The liberal arts component of my education focused on psychology. One of the areas where I find that I’m able to draw on that base is question design. How to ask a question that doesn’t suggest or influence the answer?

I recently took a customer satisfaction survey for a brand that I use regularly. The outcomes highlighted, in my opinion, a significant gap in the questions asked and the interpretation derived.

To provide context, I review the brands that I use in terms of my use of the brand (user experience) and my contact with the brand representatives (user interface/interaction). For example, I would rate my mobile phone provider somewhere in the range of ‘neutral’ to ‘somewhat disappointing’. On the other hand, when I visit the mobile phone provider’s retail location, the staff would almost always rate a review of ‘exceeded expectations’. I recently posted on Twitter about exceptional customer experiences at a retail location.

In the mobile phone provider example, I would rate the brand lower than they would like based on intermittent service coverage, painful contract restrictions and high prices. All issues that make their competitors’ advertising compelling, and shareable on social media.

In the particular survey I referenced at the start, (Survey Monkey as I recall), there were about ten questions with the first group clearly about my experience with the brand representatives. The last few questions focused on my impressions of the brand generally.

About a week or so after completing the survey, I ran into one of the brand representatives and the survey came up during our conversation. I found out that my name and address had been shared with the brand employees, which was an unpleasant surprise given the fact that the brand had promised anonymity in the original survey document.

What was worse was that I learned that the brand, according to their employee, had used my feedback (identifying me by name and address) about the brand to deny wage and salary increases. It was explained to me that only a brand rating of ‘extremely satisfied’ was sufficient to demonstrate that the employees deserved their merit increase.

Clearly, there are several design and interpretation issues here, more than I can easily address in one blog.

I’ll leave this conversation open with a few thoughts for consideration. First, when you’re asking a question, be clear on whether you’re asking about the brand or the team. Second, if your criteria is ‘extremely satisfied’, make an objective list of the recent experiences you’ve had that would meet this criteria. Lastly, if you promote a survey as anonymous, it has to be anonymous.

Author Notes:

Jeffrey Henning

Gravatar Image
Jeffrey Henning, IPC is a professionally certified researcher and has personally conducted over 1,400 survey research projects. Jeffrey is a member of the Insights Association and the American Association of Public Opinion Researchers. In 2012, he was the inaugural winner of the MRA’s Impact award, which “recognizes an industry professional, team or organization that has demonstrated tremendous vision, leadership, and innovation, within the past year, that has led to advances in the marketing research profession.” In 2022, the Insights Association named him an IPC Laureate. Before founding Researchscape in 2012, Jeffrey co-founded Perseus Development Corporation in 1993, which introduced the first web-survey software, and Vovici in 2006, which pioneered the enterprise-feedback management category. A 35-year veteran of the research industry, he began his career as an industry analyst for an Inc. 500 research firm.