Bernie Malinoff of Element54 discussed at ESOMAR 3D 2011 the implications of Flash and gamification interfaces on market research online survey design.

While research firms have introduced interactive technology based on the premise that online surveys are boring and basic, reflecting the Word document. Survey engagement has become a space race, with the rise of sliders and other interactive survey elements from different vendors.

The problem with the rise of visual interface options is that it makes it hard to determine whether or not it is appropriate to choose a particular visual element (for instance, a slider), and — if so — what configuration of an element to use. How will all these decisions affect the quality of the data?

With GMI, Bernie Malinoff tested a basic survey against a Flash interface survey and a gamified survey. Goals are important when choosing between these designs: for instance, if you want to minimize straightlining, the Flash survey tested reduced it by half, while the game eliminated altogether by asking respondents to bet on their confidence in their answers. Some other findings:

  • If you ask the same question 5 different ways, you are going to get five different results: 33% recognized logos from a checkbox quesiton, rising to over 70% from a game approach, for instance.
  • Questionnaire enjoyment rose from 7.6 on a 10-point scale with the basic survey to 8.2 with a Flash survey and 8.3 with games.

Scalability and norms are very important when running research, and switching to Flash or gamified elements changes the metrics. “The norms become a fortress with the normative database very important.” Switching to such elements also requires significant custom work. On a container ship, each of the 1500 containers is the same size. Different size containers lowers efficiency and scalability. “The world of better, faster, cheaper comes into conflict with creativity and scalability.” The first step is to embrace substance with style, ask why we want to do it and use that as the basis for deciding what part of the process you want to fix.

How far do you want or need to go?

Author Notes:

Jeffrey Henning

Gravatar Image
Jeffrey Henning, IPC is a professionally certified researcher and has personally conducted over 1,400 survey research projects. Jeffrey is a member of the Insights Association and the American Association of Public Opinion Researchers. In 2012, he was the inaugural winner of the MRA’s Impact award, which “recognizes an industry professional, team or organization that has demonstrated tremendous vision, leadership, and innovation, within the past year, that has led to advances in the marketing research profession.” In 2022, the Insights Association named him an IPC Laureate. Before founding Researchscape in 2012, Jeffrey co-founded Perseus Development Corporation in 1993, which introduced the first web-survey software, and Vovici in 2006, which pioneered the enterprise-feedback management category. A 35-year veteran of the research industry, he began his career as an industry analyst for an Inc. 500 research firm.