Bernie Malinoff of Element54 discussed at ESOMAR 3D 2011 the implications of Flash and gamification interfaces on market research online survey design.
While research firms have introduced interactive technology based on the premise that online surveys are boring and basic, reflecting the Word document. Survey engagement has become a space race, with the rise of sliders and other interactive survey elements from different vendors.
The problem with the rise of visual interface options is that it makes it hard to determine whether or not it is appropriate to choose a particular visual element (for instance, a slider), and — if so — what configuration of an element to use. How will all these decisions affect the quality of the data?
With GMI, Bernie Malinoff tested a basic survey against a Flash interface survey and a gamified survey. Goals are important when choosing between these designs: for instance, if you want to minimize straightlining, the Flash survey tested reduced it by half, while the game eliminated altogether by asking respondents to bet on their confidence in their answers. Some other findings:
- If you ask the same question 5 different ways, you are going to get five different results: 33% recognized logos from a checkbox quesiton, rising to over 70% from a game approach, for instance.
- Questionnaire enjoyment rose from 7.6 on a 10-point scale with the basic survey to 8.2 with a Flash survey and 8.3 with games.
Scalability and norms are very important when running research, and switching to Flash or gamified elements changes the metrics. “The norms become a fortress with the normative database very important.” Switching to such elements also requires significant custom work. On a container ship, each of the 1500 containers is the same size. Different size containers lowers efficiency and scalability. “The world of better, faster, cheaper comes into conflict with creativity and scalability.” The first step is to embrace substance with style, ask why we want to do it and use that as the basis for deciding what part of the process you want to fix.
How far do you want or need to go?
Author Notes:
Jeffrey Henning
