To improve the representativeness of our online surveys, we – like most survey researchers – use quota sampling to closely mirror the U.S. population. In our case, we may go a little overboard. We allocate 32 quota cells for each combination of age range, gender, and region, and stop accepting results after we’ve achieved the quota limit for each cell. For instance, in a survey of 2,000 U.S. adults aged from 18- to 71-years old, the quota cell for 18- to 22-year-old males in the Northeast will require 19 completes, and the quota cell for 36- to 55-year-old females in the South (a wider age range in a more populous region) will require 158 completes. Once the target for a quota cell is achieved, subsequent respondents of that demographic are screened out of the survey.

If we didn’t use quota cells, our samples would over-represent middle-aged adults (and conversely under-represent 18-22 year olds and 56-71 year olds) and would over-represent women (and conversely under-represent men).

Because the demographics of online panels don’t exactly mimic the U.S. population, early incoming results to live surveys exhibit exactly the bias we are seeking to correct for! As a result, early dashboards can be misleading about final results, in those cases where attitudes and behaviors prove to differ by age or by gender.

Here are how our quota cells typically fill up, from first to last, in order of hitting the target number of completes:

  1. 36-55, Female, South
  2. 23-35, Female, South
  3. 36-55, Female, Northeast
  4. 36-55, Female, Midwest
  5. 36-55, Female, West
  6. 23-35, Female, West
  7. 56-71, Female, South
  8. 23-35, Female, Northeast
  9. 23-35, Female, Midwest
  10. 36-55, Male, South
  11. 23-35, Male, South
  12. 23-35, Male, West
  13. 23-35, Male, Northeast
  14. 36-55, Male, West
  15. 56-71, Female, Midwest
  16. 36-55, Male, Northeast
  17. 56-71, Female, West
  18. 36-55, Male, Midwest
  19. 56-71, Female, Northeast
  20. 23-35, Male, Midwest
  21. 56-71, Male, South
  22. 18-22, Female, South
  23. 56-71, Male, Midwest
  24. 56-71, Male, West
  25. 56-71, Male, Northeast
  26. 18-22, Female, West
  27. 18-22, Male, South
  28. 18-22, Female, Midwest
  29. 18-22, Female, Northeast
  30. 18-22, Male, West
  31. 18-22, Male, Northeast
  32. 18-22, Male, Midwest

This means surveys can get 80% of their results right away and then slowly march on trying to fill up the final quota cells.

But, by eliminating a known source of bias, the wait ends up being worth it.

For more on quota sampling and other techniques for improving the quality of online research, see the free Researchscape white paper, “Improving the Representativeness of Online Surveys”. Download your own copy now.

  • This field is for validation purposes and should be left unchanged.

Author Notes:

Jeffrey Henning

Gravatar Image
Jeffrey Henning, IPC is a professionally certified researcher and has personally conducted over 1,400 survey research projects. Jeffrey is a member of the Insights Association and the American Association of Public Opinion Researchers. In 2012, he was the inaugural winner of the MRA’s Impact award, which “recognizes an industry professional, team or organization that has demonstrated tremendous vision, leadership, and innovation, within the past year, that has led to advances in the marketing research profession.” In 2022, the Insights Association named him an IPC Laureate. Before founding Researchscape in 2012, Jeffrey co-founded Perseus Development Corporation in 1993, which introduced the first web-survey software, and Vovici in 2006, which pioneered the enterprise-feedback management category. A 35-year veteran of the research industry, he began his career as an industry analyst for an Inc. 500 research firm.