Given that effective sample size declines dramatically when the characteristics of the sample don’t approximate national representativeness, one method of improving non-probability sampling is quota sampling, dividing the sample into cells and recruiting to fill those cells. Once 51% of the responses are women, you stop letting additional women take the survey, for instance. In this way, you minimize the effect of weights.

Quota sampling gets a bad rap because of “Dewey defeats Truman” yet in fact quota sampling had been effective in the elections before 1948. This is a recurring theme – non-probability sampling is often close enough, but it is wrong much more often than probability sampling. And in fact the issues in 1948 may have been less with quota sampling than the respondent selection methods.

Corporate researchers purchasing panel would do well to regularly use quota sampling to help reduce sampling bias, though panel providers differ in how they price quotas. I recently had the price for one study increase 50% for adding 3 quota cells and increase 100% for adding 12 quota cells.

 

This is an excerpt from the free Researchscape white paper, “Improving the Representativeness of Online Surveys”. Download your own copy now.

Author Notes:

Jeffrey Henning

Gravatar Image
Jeffrey Henning, IPC is a professionally certified researcher and has personally conducted over 1,400 survey research projects. Jeffrey is a member of the Insights Association and the American Association of Public Opinion Researchers. In 2012, he was the inaugural winner of the MRA’s Impact award, which “recognizes an industry professional, team or organization that has demonstrated tremendous vision, leadership, and innovation, within the past year, that has led to advances in the marketing research profession.” In 2022, the Insights Association named him an IPC Laureate. Before founding Researchscape in 2012, Jeffrey co-founded Perseus Development Corporation in 1993, which introduced the first web-survey software, and Vovici in 2006, which pioneered the enterprise-feedback management category. A 35-year veteran of the research industry, he began his career as an industry analyst for an Inc. 500 research firm.