Sampling for online surveys varies greatly in technique and representativeness. Important factors to consider are types of panels, selection of participants from those panels, and how and whether to weight survey results. While demographic variables are often compared between the survey and the target population, equally important are comparing attitudinal and behavioral measures.

While there are many sources of survey error, one of the leading sources of error in online surveys is sampling bias: failing to include in representative numbers some groups of the target population. For instance, the 2011 American Community Survey estimates that in 4.6% of U.S. households no one over the age of 13 speaks English “very well” — these non-English speakers are unrepresented in most commercial surveys, leading to sampling bias.

Online surveys are often used to extrapolate to the entire U.S. population, including people who never go on online. This is a clear sampling bias, as repeated academic work reveals that non-Internet users differ materially, socially, and psychologically from Internet users, making it problematic to extrapolate to this population from online-only samples.

With online surveys it is more accurate to extrapolate to the online population – either the 73% of U.S. adults with Internet access at home or the 87% of U.S. adults with occasional Internet access from home or work, from computer or smartphone or tablet.

This is an excerpt from the free Researchscape white paper, “Improving the Representativeness of Online Surveys”. Download your own copy now.

Author Notes:

Jeffrey Henning

Gravatar Image
Jeffrey Henning, IPC is a professionally certified researcher and has personally conducted over 1,400 survey research projects. Jeffrey is a member of the Insights Association and the American Association of Public Opinion Researchers. In 2012, he was the inaugural winner of the MRA’s Impact award, which “recognizes an industry professional, team or organization that has demonstrated tremendous vision, leadership, and innovation, within the past year, that has led to advances in the marketing research profession.” In 2022, the Insights Association named him an IPC Laureate. Before founding Researchscape in 2012, Jeffrey co-founded Perseus Development Corporation in 1993, which introduced the first web-survey software, and Vovici in 2006, which pioneered the enterprise-feedback management category. A 35-year veteran of the research industry, he began his career as an industry analyst for an Inc. 500 research firm.