Convenience sampling is almost the exact opposite of probability sampling: there is no sampling frame, there is no external random selection – anyone who comes across the survey can answer it. The results can be quite disproportionate to the target population: for instance, one convenience survey we were sent to analyze was intended to be representative of U.S. adults in general, yet only 17% of the sample was 35 years old or older – in the total adult population, this group represents 69% of adults, making for a large sampling bias.

A common mistake that publicists make when talking about PR surveys is to talk about random samples of panelists as probability samples. A random sample of a convenience sample is still a convenience sample. For instance, qSample’s U.S. panel is 54% female, 8% 55 years old and older, and 8% Hispanic; a random sample of this panel would produce results roughly the same, whereas the American population is 51% female, 33% 55+ and 16% Hispanic, according to the U.S. Census. All panels vary in similar ways. (Random selection from a panel does mean the results are representative of that panel, which is one method for providing consistent longitudinal research using that panel.)

Convenience samples are sometimes the only practical solutions for reaching markets with low-incidence populations. For instance, we’ve recently conducted research to reach the following audiences:

  • Mothers of children 4 and under
  • Families planning a home-school curriculum
  • Families with chronically ill members
  • Women who do yoga workouts
  • Adventure racing enthusiasts
  • Video game players
  • Board and card game purchasers
  • Crowdfunding backers (e.g., Kickstarter backers)
  • Purchasers of apps for smartphones and tablets
  • E-book purchasers
  • Purchasers of self-help books
  • Golfers
  • Small-business owners
  • Middle managers

The higher the incidence of the population being studied, the more techniques can be applied to improve the representativeness of the convenience sample. However, the average absolute error observed will still exceed that of probability panels and samples.

This is an excerpt from the free Researchscape white paper, “Improving the Representativeness of Online Surveys”. Download your own copy now.

Author Notes:

Jeffrey Henning

Gravatar Image
Jeffrey Henning, IPC is a professionally certified researcher and has personally conducted over 1,400 survey research projects. Jeffrey is a member of the Insights Association and the American Association of Public Opinion Researchers. In 2012, he was the inaugural winner of the MRA’s Impact award, which “recognizes an industry professional, team or organization that has demonstrated tremendous vision, leadership, and innovation, within the past year, that has led to advances in the marketing research profession.” In 2022, the Insights Association named him an IPC Laureate. Before founding Researchscape in 2012, Jeffrey co-founded Perseus Development Corporation in 1993, which introduced the first web-survey software, and Vovici in 2006, which pioneered the enterprise-feedback management category. A 35-year veteran of the research industry, he began his career as an industry analyst for an Inc. 500 research firm.