Convenience sampling is almost the exact opposite of probability sampling: there is no sampling frame, there is no external random selection – anyone who comes across the survey can answer it. The results can be quite disproportionate to the target population: for instance, one convenience survey we were sent to analyze was intended to be representative of U.S. adults in general, yet only 17% of the sample was 35 years old or older – in the total adult population, this group represents 69% of adults, making for a large sampling bias.
A common mistake that publicists make when talking about PR surveys is to talk about random samples of panelists as probability samples. A random sample of a convenience sample is still a convenience sample. For instance, qSample’s U.S. panel is 54% female, 8% 55 years old and older, and 8% Hispanic; a random sample of this panel would produce results roughly the same, whereas the American population is 51% female, 33% 55+ and 16% Hispanic, according to the U.S. Census. All panels vary in similar ways. (Random selection from a panel does mean the results are representative of that panel, which is one method for providing consistent longitudinal research using that panel.)
Convenience samples are sometimes the only practical solutions for reaching markets with low-incidence populations. For instance, we’ve recently conducted research to reach the following audiences:
- Mothers of children 4 and under
- Families planning a home-school curriculum
- Families with chronically ill members
- Women who do yoga workouts
- Adventure racing enthusiasts
- Video game players
- Board and card game purchasers
- Crowdfunding backers (e.g., Kickstarter backers)
- Purchasers of apps for smartphones and tablets
- E-book purchasers
- Purchasers of self-help books
- Golfers
- Small-business owners
- Middle managers
The higher the incidence of the population being studied, the more techniques can be applied to improve the representativeness of the convenience sample. However, the average absolute error observed will still exceed that of probability panels and samples.
This is an excerpt from the free Researchscape white paper, “Improving the Representativeness of Online Surveys”. Download your own copy now.
Author Notes:
Jeffrey Henning
