As part of CASRO’s great series of webinars, John Bremer of The NPD Group discussed “Elements of Non-Probability Seminar.” Besides touching on probability sampling, sample matching, and calibration, he presented an excellent taxonomy of the different types of non-probability sampling.
- Quota Sampling
- Proportional Quota Sampling – The “proportional” in the name is because the population of interest is represented almost exactly by the percentage of each cell (major demographic group) in the final survey results. So if a population is 52% female, then 520 of 1,000 respondents would be female. “Strata may be joint or interlocking” – so you might have quotas of younger women, older women, younger men, and older men. Sometimes known as “hard quotas”.
- Non-Proportional Quota Sampling – Also known as “soft quotas”, non-proportional quota sampling captures a minimum number of respondents in a specific group. For hard-to-reach populations, it might be an undersample (less than in a proportional sample), and for populations of especial interest in and of themselves it might be an oversample (more than in a proportional sample). “Here, weighting [of final results] is important because the sample doesn’t look like the population of interest,” John said.
- Convenience Sampling – Respondents are simply those “who are easily available or convenient to interview”: fellow employees, people intercepted on the street, Facebook fans of a brand, members of mailing lists, and online panelists. While an online panel can use a stratified random sample of panel members, it is still a convenience sample.
- Snowball Sampling – Snowball sampling asks respondents to recommend other respondents who might subsequently be invited to take the survey. This is useful for very hard-to-reach audiences, such as AIDS sufferers or others with rare diseases. “It’s hard to get a representative view of such a population, because of the clustering effect and the representativeness effect and because you often won’t know what the population looks like, so it is what it is—but with unknown biases.”
- Self-Selection Sampling – The respondent decides whether or not to participate, typically in one request without the chance for follow up. Classic psychology experiments are self-selection sampling, as are “surveywall”-style intercept surveys.
- Purposive Sampling – The interview or study designer chooses sampled units who, by their judgment, will meet the specific purpose of the survey.
- Maximum Variance Sampling (Heterogeneous Sampling) – For such a study, which may be more qualitative in nature, the goal is not to be representative of views on an issue but “to look at it from all angles”. Understanding the extremes is as important as understanding the typical viewpoint.
- Homogenous Sampling – The opposite of heterogeneous sampling is to deeply explore the views of a group of respondents with the same characteristics.
- Typical Case Sampling – Rather than understand all viewpoints, including the extremes, typical case sampling is interested in an in-depth assessment of the typical viewpoint while not developing.
- Extreme Case Sampling – [In the interests of time, John skipped these final four examples of purposive sampling.] Extreme case sampling is interested in understanding unusual cases such as successes or failures. For instance, a review of “unicorn” CEOs (startups worth $1 billion or more) or a study of millionaires.
- Critical Case Sampling – Studying those cases that have the most to offer in terms of understanding the population.
- Expert Sampling – Surveying experts on a particular topic, with their expertise left to the judgment of the interviewer or study designer.
- Total Population Sampling – Surveying every single member of a qualifying subgroup – for instance, employees of a firm or employees at a specific branch of a firm. Failure to survey even a few respondents can make it difficult to generalize, if non-respondents share certain characteristics that differ from the wider population.
John’s presentation provided a valuable overview of the many uses of non-probability samples. Make sure to check out other upcoming CASRO webinars!
“Simple random sampling” by Dan Kernler – Own work. Licensed under CC BY-SA 4.0 via Commons
This is an excerpt from the free Researchscape white paper, “Improving the Representativeness of Online Surveys”. Download your own copy now.
Author Notes:
Jeffrey Henning
