Our standard practice for business-to-business surveys and surveys of our clients’ own house lists is to include the responses from incomplete surveys. Depending on the topic and the length of the survey, 10% to 30% of respondents may not complete the entire questionnaire. A common reason that respondents abandon surveys is because of topic salience: they simply find the subject of a survey to be uninteresting to them. Including their answers for those questions to which they did respond improves the representativeness of results, which would otherwise skew towards those with a higher engagement with the topic of the study.

Of course, another reason that respondents abandon surveys is simply that they’ve been interrupted while answering. We will often see examples of this in the completion data; for instance, in one survey we data-cleansed, a respondent was recorded as having taken 28 hours to complete a 7-minute questionnaire. Clearly, they were interrupted, then returned to that tab in their browser and finished the survey the next day.

Now that said, some partial responses are too trivial to be included. For instance, responses from those who screened into a survey by answering the up-front demographic questions but who then abandoned it before answering the first topic-specific question. Those partial responses might provide some clues as to how nonresponse bias varied demographically, and could be analyzed in the context of screen outs, but aren’t worth including in the main analysis.

The standard practice for many organizations is to remove all partial responses, but this has always struck me as disrespectful of the time that respondents spent answering as much of the survey as they did. (For surveys with incentives, respondents only earn the incentive if they complete the survey, so there’s no added cost to including partial responses.) And given that weighting is designed to compensate for different rates of representation, for addressing the fundamental issue that not every group takes surveys at the same rate, dropping partial responses seems counterproductive.

For nationally representative surveys – which use quota sampling to approximate the proportions of the population by age, gender, region, ethnicity, education, or other attributes – partial responses may skew the results away from this representativeness. For instance, respondents with only a high school degree have higher abandonment rates than college-educated respondents. Because of the problems with partial responses when using quota sampling, for such surveys we typically only include complete responses.

In summary:

  • Pros of including partial responses
    • Respectful of all respondents’ contributions
    • Increases the sample size for early questions
    • Increases the reportable sample size for newsmaker surveys
    • Reduces the bias of topic salience
  • Cons
    • Incompatible with quota sampling or weighting
    • Confusing to readers when the sample size declines from question to question

First published on May 21, 2018, but updated to point to more recent links.

Author Notes:

Jeffrey Henning

Gravatar Image
Jeffrey Henning, IPC is a professionally certified researcher and has personally conducted over 1,400 survey research projects. Jeffrey is a member of the Insights Association and the American Association of Public Opinion Researchers. In 2012, he was the inaugural winner of the MRA’s Impact award, which “recognizes an industry professional, team or organization that has demonstrated tremendous vision, leadership, and innovation, within the past year, that has led to advances in the marketing research profession.” In 2022, the Insights Association named him an IPC Laureate. Before founding Researchscape in 2012, Jeffrey co-founded Perseus Development Corporation in 1993, which introduced the first web-survey software, and Vovici in 2006, which pioneered the enterprise-feedback management category. A 35-year veteran of the research industry, he began his career as an industry analyst for an Inc. 500 research firm.