“We don’t have to worry about data cleansing, since we’re using our own list,” you may think.
Ah, dear reader, if only that were so.
Many such surveys run into difficulties from poor list hygiene, open recruiting, and viral incentives.
List Hygiene
Have you invested resources in list hygiene? Reviewing list data to make sure it is accurate and up-to-date. For instance, once a month I review our prospecting list to remove new customers from it. But sometimes I miss a customer, or they are on the prospecting list with a personal email address while I’m looking for their work email address. Prior to inviting your list to take a survey, it’s worth reviewing the list.
One of my favorite examples of poor list hygiene came at a prior employer. We were doing a CX study for a CRM vendor. They wanted us to populate certain fields with information from their CRM system, to save time for their customers: respondents would just need to update a few fields. Alas, the data quality was so bad that they immediately had us stop the survey. It cast doubt on the quality of their own use of their flagship CRM system.
List hygiene is hard. The list you email will be imperfect.
Preventing Duplicates
Will your survey tool and email management platform let you send a unique link to each respondent, letting them resume where they left off? It’s more work to pull off, but it immediately improves data quality. First, no one will accidentally stuff the ballot box. Second, only those who haven’t completed the survey will get reminders.
If not, you’ll find that respondents might take the survey, get distracted, then start it over again when they get a reminder. We look for that pattern and delete their shortest response when that happens.
Are you inviting people to take the survey from LinkedIn, Facebook, TikTok or other social media accounts? That’s another way for people to come in more than once. We always use a separate tracking link for such accounts, so that we can review them with that context.
Might they be on your list with more than one email address? Capturing a digital fingerprint can help you identify those duplicates. We subscribe to a service with known survey spammers, so that we can flag them immediately.
Viral Incentives Invite Ballot Box Stuffing
Even minor incentives can encourage ballot-box stuffing. A U.S. university’s response rate was lagging, so they posted to their social media that each student response would earn a $5 Starbucks gift card. Suddenly we had a hundred responses from Chile. A CPG manufacturer posted a coupon; suddenly they had tens of thousands of responses, and potential redemption discounts.
House-list surveys are invaluable for many types of research. Pay attention to list hygiene, deduplication, and incentive structure to guarantee your success.
Author Notes:
Jeffrey Henning
