The Insights Association ran a great webinar on “Migrating Qualitative Projects Online” with Katrina Noelle of KNow Research and Scoot Insights and Jen Dale of Inside Heads and MyVFF. I was lucky enough to moderate the session and got to curate some of the attendees’ questions at the end. Here is a lightly edited transcript of our discussion.
Jeffrey: So with the migration [to online for qualitative research], certainly a lot of people have been caught flat-footed. One of the questions that has come up a few times is “How do we think about the mode effects, the way that doing this online differs from when it would be done in person? Are people more candid, less candid, are there certain topics they’ll talk about more or less?”
Jen: I got started in online marketing research for that very reason, because we were doing studies that were very, private, right? And people online are incredibly candid and there are ways you can set up the structure or pick a method that fosters even greater candor. So obviously with webcam that’s not going to happen quite as much but I know with chat groups and IDIs and Screen Share, if they don’t have to look me in the eye, and they don’t have to look their other neighbor in the eye, they are much more apt to tell you the truth? And that can be hard too for clients on the other side, when people are very candid, so it can be painful for clients. But it is all the information you need, right? And all we want is the truth. And if somehow or another this gives people the freedom to open up a little bit more to us? What a joyous thing.
Katrina: And you know people always ask me: “Body language, the 90% and what’s happening from the waist down, do you get all that great information if you’re doing it digitally?” I would say sometimes that some of that is compromised. There is something about feeling and energy in a room that you don’t necessarily have, but it just means you have to find out in different ways. It means you have to get very good at figuring out what someone is getting at while they’re on their webcam, and what an eyebrow move means rather than a leg cross, so there’s just different signals.
Jen: Absolutely, and it’s not just the moderator that’s working with different signals, so are participants. And I’ve said this more than once; everybody knows you’re online, they know I can’t see them if that is the situation. I mean just like on a phone interview! We didn’t suddenly have, “Well you can’t see their face”, we both know we’re on a phone, we can make this work! So, that’s it, there’s an understanding of the engagement means, so I think that helps a lot too.
Jeffrey: Great! Another question as people migrate is how do the timings of Qual projects online as compared to offline?
Katrina: I’ll take that one. So for synchronous conversations I have noticed, just doing this over the years and learning by failing, it’s harder to hold attention. It’s harder to have someone sit in a chair looking at their webcam and, to the same degree that they can sit in a room with other people or by themselves, or in their home and chat. There’s a little antsiness that gets in. I’m just gonna throw some numbers out there this is just what we tend to use: we tend to not do webcam groups for longer than ninety minutes, and we don’t tend to do interviews longer than sixty minutes, we just noticed that the twitchy factor gets in right about there. Sometimes you can stretch it a little bit, but that is a situation. The other consideration that Jen mentioned earlier which deserves emphasis is size of group. So again, for synchronous, this is not a time for eight people. I know Brady Bunch is used as an example of it, the squares and talking to a bunch of people: don’t use as many people as the Brady Bunch in your webcam group! Try with four or five, see how you like it, maybe two. Try to keep it small, maybe do a few more. Janet and I at Scoot, talk a lot about how more, smaller sessions gets you the same number of people you’re just doing more, shorter sessions with less people, but the N is the same. So, yeah, there are some considerations design-wise.
Jeffrey: OK, how do incentives for participating online compare to participating offline? Is it comparable?
Jen: The same. It really goes back to the old rule of thumb, which still stands, what is the value of that person’s time for that sixty minutes, ninety minutes, two hours, three days, whatever it happens to be, and you know that number has gone like this *gestures* over the last three years. God knows where it’s going next, I’m not even going to try and guess, but I would say it’s absolutely comparable. People say, “Well they don’t have to drive, it’s easier, they can just log in.” I don’t care, I mean, in the whole sum it’s saving hundreds, and it’s not getting us anywhere. So I think a comparable incentive is right-on.
Jeffrey: And for people used to over-recruiting for a focus group, how should they over-recruit for an online Qual?
Jen: What a good question.
Katrina: Yeah, that depends on methodology.
Jen: I can take the individual, I mean I could do real-time, you want to take over time?
Katrina: Sure.
Jen: Alright. Real time, if I invite, a lot of it is going to depend on how much you communicate with your participants: how well are you preparing them, are you reminding them via phone or via text, via email, all three. Yes, that’s what you want. How well do you know them prior to them being your participant? Do they have any kind of involvement with you? But typically, if you’re going to invite 25, anywhere between 18 to 20 will show up. I used to say that a certain percentage would have problems but that doesn’t really happen anymore. When it comes to IDIs, I’ve rarely had anyone not show up. And, again, it’s because of the effort you put into getting them a calendar invite, reminding them, all of that good stuff.
Katrina: No, I mean in that way, Jen, but it’s not different. That’s not different than in person. You’re doing exactly the same things.
Jen: True, true, it might be even easier for someone, I mean, I don’t have the in-person show rates off the top of my head but it might even be a little higher only because it’s the ease of hitting a button: you’re not dealing with traffic.
Katrina: Yeah, you know, I’m always a fan of not over-recruiting, but having standby people. So, often with some of the bulletin boards and diary studies we won’t totally overload it at the beginning but we’ll have some people to swap out with if we notice some people who are not enthused and not committed to the process. That should also really, you keep in mind, it depends on the length, if you’re talking about asynchronous, how long is this project? And be ready to refresh if people drop out or you want to replace anybody. I think that a lot of the rules of thumb that you use for in person research can be directly translated. Just don’t take care of them less.
Jen: True, right, that’s great advice!
Jeffrey: Thanks everyone for the great questions, we will be getting to the rest of those asynchronously, but I do want to remind folks that the NEXT Conference which was going to be in NYC is now a virtual event, and I encourage you to check that out. It’ll be a three-day program of online sessions. I’ve already looked at the schedule, and it looks great and it’s a great price, so I really encourage you check that out. Again, I’d like to thank everyone for contributing your input. We will be sending out a recording of this session as soon as we can, Thank you so much for participating. Have a great day, and thank you, Jen and Katrina.
You can view the whole webinar here: “Migrating Qualitative Projects Online”.
Need more? Check out the affordable online Principles Express course, Qualitative Market Research. We are grateful for the course to be sponsored by Recollective, an online research platform for conducting qualitative studies and developing insights communities. Powerful and flexible, Recollective is chosen by the world’s leading research organizations to meet their sophisticated and constantly advancing research needs. Sponsorships like this have funded the development of our new line of Principles Express courses, a portfolio of $359 online courses that let you master a research skill at your own pace, with just 9 to 14 hours of study.
Author Notes:
Jeffrey Henning
