Every week new email invitations arrive asking me to participate in an online survey concerning some product or service I recently used. And each time, as I read the stated reasons why I should comply with the request, I find myself taking a mental inventory of what I know or don’t know about the subject matter, what I can or cannot recall about my user experience, how positive or not positive the user experience was, and how important I think this product or service is in my life to be worthy of my time to answer their survey questions.
Last week I was asked by one of my trade organizations to participate in an online survey about their quarterly magazine. Or is it a monthly magazine? Maybe every two months? I am not sure, but I do know that I receive it and I read it. I stared at the email invitation taking the usual inventory, sifting through my usual battery of qualifying questions, pondering whether I should complete this survey or not. Yes, I told myself, I remember receiving this magazine, I know that I read it when it arrives, but do I really have anything to say about this magazine? My opinion of this magazine falls in some neutral territory Read Full Text
Nonresponse, and inaccuracies in the data due to nonresponse, is more than a quantitative issue. While qualitative researchers may shudder at the thought, the typically-ignored impact of nonresponse is just as important in the qualitative realm. Why is nonresponse in qualitative research important? Because we are conducting qualitative research. Not qualitative let’s get a few warm bodies around the table for our face-to-face focus group, but actually research methods that, like all research, demand certain protocols that address potential biasing effects. One of these is nonresponse. The warm bodies in our group discussion may make the moderator and client observers feel great – Thank goodness, someone showed up! – but the uncomfortable reality is that the people who chose not to participate – or were never contacted by a recruiter and asked to participate in the first place – greatly affect our research outcomes. Indeed, the trajectory of a group discussion Read Full Text
A number of years ago I experimented with using a toll-free number in an ongoing satisfaction survey for a large organization. For a lot of reasons that I don’t need to go into here, the design was a self-administered paper questionnaire delivered in the U.S. mail; and because of an unexpectedly low response rate, I elected to send a second follow-up mailing that, among other things, included a toll-free number (that linked directly into a voice mail box where callers were not required to leave their name or contact information, although many did).
My initial thinking was that the toll-free number would be a convenient way for nonrespondents to request another copy of the questionnaire. As it turned out, however, nearly 60% of the calls received were from people who stated that they wouldn’t or couldn’t respond to the survey request. And we determined that the vast majority of these people either shouldn’t have been in the database in the first place or were unable to respond to the questionnaire due to design issues.
This was an important result in that we were able to get our hands around the key sources of nonresponse – inaccuracies in the database & questionnaire design – and focus our attention in these areas. The toll-free number did, indeed, help us explain non-response behavior.
A short presentation of this effort follows: