Who can argue with the idea of choice?  Having the ability to choose between two or more options is inherently attractive and frequently motivates behavior.  I may pass on vanilla ice cream if that is the only flavor on the menu but more likely to buy if offered at least one other taste variety.  And every car, computer, packaged goods, and other manufacturer understands that no two people are alike and that product-line differentiation intensifies appeal by giving choice to a broader audience.

So it makes some sense that offering research respondents a choice in how they would like to respond to our survey requests would be a good thing.  After all, the idea that different people prefer different survey modes has been around for a while and the concept of mode preference is one justification for mixed-mode research designs that integrate, for example, traditional modes – such as CATI interviews and self-administered questionnaires via USPS mail – and Web surveys.  While other more practical reasons may prevail for a mixed-mode approach – not the least of which is the ability to reach the target respondent given the contact information available to the researcher – taking advantage of mode preferences by giving respondents a choice has been considered a way to improve overall rate of response.

But choice is a curious thing in research design.  It turns out that providing respondents with a choice between two modes of responding can actually decrease the response rate.  A number of experiments have shown this to be the case.  One such study was recently published in Public Opinion Quarterly (“Improving Response to Web and Mixed-mode Surveys”) by Morgan Millar and Don Dillman from Washington State University.  In the first of two experiments, Millar and Dillman tested four conditions, all of which included initial contact via the US mail and a $2 incentive.  Two of the test conditions were as follows: 1) respondents were offered a choice of responding by way of US mail or the Web; 2) response was requested by way of US mail with no other option available (i.e., no choice was given).  What is interesting is that the response rate among the sample given a choice of mode was 47.7% while the rate of response within the no-choice (mail-only) sample was 51.3%.  While not hugely significant (p<0.10) “it nevertheless provides strong evidence that offering a choice of modes is not superior [authors’ italics] to using only mail” in survey design.  And, by the way, this study was conducted with a “highly Internet-literate population,” all of whom had Internet access.

There is any number of explanations for why individuals may elect to not respond when given a choice.  Barry Schwartz (The Paradox of Choice) might argue that a choice of modes “produces paralysis” and creates anxiety which, in turn, leads to inaction; or that the appeal of (or satisfaction with) any one option is diminished by the introduction of another.  Linelle Blais and I – in our study of paper vs. Web modes among American Cancer Society volunteers – also found a lower response rate among people given a choice of response mode and posited that “the choice, in and of itself, communicated the notion that volunteers could respond at their leisure” or not respond at all.

Whatever the explanation, it may be that our best of intentions to do something positive for our respondents (by giving them choice) is actually encouraging a do-nothing response.