Back in February 2012, Research Design Review posted a discussion titled “Accounting for Social Desirability Bias in Online Research,” questioning the idea that social desirability is less a factor in the online mode (compared to more traditional research methods) and that, to the contrary, “individual attitudes and behavior we capture online are potentially distorted by an underlying need for social approval.”  Social desirability is an interesting and important source of error in research regardless of mode and worthy of consideration in all of our design strategies.

Social desirability as a factor in measurement error is particularly relevant when people assume there are socially-acceptable behaviors or attitudes connected with the research topic.  Behavior and attitudes, for instance, associated with healthy eating, exercise, religious activities, and (in the old days) visits to the library.  Because direct questioning about issues potentially laden with social (or personal-worth) status can distort a true picture of behavior, researchers have used time-diary data based on a person’s methodical recording of daily activities.  Concerning library visits, Iiris Niemi’s 1993 paper compared interview survey data with actual behavior recorded in time-use diaries showing that “the interview method produced 70% more library visits than the diary method.”  Similarly, Niemi reports that a much higher percentage of people claimed to engage in daily physical exercise when asked directly compared to the diary data documenting actual frequency of exercise activity.

Over reporting of religious activities, and specifically religious service attendance, has been the focus of Philip Brenner and other researchers.  Looking at religious service attendance among the U.S. population, researchers have concluded that there is a positive relationship between direct measures (i.e., direct questioning) and socially-desirable responding.  For example, 40%-50% report attending church every week when asked directly in a survey question, yet time-use diary studies (that do not tell participants what behaviors are of particular interest) indicate that only 20%-30% actually attend church weekly.  Some have linked this disparity to the idea that people have a tendency to interpret the survey question, ‘How often do you attend church?’ as ‘Are you a good, church-going person?’  Interestingly, the gap in direct versus indirect data on this issue as well as this tendency – that is, to re-interpret the church-attendance question to one about personal identity or worth – appears to be a U.S. phenomenon with people in other countries reporting very similar frequency of attendance regardless of research method.

Until social stigmas disappear and people do not feel threatened by reporting their true nature, research design will always struggle with the problem of social desirability.  Even though time-use diaries – as well as other more modern, in-the-moment approaches, such as mobile research – may be just as susceptible to censorship as typical survey responses, it seems right that unobtrusive yet personal modes of data collection may move us closer to knowing what we don’t know about consumer behavior.