Social Desirability

Finding Meaning: 4 Reasons Why Qualitative Researchers Miss Meaning

Research of any kind that is interested in the human subject is interested in finding meaning.  It Meaningis typically not enough to know that a behavior has occurred without knowing the significance of that behavior for the individual.  Even survey research, with its reliance on mostly preconceived closed-ended questions, is designed with some hope that sense (i.e., meaning) can be derived by cross tabbing data from one question with another, factor analyzing, t-testing, z-testing, regressing, correlating, and any number of statistical techniques.

Yet, it is qualitative research that is usually in charge of finding meaning.  It is not good enough to know who does what, for how long, or in what manner.  Qualitative researchers are not so Read Full Text

Knowing What We Don’t Know: Social Desirability & Time-use Diaries

Back in February 2012, Research Design Review posted a discussion titled “Accounting for Social Desirability Bias in Online Research,” questioning the idea that social desirability is less a factor in the online mode (compared to more traditional research methods) and that, to the contrary, “individual attitudes and behavior we capture online are potentially distorted by an underlying need for social approval.”  Social desirability is an interesting and important source of error in research regardless of mode and worthy of consideration in all of our design strategies.

Social desirability as a factor in measurement error is particularly relevant when people assume there are socially-acceptable behaviors or attitudes connected with the research topic.  Behavior and attitudes, for instance, associated with healthy eating, exercise, religious activities, and (in the old days) visits to the library.  Because direct questioning about issues potentially laden with social (or personal-worth) status can distort a true picture of behavior, researchers have used time-diary data based on a person’s methodical recording of daily activities.  Concerning library visits, Iiris Niemi’s 1993 paper compared interview survey data with actual behavior recorded in time-use diaries showing that “the interview method produced 70% more library visits than the diary method.”  Similarly, Niemi reports that a much higher percentage of people claimed to engage in daily physical exercise when asked directly compared to the diary data documenting actual frequency of exercise activity.

Over reporting of religious activities, and specifically religious service attendance, has been the focus of Philip Brenner and other researchers.  Looking at religious service attendance among the U.S. population, researchers have concluded that there is a positive relationship between direct measures (i.e., direct questioning) and socially-desirable responding.  For example, 40%-50% report attending church every week when asked directly in a survey question, yet time-use diary studies (that do not tell participants what behaviors are of particular interest) indicate that only 20%-30% actually attend church weekly.  Some have linked this disparity to the idea that people have a tendency to interpret the survey question, ‘How often do you attend church?’ as ‘Are you a good, church-going person?’  Interestingly, the gap in direct versus indirect data on this issue as well as this tendency – that is, to re-interpret the church-attendance question to one about personal identity or worth – appears to be a U.S. phenomenon with people in other countries reporting very similar frequency of attendance regardless of research method.

Until social stigmas disappear and people do not feel threatened by reporting their true nature, research design will always struggle with the problem of social desirability.  Even though time-use diaries – as well as other more modern, in-the-moment approaches, such as mobile research – may be just as susceptible to censorship as typical survey responses, it seems right that unobtrusive yet personal modes of data collection may move us closer to knowing what we don’t know about consumer behavior.