consumer research

Unilever’s Qualitative Accreditation Program & Misdirected Quest for “Fresh Ideas”

Many researchers have discussed Unilever’s accreditation program for qualitative research.  Among others, the Market Research Society, ESOMAR’s Research World, and Katfresh ideas2hryn Korostoff (Research Rockstar) have all outlined what led up to this program, the objectives of the program, and the accreditation process.  In a nutshell, Unilever assessed the outcomes of their many qualitative studies around the globe and determined that the qualitative researchers Unilever has employed to conduct their qual studies have generally failed in providing management with a sufficient caliber of new ideas and insights that serve to move the company forward.

Manish Makhijani, a consumer insights director at Unilever, stated in an interview discussing the program that one of his top concerns with their qualitative research is the inconsistency in “the quality of insights and debriefs” among their qualitative researchers, emphasizing that “what matters in qual more than anything else is the quality of thinking that you put on the table.”  And, indeed, Makhijani brought home this point at the November 2012 ESOMAR conference when he presented the notion that “good” qualitative research is derived Read Full Text

Knowing What We Don’t Know: Social Desirability & Time-use Diaries

Back in February 2012, Research Design Review posted a discussion titled “Accounting for Social Desirability Bias in Online Research,” questioning the idea that social desirability is less a factor in the online mode (compared to more traditional research methods) and that, to the contrary, “individual attitudes and behavior we capture online are potentially distorted by an underlying need for social approval.”  Social desirability is an interesting and important source of error in research regardless of mode and worthy of consideration in all of our design strategies.

Social desirability as a factor in measurement error is particularly relevant when people assume there are socially-acceptable behaviors or attitudes connected with the research topic.  Behavior and attitudes, for instance, associated with healthy eating, exercise, religious activities, and (in the old days) visits to the library.  Because direct questioning about issues potentially laden with social (or personal-worth) status can distort a true picture of behavior, researchers have used time-diary data based on a person’s methodical recording of daily activities.  Concerning library visits, Iiris Niemi’s 1993 paper compared interview survey data with actual behavior recorded in time-use diaries showing that “the interview method produced 70% more library visits than the diary method.”  Similarly, Niemi reports that a much higher percentage of people claimed to engage in daily physical exercise when asked directly compared to the diary data documenting actual frequency of exercise activity.

Over reporting of religious activities, and specifically religious service attendance, has been the focus of Philip Brenner and other researchers.  Looking at religious service attendance among the U.S. population, researchers have concluded that there is a positive relationship between direct measures (i.e., direct questioning) and socially-desirable responding.  For example, 40%-50% report attending church every week when asked directly in a survey question, yet time-use diary studies (that do not tell participants what behaviors are of particular interest) indicate that only 20%-30% actually attend church weekly.  Some have linked this disparity to the idea that people have a tendency to interpret the survey question, ‘How often do you attend church?’ as ‘Are you a good, church-going person?’  Interestingly, the gap in direct versus indirect data on this issue as well as this tendency – that is, to re-interpret the church-attendance question to one about personal identity or worth – appears to be a U.S. phenomenon with people in other countries reporting very similar frequency of attendance regardless of research method.

Until social stigmas disappear and people do not feel threatened by reporting their true nature, research design will always struggle with the problem of social desirability.  Even though time-use diaries – as well as other more modern, in-the-moment approaches, such as mobile research – may be just as susceptible to censorship as typical survey responses, it seems right that unobtrusive yet personal modes of data collection may move us closer to knowing what we don’t know about consumer behavior.

Ask Someone a Question, You’ll Get an Answer

A recent online survey questionnaire asked me to compare my salary in 2009 with that in 2010.  Because the question was impossible to answer (I won’t know my 2010 salary for another eight months), I didn’t respond and attempted to move forward to the next question.  Unfortunately the salary question was mandatory and so I was left with the choice of either abandoning the study or falsifying a response in order to qualify for the next question.  I opted for the latter (I was keen on learning where the battery of questions would take me), then immediately upon Read Full Text