Cognitive Interviewing

Cognitive Interviewing: A Few Best Practices

Cognitive interviewing is a method used by survey researchers to investigate the integrity of their questionnaire designs prior to launching the field portion of the study. In the edited volume Cognitive Interviewing Methodology, Kristen Miller (2014) describes cognitive interviewing as “a qualitative method that examines the question-response process, specifically the processes and considerations used by respondents as they form answers to survey q4 attributes of the CI methoduestions,” further explaining that “through the interviewing process, various types of question-response problems that would not normally be identified in a traditional survey interview, such as interpretive errors and recall accuracy, are uncovered” (p. 2). In this way, survey researchers identify the users’ (i.e., survey respondents’) possible meaning and interpretation of survey questions – having to do with question structure or format and terminology – that may or may not deviate from the researcher’s intent. Importantly, the objective of the cognitive interview is not to simply determine whether a questionnaire item “makes sense” to an individual  but to go beyond that to explore the individual’s lived experience (personal context, attitudes, perceptions, behavior) in relationship to their interpretation and/or ability to answer a particular question.

Although not typically included under the “qualitative research” umbrella (with in-depth interviewing, focus group discussions, and observation), four of the 10 unique attributes associated with qualitative research are notably relevant to the cognitive interviewing method. They are the: importance of meaning, flexibility of design, participant-researcher relationship, and researcher skill set. These distinctive qualities of the cognitive interviewing method, and qualitative methods generally, define why researchers opt for Read Full Text

Integrating Cognitive Interviewing into Research Design

If maximizing our understanding of how people think is fundamental to research design – a common theme throughout Research Design Review – then why is so little attention paid to the idea that thinking is not stagnant but something that is continually changing from moment to moment.  If I ask a survey respondent to name the primary reason she likes store A over store B, the response may be something entirely different than if I ask the same question the following day, or possibly even later the same day.  And if I ask how many miles you drive to the office each day, you might say 10 miles today but 15-20 miles if I ask the same question tomorrow.

Why is that?  In the February 2010 RDR post, “Qualitative Research & Thinking About How People Think,” I reference the four-step cognitive process (posited by others) considered necessary to respond “optimally” to research questions: interpretation, searching for relevant information, integration towards a judgment, and translation of a judgment into a response.   These four stages alone suggest that isolating the key reason for choosing store A over store B may be more complex, requiring more thoughtful contemplation than the quick response researchers typically encourage in order to keep the interview to a manageable length.  And that is why my judgment about Read Full Text