Total Quality Framework

In-depth Interviewer Effects: Mitigating Interviewer Bias

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 83-84).

The outcome of a qualitative in-depth interview (IDI) study, regardless of mode, is greatly affected by the interviewer’s conscious or unconscious influence within the context of the IDIs—that is, the absence or presence of interviewer bias. The interviewer’s Interviewer Effects-Biasdemographic characteristics (e.g., age, race), physical appearance in face-to-face IDIs (e.g., manner of dress), voice in face-to-face and telephone IDIs (e.g., a regional accent), and personal values or presumptions are all potential triggers that may elicit false or inaccurate responses from interviewees. For example, imagine that an IDI study is being conducted with a group of public school teachers who are known to harbor negative feelings toward the district’s superintendent but who express ambivalent attitudes in the interviews as the result of the interviewers’ inappropriate interjection of their own personal positive opinions. In this way, the interviewers have caused the findings to be biased. In order to minimize this potential source of distortion in the data, the researcher can incorporate a number of quality enhancement measures into the IDI study design and interview protocol:

  • The IDI researcher should conduct a pretest phase during which each interviewer practices the interview and learns to anticipate what Sands and Krumer-Nevo (2006) call “master narratives” (i.e., the interviewer’s own predispositions) as well as “shocks” that may emerge from interviewees’ responses. Such an awareness of one’s own predispositions as an interviewer and possible responses from interviewees that might otherwise “jolt” the interviewer will more likely facilitate an uninterrupted interview that can smoothly diverge into other appropriate lines of questioning when the time presents itself. In this manner, the interviewer can build and maintain strong rapport with the interviewee as well as anticipate areas within the interview that might bias the outcome.

For example, Sands and Krumer-Nevo (2006) relate the story of a particular interview in a study among youth who, prior to the study, had been involved in drug use and other criminal behavior. Yami, the interviewer, approached one of the interviews with certain assumptions concerning the interviewee’s educational background and, specifically, the idea that a low-level education most likely contributed to the youth’s illicit activities. Because of these stereotypical expectations, Yami entered the interview with the goal of linking the interviewee’s “past school failures” to his current behavior and was not prepared for a line of questioning that was not aimed at making this connection. As a result <!–more–Read Full Text>of her predisposition, Yami failed to acknowledge and question the interviewee when he talked about being a “shy, lonely boy” and, consequently, stifled the life story that the interviewee was trying to tell her.

 

  • The interviewer should use follow-up and probing questions to encourage the interviewee to elaborate on a response (e.g., “Can you tell me more about the last time the other students harassed you at school?”), but not in a manner that could be perceived as seeking any particular “approved” substantive response.

 

  • Using a reflexive journal is an important and necessary feature of an IDI study design. This device enhances the credibility of the research by ensuring that each interviewer keeps a record of his/her experiences in the field and how he or she may have biased interview outcomes. The interviewer reflects carefully after each completed IDI and records how he or she may have distorted the information gathered in the interview (inadvertent as it may have been) and how the interviewee’s behavior and other factors may have contributed to this bias. This “reflexive objectivity” (Brinkmann & Kvale, 2015) helps the interviewer gain “sensitivity about one’s [own] prejudices, one’s subjectivity” (p. 278) and consider the impact of these influences on the credibility of the data. This objectivity might also reflect on any personal characteristics of the interviewer (e.g., voice parameters, personality traits, demographics) that affected the interview and resulted in unintended variation across all IDIs. By way of the reflexive journal, the research is enriched with a documented firsthand account of any interviewer bias or presumptions as well as variations in the interviewer’s handling of interviews throughout the study.

 

  • A reflexive journal can also be used in the triangulation of interview data. From a Total Quality Framework perspective, a best practice is to have an impartial research team member review the audio or video recordings from one or more IDIs to identify how and under what circumstance the interviewer may have biased interviewees’ responses. In turn, this review can be used in cross-reference with the interviewer’s reflexive journal and discussed with the interviewer to help him/her better understand lapses in self-awareness. This journal also becomes an important component of the study’s audit trail and a tool in the final data analysis and interpretation.

 

Brinkmann, S., & Kvale, S. (2015). Interviews: Learning the craft of qualitative research interviewing (3rd ed.). Thousand Oaks, CA: Sage.

Sands, R. G., & Krumer-Nevo, M. (2006). Interview shocks and shockwaves. Qualitative Inquiry, 12(5), 950–971. https://doi.org/10.1177/1077800406288623

Image captured from: http://www.jeannievodden.com/light-effects-11-x-15-c2009/

 

 

Making Sense of the Human Experience with Qualitative Research

The following is a modified excerpt of the introduction to Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 1-2).

Human beings engage in some form of “qualitative research” all of the time. This is because there is not a context in which humans engage that does not require some process of taking in (i.e., gathering) information from the environment and developing that information into an interpretive nugget that can then be used to make sense of and react to particular situations. Humans do this so routinely that they are rarely aware of the information-gathering stages they process, or even their constant and natural proclivities to do so. Although some human beings may be more successful at processing contextual information than others, humans generally do not consciously think about the quality of the information they take in and the quality of the decision-making processes they apply to that information as they go through their daily lives.

As a formal method of inquiry, qualitative research—with its emphasis on the individual and the role that context and relationships play in forming thoughts and behaviors—is at the core of what it means to conduct research with human subjects. Qualitative research assumes that the answer to any single research question or objective lies within a host of related questions or issues pertaining to deeply seeded aspects of humanity. A qualitative inquiry into breast cancer treatment, for example, might begin by asking “How do women cope with breast cancer treatment?”, from which the researcher considers any number of relevant personal issues around “coping” and then addresses further and deeper questions, such as “What is the quality of life among women undergoing breast cancer treatment?”, “How do various aspects of this quality of life compare to life before their cancer treatment, before breast surgery, and before breast cancer diagnosis?”, “What words do women use to describe their life experiences and what is the relevance Read Full Text

Qualitative Data Processing: Minding the Knowledge Gaps

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 34-37).

Once all the data for a qualitative study have been created and gathered, they are rarely ready to be analyzed without further analytic work of some nature being done. At this stage the researcher is working with preliminary data from a collective datasetKnowledge gap that most often must be processed in any number of ways before “sense making” can begin.

For example, it may happen that after the data collection stage has been completed in a qualitative research study, the researcher finds that some of the information that was to be gathered from one or more participants is missing. In a focus group study, for instance, the moderator may have forgotten to ask participants in one group discussion to address a particular construct of importance—such as, the feeling of isolation among newly diagnosed cancer patients. Or, in a content analysis, a coder may have failed to code an attribute in an element of the content that should have been coded.

In these cases, and following from a Total Quality Framework (TQF) perspective, the researcher has the responsibility to actively decide whether or not to go back and fill in the gap in the data when that is possible. Regardless of what decision the researcher makes about these potential problems that are discovered during the data processing stage, the researcher working from the TQF perspective should keep these issues in mind when the analyses and interpretations of the findings are conducted and when the findings and recommendations are disseminated.

It should also be noted that the researcher has the opportunity to mind these gaps during the data collection process itself by continually monitoring interviews or group discussions. As discussed in this Research Design Review article, the researcher should continually review the quality of completions by addressing such questions as Did every interview cover every question or issue important to the research? and Did all interviewees provide clear, unambiguous answers to key questions or issues? In doing so, the researcher has mitigated the potential problem of knowledge gaps in the final data.

 

 

Image captured from: https://modernpumpingtoday.com/bridging-the-knowledge-gap-part-1-of-2/