Total Quality Framework

In-depth Interviewer Effects: Mitigating Interviewer Bias

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 83-84).

The outcome of a qualitative in-depth interview (IDI) study, regardless of mode, is greatly affected by the interviewer’s conscious or unconscious influence within the context of the IDIs—that is, the absence or presence of interviewer bias. The interviewer’s Interviewer Effects-Biasdemographic characteristics (e.g., age, race), physical appearance in face-to-face IDIs (e.g., manner of dress), voice in face-to-face and telephone IDIs (e.g., a regional accent), and personal values or presumptions are all potential triggers that may elicit false or inaccurate responses from interviewees. For example, imagine that an IDI study is being conducted with a group of public school teachers who are known to harbor negative feelings toward the district’s superintendent but who express ambivalent attitudes in the interviews as the result of the interviewers’ inappropriate interjection of their own personal positive opinions. In this way, the interviewers have caused the findings to be biased. In order to minimize this potential source of distortion in the data, the researcher can incorporate a number of quality enhancement measures into the IDI study design and interview protocol:

  • The IDI researcher should conduct a pretest phase during which each interviewer practices the interview and learns to anticipate what Sands and Krumer-Nevo (2006) call “master narratives” (i.e., the interviewer’s own predispositions) as well as “shocks” that may emerge from interviewees’ responses. Such an awareness of one’s own predispositions as an interviewer and possible responses from interviewees that might otherwise “jolt” the interviewer will more likely facilitate an uninterrupted interview that can smoothly diverge into other appropriate lines of questioning when the time presents itself. In this manner, the interviewer can build and maintain strong rapport with the interviewee as well as anticipate areas within the interview that might bias the outcome.

For example, Sands and Krumer-Nevo (2006) relate the story of a particular interview in a study among youth who, prior to the study, had been involved in drug use and other criminal behavior. Yami, the interviewer, approached one of the interviews with certain assumptions concerning the interviewee’s educational background and, specifically, the idea that a low-level education most likely contributed to the youth’s illicit activities. Because of these stereotypical expectations, Yami entered the interview with the goal of linking the interviewee’s “past school failures” to his current behavior and was not prepared for a line of questioning that was not aimed at making this connection. As a result <!–more–Read Full Text>of her predisposition, Yami failed to acknowledge and question the interviewee when he talked about being a “shy, lonely boy” and, consequently, stifled the life story that the interviewee was trying to tell her.

 

  • The interviewer should use follow-up and probing questions to encourage the interviewee to elaborate on a response (e.g., “Can you tell me more about the last time the other students harassed you at school?”), but not in a manner that could be perceived as seeking any particular “approved” substantive response.

 

  • Using a reflexive journal is an important and necessary feature of an IDI study design. This device enhances the credibility of the research by ensuring that each interviewer keeps a record of his/her experiences in the field and how he or she may have biased interview outcomes. The interviewer reflects carefully after each completed IDI and records how he or she may have distorted the information gathered in the interview (inadvertent as it may have been) and how the interviewee’s behavior and other factors may have contributed to this bias. This “reflexive objectivity” (Brinkmann & Kvale, 2015) helps the interviewer gain “sensitivity about one’s [own] prejudices, one’s subjectivity” (p. 278) and consider the impact of these influences on the credibility of the data. This objectivity might also reflect on any personal characteristics of the interviewer (e.g., voice parameters, personality traits, demographics) that affected the interview and resulted in unintended variation across all IDIs. By way of the reflexive journal, the research is enriched with a documented firsthand account of any interviewer bias or presumptions as well as variations in the interviewer’s handling of interviews throughout the study.

 

  • A reflexive journal can also be used in the triangulation of interview data. From a Total Quality Framework perspective, a best practice is to have an impartial research team member review the audio or video recordings from one or more IDIs to identify how and under what circumstance the interviewer may have biased interviewees’ responses. In turn, this review can be used in cross-reference with the interviewer’s reflexive journal and discussed with the interviewer to help him/her better understand lapses in self-awareness. This journal also becomes an important component of the study’s audit trail and a tool in the final data analysis and interpretation.

 

Brinkmann, S., & Kvale, S. (2015). Interviews: Learning the craft of qualitative research interviewing (3rd ed.). Thousand Oaks, CA: Sage.

Sands, R. G., & Krumer-Nevo, M. (2006). Interview shocks and shockwaves. Qualitative Inquiry, 12(5), 950–971. https://doi.org/10.1177/1077800406288623

Image captured from: http://www.jeannievodden.com/light-effects-11-x-15-c2009/

 

 

Critical Thinking in Qualitative Research Design

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 20-21).

Many researchers and scholars have advanced strategies, criteria, or frameworks for thinking about and promoting the importance of “the quality” of qualitative research at some stage in the research design. Critical thinking skills in qualitative researchOne such strategy is the framework developed by Levitt et al. (2017) that centers on methodological integrity. Another is the Total Quality Framework (TQF) which has been discussed throughout Research Design Review, as in the article titled “The ‘Quality’ in Qualitative Research Debate & the Total Quality Framework.”

The strategies or ways of thinking about quality in qualitative research that are most relevant to the TQF are those that are (a) paradigm neutral, (b) flexible (i.e., do not adhere to a defined method), and (c) applicable to all phases of the research process. Among these, the work of Lincoln and Guba (e.g., 1981, 1985, 1986, and 1995) is the most noteworthy. Although they profess a paradigm orientation “of the constructionist camp, loosely defined” (Lincoln et al., 2011, p. 116), the quality criteria Lincoln and Guba set forth nearly 30 years ago is particularly pertinent to the TQF in that it advances the concept of trustworthiness as a major criterion for judging whether a qualitative research study is “rigorous.” In their model, trustworthiness addresses the issue of “How can a [qualitative researcher] persuade [someone] that the findings of a [study] are worth paying attention to, worth taking account of?” (Lincoln & Guba, 1985, p. 290). That is, what are the criteria upon which such an assessment should be based? In this way, Lincoln and Guba espouse standards that are flexible (i.e., can be adapted depending on the research context) as well as relevant throughout the research process.

In answering, they put forth the criteria of credibility, transferability, dependability, and confirmability. For Lincoln and Guba (1985), credibility Read Full Text

Ethnography: A Case Study in a Quality Approach

As discussed elsewhere in Research Design Review, the Total Quality Framework (TQF) is “a useful tool for qualitative researchers to apply in designing, conducting, and interpreting their research so that the studies are more likely to (a) gather high-quality data, (b) lead to more robust and valid social justiceinterpretations of the data, and (c) ultimately generate highly useful outcomes.” The basic research principles that underlie the TQF can be applied to various qualitative methods.

The following is an excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (pp. 227-229) which summarizes an ethnographic study conducted by Todd (2012) concerning religious network organizations and their association with social justice at the local community level. This case study exemplifies many of the principles supported by the TQF approach — illustrated by the clearly stated purpose, the stated justification for the chosen method, and the attention to quality-enhancing details throughout the study.

Purpose

Religious networking organizations are structured groups consisting of people from multiple religious congregations that meet regularly to discuss common interests. The primary purpose of this study was to examine how and why these organizations work for social justice in their local community and how religion is integrated into the organizations’ work in social justice.

Method

An ethnographic approach was considered appropriate because of the distinctive insight it could give into the organization members’ personal experiences, as well as the proven benefit of ethnography, by other researchers in community psychology, in identifying and understanding the storied lives of individuals and social processes within community-based environments.

Design

Credibility (Data Collection)

Scope

Two networking organizations were included in this study. Both organizations are located in the same Midwestern community. The researcher became aware of, and was introduced to, these organizations by way of contacts (gatekeepers) within the community. The researcher assumed the role of an overt participant observer, attending monthly 2-hour meetings at both organizations for approximately 1½ years. The ethnographer’s involvement Read Full Text