Researcher Bias

Shared Constructs in Research Design: Part 2 — Bias

Part 1 of the discussion of shared constructs — “Shared Constructs in Research Design: Part 1 – Sampling” — acknowledges the distinctiveness between quantitative and qualitative research while Research biashighlighting the notion that there are fundamental constructs common to a quality approach to research design regardless of method or, in the case of qualitative research, paradigm orientation. Three such constructs are sampling, bias, and validity. Part 1 of this discussion focused on sampling (prefaced by a consideration of paradigms in qualitative research and the importance of quality research design regardless of orientation). This article (Part 2) discusses bias.

Bias in qualitative research design has been the topic of a number of articles in Research Design Review over the years. One of these articles is a broad discussion on paying attention to bias in qualitative research and another explores social desirability bias in online research. An article written in 2014 examines the role of empathy in qualitative research and its potential for enhancing clarity while reducing the bias in qualitative data, and another article in RDR talks about visual cues and the importance of visual cues in mitigating sources of bias in qualitative research. Other articles concerning bias in RDR are specific to methods. For example, a couple of articles discuss mitigating interviewer bias in the in-depth interview method — “In-depth Interviewer Effects: Mitigating Interviewer Bias” and “Interviewer Bias & Reflexivity in Qualitative Research” — while another article focuses on ethnography and mitigating observer bias, and a fourth article considers the potential bias in mobile (smartphone) qualitative research.

Others in the field of psychology have discussed various aspects of bias in qualitative research. For example, Linda Finlay (2002) discusses the value of reflexivity as a tool to, among other things, “open up unconscious motivations and implicit biases in the researcher’s approach” (p. 225). Ponterotto (2005) looks at the varying role and understanding of bias across paradigm orientations in qualitative research among the postpositivists, constructivist–interpretivist researchers, and critical–ideological researchers. In psychiatry, Whitley & Crawford (2005) suggest ways to mitigate investigator bias and thereby increase the rigor in qualitative studies. Morrow (2005) asserts that “all research is subject to researcher bias” and highlights the subjectivity inherent in qualitative research and explores bracketing and reflexivity as a means of “making one’s implicit assumptions and biases overt to self and others” (p. 254). And researcher bias is central to the Credibility component of the Total Quality Framework (Roller & Lavrakas, 2015).

Social scientists such as Williams & Heikes (1993) examine the impact of interviewer gender on social desirability bias in qualitative research; while Armour, Rivaux, and Bell (2009) discuss researcher bias within the context of analysis and interpretation of two phenomenological studies. In a recent paper, Howlett (2021) reflects on the transition to online technical research solutions and the associated methodological considerations, such as the negative impact of selection bias due to weak recruitment and engagement strategies.

Among healthcare researchers, Arcury & Quandt (1999) discuss recruitment with a focus on sampling and the use of gatekeepers, with an emphasis on the potential for selection bias which they monitored by way of reviewing “the type of clients being referred to us, relative to the composition of the site clientele” (p. 131). Whittemore, Chase, & Mandle (2001) define quality in qualitative research by way of validity standards, including investigator bias — “…a phenomenological investigation will need to address investigator bias (explicitness) and an emic perspective (vividness) as well as explicate a very specific phenomenon in depth (thoroughness)” (p. 529). And Morse (2015), who is a pioneer in qualitative health research and has written extensively on issues of quality in qualitative research design, highlights the mitigation of researcher bias as central to the validity of qualitative design, offering “the correction of researcher bias” as one recommended strategy for “establishing rigor in qualitative inquiry” (p. 33).

Another shared and much discussed construct among qualitative researchers — validity — is the focus of Part 3 in this discussion.

Arcury, T. A., & Quandt, S. A. (1999). Participant recruitment for qualitative research: A site-based approach to community research in complex societies. Human Organization, 58(2), 128–133. Retrieved from http://www.metapress.com.proxy.wm.edu/content/t5g838w7u1761868/fulltext.pdf

Armour, M., Rivaux, S. L., & Bell, H. (2009). Using context to build rigor: Application to two hermeneutic phenomenological studies. Qualitative Social Work, 8(1), 101–122. https://doi.org/10.1177/1473325008100424

Finlay, L. (2002). Negotiating the swamp: The opportunity and challenge of reflexivity in research practice. Qualitative Research, 2(2), 209–230. Retrieved from http://qrj.sagepub.com/cgi/doi/10.1177/146879410200200205

Howlett, M. (2021). Looking at the ‘field’ through a Zoom lens: Methodological reflections on conducting online research during a global pandemic. Qualitative Research, 146879412098569. https://doi.org/10.1177/1468794120985691

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52(2), 250–260. https://doi.org/10.1037/0022-0167.52.2.250

Morse, J. M. (2015). Critical analysis of strategies for determining rigor in qualitative inquiry. Qualitative Health Research, 25(9), 1212–1222. https://doi.org/10.1017/CBO9781107415324.004

Ponterotto, J. G. (2005). Qualitative research in counseling psychology: A primer on research paradigms and philosophy of science. Journal of Counseling Psychology, 52(2), 126–136. https://doi.org/10.1037/0022-0167.52.2.126

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach. New York: Guilford Press.

Whitley, R., & Crawford, M. (2005). Qualitative research in psychiatry. Canadian Journal of Psychiatry, 50(2), 108–114. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/15807227

Whittemore, R., Chase, S. K., & Mandle, C. L. (2001). Validity in qualitative research. Qualitative Health Research, 11(4), 522–537. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11521609

Williams, C. L., & Heikes, E. J. (1993). The importance of researcher’s gender in the in-depth interview: Evidence from two case studies of male nurses. Gender and Society, 7(2), 280–291.

In-depth Interviewer Effects: Mitigating Interviewer Bias

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 83-84).

The outcome of a qualitative in-depth interview (IDI) study, regardless of mode, is greatly affected by the interviewer’s conscious or unconscious influence within the context of the IDIs—that is, the absence or presence of interviewer bias. The interviewer’s Interviewer Effects-Biasdemographic characteristics (e.g., age, race), physical appearance in face-to-face IDIs (e.g., manner of dress), voice in face-to-face and telephone IDIs (e.g., a regional accent), and personal values or presumptions are all potential triggers that may elicit false or inaccurate responses from interviewees. For example, imagine that an IDI study is being conducted with a group of public school teachers who are known to harbor negative feelings toward the district’s superintendent but who express ambivalent attitudes in the interviews as the result of the interviewers’ inappropriate interjection of their own personal positive opinions. In this way, the interviewers have caused the findings to be biased. In order to minimize this potential source of distortion in the data, the researcher can incorporate a number of quality enhancement measures into the IDI study design and interview protocol:

  • The IDI researcher should conduct a pretest phase during which each interviewer practices the interview and learns to anticipate what Sands and Krumer-Nevo (2006) call “master narratives” (i.e., the interviewer’s own predispositions) as well as “shocks” that may emerge from interviewees’ responses. Such an awareness of one’s own predispositions as an interviewer and possible responses from interviewees that might otherwise “jolt” the interviewer will more likely facilitate an uninterrupted interview that can smoothly diverge into other appropriate lines of questioning when the time presents itself. In this manner, the interviewer can build and maintain strong rapport with the interviewee as well as anticipate areas within the interview that might bias the outcome.

For example, Sands and Krumer-Nevo (2006) relate the story of a particular interview in a study among youth who, prior to the study, had been involved in drug use and other criminal behavior. Yami, the interviewer, approached one of the interviews with certain assumptions concerning the interviewee’s educational background and, specifically, the idea that a low-level education most likely contributed to the youth’s illicit activities. Because of these stereotypical expectations, Yami entered the interview with the goal of linking the interviewee’s “past school failures” to his current behavior and was not prepared for a line of questioning that was not aimed at making this connection. As a result Read Full Text

Ethnography: Mitigating Observer Bias

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 207-212).

In qualitative research, the researcher – including the in-depth interviewer, focus group moderator, coder in content Observationanalysis, and observer – is the instrument, meaning that the qualitative researcher wields substantial control in the design content, the gathering of data, the outcomes, and interpretation of the research.  Ethnography is no different in that the observer – albeit not controlling participants’ natural environment – plays a central role in creating the data for the study by way of recording observations.  In this respect, the credibility of an ethnographic study essentially rests on the observer’s ability to identify and record the relevant observations.

The necessary observer skills have been discussed elsewhere in Research Design Review – for example, “The Importance of Analytical Sensibilities to Observation in Ethnography.” Without these skills, an observer has the potential for biasing the data which in turn will negatively impact the analysis, interpretation, transferability, and ultimate usefulness of an ethnographic study.  The potential for bias exists regardless of observer role. An offsite, non-participant observer may knowingly or not impose subjective values on an observed event – e.g., ignoring certain comments the observer finds personally offensive in a study of an online forum discussing alcohol use – while an onsite observer, operating either overtly or covertly, may bias results by way of Read Full Text