The impact of bias (in various forms) on research outcomes is well-documented. In Research Design Review alone, there are many articles related to this issue; bias in the world of both quantitative – such as “Ask Someone a Question, You’ll Get an Answer” and “Accounting for Social Desirability Bias in Online Research” – as well as purely qualitative – “Selection Bias & Mobile Qualitative Research” and “Visual Cues & Bias in Qualitative Research” – research.   One of the more significant sources of bias in qualitative research is the researcher, i.e., the in-depth interviewer, focus group moderator, or observer in ethnography. This bias is specifically addressed in the RDR article “Interviewer Bias & Reflexivity in Qualitative Research,” which highlights the importance of the reflexive journal to help address “the bias that most assuredly permeates the socially-dependent nature of qualitative research.”

An interviewer may bias research outcomes in any number of ways. For instance, he or she may allow personal beliefs or expectations to skew how questions are asked and/or responses are recorded. Or, the interviewer’s physical characteristics (e.g., associated with gender, race, ethnicity, as well as manner of dress and demeanor) may weaken the interviewer-interviewee relationship and an otherwise trusting research environment which is essential to gaining accurate and useful qualitative data.

It is not, however, only interviewer bias that can lead to distorted outcomes but also interviewer inconsistency. This is an important distinction. An interviewer that has biased the results has done something to provoke false information from the interviewee in response to the research questions. Interviewer inconsistency, on the other hand, does not lead to inaccurate information from the interviewee but rather variation in the data that does not truly exist. A researcher, for example, conducting face-to-face interviews with public school teachers about their use of electronic media in the classroom may do nothing to elicit erroneous information from the teachers yet produce data suggesting a wide range of media use when, in fact, this is not the reality. The researcher might do this by: 1) not specifying what is included under “electronic media” for some participants; 2) defining it as audio and video recordings and PowerPoint presentations for other participants; and 3) defining electronic media as audio/video recordings, slide presentations, the Internet, television, radio, phone, and computer devices for yet another set of participants. The interviewer’s inconsistent reference to “electronic media” will ultimately produce an unrealistic picture of what actually goes on in the classroom, a picture that suggests a greater variation in the use of electronic media than is true.

A classic example of the perils of inconsistency can be found in research intended to gain participants’ reactions to something new – such as a new product, service, or program. In order to gain an accurate measure of the viability of a new concept, it is critical that the interviewer or focus group moderator introduce this new idea by way of a prepared concept statement that is simple to understand and void of any “sales talk.” This statement must consistently be read to all interviewees or group participants. If not – if some interviewees/participants are read a well-prepared descriptive statement while others are introduced to the new concept via the researcher’s off-the-cuff remarks – participants wind up reacting to different versions of the concept and, in the end, the researcher has no way of honestly knowing whether the proposed product, service, or program “has legs.”

Inconsistency also presents problems in observational research (i.e., ethnography). Consider an ethnographic study involving the observation of passengers at major train stations on the East Coast, with a particular focus on observations related to 1) passengers waiting for a train, 2) unexpected delays in the train schedules, and 3) passengers boarding a train. An observer who (due to fatigue or for other reasons) fails to consistently observe these three target scenarios – e.g., observes all three situations at some stations, passengers waiting and unexpected delays at other stations, and only passengers boarding at still other stations – jeopardizes the research outcomes and ultimately provides data of little value.

Let’s be clear. Qualitative research thrives on the flexibility and nimbleness of the researcher. This is an important quality that allows the researcher to reap all the complexity and context inherent in gaining meaning in qualitative research. But a good qualitative researcher understands that flexibility is not the same thing as an “anything goes” approach where no consideration is given to how the data are gathered. Like researcher bias, knowing when and how to avoid inconsistency – and add consistency – in data collection is an essential ingredient in the recipe for a quality qualitative research design.

Image captured from: http://headache.answers.com