researcher bias

Limitations of In-person Focus Group Discussions

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 116-119).

The interactive, dynamic aspect of the focus group discussion method is its greatest potential strength as well as its greatest potential liability. This is especially the case in the face-to-face, in-person limitations of focus groupsmode where the close physical proximity of participants can unleash any number of factors that will threaten data quality if left unchecked.

One of the most important factors is the caliber of the discussion; specifically, the extent to which all participants have a fair chance of voicing their input. This is critical because the success of the group discussion method hinges on generating a true discussion where everyone present participates in a dialogue with the other group members and, to a lesser degree, with the moderator. A true participatory discussion, however, can be easily jeopardized in the social context of the in-person focus group (as well as the online synchronous discussion mode) because one or more participants either talk too much (i.e., dominate the discussion) or talk too little (i.e., are hesitant to express their views). In either case, the quality of the data will be compromised by the failure to capture the viewpoints of all participants, leading to erroneous interpretations of the outcomes.

The potentially negative impact that the face-to-face group interaction can have on data quality is an important consideration in qualitative research design, yet this impact—or, the effect of group interaction on the research—is often overlooked when conducting the analyses and reporting the outcomes. Researchers who have explored the role of interaction in focus group research include Grønkjær et al. (2011) and Moen, Antonov, Nilsson, and Ring (2010). Grønkjær et al. analyzed the “interactional events” in five focus groups they conducted with Danes on Read Full Text

Ethnography: Mitigating Observer Bias

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 207-212).

In qualitative research, the researcher – including the in-depth interviewer, focus group moderator, coder in content Observationanalysis, and observer – is the instrument, meaning that the qualitative researcher wields substantial control in the design content, the gathering of data, the outcomes, and interpretation of the research.  Ethnography is no different in that the observer – albeit not controlling participants’ natural environment – plays a central role in creating the data for the study by way of recording observations.  In this respect, the credibility of an ethnographic study essentially rests on the observer’s ability to identify and record the relevant observations.

The necessary observer skills have been discussed elsewhere in Research Design Review – for example, “The Importance of Analytical Sensibilities to Observation in Ethnography.” Without these skills, an observer has the potential for biasing the data which in turn will negatively impact the analysis, interpretation, transferability, and ultimate usefulness of an ethnographic study.  The potential for bias exists regardless of observer role. An offsite, non-participant observer may knowingly or not impose subjective values on an observed event – e.g., ignoring certain comments the observer finds personally offensive in a study of an online forum discussing alcohol use – while an onsite observer, operating either overtly or covertly, may bias results by way of Read Full Text

Paying Attention to Bias in Qualitative Research: A Message to Marketing Researchers (& Clients)

Researchers of all ilk care about bias and how it may creep into their research designs resulting in measurement error.  This is true among quantitative researchers as well as among qualitative researchers who routinely head-in-the-sand-2demonstrate their sensitivity to potential bias in their data by way of building interviewer training, careful recruitment screening, and appropriate modes into their research designs.  It is these types of measures that acknowledge qualitative researchers’ concerns about quality data; and yet, there are many other ways to mitigate bias in qualitative research that are often overlooked.

Marketing researchers (and marketing clients) in particular could benefit from thinking more deeply about bias and measurement error.  In the interest of “faster, cheaper, better” research solutions, marketing researchers often lose sight of quality design issues, not the least of which concern bias and measurement error in the data.  If marketing researchers care enough about mitigating bias to train interviewers/moderators, develop screening questions that effectively target the appropriate participant, and carefully select the suitable mode for the population segment, then it is sensible to adopt broader design standards that more fully embrace the collecting of quality data.

An example of a tool that serves to raise the design standard is the reflexive journal.  The reflexive journal has been the subject (in whole or in part) of many articles in Research Design Review, most notably Read Full Text