transparency

Ethnography: An Example of Transparent Reporting

A portion of the following is an excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, p. 221).

Ethnography: An Example of TransparencyThe important component in research design concerning transparency has been discussed many times in Research Design Review. And indeed, Transparency is the third component of the Total Quality Framework. The integrity and ultimate Usefulness of qualitative research hinges on exposing design and data collection details in the final reporting documents.

An excellent example of transparency can be found in “Impacts of intensified police activity on injection drug users: Evidence from an ethnographic investigation” (Small et al., 2006). Here, the authors report on a participant–observation study that was conducted to complement a broader study concerning the impact of enforcement on illicit drug-use-related behavior. Their description of what went on in the field is a good example of giving the reader a clear understanding of the field activity:

Trained observers spent time “hanging out” in and around locales where drug sales and injecting took place, talking to and interacting with drug users. Discussions, occurrences, and observations were documented in fieldnotes. Observational data recorded in extensive fieldnotes included: location and character of public injection venues; syringe acquisition, availability, and disposal; public drug consumption patterns for injection and non-injection drugs; and description of public drug users. . . . Each observational field visit incorporated two hours of participant–observation conducted in streets and alleys as well as time spent writing fieldnotes to document observations and discussions. A target area and schedule of observations was devised, drawing on previous ethnographic research examining needle exchange practices. . . . The observations targeted both street-side and in the alleyways along 10 blocks of Hastings Street, where numerous clusters of drug market and consumption activity were identified by ethnographic mapping techniques. . . . Observations were distributed between morning, afternoon, and evening hours, with an increased number of observations occurring around monthly welfare payments when public drug scene and police activity increases. As some drug market and using locales shifted and new ones emerged, ethnographic data collection activities were altered accordingly to survey the largest portion of the open drug using scene, including areas far outside the central Hastings corridor. (pp. 86–87)

 

Small, W., Kerr, T., Charette, J., Schechter, M. T., & Spittal, P. M. (2006). Impacts of intensified police activity on injection drug users: Evidence from an ethnographic investigation. International Journal of Drug Policy, 17(2), 85–95. https://doi.org/10.1016/j.drugpo.2005.12.005

A TQF Approach to Construct Validity

TQF approach to construct validity

Construct validity plays an important role in the design, implementation, analysis, and ultimate usefulness of qualitative research methods. The construct of validity itself in qualitative research is discussed in this article and cites qualitative researchers across disciplines who explore “unique dimensions” and other considerations  relating to validity in qualitative research.

The Total Quality Framework (TQF) relies heavily on construct validity in its quality approach to each phase of the qualitative research process. At each phase, the researcher must ask “Am I gaining real knowledge about the core concepts that are the focus of this research?” For example,

  • An important step when developing a research design is to identify the key constructs associated with the research objectives to investigate, and the particular attributes of each construct that the researcher wants to explore. So, for example, a researcher conducting a study on dietary behavior may have interest in “health consciousness,” including shopping behavior related to organic and fresh foods.
  • In the in-depth interview and focus group discussion methods, careful attention needs to be paid to guide development and the inclusion of questions relevant to the constructs of interest. When developing the guide, the researcher needs to ask “Is this [topic, question, technique] relevant to the construct we are investigating?”, and “Does this [topic, question, technique] provide us with knowledge about the aspect of the construct that we intended to explore in the interviews/discussions?”
  • In ethnography, the observation guide and observation grid are important tools. “The grid is similar to the guide in that it helps to remind the observer of the events and issues of most import; however, the observation grid is a spreadsheet or log of sorts that enables the observer to actually record and reflect on observable events in relationship to the research constructs of interest” (Roller & Lavrakas, 2015, p. 206).
  • The quality of qualitative data analysis hinges on the researcher’s ability to effectively identify, analyze, and develop valid interpretations of the data around the important constructs associated with the research objectives. To assist the researcher, a TQF approach to analysis recommends a codebook format and coding form (which is basically a reflexive journal for the coder[s] to record thoughts and justifications for their coding decisions) that highlights constructs of interest. For example,

TQF codebook and coding form

  • Construct validity also plays an important role in the transparency of the final research document. In the study report, the researcher can (and should) elaborate on the design, data gathering, and analysis decisions that were made pertaining to the key constructs, as well as the main themes that were derived from the data — i.e., the knowledge that was gained from the research — concerning these constructs.

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach. New York: Guilford Press.

 

Photo by Bon Vivant

Research Integrity & a Total Quality Framework Approach to Qualitative Data Sharing

The September 2021 issue of Monitor on Psychology from the American Psychological Association includes an article “Leading the Charge to Address Research Misconduct” by Stephanie Pappas. The article discusses the various Qualitative data sharingcircumstances or “pressures” that may lead researchers towards weak research practices that result in anything from “honest” mistakes or errors (e.g., due to insufficient training or oversight) to deliberate “outright misconduct” (e.g., falsifying data, dropping outliers from the analysis and reporting). The article goes on to talk about what psychologists are doing to tackle the problem.

One of those psychologists is James DuBois, DSc, PhD at Washington University School of Medicine. Dr. DuBois and his colleague Alison Antes PhD direct the P.I. (professionalism and integrity in research) Program at Washington University. This program offers one-on-one coaching to researchers who are challenged by the demands of balancing scientific and compliance requirements, as well as researchers who have (or have staff who have) been investigated for noncompliance or misconduct. The P.I. Program also conducts an On the Road Workshop which is an onsite session for researchers “doing empirical research in funded research environments” covering such areas as decision-making strategies, effective communication, and professional growth goals.

Another approach to the problem of misconduct and the goal of research integrity is transparency by way of sharing data (and other elements of design), allowing other researchers the opportunity to examine research practices and substantiate the reported results. Dr. DuBois and his co-authors discuss this and other advantages to sharing qualitative data in their 2018 article “Is It Time to Share Qualitative Research Data?” The authors assert that allowing other researchers to assess supporting evidence and “comprehensiveness by examining our data may improve the quality of research by enabling correction and increasing attention to detail” (p. 384).

In response to DuBois et al., Roller and Lavrakas (2018) published a commentary expressing Read Full Text