Best Practices

Research Integrity & a Total Quality Framework Approach to Qualitative Data Sharing

The September 2021 issue of Monitor on Psychology from the American Psychological Association includes an article “Leading the Charge to Address Research Misconduct” by Stephanie Pappas. The article discusses the various Qualitative data sharingcircumstances or “pressures” that may lead researchers towards weak research practices that result in anything from “honest” mistakes or errors (e.g., due to insufficient training or oversight) to deliberate “outright misconduct” (e.g., falsifying data, dropping outliers from the analysis and reporting). The article goes on to talk about what psychologists are doing to tackle the problem.

One of those psychologists is James DuBois, DSc, PhD at Washington University School of Medicine. Dr. DuBois and his colleague Alison Antes PhD direct the P.I. (professionalism and integrity in research) Program at Washington University. This program offers one-on-one coaching to researchers who are challenged by the demands of balancing scientific and compliance requirements, as well as researchers who have (or have staff who have) been investigated for noncompliance or misconduct. The P.I. Program also conducts an On the Road Workshop which is an onsite session for researchers “doing empirical research in funded research environments” covering such areas as decision-making strategies, effective communication, and professional growth goals.

Another approach to the problem of misconduct and the goal of research integrity is transparency by way of sharing data (and other elements of design), allowing other researchers the opportunity to examine research practices and substantiate the reported results. Dr. DuBois and his co-authors discuss this and other advantages to sharing qualitative data in their 2018 article “Is It Time to Share Qualitative Research Data?” The authors assert that allowing other researchers to assess supporting evidence and “comprehensiveness by examining our data may improve the quality of research by enabling correction and increasing attention to detail” (p. 384).

In response to DuBois et al., Roller and Lavrakas (2018) published a commentary expressing Read Full Text

Focus Groups: Moving to the Online Face-to-face Mode

There are many articles in Research Design Review about the focus group method. They range from broad discussions concerning the strengths and limitations of focus group discussions in qualitative research, to determining the number of groups to conduct for a particular study, to considerations Online synschronous focus groupwhen deciding on the heterogeneity or homogeneity of focus group participants, to matters of moderating such as the importance of gaining individual thinking in the group environment.

Most of these articles pertain to the in-person mode, where the moderator meets group participants at a local facility to discuss the research topic for 90 minutes to two hours. Alternatively, there are a variety of online solutions for the focus group method. One of the most popular are online asynchronous discussions (sometimes called “bulletin boards”) that take place over two to three or more days. As discussed in a brief 2018 article, there are a number of strengths and limitations to the online asynchronous mode, including the advantages of flexibility, geographic spread of participants, and potential for multi-media input; as well as limitations such as that having to do with the absence of visual cues, managing participant engagement, and conducting the analysis.

As I write this in mid-March 2020, many researchers are scrambling to find ways to re-design their in-person focus group research during the current coronavirus pandemic crisis. In doing so, these researchers are taking a close look at moving from in-person discussions to an online mode that allows for some semblance of in-person groups by way of face-to-face, real-time interaction, i.e., synchronous video conferencing. For some (if not, most) of these researchers, the online face-to-face mode is a new experience and, as such, researchers are uncertain on how to proceed on two key facets of the research design: 1) the online service or platform they should use and 2) best practices when conducting online synchronous group discussions for research purposes.

With respect to the online service or platform, the researcher needs to weigh the scope of the study (e.g., type of participant) as well as the depth and breadth of the discussion guide. While simple interfaces such as those provided by Zoom, Webex, or GoToMeeting may offer the video interface, the researcher needs to think about what they may or may not be giving up in terms of the quality of the discussion. For instance, dedicated online qualitative research platforms – such as itracks, 20/20 Research, Civicom, Discuss.io, and others – offer features and capabilities designed specifically for the demands of qualitative research. This includes the capacity to go beyond simple video conferencing (e.g., recording, screen sharing, and transcripts) by way of: recruiting participants; providing a community dashboard; aiding in question development; enabling in-discussion participant activity capabilities such as marking up images and creating collages; an observer “back room”; and various analytical functions such as image tagging as well as keyword and sentiment analysis.

In terms of best practices when conducting online synchronous discussions, here are a few resources:

“Considerations for and Lessons Learned from Online, Synchronous Focus Groups” (Forrestal, D’Angelo, and Vogel, 2015)

“Best Practices for Synchronous Online Focus Groups” (Lobe, 2017)

Online Moderator Training with Casey Sweet and Jeff Walkowski

Although there are clearly limitations to the online mode in qualitative research (as mentioned earlier), there are also times and extraordinary situations (such as the current pandemic) when it is the best approach. In these times, it is incumbent on the researcher to think carefully about maintaining the integrity of their research as they move to an online face-to-face mode, to reflect on what was lost and gained in this approach, and to be transparent in the reporting of this research.

Forrestal, S. G., D’Angelo, A. V., & Vogel, L. K. (2015). Considerations for and lessons learned from online, synchronous focus groups. Survey Practice, 8(2), 1-8.

Lobe, B. (2017). Best Practices for Synchronous Online Focus Groups. In A New Era in Focus Group Research (pp. 227-250). Palgrave Macmillan, London.

Images captured from: https://pixabay.com/vectors/monitor-screen-computer-electronics-1143202/ and https://www.istockphoto.com/illustrations/cartoon-people?mediatype=illustration&phrase=cartoon%20people&sort=mostpopular

Critical Thinking in Qualitative Research Design

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 20-21).

Many researchers and scholars have advanced strategies, criteria, or frameworks for thinking about and promoting the importance of “the quality” of qualitative research at some stage in the research design. Critical thinking skills in qualitative researchOne such strategy is the framework developed by Levitt et al. (2017) that centers on methodological integrity. Another is the Total Quality Framework (TQF) which has been discussed throughout Research Design Review, as in the article titled “The ‘Quality’ in Qualitative Research Debate & the Total Quality Framework.”

The strategies or ways of thinking about quality in qualitative research that are most relevant to the TQF are those that are (a) paradigm neutral, (b) flexible (i.e., do not adhere to a defined method), and (c) applicable to all phases of the research process. Among these, the work of Lincoln and Guba (e.g., 1981, 1985, 1986, and 1995) is the most noteworthy. Although they profess a paradigm orientation “of the constructionist camp, loosely defined” (Lincoln et al., 2011, p. 116), the quality criteria Lincoln and Guba set forth nearly 30 years ago is particularly pertinent to the TQF in that it advances the concept of trustworthiness as a major criterion for judging whether a qualitative research study is “rigorous.” In their model, trustworthiness addresses the issue of “How can a [qualitative researcher] persuade [someone] that the findings of a [study] are worth paying attention to, worth taking account of?” (Lincoln & Guba, 1985, p. 290). That is, what are the criteria upon which such an assessment should be based? In this way, Lincoln and Guba espouse standards that are flexible (i.e., can be adapted depending on the research context) as well as relevant throughout the research process.

In answering, they put forth the criteria of credibility, transferability, dependability, and confirmability. For Lincoln and Guba (1985), credibility Read Full Text