qualitative research design

Qualitative Tech Solutions: Coverage & Validity Considerations

Back in 2018, Research Design Review posted an article titled “Five Tech Solutions to Qualitative Data Collection: What Strengthens or Weakens Data Quality?” The focus of this article is on a presentation given in May 2018 concerning technological alternatives TQF Credibilityto qualitative research data collection. Importantly, the aim of the presentation was, not to simply identify different approaches to data collection beyond the in-person and telephone modes but rather, to examine the strengths and limitations of these technological solutions from a data quality – specifically, Credibility – standpoint.

Broadly speaking, technological approaches to qualitative research data gathering offer clear advantages over in-person methods, particularly in the areas of:

  • Representation, e.g., geographic coverage, potential access to hard-to-reach population segments;
  • Cooperation, e.g., convenience and flexibility of time and place for participants, appropriateness for certain demographic segments (18-49 year olds*);
  • Validity associated with data accuracy, e.g., research capturing in-the-moment experiences do not rely on memory recall;
  • Validity associated with the depth of data, e.g., capturing multiple contextual dimensions through text, video, and images;
  • Validity associated with data accuracy and depth allowing for the triangulation of data;
  • Researcher effects, e.g., mitigated by the opportunity for greater reflection and consistency across research events;
  • Participant effects, e.g., mitigated by the multiple ways to express thoughts, willingness to discuss sensitive issues, and (possibly) a lower tendency for social desirability responding; and
  • Efficient use of resources (i.e., time, money, and staff).

There are also potential drawbacks to any technological solution, including those associated with:

  • Uneven Internet access and comfort with technology among certain demographic groups (e.g., sampling favors “tech savvy” individuals), hard-to-reach and marginalized segments of the population;
  • Difficulty in managing engagement, including the unique researcher skills and allocation of time required;
  • Potential participant burnout from researcher’s requests for multiple input activities and/or days of engagement. This is a type of participant effect that negatively impacts validity;
  • Nonresponse due to mode, e.g., unwillingness or inability to participate to a mostly text-based discussion;
  • Data accuracy, e.g., participant alters behavior in a study observing in-home meal preparation;
  • Missing important visual &/or verbal cues which may interfere with rapport building and an in-depth exploration of responses;
  • Difficulty managing analysis due to lots and lots of data (in volume & formats);
  • Fraud, misrepresentation – “Identity is fluid and potentially multiple on the Internet” (James and Bushner, 2009, p. 35) and people may not share certain images or video that reveal something “embarrassing” about themselves**; and
  • Security, confidentiality, anonymity (e.g., data storage, de-identification).

 

 

* https://www.pewresearch.org/internet/fact-sheet/internet-broadband/

** https://www.businesswire.com/news/home/20180409006050/en/Minute-Maid-Debuts-New-Campaign-Celebrates-Good

James, N., & Busher, H. (2009). Online interviewing. London: Sage Publications.

Focus Groups: Moving to the Online Face-to-face Mode

There are many articles in Research Design Review about the focus group method. They range from broad discussions concerning the strengths and limitations of focus group discussions in qualitative research, to determining the number of groups to conduct for a particular study, to considerations Online synschronous focus groupwhen deciding on the heterogeneity or homogeneity of focus group participants, to matters of moderating such as the importance of gaining individual thinking in the group environment.

Most of these articles pertain to the in-person mode, where the moderator meets group participants at a local facility to discuss the research topic for 90 minutes to two hours. Alternatively, there are a variety of online solutions for the focus group method. One of the most popular are online asynchronous discussions (sometimes called “bulletin boards”) that take place over two to three or more days. As discussed in a brief 2018 article, there are a number of strengths and limitations to the online asynchronous mode, including the advantages of flexibility, geographic spread of participants, and potential for multi-media input; as well as limitations such as that having to do with the absence of visual cues, managing participant engagement, and conducting the analysis.

As I write this in mid-March 2020, many researchers are scrambling to find ways to re-design their in-person focus group research during the current coronavirus pandemic crisis. In doing so, these researchers are taking a close look at moving from in-person discussions to an online mode that allows for some semblance of in-person groups by way of face-to-face, real-time interaction, i.e., synchronous video conferencing. For some (if not, most) of these researchers, the online face-to-face mode is a new experience and, as such, researchers are uncertain on how to proceed on two key facets of the research design: 1) the online service or platform they should use and 2) best practices when conducting online synchronous group discussions for research purposes.

With respect to the online service or platform, the researcher needs to weigh the scope of the study (e.g., type of participant) as well as the depth and breadth of the discussion guide. While simple interfaces such as those provided by Zoom, Webex, or GoToMeeting may offer the video interface, the researcher needs to think about what they may or may not be giving up in terms of the quality of the discussion. For instance, dedicated online qualitative research platforms – such as itracks, 20/20 Research, Civicom, Discuss.io, and others – offer features and capabilities designed specifically for the demands of qualitative research. This includes the capacity to go beyond simple video conferencing (e.g., recording, screen sharing, and transcripts) by way of: recruiting participants; providing a community dashboard; aiding in question development; enabling in-discussion participant activity capabilities such as marking up images and creating collages; an observer “back room”; and various analytical functions such as image tagging as well as keyword and sentiment analysis.

In terms of best practices when conducting online synchronous discussions, here are a couple of resources:

“Considerations for and Lessons Learned from Online, Synchronous Focus Groups” (Forrestal, D’Angelo, and Vogel, 2015)

“Best Practices for Synchronous Online Focus Groups” (Lobe, 2017)

Online Moderator Training with Casey Sweet and Jeff Walkowski

Although there are clearly limitations to the online mode in qualitative research (as mentioned earlier), there are also times and extraordinary situations (such as the current pandemic) when it is the best approach. In these times, it is incumbent on the researcher to think carefully about maintaining the integrity of their research as they move to an online face-to-face mode, to reflect on what was lost and gained in this approach, and to be transparent in the reporting of this research.

 

Forrestal, S. G., D’Angelo, A. V., & Vogel, L. K. (2015). Considerations for and lessons learned from online, synchronous focus groups. Survey Practice, 8(2), 1-8.

Lobe, B. (2017). Best Practices for Synchronous Online Focus Groups. In A New Era in Focus Group Research (pp. 227-250). Palgrave Macmillan, London.

 

 

Images captured from: https://pixabay.com/vectors/monitor-screen-computer-electronics-1143202/ and https://www.istockphoto.com/illustrations/cartoon-people?mediatype=illustration&phrase=cartoon%20people&sort=mostpopular

Focus Groups: Heterogeneity vs. Homogeneity

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 107-109).

Heterogeneity

homogeneity

Fundamental to the design of a focus group study is group composition. Specifically, the researcher must determine the degree of homogeneity or heterogeneity that should be represented by the group participants. As shown below, there are many questions the researcher needs to contemplate, such as the extent of similarity or dissimilarity in participants’ demographic characteristics, as well as in their experiences and involvement with the subject matter.

Questions When Considering Heterogeneity vs. Homogeneity
A few of the questions the focus group researcher might consider when determining the desired heterogeneity or homogeneity among group participants include:

  • Should participants be in the same age range and/or stage of life?
  • Should participants be the same gender, race, and/or ethnicity?
  • Should participants be at a similar income, socio-economic, or educational level?
  • Should participants reside in the same community, be members of the same organization(s)?
  • Should participants have similar professions or jobs (including, job titles)?
  • Should participants have a similar involvement, experience, or knowledge with the research topic, e.g., the same types of problems with their 13 year old boys? the same healthcare service provider? the same purchase behavior? the same level of expertise with a new technology?

Whether or not—or the degree to which—group participants should be homogeneous in some or all characteristics has been at the center of debate for some years. On the one hand, Grønkjær, Curtis, Crespigny, and Delmar (2011) claim that at least some “homogeneity in focus group construction is considered essential for group interaction and dynamics” (p. 23)—for example, participants belonging to the same age group may have similar frames of reference and feel comfortable sharing their thoughts with people who have lived through the same experience. In the same vein, Read Full Text