Credibility

The TQF Qualitative Research Proposal: Credibility of Design

A Total Quality Framework (TQF) approach to the qualitative research proposal has been discussed in articles posted elsewhere in Research Design Review, notably “A Quality Approach to the Qualitative Research Proposal” (2015) and “Writing Ethics Into Your Qualitative Proposal” (2018). The article presented here focuses on the Research Design section of the TQF proposal and, specifically, the Credibility component of the TQF. The Credibility component has to do with Scope and Data Gathering. This is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 339-340).

TQF Proposal Image-DesignScope

A TQF research proposal clearly defines the target population for the proposed research, the target sample (if the researcher is interested in a particular subgroup of the target population, e.g., only African American and Hispanic high school seniors in the district who anticipate graduating in the coming spring), how participants will be selected for the study, what they will be asked to do (e.g., set aside school time for an in-depth interview [IDI]), and the general types of questions to which they will be asked to respond (i.e., the content areas of the interview). In discussing Scope, the researcher proposing an IDI study with African American and Hispanic high school students would identify the list that will be used to select participants (e.g., the district’s roster of seniors who are expected to graduate); the advantages and drawbacks to using this list (e.g., not everyone on the roster may consider themselves to be African American or Hispanic); the systematic (preferably random) procedure that will be used to select the sample; and the number of students that will be selected as participants, including the rationale for that number and the steps that will be taken to gain cooperation from the students and thereby ideally ensure that everyone selected actually completes an interview (e.g., gaining permission from the school principal to allow students to take school time to participate in the IDI, and from parents/guardians for students under 18 years of age who cannot give informed consent on their own behalf).

Data Gathering

The data-gathering portion of the Research Design section of the proposal highlights the constructs and issues that will be examined in the proposed research. This discussion should provide details of the types of questions that will be asked, observations that will be recorded, or areas of interest Read Full Text

Qualitative Tech Solutions: Coverage & Validity Considerations

Back in 2018, Research Design Review posted an article titled “Five Tech Solutions to Qualitative Data Collection: What Strengthens or Weakens Data Quality?” The focus of this article is on a presentation given in May 2018 concerning technological alternatives TQF Credibilityto qualitative research data collection. Importantly, the aim of the presentation was, not to simply identify different approaches to data collection beyond the in-person and telephone modes but rather, to examine the strengths and limitations of these technological solutions from a data quality – specifically, Credibility – standpoint.

Broadly speaking, technological approaches to qualitative research data gathering offer clear advantages over in-person methods, particularly in the areas of:

  • Representation, e.g., geographic coverage, potential access to hard-to-reach population segments;
  • Cooperation, e.g., convenience and flexibility of time and place for participants, appropriateness for certain demographic segments (18-49 year olds*);
  • Validity associated with data accuracy, e.g., research capturing in-the-moment experiences do not rely on memory recall;
  • Validity associated with the depth of data, e.g., capturing multiple contextual dimensions through text, video, and images;
  • Validity associated with data accuracy and depth allowing for the triangulation of data;
  • Researcher effects, e.g., mitigated by the opportunity for greater reflection and consistency across research events;
  • Participant effects, e.g., mitigated by the multiple ways to express thoughts, willingness to discuss sensitive issues, and (possibly) a lower tendency for social desirability responding; and
  • Efficient use of resources (i.e., time, money, and staff).

There are also potential drawbacks to any technological solution, including those associated with:

  • Uneven Internet access and comfort with technology among certain demographic groups (e.g., sampling favors “tech savvy” individuals), hard-to-reach and marginalized segments of the population;
  • Difficulty in managing engagement, including the unique researcher skills and allocation of time required;
  • Potential participant burnout from researcher’s requests for multiple input activities and/or days of engagement. This is a type of participant effect that negatively impacts validity;
  • Nonresponse due to mode, e.g., unwillingness or inability to participate to a mostly text-based discussion;
  • Data accuracy, e.g., participant alters behavior in a study observing in-home meal preparation;
  • Missing important visual &/or verbal cues which may interfere with rapport building and an in-depth exploration of responses;
  • Difficulty managing analysis due to lots and lots of data (in volume & formats);
  • Fraud, misrepresentation – “Identity is fluid and potentially multiple on the Internet” (James and Bushner, 2009, p. 35) and people may not share certain images or video that reveal something “embarrassing” about themselves**; and
  • Security, confidentiality, anonymity (e.g., data storage, de-identification).

 

 

* https://www.pewresearch.org/internet/fact-sheet/internet-broadband/

** https://www.businesswire.com/news/home/20180409006050/en/Minute-Maid-Debuts-New-Campaign-Celebrates-Good

James, N., & Busher, H. (2009). Online interviewing. London: Sage Publications.

Credibility & the Online Asynchronous Focus Group Method

The Total Quality Framework (TQF) offers researchers a way to think about basic research principles at each stage of the qualitative research process – data collection, analysis, reporting – with the goal of doing something of value with the outcomes (i.e., the usefulness of the research). The first of the four components of the TQF is Credibility which pertains to the data collection phase of a qualitative study. A detailed discussion of Credibility can be found in this 2017 Research Design Review article.

This article – and in similar fashion to the companion articles associated with the other three components of the TQF – explains the chief elements that define Credibility, stating that “credible qualitative research is the result of effectively managing data collection, paying particular attention to the two specific areas of Scope and Data Gathering.” Although a great deal of the discussions thus far have been centered on traditional qualitative methods, the increasingly important role of technological solutions in qualitative research makes it imperative that the discussion of Credibility (and the other TQF components) expand to the digital world.

The online asynchronous focus group (“bulletin board”) method has been around for a long time. It is clearly an approach that offers qualitative researchers many advantages over the face-to-face mode while also presenting challenges to the integrity of research design. The following presents a snapshot of the online bulletin board focus group method through the lens of the two main ingredients of the TQF Credibility component – Scope and Data Gathering. This snapshot is not an attempt to name all the strengths and limitations associated with the Credibility of the online asynchronous focus group method but rather highlight a few key considerations.Credibility of Asynch Focus Group Method