Total Quality Framework

Qualitative Research Participants: Gaining Access & Cooperation

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, p. 28).

Gaining cooperationWhen developing the sample design, including the sample size for a qualitative study, careful attention needs to be paid to how the researcher will gain access to individuals in the sample and then gain their cooperation to participate in the research.

In doing a company-sponsored in-depth interview study of employees, for example, gaining access to the employees who have been sampled may be as simple as sending each of them a notification that their employer has authorized the researcher to contact them to request their participation in the research study. Or it may be as challenging as gaining permission from “gatekeepers” who have the right to deny access to the individuals the researcher wants to study — e.g., parents of the children who will be studied, presidents of the professional organizations whose members will be studied, wardens of prisons whose inmates will be studied, etc. The challenge of gaining access from gatekeepers is essentially finding successful strategies that (a) provide guarantees to the gatekeepers that no harm will come to the participants, (b) communicate the worthiness of the research study, and (c) offer some benefit to the gatekeeper or the organization.

Once access to the sampled participants has been granted, the researcher must use strategies to gain cooperation from those who have been chosen. Ideally a very large portion of those who have been sampled will agree to participate. Gaining cooperation is important. This is because, from a Total Quality Framework standpoint, individuals who are chosen to be included in the study but do not participate (e.g., because they refused to cooperate) may differ in important ways from those who do participate, jeopardizing the integrity of the data  which can lower or even undermine the credibility of the qualitative study. If, for example, a disproportionately greater number of males, compared to females, who have been sampled from a list of college freshmen can never be contacted or refuse to participate, and if these sampled males would have provided data that are materially different from the data provided by the other freshmen on the list who did participate in the study, then the research findings will be biased because of the data missing from a major subgroup of the population.

To avoid these problems, qualitative researchers need to utilize strategies meant to overcome the reason(s) that causes some people who are sampled to not cooperate and fail to participate. Such strategies include:

  • Building rapport early with the participants, thereby gaining their trust.
  • Assuring the participants of complete confidentiality.
  • Explaining the non-material benefits to be gained by participating (e.g., helping to raise the quality of life in the neighborhood).
  • Explaining the material benefits, if any, to be gained by participating (e.g., the offer of an Amazon gift card).

Whichever strategies the researchers choose to deploy, ideally they will be tailored (at the individual level) to appeal to the particular types of participants in the sample in order to overcome reluctance or unequivocal refusal during the recruiting process.

A TQF Approach to Construct Validity

TQF approach to construct validity

Construct validity plays an important role in the design, implementation, analysis, and ultimate usefulness of qualitative research methods. The construct of validity itself in qualitative research is discussed in this article and cites qualitative researchers across disciplines who explore “unique dimensions” and other considerations  relating to validity in qualitative research.

The Total Quality Framework (TQF) relies heavily on construct validity in its quality approach to each phase of the qualitative research process. At each phase, the researcher must ask “Am I gaining real knowledge about the core concepts that are the focus of this research?” For example,

  • An important step when developing a research design is to identify the key constructs associated with the research objectives to investigate, and the particular attributes of each construct that the researcher wants to explore. So, for example, a researcher conducting a study on dietary behavior may have interest in “health consciousness,” including shopping behavior related to organic and fresh foods.
  • In the in-depth interview and focus group discussion methods, careful attention needs to be paid to guide development and the inclusion of questions relevant to the constructs of interest. When developing the guide, the researcher needs to ask “Is this [topic, question, technique] relevant to the construct we are investigating?”, and “Does this [topic, question, technique] provide us with knowledge about the aspect of the construct that we intended to explore in the interviews/discussions?”
  • In ethnography, the observation guide and observation grid are important tools. “The grid is similar to the guide in that it helps to remind the observer of the events and issues of most import; however, the observation grid is a spreadsheet or log of sorts that enables the observer to actually record and reflect on observable events in relationship to the research constructs of interest” (Roller & Lavrakas, 2015, p. 206).
  • The quality of qualitative data analysis hinges on the researcher’s ability to effectively identify, analyze, and develop valid interpretations of the data around the important constructs associated with the research objectives. To assist the researcher, a TQF approach to analysis recommends a codebook format and coding form (which is basically a reflexive journal for the coder[s] to record thoughts and justifications for their coding decisions) that highlights constructs of interest. For example,

TQF codebook and coding form

  • Construct validity also plays an important role in the transparency of the final research document. In the study report, the researcher can (and should) elaborate on the design, data gathering, and analysis decisions that were made pertaining to the key constructs, as well as the main themes that were derived from the data — i.e., the knowledge that was gained from the research — concerning these constructs.

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach. New York: Guilford Press.

 

Photo by Bon Vivant

A TQF Approach to Sample Size

Credibility TQF componentSample size and sampling in qualitative research design have been discussed elsewhere in Research Design Review, see “Sample Size in Qualitative Research & the Risk of Relying on Saturation” and “Shared Constructs in Research Design: Part 1 — Sampling.” In June 2022, “A TQF Approach to Choosing a Sample Design” was posted to RDR and considers ways to ensure that research participants are representative (share defining characteristics) of the population being studied.

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 26-27) that briefly examines a Total Quality Framework (TQF) approach to another facet of sample design, i.e., sample size.

 


How large a sample to use is a decision that qualitative researchers need to make explicitly and carefully in order to increase the likelihood that their studies will generate credible data by well representing their population of interest. Unlike quantitative researchers who most often rely on statistical formulae to determine the sample sizes for their studies, qualitative researchers must rely on (a) past experience and knowledge of the subject matter; and (b) ongoing monitoring during the data-gathering period, which includes applying a set of decision rules, such as those listed in “Designing a Quality In-depth Interview Study: How Many Interviews Are Enough?” These decision rules consider (a) the complexity of the phenomena being studied, (b) the heterogeneity or homogeneity of the population being studied, (c) the level of analysis and interpretation that will be carried out, and (d) the finite resources available to support the study. These types of decision guidelines, along with past experience, should provide qualitative researchers with the considerations they need to carefully judge the amount of data necessary to meet their research objectives. (Of note, if a researcher does not have sufficient past personal experience, a literature review, or speaking directly with other researchers who do have such experience, should serve well.)

As importantly, during the period when data are being gathered, researchers should also closely monitor the amount of variability in the data, compared to the variability that was expected, for the key measures of the study. Based on this monitoring, researchers are responsible for making a “Goldilocks decision” about whether the sample size they originally decided was needed is too large, too small, or just about right. In making a decision to cut back on the amount of data to be gathered, because there is less variability in what is being measured than anticipated, the researcher needs to make certain that those cases that originally were sampled, but would be dropped, are not systematically different from the cases from which data will be gathered. In making a decision to increase the size of the sample, because there is more variability in what is being measured than anticipated, the researcher needs to make certain that the cases added to the sample are chosen in a way that is representative of the entire population (e.g., using the same orderly approach that was used to create the initial sample).

In all instances, and if the necessary resources (staff, time, budget) are available, it is prudent for a researcher to error on the side of having more rather than less data. Gathering too much data does no harm to the quality of the study’s findings and interpretations, but having too little data leaves the researcher in the untenable position of harming the quality of the study because the complexity of what was being studied will not be adequately represented in the available data. For example, case study research to investigate new public school policies related to the core science curriculum might include in-depth interviews with school principals and science teachers, observations of science classes in session, and a review of students’ test papers; however, as a complex subject matter, the research may be weakened by not including discussions with the students and their parents as well as by a failure to include all schools (or a representative sample of schools) in the research design.

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach. New York: Guilford Press.