The Total Quality Framework (TQF) (Roller & Lavrakas, 2015) offers researchers a way to think about qualitative research design from the vantage point of core principles. It is an approach that helps qualitative researchers develop critical thinking skills by giving explicit attention to the quality of the conceptualization and implementation of their qualitative studies. The TQF is composed of four components, each pertaining to a phase of the research process – data collection (Credibility), analysis (Analyzability), reporting (Transparency), and the ability to do something of value with the outcomes (Usefulness).

Qualitative research is most often conducted as a standalone study but frequently conducted in conjunction with quantitative methods. A mixed methods research (MMR) design involves collecting both qualitative and quantitative data, then integrating or connecting the two datasets to draw interpretations derived from the combined strengths of both sets of data (Creswell, 2015). The integration of, or making the connection between, the qualitative and quantitative components is fundamental to MMR and distinguishes it from a multi-method approach that simply utilizes different methods.  In contrast, a mixed methods design incorporates any number of qualitative and quantitative methods (and modes) with the specific intention of blending the data in some fashion. Mixed methods research is the subject of an earlier article in Research Design Review.

So, how do we apply the TQF to a MMR design? It is not good enough to simply think of the qualitative component of MMR as a separate feature to the overall design and apply a TQF approach to the qualitative method(s). For MMR, the TQF needs to be adapted to accommodate a qualitative-quantitative connection as discussed earlier. There are many ways to do this. A few practical applications of the TQF in MMR are outlined below.

Credibility (Data Collection)

A necessary and highly practical consideration in the course of collecting in-depth interview data is the question of the number of interviews to complete. To address this question, the TQF presents 10 related questions* for the researcher to contemplate when in the field, such as

• Did all interviewees provide clear, unambiguous answers to key questions or issues, or does the researcher need to go back to some interviewees for clarification?
• Can the researcher identify the sources for variation and contradictions within the data?
• Do the data confirm or deny what is already known about the subject matter?

The kinds of questions the researcher might contemplate in a MMR design are similar but are now tweaked to connect qualitative data gathering with the quantitative component. In each case, the researcher is expanding his/her thinking to consider the implications associated with the collecting of qualitative data as well as that associated with the quantitative. The researcher conducting a MMR study might now consider,

• Did all interviewees provide clear, unambiguous answers to key questions or issues; if not, does the researcher need to go back to the participant(s) or leave clarification for the quantitative component?
• Can the researcher identify the sources for variation and contradictions within the qualitative data as well as between the qualitative and quantitative data?
• Do the data confirm or deny what is known from the quantitative data?

Analyzability (Analysis)

The TQF offers numerous ways to approach the processing and verification of qualitative data. One of the suggested verification strategies has to do with reflexivity and, specifically, the reflexive journal. The reflexive journal gives researchers the opportunity to respond to questions intended to foster introspection along with an understanding of the researcher’s effect on the qualitative data. These reflections further the researcher’s ability to verify the interpretations of qualitative data during the analysis process. In a standalone qualitative study, the researcher’s reflexive journal might include the contemplation of such questions as*

• What do I think I “know” from this/these participant(s) and how do I think I “know” it?
• What assumptions did I make (what did I assume to be true) about the participant(s)?
• How did my personal values, beliefs, life story, and/or social/economic status affect or shape the questions I asked, the interjections I made, my listening skills, and/or behavior?

If the researcher was conducting MMR, the reflexive journal would address similar questions but now in the context of the broader MMR scheme. To connect the qualitative component with the quantitative, the reflexive journal asks the researcher to think about

• What do I think I “know” from this/these participant(s) and how has that been influenced by what I may know from the quantitative data?
• What assumptions did I make (what did I assume to be true) about the participant(s) based on what I may know about respondents to the quantitative survey?
• How did my understanding of the quantitative data affect or shape the questions I asked, the interjections I made, my listening skills, and/or behavior?

Transparency (Reporting)

The Transparency component of the TQF has to do with reporting the outcomes in the final document; specifically, reporting a “thick description” of study details (NOTE: For earlier RDR articles on thick description, see this April 2017 article and this 2015 article). By conveying the details of the data collection and analysis processes, the researcher allows the users of the research (e.g., other researchers, the sponsor) to examine the researcher’s work and draw their own conclusions as well as transfer the design to other contexts. There are many details about the study that the researcher may want to address in the final document*, including the

• Adequacy (i.e., comprehensiveness) of the lists that were used to represent the target population.
• Failure to interview all interviewees sampled, efforts that were made to avoid this, and possible biases or weakness this may have caused.
• Field notes (e.g., note-taking procedures, examples from the field notebook).

In MMR, the qualitative researcher needs to pay attention to connecting the qualitative component with the quantitative portion of the study. To do this in the reporting phase, the researcher interjects the thick description with details relevant to both the qualitative and the quantitative research. For example, the details might include the

• Compatibility of the lists with that used in the quantitative phase.
• Failure to interview comparable types of people, efforts that were made to avoid this, and possible biases or weakness this may have caused.
• Field notes (e.g., examples when qualitative data converged/diverged with quantitative data).

Usefulness (Doing something of value with the outcomes)

Ultimately, the objective of our research efforts is to derive outcomes that respond to the research question and provide outcomes that serve a valuable purpose. In many instances, a MMR approach fulfills this goal more so than a standalone qualitative or quantitative study by expanding and enriching the researcher’s understand beyond the “borders” of a mono-method study. The Journal of Mixed Methods Research and other resources are filled with examples of ways MMR has contributed to important societal issues:

Health

• Cultural nuances among dementia caregivers, e.g., social stigma of dementia (Weitzman & Levkoff, 2000)

Education

• Procrastination & motivation among students with learning disabilities (Klassen et al., 2008)

Conservation

• Conservation adoption decision process among farmers, e.g., importance of communication, rapport, & incentives (Nyanga, 2012)

Psychology

• Meaning-making underlying bereaved mothers’ adaptive and complicated grief responses to the death of a child from cancer (Gerrish, et al., 2014)

Food Safety

• Gap between knowledge & behavior (Meysenburg et al., 2014).

When adapting a quality approach to the qualitative component of MMR, it is not sufficient to simply treat the qualitative portion as an independent element in the overall MMR design. Indeed, it is critical and fundamental to the MMR approach to make a connection between the qualitative and quantitative facets of the study. The few practical examples discussed in this article illustrate how qualitative researchers can make these connections while, at the same time, maintaining the integrity of the unique epistemology underpinning qualitative inquiry.

*See Roller & Lavrakas (2015) for a complete list of questions / thick description details.

Creswell, J. W. (2015). A concise introduction to mixed methods research. Thousand Oaks, CA: Sage.

Gerrish, N. J., Neimeyer, R. A., & Bailey, S. (2014). Exploring maternal grief: A mixed-methods investigation of mothers’ responses to the death of a child from cancer. Journal of Constructivist Psychology, 27(3), 151–173.

Klassen, R. M., Krawchuk, L. L., Lynch, S. L., & Rajani, S. (2008). Procrastination and motivation of undergraduates with learning disabilities: A mixed-methods inquiry. Learning Disabilities Research & Practice, 23(3), 137–147.

Meysenburg, R., Albrecht, J. A., Litchfield, R., & Ritter-Gooder, P. K. (2014). Food safety knowledge, practices and beliefs of primary food preparers in families with young children: A mixed methods study. Appetite, 73, 121–131.

Nyanga, P. H. (2012). Factors influencing adoption and area under conservation agriculture: A mixed methods approach. Sustainable Agriculture Research, 1(2), 27–40.

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach. New York: Guilford Press.

Weitzman, P. F., & Levkoff, S. E. (2000). Combining qualitative and quantitative methods in health research with minority elders: Lessons from a study of dementia caregiving. Field Methods, 12(3), 195–208.

Image captured from: https://blog.wiziq.com/tag/connecting-online/