The Total Quality Framework (TQF) has been discussed many times in Research Design Review. Typically, the discussions center around: applying a quality approach to different aspects of qualitative research design such as sample design or a quality approach to different research methods such as qualitative content analysis, or an explanation of one of the four components of the TQF — Credibility, Analyzability, Transparency, Usefulness — or the TQF approach to the qualitative research proposal. Although there have been a few articles on the limitations of qualitative methods and modes from a TQF perspective such as “Limitations of the Focus Group Method: An Overview,” there has not been an explicit discussion of utilizing the TQF to examine the limitations of a particular research study.

Luke David Bevan, a recent doctoral graduate of University College London, wrote a compelling thesis on “Through a Model, Darkly: An Investigation of Modellers’ Conceptualisation of Uncertainty in Climate and Energy Systems Modelling and an Application to Epidemiology.” This study set out “to understand how [computer modellers in climate science and energy systems] conceptualise the uncertainty of their work.” To this end, Dr. Bevan conducted 38 semi-structured in-depth interviews with modellers. Included in this nearly 500-page doctoral thesis is a discussion on the limitations of the research study. This discussion is organized around the TQF and its four components. It does this by highlighting potential shortcomings of the research design associated with the sample and data gathering (the TQF Credibility component), data processing and verification (the TQF Analyzability component), Transparency, and the ultimate Usefulness of the research outcomes.

The following are a few brief examples of the limitations identified in Dr. Bevan’s thesis:

As this case example shows, the TQF is an effective and efficient facilitator for thinking critically about qualitative research design. In particular, this case reminds researchers that critical thinking skills are essential throughout the research process, i.e., before and during the conduct of the research as well as when reflecting back on the completed study.