Evaluating Proposals Using the Total Quality Framework

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 345-346)

In addition to using the Total Quality Framework (TQF) to structure more rigorous and comprehensive research proposals, the TQF can be used by anyone who is evaluating a proposal for a research study that will use qualitative methods (e.g., members of a thesis or dissertation committee, funders at a granting agency or foundation, clients in the commercial sector). A TQF approach to evaluating research proposals effectively holds the proposal author(s) accountable for doing research that is likely to be accurate and, in the end, useful. The TQF provides a comprehensive system to methodically think about the strengths and limitations of the proposed study design and helps the reviewer ascertain whether there are outstanding threats to the quality of the proposed research that have been ignored or remain unanticipated by the researcher(s).

In essence, the TQF is a reminder to proposal evaluators that research integrity built around fundamental principles is equally important in qualitative as it is in quantitative research design.

The TQF criteria to be considered in the proposal review, within each of the four TQF components, are the following:

Credibility

• How the target population has been defined.
• How the list representing the target population will be created.
• How the sample of participants will be chosen from the list(s) that will be used.
• How many participants the researcher proposes to gather data from or about and the justification that is provided for this number, including its adequacy for the purposes of the study; a discussion of how the researcher will monitor and judge the adequacy of this number while in the field should also be included.
• How the researcher will gain cooperation from, and access to, the sampled participants.
• How the researcher will determine if those in the sample from whom data was not gathered differ in critical ways on the topics being studied from those participants who did provide data.
• What the researcher will do to account for the potential bias that may exist because not everyone in the sample participated in the research (i.e., no data was gathered from some individuals).
• The extent to which the relevant concepts that will be studied have been identified.
• How the researcher has operationalized these concepts in order to effectively collect data on them in the research approach.
• How the researcher has articulated and supported the research objectives and questions.
• How the data collection method(s) will be pilot-tested and revised as necessary.
• The precautions that will be taken to minimize (or at least better understand) the potential biases and inconsistencies that might be created in the data by those involved in data collection.
• The precautions that will be taken to assure high ethical standards throughout the entire study.

Analyzability

• How the researcher will process the gathered data for analysis.
• How the data analyses will be carried out to make sense and interpret the data.
• The particular verification procedures that will be used to assess the reliability and validity of the findings from the analyses.

Transparency

• How the researcher will make the methods and details of the research transparent and accessible to all who seek such information about the study.

Usefulness

• The extent to which the researcher has identified the value of the proposed study and built a compelling case that the findings will advance the state of knowledge on the topic, provide actionable next steps, and/or enable the research to be transferred to other comparable contexts.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.