Designing Qualitative Research to Produce Outcomes You Can Use
November 29, 2012
A November 2011 post in Research Design Review briefly discussed the “Four Components of the Quality Framework for Qualitative Research Design” – Credibility, Transparency, Usefulness, and Analyzability. Of the four, Usefulness is clearly the most important for two obvious reasons. First, all research – qualitative and quantitative and across all modes – is designed to be used. Our research efforts are meaningless if they don’t produce outcomes that actually advance the researcher and the research end-user towards a desired goal and spurs action (to actually do something). Whether it is the discovery of new concepts to explore further, a roadmap for next steps, or insights on how to apply the research to similar contexts, any research design worth its weight is utilitarian by nature.
Second, usefulness in qualitative research is important because it is central to the other three framework components. Research that is designed to give useful – implementable – results is also one that integrates design features that maximize: the completeness and accuracy of the data (Credibility), the completeness and accuracy of the analysis (Analyzability), as well as the completeness and fullness of disclosure in the final research documents (Transparency). By exploiting Credibility, Analyzability, and Transparency in our qualitative designs, we have fulfilled our promise of providing research that motivates new thinking and lays the groundwork for steps forward.
The relationship between Usefulness and the other components of qualitative design can be considered in association with the three broad phases of research – fieldwork, analysis, and reporting.
Credibility is concerned with the fieldwork or data collection phase when the completeness and accuracy of the interview responses, focus group discussions, or participant observations rely on both the scope of the qualitative research (representativeness of the sample and sample size) as well as the trustworthiness of the measurement (e.g., Were the research questions actually relevant to the objectives? Did the interviewer or moderator verify that the participants understood the questions as intended? What was the role of the researcher and how did the interviewer, moderator, or observer bias the responses?).
The Analyzability of a qualitative research design has to do with the completeness or comprehensiveness of the analysis procedures (e.g., the verification of outcomes via peer debriefings, triangulation, and a review of deviant cases) and the accuracy of data processing, specifically the transcriptions and coding of text responses.
Transparency – or providing in the final document, what some call, “thick description” of every detail related to the data collection and analysis – is critical to the reporting phase of qualitative research design because it empowers the reader of the research to make his or her own judgments as to the integrity of the research (Is it good research?) as well as its usefulness in furthering new ideas, next steps, and new applications.
Our qualitative research should strive to be nothing if not useful. And our research is not useful unless the scope and measurement of our data is credible, our analysis is comprehensive and accurate, and our final research report offers a transparent view of what we hoped to do in the research, how we did it, why we did it that way, and how verification and textual processing procedures shaped our final analysis. In this way, our qualitative research designs ultimately produce the most important outcome of all – information we can actually do something with.