Research Design Review has discussed the idea of transparency on several occasions. Last month’s post, titled “Designing Qualitative Research to Produce Outcomes You Can Use,” briefly mentioned the contribution transparency makes to the ultimate usefulness of a qualitative research study emphasizing that full disclosure of the study’s details “empowers the reader of the research to make his or her own judgments as to the integrity of the research (Is it good research?) as well as its usefulness in furthering new ideas, next steps, and new applications.” The goal of transparency is to provide an audit trail in the final research document that allows the reader to duplicate the research (if that were possible), derive similar conclusions from the data as presented, or apply the research in other contexts. Transparency is important.
Transparency in the final document goes way beyond a simple account of the number and time frame when interviews, groups, or observations were conducted and a rundown of participants’ characteristics. In order for clients and other users of the research to ascertain the reliability, validity, and transferability of the outcomes, the researcher’s final deliverables need to include details concerning the:
- Researcher’s justification and assumptions prior to the fieldwork concerning the sample population, data collection techniques, and expected outcomes;
- Sampling, esp. the determination of the appropriate number of events (interviews, groups, observations) to conduct, the sampling frame and process of participant selection, and the efforts that were made to select a representative sample of the target population, including possible biases or weaknesses in the data due to the lack of representation;
- Decisions that were made while the research was in the field that modified the original research objectives or design elements (e.g., reasons for switching from face-to-face to online mode because of unexpected costs and time delays) and how these decisions may have impacted outcomes;
- Researcher’s reflexive journal (a diary of in-the-field feelings, hunches, insights), including a critical account of his/her attitudes and behavior during the research event that may have biased the outcomes;
- Transcription and coding processes; and,
- Steps that were taken to verify the outcomes, such as detailed accounts of peer debriefings, triangulation efforts (e.g., inter-interviewer reliability, “member checking”), and analysis of negative or deviant cases.
Without transparency in our qualitative research designs, how are the buyers and users of our research to know what they are getting? How are they to know if what is being shown as the outcomes is actually worthy of attention, actually true to the research objectives, to the people participating in the research, and to the researcher’s conclusions and recommended next steps? By casting a light into the inner workings of our research – from conceptualization to completion – we allow others to see how the pieces of the design connect with each other, including the dips and turns the research took to eventually produce a functioning final result. This is transparency.