It is a common misperception among researchers that the analysis of research data is a process that is confined to the data itself. This is probably truer among qualitative researchers than survey researchers given that the latter frequently publish their work in the literature comparing and contrasting their data with relevant earlier studies. Qualitative research, on the other hand, is typically held up to less scrutiny; and, except for the usual comparisons of populations segments, it is rare to find an analytical discussion that goes beyond the patterns and themes derived from the qualitative data itself. This may be for any number of reasons. It may be associated with the idea that qualitative research by definition is chock full of uncontrollable variables that vary from study to study making data comparisons across studies unreliable, or it may be researchers’ unfamiliarity Read Full Text
A challenge to any research design is the pesky issue of bias or the potential distortion of research outcomes due to unintended influences from the researcher as well as research participants. This is a particularly critical issue in qualitative research where interviewers (and moderators) take extraordinary efforts to establish strong relationships with their interviewees (and group participants) in order to delve deeply into the subject matter. The importance of considering the implications from undo prejudices in qualitative research was discussed in the April 2011 Research Design Review post, “Visual Cues & Bias in Qualitative Research,” which emphasized that “there is clearly much more effort that needs to be made on this issue.” Reflexivity and, specifically, the reflexive journal is one such effort that addresses the distortions or preconceptions researchers unwittingly introduce in their qualitative designs.
Reflexivity is an important concept because it is directed at the greatest underlying threat to the validity of our qualitative research outcomes – that is, the social interaction component of the interviewer-interviewee relationship, or, what Steinar Kvale called, “the asymmetrical power relations of the research interviewer and the interviewed subject” (see “Dialogue as Oppression and Interview Research,” 2002). The act of reflection Read Full Text
The October 2012 issue of the American Psychological Association’s Monitor on Psychology includes an interview with developmental psychologist, Jerome Kagan. In this interview he talks about psychology’s research “ghosts,” referring to the dubious generalizations psychologist’s make from their often-limited research. Kagan’s primary point is that “it’s absolutely necessary to gather more than one source of data, no matter what you’re studying,” and that these multiple sources of data should come from verbal and behavioral as well as physiological measures. Only by combining these various perspectives on an issue or situation – that is, utilizing data taken in different contexts and by way of alternative methods and modes – can the researcher come to a legitimate conclusion.
This is not unlike triangulation, esp., in the social and health sciences, which is used to gauge the trustworthiness of research outcomes. Triangulation is the technique of examining a specific research topic by comparing data obtained from: two or more methods, two or more segments of the sample population, and/or two or more investigators. In this way, the researcher is looking for patterns of convergence and divergence in the data. Triangulation is a particularly important design feature in qualitative research – where measures of validity and transferability can be elusive – because it furthers the researcher’s ability to gain a comprehensive view of the research question and come closer to a plausible interpretation of final results.
Scholars teach the importance of including some form of triangulation in research designs yet there is not a lot of evidence that this occurs in the real world of applied qualitative research. While there are an increasing number of ways to gather qualitative feedback – particularly via social media and mobile – that provide researchers with convenient sources of data, applied researchers would benefit from more discussion on case studies that have utilized multiple data sources and methods to find reliable themes in the outcomes. Importantly, it is further hoped that applied researchers use this contrast-and-compare approach to scrutinize the research issue from both traditional (e.g., in-person group discussions, in-depth interviews, in-home ethnography) and newer (e.g., online based, mobile device) information-gathering strategies.
The triangulation concept is just one way that researchers can add rigor to their research designs and manage the potential “ghosts” of groundless assumptions and misguided interpretations.