There are lots of articles discussing question design, focusing on such things as how to mitigate various forms of bias, clearly communicate the intended meaning of the question, and facilitate response.  Survey question wording is discussed in this “tip sheet” from Harvard University as well as in “Questionnaire Design” from Pew Research Center, and an article in Research Design Review discusses the not-so-simple “why” question in qualitative research (see “Re-considering the Question of ‘Why’ in Qualitative Research”).

Getting the question “right” is a concern of all researchers, but qualitative researchers need to be particularly mindful of the responses they get in return. It is not good enough to use an interview guide to ask a question, get an answer, and move on to the next question. And, it is often not good enough to ask a question, get an answer, interject one or two probing questions, and move on to the next question. Indeed, one of the toughest skills a qualitative interviewer must learn is how to evaluate a participant’s answer to any given question. This goes way beyond evaluating whether the participant responded in line with the intention of the question or the potential sources of bias. Rather, this broader, much-needed evaluation of a response requires a reflexive, introspective consideration on the part of the interviewer.

Reflexivity is central to a qualitative approach in research methods. It is a topic that is discussed often in RDR – see “Interviewer Bias & Reflexivity in Qualitative Research,” “Reflections from the Field: Questions to Stimulate Reflexivity Among Qualitative Researchers,” and “Facilitating Reflexivity in Observational Research: The Observation Guide & Grid” – because of its role in qualitative research design. There are many wonderful papers and studies on reflexivity. A few examples can be found in the August 2017 issue of Qualitative Psychology which is devoted to reflexivity and includes such thoughtful and insightful articles as Shari Goldstein’s “Reflexivity in Narrative Research.”

Most accounts on reflexivity focus on the reflexive journal and, specifically, the researcher’s recording of observations related to the participant and the research environment as well as the researcher’s assumptions and beliefs that may have affected the outcomes. These after-the-fact considerations are essential to the integrity of the research. However, equally essential is the reflexive exercise that researcher’s practice in situ, i.e., during an in-depth interview (IDI) or group discussion. This in-the-moment reflection, while in the research environment with the participant(s), is the time when the researcher must think carefully about what is being said, the extent to which the researcher understands what is being said, and the degree to which this understanding actually mirrors the participant(s) true intent.

Here are a few of the questions the researcher might contemplate throughout an IDI or group discussion:

• Can I explain, in my own words, what was said?
• Can I explain, in my own words, the meaning of what was said as it relates to the research question?
• How much of what I think I understand stems from the participant(s) rather than something I heard from other study participants?
• How much of what I think I understand stems from the participant’s meaning rather than my subjective assumptions, beliefs, or personal experiences?
  • What are the words or phrases that I may be misinterpreting because I am contaminating them with my own assumptions, beliefs, or personal experiences?
• Have my emotional reactions to the participant’s responses affected (biased) my understanding?
• Can I conclude the research event confident of what I learned from this/these participant(s) or do I need to prolong the event to ask clarifying questions?

It is this kind of in-the-moment reflexive exercise that ensures the integrity and the ultimate usefulness of the qualitative data.

Image captured from: http://futureofcio.blogspot.com/2015/02/reflection-in-design-thinking.html