Too often qualitative researchers present their findings with an assertion along the lines of, ‘We conducted 25 focus groups with a total of 250 participants making this study more quantitative than qualitative’; or ‘We conducted 10 online bulletin boards with 15 participants in each divided between males and females, so we wound up with good quantitative data’; or ‘We planned on conducting 30 qualitative in-depth interviews (IDIs) but extended the research to include 100 interviews so that we can quantify the results.’ Unfortunately, comments like these reflect a misguided attempt to equate apples with oranges – lumping them both into the category of “fruit” although their essence – the properties that characterize them – are radically different.
Conducting a lot of qualitative research does not transform it into a quantitative study. To say otherwise assumes that the only distinguishing factor between a qualitative and quantitative research design is the number of participants or respondents who contribute to the research outcomes. This way of thinking would deem a study conducted with less than 30 individuals as qualitative while something more than that – and certainly more than 100 – as quantitative. Oh, if the workings of research were so simple. Research, like apples and oranges, may all be “fruit” but the essence of design maintains the individuality of each method.
Underlying the 10 unique attributes to qualitative research, there are three big reasons why a qualitative study of any size or shape will never – or should never – be confused with anything remotely looking like quantitative research.
Big Reason #1: By its very nature, qualitative research thrives on the use of unstructured or semi-structured question formats. Unlike survey questions which are highly structured requiring explicit interviewer training so that questions are asked precisely as written, qualitative questioning is typically more relaxed and, though following a topic outline, the researcher will most likely word questions in varying ways as well as introduce new topics as they emerge during the course of the study. It is this flexible nature of qualitative research that allows for the in-depth, rich input that serves to clarify and contextualize quantitative data. Allowing for new content brings us to Big Reason #2…
Big Reason #2: The content and therefore the context of a qualitative event (e.g., focus group discussion or IDI) will vary from event to event. This is because research participants invariably introduce new ideas or thoughts that the qualitative researcher explores. The introduction of new, not-previously-discussed content creates a unique context within each qualitative event which ipso facto serves to shape participants’ comments in a discussion or interview to some degree. Along with varying content and contexts, there is a host of other factors that act as variables in qualitative research, which brings us to Big Reason #3…
Big Reason #3: The aggregation of a whole bunch of qualitative research events can never be interpreted as quantitative data because there are simply too many variables at play within any one event. While quantitative research design incorporates certain measures as an attempt to control for an even playing field in the execution stage, the qualitative environment is replete with variables that counter any effort to create a controlled context. Here are just three of the major variables affecting face-to-face (in-person) qualitative research:
- Venue – In face-to-face research the venue from one focus group discussion or IDI to another continually changes as the moderator/interviewer moves from one research facility or interviewing site to another. Each site has its own aura – emitting from the size of the room, the lighting, the décor, or hospitality of the facility staff – that can impact participants’ comfort level and hence their engagement with the research. Whether or not client viewers are present – as well as the number of clients viewing – is another contributing variable to the venue impacting the research experience.
- Moderator/interviewer – Even if the same moderator or interviewer conducts all discussions or IDIs, the researcher’s particular mood (affecting what and how questions/issues are raised) or style of dress will modify outcomes in some way.
- Show rate – The dynamics – and therefore research findings – will vary dramatically in group discussions (face-to-face or otherwise) depending on: 1) who decides to show up and 2) how many show up. The group composition (i.e., who shows up) in terms of demographics as well as personality types is a key variable that directly affects results. And clearly a discussion with 10 participants will produce a different dynamic as well as quantity and quality of outcomes compared to a discussion with six individuals.
It is curious why any researcher would need to equate their large qualitative study to a quantitative effort. By its very nature, qualitative research design is not intended to be nor does it aspire to become a newfangled version of quantitative. It is not the mere sample size that separates qualitative from quantitative but rather the multifaceted essence of their designs.
Image captured from: http://cobornsdelivers.wordpress.com/2010/02/25/apple-and-oranges-don%E2%80%99t-mix/
Interesting article. I agree with everything you say, though the article is predicated on quantitative versus in-person qualitative techniques. I would argue that asynchronous (I.e. not live) qualitative techniques such as that collected through online platforms where questions are served up identically to each and every respondent could be analyzed from a quantitative perspective. There is no moderator, no facility, no other peers to influence responses. Thoughts?
LikeLike
Hello Dean,
Thank you for the comment.
Yes, you are right, my “big reasons” are directed at the face-to-face mode. But asynchronous online groups and IDIs also present the researcher with a host of variables. Yes, specific questions are programmed into the platform but, other than that, similar factors exist in both online and face-to-face. You mentioned one of them, i.e., having to do with peers. When I conduct online bulletin boards I make a strong effort to engage with the participants and to foster group interaction (just as I would in a face-to-face group); so, indeed, the level and type of group interaction that goes on between participants in online boards is not unlike that in the face-to-face mode.
And, in both groups and IDIs, I am only programming in the main questions which still leaves me with lots of probing to do — and the level and type of probing questions I interject will vary from online event to online event depending on participants’ input; which, in turn, may change the flavor of the discussion/IDI.
Finally, keep in mind that most (all?) online platforms enable participants to share images, documents, etc. which, again, will vary with each group of participants or IDI interviewee and impact the outcomes.
So, I take your comment. One would think that the absence of a facility and programmed questions might give the researcher permission to think quantitatively but, alas, online qualitative is still qualitative. Thanks again for the comment. Much appreciated.
LikeLike