Pigeonholing Qualitative Data: Why Qualitative Responses Cannot Be Quantified

A recent webinar on the ins-and-outs of qualitative research stated that qualitative data could be quantified by simply counting the codes associated with some aspect of the data content, such as the number of times a particular brand name is mentioned or a specific sentiment is expressed towards a Pigeonholetopic of interest.  The presenter asserted that, by counting these codes, the researcher has in effect “converted” qualitative to quantitative data.

This way of thinking is not unlike those who contend that useful quantitative data can be calculated with qualitative findings by counting the number of “votes” for a particular concept or some aspect of the research subject matter.  Let’s say a moderator asks group participants to rate a new product idea on a modest four-point scale from “like very much” to “do not like at all.”  Or, an interviewer conducting qualitative in-depth interviews (IDIs) asks each of the 30 participants to rate their agreement with statements pertaining to the advantages of digital technology on a scale from “strongly agree” to “strongly disagree.”  It is the responses to these types of questions that some researchers gather up as votes and report as quantitative evidence.

By asserting that codes and votes can be counted and hence transform a portion of qualitative findings into quantitative data, these researchers are making the case, knowingly or not, that these codes and votes are discrete items.  But, of course, they are not.

Unlike the structured environment of survey research, qualitative data is the product of a host of variables that influence outcomes in any number of ways.   When a survey respondent picks a brand name from a list or rates a concept on a given scale, he/she is responding to a specifically-worded question that is: being asked of all respondents in exactly the same way, typically positioned in the same or similar context in relationship to the other survey questions, and not preceded by researcher-respondent conversations concerning the topic.  Qualitative methods, on the other hand, do not abide by these standards.  By definition, qualitative research embraces flexible question-and-answer environments where the researcher (interviewer, moderator) is never quite sure what byways the discussion will take as it journeys to the final destination of the research objective.  It is the multi-faceted context of this environment that steers the course to some degree.

As a result, there is no telling what influences impinge on a participant’s responses in an IDI or focus group.  Did the discussion leading up to the question familiarize the participant with otherwise unknown information about the topic at hand?  In what way did the interviewer/moderator modify how questions were asked based on the participant’s responses to earlier questions?  How did the research environment – e.g., the highly talkative “dominator” in a focus group discussion – alter a participant’s attitude or willingness to answer honestly?

In qualitative research, context is everything.  By paying attention to context, qualitative researchers are able to identify meaningful connections and draw useful – more profound – interpretations about “what makes people tick” that go beyond survey data.  But context also limits how qualitative data can be used.  Just as context precludes a qualitative researcher from generalizing qualitative outcomes, so too context prevents the researcher from treating the data as discrete, independent responses to be counted and thereby hoping to pigeonhole qualitative data as something it is not.

 

Image captured from: http://www.susan-ingram.com/2016/04/divorce-mediation-and-the-pigeonhole-effect/

4 comments

  1. The case made does completely apply to data garnered from focus group discussion. Does it hold true to semi-structured individual interviews where every answers the way they please? Even in the Likert scales of strongly agree to strongly disagree have at times nuances that cannot be captured discretely. A part from this blog, any other empirical sources of why we cannot quantitize qualitative data and qualitize quantitative data?
    Thanks for this important topic!

  2. This is the kind of near sighted response that happens when teachers try to take holistic rubrics and assign a grade to each level or use tests to limit kids’ reading choices. We have become a society so focused on numbers as a means of making meaning that we’ve forgotten what’s important in in qualitative research and why context and relationships defy being quantified.

    1. Thanks Elisa for your intervention. I hold similar views like you. I do not know why we attempt to quantify everything before we agree that it is true meaningful knowledge. As you have rightly pointed out the import of qualitative research is not the same as quantitive research. Instead of attempting to make the two designs the same let us celebrate what each bring on board the world of knowledge generation.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s