question design

Pigeonholing Qualitative Data: Why Qualitative Responses Cannot Be Quantified

A recent webinar on the ins-and-outs of qualitative research stated that qualitative data could be quantified by simply counting the codes associated with some aspect of the data content, such as the number of times a particular brand name is mentioned or a specific sentiment is expressed towards a Pigeonholetopic of interest.  The presenter asserted that, by counting these codes, the researcher has in effect “converted” qualitative to quantitative data.

This way of thinking is not unlike those who contend that useful quantitative data can be calculated with qualitative findings by counting the number of “votes” for a particular concept or some aspect of the research subject matter.  Let’s say a moderator asks group participants to rate a new product idea on a modest four-point scale from “like very much” to “do not like at all.”  Or, an interviewer conducting qualitative in-depth interviews (IDIs) asks each of the 30 participants to rate their agreement with statements pertaining to the advantages of digital technology on a scale from “strongly agree” to “strongly disagree.”  It is the responses to these types of questions that some researchers gather up as votes and report as quantitative evidence.

By asserting that codes and votes can be counted and hence transform a portion of qualitative findings Read Full Text

Life Is Meaningful, Or Is It?: The Road To Meaning In Survey Data

Samantha Heintzelman and Laura King, at the University of Missouri, published an article in American Psychologist in 2014 titled, “Life is Pretty Meaningful.” In this article the authors meaningful walkdiscuss their work that explores the answer to the “lofty” question “How meaningful is life, in general?” To do this, Heintzelman and King examined two broad categories of data sources: 1) large-scale surveys – six representative surveys conducted in the U.S. and a worldwide poll; and 2) articles published in the literature that explicitly report on research studies utilizing one of two established measures of meaning in life – the Purpose in Life Test (PIL) and Meaning in Life Questionnaire (MLQ). The large-scale surveys asked yes-and-no questions such as “Did you feel that your life has meaning [in the past 12 months]?” as well as agree-disagree rating scale items such as “My life has a real purpose.” Their analysis of these surveys concluded that “for most people, life is meaningful [and] comparatively few Read Full Text

Feelings & Sensations: Where Survey Designs Fail Badly

Survey research is pretty good at allowing people to describe “things” in such a way that the researcher winds up with a fairly accurate idea of the thing being described. The most straight-forward example is a survey question that asks, “Which of the following features came with Hotel experienceyour new Toyota Corolla?” followed by a list of possible features. However, survey research can also get at descriptions of more experiential phenomena with questions such as, “On a scale from ‘1’ to ‘5’, how does each of the following statements describe your experience in buying a new home?” In these cases, the use of survey methods to research a great number of people, and compile and report the data as efficiently as possible, make good use of closed-ended questions to gain an understanding of respondents’ accounts of the “things” of interest. This can also be said of beliefs. Pew’s recent survey pertaining to the Christmas story that asked, Read Full Text