December 16, 2014
Survey research is pretty good at allowing people to describe “things” in such a way that the researcher winds up with a fairly accurate idea of the thing being described. The most straight-forward example is a survey question that asks, “Which of the following features came with your new Toyota Corolla?” followed by a list of possible features. However, survey research can also get at descriptions of more experiential phenomena with questions such as, “On a scale from ‘1’ to ‘5’, how does each of the following statements describe your experience in buying a new home?” In these cases, the use of survey methods to research a great number of people, and compile and report the data as efficiently as possible, make good use of closed-ended questions to gain an understanding of respondents’ accounts of the “things” of interest. This can also be said of beliefs. Pew’s recent survey pertaining to the Christmas story that asked, Read Full Text
The fourth edition of Michael Quinn Patton’s book Qualitative Research & Evaluation Methods has just been published by Sage. It is a big book – over 800 pages – with updated and new content from earlier editions, including something he calls “ruminations” which are highlighted sections in each chapter that present Patton’s commentary and reflections on issues that have “persistently engaged, sometimes annoyed” him throughout his long career in qualitative research. Patton has made some of these ruminations available online via his posts on the betterevaluation.org blog.
In his November 14th post, Patton shares his “Rumination #2: Confusing empathy with bias.” In it, he raises an important issue – having to do with the personal nature of qualitative research and how that impacts data collection – that, on some level, runs through the qualitative-quantitative debates waged by researchers who argue for one form of research over another. Such a debate might involve a survey researcher who, entrenched in statistical analysis, wonders, Read Full Text
November 22, 2014
Back in April 2013, a post in RDR talked about the “daunting job of conducting a content analysis that reveals how people think [the “stream of consciousness”] while at the same time answers the research question and takes the sponsoring client to the next step.” The article outlines the basic steps in a content analysis, including the analysis and interpretation phases of the process. Making interpretations from a content analysis are tricky things, esp., when conducting a “primary content analysis” when the content being analyzed is derived from non-research-related, pre-existing sources such as newspapers, blog posts, Hollywood films, YouTube videos, television broadcasts, and the like. The issue here is the “trap” content analysts can fall into by (a) thinking there are causal relationships in the data when there are not, and/or (b) trying to build a story in the shape of their interpretations when the story (based on the data) has little merit. In this way, an overabundance of subjectivity can creep into the qualitative content analysis method.
These traps, related to causality and storytelling, are fairly easy to fall into unless a systematic and conscientious approach is taken in the analysis and interpretation phases. In particular, Read Full Text
October 27, 2014
Researchers know that “good” survey questionnaire design begins with a preliminary qualitative research phase that serves to expose the nuances of the research topic or category – such as the most pertinent issues and the relevant concerns or “issues within the issues” – along with the manner by which the target population talks about these issues – that is, the particular words, expressions, and terminology used by the target group. In this way, the survey researcher can hope to create user-friendly survey questions that speak to respondents rather than at respondents.
A preliminary qualitative phase is good and necessary, but employing the talents of a qualitative researcher during survey question development is an equally-important step. Qualitative researchers spend much of their lives listening to people talk about a host of attitudinal and behavioral issues, listening to the use of language, and using these conversations to interpret Read Full Text
October 13, 2014
Qualitative research is not any one thing. It is clearly not any one method but it is also not any one technique or process. Much of the diversity in how and in what manner qualitative research is utilized can be attributed to the researcher’s particular discipline or field of study. This is because each area of study brings with it its own set of priorities and concerns that mandate a particular qualitative approach. Importantly, this provides an opportunity for all qualitative researchers to extend their reach to learn from other researchers both within and outside their own disciplines. By broadening their boundaries and world view of what constitutes qualitative research, researchers can make better – more informed – choices in the development and implementation of their research designs.
Here are just a few examples of how the qualitative-research focus can vary across Read Full Text
September 30, 2014
With a lot of discussion about new methods of observation among qualitative researchers – in-the-moment mobile research and the like – it is terrific to witness an increasing appreciation of broader contexts. This perspective embraces the idea that individual behavior and thought are not so easily and singularly confined to any one moment in time. One could argue that it is because of this new-found obsession with observation that many researchers have come to discover – as if for the first time – the essential role that context plays in our qualitative studies. In this way, observational research – a method often bypassed for focus groups and other qualitative methods in the past – has led the research community into what is becoming a growing and healthy dialogue concerning the contextual nature of being human. Here are just four contributors to the dialogue that have recently come my attention:
An interview with Christian Madsbjerg at ReD Associates appears in the September issue of Marketing News – “What it Means to be Human” by Elisabeth A. Sullivan. In it, Madsbjerg asserts that “people are different from the way that we research them,” emphasizing the point that “the respondent is not a person” but rather “an ecology of people, a culture of people” Read Full Text
September 8, 2014
For most of us, it is important to write a final research report that goes beyond the questions we asked and the responses we received. Unlike a topline debriefing that may require a simple rundown of the questions and responses, our qualitative and quantitative studies typically culminate in write-ups that provide thoughtful discussions of our analyses and interpretations of the data.
The consumers of our research reports take it on blind faith that the data along with the corresponding questions and issues are reported accurately, and that the researchers’ interpretations of the findings are consistent with both the data and the questions asked or issues raised. And yet blind faith is not always enough. Those are the times when a closer look at what the research actually asked and what is actually reported is needed.
August 14, 2014
On August 5th Vision Critical held a webinar titled, “How NASCAR Increases Fan Engagement and Drives Business Decisions” where Brian Moyer discussed the NASCAR Fan Council, its online community of more than 10,000 NASCAR fans. In his presentation, he talked about the camaraderie feel of the community and the efforts they take to create an atmosphere in which fans believe they are “talking directly to NASCAR.” So strong is this community atmosphere that fans actually challenge whether Moyer and his team are hearing what fans have to say and wonder if NASCAR is internalizing their comments for a greater good. One fan asked, “How do I know you are listening?”
What a great question. To NASCAR’s credit, they make a habit of providing one-on-one feedback to their community members and did so in response to the are-you-listening question. But how well do researchers of any kind pay attention to this all-important facet of research design? Where in our research designs Read Full Text
Conducting qualitative research by way of a mobile device presents the researcher with unique challenges in terms of how to design a mobile study that results in valid outcomes. There are, however, a number of quality measures that researchers can build into their qualitative mobile studies that will serve to elevate their research designs and bring added confidence to the final results. The following are just a few quality considerations that qualitative researchers should think about and incorporate throughout the mobile research process. This list simply highlights a few design aspects related to mobile research and in no way supersedes the additional quality features (discussed throughout this blog) that should be part of any qualitative research design.
These design aspects are discussed from the perspective of the Total Quality Framework* which is comprised of four components – Credibility, Analyzability, Transparency, and Usefulness. In essence, the framework is based on the idea that all qualitative research Read Full Text
If I Conduct a Large Qualitative Study with 100 Participants, is it Quantitative Research? Three Big Reasons Why the Answer is “No!”
July 10, 2014
Too often qualitative researchers present their findings with an assertion along the lines of, ‘We conducted 25 focus groups with a total of 250 participants making this study more quantitative than qualitative’; or ‘We conducted 10 online bulletin boards with 15 participants in each divided between males and females, so we wound up with good quantitative data’; or ‘We planned on conducting 30 qualitative in-depth interviews (IDIs) but extended the research to include 100 interviews so that we can quantify the results.’ Unfortunately, comments like these reflect a misguided attempt to equate apples with oranges – lumping them both into the category of “fruit” although their essence – the properties that characterize them – are radically different.
Conducting a lot of qualitative research does not transform it into a quantitative study. To say otherwise, assumes that the only distinguishing factor between a qualitative and quantitative research design is the number of participants or respondents who contribute to the research outcomes. This way of thinking would deem Read Full Text