September 10, 2013
Approximately two years ago, a post in Research Design Review described a quality framework that is recommended as a guide to researchers in their qualitative research designs. This post – “Four Components of the Quality Framework for Qualitative Research Design” – talks about the benefits of grounding qualitative design in a framework by which the researcher can “judge the efficacy” as well as “examine the sources of variability and establish critical thinking in the process of qualitative research design.” The four components of the quality framework (QF) revolve around the idea that all qualitative research must be: credible, analyzable, transparent, and ultimately useful.
In the current post, qualitative researchers are encouraged to put the QF to work in a very important applied arena – i.e., the crafting and evaluating of research proposals. For instance, a QF approach to qualitative research deserves prominence in: (a) the proposals written by graduate students working towards their theses and dissertations; (b) proposals written by researchers in the academic, government, not-for-profit, and commercial sectors responding to clients’ requests for proposal (RFPs); and (c) proposals written for grants. Taking a quality perspective in the research proposal raises the bar on the Read Full Text
August 19, 2013
Multi-method research enables the qualitative researcher to study relatively complex entities or phenomena in a way that is holistic and retains meaning. The purpose is to tackle the research objective from all the methodological sides. Rather than pigeonholing the research into a series of IDIs, focus groups, or observations, the multi-method approach frees the researcher into total immersion with the subject matter. Multi-method strategies are particularly relevant in case-centered research such as case studies and narrative research. For instance, a case study concerning a state-wide drug prevention program might include IDIs with the program staff and volunteers, observations of program activities, group discussions with program participants, and a review of administrative documents. Similarly, a narrative study to explore the manner in which 8th grade science is taught in the city schools might be designed to include many methods in order to frame the narrative environment such as: in-class teacher observations, teachers’ lived stories Read Full Text
July 31, 2013
Researchers conduct qualitative research because they acknowledge the human condition and want to learn more, and think differently, about a research issue than what is usual from mostly numerical quantitative survey research data. Not surprisingly, the unique nature of qualitative inquiry is characterized by a distinctive set of attributes, all of which impact the design of qualitative research one way or the other. The 10 unique attributes of qualitative research are the:
- Absence of “truth” With all the emphasis in qualitative research on reality and the human condition, it might be expected that qualitative inquiry is in the business of garnering “the truth” from participants. Instead of “truth,” the qualitative researcher collects information from which some level of knowledge can be gained. The researcher does not acquire this information and knowledge in a vacuum but rather in a context and, in this way, the research data are a product of various situational factors. For this reason, qualitative researchers do not talk about the “truth” of their findings but rather the “plausibility” of their interpretations.
- Importance of context A relevant factor in the elusiveness of “truth” is the central and significant role context plays in qualitative research. Whether it be the physical environment or mode by which an in-depth interview (IDI), group discussion, or observation is conducted the outcomes in qualitative research hinge greatly on the contexts from which we obtain this data.
- Importance of meaning Although the goal of all research is to draw meaning from the data, qualitative research is unique in the dimensionality of this effort. Qualitative researchers derive meaning from the data by way of multiple sources, Read Full Text
July 15, 2013
The assertions of marketing researchers (in particular) who continue to promote speed and techno-whiz over design principles leaves the rest of us wondering if rigorous design considerations really matter and whether we need to “buckle our seat belts” as we race to an anything-goes research paradigm. Marketing researchers (in particular) have been in this race for quite some time. Even before the Internet and all the gadgetry, there has been an over-emphasis on finding the path of least resistance – a path absent of speed limits and tolls, delivering results as quickly and cheaply as possible. The Internet and gadgetry have just transformed this path into a popular, well-paved superhighway.
In recent articles, we learn that – costly and time-consuming – face-to-face focus groups are “on life support,” that “micro-surveys” are the future, and that feedback from “brand ambassadors” in the marketplace can fill in when management’s need-to-know can’t wait for the oh-so-slow process of real research. All of this is beginning to sound a lot like really bad qualitative research design where:
- Sample representativeness is of little concern.
- No thought is given to the transferability of the outcomes.
- The final deliverable is full of great – colorful, fun, creative – quotes and images.
- There are as few demands as possible on the participants, and even the researcher.
- An attempt to make meaningful connections based on how people think is nonexistent.
What if, instead of promoting the research superhighway, folks discussed with their buyers/users of research the design issues inherent in various approaches, the trade-offs involved, and how to construct the best-quality research design possible within the reality of cost and time parameters. The superhighway is great for advancing the technology that advances our quality of life, including our ability to enjoy new options in our research designs. But when the highway itself becomes our focus – and not the quality measures in design that we know translate into reliable research – it may be time to take the next exit, turn off the engine, and just chill.
June 30, 2013
Ethical considerations play an important role in the research we do. Of all researchers, however, the ethnographer may be the most likely to face difficult ethical considerations and decisions that directly impact study design. One reason is that covert observation is a fairly common design feature in ethnographic research and these researchers live with the secrecy of deception.
There are many well-documented covert ethnographic studies, some of which became highly controversial for their use of deceptive tactics. Carolyn Ellis (1986), for example, conducted a nine-year observation of a Guinea (traditional watermen) community in the tidewater region of Virginia whose townspeople befriended her unaware that the sole purpose of her visits was to further her research endeavor. She quickly became a “traitor” when her prize-winning book on the research went public.
Deviant and subculture groups have also been the target of covert ethnographies. Humphreys’ (1970) classic study on male homosexual bathroom trysts involved the researcher serving as a watchdog for quick sexual liaisons in public bathrooms between male strangers. The researcher obtained the names and addresses of these men by using public records to look up their automobile license plate numbers. One year later, he visited these men, pretended to be conducting survey research on mental health and, in so doing, conducted 50 interviews that Read Full Text
June 16, 2013
Storytelling is the ultimate goal of all research. In the end, researchers of all kinds are in the business of understanding how people think, and what better way than to hear their stories. Storytelling may sound like something only qualitative researchers should care about but survey researchers, knowingly or not, are equally concerned about the stories people have to tell. The recent brouhaha over Gallup’s failure to correctly predict the winner of the 2012 presidential election is a case in point. One of the fundamental weaknesses that contributed to the Gallup polls favoring a Romney win is how Gallup went about determining likely voters, including respondents’ past voting behavior and how much attention they were paying to the election. Like all pollsters, Gallup simply used the responses to these and other questions to calculate which respondents were most likely to vote in the national election. One of the problems that Gallup ran into, however, is that “many” of the Obama voters claimed not to be paying much attention to the election which, of course, disqualified them as likely voters. In essence, Gallup simply wanted to know each Read Full Text
May 31, 2013
Nonresponse and non-response error is more than a quantitative issue. While qualitative researchers may shudder at the thought, the typically-ignored impact of nonresponse is just as important in the qualitative realm. Why is nonresponse in qualitative research important? Because we are conducting qualitative research. Not qualitative let’s get a few warm bodies around the table for our face-to-face focus group, but actually research methods that, like all research, demand certain protocols that address potential biasing effects. One of these is nonresponse. The warm bodies in our group discussion may make the moderator and client observers feel great – Thank goodness, someone showed up! – but the uncomfortable reality is that the people who chose not to participate – or were never contacted by a recruiter and asked to participate in the first place – greatly affect our research outcomes. Indeed, the trajectory of a group discussion Read Full Text
May 18, 2013
If you are one of those researchers who work in both quantitative and qualitative design, something you are reminded of fairly quickly at the AAPOR annual conference – currently being held in Boston – is that there is really little separating the two genres. Survey researchers may deem the ‘hard facts’ of their quantitative data as a gold standard of sorts; and qualitative researchers may look questioningly at the righteousness of these ‘hard facts’, asking “Where’s the beef?” that explains the “why” behind the data, but there is no debate that we are all after the same thing. The following are just a few of the common areas of interest among quantitative and qualitative researchers:
- Question administration – What to ask & how to ask it
- Interviewer effect – Impact of interviewer’s behavior, appearance, & attitude on response
- Mode – Which mode for which population segment & its impact on response
- Cooperation – How to increase participation & decrease respondent/participant burden
- Analysis – How to organize data & develop coding schemes that accurately represent the data
- Cost – How to “do more” with smaller research budgets
And, interestingly, researchers of all stripes are addressing similar Read Full Text
April 30, 2013
An article posted on Research Design Review back in 2010 discussed the work of William James and, specifically, his concept that consciousness “flows” like a river or stream. The article goes on to say that James’ “stream of consciousness” is relevant to researchers of every stripe because we all share in the goal of designing research “to understand the subjective links within each individual.” Yet these subjective links come at a price, not the least of which is the “messiness” of the analysis as we work towards identifying these links and finding meaning that addresses our objectives.
Whether it is the verbatim comments from survey respondents to open-end questions or the transcripts from focus group discussions or ethnographic interviews, the researcher is faced with the daunting job of conducting a content analysis that reveals how people think while at the same time answers the research Read Full Text
“Keep it simple,” “keep it short,” and “make it fast.” These are the words that many qualitative researchers live by as they sit down to produce the final written report for their clients. The prevailing sense among some, particularly in the marketing research field, is that their all-too-busy clients don’t have the time, inclination, or research backgrounds to read lengthy reports detailing nuanced findings and method. Instead, clients want a brief summary of outcomes that are actionable in the short term. It is no wonder that PowerPoint reporting has become so popular. Who needs complete sentences when a key implication from the research can be reduced to a bullet list or an alluring infographic?
But what has become lost in the ever-increasingly-shrinking report is the discussion of research design. Where once at least cursory attention would be given to the basic design elements – this is what we did, this is when we did it, this is where we did it, and these are the demographics of the participants – in the first few pages of the report, this all-important information has been pushed to the back, sometimes to the appendix where it sits like frivolous or unwanted content begging to be ignored. Not only should the research design not be sequestered to the badlands of reporting but the discussion of research design in qualitative research should be expanded and enriched with details of the:
- qualitative method that was used (along with the rationale for using that method),
- target population,
- sample selection and composition of the participants,
- basis by which the interviewer’s/moderator’s guide was developed,
- reason that particular field sites and not others were chosen for the research,
- interviewer’s/moderator’s techniques for eliciting participants’ responses,
- measures that were taken to maximize the credibility and analyzability of the data, and
- coding and other analysis procedures that were used to arrive at the reported interpretations and implications from the outcomes.
The inclusion and elaboration of the research design in qualitative reports matters. It matters because qualitative research has a life, and it is only the researcher’s thick description of the paths and byways the research traveled that allows the life of qualitative research to thrive Read Full Text