November 26, 2013
Eric Anderson and Duncan Simester published a paper in May 2013 titled “Deceptive Reviews: The Influential Tail.” It talks about their analysis of many thousands of reviews for a major apparel “private label retailer” with the focus on a comparison of reviews made by customers who actually made a prior transaction (i.e., customers who actually purchased the item they were reviewing) and customers who had not made a prior transaction (i.e., customers who reviewed items they had not actually purchased). Their comparisons largely revolved around four key measures or indicators that characterize deception in online reviews and messaging: 1) a greater number of words (compared to reviews from customers who had bought the item); 2) the use of simpler, shorter words; 3) the inappropriate reference to family (i.e., referring to a family event unrelated to the product being reviewed such as “I remember when my mother took me shopping for school clothes…”); and 4) the extraordinary use of exclamation points (i.e., “!!” or “!!!”). Apparently, deceivers tend to overcompensate for their lack of true knowledge and wax eloquent about something they know nothing about. This wouldn’t matter except that deceivers’ deceptive reviews (i.e., reviews from customers who have not purchased the item reviewed) are more likely to be Read Full Text
November 15, 2013
A focus group moderator’s guide will often include group exercises or facilitation techniques as alternative approaches to direct questioning. While many of these alternative tactics are not unique to the group discussion method, and are also used in in-depth interview research, they have become a popular device in focus groups, esp., in the marketing research field. These alternative approaches can be broadly categorized as either enabling or projective techniques, the difference being whether the moderator’s intent is to simply modify a direct question to make it easier for group participants to express their opinions (enabling techniques) or delve into participants’ less conscious, less rational, less socially-acceptable feelings by way of indirect exercises (projective techniques). Examples of enabling techniques are: sentence completion – e.g., “When I think of my favorite foods, I think of _____.” or “The best thing about the new city transit system is _____.”; word association – e.g., asking prospective college students, “What is the first word you think of when Read Full Text
October 30, 2013
Last week, Susan Eliot posted a terrific piece on listening (a common theme on her blog The Listening Resource) titled “Listening For Versus Collecting Data.” In it, she talks about the power imbalance – and, I would add, the insensitive mindset – implied by the idea that researchers are “collecting data from subjects” compared to the more useful notion that we are listening “one human to another.” Eliot goes on to cite Martin Buber and his distinction of I-Thou and I-It interactions or relationships between people, with Eliot stating “When we look upon the other person as a ‘thou’ (a unique, sentient human being) rather than an ‘it’ (a data repository), we approach the research with a humanistic perspective, one that is likely to net us rich and meaningful data.”
Extolling the virtues of listening seems almost trite (we all claim to “listen” in some shape or form) yet why is it so very difficult? It is difficult, not only among researchers where listening is (should be) a required skill but, among all of us where listening is a fundamental component of human interaction.
The October 18, 2013 NPR TED Radio Hour program “Haves and Have-Nots” presents two important examples on the importance of listening and, more particularly, the negative effects of not listening well. The first is a TED talk given by Ernesto Sirolli titled “Want to help someone? Shut up and listen!” where he tells the story of an ill-fated attempt to teach people in Zambia Read Full Text
October 15, 2013
Through history, research people have discussed and debated the virtues and fallibilities of quantitative versus qualitative research. “Versus” because there is typically a ‘one or the other’ mentality in thinking and talking about quantitative and qualitative research that may ultimately pit one against the other. This dichotomy makes obvious sense from the standpoint of the very different purposes and approaches prescribed by these two research genres, fostering as it often does two very different types of researchers with sometimes radically different mind and skill sets.
There are situations – we can all probably think of some – when a survey or focus group (or IDI or observation) research design is opted for simply because it is the type of research that falls within someone’s comfort zone. We go with what we know. This is true of researchers; it is also true of corporate clients and other research funders.
Many qualitative researchers, for instance, are loath to venture into survey territory where the stark realities of black and white numbers, percentages, and correlations are too confining as they are mind-blowing. And it is usually this qualitative-fear-of-quantitative that we hear so much about. But what about survey researchers and the clients who find a safe haven in quantitative methods? Do they share a similar dread of qualitative research and, if so, why? Read Full Text
September 21, 2013
Reliability, in the sense of being able to obtain identical findings from repeated executions of a qualitative research design, is debatable. Validity, however, is another matter. Validity, in the sense of whether the qualitative researcher is collecting the information (data) he or she claims to be gathering (i.e., the accuracy of the data), is a topic worthy of much more discussion in the research community, or at the least a greater emphasis in our qualitative research designs. While qualitative researchers may not be able to replicate their studies, they surely have the means to consider the authenticity of the data.
There was a Research Design Review post back in 2010 that discussed the importance and appropriateness of validity in qualitative research, including the idea that there are ready-made techniques for looking at validity in qualitative research and that, in some ways, validity is already built into our research methods. To illustrate how qualitative researchers typically incorporate validity Read Full Text
September 10, 2013
Approximately two years ago, a post in Research Design Review described a quality framework that is recommended as a guide to researchers in their qualitative research designs. This post – “Four Components of the Quality Framework for Qualitative Research Design” – talks about the benefits of grounding qualitative design in a framework by which the researcher can “judge the efficacy” as well as “examine the sources of variability and establish critical thinking in the process of qualitative research design.” The four components of the quality framework (QF) revolve around the idea that all qualitative research must be: credible, analyzable, transparent, and ultimately useful.
In the current post, qualitative researchers are encouraged to put the QF to work in a very important applied arena – i.e., the crafting and evaluating of research proposals. For instance, a QF approach to qualitative research deserves prominence in: (a) the proposals written by graduate students working towards their theses and dissertations; (b) proposals written by researchers in the academic, government, not-for-profit, and commercial sectors responding to clients’ requests for proposal (RFPs); and (c) proposals written for grants. Taking a quality perspective in the research proposal raises the bar on the Read Full Text
August 19, 2013
Multi-method research enables the qualitative researcher to study relatively complex entities or phenomena in a way that is holistic and retains meaning. The purpose is to tackle the research objective from all the methodological sides. Rather than pigeonholing the research into a series of IDIs, focus groups, or observations, the multi-method approach frees the researcher into total immersion with the subject matter. Multi-method strategies are particularly relevant in case-centered research such as case studies and narrative research. For instance, a case study concerning a state-wide drug prevention program might include IDIs with the program staff and volunteers, observations of program activities, group discussions with program participants, and a review of administrative documents. Similarly, a narrative study to explore the manner in which 8th grade science is taught in the city schools might be designed to include many methods in order to frame the narrative environment such as: in-class teacher observations, teachers’ lived stories Read Full Text
July 31, 2013
Researchers conduct qualitative research because they acknowledge the human condition and want to learn more, and think differently, about a research issue than what is usual from mostly numerical quantitative survey research data. Not surprisingly, the unique nature of qualitative inquiry is characterized by a distinctive set of attributes, all of which impact the design of qualitative research one way or the other. The 10 unique attributes of qualitative research are the:
- Absence of “truth” With all the emphasis in qualitative research on reality and the human condition, it might be expected that qualitative inquiry is in the business of garnering “the truth” from participants. Instead of “truth,” the qualitative researcher collects information from which some level of knowledge can be gained. The researcher does not acquire this information and knowledge in a vacuum but rather in a context and, in this way, the research data are a product of various situational factors. For this reason, qualitative researchers do not talk about the “truth” of their findings but rather the “plausibility” of their interpretations.
- Importance of context A relevant factor in the elusiveness of “truth” is the central and significant role context plays in qualitative research. Whether it be the physical environment or mode by which an in-depth interview (IDI), group discussion, or observation is conducted the outcomes in qualitative research hinge greatly on the contexts from which we obtain this data.
- Importance of meaning Although the goal of all research is to draw meaning from the data, qualitative research is unique in the dimensionality of this effort. Qualitative researchers derive meaning from the data by way of multiple sources, Read Full Text
July 15, 2013
The assertions of marketing researchers (in particular) who continue to promote speed and techno-whiz over design principles leaves the rest of us wondering if rigorous design considerations really matter and whether we need to “buckle our seat belts” as we race to an anything-goes research paradigm. Marketing researchers (in particular) have been in this race for quite some time. Even before the Internet and all the gadgetry, there has been an over-emphasis on finding the path of least resistance – a path absent of speed limits and tolls, delivering results as quickly and cheaply as possible. The Internet and gadgetry have just transformed this path into a popular, well-paved superhighway.
In recent articles, we learn that – costly and time-consuming – face-to-face focus groups are “on life support,” that “micro-surveys” are the future, and that feedback from “brand ambassadors” in the marketplace can fill in when management’s need-to-know can’t wait for the oh-so-slow process of real research. All of this is beginning to sound a lot like really bad qualitative research design where:
- Sample representativeness is of little concern.
- No thought is given to the transferability of the outcomes.
- The final deliverable is full of great – colorful, fun, creative – quotes and images.
- There are as few demands as possible on the participants, and even the researcher.
- An attempt to make meaningful connections based on how people think is nonexistent.
What if, instead of promoting the research superhighway, folks discussed with their buyers/users of research the design issues inherent in various approaches, the trade-offs involved, and how to construct the best-quality research design possible within the reality of cost and time parameters. The superhighway is great for advancing the technology that advances our quality of life, including our ability to enjoy new options in our research designs. But when the highway itself becomes our focus – and not the quality measures in design that we know translate into reliable research – it may be time to take the next exit, turn off the engine, and just chill.
June 30, 2013
Ethical considerations play an important role in the research we do. Of all researchers, however, the ethnographer may be the most likely to face difficult ethical considerations and decisions that directly impact study design. One reason is that covert observation is a fairly common design feature in ethnographic research and these researchers live with the secrecy of deception.
There are many well-documented covert ethnographic studies, some of which became highly controversial for their use of deceptive tactics. Carolyn Ellis (1986), for example, conducted a nine-year observation of a Guinea (traditional watermen) community in the tidewater region of Virginia whose townspeople befriended her unaware that the sole purpose of her visits was to further her research endeavor. She quickly became a “traitor” when her prize-winning book on the research went public.
Deviant and subculture groups have also been the target of covert ethnographies. Humphreys’ (1970) classic study on male homosexual bathroom trysts involved the researcher serving as a watchdog for quick sexual liaisons in public bathrooms between male strangers. The researcher obtained the names and addresses of these men by using public records to look up their automobile license plate numbers. One year later, he visited these men, pretended to be conducting survey research on mental health and, in so doing, conducted 50 interviews that Read Full Text