November 22, 2015
Is all qualitative research of equal value? Are the findings derived from one focus group study just as useful as those obtained from another focus group study? Are the outcomes from observational research or in-depth interviews (IDIs) valuable regardless of the design peculiarities (i.e., how the research was conducted)?
More specifically, what are the strengths and limitations of the design elements that inform the usefulness of research outcomes? Was the research objective and approach well-conceived, realistic? What was the sampling method? How was recruitment conducted? What procedures were in place to maximize cooperation and rapport, and minimize nonresponse? Was the moderator/observer/interviewer guide carefully thought out and designed to achieve the research objective (e.g., using a funnel approach to develop a moderator’s outline)? Is it clear how the researcher conducted the Read Full Text
October 31, 2015
An important lesson in research design is the idea of learning from past research in order to not repeat the “mistakes” from comparable research in a given area. In qualitative research, if recruiting participants via email has reaped mediocre levels of response and cooperation in the past, a different recruiting strategy (e.g., personal letters by way of FedEx followed by phone) would be adopted for future studies with this population segment. And, if a particular moderating technique has not resulted in a dynamic and open focus group discussion on a certain topic, the researcher will dig deeper next time into the proverbial “toolbox” to find a more effective approach.
To facilitate the design process, while keeping in mind what has “worked” and “not worked” in the past, it is useful to create some type of grid or display of earlier research. This grid might include the researcher’s own work in the particular area of interest as well as that of others’ research published in peer-reviewed journals. For each study cited, the researcher’s display should include information pertaining to effective as well as ineffective elements of data collection. [NOTE: Similar grids could be developed relating to analysis and reporting.] For instance, a display looking at sampling and recruitment for face-to-face focus group research with cancer patients or survivors might look something like this: [NOTE: Click on image to enlarge]
By expanding the display and allowing it to guide the design process, the qualitative researcher can efficiently develop qualitative studies that build on past successes and result in useful outcomes.
Brown, R. F., Shuk, E., Leighl, N., Butow, P., Ostroff, J., Edgerson, S., & Tattersall, M. (2011). Enhancing decision making about participation in cancer clinical trials: Development of a question prompt list. Supportive Care in Cancer, 19(8), 1227–1238.
Ferrell, B. R., Grant, M. M., Funk, B., Otis-Green, S., & Garcia, N. (1997). Quality of life in breast cancer survivors as identified by focus groups. Psycho-Oncology, 6(1), 13–23.
Frazier, L. M., Miller, V. A., Horbelt, D. V., Delmore, J. E., Miller, B. E., & Paschal, A. M. (2010). Comparison of focus groups on cancer and employment conducted face to face or by telephone. Qualitative Health Research, 20(5), 617–627.
October 13, 2015
Research Design Review is a blog devoted to qualitative and quantitative research design issues. Yet, there is an imbalance in these discussions with many of the posts dedicated to qualitative design and methods. The reason boils down to the fact that there is simply a lot to say about qualitative design. And this is because relatively little is written or discussed in the research community in answer to such questions as, “What is the basis of sound qualitative research design?” “What are the necessary components to a ‘quality’ qualitative design?” and “How does the researcher effectively put into practice these quality design elements?” These are the questions routinely addressed among dedicated survey researchers yet too often absent in the qualitative orbit.
An underlying current running throughout RDR is the idea that quality design issues are important to all research, regardless of whether the researcher leans more to the qualitative or to the quantitative side of the equation. Pushing this idea one step further, there is an even more subtle suggestion lingering in RDR that researchers might do well to free themselves from their qualitative or quantitative “hats” and instead take on the mantle of Read Full Text
September 23, 2015
September 10, 2015
The Qualitative Methods in Psychology section (QMiP) of the British Psychological Society held its annual conference in Cambridge, England last week. It was a conference packed with varied and insightful presentations, workshops, and symposia covering such topics as: using conversation analysis to understand online communication, pluralism in qualitative research, visual methods such as photo-elicitation interviewing, the emotional demands associated with conducting phenomenological research, and discourse analysis of the media coverage of the conflict in Gaza.
In many instances the presenter’s focus was on the outcomes, e.g., what was learned after conducting a certain number of interviews, with little attention to the research design. This attention on the outcomes was to the exclusion of the path by which the outcomes were derived, i.e., the research process. Exploring the lived experiences among victims of brain injuries, or the life stories of women who have experienced Read Full Text
August 15, 2015
The cadre of modes available to researchers as they design their studies has grown hugely over the past decade. When researchers once had few choices – relying on face-to-face, landline phone, and mail – they now need to think carefully as they sift through an increasing number of options. In addition to the old standbys, other viable, and often preferable, modes must be considered, including mobile phone, online (without webcam use), and online (with webcam use).
- “Natural” characteristics, i.e., its ability to foster a natural, social conversation environment.
- The ability to share content, e.g., photos, video, documents.
- Rapport building, i.e., its ability to foster researcher-participant rapport.
- The ability to identify cues – verbal and non-verbal – that provide insights beyond direct responses.
- Coverage, i.e., the breadth and depth of geography and the population segment the mode can reach.
- Cost, i.e., the total cost of the study attributable to the mode.
There are, of course, other considerations – such as, convenience, depth of response, and so on – but the six listed are certainly important.
Using these considerations, it can be helpful to visualize the relative value Read Full Text
Researchers are desperate to understand behavior. Health researchers want to know what leads to a lifetime of smoking and how the daily smoking routine affects the quality of life. Education researchers examine the behavior of model teaching environments and contemplate best practices. Psychologists look for signs of social exclusion among victims of brain injuries. Marketing researchers chase an elusive explanation for consumer behavior, wanting to know product and service preferences in every conceivable category. And, if that were not enough, researchers of all ilk, to a lesser or greater extent, grapple with an often ill-fated attempt to predict (and shape) behaviors to come.
But researchers have come to appreciate that behavior is not enough. It is not enough to simply ask about past behavior, observe current behavior, or capture in-the-moment experiences via mobile. Behavior only tells part of a person’s story and, so, researchers passionately beef-up their research designs to include “why” – focusing on not just what people do but why they do it. “Why,” of course, is often phrased as “what,” “how,” or “when” questions – “What was going on at the time you picked up your first cigarette?” – but, whatever the format, the goal is the same, i.e., to get Read Full Text
Janette Brocklesby recently wrote an article in QRCA Views magazine concerning the conduct of qualitative research with the Māori population of New Zealand. Specifically, she addresses the issue of whether “non- Māori researchers have the cultural competency, expertise and skills to undertake research with Māori.” Brocklesby makes the case in the affirmative, emphasizing that non- Māori qualitative researchers are “well equipped to undertake research with Māori and to convey the Māori perspective.”
In making her case, Brocklesby discusses many of the best practices mentioned repeatedly in Research Design Review. As for all qualitative research, a researcher studying Māori groups must place a high importance on:
Reflexivity – Continually questioning and contemplating the researcher’s role or impact on research outcomes is a critical step towards quality results. In March 2014, an article in RDR talked about using a reflexive journal to think about the assumptions, values, and beliefs Read Full Text
June 28, 2015
In-depth interviewers and focus group moderators typically work from an outline of relevant topics and questions that guides them through the interview or discussion. The guide is intended to be just that, a guide, and not a strict, prescriptive document. With the guide, the ultimate goal is to enable the interviewer or moderator to efficiently incorporate all of the issues that are important to achieving the research objectives. Maintaining clarity throughout the interview or discussion on the related issues is actually a more essential purpose of the guide than the actual questions or follow-up probes it may contain.
The most typical and effective approach in constructing an interview or discussion guide is to begin broadly and progressively narrow the topic area to the subject matter of greatest importance to the research objectives, i.e., a “funnel” approach. The funnel consists Read Full Text
June 15, 2015
The impact of bias (in various forms) on research outcomes is well-documented. In Research Design Review alone, there are many articles related to this issue; bias in the world of both quantitative – such as “Ask Someone a Question, You’ll Get an Answer” and “Accounting for Social Desirability Bias in Online Research” – as well as purely qualitative – “Selection Bias & Mobile Qualitative Research” and “Visual Cues & Bias in Qualitative Research” – research. One of the more significant sources of bias in qualitative research is the researcher, i.e., the in-depth interviewer, focus group moderator, or observer in ethnography. This bias is specifically addressed in the RDR article “Interviewer Bias & Reflexivity in Qualitative Research,” which highlights the importance of the reflexive journal to help address “the bias that most assuredly permeates the socially-dependent nature of qualitative research.”
An interviewer may bias research outcomes in any number of ways. For instance, he or she may allow personal beliefs or expectations to skew how questions are asked and/or Read Full Text