The cadre of modes available to researchers as they design their studies has grown hugely over the past decade. When researchers once had few choices – relying on face-to-face, landline phone, and mail – they now need to think carefully as they sift through an increasing number of options. In addition to the old standbys, other viable, and often preferable, modes must be considered, including mobile phone, online (without webcam use), and online (with webcam use).

Relative value of modes

  • Natural” characteristics, i.e., its ability to foster a natural, social conversation environment.
  • The ability to share content, e.g., photos, video, documents.
  • Rapport building, i.e., its ability to foster researcher-participant rapport.
  • The ability to identify cues – verbal and non-verbal – that provide insights beyond direct responses.
  • Coverage, i.e., the breadth and depth of geography and the population segment the mode can reach.
  • Cost, i.e., the total cost of the study attributable to the mode.

There are, of course, other considerations – such as, convenience, depth of response, and so on – but the six listed are certainly important.

Using these considerations, it can be helpful to visualize the relative value Read Full Text

Researchers are desperate to understand behavior. Health researchers want to know what leads to a lifetime of smoking and how the daily smoking routine affects the quality of life. Education researchers examine the behavior of model teaching environments and contemplate best practices. Psychologists look for signs of social exclusion among victims of kid-waterslidebrain injuries. Marketing researchers chase an elusive explanation for consumer behavior, wanting to know product and service preferences in every conceivable category. And, if that were not enough, researchers of all ilk, to a lesser or greater extent, grapple with an often ill-fated attempt to predict (and shape) behaviors to come.

But researchers have come to appreciate that behavior is not enough. It is not enough to simply ask about past behavior, observe current behavior, or capture in-the-moment experiences via mobile. Behavior only tells part of a person’s story and, so, researchers passionately beef-up their research designs to include “why” – focusing on not just what people do but why they do it. “Why,” of course, is often phrased as “what,” “how,” or “when” questions – “What was going on at the time you picked up your first cigarette?” – but, whatever the format, the goal is the same, i.e., to get Read Full Text

Janette Brocklesby recently wrote an article in QRCA Views magazine concerning the conduct of qualitative research with the Māori population of New Zealand. Specifically, she addresses the issue of whether “non- Māori researchers have the cultural competency, expertise and skills to undertake research with Māori.” Brocklesby korumakes the case in the affirmative, emphasizing that non- Māori qualitative researchers are “well equipped to undertake research with Māori and to convey the Māori perspective.”

In making her case, Brocklesby discusses many of the best practices mentioned repeatedly in Research Design Review. As for all qualitative research, a researcher studying Māori groups must place a high importance on:

Reflexivity – Continually questioning and contemplating the researcher’s role or impact on research outcomes is a critical step towards quality results. In March 2014, an article in RDR talked about using a reflexive journal to think about the assumptions, values, and beliefs Read Full Text

In-depth interviewers and focus group moderators typically work from an outline of relevant topics and questions that guides them through the interview or discussion. The Guide developmentguide is intended to be just that, a guide, and not a strict, prescriptive document. With the guide, the ultimate goal is to enable the interviewer or moderator to efficiently incorporate all of the issues that are important to achieving the research objectives. Maintaining clarity throughout the interview or discussion on the related issues is actually a more essential purpose of the guide than the actual questions or follow-up probes it may contain.

The most typical and effective approach in constructing an interview or discussion guide is to begin broadly and progressively narrow the topic area to the subject matter of greatest importance to the research objectives, i.e., a “funnel” approach. The funnel consists Read Full Text

The impact of bias (in various forms) on research outcomes is well-documented. In Research Design Review alone, there are many articles related to this issue; bias in the world of both quantitative – such as “Ask Someone a Question, You’ll Get an Answer” and Dose“Accounting for Social Desirability Bias in Online Research” – as well as purely qualitative – “Selection Bias & Mobile Qualitative Research” and “Visual Cues & Bias in Qualitative Research” – research.   One of the more significant sources of bias in qualitative research is the researcher, i.e., the in-depth interviewer, focus group moderator, or observer in ethnography. This bias is specifically addressed in the RDR article “Interviewer Bias & Reflexivity in Qualitative Research,” which highlights the importance of the reflexive journal to help address “the bias that most assuredly permeates the socially-dependent nature of qualitative research.”

An interviewer may bias research outcomes in any number of ways. For instance, he or she may allow personal beliefs or expectations to skew how questions are asked and/or Read Full Text

There are certain types of qualitative research studies that employ more than one qualitative research method to explore a particular topic or phenomenon, i.e., the researcher uses multiple Multiplesmethods. These studies generally fall into the category of case study or narrative research, which are both designated by the label of “case-centered research.” The attributes that differentiate these forms of research from other qualitative approaches were discussed in an earlier Research Design Review post (“Multi-method & Case-centered Research: When the Whole is Greater Than the Sum of its Parts”). These differentiating attributes are largely associated with the use of multiple methods to gain a complete understanding of complex Read Full Text

Last week at the AAPOR 70th Annual Conference in Florida, Paul Lavrakas and I taught a “short course” on qualitative research design. The bulk of the class was spent on applying the unique constructs and techniques associated with the Total Quality Framework (TQF) to five qualitative research methods – in-depth interviews, focus group ducks-in-a-row-threediscussions, ethnography, qualitative content analysis, and case-centered research (i.e., case study and narrative research). But before jumping into the application of the TQF, we began by talking about the distinctive attributes of qualitative research, particularly the emphasis on context and interconnectedness that is inherent in qualitative data. Indeed, we stressed the complexity – the “messiness” – of qualitative data collection and analysis, along with the unparalleled researcher skills (such as flexibility) needed to perform high-quality and ultimately useful qualitative research.

This course was one of a handful of discussions pertaining to qualitative research at a conference that is heavily weighted toward survey methods. As both a Read Full Text

In “‘I Wonder About God’ & Other Poorly-Designed Questions” (Research Design Review, July 25, 2012), it is argued that weak survey question design has a “potentially negative impact on Sunflower lightanalysis, which in turn leads to wrong conclusions, which in turn leads end users along a path of misguided next steps.” As one of several examples, this article highlights the ambiguity embedded in SurveyMonkey’s “The God Survey”; specifically, the problematic first question that asks how often “I wonder about God.” Poorly-designed questions raise serious concerns about how or if the researcher can legitimately analyze the resulting data (while also tackling issues of reliability and validity), a concern made more profound by the frequent failure to even consider the alternative interpretations respondents may give to survey questions. By failing to Read Full Text

The analysis of qualitative research data is no small thing. Because the very nature of qualitative research is complicated by the complexities inherent injigsaw being human, attempting to qualitatively measure and then make sense of behavior and attitudes is daunting. In fact, it is this overwhelming aspect of qualitative research that may lead researchers – who live in the real world of time and budget constraints – to succumb to a less-than-rigorous analytical process.

And yet, Analyzability* is a critical component in qualitative research design.

All of the data collection in the world – all the group discussions, IDIs, observations, storytelling, or in-the-moment research – amounts to a meaningless exercise unless and until a methodical processing and verification of the data is conducted. Without the thoughtful work required to achieve a quality research product, qualitative data simply sits as an inert compilation of discrete elements lacking import.

Finding the connections in the qualitative data that make sense of the phenomenon, concept, or construct under investigation may, for some, be difficult and worthy of shortcuts; but proper analysis is the only thing that separates an honest, professional qualitative study from a random amalgamation of conversations or online snapshots.

In April of last year, this blog discussed one facet of Analyzability, i.e., verification. Verification, however, only comes after the researcher has conducted the all-important processing phase that converts qualitative data – that amalgamation of discrete elements – into meaningful connections that give rise to interpretations and implications, and the ultimate usefulness, of the research.

A quality approach to qualitative research design necessitates a well-thought-out plan for finding connections and making sense of the data. Here are six recommended steps in that process*:

•  Select the unit of analysis – a subject matter, an activity, a complete narrative or interview.
•  Develop unique codes – an iterative process utilizing a codebook that pays particular attention to context to derive explicit, closely-defined code designations.
•  Code – a dynamic process that incorporates pretesting of codes, inter-coder checks, and coder retraining as necessary.
•  Identify categories – a group of codes that share an underlying construct.
•  Identify themes or patterns – by looking at the coding overall and the identified categories to reveal the essence of the outcomes. This is made easier by way of visual displays via various programs such as PowerPoint and CAQDAS**.
Draw interpretations and implications – from scrutinizing the coded and categorized data as well as ancillary materials such as reflexive journals, coders’ coding forms (with their comments), and other supporting documents.

 

* Analyzability is one of four components of the Total Quality Framework. This framework and the six general steps in qualitative research analysis are discussed fully in Applied Qualitative Research Design: A Total Quality Framework Approach (Roller, M. R. & Lavrakas, P. J., 2015).

** Computer-assisted qualitative data analysis software, such as nVivo, Atlas.ti, MAXQDA.

Image captured from: http://www.breakthroughresults.co.uk/interim-management.php/

Every researcher working with human subjects strives to ensure the highest ethical standards. Regardless of whether the research is quantitative or qualitative in fingerprint-illusions-6nature – or in the field of health, communications, education, psychology, marketing, anthropology, or sociology – researchers care about protecting the confidentiality, anonymity, and basic “rights” (such as privacy and freedom of thought) of the people who agree to be part of their studies. It is with this in mind that, in addition to gaining IRB approval (as required), researchers openly discuss the goals and intended use of their research with participants, as well as asking them to carefully read and agree to the appropriate consent forms. Online group discussions (focus groups) present a particularly delicate matter. Unlike any other overt form of research – unlike an online survey dominated by closed-end questions, or an online in-depth interview with one person at any moment in time – the online group discussion – with its amalgamation of many people (typically, strangers to each other) responding at length to many open-ended questions over the course of multiple (possibly, many) days – potentially raises important security and identity concerns among participants. Even with a signed consent form, online group participants may still have serious doubts about the containment of their input to the discussion and, hence, their willingness to contribute Read Full Text

Follow

Get every new post delivered to your Inbox.

Join 309 other followers

%d bloggers like this: