qualitative analysis

Sample Size in Qualitative Research & the Risk of Relying on Saturation

Qualitative and quantitative research designs require the researcher to think carefully about how and how many to sample within the population segment(s) of interest related to the research objectives. In doing so, the researcher considers demographic and cultural diversity, as well as other distinguishing characteristics (e.g., usage of a particular service or product) and pragmatic issues Risk of Relying on Saturation(e.g., access and resources). In qualitative research, the number of events (i.e., the number of in-depth interviews, focus group discussions, or observations) and participants is often considered at the early design stage of the research and then again during the field stage (i.e., when the interviews, discussions, or observations are being conducted). This two-stage approach, however, can be problematic. One reason is that giving an accurate sample size prior to data collection can be difficult, particularly when the researcher expects the number to change as the result of in-the-field decisions.

Another potential problem arises when researchers rely solely on the concept of saturation to assess sample size when in the field. In grounded theory, theoretical saturation

“refers to the point at which gathering more data about a theoretical category reveals no new properties nor yields any further theoretical insights about the emerging grounded theory.” (Charmaz, 2014, p. 345)

In the broader sense, Morse (1995) defines saturation as “‘data adequacy’ [or] collecting data until no new information is obtained” (p. 147).

Reliance on the concept of saturation presents two overarching concerns: 1) As discussed in two earlier articles in Research Design ReviewBeyond Saturation: Using Data Quality Indicators to Determine the Number of Focus Groups to Conduct and Designing a Quality In-depth Interview Study: How Many Interviews Are Enough? – the emphasis on saturation has the potential to obscure other important considerations in qualitative research design such as data quality; and 2) Saturation as an assessment tool potentially leads the researcher to focus on the obvious “new information” obtained by each interview, group discussion, or observation rather than gaining a deeper sense of participants’ contextual meaning and more profound understanding of the research question. As Morse (1995) states,

“Richness of data is derived from detailed description, not the number of times something is stated…It is often the infrequent gem that puts other data into perspective, that becomes the central key to understanding the data and for developing the model. It is the implicit that is interesting.” (p. 148)

With this as a backdrop, a couple of recent articles on saturation come to mind. In “A Simple Method to Assess and Report Thematic Saturation in Qualitative Research” (Guest, Namey, & Chen, 2020), the authors present a novel approach to assessing sample size in the in-depth interview method that can be applied during or after data collection. This approach is born from Read Full Text

Qualitative Data Analysis: 16 Articles on Process & Method

“Qualitative Data Analysis: 16 Articles oQualitative Data Analysisn Process & Method” is a new compilation of selected articles appearing in Research Design Review from 2010 to December 2019 concerning various facets of qualitative data analysis. Although there are other RDR articles posted in this time period related to analysis — such as articles on transparency, e.g., “The Use of Quotes & Bringing Transparency to Qualitative Analysis” and those pertaining to quantitative-qualitative research topics, e.g., “Qualitative Analysis: The Biggest Obstacle to Enriching Survey Outcomes”  — the 16 articles in this compilation are narrowly focused on issues relevant to applying a quality approach to the analytical process — e.g., identifying the unit of analysis, coding, and use of “buckets” — and the qualitative content analysis method.

“Qualitative Data Analysis: 16 Articles on Process & Method” is available for download here.

Two other compilations are also available for download: “The Focus Group Method: 18 Articles on Design & Moderating is available for download here. And “The In-depth Interview Method: 12 Articles on Design & Implementation” is available for download here.

Supporting Observational Research

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 217-219) which is a qualitative methods text covering in-depth interviews, focus group discussions, ethnography, qualitative content analysis, case study, and narrative research.

An important element in the Total Quality Framework Analyzability component is Verification, i.e., taking steps to establish some level of support for the data gathered in order to move the researcher closer to achieving high quality outcomes. The verificationSupporting qualitative data tools at the ethnographer’s disposal go beyond those identified for the in-depth interview (IDI) and group discussion methods in that they include the technique of expanded observation. For example, Lincoln and Guba (1985) stated that it is “more likely that credible findings and interpretations” will come from ethnographic data with “prolonged engagement” in the field and “persistent observation” (p. 301). The former refers to spending adequate time at an observation site to experience the breadth of stimuli and activities relevant to the research, and the purpose of the latter (i.e., persistent observation) is “to identify those characteristics and elements in the situation that are most relevant to the problem or issue” (p. 304)—that is, to provide a depth of understanding of the “salient factors.” Both prolonged engagement and persistent observation speak to the idea of expanding observation in terms of time as well as diligence in exploring variables as they emerge in the observation. Although expanding observations in this way may be unrealistic due to the realities of deadlines and research funding, it is an important verification approach unique to ethnography. When practicable, it is recommended that researchers maximize the time allotted for observation and train observers to look for the unexpected or examine more closely seemingly minor occurrences or variables that may ultimately support (or contradict) the observer’s dominant understanding.

The ultimate usefulness of expanded observation is not unlike deviant or negative case analysis (see earlier link). In both instances, the goal is to identify and investigate observational events (or particular variables in these events) that defy explanation or otherwise contradict the general patterns or themes that appear to be emerging from the data. For example, a researcher conducting in-home nonparticipant observations of young mothers Read Full Text