research design

A TQF Approach to Choosing a Sample Design

The Total Quality Framework (TQF) offers qualitative researchers a way to think critically about their research Credibility TQF componentdesigns and helps to guide their decision making. The TQF consists of four components, with each component devoted to the critical thinking considerations associated with a phase in the research process. The first component of the TQF is Credibility which is focused on data collection; specifically, Scope and Data Gathering. One of the many considerations related to Scope has to do with the sample design.

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 25-26) on the different aspects of sampling that researchers might want to think about as they develop their qualitative research designs.


Once the researcher has identified the list (or lists) that will be used to select the sample, a decision must be made about which sampling approach will be used. If the decision is to gather data from each member of the population on the list (e.g., all 20 students enrolled in an honors science class), then there is nothing more for the researcher to consider. But for those studies where something less than the entire population will be chosen for study, additional Total Quality Framework (TQF) decisions need to be made about sampling.

Here, qualitative researchers may needlessly lessen the quality of their studies by not giving these decisions sufficient consideration. In fact, some qualitative researchers may think that how they create a sample of the population is unimportant. Qualitative researchers may proceed in this manner because they mistakenly believe that systematic sampling is too hard to carry out (i.e., too complex, too expensive, and too time-consuming) and that it is “too quantitative” a concern. Yet, in the vast majority of qualitative studies, systematic sampling is neither complex, expensive, nor time-consuming, and should not only be a quantitative issue. And by using an organized approach for choosing which members of their key population to study, as opposed to merely using a convenient and disorderly approach to sampling, qualitative researchers avoid a major threat to the credibility of the data they gather. That threat is the possibility that those from whom they gather data are not, in fact, representative (do not share defining characteristics) of the population being studied.

Take, for example, a focus group researcher that has a list of men and women who completed a cardiopulmonary resuscitation (CPR) training class in the past year. The researcher can choose one of two basic approaches to selecting those who will be invited to participate in a group discussion. The often-used but misguided approach is to start at the top of the list and contact people, one after another, until the focus groups have been filled with ostensibly willing attendees. The rigorous and correct approach is to use an organized scheme to sample CPR class graduates from across the entire list (i.e., stratifying the list and taking an ‘nth’ name approach). The second approach is preferred because it avoids the possible problem that the names on the list are ordered in a way that is not representative of the entire population of CPR graduates that the researcher wants to study.

A final TQF issue related to choosing a sample applies to qualitative studies that utilize observations of naturally occurring human behavior to gather data, such as in ethnographic research. In these studies, sampling considerations need to be applied to the times and the locations during which the behaviors of interest will be observed. By systematically choosing which locations and which times to conduct the observations—among all possible locations and times in which the behaviors of interest will be taking place—the qualitative researcher is greatly raising the likelihood that the observations included in the study are a representative subset of all the possible behaviors of interest to the study.

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach. New York: Guilford Press.

Is It Good Research?

To see this and other slide decks on best practices in research methods and design, go to https://www.slideshare.net/MargaretRoller.

 

Actively Conducting an Analysis to Construct an Interpretation

It is not uncommon for researchers who are reporting the results of their quantitative studies to go beyond describing their numerical data and attempt to interpret the meaning associated with this data. For example, in a survey concerning services at a healthcare facility, the portion of respondents who selected the midpoint on a five-point scale to rate the improvement of these services from the year before might be interpreted as having a neutral opinion, i.e., these respondents believe the caliber of services has remained the same, neither better nor worse than a year earlier. And yet there are other interpretations of the midpoint response that may be equally viable. These respondents may not know whether the services have improved or not (e.g., they were not qualified to answer the question). Or, these respondents may believe that the services have gotten worse but are reluctant to give a negative opinion.

Survey researchers fall into this gray area of interpretation because they often lack the tools to build a knowledgeable understanding of vague data types, such as scale midpoints. Unless the study is a hybrid research design (i.e., a quantitative study that incorporates qualitative components), the researcher is left to guess respondents’ meaning.

In contrast, the unique attributes of qualitative research methods offer researchers the tools they need to construct informed interpretations of their data. By way of context, latent (coupled with manifest) meanings, the participant-researcher relationship, and other fundamentals associated with qualitative research, the trained researcher collects thick data from which to build an interpretation that addresses the research objectives in a profound and valuable manner for the users of the research.

Qualitative data analysis is a process by which the researcher is actively involved in the creation of themes from the data and the interpretation within and across themes to construct results that move the topic of investigation forward in some meaningful way. This active involvement is central to what it means to conduct qualitative research. Faithful to the principles that define qualitative research, researchers do not rest on manifest content, such as words alone, or on automated tools that exploit the obvious, such as word clouds.

This is another way of saying — as stated in this article on sample size and saturation — that “themes do not simply pop up…but rather are the result of actively conducting an analysis to construct an interpretation.” As Staller (2015) states, “In lieu of the language of ‘discovering’ things with its positivistic roots, the researcher is actually interpreting the evidence” (p. 147).

Braun and Clarke (2006, 2016, 2019, 2021) have written extensively about the idea that “themes do not passively emerge” (2019, p. 594, italics in original) from thematic analysis and that meaning

is not inherent or self-evident in data, that meaning resides at the intersection of the data and the researcher’s contextual and theoretically embedded interpretative practices – in short, that meaning requires interpretation. (2021, p. 210)

An article posted in 2018 in Research Design Review“The Important Role of ‘Buckets’ in Qualitative Data Analysis” — illustrates this point. The article discusses the analytical step of creating categories (or “buckets”) of codes representing shared constructs prior to building themes. As an example, the discussion focuses on three categories that were developed from an in-depth interview study with financial managers — Technology, Partner, Communication. The researcher constructed themes by looking within and across categories, considering the meaning and context associated with each code. One such theme was “strong partnership,” as illustrated below.

Themes from buckets

The theme “strong partnership” did not simply emerge from the data, it was not lying in the data waiting to be discovered. Rather, the researcher utilized their analytical skills, in conjunction with their constructed understanding of each participant’s contribution to the data, to create contextually sound, meaningful themes such as “strong partnership.” Then, with the depth of definition associated with each theme, the researcher looked within and across themes to build an interpretation of the research data targeted at the research objectives, and provided the users of the research with a meaningful path forward.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun, V., & Clarke, V. (2016). (Mis)conceptualising themes, thematic analysis, and other problems with Fugard and Potts’ (2015) sample-size tool for thematic analysis. International Journal of Social Research Methodology, 19(6), 739–743. https://doi.org/10.1080/13645579.2016.1195588

Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, Vol. 11, pp. 589–597. https://doi.org/10.1080/2159676X.2019.1628806

Braun, V., & Clarke, V. (2021). To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qualitative Research in Sport, Exercise and Health, 13(2), 201–216. https://doi.org/10.1080/2159676X.2019.1704846

Staller, K. M. (2015). Qualitative analysis: The art of building bridging relationships. Qualitative Social Work, 14(2), 145–153. https://doi.org/10.1177/1473325015571210