Total Quality Framework

Analyzability & a Qualitative Content Analysis Case Study

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 284-285).

Kuperberg and Stone (2008) present a case study where content analysis was used as the primary research method. Gender & SocietyIt is an example of how many of the Total Quality Framework (TQF) concepts can be applied — not only to the in-depth interview, focus group, observation, and case centered methods, discussed elsewhere in Research Design Review, but — to qualitative content analysis. The discussion below spotlights aspects of this study relevant to one of the four TQF components, Analyzability.

Purpose & Scope
The primary purpose of this content analysis study was to extend the existing literature on the portrayal of women’s roles in print media by examining the imagery and themes depicted of heterosexual college-educated women who leave the workforce to devote themselves to being stay-at-home mothers (a phenomenon referred to as “opting out”) across a wide, diverse range of print publications. More specifically, this research set out to investigate two areas of media coverage: the content (e.g., the women who are portrayed in the media and how they are described) and the context (e.g., the types of media and articles).

This study examined a 16-year period from 1988 to 2003. This 16-year period was chosen because 1988 was the earliest date on which the researchers had access to a searchable database for sampling, and 2003 was the year that the term “opting out” (referring to women leaving the workforce to become full-time mothers) became popular. The researchers identified 51 articles from 30 publications that represented a wide diversity of large-circulation print media. The researchers acknowledged that the sample “underrepresents articles appearing in small-town outlets” (p. 502).

Analyzability
There are two aspects of the TQF Analyzability component — processing and verification. In terms of processing, the content data obtained by Kuperberg and Stone from coding revealed three primary patterns or themes in the depiction of women who opt out: “family first, child-centric”; “the mommy elite”; and “making choices.” The researchers discuss these themes at some length and support their findings by way of research literature and other references. In some instances, they report that their findings were in contrast to the literature (which presented an opportunity for future research in this area). Their final interpretation of the data includes their overall assertion that print media depict “traditional images of heterosexual women” (p. 510).

Important to the integrity of the analysis process, the researchers absorbed themselves in the sampled articles and, in doing so, identified inconsistencies in the research outcomes. For example, a careful reading of the articles revealed that many of the women depicted as stay-at-home mothers were actually employed in some form of paid work from home. The researchers also enriched the discussion of their findings by giving the reader some context relevant to the publications and articles. For example, they revealed that 45 of the 51 articles were from general interest newspapers or magazines, a fact that supports their research objective of analyzing print media that reach large, diverse audiences.

In terms of verification, the researchers performed a version of deviant case analysis in which they investigated contrary evidence to the assertion made by many articles that there is a growing trend in the proportion of women opting out. Citing research studies from the literature as well as actual trend data, the researchers stated that the articles’ claim that women were increasingly opting out had weak support.

Kuperberg, A., & Stone, P. (2008). The media depiction of women who opt out. Gender & Society, 22(4), 497–517.

In-depth Interview Data: Achieving Quality From Cooperation

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 75-77)

cooperationAn important aspect related to Scope within the Credibility component of the Total Quality Framework (TQF) for qualitative research design is the extent to which the researcher is successful in gaining cooperation from the participants. In an in-depth interview (IDI) study, the researcher is concerned with the impact that the proportion of selected interviewees not interviewed or only partially interviewed has on the integrity of the data. This is the domain of research that is often termed “nonresponse.” If this proportion is large and/or if the group that is selected but not interviewed differs in meaningful ways from those who are interviewed, bias can infiltrate the final data of an IDI study and compromise the credibility of the research.

To avoid this, qualitative researchers need to give serious a priori thought to how they will gain high and representative levels of cooperation from the persons they have selected to interview, and how individuals who do not cooperate may differ in past experiences, attitudes, behaviors, and knowledge compared to interviewees. The researcher must keep in mind that bias may enter into the outcomes, and the credibility of the study’s findings and interpretations thereby weakened, if the characteristics of those in the sample who do not cooperate with an IDI study are correlated with the key topics the study is investigating. Likewise, qualitative researchers using the IDI method should also constantly monitor the representativeness of the group of selected participants that does cooperate and watch whether the characteristics of that group deviate from the characteristics of the target population. This may be difficult in the case of the email IDI (or other asynchronous text-based mode) where the interviewer must stay alert to the consistency of participants’ responses and recognize when the identity of the interviewee may have changed (i.e., someone other than the recruited research participant is the one now responding). For instance, in an email IDI study among Read Full Text

Gathering Quality Ethnographic Data: 3 Key Considerations

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 204-206).

Data Gathering is one of two broad areas of the Total Quality Framework Credibility component that affects all qualitative research, incEthnography peacockluding ethnographic research. There are three primary aspects concerning the gathering of data in ethnography that require serious consideration by the researcher in the development of the study design. To optimize the measurement of ethnographic data, and hence the quality of the outcomes, researchers need to pay attention to:

  • How well the observers have identified and recorded all the information (e.g., verbal and nonverbal behavior, attitudes, context, sensory cues) pertinent to the research objectives and constructs of interest. A well-developed observation guide and observation grid can assist greatly in this effort. Not unlike the development of an in-depth interview or discussion guide, the ethnographer seeks to identify those observable events—including the specific individuals (or types of individuals), the verbal and nonverbal behaviors, attitudes, sensory and other environmental cues—that will further the researcher’s understanding of the issues. During the design development phase, the researcher might isolate the observations of interest by:
    • Looking at earlier ethnographic research on the subject matter and/or with similar study populations.
    • Interviewing the clients or those who have requested the research to learn everything they know about the topic and   their past work in the area.
    • Consulting the literature or other experts concerning the behaviors and other occurrences associated with particular constructs.
    • “Shagging around” (LeCompte & Goetz, 1982) the observation site(s) to casually assess the environment and begin to learn about the participants.

 

  • Observer effects, specifically—
    • Observer bias, that is, behavioral and other characteristics (e.g., personal attitudes, values, traits) of the observer that may alter the observed event or bias their observations. For example, an observer as a complete participant would bias the observational data if there was an attempt to “educate” participants on a subject matter for which the observer had personal expertise or knowledge.
    • Observer inconsistency, that is, an inconsistent manner in which the observer conducts the observations that creates unwarranted and unrepresentative variation in the data. For example, an on-site nonparticipant observer conducting in-home observations of the use of media and technology would be introducing inaccuracies in the data by observing and recording the use of television and gaming in some households but not in others where television and gaming activities took place.

 

  • Participant effects, specifically, the extent to which observed participants alter a naturally occurring event, leading to biased outcomes. This is often called the Hawthorne effect, whereby the people being observed, either consciously or unconsciously, change what is being measured in the observation because they are aware of the observer. For example, an ethnographer conducting an overt, on-site passive observation of teaching practices in a school district would come away with misleading data if one or more school teachers deviated from their usual teaching styles during the observations in order to more closely conform with district policies.

 

LeCompte, M. D., & Goetz, J. P. (1982). Ethnographic data collection in evaluation research. Educational Evaluation and Policy Analysis, 4(3), 387–400.

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach. New York: Guilford Press.