validity

Satisfaction Research & Other Conundrums

Greg Allenby, marketing chair at Ohio State’s business school, published an article in the May/June issue of Marketing Insights on heterogeneity or, more specifically, on the idea that 1) accounting for individual differences is essential to understanding Conundrumthe “why” and “how” that lurks within research data and 2) research designs often mask these differences by neglecting the relative nature of the constructs under investigation. For instance, research concerning preference or satisfaction is useful to the extent it helps explain why and how people think differently as it relates to their preferences or levels of satisfaction, yet these are inherently relative constructs that only hold meaning if the researcher understands the standard (the “point of reference”) by which the current question of preference or satisfaction is being weighed – i.e., my preference (or satisfaction) compared to…what? Since the survey researcher is rarely if ever clued-in on respondents’ points of reference, it would be inaccurate to make direct comparisons such as stating that someone’s product preference is two times greater compared to someone else’s.

The embedded “relativeness” associated with responding to constructs such as preference and satisfaction is just one of the pesky problems inherent in designing this type of research. A related but different problem revolves around the personal interpretation given Read Full Text

Exploring the True Colors in Qualitative Data

Reliability, in the sense of being able to obtain identical findings from repeated executions of a qualitative research design, is debatable.  Validity, however, is another matter.  Validity, in the sense of whether the qualitative researcher is collecting the information (data) he or she claims to be gathering (i.e., ttrue colorshe accuracy of the data), is a topic worthy of much more discussion in the research community, or at the least a greater emphasis in our qualitative research designs.  While qualitative researchers may not be able to replicate their studies, they surely have the means to consider the authenticity of the data.

There was a Research Design Review post back in 2010 that discussed the importance and appropriateness of validity in qualitative research, including the idea that there are ready-made techniques for looking at validity in qualitative research and that, in some ways, validity is already built into our research methods.  To illustrate how qualitative researchers typically incorporate validity Read Full Text

Managing Ghosts & the Case for Triangulation in Qualitative Research

The most recent issue of the American Psychological Association’s Monitor on Psychology includes an interview with developmental psychologist, Jerome Kagan.  In this interview he talks about psychology’s research “ghosts,” referring to the dubious generalizations psychologist’s make from their often-limited research.  Kagan’s primary point is that “it’s absolutely necessary to gather more than one source of data, no matter what you’re studying,” and that these multiple sources of data should come from verbal and behavioral as well as physiological measures.  Only by combining these various perspectives on an issue or situation – that is, utilizing data taken in different contexts and by way of alternative methods and modes – can the researcher come to a legitimate conclusion.

This is not unlike triangulation, esp., in the social and health sciences, which is used to gauge the trustworthiness of research outcomes.  Triangulation is the technique of examining a specific research topic by comparing data obtained from: two or more methods, two or more segments of the sample population, and/or two or more investigators.  In this way, the researcher is looking for patterns of convergence and divergence in the data.  Triangulation is a particularly important design feature in qualitative research – where measures of validity and reliability can be elusive – because it furthers the researcher’s ability to gain a comprehensive view of the research question and come closer to a plausible interpretation of final results.

Where is this multifaceted process in the commercial world of qualitative marketing research?  Academics talk about the importance of including some form of triangulation in research design yet there is not a lot of evidence that this occurs in marketing research.  While there are an increasing number of ways to gather qualitative feedback – particularly via social media and mobile – that provide researchers with convenient sources of data, there needs to be more discussion on case studies that have utilized multiple data sources and methods to find reliable themes in the outcomes.  Importantly, it is further hoped that marketing researchers use this contrast-and-compare approach to scrutinize the research issue from both traditional (e.g., face-to-face group discussions, in-depth interviews, in-home ethnography) and new (e.g., online based, smartphone) information-gathering strategies.

The triangulation concept is just one way that marketing researchers can begin to bring rigor to their research designs and manage the “ghosts” of groundless assumptions and misguided interpretations.