Data quality matters. Regardless of the research method or approach, our ability to say anything meaningful about our research outcomes hinges on the integrity of the data. The greater care the researcher takes to ensure the basic ingredients of “good” research design, the more confident the researcher and importantly the user of the research will be in the recommendations drawn from the research and its ultimate usefulness.

This focus on data quality applies to all research. And although it is most often a topic of discussion among survey researchers, data quality considerations are increasingly (I hope!) a discussion among qualitative researchers as well. Indeed, the underlying validity of our qualitative data is an important consideration regardless of the researcher’s paradigm orientation or the qualitative method, including the more recent methodological options – that is, mobile and online qualitative research.

Mobile and online technology – in particular, tech solutions that combine observation with a multimethod/mode approach – offer qualitative researchers new ways to investigate a variety of situations that give them a closer understanding of participants’ lived experiences as never before possible. Three such situations are:

• “Day in the life,” e.g., to explore daily activities such as daily eating or medication habits, or the mobility patterns among children (Christensen, Mikkelsen, Nielsen, & Harder, 2011).
• “Journey” or decision making, e.g., to explore the path people take to achieve a certain goal such as the journey from a cancer diagnosis through the course of treatment, or the path to purchase among consumers, or how educators make decisions to use the Internet in the classroom and its influence on students’ literacy (Karchmer, 2001).
• Specific assignments/tasks, e.g., a “show and tell” study to explore how people prepare a meal using a particular product, or asking patients to show and discuss particular areas within their homes (such as the refrigerator and medicine cabinet) to understand how they cope with their disease (Hancock, 2012).

Regardless of the particular situation under investigation, there are clearly data quality advantages to mobile and online solutions. From the perspective of the Total Quality Framework’s Credibility component (which pertains to data collection), elements of both Scope and Data Gathering strengthen the quality of mobile and online data. In particular, these tech solutions: expand the researcher’s geographic coverage and potentially garner high levels of cooperation among participants who are comfortable with the technology (Scope). These approaches also: add depth to (and the ability to triangulate) the data due to the multifaceted layers of methods and modes, help mitigate researcher bias, and enable participants to more fully engage with the research by way of the various tasks and length of the research process (Data Gathering).

There are, however, a number of ways in which data quality resulting from mobile and online approaches is seriously weakened. Contrary to the idea (shared by some) that these tech solutions are the answer to a host of research design dilemmas – such as the ability to include many (up to 100) participants in a qualitative study, offering participants their most preferred way to participate, and efficient project management (by way of the available platforms) – the we-can-do-it-all thinking around mobile and online methods ignores the negative implications associated with the quality of the data. It would be a gross oversight with detrimental consequences to ignore the fact that: mobile and online solutions bias the sample towards tech savvy segments of the population as well as potentially limit coverage due to reduced cooperation associated with the tasks and length of these studies (Scope). Importantly, the quality of the data is also potentially weakened by: researcher effects associated with impinging on participants’ lives and thus tainting the study environment as well as poor study management due to weak multi-tasking skills, as well as participant effects resulting from a Hawthorne-type effect (i.e., altered behavior and attitudes due to the act of participation and the researcher’s remote presence) as well as “selection bias” or the participant’s control of what is and is not shared with the researcher (Data Gathering).

The quality of our qualitative data needs to be assessed at each turn of the research process. This is no less true for newer, technology-based qualitative methods and modes than traditional approaches. Just a few of the data quality considerations associated with these tech solutions have been proposed here.

 

Christensen, P., Mikkelsen, M. R., Nielsen, T. A. S., & Harder, H. (2011). Children, mobility, and space: Using GPS and mobile phone technologies in ethnographic research. Journal of Mixed Methods Research, 5(3), 227–246.

Hancock, K. (2012, October). Online qual guides health care foundation to shift its focus. Quirk’s Marketing Research Review, 30–32. Retrieved from http://www.quirks.com/articles/2012/20121006.aspx?searchID=702818743&sort=5&pg=1

Karchmer, R. A. (2001). The journey ahead: Thirteen teachers report how the Internet influences literacy and literacy instruction in their K-12 classrooms. Reading Research Quarterly, 36(4), 442–466.

Image captured from: https://www.scoop.it/t/art-and-craft-by-sunish-sebastian