Participant Effects

Research Quality & the Impact of Monetary Incentives

The following is adapted from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 78-79).

Monetary incentives & quality researchGaining cooperation from research participants and respondents is important to the quality of qualitative and survey research. A focus on gaining cooperation helps to mitigate potentially weakened data due to the possibility that the individuals who do not cooperate — do not participate in the research — differ in meaningful ways compared to those who do cooperate. As mentioned in an article posted earlier in Research Design Review, an effective component to the researcher’s strategy for gaining cooperation among participants is the offer of material incentives (e.g., cash, a gift card, prized tickets to a sporting event, donation to a favorite charity).

Although monetary incentives are routinely given to qualitative research participants to boost cooperation, the researcher needs to keep in mind that the offer of a cash (or equivalent) incentive may also jeopardize the quality of the actual focus group discussion, in-depth interview, or observation. The following is one example of how monetary incentives may have the unwanted effect of skewing participants’ responses in an in-depth interview (IDI) study.

Cook and Nunkoosing (2008) conducted an in-person IDI study with 12 “impoverished elders” in Melbourne, Australia to investigate community services for the poor among those “who are excluded or at risk of exclusion from their communities.” Research participants could participate in up to two interviews and were given $20 for each interview.

In reviewing the key findings, the researchers observed many “interview interactions that were atypical.” At least part of these irregularities was attributed to the monetary incentive which, according to Cook and Nunkoosing, helped to create an interview environment where interviewees were motivated “to manage the presentation of self, retain control over the exchange of information, and reduce the stigma of poverty by limiting disclosure and resisting researcher questioning” (p. 421).

The importance of the incentive in the interview process became clear when interviewees volunteered comments such as “I need the $20 . . . ” and critically compared the $20 to better (i.e., higher) cash incentives offered by other research studies. In this way interviewees were in effect “selling” their stories to the interviewer (and, some would say, at a bargain price) which, based on the researchers’ analyses, tainted interviewees’ responses with “stylized accounts” (or “rehearsed narratives”) as well as “minimal disclosure,” as seen in this excerpt from the transcripts (p. 424):
Participant: What did you want to know?
Interviewer: All about you.
Participant: That’s about it, like, there’s not too much.
Interviewer: Do you want to tell me a bit more? I don’t really know who you are yet.
Participant: You do.
Interviewer: Tell me a bit about who you are, what you like, what you don’t like.
Participant: I don’t like him [Gesturing toward the other agency client].

This dialog came towards the end of a 30-minute interview and helps to illustrate “the researcher’s frustration at [their] inability to engage the participant in in-depth discussion” (p. 423).

Research design is always a balancing act involving various trade-offs associated with meeting the key objectives, method(s) and strategy for engaging the target population(s), and the efficient use of available resources. An important researcher skill is understanding the implications of these trade-offs to the integrity of the final data and overall quality of the research investigation. A monetary incentive may be highly effective in securing participation in our research but what is its ultimate impact on data quality? This is the concern of a skilled researcher.

Cook, K., & Nunkoosing, K. (2008). Maintaining dignity and managing stigma in the interview encounter: The challenge of paid-for participation. Qualitative Health Research, 18(3), 418–427. https://doi.org/10.1177/1049732307311343

The Asynchronous Focus Group Method: Participant Participation & Transparency

There is a great deal that is written about transparency in research. It is generally acknowledged that researchers owe it to their research sponsors as well as to the broader research community to divulge the details of their designs and the implementation of their studies. Articles pertaining to transparency Participant participation in asynchronous focus group discussionshave been posted throughout Research Design Review.

The need for transparency in qualitative research is as relevant for designs utilizing off-line modes, such as in-person interviews and focus group discussions, as it is for online research, such as asynchronous focus groups. A transparency detail that is critical for the users of online asynchronous – not-in-real-time – focus group discussions research is the level of participant participation. This may, in fact, be the most important information concerning an asynchronous study that a researcher can provide.

Participation level in asynchronous discussions is particularly important because participation in the online asynchronous mode can be erratic and weak. Nicholas et al. (2010) found that “online focus group participants offered substantially less information than did those in the [in-person] groups” (p. 114) and others have underscored a serious limitation of this mode; that is, “it is very difficult to get subjects with little interest in [the topic] to participate and the moderator has more limited options for energising and motivating the participants” (Murgado-Armenteros et al., 2012, p. 79) and, indeed, researchers have found that “participation in the online focus group dropped steadily” during the discussion period (Deggs et al., 2010, p. 1032).

The integrity and ultimate usefulness of focus group data hinge solidly on the level of participation and engagement among group participants. This is true regardless of mode but it is a particularly critical consideration when conducting asynchronous discussions. Because of this and because transparency is vital to the health of the qualitative research community, focus group researchers employing the online asynchronous method are encouraged to continually monitor, record, and report on the rate and level of participation, e.g., how many and who (in terms of relevant characteristics) of the recruited sample entered into the discussion, how many and who responded to all questions, how thoughtful and in-depth (or not) were responses, how many and who engaged with the moderator, and how many and who engaged with other participants.

This transparent account of participant participation offers the users of asynchronous focus group research an essential ingredient as they assess the value of the study conducted.

Deggs, D., Grover, K., & Kacirek, K. (2010). Using message boards to conduct online focus groups. Retrieved from http://www.nova.edu/ssss/QR/QR15-4/deggs.pdf

Murgado-Armenteros, E. M., Torres-Ruiz, F. J., & Vega-Zamora, M. (2012). Differences between online and face-to-face focus groups, viewed through two approaches. Journal of Theoretical and Applied Electronic Commerce Research, 7(2), 73–86.

Nicholas, D. B., Lach, L., King, G., Scott, M., Boydell, K., Sawatzky, B., … Young, N. L. (2010). Contrasting Internet and face-to-face focus groups for children with chronic health conditions : Outcomes and participant experiences. International Journal of Qualitative Methods, 9(1), 105–122.

Image captured from: https://uwm.edu/studentinvolvement/student-organizations-2/our-communityinvolvement/

The Social Environment & Focus Group Participants’ Willingness to Engage

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 148-150).

Beyond discussion guide development and the effects of the moderator, there is another critical component that threatens the quality of data gathered in the focus group discussion method: the participants themselves. The social environment of a focus group discussionparticipants in a group discussion face a more daunting social environment than in-depth interviewees, an environment in which participants are typically expected to meet (in-person, on the phone, or online) and engage with a group of strangers. At the minimum, participants in a dyad find themselves among two other individuals they have never met (the moderator and other participant); and, in the opposite extreme, participants in an online asynchronous group may be one of 10 or 12 or more people who have been asked to join the discussion.

As with the in-depth interview (IDI) method, focus group participants in any mode (i.e., in-person, phone, or online) may threaten the integrity and credibility of group discussion data by their unwillingness or reluctance to divulge certain information, leading them to say nothing or to make an inaccurate statement. For instance, in some focus group studies, what people do not know (or have not done) is a central part of what the study is exploring (e.g., recruiting people who have not been involved with a local nonprofit organization to learn about their Read Full Text