If you are one of those researchers who work in both quantitative and qualitative design, something you are reminded of fairly quickly at the AAPOR annual conference – currently being held in Boston – is that there is really little separating the two genres. Survey researchers may deem the ‘hard facts’ of their quantitative data as a gold standard of sorts; and qualitative researchers may look questioningly at the righteousness of these ‘hard facts’, asking “Where’s the beef?” that explains the “why” behind the data, but there is no debate that we are all after the same thing. The following are just a few of the common areas of interest among quantitative and qualitative researchers:
- Question administration – What to ask & how to ask it
- Interviewer effect – Impact of interviewer’s behavior, appearance, & attitude on response
- Mode – Which mode for which population segment & its impact on response
- Cooperation – How to increase participation & decrease respondent/participant burden
- Analysis – How to organize data & develop coding schemes that accurately represent the data
- Cost – How to “do more” with smaller research budgets
And, interestingly, researchers of all stripes are addressing similar issues within each of these areas, such as:
- Question administration –
- What role does context – question context &/or physical environment – play in response?
- How does conversational interviewing affect response?
- How do you avoid “crummy” questions such as the one posed by Jack Fowler, Jr., PhD, the 2013 AAPOR Award winner in his acceptance speech tonight – ‘How often do you buckle your seat belt when sitting in the backseat of a car?’ (Hint: It is a double-barrel question)
- Interviewer effect –
- What effect do the interviewer’s probing questions have on response?
- How does the moderator’s appearance alter a face-to-face discussion?
- Mode –
- How does the mode impact interviewer-participant rapport & interaction?
- What effect will choice of mode have on the particular study population?
- Which is the “best” mode for sensitive topics?
- Cooperation –
- What are the most effective recruitment strategies to gain cooperation?
- What role does incentive – amount & type – play in gaining cooperation?
- Analysis –
- What is the best inductive approach for this particular study?
- Is the interpretation of the data supported by the analytical process?
- Cost –
- What are the tradeoffs between opting for a less expensive approach?
- If the incentive is decreased, what will this do to cooperation?
There is, however, one important difference. It is a difference that rings loudly while sitting at AAPOR listening to the work of these mostly quantitative researchers. And that difference, of course, is that the survey folks grapple with these issues head on. They experiment and test and look at the myriad of design issues upside down and sideways, always searching for ways to tweak their designs in order to achieve more reliable projectable outcomes. Qualitative research never will and never should be about projectable or reliable outcomes but there are any number of ways that qualitative researchers could be learning more about the effectiveness of their designs and the realities of their findings. The art of question design, behavioral coding, selection bias, and non-response – these and so much more should be fertile areas for qualitative researchers to explore in their work with the goal of producing research that is credible, analyzable, transparent, and ultimately useful. These are the quality components that all researchers can agree on.