question design

Feelings & Sensations: Where Survey Designs Fail Badly

Survey research is pretty good at allowing people to describe “things” in such a way that the researcher winds up with a fairly accurate idea of the thing being described. The most straight-forward example is a survey question that asks, “Which of the following features came with Hotel experienceyour new Toyota Corolla?” followed by a list of possible features. However, survey research can also get at descriptions of more experiential phenomena with questions such as, “On a scale from ‘1’ to ‘5’, how does each of the following statements describe your experience in buying a new home?” In these cases, the use of survey methods to research a great number of people, and compile and report the data as efficiently as possible, make good use of closed-ended questions to gain an understanding of respondents’ accounts of the “things” of interest. This can also be said of beliefs. Pew’s recent survey pertaining to the Christmas story that asked, Read Full Text

Humanizing Survey Question Design with a Qualitative Touch

Researchers know that “good” survey questionnaire design begins with a preliminary qualitative research phase that serves to expose the nuances of the research touchtopic or category – such as the most pertinent issues and the relevant concerns or “issues within the issues” – along with the manner by which the target population talks about these issues – that is, the particular words, expressions, and terminology used by the target group. In this way, the survey researcher can hope to create user-friendly survey questions that speak to respondents rather than at respondents.

A preliminary qualitative phase is good and necessary, but employing the talents of a qualitative researcher during survey question development is an equally-important step. Qualitative researchers spend much of their lives listening to people talk about a host of attitudinal and behavioral issues, listening to the use of language, and using these conversations to interpret Read Full Text

Reporting What We Know From What We Ask

For most of us, it is important to write a final research report that goes beyond the questions we asked and the responses we received. Unlike a topline debriefing that may require a simple rundown of the questions and responses, our qualitative and quantitative studies typically Pop-tartsculminate in write-ups that provide thoughtful discussions of our analyses and interpretations of the data.

The consumers of our research reports take it on blind faith that the data along with the corresponding questions and issues are reported accurately, and that the researchers’ interpretations of the findings are consistent with both the data and the questions asked or issues raised.   And yet blind faith is not always enough. Those are the times when a closer look at what the research actually asked and what is actually reported is needed.

One example is a July 2014 report from Gallup on its research concerning Americans’ consumption habits. The report, in part, shows that nearly all (more than 90%) Read Full Text