question design

Focusing on a Research Design Reality: Questions Lead to Answers

Questions Lead to AnswersAt the core of research design development — in quantitative and qualitative methods — is the reality that individuals who have agreed to participate in our research studies generally answer the questions we ask. This fundamental reality places a heavy burden on the researcher developing a quality research design. Survey research that relies on closed-end questionnaire items is vulnerable to unreliable data due to question design that confuses respondents or fosters interpretations outside the true intention of the question asked. All of which leaves the researcher with weak data and consequently flawed analysis and erroneous final results. The need for more involved research designs that effectively investigate complex subject matter is discussed throughout Research Design Review, including in “Life Is Meaningful, Or Is It?: The Road To Meaning In Survey Data” and “Feelings & Sensations: Where Survey Designs Fail Badly.”

Ask a willing research respondent/participant a question and you are likely to get an answer. It may not be the question the researcher intended, it may confuse the responding individual, but the Read Full Text

Re-considering the Question of “Why”

It is easy to fall into the trap of relying on the “why” question when conducting qualitative research. After all, the use of qualitative research is often supported with the claim that qualitative methods enable the researcher to reach beyond quantitative numerical data to grasp the meaning and motivations – that is, the why – associated with particular attitudes and behavior. And it is in this spirit that researchers frequently find themselves with interview and discussion guides full of “why” questions – Why do you say you are happy? Why do you prefer one political candidate over another? Why do you diet? Why do you believe in God? Why do you use a tablet rather than a laptop computer?

Yet “why” is rarely the question worth asking. In fact, asking “why” questions can actually have a negative effect on data collection (i.e., Credibility) and may contribute to a distortion in qualitative data. This happens for many reasons, here are just four:

The “why” question potentially

• Evokes rationality. By asking the “why” question, researchers are in essence asking participants to justify their attitudes and behavior. In contemplating a justification, it is not unusual for participants to seek Read Full Text

Pigeonholing Qualitative Data: Why Qualitative Responses Cannot Be Quantified

A webinar on the ins-and-outs of qualitative research stated that qualitative data could be quantified by simply counting the codes associated with some aspect of the data content, such as the number of times a particular brand name is mentioned or a specific sentiment is expressed towards a Pigeonholetopic of interest.  The presenter asserted that, by counting these codes, the researcher has in effect “converted” qualitative to quantitative data.

This way of thinking is not unlike those who contend that useful quantitative data can be calculated with qualitative findings by counting the number of “votes” for a particular concept or some aspect of the research subject matter.  Let’s say a moderator asks group participants to rate a new product idea on a modest four-point scale from “like very much” to “do not like at all.”  Or, an interviewer conducting qualitative in-depth interviews (IDIs) asks each of the 30 participants to rate their agreement with statements pertaining to the advantages of digital technology on a scale from “strongly agree” to “strongly disagree.”  It is the responses to these types of questions that some researchers gather up as votes and report as quantitative evidence.

By asserting that codes and votes can be counted and hence transform a portion of qualitative findings Read Full Text