Don Gloeckler’s recent article* in Quirk’s concerning the quality of online research is important.  While it is not news that Procter & Gamble (where Gloeckler is senior manager) is at the forefront of demanding change – or at least critical introspection – within the marketing research industry, a reminder of the need for research design quality standards is welcomed.

Many of us can relate to Gloeckler’s discussion of “less-than-perfect” experiences with online research.  He mentions “test/re-test inconsistencies” and “illogical research results”; I would add my own experiences that directly contribute to these data issues, such as proprietary posturing regarding panel sourcing and composition, user-unfriendly questionnaire design, and the lack of disclosure concerning fraudulent or satisficing respondent behavior.  It is clearly these negative experiences that cry out for an industry-wide protocol by which both research buyers and providers can design, execute, and analyze their studies.  By setting design standards we are ensuring that everyone involved in the research process is playing from the same playbook while raising the bar on the transparency and accuracy of our data.

P&G is part of a consortium of companies – Microsoft, Unilever, General Mills, providers such as MarketTools, and others – participating in the TrueSample Quality Council.  To their credit, this council has developed the “Online Consumer Research Quality Guidelines” which delineates the standards buyers should request from research providers pertaining to three areas: respondent quality – e.g., ensuring that “each survey respondent completes a survey only once” – survey instrument quality, and “quality solution” (i.e., the provider’s ability to implement and deliver according to quality measures).

This quality initiative is a good start.  But like the first steps of all good things, it begs for enrichment.  Let’s enhance these quality guidelines by thinking more deeply as well as more broadly about research design.  Here are just two suggestions for ways to build on the Council’s quality standards:

• The weight of quality research design does not rest solely on the shoulders of research providers. 

Research buyers share in the burden of ensuring that quality standards are met.  This is why buyers need to be as up-to-date on design issues as research providers.  It would be good to see a reference to this level of partnership in the guidelines.

•  Quality standards for all modes of marketing research – not just online – are needed. 

The weak or clear absence of quality guidelines in the marketing research industry is striking.  While individual buying, provider, and professional organizations may espouse a set of rules, recommendations, or expectations, there is a general disconnect across the industry, manifested at times by disagreements along with chest-thumping for branded solutions (e.g., Net Promoter Score).

Gloeckler is absolutely right in calling for marketing researchers to “take the reins of research quality today.”  By continually developing and adopting quality design standards, the marketing research industry will produce consistently reliable data that will benefit buyers and providers alike.

* See “How Procter & Gamble worked to develop online data quality guidelines”, July 2011.