The Truth about Truth in Survey Research

Posted on 14. Jun, 2009 by in Customer Research

We Can’t Trust Survey Respondents?  You’re Kidding Me!

A journal that I (used to) respect, the Journal of Consumer Research is printing an article in its August issue, that ‘reveals’ a truth that most of us who can remember black and white television were taught in marketing research 101.  This astounding fact, (hold on to your seats) is that one can’t always trust the results of surveys or opinion polls, because not all respondents tell the truth!  Now perhaps the editors of the JCR simply thought it was about time to remind readers and the marketing community of this truism, but they surely can’t consider it a significant new finding!  When researchers, perhaps like the author of this article, struggle to find topics to research, sometimes absurd topics emerge.  Apparently  this article’s author decided to re-investigate the veracity of marketing surveys and polls, but the article certainly doesn’t extend the state of current knowledge!

That’s Why Dichter Created “Projective Questions”

Social scientists, at least those candid enough to admit to the failings of their profession, have always recognized that respondents’ answers to survey questions need to be carefully interpreted.  Some respondents–it is hoped–tell the truth (as far as they know or understand it).  But other respondents lie or fabricate answers either to confuse, mislead, impress or please the surveyor or to safeguard their own image.  Ernest Dichter, father of motivation research in marketing, applied principles of Freudian psychology to create the “projective question”.  This questioning technique allows respondents to project into a relatively ambiguous question, their own (sometimes impure) feelings and beliefs without sullying their own persona – much the way one sees butterflies and witches in Rorschach inkblot tests

Inherent Bias

The reality of any data gathering in marketing is always that respondents who fabricate their responses will unfavorably influence (bias) the survey results.  So, in addition to using options like Dichter’s projective question, smart marketing researchers have built fail-safes into their questionnaires.  First, researchers will attempt to somewhat mask their intention for asking questions.  This neutralizes those out to mislead or please the surveyor.  Next, researchers use procedures to identify or trap those telling lies so that their answers can be excluded from the final survey tally.  (The systematic identification and exclusion of “yea-sayers” and “nay-sayers” and other “liars” is probably [unfortunately] less practiced than it once was.)  But most of all, good researchers know the limitations of survey research and therefore are disciplined in the amount and type of information for which they ask respondents.  And finally, good researchers warn that even the most accurately collected survey responses are still only right up to some probability.

Caveat Cluens (Client)

These realities impose serious mandates on the responsible survey researcher.  First, one should never ask what one can’t logically expect respondents to honestly reveal.  Second, questions need to be strategically framed and worded.  Third, responses must be carefully and wisely interpreted.  And finally, fourth, the limitations of the survey research process should be fully explained to the client and anyone else hoping to utilize the information gathered.

 

(The article referenced is: Ashok K. Lalwani. “The Distinct Influence of Cognitive Busyness and Need for Closure on Cultural Differences in Socially Desirable Responding.” Journal of Consumer Research: August 2009.)

Tags: ,

Leave a Reply