Member-only story
Are Survey Respondents Honest?
A question I’m frequently asked about consumer opinion research addresses whether we should “trust” the answers generated. We are all likely familiar with the quote: “lies, damned lies, and statistics” — popularized by Mark Twain and (apparently erroneously) attributed to Benjamin Disraeli.
The above chart breaks down the answers given by respondents to the question “For what kind of organization do you work for pay or profit each week?” to those given by the same respondents to the prior question “Which of the following categories best describes your current employment situation?”
You will note that five of the people who previously answered “Employed, working full time” or “Employed, working part-time” to describe their current employment situation now answer “None, I’m not currently working” to describe the kind of enterprise they are working for. That’s a 5/43≈12% error rate.
To understand how this happens, we need to understand the survey process. We are no-longer in the era of George Gallup standing on a street corner in Princeton and interviewing passers by.
In the 21st Century, many consumer surveys are delivered via an online platform and…
