The story behind the survey.

On average this household gets one or two surveys per week. That is more than the average household but we encourage it. We answer interactive voice response (IVR) calls because they are fun and we enjoy confusing them. We are more cautious with live callers and we listen carefully to the technique and the mix of questions. We are particularly interested in any bias we find or any trick questions. We are probably more truthful on Internet panels in which we participate. It has been a while since we have written survey questionnaires or trained interviewers.

But we wish we could have written the survey for a friend that came to our e-mail in-box in the last week. We sent him an e-mail after answering the questionnaire and suggested that he should just ask his family and friends what he was trying to find out. The problem was that he had used one of the new Internet do-it-yourself surveys and it not only lacked subtlety but it would hardly provide an impartial answer from the people being interviewed. He was trying to answer a question that needed to be answered by people who cared.

At the same time another survey was going on in a local city ward where there will be a by-election to replace a councillor who resigned. This was an IVR telephone survey to see what the local identification was for a potential candidate. Since these are just beauty contests anyway, this potential candidate wanted to see the identification he had in the ward. He has now said he will run. Mind you, the usual group of suspects and also-rans have already declared and he is not exactly joining a very sparkling array of talent.

And that was just two surveys that were a waste of time.

Another survey we were looking at recently was the one being pushed to the media by the Broadbent Institute. Being no fan of Ed Broadbent, nor he of yours truly, we tend to disparage anything from that institute as we both treat studies sponsored by the right wing Fraser Institute. The problem with the recent Broadbent Institute study and with many Fraser Institutes studies is the bias these people have already expressed before bringing out surveys to support their cause.

The first requirement for any believable survey is the impartiality of the questionnaire and the interviewers. The second is the sampling technique and understanding you get what you pay for. The third requirement is that you have to know the people interpreting the raw data are also impartial. And finally, you have to bear in mind that a survey is about what people thought—not what they are necessarily going to do.


Copyright 2015 © Peter Lowry

Complaints, comments, criticisms and compliments can be sent to

Comments are closed.