Friday, December 11, 2009

We're all on Family Feud: The abuse of opinion polls

Back when I was in the single digits and living in Virginia, there wasn't much on TV. No cable yet. You end up watching whatever you can. I remember seeing Richard Dawson (a guy who got his start in a sitcom about a concentration camp) smooching away in his inimitable style.

Family Feud was an awful, awful show. The best/worst part was how the game worked. You were asked questions and were supposed to guess what "average polled Americans" said about things. The famous line "The survey SAYS...", followed by the answer flipping down from the giant Whitman's sampler chocolate box display. Teams lived or died not by their knowledge, but by their ability to guess what other people thought. This has been famously parodied before, because it's a pretty silly and fatuous game idea.

The Internet has given us many great things. Lots of entertainment. Video and music on demand, sometimes even legally. It has heightened communication between individuals and communities. It has also provided many new ways for people to provide feedback and opinion, frequently anonymously.

Naturally, that ability to comment at any time, on any subject, regardless of whether we know what we're talking about or not, has some downsides.

One is the rise of a culture where everyone thinks their opinion matters and deserves to be heard. This is followed by Big News feeding that for sensationalistic purposes.

Yesterday I happened to be out to lunch with a friend and out of the corner of my eye I caught CNN's coverage of the health care reform bill. They had one of their instant polls up about "Whether you thought the bill was going to increase the federal deficit" [emphasis mine].

Of course, the numbers were shocking. Unsurprisingly, something like 75% of the respondents said they thought the bill was going to increase the deficit. Were one passively watching and not paying careful attention, one would get a strong impression the deficit would increase under the bill. Much of this had to do with the typically clipped and simplified presentation of the polling data, which downplayed the "we asked a bunch of dopes on the street" aspect.

This is not what the poll was saying or proving. Nor was it what the poll was for.

Fundamentally, the idea of collecting and presenting this information is just silly. This is largely inessential data for everyone except a small number politicos watching the polls. Even then, what they should do with this data is figure out how to better educate people about the issue, or perhaps understand whether people are actually educated about the situation. Instead, these herd mentality numbers are used as if they are some sort of important facts in the "ongoing debate". As if what people think about the bill and its possible impacts are more important than the actual bill and its impact

This really irks me. It's dangerous, lazy thinking. The bill itself isn't even "finished" yet, with many key provisions controlling cost - things like "who gets covered?" and "how long do they get covered?" and "what are they covered for?" - not yet locked down.

Most of the people actually voting on the bill haven't even read it, so they don't even know for sure. Each analysis of the bill's costs makes a number of different assumptions, so none of those can be definitive, either. The end result is the knowledgeable folks can provide a more informed opinion, but even they cannot definitively say what the bill's impact on the deficit will be.

The average polled person has nowhere near enough information to weigh in on this issue. News presenting these "opinion polls" as facts do everyone a disservice. This data could be used as a lead-in to a detailed discussion of what the bill actually proposes, to either bolster or refute the opinion polls. But that never happens.

By emphasizing and focusing on what uninformed people think or feel might happen, we further destroy our ability to have reasoned, proper debates and turn more towards charisma and force of personality and presentation. We turn away from "what is true" to "what people will believe". This encourages the kind of knee-jerk demagoguery where outright lies get traction because they "feel true".

Once it's OK to respect and discuss what people "feel is true", rather than what is actually true, you end up wasting time addressing and relentlessly debunking fringe positions already disproven: Vaccinations cause autism. Obama's not eligible to be president. Climate change isn't real. The list goes on.

And once there's enough of these topics circulating in the news, even more people start to pick up the thought virus. They figure "well, people are talking about it, where there's smoke there's fire, it must be true or something". And the cycle becomes ever-harder to break.

Stephen Colbert may be loving it. I'm not.

Put another way, only 40% of people in the USA say they believe in evolution. Think about that, and then ask yourself if you really want the carefully considered, highly informed opinions of everyone else driving the agenda for national debate on complex subjects like health care reform.

At this point, we're all contestants on Family Feud. What you know doesn't matter. How well you understand how little everyone else knows does.

The survey says? Bzzzzzzzzzzzzz.

1 comment:

Mark Johnson said...

Perception is reality.