The Wisdom of Crowds

I have only a vague idea where I'm going here, so bear with me. Or not. Don't let the browser whack you on the way out ;-)

An individual ant is not very smart. It probably has fewer neurons than comes in a box of tinker-toys. An ant colony, on the other hand, is pretty clever. This type of "collective intelligence" is an area of active research on many fronts. Its effects aren't readily apparent, unless we know what we're looking for. Or rather, how we are looking.

We aren't used to seeing systems. I blame language. Case in point: did you notice the structure of that last sentence? Subject (I) action (blame) and object (language.) Language tends to put things in a straight line. One thing affecting or being affected by another. If "a" then "b". Most of our human wisdom is ascribed to knowing either what "a" to use to get the "b" we want or why "b" happened based on some "a". And if "b" happens there's sure to be some "a" to blame for it.

But systems are more subtle. Based on the "state" a system is in, sometimes "a" can cause "b" and sometimes it may cause "c". Sometimes the causes of "b" are because of multiple contributing factors and sometimes it's because of something that predates "a", leaving "a" holding a bag it would not have held otherwise.

We deal with systems everyday. Even our thoughts are part of systems of identity, beliefs and values that preselect the nature of information we receive and the range of behaviors available to us. Most of this works outside of our conscious awareness. Systems have behaviors very different from the individuals or components that make them up. They process information differently, often with different priorities and towards different ends. A cell has very different functions than the organ to which it is a part.

This is all well and good, and if you are interested in how systems thinking may lead you to different insights about things than linear thinking, I recommend this primer.

But that's not the real topic (I just realized ^^) The real topic is measuring "public" opinion.

Most polls seem to indicate that the "wisdom of crowds" is an oxymoron. But there are some fascinating counter-examples. Theoretically a collection of people has more experience, sensory data and aggregate brain power than a single individual. They may come up with things a single mind could not. In theory. The book, The Wisdom of Crowds, explores some conditions which permit this to happen. It's not a given though, as our election process demonstrates. But there are situations where group-think offers some interesting data.

Predication markets, for example. I was amazed to discover the number of these places where people are betting real (and play) money on predicting outcomes of various events. Like the election. Note that these are illegal in the US. So you can go offshore to places like Betfair, in England.

I'm not advocating this as a way of supplementing your income (although it may pay to be psychic), but since real money is at stake, there's probably more serious thought going into issues here than the *cough*stupid*cough* election polls.

For example, the day after Sarah Palin's disastrous (but hilarious) interview, the bookies odds for McCain winning went *way* down. Newsfutures is an innovative approach to predictive markets as well, you bet real money, but the results go to charity.

Collective intelligence is a fascinating topic for me. For you hackers in the lot who want a hands on start in playing with it, try this.

Well, that didn't go anywhere near the direction I thought it would. Guess there are some *liabilities* to non-linear thinking as well ;-)