Rugby: The Wisdom of Crowds and Super 14 Picks

In his book, The Wisdom of Crowds, James Surowiecki argues that the the aggregation of information in groups results in better decisions than could be made by a single member of the group.  I have to admit that I haven’t read the book, yet, but I thought I would test the wisdom of rugby followers who participated in the Sky Sport Virtual Rugby game that ran for the 2009 season.

The hypothesis would be that the aggregation of picks would tend to be pretty close to actual results.  However, we need to look at the criteria that separate wise crowds from irrational crowds, as set out in the Wikipedia entry on the book and subject:

Diversity of Opinion: each person participating will tend to have private information, which will (primarily) be their own eccentric interpretation of known facts

Independence: Not so sure about this one, since I am sure that some people (e.g. me) were influenced by the views of others as reflected in the level of support for particular results, especially where they don’t have any strong eccentric interpretation of their own

Decentralization: People do specialize and draw on local knowledge, but the other side of this coin is that they may well support an outcome favouring their local side irrespective of the facts and experience

Aggregation: Sky Sport Virtual Rugby provides the mechanism for turning private judgments into a collective decision

On this basis, the elements for a wise crowd appear to be there in Virtual Rugby.

Bad judgments can result when the crowd is:

Too homogenous: if there is not sufficient diversity within a crowd – in the case of Virtual Rugby it is likely that there is too great a focus on New Zealand teams, since the majority of the participants are (I assume) New Zealanders

Too centralized: I don’t think this is a problem since there were about 122,000 participants

Too divided: there is ample scope to share information through media reporting and commenting, and through other forms of information exchange, so people can choose what information they need

Too imitative: Choices are visible, in the aggregate, which could lead people to reflect the majority view

Too emotional: Well of course, clearly there will be biases, hearts will dominate over heads, Wellingtonians will support the Hurricanes, despite the evidence

Anyway, that’s enough about the theory, what abut the analysis.

Virtual Rugby is played by making a prediction for each game in each round of the Super 14, with choices being one side or the other to win by either 12 points or less, or more than 12 points, or a draw.

For the purposes of the initial analysis, I looked only at the outcome of the game and regarded a crowd prediction as successful if the actual outcome reflected the views of the highest proportion of players.  This includes predictions of both wins and losses.

On this basis, the crowd had a 69% success rate over the 14 rounds of the 2009 Super 14 season, i.e. it got slightly better than 2 out of 3 right.  Is this good?  I would have thought that it’s not so good.

A more detailed analysis by team and country is interesting:

Bar chart of success rate of crowd picks in Virtual Rugby, by team and country

Bar chart of success rate of crowd picks in Virtual Rugby, by team and country

The most predictable teams tended to be those that ended up in the bottom third of the competition, i.e. they were predicted to lose most of the time.  Not unrelated, given that they had three teams in the bottom third, the crowd did best with South African sides (75%).  It did worst with New Zealand sides, which could well reflect the emotional attachment to local teams.  The most unpredictable teams were the Crusaders and Waratahs, which can possibly be explained by their changing fortunes through the competition.

What about picking winners. The analysis shows that the crowd got it right for 80% or more of the time for the Hurricanes, Stormers and Blues, but less than half the time for the Reds, while crowd opinion was definitely against the Cheetahs.  It could also mean that Hurricanes supporters always pick their team to win.


When picking losses it was a bit of a mixed bag.  The crowd got it right for the Sharks, Bulls and Lions, but was nowhere near it for the Hurricanes and Waratahs.  I think that this supports the view that supporters of these teams let their emotion cloud their judgment.


Did the crowd learn anything as the season went on?  Not sure about that, although there is some evidence to support the proposition in that for the last three rounds at least the outcomes were well-predicted.


So what does this all mean, apart from a suggestion I have too much time on my hands?  What I think it means is that the crowd that does Virtual Rugby is reasonably well-informed but is not too wise. It also means that if you want to win at Virtual Rugby, don’t follow the crowd.

Further analysis comparing the crowd verdicts to the TAB betting odds would be very interesting.


5 Responses to Rugby: The Wisdom of Crowds and Super 14 Picks

  1. Matt says:

    Hey Hugh (only just ran into this blog). Related to this post, you might find Ebbsfleet Football club in the doco “Us Now” interesting. It also reminds me a bit of prediction markets, such as NZ’s (essentially the TAB for non sporting NZ stuff). I suspect that organisations that harness at least the predictive power of the wisdom of the crowds (forgetting the other wisdom you can harness) will have a huge advantage over those who don’t.

  2. Matt says:

    …while also understanding its limitations

  3. Clifton says:

    Hi Hugh,

    Matt directed me to you post.

    One year I came 1st equal in the Virtual Super 12 competition at work. I picked the results at random, based on the random numbers in the back of the Eton’s Tables! I can’t remember the exact algorithm. The amazing thing was that the other 1st place getter was probably the most knowledgeable rugby head in the team.

    I put it down to the teams being very even that year.

  4. Juan Diskin says:

    Rugby World Cup packages to tour with other spectators and players to take part in the ultimate festival of rugby will appeal to followers of the game. Every four years the World Cup tournament comes around. This is like a climax that aficionados will patiently wait for. It builds up over years, months, weeks, days, and hours until the moment of the finals kick-off. Many people will save assiduously for years o be part of a tour.A party of rugby followers is likely to be a diverse group. The game knows no social, physical or psychological barriers. Quick small people have become legends of the game. Giants, both tall and wide, are central to the scrum and normal men are needed on the flanks. It is known as the gentlemen’s game for hooligans and both types are likely to be represented in any game.”

    Most recently released posting on our blog site

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: