Sunday, May 31, 2009

The Dao of Strategic Assessment (22): Let's Look at the Data

Quality intelligence gathering defines the quality of strategic assessment.

May 29, 2009

Lebron James, and Naked Without Data

In today’s world, companies, entities, people and concepts are naked without data. Making business decisions or predicting trends using data analysis is perhaps the only way to survive the future. However, it is not the raw data, but how you analyze it that matters. In other words, asking the right questions. Like the ones that many Wall Street firms didn’t ask when betting the farm, the house and the cars on mortgage-backed securities.

A good reminder of how to use data smartly came this week, thanks to Lebron James and my beloved Cleveland Cavaliers. As a frame of reference, none of my three professional teams have ever won a championship in their sport in my lifetime. That would be zero wins in approximately 120 attempts. In fact during such time these teams have made it to the finals on just three occasions. Ouch!

Here comes the “naked without data” part:

Moments after the Cavs lost Game 4 in a best-of-seven series, one the “analysts” (not statisticians but rather former NBA stars who likely have not been trained in statistics) “revealed” how stacked the odds were against them coming back to win it. In the history of the NBA playoffs there have been 3-1 series leads exactly 187 times, and only eight times has a team come back to win. So the Cavs odds are only slightly better than 4 percent, right?

Wrong. What Kenny Smith –- one of my favorite entertainers and basketball analysts — forgets when dealing with stats as opposed to offering up his sensational, articulate and simple breakdown of the subtleties of the pro game is that his sample set of 187 prior events had a bias to it that makes it not directly relevant at all in assessing the Cavs odds.

Why? Well, across the 187 instances of 3-1 series leads we would expect that the majority of them were generated by superior teams. For instance if the Lakers, Celtics or Bulls at the height of their respective Magic, Bird and Jordan years held 3-1 leads in the first or second rounds we would EXPECT them to win those series NOT because of the numbers but because of their superiority! In the case of the Cavs and Magic, the Cavs were actually favored to win the series. If the Magic win the series it will be a well-earned upset. Not a shocker, since the Magic won almost as many games as the Cavs during the regular season and we are now in the semi-finals, where only very good teams remain to compete.

So Kenny was starting with a very faulty sample set, which in this case made the Cavs odds look horrendous. What would have been impressive was if Kenny Smith understood his flawed sample set thanks to his statistical and logical training in grade school and provided the number of instances when a favored team was behind 3-1 in the late rounds of the NBA playoffs. THAT would be darn useful. If you threw in home court advantage, since the Cavs would be playing two out of the three remaining games at home, it would be all the better.

Since there are a mere eight instances of teams overcoming those odds, Kenny might be able to do an easy case study to see what happened in those games, to find out if there was a pattern or not and then use his basketball knowledge to tell a truly powerful story. I’m just guessing, but maybe then the odds move to, say, 20 percent from 4 percent. Still far from ideal for Cavs fans, but the Cavs do have the league MVP on their squad in Lebron James. So that should matter. As it did in Game 5.

So how does this relate to the technology industry? Data is a critical weapon and a difference-maker. Real-time analysis of data is one of the reasons why a company like Google has been able to make itself the king of online advertising, because with the ability to quickly, constantly analyze the data, it can then match ads with pretty much any content page across the web.

Of course, no one is giving Kenny Smith a hard time for the statistical irrelevancy of his comment. But we can probably chalk that up to the eternal optimism of true fans.

Copyright 2009 GigaOm. All Rights Reserved.

No comments: