Why did Gartner fail to spot 77 million devices and what it means for prediction methodologies ..

In a recent analysis, Gartner failed to spot 77 million devices from emerging Indian and Chinese manufacturers

Unverified sources in the Guardian article point to a leaning towards: preserve the growth rates; to hell with the actual numbers. and . Thing is, real executives got real compensation based on our numbers ..

Then again, over at Fortune, Philip Elmer-DeWitt was contacted by someone who says they used to work at Gartner, and analysed the PC market – “but the methodolgy is the same for phones”. (You can find their comment below the main article, timed at 11.29; the comments run in reverse chronological order.)

S/he says, inter alia:

“So, in 3Q98, I analyzed the “choke points,” those parts of the supply chain where the channel narrowed enough to get a definitive count. At the time, it was OS, processor, graphics, and hard drive. As I recall, I found 20 million processors with no homes. The market at the time was about 100 million, so this was a 20% discrepancy.

“The process that ensued was a marvel of obfuscation. The leader of the Tracker team figured out a way to rationalize away all the extra units (e.g., multiprocessor servers, inventory, speculation, etc.). It was politically impossible to force the extra units on the regions because it would introduce gross distortions to the historical trends.

So, the mantra became, preserve the growth rates; to hell with the actual numbers. Even the growth rates are fiction. The fudge is in the “others” category, which is used as a plug to make the numbers work out. In fairness, we did do survey work, calling around, and attending white box conferences and venues to try to get a feel for that market, but in the end, the process was political. I used to tell customers which parts of the data they could trust, essentially the major vendors by form factor and region. The rest was garbage.

The industry itself was aware of these issues, but agreed to maintain the fiction because it was convenient. Most vendors kept their own numbers, but referred to IDC for public purposes. Thing is, real executives got real compensation based on our numbers. There were other games played, but that’s for another time.”

This is familiar territory. The analyst forecasts for Location based services in early 2000s are now laughable

“Successful plays in mobile data will ultimately exploit that which makes wireless unique. There is an element inherent to wireless that wired networks, by definition, will never possess – untethered mobility. Mobility, and hence location, is therefore a critical attribute to be exploited by all involved in the wireless value chain,” said Cliff Raskind, Sr. Industry Analyst with Strategy Analytics.

My view is:

Analysts can work on incremental trends

These are nice to ‘model’

These models have their accompanying assumptions

Mainly to protect the analysts anatomy ..

But ..

Disruptive trends are not incremental ..

The cannot be modelled by applying an incremental formula to a historical trend

Thats why LBS trends were so wrong(Google, foursquare etc are the big winners for LBS) and thats why the unconfirmed report in the Guardian “So, the mantra became, preserve the growth rates; to hell with the actual numbers” about Gartner’s methodology is so interesting .. more so because much of the industry analysis is geared towards what the industry(in this case Operators) want to hear

But its not only Gartner ..

The process which analysts use to ‘predict’ works only under incremental conditions where it has limited utility for sure, but the methodology fails to detect disruptive trends

See also: With the warp speeds of android, can Klingons win or do we need faster features?

Image: rajkumar1220 on Flickr

Comments

  1. Excellent post, thanks for wrapping it up beautifully. The uses of cell phones is increasing in India, especially rural India. I came across this PR from Gartner old but interesting.