Trust Analysts Blindly - Lose Money

Posted on May 5, 2008
Filed Under BS-Free Advice |

In recent posts ( and here) I’ve made it clear that I apply the “trust no one” mantra to many things, including research firms and well-connected individuals.

This, of course, is not to say that I don’t listen to what “experts” have to say. “Expert opinions” can be extremely valuable when used appropriately.

Unfortunately, some “experts” aren’t “experts” and implicitly trusting that every statement or prediction that comes from somebody you perceive as being an “expert” is accurate or wise is not the best strategy.

I try as much as possible to rely on common sense and logic to evaluate situations and even if the results prove my critical thinking to be wrong, I much prefer to make bad decisions on my own as opposed to making bad decisions based on third-party information I accepted blindly.



Another amusing example of just how foolish it can be to rely on “experts” was provided by Citi Internet Analyst Mark Mahaney. On February 2, the day after Microsoft announced its bid for Yahoo, that there was a 60% chance that Microsoft would wind up acquiring Yahoo, noting a 20% probability that the original bid would be accepted and a 40% probability that a higher bid would be accepted.

On April 25, he upped the total probability that Microsoft would acquire Yahoo to 90%.

On April 28, in a TechCrunch interview with Mahaney that Mahaney did not think Microsoft was likely to simply walk away.

The interview, of course, demonstrates that Mahaney, as to be expected, based his predictions on certain assumptions and the best knowledge he had at the time. He appropriately admits when he is unable to answer a question, at two points stating “Sorry Michael, I don’t have a good opinion on that” and “That’s a good question, Erick. I don’t have a well-formed opinion on that.”

Fast forward to today.

Microsoft has walked away and although there remains the possibility that the move will eventually lead to a scenario in which Microsoft and Yahoo get together, Mahaney now says a Microsoft-Yahoo deal is “still 15% likely” the TechCrunch headline.

If that isn’t headline spin, I don’t know what is.

Of course, the purpose of this post is not to criticize Mahaney, who may or may not be worth listening to (I’m not familiar enough with him to express an educated opinion). The purpose of this post is to reiterate the same point I made about research firms:

To be fair, research firms like Forrester are tasked with an almost impossible task - predict the future. Nobody can. At best, educated speculation can be provided.

Even then, significant differences are bound to exist, highlighting the fact that even educated speculation is still speculation.

Predicting the future is not possible, and the vast majority of the time, numeric probabilities coming from non-mathematical analyses should be avoided like the plague. After all, they try to put a quantitative spin on a qualitative evaluation - something that just doesn’t work. Only numbers derived from a fundamentally-sound mathematical analysis should be (potentially) taken seriously.

While I always chuckle when guys like Mahaney give such probabilities and then are forced to change them dramatically as is the case here, the real problem, of course, is that many people actually give significant weight to these numbers and consider them credible because they trust the expertise of the person who came up with them.

That’s precisely why research firms and analysts create them in the first place - people want a simple piece of information that requires little thinking to digest.

This is highlighted by Michael Arrington’s rationale for interviewing Mahaney:

Given how consistently right Mahaney has been in his analysis, Erick and I reached out to him today to pick his brain on the deal.

And is further demonstrated by Arrington’s statement that, after being told by Mahaney that (if you’re Ballmer) “You can’t walk away”:

There are very few people who understand this situation as well as Mahaney. If he says Microsoft needs Yahoo, I’m not going to disagree.

This is emblematic of the challenge that exists when it comes to research firms and analysts. As I stated:

The big problem, of course, is that research firms are often looked upon as authoritative sources. Many investors are easily seduced by the projections that are produced by research firms and if a research firm predicts that the podcasting market will be worth $1 trillion by 2012, there are some individuals who can’t be convinced otherwise.

Implicit, unquestioning trust is never good and reliance on the belief that the past predictions of a particular firm or individual have been good ignores a simple, yet wise, adage: “Past performance is not necessarily indicative of future results.”

Had you made trading decisions on MSFT and YHOO, for instance, based largely on Mahaney’s predictions, the fact that you found his past predictions to have been “solid” would probably not provide much comfort right now given your potential losses - despite the glimmer of hope that there’s “still” a 15% chance of a deal.

Perhaps that most important take-away is that, even if you have established that a person or business has “expertise,” the type of information that is most appealing, such as predictions (i.e. “there is a 90% chance that Yahoo will be bought by Microsoft” or “the podcasting market will be worth $50 billion by 2012″), is exactly the type of information that is still not very reliable no matter who it is coming from.

In general, I’ve found real experts to be capable of predicting general trends quite well but generally incapable of reliably predicting the magnitudes of those trends and the timeframes in which they will be realized. And when it comes to the outcomes of specific short-term events, in particular, I will say little more than caveat emptor.



Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • del.icio.us
  • Netvouz
  • ThisNext
  • Reddit
  • SphereIt
  • Slashdot
  • StumbleUpon
  • Technorati
Print This Post Print This Post

Comments

4 Responses to “Trust Analysts Blindly - Lose Money”

  1. Laurence on May 10th, 2008 4:12 am

    Economists, experts, analysists…. They’re all the same.

    Enough of them predict equally diverse and varied outcomes and generally one of them has a decent chance of predicting spot on.

    I guess they just hope they are that person.

  2. Paul on May 10th, 2008 11:31 am

    While I would never shy from throwing stones at investment bankers just for the fun of it, I don’t think you are entirely justified in this case.

    You seem to have made the mistake that many people make; when they hear the possibility of X occurring is 90%, they think that means 100%, and when X does not occur, they cry that the initial estimate was incorrect. Had he said it had 100% chance of occurring then I would agree with you. But many were surprised at Yahoo’s refusal to accept the higher offer, and in hindsight his 60-90% probability range was not unreasonable; the outcome just fell in the 40%-10% range of unfavorable outcomes.

    It’s like poker - you will win over time by playing the odds, but you will still lose hands occasionally even when the odds are significantly in your favor.

    Cheers
    Paul

  3. Others Start to Ask “What is Mark Mahaney Smoking”? : The Drama 2.0 Show on May 19th, 2008 10:28 pm

    […] my recent post about the absurdity of the predictions and projections frequently promulgated by research firms and […]

  4. Silicon Valley’s Circle Jerk Finds a Friend on Wall Street : The Drama 2.0 Show on August 11th, 2008 4:31 pm

    […] my past criticism of “analysts,” I’ve used the “analyses” provided by Citigroup’s Internet analyst Mark Mahaney as examples […]

Leave a Reply

Sponsors

Lead Sponsor

Drama 2.0's indulgent jet-set lifestyle is supported by Drama 2.0 Show lead sponsor . For more information about MySites, click here.

About

Drama 2.0 spikes the Web 2.0 kool aid by providing critical analyses of Web 2.0, its people, its startups and its impact on the world of media. Other topics are explored when Drama 2.0 has been drinking too much 1975 Dom Perignon. Read more about the Internet's version of Keyser Söze here.

Ads


Subscribe