Yesterday, in my article about activist investors, I referred to a Yale academic who said that one hedge fund index showed that activists had sluggish performance, but then said that this index series is widely known to be troublesome without any explanation for what makes them problematic.

Hedge fund ‘indexes’ are well known to overstate performance and understate risk, making them appear more appealing to investors than they really are.  The first thing that you have to know is that a hedge fund index isn’t an index at all.

An index measures securities in some way.  The S&P 500 tracks the value of 500 stocks, also picked by a committee, based on their market value.  The Barclays Aggregate Bond Index does the same thing for taxable, investment grade bonds, the MSCI EAFE tracks non-US stocks and the CRB Commodity Index follows commodity futures.

Unlike stocks, bonds or commodities, hedge funds aren’t assets – they are funds operated by managers.  Hedge fund indexes are categories of actual funds that reflect peer performance.

Let’s say that there are 10 hedge funds in the world: five are equity market neutral and five do merger arbitrage deals.  We could create two hedge fund indexes (or categories, really), one that tracks each of the two styles.

At the end of a year (or some period), we could see how the two categories of funds fared.  Even though they aren’t indexes, you can use them as benchmarks because it tells how other merger arbitrage funds did to give you a sense of how your fund performed relative to its peers.

This can be very useful information, especially if the categories are well built.  If the categories aren’t well constructed, the information about your relative performance isn’t so helpful.

Although there are a lot of variants, the three basic construction problems are known as selection bias, survivorship bias and backfill bias.

Selection bias refers to the fact that reporting to the hedge fund databases that create these category returns is voluntary.

In my example, I assumed there were five equity market neutral funds.  What if there were really six funds, but one had such bad performance that they didn’t want to report their numbers?  That’s right, the category would look a lot better without the worst performing fund.

Backfill bias is the flip side of that same coin.  Let’s say that you started a merger arbitrage fund but didn’t feel like reporting it because you didn’t know how you would really do in the market.

Good news, you blow the doors off of everyone else, so after the race has run, you report your results.  The database adds your ‘instant history’ to the category and all of a sudden it looks a lot better with the benefit of 20/20 hindsight.

The last problem, survivorship bias, is related to funds that had to close operations for one reason or another (although if it’s going well, you’re more likely to stay open).

Let’s say that you started a fund 10 years ago, but closed it after the first five because performance was lackluster.  If a fund category is created today by looking at the history of all of the current funds, it won’t catch a fund that was closed five years ago because of bad performance.

It’s been estimated that hedge fund indexes overstate performance by three to five percent per year because of these factors.  Although there aren’t solid numbers, you can imagine how cutting out all of the losers and over-emphasizing the winners would also understate the risk in these indexes.

Because we don’t invest in hedge funds, these biases don’t represent a particular problem for us – although it’s partly because of these issues that we don’t invest in hedge funds.