Who’s in your Top Ten? – LP demands push decile rankings
Deciles and quartiles have been around for a long time and have been used to rank everything from hereditary traits …
“You can’t eat IRR.” Many PE investors have read or heard this phrase somewhere. And as will be demonstrated below, IRR truly is a terrible measure of PE performance. This fact, combined with the fund managers’ ability to shop for the best statistically weak vintage year benchmarks with small sample sizes from many vendors which can allow over 60% of funds to claim top quartile status, and, the ability to further increase chances of these peak performance measure by adjusting the vintage year by a year forward or back, provide compelling arguments to come up with something much better NOW.
But this is not new. Leading MBA finance text books dedicate entire chapters to the shortcomings of the IRR measure in general (1) and academics specializing in PE performance measurement have made powerful arguments warning investors about the possible biases in any IRR-based performance assessment (2).
Still, investors continue to rely on this performance measure and one wonders why they are doing so. One of the reasons may be that we know that (in theory) IRR can be biased, but, UNTIL NOW, there is little evidence as to how substantial this bias really is. This study provides such evidence and the results are somewhat frightening. In collaboration with the PE Observatory at HEC Paris, we have been able to leverage a unique set of data on PE performance for over 5000 realized investments and 422 private equity funds to assess the magnitude of the IRR bias across a large and diversified sample of investments and funds.
Based on an assessment of the exact timing and amount of cash flows for each of these deals, we have been able to compare results from the standard IRR method to those of a simple but vastly superior performance measure, the Annualized Rate of Return (see box), which is calculated based on an investment’s total return multiple and its duration.
We have performed these calculations at the deal-level for 5038 realized PE deals made of the past three decades in Europe and North America (3). These deals had an average (median) IRR of 32% (28%) and an average (median) total return multiple or 3,4 (2,3). The average (median) duration of these deals is 3,9 (3,5) years (4). Drawn from a sample of realized investments made by established PE houses, these average performance values are not representative for the entire industry and indicate an upward biased sample. However this data is perfectly suitable to assess the magnitude of distortions attributable to the IRR method.
Comparing IRR and Annualized Rate of Return for the 5038 deals, we find that in 12,8% of this sample, the IRR measure deviates by more than 10% (in relative terms) from the more accurate Annualized Rate of Return measure. This implies that whenever you see a quoted deal IRR of, say 30%, there is a chance of one to eight that the real performance (based on the Annualized Rate of Return) is greater than 33% or less than 27%. Not exactly a negligible problem. For 3,7% of the deal sample (still 186 real-world investments), the distortion even exceeds ¼ of the stated IRR.
It is further interesting to observe that the IRR bias can go either way, as IRR may be above or below the Annualized Rate of Return. However we find that given the typical cash flow patterns in PE, it is much more common to find an upward bias. For 75% of the 645 deals with an IRR bias of greater than 10%, this bias was upward, creating an average upward bias of these 645 deals of +12% (in relative terms) over their actual performance as measured by the Annualized Rate of Return.
We shift our focus from deal-level performance measurement to fund-level performance measures. As the streams of cash flows for PE funds are generally longer and more irregular than for those of single investments, one would expect the distortions of the IRR performance measure to be even greater at the fund level. To verify this expectation empirically, we analyzed fund-level cash flows for 422 PE funds in our database. All these funds have made at least 10 (realized or unrealized) investments and we consider the final net-asset-values for the unrealized deals as a final cash flow.
Replicating the prior approach to calculate both the standard IRR and the Annualized Rate of Return for these 422 funds, we find indeed that the use of IRR as a performance measure is even more problematic at the fund level. We observe that for over 30% of the funds, IRR deviates from the Annualized Rate of Return by more than 10% (in relative terms). For 44 funds (or a good 10% of our sample), this distortion exceeds even 50%! In other words, if a fund reports a 25% IRR, then there is a 30% chance that its actual performance as captured by the Annualized Rate of Return is below 22,5% or above 27,5% … and with a 10% probability it will be below 15% or above 45%.
One can easily imagine how such difference may wrongly put a fund in a given performance quartile relative to its peers. The danger lies in the fact that these biases occur almost at random, largely driven by the cash flows that occur early in the life or a fund (or deal). If a fund’s performance turns out to be misrepresented by the IRR measure, the fund managers can hardly be blamed … in many cases they may not even be aware of it … and after all they only apply the most standard performance measure of the PE industry, wrongfully assuming that this would lead to a correct apples-to-apples comparison. But given the findings from our study, it may be high time to change this standard!
The IRR of Deal 1 (62%) is largely driven by the positive cash flow of 180 after only one year. According to the so-called ‘reinvestment assumption’, the standard IRR formula assumes the excess cash flow accumulated to that date (180-120=60) to be reinvested at the IRR realized to this date (i.e. 50%). This hypothetical investment, combined with the actual realizations in the following years pushes the overall IRR of the deal to 62%, despite a total return multiple of “only” 1,8x over a 5 year holding period. One would expect an IRR of 62% over 5 years to generate a multiple of (1+0,62)^5=11,1x ! Clearly, the capital was not fully invested over 5 years. The deal’s duration, i.e. the difference between the capital weighted average date of all realizations and the capital weighted average date of all investments, for deal 1 is only 1,38 years. But even if we consider the duration instead of the simple holding period, the IRR returns still do not match as (1+0,62)^1,38=1,94x, which is different from the multiple of 1,8x. The simple but powerful solution is to ignore the IRR measure altogether, as it is simply unsuitable for PE type cash flows and to calculate the Annualized Rate of Return based on multiple of investment and duration as follows:
Annualized Rate of Return = (Multiple^(1/Duration in years))-1
This Annualized Rate of Return, 53% in our case, is an unbiased and accurate measure of performance at the deal-level. We see that deal one has an upward biased IRR of 9% in absolute terms, i.e. 17% relative to the Annualized Rate of Return of 53%. If we combine the two deals in this example, the difference between IRR (51%) and Annualized Rate of Return (37%) becomes even larger, as the 14% difference in absolute terms corresponds to over 1/3rd of the Annualized Rate of Return. As our empirical analysis of actual data from 5038 deals and 422 fund shows, such performance distortions due to the use of the inappropriate IRR measure are much more frequent than one would have expected.
(1) See, for example, Brealey, R. and S. Myers, 2001, Principles of Corporate Finance, Sixth Edition, McGraw Hill.
(2) See Gottschalg&Phalippou (2007) Truth About Private Equity Performance, Harvard Business Review, Dec 01, 2007 (http://hbr.org/2007/12/the-truth-about-private-equity-performance/ar/1) or Phalippou, “The Hazards of Using IRR to Measure Performance: The Case of Private Equity: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1111796.
(3) To eliminate extreme cases, we excluded deals with a duration of less than 6 months and with an IRR of less than -50% or more than 200%. For these cases, the bias of the IRR methods would be even greater.
(4) The observation that 50% of our deals have a duration of more than 5 years or less than 2,3 years illustrates that it is not an option to simply use the Return Multiple alone as performance indicator. After all, it makes a huge difference if one “doubles the money” in two or in five years…
An article by Oliver Gottschalg, Administrator of Research, PERACS