Building with uncertain numbers

Markowitz gave the world a formula. Put in what each asset is expected to return, how much each one swings, and how they move together, and the formula gives back the best portfolio you can build with them. It is one of the cleanest ideas in finance and it won a Nobel Prize.

There is one problem with it, and this chapter is about that problem. The formula assumes you know the three inputs. You do not. Nobody does. The expected return of a stock market is a number you estimate from the past, and the past is short and noisy. Anyone who has watched a market for twenty years knows how little twenty years tells you about the next ten.

Through the seventies and eighties, big institutions bought optimization software, fed it their data, looked at the answer, and quietly stopped using it. The answers were absurd. Everything in two assets and nothing in the rest, and a completely different portfolio if you changed one input by a hair. The four papers here explain why that happened, put a number on it, propose the fix that every serious allocator now uses, and then ask the uncomfortable question of whether the whole exercise beats splitting your money into equal parts.

Michaud, 1989

Richard Michaud worked in quantitative research on Wall Street and had seen the software fail from the inside. His paper said out loud what practitioners had learned in private. An optimizer, he wrote, is an estimation-error maximizer.

The mechanics are simple once you see them. The formula treats every input as the truth. If your data happens to overstate the return of one asset, or understate its risk, the optimizer does not know that. It sees an asset that looks better than the others and loads up on it. The assets whose numbers flatter them most are exactly the ones that get the biggest weights. The machine is built to find the errors in your data and bet the portfolio on them.

That is why the answers were extreme, and why they changed so much from one run to the next. Nothing was wrong with Markowitz. Michaud was clear about that. The framework was right. What was wrong was feeding it raw historical averages and believing the output.

What I take from it is a rule I apply to every optimized allocation anyone shows me, including my own. If the output surprises you, it is a report on your inputs. Ask which numbers drove the result, and move each of them a little. If the portfolio swings, the optimization was decoration.

Chopra and Ziemba, 1993

Michaud had made the diagnosis. Four years later Vijay Chopra and William Ziemba measured it, in six pages. They took real data on stocks, bonds and cash, built the best portfolio for an investor with an ordinary appetite for risk, and then deliberately got each input wrong by a small amount, one at a time, to see how much the investor lost.

The result set the priorities for everything that came after. Errors in expected returns did about eleven times as much damage as errors in volatility, and about twenty times as much as errors in the correlations. For an investor with a big appetite for risk the gap was wider still.

Read that again, because it is the most useful number in this chapter. Of the three things the formula asks for, the one that hurts most when you get it wrong is also the one you are worst at estimating. Expected returns are the thing you cannot know. Volatility and correlation you can estimate reasonably well from a few years of data, and they hurt far less when you are a little off.

So the paper tells you where to spend your effort. Measure risk carefully, because you can. Treat your return forecasts with suspicion, because you must. And never let a decision hinge on an expected return, because a change of one percent in that number, well inside the range of honest disagreement, can reverse the decision.

Black and Litterman, 1992

The fix came from Goldman Sachs, and it came from a real failure. In the late eighties the firm’s fixed income research group was building global bond portfolios, and it tried the obvious thing: take the historical returns of the world’s bond markets, put them in the optimizer, and hold what it says. The optimizer said to make enormous bets. Huge long positions in a couple of markets, huge short positions in others, and a different set of enormous bets the moment anyone touched the inputs. No client could hold such a portfolio and no trader believed in it. The paper’s opening pages are a description of that problem.

Fischer Black, already famous for the option pricing formula, and Robert Litterman, who ran the research group, took it apart and asked a different question. Instead of asking what returns the past shows, they asked what returns would justify the portfolio the world already holds. Every asset has a market weight. If investors are roughly sensible, those weights tell you what the market expects each asset to earn. Run the optimizer backwards, from weights to returns, and you get a set of expected returns that will produce, by construction, the market portfolio. They called it equilibrium.

Then they let the investor add views. If you believe German bonds will beat French bonds by half a percent, you say so, and you say how confident you are. The model blends your view with the equilibrium, and the portfolio moves away from the market only where you have a view, and only as far as your confidence goes. No views, and you hold the market. Weak views, small bets. Strong views, larger ones, and the model makes you write down why.

What survived is less the formula than the grammar. Nearly every large allocator today runs some version of it, whether or not they run the equations. Start from the market. Every deviation has to name its reason, its size and its confidence. That discipline is worth more than the mathematics, and a family can use it without a single equation. When a bank proposes a portfolio, the question is which views it contains, how confident anyone is in them, and what the portfolio looks like with the views taken out. Usually it looks like the market with fees on top.

DeMiguel, Garlappi and Uppal, 2009

Twenty years after Michaud, the construction literature had produced a long list of clever repairs: shrink the estimates, constrain the weights, use Bayesian priors, combine methods. Victor DeMiguel, Lorenzo Garlappi and Raman Uppal asked whether any of it cleared the simplest bar imaginable. They raced fourteen optimization models against one rule: split the money equally among the assets, and rebalance. The rule has a name in the literature, 1/N, and a very old pedigree. The paper opens with a rabbi in the Talmud advising a man to keep a third of his wealth in land, a third in merchandise, and a third ready to hand.

They ran the race on seven sets of data, out of sample, the way it would have played out in real time, and scored it on return per unit of risk, on what the investor would have paid to hold each portfolio, and on trading costs. None of the fourteen models beat equal weighting consistently. Not the original Markowitz formula, and not its best repairs.

The reason is arithmetic. Optimization only wins if the gain from getting the weights right exceeds the loss from getting the inputs wrong, and the inputs are wrong for a long time. The authors computed how much history the plain formula would need before it reliably beat equal weighting. For twenty-five assets, about three thousand months. Two hundred and fifty years of data, from a world that stays the same the whole time. For fifty assets, twice that.

There is a boundary to the result and the authors state it. The race was run mostly among assets of the same kind, groups of stock portfolios, where equal weights are a reasonable starting point. Nobody proposes putting a third of a family’s money in cash on the strength of it. But the burden of proof moved permanently. Since this paper, any method that claims to be smarter than equal weighting has to prove it out of sample, after costs, on the assets you actually hold. Simplicity stopped being an embarrassment and became the default. Markowitz himself, asked years later how he had invested his own retirement money, said he had split it half and half between stocks and bonds.

What I do

I have run optimizers on my own portfolio, more than once, and each time I have looked at the output and thrown it away. The weights were wild, and a small change in the return assumptions produced a different set of wild weights.

What I do now is take Chopra and Ziemba literally. I put a zero in the expected return column for every asset. It is the same for all of them because I do not know which will do better. That is a battle I choose not to fight. With the returns switched off, the machine can only work on the other two inputs, the ones that can be estimated: how much each asset moves and how the assets move together. It uses them to find the mix that swings the least. That is the whole job I give it. It builds the portfolio that reduces risk, and the returns are whatever the market pays for holding it.

Two things follow from that choice. My weights are stable, because volatility and correlation drift slowly and the returns column never changes. And I spend my time on the risk side, on what falls together and when, because that is where the effort pays.

Three habits

Anchor before you think. Write down what the market portfolio looks like for the assets you hold, and treat every difference between it and your own weights as a bet that needs a reason, a size and a confidence. If you cannot supply all three, the difference is noise, and noise costs money.

Test the sensitivity of anything optimized. Before you accept an allocation from a model, a bank or a consultant, move each expected return up and down by one percent and watch what happens to the weights. If they swing, the allocation is a report on someone’s guesses.

Let simple be the default. Coarse weights, in round numbers, changed rarely and rebalanced on a rule, are close to the best that estimation allows. Complexity has to earn its place by beating that, out of sample and after costs, on your own assets. Most of it does not.

Gala: start with weights that are simple and stable, the market’s own, or equal parts, and hold them. Anything cleverer has to prove it does better on your own money, after costs, before it earns a place. Complexity must pay rent. Simple and stable beats clever and fragile.