When a friend asks me what I think of investing in the SP500 I have a standard answer:

It’s a broad basket diversified across industries including foreign revenue via multinationals. If the economy grows and companies make profits in the long run you should do fine.

The next question is inevitably “what percentage of my investable assets should I put in it?”

The quick answer is that custom is not a bad guide. If you don’t want to do any work 60/40 is a reasonable weighting or you can follow the glide path of a target-date fund. Ed Thorp admits he puts all his money in the index because that’s the recipe for the highest long term return. But he’s also admits that if he suffered a 90% drawdown he still has more money than or his heirs ever need. Most of us would be injured if the market swept the leg right before we retired.

So while 60/40 or age-based weight is a good starting point if you know nothing, most people who care enough to ask “how much should I put?” don’t know nothing. They know their life circumstances. They probably have goals. And concerns. For some goals loom larger than concerns in their minds. For others it’s the opposite. Both types of people find that the balance of these considerations makes them a candidate for a non-standard answer.

Of course you don’t just go full dominatrix and say “I heard what you need, now do THIS”. You let them own their decisions by laying out the proposition and letting them come to a conclusion. This is how I describe the proposition of investing in the SP500 or broad index:

It’s a box that has historically paid about 9% per year with a standard deviation of about 19%. The sharpe ratio has been close to .50. Under that hood, you have negative skew and fat-tails which means you are getting paid to hang on through some scary turbulence.

[A more granular answer would say something like “you earn the risk-free rate + 3 to 6%”]

They’ll ask if that performance will continue. I’ll die a little inside because the presumption is they think someone could know. Any answer is merely epistemic icing on a layer cake of conditional probabilities emanating from whether humanity will self-destruct. But ruining your Thursday was not on my to-do list so I play along:

“You see, it depends on whether you think there’s something persistent about the last 40 years that padded those hundred year stats. Have you wondered if the fact that companies stay private for longer matters for future returns? I mean it was kinda cool that Gates, Jobs, Bezos, Zuck, Brin, and Page had the public as their LPs during their massive growth phases. Not sure if they started their companies today if that would happen. On the other hand, the GFC marked the end of defined benefits for private companies in exchange for defined contribution. If the stock market is now the new social security I can see the political argument for not only a Fed put but a permanent federal jobs program.

You know, I have this saying that markets are biology not phys— um, hello…oh…at what point did I cut out, ahhh, sorry, go ahead…no, no, it’s ok, really go ahead, you should never be to a late cat seance”

You get the idea. Any discussion of the future has speculative errors bars that pale against the tyranny of circumstance. I’ve always appreciated the humility behind my buddy Nick Maggiulli’s observation that if you had invested from 1960-1980 and beaten the market by 5% each year, you would have made less money than if you had invested from 1980-2000 and underperformed the market by 5% a year. May your capital appreciation years coincide with economic growth where you live. Deciding to invest is a faith-based exercise before anything. Reminds me of a riddle:

If you have 3 of me, you have 3.

If you have 2, you have 2.

If you have 1, you have 0.

What am I?¹

Discussing the future with me is nothing but entertainment.

What I can help with is deconstruct the properties of historical proposition that the SP500 offered and pull out a bunch of interesting lessons. If the world carries on, they’ll be useful context for considering how much to invest. But I’d bet you’ll learn a lot more than that.

Let’s roll.

We will play show-and-tell with SP500 returns from Jan 1928 until April 15, 2024. By stepping through several exhibits with commentary you will come away with intuition for properties of the historical returns.

First, I computed non-overlapping logreturns (ie compounded) and volatilities for the following intervals:

• 1-day (“daily” returns)

• 5-day (“weekly”)

• 21-day (“monthly” — note these don’t line up to calendar months, I’m just labeling a 21-day return as “monthly”)

• 63-day (“quarterly”)

• 126-day (“semi-annual”)

• 252-day (“annual”)

For each partition of returns, we compute:

• mean return (µ)

• standard deviation (σ_raw: this is an unannualized measure of volatility)

• mean absolute deviation (MAD: mean absolute move size — this is another measure of volatility. Unlike standard deviation the return data is not squared so large moves are not given extra weight.)

• Sharpe ratio (SR: µ/σ_raw — this is a measure of return per risk)

• St dev of MAD (This is a measure of volatility of volatility — the standard deviation of a typical change)

  • (St dev of MAD) / MAD normalizes this for the size of the typical move. The St dev of MAD will obviously be higher for 1-year moves than 1-day moves so we divide by the MAD of 1-year or 1-day respectively

• MAD/SD (For a normal distribution MAD ~ .8 * volatility. Options folks will recognize the MAD as the straddle or expected move size. Observing MAD/SD ratios less than .80 is a quick way to recognize data is not normally distributed but has fat tails or skew)

• annualized volatility [σ_annualized is σ_raw annualized by a factor of √(252/interval days…so monthly return are annualized by multiply the raw vol by √(252/21)]

Here’s the table:

What I notice:

1) Annualized vol is similar regardless of the sampling interval

Option traders split hairs over how to measure realized volatility because they are in the business of discerning 19% vol versus 20%. The general heuristic is more frequent sampling periods will converge on a sensible estimate of volatility faster. Having 252 daily data points is better than a single annual point. Option traders will often use more than just the closing price from a single day. There are methods to incorporate OHLC (open, high, low, close) or even use tick data to for even faster updating measure of realized volatility.

But the good news is — for general investing applications, this table says “just pick something reasonable”. It’s true that if the market sells off 3% and rallies back to unchanged intraday that using closing prices will mask the true volatility, but if a market chops around violently everyday but reverts back to a steady price the less frequent sampling might be fine for your context.

Picking a vol for portfolio weights doesn’t need the same scalpel you’d use to price a straddle.

2) The vol of vol declines with time

Look how (St Dev of the MAD/MAD) declines as you lengthen the sampling period. In other words, vol is more predictable over longer periods.

This is the same idea behind a popular option trader tool: the vol cone (explained in Understanding Vega Risk)

QQQ realized vol cone via moontower.ai

3) Sharpe ratio increases with time

We define SR as mean return / volatility

Daily SR = .04% / 1.19% = .03

Annual SR = 9.21% / 19.52% = .47

Why the disparity?

Friends — this is the entire basis of trading, the casino business, thinking in terms of repeated positive expectancy bets. It’s the gravity of investing that says a small positive edge ensures you get rich while a negative one means you go broke.

In a line: edge scales linearly, volatility scales by √(time)

*If you were a poker player you’d substitute “hands” for time.

Let’s try something with those daily numbers:

.04% * 252 = 10.08%

1.19% x √252 = 18.89%

SR estimated by annualizing from daily samples = .57

I’ve explained this before in Understanding Edge

4) The distribution, at least for periods less than 6 months, looks like it might be non-normal

Look at those MAD/SD ratios —> all below .80 with the daily ratio significantly lower at .64 (I’m going off memory, but I believe the price returns of oil future spreads which are definitely not normal have MAD/SD ratios in the .60 range). The ratios are a clue to look deeper for skew and/or fat-tails.

Fat tails and Skew

Let’s examine the data by filtering for larger moves. Let’s start with moves greater than 1 standard deviation for each sampling period.

Note that we expect 31.73% of all returns to exceed 1 standard deviation if the distribution is normal.

We count how many moves exceed 1 st dev and divide that percentage of the sample by the expected 31.73%.

Whoa…the ratio is less than 1.

We observe that there are actually less moves of > 1 st dev than we expect!

But then we remember — if we filter for moves greater than 1 st dev we are also catching moves greater than 2 st devs, 3 st devs, etc. We need a finer look.

Here’s the full table with me highlighting points of interest. Observations follow.

Observations:

• There are less > 1 st dev moves than we expect, but far more moves of > 3, 4, or even 5 standard deviations. This is a distribution with a high peak (more small moves than we expect) but also fatter tails (more extreme moves) than a Gaussian or normal curve predicts.

• 4 and 5+ standard deviation moves occur several orders of magnitude more often than you expect but of course are still low probability. This isn’t shocking — the Black Monday crash of ‘87 was a single day freefall ~23%

• Negative skew: the overwhelming majority of extreme (3 sd or more) moves are negative for all sampling periods. Extreme single day moves are the most balanced with a down move being a slight 6-5 favorite.

• We have never seen a 5 standard deviation 1 year move (~90%) but during the early 1930’s there was a 70% drop over one year

• [Be careful: this is not a drawdown study — it’s a survey of point-to-point non-overlapping total compounded return]

Let’s pause for pictures.

The peak of the distribution is centered around the mean monthly return of .77% reflecting the upward drift

• The shoulder region between 1 and 2 st devs (sd = 5.64%) has less density than the normal curve predicts.

• The tails are fatter with actual observations when we would expect those outcomes to almost never occur

Here’s the same picture zoomed in so the range cuts off tails beyond 20%:

The zoomed in pic makes the left skew, upward bias, and taller peaks around the mean more visible.

Relating to options

When SP500 put skew increases implying a higher probability of a far left tail move. But by making the far OTM puts more expensive, the ATM/OTM put spread gets cheaper which means the probability of going down at all is implied to be lower. This makes sense if option prices change while the spot price remains fixed…the implied densities are shifting around but the sumproduct of probability x outcome must still cash out at the same expected stock price.

This tends to make more sense when you use an extreme bimodal distribution — like a stock worth $100 because it’s 10% to be worth $1000 and 90% to be a zero. Such a stock will have:

• Extremely expensive ATM volatility — in fact the straddle is worth $180 when the stock is worth $100! If you could buy the straddle for less than $180 you can construct a riskless arbitrage by shorting shares against the straddle. (I’ll leave it to inclined readers to construct a table of straddle prices vs share hedge ratios that ensure a profit).

• Very expensive OTM call skew. The $900 strike call is worth $10 (but I’m sure there’s someone on Reddit selling at $5 “for income”)

• Very expensive put spreads (and conversely cheap put skew!)

You have enough info to price calls and puts for any strike in this toy example to prove all this to yourself. If you need guidance see any of these:

✍🏽Practice Pricing Options By Hand

🐍The Snake Eyes Option

📏What We Can Learn From Vertical Spreads

Wrapping Up

Here are the main points of what we covered:

• Annualized vol is similar regardless of the sampling interval (it just takes longer to get data that is sampled less frequently).

• Vol of vol declines with time. Remember the “vol cones”.

• Edge stacks linearly while volatility or risk scales sublinearly. This is the entire basis of repeated edge thinking.

• SP500 historical returns exhibit negative skew, especially for large moves, and fat tails.

If history is a guide it’s reasonable to see:

• a 15% selloff in a single month in a 4 to 5 year period

• a 25% selloff in a single month in a 20 year period and to lose over 1/3 over the the course of a year

Large moves are rare so it’s hard to make guesses about them but when the next financial panic strikes remember such moves would be entirely precedented.

And to think we are just looking at the ultimate survivorship bias market — America and only in nominal terms at that! There’s no point in being humorless about this stuff. It is certain that future humans will remember that English was spoken in the United States the way we remember the Romans spoke Latin. May that not happen during the investing years of anyone you know.

(I have a strange urge to go watch the Enter Sandman video now…”if I die before I wake…”)

Related reading:

• How Tails Contrain Investment Allocations

• Risk Depends On The Resolution