**056: Campbell Harvey on Improving Significance Tests, the Importance of Positive Skew and the Future of Blockchain**

* Campbell R. Harvey* is Professor of Finance at the

*Fuqua School of Business*at

*Duke University*and a

*Research Associate*of the

*National Bureau of Economic Research*in Cambridge, Massachusetts. He served as Editor of

*The Journal of Finance*from 2006-2012 and is President-elect of the

*American Finance Association*.

Professor Harvey obtained his doctorate at the *University of Chicago* in business finance. He has served on the faculties of the *Stockholm School of Economics*, the *Helsinki School of Economics*, and the *Booth School of Business* at the *University of Chicago*. He has also been a visiting scholar at the *Board of Governors of the Federal Reserve System*.

Campbell received the *2014 Reader’s Choice Award* for the best paper published in the *Financial Analysts Journal* and the 2015 prize for the best paper published in the *Journal of Portfolio Management*. His recent work on evaluating trading strategies has won best paper awards.

Campbell’s research interests include statistical methods, risk management, asset allocation, real assets and cryptocurrencies. He is the *Investment Strategy Advisor* to the *Man Group plc*, the world’s largest, publicly listed, global hedge fund.

**Economics:**

**In this interview, Campbell mentions: **t-statistics, significance tests, trading strategies, investment premium, beta, correlation, standard deviation, confidence interval, P-value, Bonferroni multiple testing method, Type I error, Type II error, probability, normal distribution, optimal portfolio, volatility, expected returns, portfolio, pay-off, skew, over-fitting, regularisation, Efficient Market Hypothesis, Fractal Markets, stock market anomalies, Straw Man Model, momentum effect, mis-pricing and outliers.

**Economists: **

**In this interview, Campbell mentions: **Nassim Taleb, Benoit Mandlebrot, Peter Edgar, Yan Liu and Eugene Fama.

**In this episode you will learn:**

- why it’s important to use
and**t-statistics**and how it can be improved.**significance tests**

- about the very simple idea Professor Campbell Harvey applies to his
to improve the robustness of his tests.**statistical modelling**

- why it’s wrong to use 2
to have**standard deviations**when running many tests.**95% confidence**

- about ‘Significant’, the
that illustrates the vulnerability of statistical significance testing.**XKCD cartoon**

- do green jelly beans really cause acne? How significance tests can mislead with a fluke.

- how a
based upon picking a portfolio of shares based upon the first letter of a ticker symbol showed that those tickers that began with the letter A outperformed other stocks.**trading strategy**

- how testing multiple times is effectively
and what should be done about it.**data mining**

- about the meaning of 95% confidence and 5% level of significance.

- what a
is and why we ant it to be as small as possible.**p-value**

- if it’s important for the finance and economics profession to look at how other sciences are applying testing methods?

- whether we need a tougher standard to lower the possibility of
?**false discoveries**

- if there is a chance of a fluke finding and why we should apply the
solve this?**Bonferroni multiple testing method**

- about the decay signature of the
and whether it is just background noise.**Higgs Boson**

- whether the findings of many published academic peer-reviewed papers are wrong.

- about
and their trade-off.**Type I and Type II errors**

- about
’ mission to make all randomised control trials made public.**All Trials**

- the problems when measuring and using
.**volatility in asset returns**

- why the
in a distribution must play more of an important role in risk management and portfolio selection.**level of skew**

- why Taleb’s
only looks at one side of the distribution – the negative side, and why we must also look at the positive side.**Black Swan**

- how applying
avoids ‘over-fitting’ the data so that unexpected future outcomes can be considered.**‘regularization’ to portfolio selection**

- about the
and the 316**efficient market hypothesis**that have been published to refute this hypothesis.**anomalies**

- why the best traders are in Asia and how
makes them so.**insider activity**

- about the rise of
and**crypto currencies**and why schools across US universities are introducing modules on it.**Bitcoin**

- what is
and why its is safe.**blockchain**

- about the bank’s idea of creating a
.**permission blockchain**

**The Problem with Significance Testing and How to Solve It**

If you’re trying to see if a variable Y is associated with a variable in a significant way, we usually think of looking at that correlation and determining whether you’re 95% confident that you’ve got it right. Usually what that means is that you’re 2 standard deviations away from zero. So, zero would be there’s no relation.

It turns out that that is perfectly acceptable if we’re looking at one correlation between Y and X. However, if it’s not X, it’s X_{1} you try. You try X_{2}. You try X_{3}, you try … X_{100}. You try 100 different things. Then the criteria of using 2 standard deviations to have 95% confidence is just plain wrong.

The reason why this is wrong, is that when you’re running 100 tests, there is going to be a high probability that something will turn up that’s 2 standard deviations from zero just by chance.

The ‘Jelly Bean’ cartoon by **XKCD** called *‘Significant’* illustrates how testing a hypothesis can become misleading when conducting a significance test. The hypothesis being tested here is whether jelly beans causes acne.

A randomised control trial is ‘conducted’ by scientists. This is done where, say we have 50 people with jelly beans and 50 people with no jelly beans and we count the acne. And what basically happens is that there is no significance. So the scientists don’t achieve the 95% and conclude that there is no relation between jelly beans and acne.

However, the cartoon further illustrates what happens when the color of each jelly bean is tested to see if a particular color causes acne. 20 additional randomised control trials are conducted. The cartoon shows that the link between the Red Jelly Bean and acne is insignificant. Blue Jelly Bean – insignificant. Until you get to the last jelly bean, the 20th, which is the Green Jelly Bean. They find that there is a significant relation between Green Jelly Beans and acne. The final frame in the cartoon is a headline saying ‘Green Jelly Beans Linked to Acne’.

So, if you do 20 trials, one of those is likely to show up as significant using the standard criteria and it’s a fluke.

*“The idea of my research is that we need to raise the bar that 2 standard deviations is no longer – that 2 sigma is no longer – something that should be considered. We need to go much higher.”* – Professor Campbell Harvey

http://imgs.xkcd.com/comics/significant.png

**The Bonferroni Multiple Testing Method**

When we say that there is 95% confidence, we are saying that there is a 5% chance that the finding is a fluke. The 5% is called the p-value. What you would like is for that p-value to be as small as possible. You want as small as possible probability that the finding is a fluke. So the usual p-value for a single test with just X and Y for 5%, would imply 2 standard deviations. When you do multiple tests, you need more than 2 standard deviations from zero. If there is a chance of a fluke finding, then we should apply the Bonferroni multiple testing method solve this.

What the Bonferroni does is a simple correction. What it says is ‘you discover a p-value which is, say, 0.004 and you multiply by the number of things or X’s you’ve tried, which is, say, X_{1} to X_{100}. All of a sudden, your p-value transforms to 0.4 or 40%. That means there is a 40% chance that in repeated trials that this thing you’ve identified, say X_{57}, is a fluke. So when you use this adjustment, you discard that variable.

**Quotes by Professor Campbell Harvey in Episode 56 of the Economic Rockstar Podcast:**

In the practice of finance, some investment manager goes to a client and shows a great strategy and looks amazing. But they don’t tell the client or potential client that they tried 499 other possibilities and this is the only one out of 500 that worked– Professor Campbell Harvey.

*“The problem with volatility is that it is a symmetric measure, that if you’re way above the average that contributes to the same volatility as if you’re way below the average” – Professor Campbell Harvey*

*“I’ve being pushing for the last 15 years to reform the way that we do our portfolio analysis, our standard models, to have the skew play a role.” – Professor Campbell Harvey*

*“It’s also a fact that it’s really hard to find any asset return that adheres to a normal distribution. If it does, it is very unusual.” – Professor Campbell Harvey*

**Books:**

- The New York Times Dictionary of Money and Investing: The Essential A-to-Z Guide to the Language of the New Market by Campbell Harvey and Gretchen Morgenson
- The Black Swan by Nassim Taleb
- The Ascent of Money by Neil Ferguson

**Papers:**

- Evaluating Trading Strategies. by Campbell Harvey and Yan Lui
- Where are the World’s Best Analysts? Campbell Harvey, Sam Radnor, Khalil Mohammed and William Ferreira
- Conditional Skewness in Asset Pricing Tests. Campbell Harvey and Akhtar Siddique, Journal of Finance 55, (2000): 1263-1295. (P56)

**Other Resources:**

- Garden of Econ podcast
- Hypertextual Finance Glossary – Over 8,000 Entries and 18,000 Hyperlinks: The largest financial glossary on the Internet
- The New York Times Dictionary of Money and Investing: The Essential A-to-Z Guide to the Language of the New Market by Campbell Harvey and Gretchen Morgenson

## **Websites:**

**Where to Find Campbell: **

**Website:** Duke University

Podcast: Play in new window | Download