Rice University logoGeorge R. Brown School of Engineering
 
Statistics
 

Yule's "Nonsense Correlation" Solved!

Colloquium

Statistics

By: Philip Ernst
Assistant Professor
From: Rice University
When: Monday, February 27, 2017
4:00 PM - 5:00 PM
Where: Duncan Hall
1070
Abstract: In this talk, I will discuss how I recently resolved a longstanding open statistical problem. The problem, formulated by the British statistician Udny Yule in 1926, is to mathematically prove Yule's 1926 empirical finding of ``nonsense correlation.” I solve the problem by analytically determining the second moment of the empirical correlation coefficient of two independent Wiener processes. Using tools from Fredholm integral equation theory, I calculate the second moment of the empirical correlation to obtain a value for the standard deviation of the empirical correlation of nearly .5. The ``nonsense'' correlation, which I call ``volatile'' correlation, is volatile in the sense that its distribution is heavily dispersed and is frequently large in absolute value. It is induced because each Wiener process is ``self-correlated'' in time. This is because a Wiener process is an integral of pure noise and thus its values at different time points are correlated. In addition to providing an explicit formula for the second moment of the empirical correlation, I offer implicit formulas for higher moments of the empirical correlation. The full paper is currently in press at The Annals of Statistics and can be found at http://www.imstat.org/aos/AOS1509.pdf.