Sunday, May 3, 2015

Significance

Statistical significance has let many industries down and built fortunes for many, riding their luck. A recent paper from Duke University explains what most non-statisticians and non-financiers always knew. Models that do not make practical sense are unlikely to work. Industries on top of the modern economy, those who attract the best and the brightest – physics, medicine and finance – have been playing with statistical fire, discovering and proving everything there is – some for money and others for fame. As the Duke paper points out, almost any hypothesis could be proven by a sufficiently large number of trials. And proving hypotheses is front and center for any “scientific profession.”

In this context, it may be interesting to make the following predictions:

1. LHC : If 6 trillion trillion collisions are made and the data analyzed, LHC could prove God exists. Just 6 trillion was enough to find the “God particle” within 5 sigma. This is a good experiment. Proving God exists may solve many of the vexing problems faced by humanity.

2. SETI : If an antenna is provided to every roof top in the world and the “search” accelerated by a billion times, SETI could prove ET with pointy ears, a cone head and red white and blue stripes across the body exists in some distant galaxy.

3. Wall Street : If the number of idiots trading securities back and forth every day is increased by an order of magnitude, Wall street could create somebody who wins on every trade for 10 years running.

Statistics, the “science” that is foundational for “accelerating knowledge” of humanity, may singlehandedly bring knowledge-seekers to a standstill in the presence of “big data.”