Tuesday, May 27, 2014

The data problem

It appears that the “data age” has arrived. Aided by century old Mathematics and latest technologies for the collection of large amounts of data, nearly everybody seems to be excited about the insights that are going to be revealed. Will data prove the existence of God? Will data tell us precisely when each of us will depart from this Earth?  Will data finally usher in the long feared “singularity?” Are data enough to create insights and make better decisions?

Economists and scientists have a checkered history in the use of data to prove what they believe. A hypothesis, once stated and believed, can always be proven with data. The man from France who made a strong case for wealth redistribution may have to now relook at the raw materials he used to build the leaning edifice. The listeners to the Satellite echoes and the finders of the ocean pings, who knew precisely where the metal bird fell and sank may now have to relook at the data they collected and analyzed. Analyzing data with preconceived expectations have proven to be dangerous in many fields – Medicine, Economics and High Energy Physics, included. Data have helped many careers and it has killed many others.

Data are not enough – Common-sense and logic are still important.