Saturday, January 30, 2016

Data science blindspot

Recent research from MIT that claims their "data science machine," does better than humans in predictive models is symptomatic of the blind spots affecting data scientists - both the human and non-human variety. Automation of data analytics is not new - some have been doing it for many decades. Feature selection and model building can certainly be optimized and that is old news. The problem remains to be how such "analytics," ultimately add value to the enterprise. This is not a "data science problem," - it is a business and economics problem.

Investments taken by companies into technologies that claim to be able to read massive amounts of data quickly in an effort to create intelligence are unlikely to have positive returns for their owners. Information technology companies, who have a tendency to formulate problems as primarily computation problems, mostly destroy value for companies. Sure, it is an easy way to sell hardware and databases, but it has very little impact on ultimate decisions that affect companies. What is needed here is a combination of domain knowledge and analytics - something the powerpoint gurus or propeller heads cannot deliver themselves. Real insights sit above such theatrics and they are not easily accessible for decision-makers in companies.

Just as the previous "information technology waves," called "Enterprise Resource Planning" and "Business Intelligence," the latest craze is likely to destroy at least as much value in the economy, if it is not rescued from academics seeking to write papers and technology companies trying to sell their wares. The acid test of utility for any "emerging technology," is tangible shareholder value.