Sunday, July 23, 2017

Condensed knowledge

A recent study (1) that appears to demonstrate mining entire body of articles than abstracts provides higher information content is obvious. The more important question is what the mining is for and how the results are expected to be used. Also interesting is how much more incremental knowledge can be extracted from full articles.
In a world of exponentially expanding data, the risk of getting caught in the details is high. The “detail oriented technician,” has been more useful than those who see “the big picture,” in a scientific context because discoveries came from pouring over the data in a detailed fashion. But now, data is arriving from every direction and time is not an affordable luxury for decision-making and advancing ideas, with half-lives tending toward days and weeks.
Humans have not been here before. For hundred thousand years, they banked on experience, accumulated over many generations. Now, experience is losing out to machine learning. Here, time is of the essence and understanding patterns reigns supreme. Patterns have a hierarchical structure, and if a pyramid needs to be built bottoms up, it may not add value if it takes too long. Hyped up technologies in the areas of deep learning and mind will likely find that brute force approaches to “artificial general intelligence,” are unlikely to yield interesting results. 
The early human survivor in the wild had an intuition that the leopard is behind the bush to the left of her and she relied on patterns at the highest levels. Data scientists toiling with oceans of data, statistical modeling platforms, machine learning, deep learning and even deeper mind, may have to look back and understand that throwing data, analytics and technology at a problem, never helped humanity.