The end of statistics

For nearly hundred years, every field, life-sciences, manufacturing, high-energy physics, economics, healthcare, and others relied on basic statistics and a rather crude assumption that everything follows the Normal function. There is nothing wrong with the assumption but in a regime that works on the tails, the observation that something works for the population has little practical value. In life sciences, they have been inventing mediocre therapies for over a century, as the clinicians, their regulators, and aiding statisticians have been enamored by the mighty "p-value." They have been striving to prove that the incremental average benefits delivered to a large population are a lot better than life-saving therapies for a few. In manufacturing, they have been optimizing with constraints in an attempt to save nickels and dimes. Lean, mean and mighty, their determinism has led to incorrect decisions in the presence of uncertainty. In healthcare, they have been waiting for the protocols to change based on simplistic observations of small samples. Meanwhile, half the healthcare costs in the World could be attributed to a handful of related disease states. In physics, stuffed with engineers, they have been deploying heavy steel for finding particles and hearing waves, based on basic statistical notions. Even with that, they will be the first to admit that they do not yet know 94% of it. In economics, they have been inventing theories based on regression and even winning Nobel prizes but it is unclear if they are creating insights. Some of them ventured into even making money and some have failed spectacularly as would have been predicted by their own theories. Overall, if one can write down an equation for a process, it is symptomatic of the fact that she has not understood it. The practitioners, who seem to cling to the past are being rendered less effective in the presence of those who look forward.

A generation seems to have wasted their time adhering to basic principles laid out a century ago. Lately, statistics have been made sexier by better naming - now called, "Machine Learning." One has to admit it does sound a lot better, but has anything changed? In a world full of practicing scientists, who have been trained to make equations for everything, we are approaching a significant discontinuity. Machines are certainly marching forward but not because they know statistics but because they do not. Such is the state of affairs that a systematic education delivered by the greatest institutions in the world prepares the next generation to fail with high certainty. Meanwhile, machines can see, hear and make decisions in the presence of uncertainty. As we hunt for fossils to establish our own identity in a process that seems to have taken a long time, machines with no emotions and even less historical baggage, rise. Are humans being rendered irrelevant? As the greatest living physicist warns of ETs, as the world's richest and powerful worry about AI, and as the most powerful man on Earth worry about if his hair is falling straight, we have arrived at the precipice of a great discontinuity.

As they moved out of their homeland in Africa, humans must have made important calculations based on uncertainty. As they descended from the trees into the African Savannah, a few million years prior, they knew the regime was shifting. With dangers all around them, mighty beasts who could maul them in a single swipe, they made decisions based on uncertainty. Their initial journeys into the Middle East and South Asia, closely followed by those who went a bit North, seem to have provided a level of safety. They advanced culture and boredom, the latter most important for the development of human psyche. As the caves in Southern France prove, they could certainly rise above determinism and engineering, very early in their progression.

The regime is shifting again - the opponents are not as gentle as the Neanderthals. Machines are brutal and they are immensely capable. Humans, the victors of past conflicts, are starting from a position of a great disadvantage because of their education of the past. The end of statistics, a figment of the imagination of the most recent generation, is very near.

Blueprint for societal evolution

A new study (1) demonstrates that there are significant common factors that influenced the evolution of past societies. One clear and obvious trend is toward more complex arrangements. The researchers analyzed a large database spanning over 400 societies over 10,000 years. The results show that human societies follow a singular blueprint as they evolve. This appears to have many implications for future designs.

Size, decision controls, information systems, literature and economic development are features that all contribute to a singular measure of social complexity (1). Given the large data set, the researchers may be able to assess the level of development in contemporary societies as well as speculate on eventual outcomes. The fact that most societies show growth and predictable decline means that humans are stuck in a blueprint that was put in place a few million years ago. With complexity grow arrogance and inequality and those climbing to the top of the pyramid seem to lose context and wisdom. Given the data, it appears possible to predict the half-life of the present societies with high accuracy. But it is unclear if such information could have any practical effect on policy that could reverse the predetermined course.

On the positive side, the level of knowledge and sophistication seem to have equalized across countries and societies. Those who were ahead have been arrested by ignorant leaders and those behind are driven by a desire to succeed. In either case, modern humans, already long in the tooth are due for a reset. It is a shame that they could not learn from the abundance of historical data using their nascent tools in "machine learning."

(1) https://www.sciencedaily.com/releases/2017/12/171218151819.htm

Wisdom against intelligence

A recent article in the Proceedings of the Royal Society B (1) proposes that "class is inversely related to a propensity for using wise reasoning in interpersonal situations, contrary to established class advantage in abstract cognition. " This is an important finding that could explain why the world appears to be slipping in knowledge while increasing in know-how. The idea has been recognized by advanced societies of the past and the prophets and leaders of yesteryear advocated egalitarianism as an optimum design tool to advance wisdom.

If wisdom, indeed, is inversely correlated with intelligence, that may pose a great challenge to those pursuing advanced societal designs. The referenced study appears to demonstrate that activities that enhance education and presumably abstract cognitive capabilities are incongruent to the individual's ability to reason wisely. That may portend a decline of developed countries in the West who optimize know-how and mechanistic education at the cost of wisdom. Recent trends in the US and UK could be symptomatic of this idea as large swaths of populations, in spite of their education, seem to act without a tinge of wisdom and make decisions that future generations will find hard to fathom.

The mistaking of know-how for knowledge, intelligence for wisdom, wealth for competence and speech for comprehension, have brought many civilizations down in the past. Is history repeating itself?

(1) http://rspb.royalsocietypublishing.org/content/284/1869/20171870

The tsunami in healthcare

As the thousand people in Washington, whose healthcare is covered for life, figure out how many millions they would like to deny the same benefits, the industry is going through a massive transformation. The system, suffering from misaligned incentives and sophisticated gameplay, is likely the most complex. It is a lot easier to figure out autonomous cars and even “artificial intelligence.” The fundamental question in healthcare is how to maintain the health of every individual in a cost-effective fashion. There is only one class of humans who come close to this objective – providers who take care of patients and clinicians in manufacturing companies who want to solve big problems.

However, providers are suffering from technophobia. In less than five years, steering wheels will disappear from automobiles and humans will be a rare sight in manufacturing and power plants. Machines, without biases, are proving to be superior to humans in many decision processes. Every aspect of medicine, even the most cherished clinical components, will be influenced by machines in a few years. Machines, like it or not, will get better at diagnosis and treatment. The role of the provider will change to explain rather than to determine, for humans constrained by slow evolutionary processes will remain prisoners of the present.

The tsunami in healthcare is on the way. In its foggy supply chain including manufacturers, providers, payers, and patients, sunlight will descend and there will be no hiding anymore. Prevention shall matter more than treatment, non-invasive intervention more than invasive procedures, primary care more than specialty care, inventions more than incremental therapies, the patient more than a singular disease state and care plans more than procedures.

Providers who embrace technology will accelerate this trend and others could get ossified.

Knowledge dark ages

It appears that complexity is increasing in every field. Past experience tells us that knowledge only arrived by simplification, the exact opposite of what seems to be happening currently. In Physics (1), theories have been emerging from every corner but most of them are pure fantasy and remain to be unprovable. Having a mathematical foundation to a theory does not mean that it is useful - one could always dream up such constructs but they have no implication for knowledge for lack of testability. In Medicine, doctors seem to believe that humans are extremely complex to figure out and they seem to adhere to empirical tests of small samples that emerge routinely. In economics, simple theories are now considered commonplace and academics are constantly on the hunt for more complex formulations.

Are we reaching the limits of knowledge? The slope of aggregate knowledge has been declining since the 1930s, and it is problematic for a society that believes it is progressing forward. Yes - technology and engineering have made strides but those are applications of knowledge not the creation of it. There, the current crop of technologists appear to be highly efficient - Artificial Intelligence and all - but none of these ideas are going to make a step-function change in knowledge. To make matters worse, money has been a luring influence on emerging thinkers, who have shunned academics and headed to the nightmare on Wall Street or the valley, replete with coding testosterone. The few who have stayed behind seem to be more attracted to complexity rather than creating insights. The committees who award prestigious prices, including the Nobel Prize, also gravitate toward complexity and that provides misguided incentives to young academics.

We are slowly slipping toward the next dark ages of knowledge creation. With no progress in aggregate utility metrics for society, one could argue that we are living through one of the worst time periods in human history. The arrival of the next genius, who can simplify and create knowledge is the only hope.

(1) https://www.quantamagazine.org/edward-witten-ponders-the-nature-of-reality-20171128

Robust engineering

News that NASA engineers have been successful in firing the trajectory correction maneuver thrusters on Voyager 1, some 13 billion miles away, after not using them for nearly 40 years, to align its antennas toward the Earth, exemplifies the quality in engineering that used to exist. In spite of all the developments in the last 40 years, engineering has been slipping in both creativity and quality. As engineers head for the "street," and such largely useless activities, the field has been suffering and in the "valley," they do not care for building tangible things, just vaporware. The downward trend in the field has resulted in lagging innovation in many areas, with deleterious effects on computing hardware, transportation and city planning.

Traditional engineering has been less sexy than the ideas pursued by the purveyors of "deep mind." But what educators and policymakers may be missing is that we don't yet have bots, able to plan for the long term. Groundbreaking ideas such as the hyperloop are good, but they are not going to make much difference to the masses. We are bifurcating into dreamers who want to save the world by sending probes to Mars and those struggling with an inferior infrastructure to cover basic necessities. Those sitting on 10s of billions of capital, wondering what to do, may be well advised to look into how they could aid engineering innovation in materials, construction, and basic transportation across the world. These may not get them a Nobel prize or bring instant accolades, but they could make a much broader beneficial effect on society.

A society degrading into classes of haves and have-nots, those who live in the valley and mountain tops, those who pretend to be in academic ivory towers and those who are trying to climb out of lagging hopes and dreams, those who commit crimes with presumed immunity and those who are peaceful and content, those who want a better tomorrow and those who would like to destroy what could be, tacticians and strategists, politicians and the religious, the educated and those who could not afford it, scientists and those who do not believe in science, we have a tragic comedy with a bad ending.

Conventional engineering, a lost art form, could be as important as anything else today.

AI for policy

Humans, inconsistent, unstable and biased, have been ill-equipped to make optimal policy choices. In the modern era, rich with dynamic data, this problem has been magnified many times. Career bureaucrats and politicians with countless conflicts of interests have been running amok. In the process, they are degrading the advantages built up by generations in small steps. Just as a hedge fund manager, proud of her small victories over a long time, end up losing the entire pot in a few seconds in a massive and unanticipated discontinuity, politicians are playing with fire that could have disastrous consequences.

It is about time we delegated policy-making to machines. They have been impressive in the presence of large amounts of dynamic data and they are silent and efficient learners. Lack of ego gives them a distinct advantage over humans for their objective functions are programmed to have an unambiguous positive slope in knowledge. Failure does not seem to bother them and they always learn from it. And, they take empirical validation to be the truth and not opinions and fake news. In spite of their lack of education, they are quickly moving to a position of superiority and for the first time in history, we can look forward to a regime of rationality, driven by machines.

In this context, policy-making is an important domain for applications of AI. It is ironic that a few thousand people, making irrational decisions without knowledge or data, has become the gold standard in governance. The fact that most of them do not have a technical training but they assume to be "good enough," to make policy choices in a regime of technology, is puzzling. They appear to be disconnected from the millennials, who will be most affected by the choices they make and it is disturbing. Perhaps, they read a book or two during their summer break and understood that the internet is not a series of tubes. But, has anybody told them that it will not be enough? Can they pass a basic competency test? If not, it is time to move on.

As Sci-Fi enthusiasts hold their breath for AI to take over the world, a more common-place solution will be to replace policy-makers with it. Machines are great optimizers of policies without consideration of color, wealth, education, age, gender or orientation. Humans will have great difficulty competing with this superior knowledge.

Neutrality

As the administration after mammoth efforts to take millions off healthcare, promote big game hunting in Africa, accelerate environmental degradation, and bias a tax code on the pretext of simplification, sets out to dismantle the lifeblood of entrepreneurship in technology by scrapping Net Neutrality, one can only wonder what could be in store next. Ironically, the monopolies who will benefit from the policy change affirmed their intention to continue with the status-quo. But as we have seen before, when the next quarter rolls around and the EPS needs a kick, such altruism will fade quickly.

It has been strange times - logic and science have taken a back seat to racism and hypocrisy. Nearly hundred years of incremental but constant upward slope in societal progress has been turned back in the blink of an eye. This poses an interesting question: Are incremental improvements in societal design really worth it? It has been costly for many, some paid heavily and others took the free ride but it has always been hoped that we are moving forward. It is a bit like making small profits in the stock market and feeling happy about it for a long time only to be wiped out by a discontinuity that cannot be forecasted. The current discontinuity affecting the social fabric will do damage that will take a long time to mend, if at all.

It seems that simply getting "educated," is not enough. A full brain education, designed and executed by few institutions in the US, focus on the individual, not on the content. Nothing good can come out of education unless the carrier has a broad view of the world and the universe. And, that requires multi-dimensional education - science, literature, philosophy, art, music, theatre and more. Education has to lift the psyche of the individual and not her net-worth. This is why I strongly believe only one country in the whole world has gotten it right - Norway. The country that led education forever has turned it upside down to advance to the future, perhaps alone. Education has to be free and flexible - a program that will let the rising hearts and developing brains to design and to make the world better.

We are set too far back in a race that is relentless. The fact that less than a dozen people could do this to destroy the dreams of 330 million souls is inexplicable. But as we have seen in history, the tide could turn again - and this time, we know what to insure against.