Sunday, August 16, 2015

Selective Bayesian Updating

Confirmation and conformation biases, symptoms of a disease that is eating into science and technology, have wrecked havoc in many areas including pharmaceutical research, astrophysics, finance, healthcare and policy, just to name a few. It appears that participants in these areas, utilize a novel mathematical technique – Selective Bayesian Updating (SBU). It is not just that data are fine tuned to prove the hypothesis but even in cases where it could not be proved, the posteriors remain the same for subsequent experiments.

Take Search for Extra-terrestrial Intelligence (SETI) as an example. It has been speculated that oceans that could support life exist in many icy and rocky remnants within the solar system – Titan, Europa and even the outcast, Pluto. Suppose the missions to these objects reveal no signs of life, will NASA and SETI use Selective Bayesian Updating for the probability of ET? Recent excitement around Kepler 452B, the most “Earth-like” planet ever found in the Milky Way, has led SETI to focus their instruments in that region. Suppose we find no signals of value, will the posterior probability of the existence of ET, remain the same? This is very convenient for those involved in the research. It is the ultimate free lunch in science – a negative result has no change in the posterior probability of the hypothesis being correct.

In pharmaceutical research, it has been shown that the efficacy of marketed drugs decline over time. This is a curious phenomenon as cutting edge research coupled with a vigilant regulator, the FDA, are unlikely to let marginal drugs into market. Was Selective Bayesian Updating deployed in the many experiments that led to the approval of the drug? Patients and providers, perhaps, are less susceptible to this problem and normal updating over a period of time, may be revealing the truth.

Selective Bayesian Updating, a disease that is substantially slowing the innovation slope in science and technology, could be treatable.