The book of why – Judea Pearl - Review

 

Am reminded of my master’s degree in finance that I had pursed back in 2009-10, one of the key problems that we had always tried to solve was the quantifiable impact of a bunch of variables on the price of a stock. Irrespective of the number of variables that were infused into an equation, there was always an element ‘X’ which was basically an unknown variable that also contributes to the price movement.

Judea Pearl uses the term confounder instead of ‘X’ but the fundamental point remains the same. In a cause-effect equation, we have not come to a position wherein the exact cause of effect can be fully explained or pointed to a definite cause. However, with a large sample in place we can state with a certain degree of confidence that can pinpoint the reasons of occurrences.

I was surprised to note that even smoking – cancer kind of cause-effect equations has taken three to four decades of debate before arriving at a concrete conclusion. The reason drives down to the fact that drilling down to the root cause is a complex operation when it’s meant for a large population. A lot of scientists have themselves been chain smokers which complicates the path for an unbiased research output.

One of the solutions to arrive at a definite cause was to extend the chain of equation and add variables that happen to be subsequent next step of a certain consistent act. For example, regular practice of smoking creates tar deposits in lungs which is a pattern observed amongst majority of smokers who had eventually contracted lung cancer. Judea terms tar deposit as mediators which results in lung cancer but is not really the root cause.

There are far more complex situations like the heat wave in the Western Europe back in 2003 that resulted in over 15,000 people losing their lives. Two possible reasons cropped up, the first one being climate change and the second being random variations in weather that’s not really connected to climate change. This is where fractional attributable risk (FAR) comes into play using which factors that significantly contributed to heat wave can be determined, of course in a debatable fashion.

Judea also touches upon the maturity levels of AI to identify root causes using Big data and draws the conclusion that even though AI has been able to beat world champions in a game of chess and AlphaGo, their effectiveness in medical, political, climate change or other field remains a work in progress by a significant measure.

Though the book provides insights, it’s meant for a very concentrated audience who are into research or into some exploratory missions using data. About half of the book was full of probability formulas that takes away the joy of reading and gets you into some math solving mood, if at all you enjoyed it.

Judgement: Read if you belong to target audience.