When Logic Goes Wrong: Survival Guide

Logic Is Dead

As discussed in a previous post, logic fails in the real world. Real-world problems do not lend themselves to logical reasoning since any non-trivial issue involves some uncertainty (Was the report accurate? Is the test result a false-positive?). When the problem is also complex, involving large amounts of information with intricate dependencies, then uncertainty can render the logical argument meaningless.

How Does Anyone Make Decisions?

Given these hurdles, how has humanity managed to make any progress at all? How do we deal with uncertain information in cases of high complexity?

Usually, it’s a combination of the following methods:

1. We ignore uncertainty.  In some fields such as physics, engineering, and computer science, we can create purely deterministic scenarios where uncertainty is negligible. This enables the effective use of logic. More importantly, it allows us to use mathematical modeling, providing a much wider and stronger set of tools than logic alone.
2. We ignore complexity. In many everyday problems, we can combine a small number of reasonably accurate premises. Then, using rules that are similar to logic, we can often reach the right conclusion.  Another way to reduce complexity is investigation. That includes looking for new evidence that is strong enough to reach an accurate conclusion, even if all other evidence is mostly ignored (a “smoking gun”).
3. We experiment. When we face problems with both uncertainty and complexity, we can treat the problem as a black box and follow a two-phase process. First, we try to come up with ideas that seem like they could work (say, an idea about how to cure some disease). In the second phase, we filter these ideas using experiments. The vast majority may turn out to be wrong. But once we see a hypothesis that does work (as in double blind medical studies), we accept it, even if the reasoning behind it is unclear (see Correlation is not Causation). This relatively inefficient methodology is restricted to cases that can be tested and controlled.

Probability Theory

There is, however, a better way to handle both high complexity and high uncertainty: probabilistic reasoning.

Probability theory gives us the mathematical tools to describe complex systems while tracking how uncertainty propagates within them.

A common misconception is to view probability as a property of an event (e.g. “The probability of rolling a six is ⅙”). In reality, probability is a measure of the uncertainty of the observer with respect to an event. This means probability can:

1. differ among observers
2. differ for the same observer over time

For example, two friends, Naomi and David, consider the probability of rolling a six. Naomi uses a computer vision system to track the die’s position and velocity immediately after the throw. Based on that information, she assesses the probability of having rolled a six at 90%. For David, standing right next to her, the probability of having rolled a six remains at ⅙.

If Naomi uses her tracking system to continually refine her assessment of the probabilities as the die flies through the air, the system will provide fine-tuned assessments at every moment. In that case, a single observer (Naomi) has different probability assessments for the same event at different times.

Applying Probability Theory

Once we understand that probability theory deals with our own uncertainty about the world, we realize that it has broad applications, far beyond gambling, games, and stock markets.  Probability theory can be used to analyze any real-world controversy.

For example, probability theory can help calculate the likelihood that Adnan Syed murdered his ex-girlfriend, Hae Min Lee (a story famously covered by the Serial podcast). About 40% of female murder victims are killed by intimate partners. Therefore Adnan is a relatively likely suspect in the murder. Additionally, a witness testified that he had helped Adnan bury the body, although this testimony may have been manipulated. On the other hand, we have Roy Davis, a seemingly unrelated resident of the area, with no forensic evidence or witness testimony tying him to the case. Davis killed another young woman in a nearby location and in a similar manner a year earlier.

Which of the two is more likely to be the murderer? The information provided can make a compelling argument against either suspect. However, only careful probabilistic modeling can establish who is the more likely suspect. In this case Roy was found to be twice as likely as Adnan (who has thus far spent 17 years in prison for the murder).

Probabilistic Analysis Is Hard

Unfortunately, performing a complete probabilistic analysis is no easy feat. One would need to collect all the evidence and try to figure out how likely each possible hypothesis is given this body of evidence. Humans often do not have the right intuitions for probability due to one of several common reasoning flaws (test yourself).

That’s Why We Built Rootclaim.

Rootclaim makes the advantages of probability theory accessible to anyone. Rootclaim uses probabilistic modeling to break down complex issues into smaller, simpler questions, methodically assessing how likely each piece of evidence is to occur under the competing hypotheses. It also considers dependencies, and evaluates the reliability of the different sources. Finally, Rootclaim uses proven mathematical models to combine those inputs into the most accurate conclusion possible.

Click here to see how we use probability theory to calculate reality.