Discounting Weak Evidence
One pitfall to avoid is prematurely discounting seemingly weak evidence. Weak evidence can take many forms. It could be evidence that seems very unlikely under all hypotheses. Or it could be evidence that is non-intuitive and doesn’t seem to fit what we consider “conclusive” evidence.
When evaluating evidence, it’s easy to get distracted looking for “irrefutable” evidence (more on that in an upcoming blog post). However, that’s a mistake. What’s really important is the ratio between how likely evidence is under the hypotheses.