Since its founding, Rootclaim has tried to bring clarity to areas of uncertainty surrounding world events. Today we are one step closer to that goal, with new discoveries that effectively resolve the major controversy of who was behind the 2013 chemical attack near Damascus.
Responsibility for the 2013 chemical attack has been a hotly contested, politically divisive issue, with a wide consensus in the West that the Syrian government was to blame, while Syria and its allies claimed that it was a “false flag” opposition attack, intended to bring about US intervention.
Rootclaim’s 2017 analysis went against this Western consensus, calculating an 87% likelihood that the Syrian opposition carried out the attack. Following the discoveries discussed below, this has now been updated to 96%, one of our most certain conclusions. Read our updated probabilistic analysis here, including a summary of the main claims of each side.
The new findings are a result of what we believe to be the most impressive independent open-source investigation in history. It was initiated nearly a year ago by several volunteers who reviewed all the evidence from the attack and managed to uncover incontrovertible evidence implicating an opposition faction, confirming Rootclaim’s conclusion. The full report is available here, and following is a summary of its findings.
The investigation began by examining the many videos of rocket impact sites that were uploaded following the attack. Each video was examined for clues pointing to its exact location and the trajectory of the incoming rocket.
For example, in this video, the chemical rocket penetrated a wall on a roof and continued to the floor below. Several landmarks in the background can be matched to satellite photos, identifying the exact location (33.519130°, 36.354841°).
Stitching together a few shots from the video shows that the rocket first hit the far wall and then the floor below. Connecting the two impacts provides an estimated trajectory for the rocket, with a launch location to the northwest.
This location is especially interesting as it singlehandedly invalidates the current hypothesis for government involvement in the attack.
Originally, the common claim was that the attack originated in a Syrian army base. But when the rockets were discovered to have a short range of around 2 km, this claim had to be retracted, as no bases were within that range.
However, this entire area lies east of the blue line we added to the diagram, which shows it could not have been the source, as a rocket shot from there would have penetrated the northern wall of the building rather than the western wall, as seen in the video.
This next video was also reexamined. Its location was identified in 2013 (33.520415°, 36.356117°), and the rocket’s trajectory is clearly evident, since it lodged in the ground without bending, pointing to its source.
In two shots from the video the camera is almost directly behind the rocket, and it is seen to align with a tree and buildings in the background.
Connecting these features in satellite imagery provides the trajectory’s azimuth (towards the yellow building).
Interestingly, the UN misreported this angle by 30(!) degrees (towards the purple building below).
The UN also misreported another trajectory, both of which conveniently intersected at a Syrian army base (which we now know is 5 times beyond the rockets’ range). Subsequently, the New York Times printed these mistaken findings on their front page as “forensic” evidence for Syrian army culpability.
This hijacking of international bodies by political and financial interests is becoming a major world threat, hurting the lives of millions. Additional examples exposed in other Rootclaim analyses include the failure of health organizations to realize the efficacy of vitamin D (and other unpatented treatments) in treating Covid-19, and when the scientific community and the WHO suppressed evidence supporting the hypothesis that Covid-19 resulted from a lab leak (see our analysis).
The open-source investigation repeated the process above for seven impact sites, producing this map of all trajectories (triangles represent uncertainty of a trajectory).
The agreement between the trajectories is remarkable, with all of them converging on a small area that also happens to be at the expected ~2 km range from the impact sites.
It is widely recognized that this location was under opposition control at the time (the significance of this spot was not known until now, so both political sides had no problem agreeing who controls it…).
Right in the middle of the identified area is this small field with enough space from which to launch rockets, whose importance will soon become evident.
Video of the chemical attack
A month after the attack, when the US threat to attack Syria had already been removed, a video surfaced, which was claimed to have been found on the bodies of “Syrian terrorists”. The video shows Islamist fighters in gas masks launching the exact same rockets, identifying themselves as Liwa al-Islam (the dominant opposition faction in the area), and announcing the date as August 21st 2013 (the day of the attack).
The existence of video evidence of opposition fighters carrying out the chemical attack is a remarkable story all by itself. What would normally be considered the highest level of evidence, was here dismissed out-of-hand as fake and wasn’t even mentioned in mainstream media, while overconfident unfounded accusations by the US government and false evidence reported by the UN made headlines.
The Rootclaim method prevents this bias by requiring a thorough investigation of all evidence, without filtering. We carefully examined those videos years ago, and also researched video fabrications in general. We found staged videos to be very rare, and that this video has multiple features that are highly uncharacteristic of a fabrication. This finding was a major factor in our initial conclusion.
Thanks to this new investigation, we now have a much deeper understanding of these videos.
The videos are fairly dark with little detail, but a frame-by-frame examination managed to uncover many features of the launch spot, and they perfectly match that same field where all the rocket trajectories intersect.
For example, in several frames the rocket illuminates the area, exposing details such as trees in the background, a field with low vegetation, and a paved platform where the cameraman stands.
In another shot we see a ditch or edge, while other shots show a few scattered trees and brush.
Other such shots in the video provided further features, which were all modeled in 3D, creating a unified view of the area:
This is a perfect match to our field:
We have a video showing opposition fighters with gas masks launching the rockets used in the chemical attack on the night of the attack. This video strongly matches the characteristics of a small field that lies right at the intersection of seven trajectories calculated from the impact sites, within rocket range of all of them.
Continuing to support the government attack hypothesis in light of this new evidence would require constructing a very unusual scenario. Nevertheless, given the political interests surrounding this issue, we will likely witness such attempts soon.
This breakthrough demonstrates the superiority of Rootclaim’s method, which was able to reach this conclusion years ago, without using the new findings, and with much less information and resources than the Western intelligence agencies who confidently claimed the opposite. That is the strength of probabilistic inference: its ability to extract better insights from less evidence.
Of course, many others also took this position, and have now been proven right, but their position was often politically influenced, causing them to reach the wrong conclusion in cases where the West’s claims happened to be true, such as the downing of flight MH17 over Ukraine (we at Rootclaim concluded that it was downed by the pro-Russian DNR, agreeing with the prevailing narrative in the West).
It is very rare to be consistently correct on contentious issues, when each time the truth supports a different political side. We believe Rootclaim is unique in its consistent success in that aspect.
To summarize the key takeaways from these new discoveries:
Having superior inference methods is far more important than gathering more evidence.
Sometimes, a “smoking gun” is already available, and there is no need to collect more evidence (the information here was all available in 2013). This is especially true for videos and photos, which are so rich in information that there is nearly always another discovery to be made.
The current crisis regarding the public’s trust in authorities and experts is not just about ‘fake news’. Experts are repeatedly failing to serve public interests, due to failures of human reasoning and heavy politicization.
Our society needs to quickly improve its inference methods and especially how our intelligence agencies, courts, international bodies, NGOs, and media operate. The current state of affairs is dramatically increasing the probability of a global catastrophe.
Rootclaim will continue to contribute its part in furthering these goals, by continuing to improve our methodology and by disseminating our analyses to a wider audience.
We’re proud to announce some exciting updates to Rootclaim. Since its inception, Rootclaim has focused on exposing the truth on many issues in public discourse using probabilistic inference. Rootclaim has established an outstanding track record, using proven mathematical models and publicly available information to overcome the flaws of human reasoning. However, this time consuming approach limited us to being able to respond to relatively few events.
We are glad to announce upgrades to Rootclaim that will deliver more agile, simplified, and timely analyses.
Previously, much of the analysis work had gone into the tricky effort of identifying and dealing with evidence dependencies. Going forward, we will group dependent evidence and analyze them jointly. This will allow Rootclaim and the readers to examine the evidence in the context of the investigation that exposed it, easily evaluate interdependencies, and assess the significance of missing evidence. Likewise, previously, a large portion of the effort had gone into the analysis of minor pieces of evidence, which have little effect on the calculated results. When analyzed as a group, minor details often lose their relevance.
These new efficiencies will allow us to publish analyses and provide the most reliable estimates about developing stories within a few days instead of the weeks of work that have been previously required.
The new analyses will be more readable while still maintaining Rootclaim’s high level of accuracy, and they can be read from top to bottom without knowledge of probability theory. Each group of evidence and its effects on the likelihood of the hypotheses will be clearly explained.
We have republished the following two analyses using the new model:
Among the many atrocities of the Syrian Civil War, the one that stood out was the use of chemical weapons, and particularly the nerve agent sarin.
While there is general agreement that there were multiple sarin attacks, most of the Western population has accepted that the attacks were carried out by the Syrian government. This assumption is so entrenched that objections to it are widely considered to be “conspiracy theories”.
Most of Rootclaim’s conclusions on other issues later became the consensus opinion, despite some initial pushback. Since this has yet to happen with regard to the sarin attacks in Syria, we decided to issue an open $20,000 challenge to debate anyone on this matter. This challenge has gone unanswered since April 2018, and we are now presenting it here in more detail, and increasing the bounty to $100,000. By doing so we hope to demonstrate the superiority of reasoning methods that integrate honest consideration of multiple hypotheses, unbiased analysis of evidence, and probabilistic inference.
Update: In June 2021, a video of opposition fighters launching rockets was matched to a field within opposition controlled territory, and that field has been shown to be at the intersection of seven rocket trajectories calculated from images of the impact sites. With this additional evidence we now consider the issue closed, demonstrating again the superiority of Rootclaim’s methods. While the $100,000 challenge is still available, we don’t expect anyone to apply.
Win a debate with a Rootclaim team member about the sarin attacks in Syria and take home $100,000. See our Rootclaim Challenge page for additional topics.
Do you have another hypothesis (e.g. Russia did it, or that some attacks were by the Syrian government while others were by the opposition)? Write to us and we’ll consider it.
The stakes: $100,000 each
This is the first in a series of Rootclaim Challenges, modeled after projects such as James Randi’s million dollar challenge, offered to anyone who can demonstrate paranormal powers in a lab setting (all attempts failed). To deter repeated submissions with the intention of winning by luck, we require the challenger to risk the same amount. Applicants who can’t afford to risk $100,000 are encouraged to pool funds together or even crowdfund it. We are willing to reduce the stakes as low as $10,000 for applicants already involved in public debate on the issue.
Both sides will first agree on two judges with strong analytical skills, relevant experience, no previous endorsement of either side, no relevant political biases, and who declare they will examine both hypotheses equally.
Choosing judges will be done publicly on Twitter, so evasion attempts by either side, such as offering biased judges, are exposed. As an example of our honest approach to this process, in a past discussion, when Nassim Taleb offered Glenn Greenwald as a judge, we agreed to bend the rules and accept him, even though he previously said there is “overwhelming” evidence the government is responsible (contrary to Rootclaim’s conclusion) – because we think he is capable of changing his mind when presented with evidence.
Each side will have 8 hours in total to present its case, including time to respond to the other side’s claims, as part of a two-day event.
The debate will be based on all currently available evidence. The goal here is not to trip up or trap the opponent, but to determine which hypothesis is better supported by the evidence. If you have new evidence, or evidence we overlooked, it should first be shared, so we can update the analysis, and if it doesn’t significantly change the conclusion, the challenge can be accepted. We are not claiming to have better evidence, but rather aim to demonstrate the superiority of probabilistic reasoning over human reasoning, when evaluating the same evidence.
Each judge has to declare which of the two hypotheses is more likely. If both agree, the prize pool, minus the debate expenses, is paid to the winner. Otherwise, it is split.
We are flexible – feel free to contact us with offers.
Who declined so far?
The following people have been sent a tweet offering to participate in the challenge but declined or failed to respond. All of them have publicly expressed very high confidence that the Syrian government is responsible.
Eliot Higgins – Founder of Bellingcat.
Brian Whitaker – Journalist and former Middle East editor of The Guardian.
Chris York – Senior editor of Huffington Post UK.
Josie Ensor – Middle East correspondent for The Telegraph.
Scott Lucas – Editor of EA WorldView and Professor at University of Birmingham.
Richard Hall – Middle East correspondent for The Independent.
Julie Leranz – Senior adviser at The Israel Project and a director at The Human Security Centre.
Kristyan Benedict – Amnesty International UK Campaigns Manager.
Dan Kaszeta – Security and CBRN specialist and writer for Bellingcat.
Tobias Schneider – Research fellow at Global Public Policy Institute (GPPi).
Gregory Koblentz – Director of Biodefense Graduate Program at George Mason University.
Numerous other individuals who were very active on social media discussing this issue.
Have you notified anyone of the challenge and they declined it? Let us know and we’ll add them to the list.
A recent study, the first randomized controlled trial of its kind, showed that high doses of vitamin D (calcifediol) reduce the severity of Covid-19 in hospitalized patients. The researchers reported a 30-fold(!) reduction in intensive care admissions of Covid-19 patients. We at Rootclaim analyzed these findings and concluded that even under conservative assumptions accounting for limitations in the study, the effect is still significant and likely around 5-fold. We further demonstrated that since the risks of treatment are low, this treatment protocol should be immediately implemented.
Many health professionals, government officials, and other decision makers worldwide have seen the study results, but they have yet to update treatment guidelines, despite their best intentions, and their vast medical knowledge. This delay may be due to the following:
Not many have the background in statistics and probability required to assess the validity of this study, and distinguish it from the dozens of previous invalidated claims.
They’re affected by omission bias – they default to the “safe” alternative of inaction, waiting for more data, rather than choosing action. It’s easier to later defend inaction than face criticism for acting too soon.
In this particular case the reasons to act now are clear:
Similar treatments have been performed for decades, and the risks are known to be low, especially in this setting, when patients can be monitored at the hospital.
The benefits of the treatment, on the other hand, are potentially enormous, effectively reducing Covid-19 severity to that of the seasonal flu.
While caution is often the correct path when dealing with public health, this is a case where decisions should be made swiftly, using the best available models. At Rootclaim we develop such models, so when our analysis exposed the implications of the new findings, we decided to promote the adoption of the proposed treatment. We hope that this unique challenge will allow the information to reach more decision makers, and save the over 100,000 lives that will likely be lost while waiting for further studies.
Rootclaim is willing to bet $100,000 that vitamin D (e.g. calcifediol) is effective in reducing the severity of Covid-19.
The specifics: On September 1st, 2022, an arbitrator, agreed upon in advance, will review the evidence available by that time, and determine whether the following statement is more likely true or false: Vitamin D (administered in a form such as calcifediol) reduces the odds of ICU admission due to Covid-19 by at least 1.5.
This is the second in a series of Rootclaim Challenges, and is intended to show that the reluctance to implement a vitamin D protocol today is irrational. A decision maker who is not pushing to adopt the proposed protocol is effectively claiming that the probability that this protocol is better than existing treatments is low. But that would also imply that taking the bet is very profitable. Therefore, any professional not accepting this challenge, is implicitly admitting that their decision not to promote the treatment is wrong.
The challenger needs to show that they can commit $100,000. We are open to discussing lower or higher amounts, and the funds can be pooled from multiple sources.
Both sides will agree on an arbitrator who will review the evidence.
The challenger needs to declare that they do not have access to any relevant non-public information. This is to protect from abuse in case of unpublished research (there is still a small chance that further research will discover the treatment is ineffective).
For the same reason, we may update these terms or withdraw the offer, as new information emerges. Of course, once a bet is made it is final and cannot be withdrawn.
If you’re not willing to risk your own money betting against vitamin D, why are you willing to risk someone else’s life?
Study participants that received a large dose of vitamin D (as calcifediol) experienced a 50-fold reduction in the odds of admissions to intensive care, which likely translates to a similar reduction in death rates (see further analysis). If these findings are accurate, the end of the pandemic is near. The study has multiple limitations that would normally warrant waiting for studies, but given the circumstances, it is important to dig deeper and accurately assess its implications. We will estimate the probability that the finding is true, and analyze the risks of adopting the treatment now vs. waiting for further studies.
Is the finding true?
A randomized controlled trial, where patients are randomly selected to receive the treatment or serve as control, makes it possible to isolate whether some clinical finding results from the treatment or from another factor. So far, there have been many studies on vitamin D and COVID-19, which demonstrated a strong link between the two, but causality had not been established. For example, people with poor health may have low vitamin D levels due to low sun exposure, creating a correlation with COVID-19 severity that is not causal.
We now have the results of the first randomized controlled trial on the effect of vitamin D on Covid-19 patients. If it was properly conducted, causality has finally been established, and an effective treatment was found. Unfortunately, the study has several limitations that may distort its result.
Let’s review these possible problems, and their significance. A more mathematically rigorous analysis may be found in the appendix below.
1. The sample size is small, so the findings may be due to chance
It is always possible that the patients that were randomly assigned to the treatment group suffered less deterioration by mere chance. This possibility is calculated using the p-value, which measures the probability of obtaining the study result (or stronger) by chance. The authors disclose it only as less than 1 in 1,000, but the actual number is less than 1 in 1,000,000 (can be verified here, using the study results of 13:13 vs 49:1).
It is important to understand that once a p-value has been obtained, the sample size no longer matters. The goal of a large sample is to reduce the random differences between the two groups, thus making the difference in treatment a larger factor in the final result. The p-value improves both with study size, and with effectiveness of the treatment.
In this case, the effect was so strong that the relatively small sample (76 people) turned out to be much larger than required.
It can be said with certainty that if the experimental results are incorrect, it is not because of the sample size or chance.
2. The control group included more people with risk factors
The control group happened to have significantly more people with hypertension, so it is expected they would have more admissions to intensive care. The researchers identified this issue and performed another analysis (logistic regression) that accounted for it, and the findings were only mildly weakened, from a 50-fold to a 30-fold reduction, with 95% confidence that the result is between 4-fold and 300-fold. We will use 12-fold as a conservative estimate.
We performed another analysis, which assumed that only those with high blood pressure could deteriorate (i.e. removing patients without hypertension from the sample), and the findings still remained very significant, with a p-value of 1 in 5,000, far better than the standard threshold of 1 in 20, or 0.05.
Another issue to evaluate is whether this imbalance indicates a deeper problem with randomization or reporting. The reported p-value of the difference in hypertension is 0.0023, meaning a 1:435 chance it would happen in a random assignment. However, this is just one of at least 10 parameters that could affect the study, and the p-value also accounts for an opposite effect (2-sided instead of 1-sided), so the probability that one of them would happen is only around 1:21, meaning 1 in 21 such studies would have such an imbalance by mere chance – hardly remarkable. Given that randomization was done electronically upon patient admission, such a mistake is unlikely, and as fraud it won’t make much sense (especially as it is later reported and corrected for).
This clearly seems like a chance occurrence, and we see no reason to reduce the estimate beyond what the investigators already did.
3. Patients in both groups were also treated with hydroxychloroquine and azithromycin
Patients in both groups received the standard treatment, which at the time was hydroxychloroquine and azithromycin, a treatment that has since fallen out of favor. Could it be that the findings result from vitamin D neutralizing negative effects of those drugs? This option is unlikely. Trials have shown differing results for hydroxychloroquine and azithromycin, with a few pointing only to a mild risk.
It is also possible that vitamin D only works in combination with the other treatments. Given these mechanisms of action, this is highly unlikely.
We estimate that, at most, this possibility reduces the effect from 12-fold to 8-fold (i.e. vitamin D may have neutralized a 50% increase in severity caused by the other drugs).
4. The experiment was not double-blind placebo-controlled
To prevent distortion of the results by the trial participants or the researchers (even unconsciously), it is preferable that neither know which patients were randomized to the control group and which to the treatment group. This was not the case in this experiment.
This is certainly a weakness of this study. It was mitigated by delegating the decision regarding transfer to intensive care to a committee of experts that included members of the hospital’s ethics committee, who were not aware which group the patient was assigned to, and reached decisions based on a structured protocol.
We have reached out to the investigators to learn more about the procedures, and learnt that this was a result of logistic problems in placebo manufacturing. We got the impression that an honest effort was made to mask the data as much as possible, and the two groups were not otherwise treated differently.
We still need to account for the possibility of outright fraud enabled by this weakness, in which case the findings are false. Since there are no commercial interests around vitamin D, and the fraud would be exposed in later studies, we assign this a probability of 10% at most.
5. There may be another, yet unidentified, factor
Of course, there may be another source for the dramatic difference between the two groups, which has not yet been identified. This would usually be the responsibility of the publishing journal to expose. In this case, the publication has been peer-reviewed and published in a small journal specializing in vitamin D. The publisher is Elsevier, which also publishes the Lancet and Cell.
Such a major finding should ideally be published in a world-leading journal, but given the limitations above, and the likely urgency to publish, it is not unreasonable to choose a smaller journal.
Given the relative simplicity of the trial, we do not see unknown factors as a major risk, at most accounting for a further reduction from 8-fold to 6-fold, and a 10% probability of it invalidating the results.
6. Is the prior probability of the study findings low?
Equally important is the likelihood that vitamin D could cure Covid-19, based on the information known before the article was published. For example, if a study finds that five minutes of neck massage cures lung cancer, it is very likely that there is some error in the study, even if its statistical significance was high.
A recent analysis associates COVID-19 severity with a “bradykinin storm”, and offers vitamin D as possible treatment.
Other effects of vitamin D on COVID-19 quoted in the study include: regulating the renin‑angiotensin system, modulating neutrophil activity, maintaining the integrity of the pulmonary epithelial barrier, stimulating epithelial repair, and tapering down the blood’s increased coagulability.
Many previous studies (such as here, here, and here) have already shown a correlative (but not causal) connection between low vitamin D and COVID-19 severity. The new publication only verifies that the connection is causal.
See additional discussions including potential mechanisms here and here.
However, so far the indication has been for a weaker effect – about a 50% reduction in severity, not 30-fold, so the new finding indicating a near cure is initially surprising. But on further examination, there may not be any contradiction between the studies. A re-examination of a study that published detailed data shows that the rate of infection drops to nearly zero with high levels of vitamin D in the blood (above 50 ng/ml). That is, it is possible that the effectiveness increases with dose, and in the very high doses, as used in the study, near healing is achieved.
It should also be noted that the earlier observational studies used vitamin D levels that were measured a significant time before infection. By the time patients got sick their levels may have changed, which would cause a possible strong correlation to appear weaker.
Another possibility is that the use of short term high dose calcifediol is more effective than the long term supplementation of vitamin D3.
Additionally, the investigators have informed us that the protocol has been used on patients after the trial completed, with similar results.
Overall, according to the prior knowledge, a mild effect is more likely than a strong or no effect, reducing our conservative estimate from 6-fold to 5-fold.
Summary – The findings are true
None of the possibilities mentioned invalidates the significant finding that emerges from the experiment. It is very possible that some minor biases occurred that exaggerated the effect, but it is unlikely that vitamin D had no positive effect.
Summarizing the numbers above, we estimate:
20% probability that vitamin D has no significant effect on COVID-19 severity
80% probability that it reduces severity and death, probably around 5-fold, and possibly much more.
In order to make a treatment policy decision, one must consider not only the likelihood that the finding is true, but also the potential harm and benefits of each possible course of action.
Alternative 1 – Wait
The easy decision is to wait for further studies to verify the new finding. This is what medical experts would normally decide after a first publication of a successful trial.
If the treatment is ineffective, there are no costs and risks to this decision.
If the treatment is effective, then based on the analysis above, the results in the study’s control group, and typical outcomes for hospitalized patients, the harm to a typical hospitalized patient, can be estimated as:
Additional 20% chance of suffering severe disease, with likely long-term implications.
Additional 5% chance of death
Alternative 2 – Adopt treatment
The second alternative is to immediately adopt the protocol for hospitalized patients. In this case the harm to patients is from the large vitamin D dose (whether or not the treatment is effective).
As vitamin D is already a popular treatment, there is abundant information on its risks.
The dose used in the study is about 10 times the maximum recommended dose for prolonged use.
However, the treatment protocol in the study is relatively short – until release of the patient or transfer to intensive care. Previous studies on short treatments at similar doses found them to be safe.
The risk in vitamin D is with increased use that maintains very high levels in the blood over a long period of time.
Even then, the risks are relatively limited, and can be corrected by a low-calcium diet and steroids. For hospitalized patients that can be monitored closely the risk is likely further reduced.
Covid-19 specific risk: vitamin D increases the expression of ACE2 in cells, which acts as a receptor for the coronavirus. Therefore, until now, there has been apprehension about its use. Since the new trial focused on COVID-19 patients and doesn’t show such negative effects, the concern seems to have been alleviated. There is still some low likelihood that the study results were completely wrong, either intentionally or due to a catastrophic mistake that hid a worse outcome in the treatment group.
Based on existing knowledge, the risks in the proposed protocol appear to be low.
The risk can be further reduced by monitoring vitamin D levels in the patients’ blood, and keeping them in a high yet safe range, for example 80 ng/ml.
It is safe to assume the risks of the protocol are much lower than:
5% chance of severe complications.
1% chance of death
Given that both:
The likelihood that the treatment is very effective is greater than 50%;
The benefit of the treatment, if effective, is far higher than twice the risk of the treatment;
it is obvious that the right decision is immediate adoption of the treatment protocol.
Hospitals deciding to wait for further studies should have very strong reasoning that outweighs the apparent harm to patients by delaying treatment.
Global Implications on the COVID-19 pandemic
This analysis shows that if the protocol is widely adopted, COVID-19 severity can likely be reduced to that of the seasonal flu, allowing alleviation of certain limitations, which could bring a major improvement in the economy and social health.
A further conclusion, although with lower confidence, is that vitamin D could be effective at earlier stages of the disease. Previous studies have shown a correlation between high vitamin D levels and lower infection rates. The new study establishes a causal connection at late stages, increasing the likelihood that the correlation at earlier stages is also causal. This would mean that widespread vitamin D therapy (e.g. bringing blood levels to a healthy 30-40 ng/ml) could reduce R0. If that reduction is as significant as indicated by the studies, R0 could drop below 1, and stop the pandemic.
Since vitamin D deficiency is already common, and risks are negligible at this dose, governments should immediately encourage and subsidize vitamin D tests and supplementation for the general population.
Update 12 November 2020: An MIT study that went deeper into the statistical aspects of the trial has reached a similar conclusion that the results of the Cordoba trial are reliable, and that vitamin D treatment is the likely explanation for the dramatic reduction in ICU admissions of hospitalized Covid-19 patients.
Update 24 November 2020: Two additional double-blind, randomized, placebo-controlled trials have reached opposing conclusions regarding vitamin D and COVID-19. One study (printed in The BMJ) showed that those receiving a high dose of vitamin D were dramatically more likely to test negative for the virus within 21 days (21% vs 62%). The other study (preprint on medrxiv) reported that vitamin D supplementation did not significantly reduce hospital length of stay for patients with COVID-19.
Our analysis of the second study shows it does not significantly change the picture, and we maintain our contention that vitamin D should be immediately adopted as a treatment and prophylaxis for COVID-19.
The main flaws are that the study was designed in a way that had a low probability of achieving any of its endpoints, and that the treatment protocol itself was not suitable to demonstrate the efficacy of vitamin D.
The study assumed that, if vitamin D were effective, it would reduce hospital stay from a mean of 7 to 3.5 days. That is an unreasonable expectation, especially given that bolus vitamin D takes a day or two to be mostly converted to 25(OH)D (calcifediol), and may take even longer for 1,25(OH)D. Previous trials were based on longer periods, or used calcifediol directly.
Due to low mortality rates in the study, the study was incapable of detecting a significant effect. Even if there were zero deaths in the treatment group, the result would still be considered not significant (i.e. p>0.05), The study had only a marginally better chance of detecting an effect in ICU admissions or mechanical ventilation, and indeed there it showed some effect, although it is still not significant (as expected).
The primary measure was length of hospital stay, which can be misleading. A short hospital stay could signify either a positive outcome due to improved healing or a negative outcome as with a quick death.
Patients were recruited 10 days after symptoms began, and 90% were already on oxygen. It is possible that vitamin D’s effects are not very relevant at that point, so the failure to improve patient outcomes at a late stage does not reflect the efficacy at earlier stages.
The effects of the vitamin D treatment could have been counteracted by the administration of steroids (to 62% of patients in the study), which weaken immune system activity.
While the study cannot demonstrate vitamin D is ineffective, due to its flawed design, it does suggest that at a late stage, alongside steroids, vitamin D is not immensely effective, i.e. it does not reduce the odds of severe outcomes by 5x or more. Our analysis at Rootclaim suggests that a 5x reduction is a reasonable outcome when vitamin D is administered correctly.
Update 24 May: A retrospective study on hospitalized patients with COVID-19 who received calcifediol showed a drop in mortality rate from 20% to 5%.Appendix – Bayesian Analysis
For those with a background in probability, following is a more rigorous analysis using Bayesian inference. By explicitly stating prior probabilities of hypotheses, and calculating the conditional probabilities of the study results under each hypothesis, a more accurate and robust result is achieved, removing the need to analyze sample sizes, p-values, or confidence intervals.
We will define five hypotheses to be considered:
Damage – Vitamin D worsens COVID-19.
Nothing – No effect
2-fold – Vitamin D reduces the odds for severe COVID-19 by around 2.
5-fold – Vitamin D reduces the odds for severe COVID-19 by around 5.
20-fold – Vitamin D reduces the odds for severe COVID-19 by around 20.
First we shall estimate the probability of each hypothesis based on what was known before the new study. As a baseline, few drugs are effective for any specific disease, but as described above, there are multiple studies showing correlation between vitamin D and COVID-19, and several proposed mechanisms of actions. On the flip side, there is the aforementioned risk that vitamin D could actually exacerbate COVID-19 by increasing ACE2.
We will represent these facts with the following prior probabilities:
Damage – 10%
Nothing – 67%
2-fold – 15%
5-fold – 5%
20-fold – 3%
Adjustments to Study
Given the limitations discussed above, we will make the following adjustments to the study results:
Move 2 cases from ICU to non-ICU in the control group, which we attribute to the higher hypertension cases there.
Move 2 cases from non-ICU to ICU within the treatment group, and do the opposite in the control group, due to unknown weaknesses not yet identified.
Estimate at 20%, as above, the probability that the study was grossly mismanaged, and should be ignored.
So instead of the reported matrix of:
Admitted to ICU
Not admitted to ICU
We will use:
Admitted to ICU
Not admitted to ICU
Next we estimate the probability of getting the adjusted study results, under each of the five hypotheses. To do that, we will use the odds of the control group (9:17 = 9/26 = 34.6%), and adjust by the hypothesis factor, to receive the expected odds in the treatment group. For example, the expected odds in the 2-fold hypotheses would be 9:17*2 = 9:34, or a probability of 20.9%. We then use a binomial distribution formula to estimate the conditional probability of getting the exact study result (3 out of 50 trials) given those expected odds. This is then normalized to sum 100%. Lastly we average with the prior probabilities at a weight of 20%:80%, accounting for the 20% possibility that the study is meaningless.
The full calculation:
Odds ratio (OR)
Convert odds to probability = 9/(9+17*OR)
Conditional probability from binomial formula
Posterior = Prior * Conditional
Posterior, normalized to 100%
Account for 20% failed study possibility (final result)
This more rigorous analysis reaches a very similar conclusion of around 80% likelihood that vitamin D is effective against COVID-19, with a 5-fold reduction being the most probable.
There have been many hypotheses surrounding Stonehenge and its long-lost origins. Can a mathematical assessment by Rootclaim help shed light on its original purpose?
The large prehistoric monument in rural England is comprised of a circle of upright stones. It was constructed around 5,000 years ago, under circumstances that have long been lost in the annals of history. Some believe that Stonehenge served a religious function, while others say it was used as a burial site or a place of mystical healing. Yet others argue that it served as a giant calendar. There are also conflicting claims about how the bluestones used to build Stonehenge were moved from their place of origin 150 miles away. One explanation is that a glacier flow moved the stones, while another view says that people transported them from quarries in Preseli.
How can probability theory help?
Probability theory helps researchers measure uncertainty. And there’s plenty of uncertainty surrounding Stonehenge. By using a probabilistic framework, we can model the likelihood of each piece of evidence relating to Stonehenge.Continue reading
The human mind doesn’t deal well with complexity. It seeks shortcuts, often being fooled by one of many cognitive biases. One of the goals at Rootclaim is to reduce uncertainty by breaking down complex questions into more manageable pieces. This whole system is strengthened by the open crowd-sourced approach, which increases the breadth, depth, and creativity of the analysis. A good case in point is the contrast between the recent UN Joint Investigative Mechanism (JIM) report on the Khan Sheikhoun chemical attack, and the Rootclaim analysis of the same incident.
On September 6, 2017, the UN Human Rights Council (HRC) published a report addressing the April 4 Khan Shaykhun attack. The report found “reasonable grounds to believe Syrian forces dropped an aerial bomb dispersing sarin in Khan Shaykhun.” This finding seems to bolster the hypothesis that the Syrian Army was responsible for the attack. That would justify inclusion in the related Rootclaim analysis. However, a closer look reveals that this is not the case.
For almost a decade, Osama Bin Laden eluded escape, despite a $25 million bounty on his head. That ended in May 2011, when two American helicopters touched down outside a walled compound in Abbottabad, Pakistan. American soldiers stormed Bin Laden’s safehouse, shooting and killing him. Bin Laden’s death raised more questions than it answered. The most glaring question: did the Pakistani government realize that Bin Laden was hiding under their noses?
The Rootclaim analysis of this question looked at extensive evidence. This included information reported about the Bin Laden compound, leaked communications, US behavior following the raid, statements by Pakistani leaders, and the findings of the Abbottabad Commission Report.Continue reading
What Caused the Disappearance of Malaysia Airlines Flight 370?
Malaysia Airlines flight 370 disappeared on March 8, 2014. The location of the plane and the reason it is missing remain unknown. In order to find the most probable solution to this mystery, the Rootclaim analysis of this story currently considers nine hypotheses: the pilot committed suicide; the co-pilot committed suicide; passengers hijacked the plane; the pilot hijacked the plane; in-flight fire; turbulence; the flight was shot down; fuselage crack; and that an improperly repaired wing-tip caused the crash. At the moment, the evidence suggests that the pilot crashed the plane while committing suicide.
Rootclaim is a collaborative analysis platform that transforms how people understand complex issues by combining the power and reach of crowdsourced information with the mathematical validity of Bayesian statistics.