COVID-19 vaccines work — misreading the data can make you think otherwise

Israel’s data shows that the COVID-19 vaccine works, but only if it’s analyzed correctly: the unfortunate story of the base rate fallacy.

The proliferation of false information about COVID-19 and the vaccines has been a significant risk factor for more disease and death throughout the pandemic. As highly vaccinated countries such as Israel have experienced an increase in cases due to the delta variant, viral disinformation about the COVID-19 vaccines’ effectiveness has also increased.

Israel had one of the earliest and most successful COVID-19 vaccine rollouts. In just four months between January and April of this year, they achieved 57 percent coverage of their total population with the Pfizer vaccine, correlating with a 98 percent decrease in COVID-19 incidence rates during the same period. This early, positive example gave the rest of the world much optimism that we would see the end of the pandemic. Unfortunately, the delta variant (associated with higher transmissibility and a higher viral load) emerged like a tidal wave in Israel around mid-June, causing a surge of new cases of COVID-19, including those among fully vaccinated people. This phenomenon led vaccine skeptics to promote the idea on social media and elsewhere that the Pfizer vaccine was not working against this new, more formidable opponent.  

No vaccine is 100 percent effective, and we expect there to be breakthrough cases. Vaccine effectiveness against asymptomatic and mild delta variant COVID-19 infections is somewhat lower than with the alpha variant and the original virus. Vaccine protection against severe disease is still strong. 

The vaccines are working, but only looking at the number of cases in Israel won’t show us this. 

Comparing numbers of vaccinated versus unvaccinated COVID-19 cases in a highly vaccinated population provides a misleading view of vaccine effectiveness. This error is known in statistics as the base rate fallacy because the absolute number of cases is not presented in the proper context. In a population like Israel, where most eligible people are vaccinated, it makes logical sense that the number of cases will be higher in vaccinated individuals. However, their rate of disease will still be lower. Comparing the rates of COVID-19 between the vaccinated and unvaccinated populations gives us a more accurate picture of what is going on. 

For example, most cases of severe COVID-19 in Israel occur in people over 60 years old. On Aug. 25 (the most recent data available), there were 257 vaccinated people over age 60 with severe COVID-19 cases, 13 partially vaccinated and 208 unvaccinated. When looking at these absolute numbers, it appears at first glance that the vaccines are not protecting against severe illness in this age group or that it is better only to have one dose. But this is because nearly everyone (about 90 percent) in this age group is fully vaccinated, and the other two groups are much smaller. 

If we look at the raw numbers of severe cases 60+ it looks like the vaccine is not protecting against severe disease in this age group but this is misleading 2/7 pic.twitter.com/e3R81kJao4

— Katrine Wallace PhD MS (@DrKatEpi) August 27, 2021

When we re-analyze the same Aug. 25 data to account for the proper base rate (cases per 100,000 in the three vaccination groups), we see that the rate of severe cases in the unvaccinated group (266 per 100,000) and partially vaccinated (58 per 100,000) are much higher than in the fully vaccinated group (19 per 100,000).

When adjusting for the base rate (rate per 100,000 in each population), the rate of severe COVID-19 is lower in vaccinated people 4/7 pic.twitter.com/N66wGbFvjZ

— Katrine Wallace PhD MS (@DrKatEpi) August 27, 2021

This is not only true for severe COVID-19 in older ages. Once adjusting for the base rate, the rates of all COVID-19 infections in Israel are lower in vaccinated people across all age groups. 

And here is the same 8/25 data adjusted for base rate 6/7 pic.twitter.com/GobXAKbL4s

— Katrine Wallace PhD MS (@DrKatEpi) August 27, 2021

The base rate fallacy is just one example of widespread misinformation that has led to public confusion surrounding the effectiveness of the COVID-19 vaccines. The vaccines do work, but viral social media posts presenting misleading data can lead others to decide against getting vaccinated, while not understanding that they are putting themselves at higher risk of illness. It is vital that the information used to make health decisions comes from reputable sources and that the data are presented with proper context to draw valid conclusions. 

Dr. Katrine Wallace is an epidemiologist and adjunct assistant professor of epidemiology and biostatistics at the University of Illinois at Chicago School of Public Health. She is also known as science communicator “Dr. Kat” on her popular social media channels where she educates about epidemiology and the COVID-19 pandemic. Follow her on Twitter: @drkatepi