Update from Steven Millman, the DC data modeler:
Aug 8th COVID Update: The Three Bears Mortality Update, Is the Death Count Too High, Too Low, or Just Right?
Short version: COVID total US and global death counts are almost certainly undercounts of true mortality. A more clear understanding of how deaths are recorded and counted is essential to understanding the claims being made about over or under-counting.
So tl:dr, but if you want a quick cut-and-paste resource for explaining to people why they’re wrong about their particular COVID-19 death count grievance, I’m hoping this will be a good resource for you. Here we go.
There is so much poor information and outright disinformation, I thought it made sense to do an update related to how COVID deaths are collected, counted, reviewed, and published. The most common claims I’m seeing on social media are:
- COVID death counts are greatly exaggerated because any death of a COVID positive person is being counted as a COVID death including such unrelated deaths as car accidents, homicides, etc.
- COVID death counts are greatly exaggerated because hospitals are financially incentivized to improperly mis-classify deaths as COVID.
- COVID death counts are greatly exaggerated because probable COVID deaths are being reported along with confirmed COVID deaths.
- COVID death counts (and cases) are being undercounted due to the new reporting system moving data from CDC to HHS.
- COVID death counts are being greatly undercounted because death certificates are missing people due to either inadequate testing and/or inaccurate attribution to another cause of death
- COVID death counts are on the decline based on the CDC website data but rising in the Hopkins/Worldometer data, so one (or both) of those data sources must be lying.
These six claims are all being used and misused to justify stories folks want to tell. This update will take you through how deaths are actually counted, why they are counted that way, and lay out the facts and falsehoods associated with each of the claims above.
HOW DEATHS ARE COUNTED
The most important thing to understand is that there are two related, but separate ways deaths from disease outbreaks are counted. One is a daily reporting system for tracking epidemics in real time and the other is a longer term process of validating the total deaths over an annualized period (like for the flu) or for the course of a disease outbreak.
Daily monitoring of mortality data allows for both public and governmental entities to get hold of vital information to understanding spread of a disease outbreak and the most important way that this is done is through the surveillance of death certificates which are registered for every death occurring in the United States. All of the mortality data presented in the news, by Johns Hopkins, Worldometer, local and federal government are taken from death certificates. For decades, flu seasons have been tracked this way to get real time information on the severity and geographic distribution of outbreaks. See (Weekly U.S. Influenza Surveillance Report | CDC) for more information on how flu mortality is tracked.
Post-outbreak, there is a longer period of evaluation leading to final estimated death counts. This process involves a number of steps, but most important are the validation of the death certificate data and excess mortality modeling. The CDC reviews the incoming data for accuracy throughout the course of the disease and makes adjustments and updates as necessary. Because of this, CDC refers to their death counts as “provisional” until they are published. CDC provisional death counts are always several weeks out of date as a result. After the course of the disease is complete, a secondary process begins in which a total death toll is estimated. That process includes what’s known as excess mortality modeling. Because many deaths from disease outbreaks such as flu and COVID-19 may not be identified as such due to inadequate testing, people dying at home or dying from co-morbid conditions worsened by the disease, statistical methods are used which compare actual deaths to expected deaths during the same period from other relatively stable conditions such as stroke, heart attack, accidents, etc. This final estimate, generally published about two years later, becomes the official death count. With the flu, for example, confirmed cases of flu deaths tend to be four to six times smaller than the ultimately published number.
States and local authorities do NOT determine how to complete death certificates, That guidance comes directly from Health & Human Services and the CDC and they have issued specific rules for how to memorialize COVID-19 deaths. You can find them here: (COVID-19 Coding and Reporting Guidance - National Vital Statistics System)
Death certificates have two sections for conditions that could have led to the death of the individual. Part I of the death certificate is for conditions that were a part of the chain of events that led to the death. COVID-19 can be the ultimate cause of death or a step on the way to death. For example, COVID-19 might lead to Acute Respiratory Distress Syndrome which leads to death. In this case, COVID-19 is the underlying cause of death and ARDS is the immediate cause of death. All of those are in Part I. Part II of the death certificate is for other significant conditions which may or may not have contributed to death. In the example above, if the patient had multiple sclerosis for the last ten years that would be included in Part II. Part II conditions would not be generally be reported as causes of death in surveillance.
In order to better track real-time spread of COVID-19, two additional guidelines have been requested by the CDC. (1) Whenever there is a COVID-19 confirmed infection that must be included on the death certificate. If it is a contributory factor in the death it is in Part I of the death certificate. If it is not a contributory factor in the death, it listed in Part II of the death certificate. Only Part I conditions are considered for reporting of death counts. As with the case above, COVID-19 leading to ARDS leading to death is a Part I death included in the COVID-19 death count. A victim of homicide who turns out to be COVID-19 positive would have homicide in Part I and COVID-19 in part II and would NOT be a part of the COVID death count. (2) The second guideline is that here there is a strong probability that COVID-19 was a contributory factor to the death of the individual, but no test was taken it should be included in Part I of the death certificate as “Probable COVID-19.” In order to be listed as probable, the CDC guidelines require a situation in which “the circumstances are compelling within a reasonable degree of certainty.” Here are the specific guidelines from (https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/about-us-cases-deaths.html):
- Meeting clinical criteria AND epidemiologic evidence with no confirmatory laboratory testing performed for COVID-19
- Meeting presumptive laboratory evidence AND either clinical criteria OR epidemiologic evidence
- Meeting vital records criteria with no confirmatory laboratory testing performed for COVID19
An example of a probable COVID death provided by the CDC includes a person who died with high fever, acute dry cough and respiratory distress who had been in direct prolonged contact with a known positive COVID patient. These kinds of situations were more prevalent early on when tests were not widely available and were being reserved for live patients with symptoms. Probable cases are evaluated in the CDC provisional data, but may or may not be reported by states in the daily data despite CDC guidance that they do so. About half of the states report probable cases in their daily reporting. (https://www.washingtonpost.com/investigations/cdc-wants-states-to-count-probable-coronavirus-cases-and-deaths-but-most-arent-doing-it/2020/06/07/4aac9a58-9d0a-11ea-b60c-3be060a4f8e1_story.html)
With that background, let’s move on to the six claims listed above.
-
COVID death counts are greatly exaggerated because any death of a COVID positive person is being counted as a COVID death including such unrelated deaths as those from car accidents, homicides, etc.
UNTRUE. While specific guidance from the CDC requires that COVID-19 appears on the death certificate (causing the confusion), it is only considered in the COVID death count if COVID-19 is in Part I as a contributing cause of the death. A homicide, suicide, car accident, or other clearly unrelated cause of death in a person also positive for COVID-19 would NOT be included in the daily death reporting, or in the eventual published count a year or more after the end of the disease spread. Contributing to this false narrative is a true story that Washington state had been erroneously counting as coronavirus deaths anyone who had tested positive, regardless of cause. Authorities caught the error and corrected it on June 17th decreasing their death count by seven including three homicides, two suicides and two overdoses. That’s just seven out of over 1,200 total deaths in WA state at the time. There is no evidence of widespread misreporting of this nature, there are no states with policies that violate the CDC reporting guidelines, and there is virtually no chance that clerical errors could have a substantial impact on total death counts.
-
COVID death counts are greatly exaggerated because hospitals are financially incentivized to improperly mis-classify deaths as COVID.
UNTRUE. This started with an April 8th comment made by MN State Sen. Scott Jensen, (who is a physician) while being interviewed on Fox News by Laura Ingraham. He raised the idea that the number of COVID-19 deaths may be inflated, saying “Right now Medicare has determined that if you have a COVID-19 admission to the hospital, you’ll get paid $13,000. If that COVID-19 patient goes on a ventilator, you get $39,000, three times as much. Nobody can tell me after 35 years in the world of medicine that sometimes those kinds of things impact on what we do.” In a later interview, however, Jensen said he did not think that hospitals were intentionally misclassifying cases for financial reasons. Either way, none of the allegations made relate to the cause of death on the death certificate, only the diagnosis code listed as the reason for treatment. Further, doctors and medical examiners are responsible for filling out death certificates, not hospitals. Finally, it should be noted that the normal reimbursements under Medicare for respiratory infections and inflammations with major comorbidities or complications in 2017 was $13,297 and for more severe hospitalizations, typical Medicare payment with ventilator support was $40,218. The current payments to hospitals for COVID patients are essentially similar to typical rates.
-
COVID death counts are greatly exaggerated because probable COVID deaths are being reported along with confirmed COVID deaths.
UNTRUE: Only about half of the states even include probable deaths in their reporting despite CDC requests that they do so. Further, the proportion of probable deaths among all COVID deaths is very small. There is an excellent description here: (Is the Coronavirus Death Tally Inflated? Here’s Why Experts Say No - The New York Times)
-
COVID death counts (and cases) are being undercounted by the Administration through the change to the new reporting system moving data from CDC to HHS.
UNTRUE: The reporting system that changed from the CDC to HHS does not actually include either case or mortality data, so could not be responsible for changes in COVID death counts. That system only includes hospital data such as available beds, ICU beds, available medical staff, etc. For my details, see my earlier Fact Check on this subject at: Steven Millman - July 27th COVID Fact Check: Did the...
-
COVID death counts are being greatly undercounted because death certificates are missing people due to either inadequate testing and/or inaccurate attribution to another cause of death
PROBABLE: There is nothing unusual about death certificates from disease epidemics being undercounts of the true death rate as described above, which is one of the reasons it takes so long to publish official death counts from annual flu. There are many potential reasons for under-reporting including people who died from related conditions (such as heart or respiratory disease), missed diagnoses due to lack of testing (especially early in the pandemic). The Journal of the American Medical Association website (JAMANetwork) published research in July which estimated as much as a 35% undercount in cases at the time. (Excess Deaths From COVID-19 and Other Causes, March-April 2020 | Population Health | JAMA | JAMA Network). This is much lower than the typical undercount in deaths associated with flu, likely because of the greatly enhanced scrutiny of the novel COVID-19 pandemic. Another excellent treatment of why we are likely undercounting the number of deaths from COVID-19 and how many can be found on the CDC’s website at (Excess Deaths Associated with COVID-19).
-
COVID death counts are on the decline based on the CDC website data but rising in the Hopkins/Worldometer data, so one (or both) of those data sources must be lying.
UNTRUE: As described above, the CDC posts PROVISIONAL counts for COVID-19, pending review and validation. The deaths reported here are, by their nature, filled in over a much longer period of time than the daily reporting system. Because it can take weeks for the data to fill in completely, the CDC provisional data for the most recent time periods always presents lower than the weeks preceding. This is abundantly clear if you look at the CDC website were these data are posted. The CDC website makes the following statements about the provisional counts: (1) “Provisional counts are not final and are subject to change. Counts from previous weeks are continually revised as more records are received and processed.” (2) “Provisional data are not yet complete. Counts will not include all deaths that occurred during a given time period, especially for more recent periods. However, we can estimate how complete our numbers are by looking at the average number of deaths reported in previous years.” (3) “Death counts should not be compared across states. Some states report deaths on a daily basis, while other states report deaths weekly or monthly.” Provisional Death Counts for Coronavirus Disease 2019 (COVID-19)
