It’s interesting to look for examples of continuous improvement ideas on display in our daily lives. A headline from NBC News on May 29th, 2018 provided provided a good example. The headline read: Hurricane Maria death toll in Puerto Rico is thousands higher than official count, study estimates.
On September 20, 2017 Hurricane Maria made landfall in Puerto Rico as a Category 4 hurricane. Damages from the storm were estimated at $45 to $95 billion. The energy grid was heavily damaged and the island was without power for an extended period (in fact many areas are still facing power issues).
Using a “Standard”
The Department of Public Safety (DPS) was responsible for reporting deaths related to the hurricane. The Department apparently uses an international standard for the reporting of deaths. The standard defines “Direct” vs “Indirect” deaths:
- Direct Death — A death directly attributable to the forces of the disaster or by the direct consequences of these forces, such as structural collapse, flying debris, or radiation exposure.
- Indirect Death — A death which occurs when the unsafe or unhealthy conditions present during any phase of the disaster (i.e., pre-event or preparations, during the actual occurrence, or post-event during cleanup after a disaster) contribute to a death.
By December 10th, the DPS had raised the death toll to 64, most of which were “direct deaths.” This is where the Operational Definition debate begins. For those unfamiliar, an Operational Definition is a detailed description that defines a measure to such a degree that everyone collects data the same way. They are critical to maintaining the integrity of data collection. If two people are counting spelling mistakes and one of them includes grammar mistakes, the results won’t match and it will be hard to know where to focus improvement efforts.
For those unfamiliar, an Operational Definition is a detailed description that defines a measure to such a degree that everyone collects data the same way. They are critical to maintaining the integrity of data collection.
Let me be clear — there is no such thing as a “proper” Operational Definition. What matters is that everyone uses the same definition. Also, given the example we’re using, any death due to this disaster is a terrible thing. In terms of deriving estimates, the debate is about which deaths were caused by the hurricane.
Using a Personal Perspective
In November, CNN contacted 279 funeral homes and reached about half of them. They asked funeral directors how many deaths they were dealing with that were related to the hurricane. These funeral directors reported a total of 499. In this case, each funeral director was providing their personal perspective (definition) of whether the death was caused by the hurricane.
It is easy to assert that the Department of Public Safety’s count of 64 was vastly understated. But their count was the number of reported deaths on the island based on the stated definitions in the standard. Keep in mind the extreme reporting environment following the disaster. Most of the island was without power. The focus, as it should be, was on serving the survivors of the disaster, not reporting casualties.
A Definition Using Comparative Data
The headlines of the undercount were due to a recently released Harvard study as reported in the New England Journal of Medicine. In the study, the researchers analyzed the number of deaths on the island from September and October for each of the years from 2010 through 2016. By comparing the average for the period to the same period of 2017, they found the number of deaths had increased by 4,645 – a 62% increase over the previous years’ average. But again, this is simply another Operational Definition. It assumes the number of deaths year to year is “stable” and that all of the increase can be attributed to the storm.
Different Definitions — Different Counts
Several other entities provided estimates for the number of deaths. Each providing a different result.
Lots of questions regarding their Operational Definitions come to mind. As you consider the following, think of them being placed on a Cause & Effect Diagram. What would you list as a potential cause of death:
- Due to power outages — Many people who had a need for a specific medical procedure were unable to receive treatment (for example, dialysis, people needing respiratory assistance). If they died within a month or two of the hurricane, would you consider their death to be caused by the disaster?
- A heart attack due to the trauma of the event — Would you consider their death to be caused by the disaster? How about someone who had a heart attack that was not specifically trauma related, but the person was unable to get medical assistance in time due to the loss of communication systems. Would they be counted?
Under the international standard, as published, probably none of these deaths would be classified as “hurricane caused.” Rather they would be listed as due to kidney, pulmonary or cardiovascular conditions. Hence the huge difference in estimates.
Do the Difference in Estimates Matter?
But the next big question is, “What difference does this estimate make?” Many of those reporting on the difference in estimates say the number of deaths reported greatly impacts the amount of financial and medical support the area receives following the event. The more deaths that occur, the more aid that’s needed.
According to an article published by theconversation.com:
“Determining the number of excess deaths after a natural disaster is not only a mathematical exercise. Undercounting deaths reduces the attention to the crisis Puerto Ricans live day by day. It can also delay international recovery efforts and the approval of policies to help those who need it the most.”
The intent is not to engage in debate. The point is that having an Operational Definition that people basically agree on goes a long way toward improving the discussion about what should be done.
Improving the Estimates
Because of the large variation in the number of deaths reported, an additional study has been commissioned and is being conducted by George Washington University. Following this event, the Center for Disease Control will publish new guidelines for states and territories to follow. Hopefully, this will result in improved reporting. But, as with every Operational Definition, it won’t be perfect and people will continue to debate which deaths should or should not be counted as part of the event.
