To truly evaluate quality, hospitals would do better to look at avoidable events such as bloodstream infections, which kill 31,000 people a year in the U.S., rather than at mortality rates, a prominent safety expert wrote in a new report.
That's because all too often, patients arrive at the hospital too sick to be saved by the time they are admitted, said Peter Pronovost, MD, professor of anesthesiology and critical care medicine at the Johns Hopkins University School of Medicine.
Mortality could be one factor considered in rating some divisions of quality care as good or bad, he says, but it should not be the only outcome measure.
Pronovost authored the paper "Using hospital mortality rates to judge hospital performance: A bad idea that just won't go away" in the British Medical Journal along with Richard Lilford, professor of clinical epidemiology at the University of Birmingham in England.
When statistical analyses compare hospitals, they usually use a risk adjustment formula. But that doesn't work, the authors said, for two reasons.
"First, risk adjustment can only adjust for factors that can be identified and measured accurately . . . Secondly, risk adjustment can exaggerate the very bias that it is intended to reduce," they wrote.
"For example, if diabetes is a more powerful prognostic factor in Glasgow than in Four Oaks, then adjusting for the average effect of diabetes will deflate expected diabetic deaths in Glasgow and inflate them in Four Oaks....This effect could tilt the playing field against Glasgow."
A key point the authors make is that there is little correlation between how well a hospital performs on one standard of safe and effective care and how well it does on another. "While commercial organisations such as Enron fail corporately, hospitals are more likely to fail on specifics–pathology in Liverpool; paediatric cardiac surgery in Bristol; radiation therapy in Missouri."
Mortality rates for the hospital overall "are silent about where any problem might lie. This combination of unfairness and non-specificity is a toxic mix, inducing what has been called ‘institutional stigma' "
Pronovost and Lilford suggested that when they know they are being graded, officials may resort to "gaming the system – for example, by upgrading risk assessments. Furthermore, a focus on hospital mortality may lead to overly aggressive care, which is inhumane and drives up costs."