Four popular national rating systems used by consumers to judge hospitals frequently come to very different conclusions about which hospitals are the best — or worst — potentially adding to the confusion over health care quality, rather than alleviating it, a new study shows.
The analysis, published on Monday in the academic journal Health Affairs, looked at hospital ratings from two publications, U.S. News & World Report and Consumer Reports; Healthgrades, a Denver company; and the Leapfrog Group, an employer-financed nonprofit organization.
No hospital was considered to be a high performer by all four, according to the study of the ratings from mid-2012 to mid-2013, and the vast majority of hospitals earned that distinction from only one of the four.
Some hospitals were even designated as a high performer by one group and a low performer by another.
“The complexity and opacity of the ratings is likely to cause confusion instead of driving patients and purchasers to higher-quality, safer care,” the authors said.
Consumers are increasingly in need of just such information on hospital quality. With the growth of narrow networks of hospitals, where the choices for patients are sharply limited, and high-deductible plans that require consumers to pay more of their medical bills, people want better ways to choose among facilities.
The Health Affairs analysis was done by a group of well-respected researchers, all of whom have recently served as experts for the Leapfrog Group. Leapfrog did not finance the study, and the study does not single out any one rating system as a model. The researchers include Dr. Ashish K. Jha, a professor of health policy at Harvard, and Dr. Robert M. Wachter, a professor at the University of California, San Francisco.
The rating groups say they are serving different purposes, which they say consumers understand.
“They’re not measuring the same thing,” said Evan Marks, the chief strategy officer for Healthgrades, who says that thinking the groups should agree would be like someone expecting car-safety rankings and the ratings in Car & Driver to be identical. “Why would they overlap?” he asked.
U.S. News & World Report, for example, says it tries to help patients find the best hospital within a given specialty that excels at treating the most complicated patients. The magazine does not currently evaluate maternity care, for example. Consumer Reports and Leapfrog focus on patient safety, while the Healthgrades list of best hospitals relies on clinical information to categorize facilities.
Leah Binder, the chief executive of the Leapfrog Group, says consumers want different types of ratings to better reflect their own priorities. “There’s room for many voices,” she said, and she played down the notion there might be confusion about the ratings. “We can all look at that information and judge it for ourselves,” she said.
While the researchers acknowledge the systems have different aims, they also say the wide variation reflects the difficulty in assigning an overall grade or categorizing any single hospital as the best over all.
“There are relatively few hospitals that excel or struggle on all measures,” J. Matthew Austin, an assistant professor at the Johns Hopkins University School of Medicine and one of the study’s authors, said in an interview.
The challenge for any rating system, he said, is to find an easy-to-understand grade or category that still captures the nuances of a single institution, which may be strong in cardiac care but have a high hospital infection rate or be weak in joint replacements.
And there is little agreement among insurers and others about what quality measures to use and how to use them. A hospital’s mortality rate, for example, is typically adjusted by how sick its patients are, but those adjustments can vary widely, depending on how they are done. “It’s a little bit of the Wild West in terms of performance measurement,” Mr. Austin said.
At Consumer Reports, the purpose of the ratings is to help patients talk with doctors and nurses about their care, said Doris Peter, the director of the health care ratings center.
While consumers rely on the lowest ratings to avoid hospitals that appear unsafe, she said the scores were also intended to put patients on alert. If a hospital is categorized as poor at communicating with patients what drugs they should be taking, patients need to spend more time making sure they understand the instructions.
The researchers say the ratings are useful, but Mr. Austin also emphasized that consumers needed to better understand the focus of the rating systems and what their limitations are, and the systems need to do more to communicate the goals of the individual ratings. “These ratings systems have been a great start, and we have a way to go,” he said.
The demand for more and better information is likely to keep increasing, said Ms. Binder of Leapfrog. “We don’t have anywhere near as much information as consumers want,” she said. “This is a major issue.”