Print      
reasons to take hospital ratings with a big grain of salt 5
Massachusetts General Hospital was third in U.S. News & World Report’s rankings. (Steven Senne/associated press/file)
By Casey Ross
STAT

It’s hospital ratings season in America, that time of year when marketing executives kick it into high gear to trumpet — and spin — the way their hospitals are graded by outside organizations.

This year, their workload has been especially heavy. In addition to annual rankings released Tuesday by U.S. News & World Report, the federal Centers for Medicare and Medicaid Services (CMS) just released its star-rating system for hospitals nationwide. While the ratings offer some valuable information to consumers, here’s why you shouldn’t put too much stock in the results.

1 No matter how objective the metrics, ratings are inherently subjective

The federal government and private organizations use a vast array of hospital data to break down their performance and boil it into easy-to-understand ratings for consumers. U.S. News & World Report formulates its rankings by relying on a mix of physician surveys and data reported to trade groups and the federal government. CMS ratings are based solely on data hospitals are required to report to the agency.

The information itself may be objective, but which data points are used — and how they are weighted — can lead to contradictory conclusions. Consider the following:

Of U.S. News & World Report’s’ top five hospitals, only one — Mayo Clinic — received a top, five-star rating from CMS. The rest — Cleveland Clinic, Massachusetts General Hospital, UCLA Medical Center, and Johns Hopkins — were not even in CMS’ top 100. (Johns Hopkins did not receive a rating from CMS because of incomplete data).

J.B. Silvers, a professor of health care finance at Case Western Reserve University, said the information, while boiled down into simple lists and stars, can be difficult for consumers to unpack, especially when hospitals are working behind the scenes to figure out ways to better their scores every year. “Some hospitals are better at working the numbers than others,’’ he said. “My guess is the safety net hospitals aren’t as good at it as richer hospitals.’’

2 One bad score, and your rating is shot

Lobbyists and some hospital executives argue that the CMS new star-rating system penalizes providers that treat greater numbers of low-income patients. That’s because the CMS system hurts hospitals with higher rates of readmission, a factor heavily influenced by a patient’s economic status.

Critics have noted that both teaching hospitals and safety net hospitals — both of which tend to deal more with sick, low-income patients — fared poorly in CMS ratings.

“On its surface that says, ‘Hey, you’ve got a bias,’’’ said Silvers. “Big teaching hospitals are taking care of the most complex cases. Hardly any of them are in the five-star category. Most of them are in the one-star category.’’

3 What’s not in the ratings matters, too

Ratings systems capture huge amounts of information on everything from whether doctors are washing their hands to mortality rates for heart attack patients. But plenty of information is not collected that might help provide a more nuanced — and accurate — assessment of performance. One hospital quality specialist noted, for example, that the severity of a cancer patient’s disease is not typically recorded by hospitals in diagnostic information and doesn’t get factored into outcome data and ratings. This can unfairly penalize hospitals that treat large numbers of patients with advanced cancers.

“Stage 4 cancer patients are more likely to die than stage 1 cancer patients,’’ said Harold Miller, chief executive of the Center for Health Care Quality and Payment Reform. “But if you don’t know that information — and you’re measuring mortality rates — then you are going to get a bad answer.’’

4 Gaming the system

Hospitals routinely launch initiatives to improve scores in particular ratings categories. They can also change how they parse and report certain information to improve their results. In some cases, those adjustments can mean positive improvements in the care provided to patients. In others, they can skew the results or cause hospitals to avoid treating some patients, altogether.

For example, a recent study by the American College of Surgeons found that organ transplant centers are removing the sickest patients from their waiting lists because of concerns that a negative outcome could harm their ratings and lead to federal scrutiny.

“You may decide not to take patients who are at risk of bad outcomes if somebody’s going to start rating you on your outcomes without adjusting properly for that,’’ Miller said. “Those are the patients that need the hospitals the most, and they are the ones that are going to potentially get disadvantaged by this.’’

5 The marketing agenda

Hospital ratings — particularly the results published by U.S. News & World Report — are often used for hospital advertising campaigns. Most institutions ranked in the top 10 will hype their ratings success on billboards, banners, and brochures that get blasted across the regions they serve.

But the glossy mass media messages offer little valuable information to consumers trying to figure out where to get treatment for a broken leg or have gallstones removed because common procedures are seldom evaluated. “It could be the hospital is bad on some things, and really good on other things,’’ Miller noted. “But if you’re the patient, how do you know? You might be going to the hospital for something it’s really good at, but you can’t always tell that from the ratings.’’

Casey Ross can be reached at casey.ross@statnews.com. Follow Casey on Twitter @byCaseyRoss