Why Are Hospital Ratings All Over the Map?

But those ratings, each using its own methodology, often come to wildly divergent conclusions, sometimes providing as much confusion as clarity for consumers. Some hospitals rated as outstanding by one group are ignored or panned by another. Ratings results from an individual group can change significantly from year to year."We've alternatively been labeled the least safe hospital in Maine and the safest hospital in Maine," said Dr. Douglas Salvador, vice president of quality at Maine Medical Center in Portland.

And the ratings do not always jibe with the views of authorities who oversee hospitals. For instance, UCSF Medical Center has gotten good grades from multiple safety raters even as California public health officials have fined it $425,000 repeatedly for endangering patients.

As ratings multiply, more and more hospitals have something they can boast about. A third of U.S. hospitals-more than 1,600 - last year won at least one distinction from a major rating group or company, according to a Kaiser Health News analysis. In the greater Fort Lauderdale hospital market, 21 of 24 hospitals were singled out as exemplary by at least one rating source. In the Baltimore region, 19 out of 22 hospitals won an award.

"I worry a lot about these ratings," said Jerod Loeb, executive vice president for health care quality evaluation at the Joint Commission. "They're all justifiable efforts to provide information, but at the end of the day every single one of them is flawed in some respect. Rather than enlightening, we may be confusing."

Not A 'Complete Picture Of The Care'

There are so many report cards on hospitals that the Informed Patient Institute runs a website that grades the raters. Carol Cronin, its executive director, said most report cards are not easy for consumers to use. "A lot of them don't help users quickly understand which hospital is better than another," she said.

But many hospitals are eager to trump these distinctions in their marketing. Healthgrades, U.S. News and Leapfrog not only encourage this but also profit from it by charging licensing fees to hospitals that want to advertise their awards. "A hospital cannot buy an award, they must achieve it," Healthgrades said in a statement.

Dr. Andrew Brotman, chief clinical officer at NYU Langone Medical Center in Manhattan, said the fees can be substantial. "Healthgrades, which is one we did well on, charges $145,000 to use this even on the website as a logo, so we don't do that," he said. "U.S. News is in the $50,000 range. Leapfrog is $12,500."

Healthgrades and Truven Health Analytics, which publishes the 100 Top Hospitals, offer consulting services to hospitals that want to improve their overall performances. Jean Chenoweth, a Truven senior vice president, said the list doesn't earn Truven any money but it "gives the company a lot of visibility."

Consumer Reports bars hospitals from using its ratings in marketing, but patients must subscribe to read them online. (Others generally provide free access to ratings on their sites.) The Joint Commission does not charge hospitals that make its top quality list.

A Pew Research Center survey found 14 percent of Internet users consulted online rankings or reviews of hospitals and medical facilities. Florence Harvey, 70, said when she moved to Washington, D.C., last fall, she picked a health plan and doctor affiliated with Washington Hospital Center after reviewing all the local hospitals rankings on U.S. News' website. "That was the one that had the best across-the-board ratings," she said.

But Harvey may be an anomaly. Dr. Peter Lindenauer, a professor with Tufts University School of Medicine based in Springfield, Mass., said the limited research on rankings "suggests they have had very little impact on patient behavior."

That's not surprising since many admissions, such as those due to a heart attack or car crash, have an immediacy that rules out comparison shopping. Also, researchers note, many patients defer to their physicians' recommendations or go to the hospital where their chosen surgeon has privileges. Still, rating groups say the ratings help keep the pressure on hospital executives to keep quality up.

"Patient safety has to be a priority 24-7," said Leah Binder, Leapfrog's president. "The minute it slips off the priority list, that impacts the rating."

The calculations that go into these ratings are complex. Most hospital assessments synthesize dozens of  pieces of data Medicare publishes on its Hospital Compare website, including death rates and the results of patient satisfaction surveys. They also examine other sources and use private surveys to create user-friendly lists or grades, which they display on their websites.

The Joint Commission looks at how frequently patients received recommended treatments, such as flu shots for those with pneumonia. Consumer Reports examines the numbers of patients who die or are readmitted, infection rates and Medicare patient surveys of their experiences. Leapfrog looks at data from its surveys of hospitals, the consistency with which hospitals followed safe surgical practices and frequencies of infections and some types of patient harm. Healthgrades analyzes detailed Medicare records to find death and complication rates for 27 procedures and conditions.

Truven considers profitability along with quality in its assessments. U.S. News surveys physicians about which places they think are the best, and those reputation ratings account for a third of most of its assessments. U.S. News ranks hospitals by geography and singles out hospitals that do well in 16 specialties, including cancer, neurology and orthopedics.

"Ratings and ranking programs certainly offer people information they can use to make their hospital selections, but we don't recommend relying on any one of them completely," Jennifer Kennedy, a spokeswoman for St. Mary Mercy, said in an e-mail. "None are able to tell the whole story or paint a complete picture of the care that is delivered."

Patient safety rankings are based on how frequently correct procedures or errors occur, so hospitals can get good grades even if they have made some egregious errors. Leapfrog gave a "B" to UCSF Medical Center in San Francisco even though California regulators have penalized UCSF eight times for infractions since 2008, most recently for leaving a sponge in one patient and a plastic clip in the skull of another. Those errors cost UCSF $200,000 in fines.

Dr. Josh Adler, the chief medical officer at UCSF, said penalties were partly a result of the hospital's policy to make sure that all errors are reported to authorities. "The key is that we constantly strive to deliver the highest quality, safest, and most satisfying care, and that we are a learning organization," he said in an e-mail.

The ratings groups believe the public benefits from the multitude of ratings. Dr. John Santa, who directs Consumer Reports' health ratings, said consumers benefit from different vantages just as they do for cars or electronic devices, and the competition spurs each rating group to get better. "We think that's consistent with good science," Santa said.

Avery Comarow, health rankings editor for U.S. News, agrees. "People go to hospitals for different reasons and priorities," he said. "I'm not sure there could be a single rating system that can do it all."

Many of the hospital graders are expanding their awards. Last year, the Joint Commission identified 620 hospitals as "top performers," up from 406 the previous year. Healthgrades now provides awards for emergency rooms, maternity care, pediatric care, bariatric surgery and gynecology surgery. In 2011, U.S. News started identifying the best hospitals in regions of the country, and identifies 748 hospitals as a "best" hospital in at least one specialty.

But because of limitations in data, the ratings cannot always offer patients the kind of specificity they seek. When Kathy Day of Bangor, Maine, needed a hysterectomy in 2011, she wanted to compare hospital infection rates for the procedure. But she said when she called Brigham and Women's Hospital in Boston, "the response I got from them was, 'We don't have to give you that information, we're not required to report those infections until next year.' "

"I said, 'I have cancer this year and I need surgery now, so next year doesn't help me,'" recalled Day, 63, a registered nurse and consumer advocate. She said the hospital ultimately told her its infection rates were average, but the attitude turned her off so much that she underwent surgery successfully at Maine Medical Center.

Brigham and Women's website publishes six types of infection rates, but not one for hysterectomies. "We are continually working on making data about more types of infections available," a hospital spokesman, Tom Langford, said in an e-mail.

Constant Turnover

Much of the quality data is rudimentary, as the science of evaluating hospitals is still in its adolescence. Adding to the confusion is that hospitals can rise and fall from year to year as groups tweak their methods of assessment and as hospital performance shifts, even slightly. A study sponsored by the Commonwealth Fund, a health care philanthropy in New York, found that only 46 percent of hospitals ranked as top performers by Thomson Reuters in 2008 were also winners in 2007. In the Joint Commission's rankings, 583 hospitals missed being designated a top performer because they fell short on just one of 45 measures.

In some cases, hospitals that have won awards are being penalized financially by Medicare for falling short on the government's quality assessment. Saint Francis Hospital in Tulsa, Okla., which is losing 0.54 percent of its Medicare payments this year under the government's quality program, is a Healthgrades Distinguished Hospital for Clinical Excellence and was ranked the second best hospital in Oklahoma by U.S. News.

Paul Levy, the former CEO of Beth Israel Deaconess Medical Center, said he is concerned that as awards multiply they may encourage complacency. "There's a danger," he said, "that some hospitals look at their excellent ranking and say 'See, we're there, we're done,' while process improving has got to be a never-ending philosophy."

Some of the hospitals that do the best in the rankings have limited respect for them. Advocate Christ Medical Center in Oak Lawn, Ill., last year received praise from Leapfrog, U.S. News, the Joint Commission, Truven and Healthgrades. But Dr. William Adair, vice president for clinical transformation, says the hospital doesn't license any of the distinctions. "We're all made a little bit uneasy, to be frank about it," Adair says. "Some of these organizations are looking for revenue. It blurs the effectiveness of the ratings processes."

Still, many hospitals are happy to use the praise. Dr. Brotman from NYU said: "Even though there's not a hospital executive who won't tell you that they have a great deal of skepticism about a lot of the methodology, there's not one who will tell you they don't want to be on the lists."