"Differences in the measures, data sources, and scoring methodologies produce contradictory results that lead to confusion for the public, providers, and governing boards, and impair the public's ability to make well-informed choices about healthcare providers," the AAMC document says.
"For data to be understood and for results to be comparable, publicly reported data should adhere to a set of guiding principles."
The guidelines suggest that hospitals look at each scorecard's merits on the validity of measures used, transparency about methodology, and purpose. For example does the scorecard specify its intended audience? Are the measures used endorsed by the National Quality Forum? Are the outcome results risk-adjusted?
'A Lot of Commercial Interest'
Does the hospital have a business relationship with the ranking company?
"Quite a few of these are what I call commercial scorecards that offer services to help improve you, and to me that's a conflict," says Michael Henderson, MD, chief quality officer for the Cleveland Clinic and a member of the committee that compiled Guiding Principles. "Because guess what? Those who take their services go up to the top of their rankings pretty quickly."
It's confusing for everybody, Henderson says, because it seems that all hospitals are number one on someone's "best" or "top" list.
Henderson says that hospitals just have to get used to the noise. "I think it's going to be exceedingly difficult to get a single agreed-upon way of doing this, because there's a lot of commercial interest in these scorecards out there, and to break that will be very hard to do."