I read this statement by Senator Kerry in a New York Times article on regional disparities in Medicare spending earlier this week and predictably went crazy:
“There is too much uncertainty about the Dartmouth study to use it as a basis for public policy,” said Senator John Kerry, Democrat of Massachusetts. “Researchers can’t explain why some areas of the country spend more on health care than others. There are many reasons spending could vary: higher costs of living, sicker people or more teaching hospitals.”
“States like Massachusetts are concentrated centers of medical innovation where cutting-edge treatments are tested and some of the nation’s finest doctors are trained,” Mr. Kerry added. “This work might cost a little more, but it benefits the entire country.”
May I also add that sticking your head in the sand (a euphemism) and ignoring statistically significant differences in average spending per capita is no basis for public policy? We now turn to THE BEST BLOG POST I HAVE READ IN MANY YEARS by Maggie Mahar of the Health Beat Blog for a less snarky and more intelligent response to Kerry's quote:
No surprise, over more than thirty years, the Dartmouth researchers considered those possibilities. That is why the researchers adjust for differences in local prices, race, and the underlying health of population when making regional comparisons. Moreover, teaching hospitals do not invariably over-treat: one of Dartmouth’s landmark studies showed major differences in spending at Yale-New Haven and Harvard.
Kerry defends his home state. “States like Massachusetts are concentrated centers of medical innovation where cutting-edge treatments are tested and some of the nation’s finest doctors are trained. This work might cost a little more, but it benefits the entire country.”
Cost a little more? See this Boston Globe story about Partners HealthCare gouging patients in Massachusetts, charging far more for commonplace procedures while also ratcheting up volume. The Mayo Clinic is also a center of “medical innovation” and boasts “some of the nation’s finest doctors.” Yet it costs Medicare 50 percent less when patients are treated at Mayo than when very similar patients are treated at some of the nation’s most prestigious academic medical centers. And Mayo is not alone. The Cleveland Clinic, and the Geisinger Health System also serve as benchmarks for better care at a lower price.
Clearly, Kerry hasn’t read the research. Someone briefed him on it—and did it poorly.
Do Senator Kerry one better and read the whole thing -- as I said, the best blog post I have read in many years.
Dartmouth Study
Andrew,
As a Professor of Economics, I am sure you are familiar with data mining or trolling. Take a large data set, run it through a statistics program and you are very likely to find data correlations that pass all statistical tests. At a 95 percent confidence level, a 100 data runs looking for correlations will come up with five meaningless correlations just by chance.
Even two random data sets, such as sunspots and the stock market appear correlated.
Add to those findings that random data clusters and has false appearances of causation. In fact, if the data did not cluster, that alone would be enough to question the validity and usefulness of the data.
In addition, there may be regional differences in reporting and sampling errors in the used data.
While, I have very little familiarity with the Dartmouth Medical Study, my impression was that they tested all the data without having a prior, testable hypothesis about why regional medical cost difference occurred. They went data mining.
If the Dartmouth study is a valid study and it did find true (as opposed to just statistical) regional differences in medical costs, the researchers should be able to develop a forecast of what regional costs differences will be going forward. Testing their projections against out of sample data would add a lot more creditability to their findings.
May be they have done that testing already, but as I said I am not familiar with their work. However, I have not seen anybody mention a regional cost difference forecast accuracy test based on the Dartmouth research. Until, that time, I will stick with John Kerry's doubts about the usefulness of the Dartmouth study.
Dear Prior Poster 6/11
Dear Prior Poster 6/11 14:30,
Suggesting and merely referencing the subject of data mining does not bring credibility to your accusation that the study comes to no viable conclusion. Yes, we've all read Freconomics and Taleb. They speak about the misunderstoond and fear of unknown unknown, not the things we know with certainty supported by a hypothesis with statistical significance.
I understand you've taken probably a few economics courses and maybe even a philosophy class or two, but as for why you are commenting about a study you "have very little familiarity with" anyway, it is beyond me. Here's my hypothesis: it's the ability to post Anonymously. No credable person would stand in front of a room and lecture NASA about Hubble findings if they have been selling knives for their entire life.
The same arguement you use to refute the study negates your own assertion about being able to predict the future; studying the past is completely seperate from being able to predict the future. We can say with 100% certainty that they sun rose yesterday. And the day before. And the day before. Yet none of us cay say with 100% certainty that the sun will rise tomorrow.
Kerry
I wouldn't expect more from a person who has made his millions from ambulance chasing and legislates to win votes rather than help the common good (and ultimately defend our liberties).
Yes, we've all read
Yes, we've all read Freconomics and Taleb. They speak about the misunderstoond and fear of unknown unknown, not the buy steroids things we know with certainty supported by a hypothesis with statistical significance.
I must read too many mystery novels
This is easy: "Follow the money." Sen. Kerry must reflexively defend the hospitals and universities in Massachusetts, facts be damned. No democratic politician could be elected without their support and the money from their highly paid, mostly liberal, employees.
Hospitals had been shielded from comparisons for many years, but most of those walls are breaking down: first outcome research, and now tying costs to outcomes, has become publicly available data. So there's no wonder that some institutions will become defensive.
Now, they might be right about some of their arguments (the chance that I will read those materials is vanishingly close to zero) but I don't need to read it to know why Kerry felt the need to try to marginalize it, even at the risk of making patently false assertions. He needs their money, so he will be a ventriloquist dummy for their lobbyists.