SCIENTIFIC DEBUNKING OF LANCET STUDY: DOES IT REALLY MATTER?
I was pleased to see that someone decided to spend the time and energy to scientifically debunk the politically motivated statistical study on deaths in Iraq since the invasion published by the Lancet just days before the 2004 election.
First of all, it is important that these charlatans be exposed for the scientific hacks they are. Dr. Les Brown, an epidemiologist, headed the 2004 study which estimated 100,000 or more excess Iraqis had died as a result of our invasion and occupation. What should have been the tip off to the study’s uselessness was the contention that “most of the excess deaths” were the result of violence and that “80% of those deaths were the result of air strikes.”
Unless the US was carrying on a massive bombing campaign that killed tens of thousands of civilians without the media, the UN, the Iraqis themselves, or anyone else knowing anything about it, that statement was either a laughable corruption of statistics or a bald faced lie.
And given this thorough destruction of the study by David Kane, Institute Fellow at the Institute for Quantitative Social Science at Harvard University, the latter explanation may be the most logical.
Much of the math here is mind-numbingly complicated, but Kane’s bottom line is simple: the Lancet authors “cannot reject the null hypothesis that mortality in Iraq is unchanged.†Translation: according to Kane, the confidence interval for the Lancet authors’ main finding is wrong. Had the authors calculated the confidence interval correctly, Kane asserts that they would have failed to identify a statistically significant increase in risk of death in Iraq, let alone the widely-reported 98,000 excess civilian deaths.
An interesting side note: as Kane observes in his paper, the Lancet authors “refuse to provide anyone with the underlying data (or even a precise description of the actual methodology).†The researchers did release some high-level summary data in highly aggregated form (see here), but they released neither the detailed interviewee-level data nor the programming code that would be necessary to replicate their results.
Failing to provide the detailed interviewee-level data and the programming code so that colleagues could duplicate their results thus validating the study is a clear indication that Brown and his crew could have cared less if the study was accurate or even scientifically useful. It is an open question whether they knew the study was flawed which would make their sin a mortal one for a scientist, a transgression that would get you fired from any respectable scientific institution in the world and leave your career in tatters.
The study was a political statement - propaganda in service to people that Brown, whose work was most praiseworthy in Rwanda, should have recognized as kin to the genocidal maniacs who hacked 800,000 tribesmen to death in the 1990’s. The beheaders and mass murderers that we are fighting in Iraq were aided by this study. And Brown and his team should be abjectly ashamed of themselves for knowingly giving them assistance and comfort.
This ethical transgression by Brown should finish his career. Instead, don’t be surprised if he gets the Nobel Peace Prize.
And what of the Lancet? Publishing the study 5 days before the presidential election and then claiming that the publication date was only a coincidence exposes them as frauds and liars. One of the oldest and most respected medical journals on the planet was put in service of a partisan political agenda and in a most cowardly manner, denied it’s motives were anything except pure as the driven snow.
Outrageous.
As we have seen with the Bush Administration, politically motivated science put in service to a specific agenda is extraordinarily damaging. For the Bushies, who have no respect for science in my opinion and see it as a tool to be used to advance their political agenda, everything from the public health to climate change was affected by their cooking the books. But Brown and The Lancet went the Bush Administration one better; they put themselves and their scientific expertise at the disposal of the enemies of civilization. They allowed their animus toward the war, or Bush, or the United States to blind them to the fact that by hurting America’s cause they were helping those who, if given the chance, would just as soon put a bullet in their brains as give them the time of day. It makes no sense.
In the end, this is an esoteric argument. Tens of thousands of Iraqis are dead, most of them innocent women and children. And while it’s true that insurgents and terrorists use civilians as human shields, it is also true that no study, no argument can be made to really defend or obscure the fact that for many Iraqis, this war has been a personal tragedy beyond their ability to bear. Loved ones who have died in crossfire or because they were in the wrong place at the wrong time when a car bomb went off, or simply because a mistake was made by American forces are lost forever. They cannot be brought back by bogus studies or “supporting the troops” or “winning through to victory” or political posturing here at home. Dead is dead. And we don’t need cooked statistics published by ethically challenged journals to tell us of the immense pain and human toll our war of choice is costing the Iraqi people.
Iraq is an open wound, bleeding as a result of our ministrations. Even though the surge is showing some signs of success in some areas - less so in others, the political differences that divide the country are a chasm that no one seems willing or able to bridge. Until the Iraqis decide they wish to live together in peace, the body count will continue to rise. The only question is will more die if we leave than if we stay.
And no one knows the answer - no one has any answers that would allow us the luxury of a quick exit.
UPDATE
Vindication for Shannon Love of Chicago Boyz whose series of posts on the study back in 2004 I relied on for my own piece questioning the study.
Kane shows that if the Falluja cluster is included in the statistical calculations, the confidence interval dips below zero, which is a big no-no. Since the study’s raw data remain a closely guarded secret, Kane cannot be absolutely certain that the inclusion of the Falluja cluster renders the study mathematically invalid…
…but that’s the way to bet.
In science, replication is the iron test. I find it revealing that no other source or study has come close to replicating the original study. All my original points still stand.
Ah, vindication is sweet.
