What’s up with the folks at the journal Health Affairs? Do they actually read the studies they publish? It’s hard to imagine that they do since they let the authors get away with outrageous claims based on poor quality data.
Last year I wrote about a paper on C-sections that was misleading and left out important data. The paper was Cesarean Delivery Rates Vary Tenfold Among US Hospitals; Reducing Variation May Address Quality And Cost Issues by Kozhimannil et al. There was just one problem; that’s NOT what the data actually showed.
Yes, it is true that the rate at the hospital that did the greatest proportion of C-section is 10 times higher than the rate at the hospital that did the lowest proportion of C-sections, but looking at the actual data shows that both hospitals are outliers. In fact, more than 90% of hospitals had C-section rates between 21%-44%. That’s still an appreciable difference (double), but very far from the 10 fold difference touted by the authors.
Now comes the latest paper playing fast and loose with the truth, Rates Of Major Obstetrical Complications Vary Almost Fivefold Among US Hospitals by Glance et al. You might think that means that the rates of major obstetrical complications vary almost 5-fold, but you’d be wrong.
What did the study actually show? We’ll get to that, but first take a look at the authors’ disclaimer unusually positioned before the results themselves:
… First, maternal health care must balance the needs of the mother and the fetus. Our data did not allow us to link maternal and newborn records to simultaneously examine the outcomes of mother and child.
Second, administrative data have been used to examine health care quality over the past three decades. However, these data have significant limitations, including the lack of important risk
factors (such as parity—that is, the number of pregnancies carried to birth—and body mass index), laboratory values, and information on other diagnostic tests; problems with coding accuracy
(for example, the extent to which comorbidities and complications are properly coded); and variability in data quality across hospitals.Third, it is likely that some of the variation in outcomes was the result of residual confounding caused by differences in unmeasured risk factors or reporting across hospitals…
In other words, they have no idea of the tradeoffs that led to the treatment decisions, there was no way to adjust for the risk status of the patients, and there may be variations in reporting across hospitals. Even the authors think that their conclusions should be taken with a ton of salt, and that’s before you look at how they manipulated the data to reach the alarming, and thoroughly unjustified, conclusion of the title.
What did the authors look at?
Our primary outcome of interest was a composite complication outcome that consisted of maternal hemorrhage; laceration or operative complication; infection; and all other complications, such as thrombotic complications.
Any vaginal tear is a major complication? No, 1st and 2nd degree lacerations are minor complications and quite common; many do not even require stitches. To include them in “major complications” is irresponsible and indefensible.
An obstetric hemorrhage is a serious complication? Not if it doesn’t require a blood transfusion, it isn’t. Indeed many studies looking at maternal morbidity do not consider any hemorrhage that does not require 2 or even 3 units of blood to be a serious complication.
Moreover, I’m always suspicious of composite indices in clinical investigations. Amalgamating the results from disparate outcomes into a “composite index” can hide a myriad of heterogeneous findings and lead to faulty conclusions. That’s exactly what happened here.
Take a look at the author’s table of results of the various individual outcomes as well as the composite outcomes.
In the entire table, encompassing multiple outcomes of vaginal births and multiple outcomes of Cesarean birth, there is only ONE individual complication that is more than 5X higher in low quality hospitals than in high quality hospitals (outlined in red on the table below). Every other complication has variations that don’t even come close to 5 fold. But since the rate of hemorrhage is so much higher than the rate of all other complications (except lacerations in vaginal births) that it makes up an outsize portion of the composite index.
The authors claim to have found:
Women undergoing a cesarean delivery at a low-performing hospital were nearly five times more likely to experience a major complication (20.93 percent) than women undergoing a
cesarean delivery at a high-performing hospital (4.37 percent) (Exhibit 1).
That conclusion is deeply misleading and therefore irresponsible.
The authors ACTUALLY found that the rate of peri-Cesarean hemorrhage was 6.5X higher in low performing hospitals than in high performing hospitals. Every other possible complication, including every complication of vaginal birth, and every other complication of Cesarean differed by approximately 2 fold. And we don’t even know if these hemorrhages were serious complications or merely above average blood loss at the time of surgery.
In other words, the title of the paper, Rates Of Major Obstetrical Complications Vary Almost Fivefold Among US Hospitals, is a lie.
Only ONE complication (not complications) differed by 5 fold, and maybe not even that since all obstetric hemorrhages are treated as serious complications even though most of them are not. Everything else did not differ by anywhere close to 5 fold.
Apparently the folks at Consumer Reports don’t read the scientific papers they report on, either. They, too, have a indefensibly misleading headline, Having a baby in the U.S. is more dangerous than you might think.
In addition to promoting the grossly inappropriate conclusion of the Health Affairs paper, they include this gem as well:
Thirteen percent of them, or some 550,000 women a year, suffer serious bleeding, blood clots, infections, or other complications, according to a study out this week in the journal Health Affairs. That’s a complication rate comparable to heart surgery.
But not all of those women suffered serious complications, most of the vaginal lacerations were not serious complications and most of the obstetric hemorrhages were not serious complications. Even the authors of the original paper acknowledge that:
the rate of obstetric complications is nearly 13 percent, which is similar to the rates of major complications for cardiac (13.4 percent) and noncardiac surgery (12.3 percent). Maternal complications such as hemorrhage, infection, and laceration are frequently less severe than complications following major surgery… (my emphasis)
So the authors of the Health Affairs paper used crappy, incomplete data, unadjusted for risk, to reach deliberately irresponsible and misleading conclusions … and then Consumer Reports proceeded to repeat the irresponsible, indefensible claim.
It’s déja vu all over again. Yet another ridiculously misleading obstetric study cheerfully published and publicized by people who apparently didn’t even bother to read it.
Headdesk. I hate this stuff. It makes my job 4000x more difficult. When you take the science out of science, you have unicorns and fairies showing up in supposedly legitimate research.
So this study possibly classified as “major haemorrhage” losses as small as 501ml?
BTW- blood donation is 475mls, and pregnant women have expanded blood volumes and physiological mechanisms designed to cope with some blood loss.
Should we be telling blood donor that their donations are 5 teaspoonfulls away from a major haemorrhage?
Ridiculous.
D’oh posted to wrong post.
OMG! Under that premise, I HAD a major hemorrhage when I donated for the first time!
When I was at the canteen, the clot in my arm popped. I proceeded to bleed all over my college mentor, the table and the carpet. I feel comfortable that I lost at LEAST 26mL before the nurse got the vein to clot again.
Sweet. This makes me oddly happy.
Blood loss in a birthing pool is always precisely calculable, however. It is consistently 499 mls or less. Just like home-APGARS are always 9 or 10. Easy.
From “reporting” in Consumer reports, where I know anything about the topic, they are completely unreliable, wrong and / or useless.
So now all of sudden birth is no longer as safe as life gets? Guess it has something to do with C-sections and the totally arbitrary WHO C-section recommendation that birth junkies always tout. *shakes head* I don’t know which is worse for women, studies like this or the idiots who keep producing more and more of The Business of Being Born.
Unfortunately, the NCB crew always blames Csections for poor maternal outcomes, and many also believe they cause great harm to the babies too. The major reason the homebirthers stay home: so those greedy, scalpel happy doctors can’t cut them and kill them and leave their widowers with babies with screwed up gut flora and no bonding ability.
This just goes to show how variable papers that are ”published in peer-reviewed publications” can be.
Many people assume that publication somehow validates a piece of research. On the contrary, the main purpose of publication is to expose the work to critical review by an informed audience. This is what Dr Amy has done, and few radical-NCBers have any idea how to do.
It’s not enough to be smart and literate to critically review medical literature – you need specific skills and knowledge – both in research and in the subject matter of the research.
It surprises me that a hemorrhage is not necessarily a major complication. But I don’t have any medical education. What is the actual definition of an obstetric hemorrhage, and why doesn’t it have to be serious?
A hemorrhage is technically any amount of excess blood loss. Most papers on maternal morbidity define a major hemorrhage as one requiring blood transfusions, since most excessive bleeds do not require anything except possibly iron supplementation.
Thank you. Follow up question: Do all hemorrhages become serious quickly if they aren’t treated in the hospital, or do they sometimes stop on their own?
Hemorrhage occurs for a variety of reasons. Some causes can be treated out of hospital, but most need forms of intervention which are difficult, if not impossible to treat at home. Also, even determining the cause can be almost impossible to ascertain at home. For example, uterine atony can often be resolved with meds and bimanual massage [but not always!] while a cervical tear or retained placenta cannot.
Severe hemorrhage means other organ systems will be involved as they are starved of oxygen. Also, when hemorrhage is severe enough that all the clotting factors are used up, even simple transfusion isn’t enough to stop bleeding.
Thank you too. Let me see if I understand this correctly (not the paper, just this complication). Hemorrhages are basically catastrophes if out of hospital, but (in hospital) not considered a major complication if they can be solved quickly enough that no follow-up treatment is needed.
Conversely, a small vaginal tear would not be a major complication anywhere (though a large one would).
When most people think hemorrhage, they think uncontrolled bleeding. However for the purposes of charting and coding, a hemorrhage is simply more bleeding than expected (generally more than 500cc for a vaginal birth and 1000 cc for a C-section) and though the amount is more than typical, the bleeding may stop easily with basic treatment like pitocin or sutures in the case of surgery.
A major hemorrhage is uncontrolled bleeding that does not respond to basic treatment and therefore large amounts of blood are lost before it is controlled. A major hemorrhage can be caused by a retained placenta, a cervical laceration, both of which are typically treated in an OR. A major hemorrhage during a C-section can be caused by a rupture of the uterus, a uterine incision that tears as the baby is delivered or a damaged major blood vessel. Extensive surgery may be required to fix it.
As far as I can determine, the authors of the study categorized any amount of excess bleeding as a major hemorrhage regardless of whether it was a small amount or easily treated. If it’s only a small amount of excess bleeding, easily treated with routine treatments, it’s not a major hemorrhage.
Thank you! Now I understand.
One major difference between bleeding in obstetrics and bleeding in most other areas of health care is that, in childbirth, SOME bleeding is inevitable.
“or do they sometimes stop on their own?”
All bleeding stops eventually. One way or the other. :-p
In my case, the only thing that was going to stop my pph was repairing the cervical laceration. Since I was in the hospital and had a fairly high hematocrit for a pregnant woman going into delivery (thank you, 3rd trimester red meat cravings), I narrowly avoided a blood transfusion. I did take iron for a month, and my one complaint about CNM care was that they did not do any blood work at my follow up. When I went in for my physical the next summer, I was still anemic. I know that if I had been at home, I would be dead. We know that HB midwives can’t even diagnose external tears correctly much of the time, and cytotec and pitocin were not stopping the bleeding.
The definition of obstetric hemorrhage varies wildly depending on where you are. One of the more commonly used definitions is 500mL for a vaginal delivery and 1000mL for a cesarean. Many, many women in the US lose 500mL in a vaginal delivery and have no side effects at all. However, it would still be classified as a hemorrhage. 500mL would be a greater problem for a woman who was severely anemic to begin with, so that loss might be more serious for a woman in a developing country with poor nutrition.
The fascinating part about calling hemorrhage a “major” complication is that the homebirth crowd, of course, DOESN’T consider it a major problem. Even severe hemorrhage, involving 500 ml or more, is no big deal. If it happens at home. But if it happens in a hospital? Horrors!
Yup. Vaginal birth at home is 5x as likely to result in hemorrhage. By the midwives’ own estimation.
Why does this not get headlines???
To add to the below comments — I believe the average amount of blood loss in a vaginal birth is 500 mL. So “hemorrhage” in this context basically means more than average.
And my docs tell me that the average blood loss in a c-section is 1 liter. That to them is a normal amount of blood loss.