Why Cochrane Childbirth Reviews are often worse than useless

image

According to Peggy O’Mara, Editor of Mothering.com:

One source that does not have a conflict of interest is the Cochrane Collaboration, internationally recognized as the highest standard in evidence-based healthcare reviews. When Cochrane compared the midwifery model of care to other models of maternity care, they concluded “that most women should be offered midwife-led continuity models of care…” Cochrane also says that there is no strong evidence to favor either planned homebirth or hospital birth.

O’Mara is wrong.

O’Mara is not alone in her reliance on Cochrane Reviews. Lay people love Cochrane pregnancy and childbirth reviews. They always include easy to understand plain language summaries and are generally written by volunteers, many with an natural childbirth ax to grind.

To understand why Cochrane Reviews, particularly childbirth reviews, are often a poor standard of evidence, it helps to understand what Cochrane Reviews are and what they are not. Cochrane Reviews are systematic analysis of randomized controlled trials on a particular topic. Randomized controlled trials (RCTs) are trials in which two treatment options assigned randomly to two groups and in ideal cases neither the patient nor the scientist knows what treatment the patient received (double blind). RCTs are considered the “gold standard” in scientific research.

But what happens when RCTS are either unfeasible or unethical or both? For example, it is unethical to ignore women’s preferences and randomize them to home or hospital birth. Does that mean that it is impossible to investigate the relative safety of home and hospital birth? No, it does not. While RCT’s are at the top of the hierarchy of research methods, there are many forms of research including cohort studies, case-control studies and case series, among others.

But Cochrane Reviews do not consider these other forms of evidence, not because they are unreliable, but because they are outside the scope of Cochrane Reviews. That’s fine, but then they go ahead and do something that is not fine at all. When there are limited RCT’s or poor quality RCT’s, they analyze them anyway as if such an analysis could provide accurate data. Since it is unethical to perform an RCT on homebirth, there should be NO Cochrane Review of homebirth. Instead there is a review that is nothing more than garbage.

Here’s what the Cochrane Review of homebirth showed:

Main results

Two trials met the inclusion criteria but only one trial involving 11 women provided some outcome data and was included. The evidence from this trial was of moderate quality and too small to allow conclusions to be drawn. (my emphasis)

Authors’ conclusions

There is no strong evidence from randomised trials to favour either planned hospital birth or planned home birth for low-risk pregnant women.

No, there is no evidence PERIOD. Therefore no conclusion can be drawn PERIOD.

The bottom line is obvious. When it is unethical to perform RCTs or the RCT’s that exist are small and underpowered, a systematic review of RCTs is worthless. Hence the Cochrane Review of homebirth is useless.

Let me amend that: by publishing a “systematic review” that includes only 11 women, the Cochrane Childbirth Group makes it clear that they are willing to publish garbage if that is the alternative to publishing nothing at all.

  • perpetual lurker

    Sorry for the OT, but have a question esp for those OB/CNM types out there:

    My friend’s wife just had her 2nd home birth, this one complicated by a posterior baby, and after thinking she was complete and starting pushing they realized she was only 7cm and the baby’s head was behind the cervix… so the midwife spent the next several hours on and off manually dilating her cervix with her hands during contractions (um, ouch) until the baby came out after 6 hours of pushing.

    My question – is this actually a thing? Has anyone heard of attendants doing this? I would imagine it’s pretty unheard of in the hospital since we have pitocin, but is this even a feasible alternative? I was taught on L&D that early pushing is a risk for cervical swelling and tearing/lac. Of course, now she’s written a blog novel about it and all my fb friends are talking about how amazing and powerful she is, and I’m just thinking why did you let this midwife torture you?

    • Ash

      On this blog, I have certainly heard of this occuring during a homebirth in the USA. There may be a story about this occurring in the Hurt by Homebirth site, actually.

    • lawyer jane

      My hospital-based CNM manually stretched my cervix from I think 1 to 3. OMG it was the single most painful thing I have ever experienced, much more painful than labor contractions ended up being. I don’t know why she did it – I had cryosurgery so she may have thought there was scar tissue.

      • fiftyfifty1

        This happened to me with my first birth. My hospital based CNM told me that I was “paper thin” effaced and that the baby was very low “right there” but that my cervix was not dilating and asked if I had ever had cryo or any other procedure because the cervix was, in her experience, acting like a cervix that had been scarred at some point (I hadn’t ever had any instrumentation/cryo etc). Then she manually dilated it, “popped it” she said. But it was a one time thing, not something she kept doing (e.g. PowerBirth). Baby was also malpositioned.
        aside: Just thinking about this reminds me of how much I wish I would have had a CS for the first. Oh well.

    • Amy M

      I have read horror stories about this, being done only by homebirth midwives. Called “powerbirthing”…several unfortunate women were basically assaulted by their midwives, when the midwife refused to stop manually stretching the cervix.

      http://midwiferyramblings.blogspot.com/2009/02/powerbirth-start-pushing-at-5cm.html
      (see Mandala Mom’s comment especially)

      The website is homebirth-friendly and there’s a nasty comment about Dr.Amy down near the bottom, but even the midwife whose blog this is can’t be down with the powerbirthing.

      • lawyer jane

        It felt like an assault when it happened to me!! I’m still kind of pissed about it. I did go on to have an uncomplicated vaginal birth so maybe it worked, but I feel like it’s not something that should have been done without the availability of pain relief. Informed consent, my ass.

    • Deborah

      I manually reduce cervixes all the time. But if it won’t reduce in a couple of pushes, I give up and have the patient NOT PUSH for a while before I recheck or try again. I’d never be trying to reduce a cervix for hours and hours – and the labor nurses certainly wouldn’t let me either.

    • yentavegan

      six hours of PUSHING! this poor mom will be lucky if she can control her urine flow as she ages. It is backwards to withhold real medical treatment like pitocin in order to experience a pelvic damaging “natural” birth.

    • perpetual lurker

      Thanks for the replies!

  • Ducky7

    Wow, seriously! 11 women??

    You are a great science communicator, Amy. You should write some sort of plain language tutorial on assessing medical research.

  • Beth S

    It seems to me as if more and more these days instead of talking out their fears and concerns with Doctors, midwives, etc.. pregnant women are turning to the internet not realizing they can find a site or study that will confirm every fear or irrational thought you have about pregnancy, motherhood, health etc… It’s part of the reason I hate Mercola so much, the man preys on people who think they’re trying to do what’s best for them, yet in the end all he sells are unicorns and fairies.
    How many women are going to go to mothering.com, see this flawed review and believe to the very depths of their soul that it proves that homebirth is safe as practiced in the United States these days? How many women are too young to remember a time when death during childbirth was not only common but the norm rather than the exception?

  • Anonymous

    Are you interested in attaining enlightenment and personal growth?

    Try doing the following two yoga exercises daily:

    Sodarshan Kriya

    http://anmolmehta.com/blog/2007/03/29/3-most-powerful-yoga-pranayamas-and-kriyas-part-1/

    Sat Nam Kriya

    http://anmolmehta.com/blog/2008/06/11/tantra-yoga-technique-tantric-sex/

    Are you interested in preventing and curing cancer, Alzheimer’s, high blood pressure, and many other common diseases using an inexpensive, natural indian spice called turmeric?

    Take two teaspoons full of turmeric powder mixed with a cup of warm soy milk (or any kind of non-diary milk that you’d like). Also eat 2 or 3 black peppercorns along with it, as the pepper helps your body absorb the turmeric better. You also need fat for the turmeric to be absorbed properly, which is why drinking turmeric mixed in water won’t work. You have to drink it with milk or some fatty based liquid. Do this twice a day and you will start to feel amazing within a few days.

    Lastly, please read the following two ancient indian scriptures which talk about the divine love of God:

    Ananda Vrindavan Campu PDF

    http://www.mediafire.com/view/0bqow2mvcifoui3/Ananda_Vmdavana_Campu.pdf

    Govinda Lilamrta PDF

    http://www.mediafire.com/view/r1ws314gg5toij2/Govinda_Lilamrita.pdf

    ,. .

  • MaineJen

    So…only one study involving 11 women qualified for this “review?” …WTH was the point of even publishing this thing? Am I missing something?

    • Daleth

      If so, I am too.

  • Busbus

    Publishing a trial with 11 women!? That’s ludicrous!

    • The Bofa, Being of the Sofa

      No, publishing a REVIEW that only contains a trial of 11 women is ludicrous.

  • theadequatemother

    I was under the impression that the cochrane thing was a consortium. As long as you stick to their meta analysis protocol and formatting the result is included. I don’t think there is much of a review process….which is clearly a problem. As a clinician I never go to the cochrane reviews for anything. Meta analyses done to a high standard in anesthesia will be published in major anesthesia journals.

    • fiftyfifty1

      “As a clinician I never go to the cochrane reviews for anything.”

      Me neither. Worthless for anything practical.

      • Beth S

        I’m not a clinician just a lowly SAHM with a creative writing degree and I know if the doctors are saying it’s worthless it’s time to leave the site.

    • Elaine

      So they’re basically only useful for laypeople who want to think they know more than doctors/are “educated” on a subject. Good to know.

      I feel like the Cochran reviews don’t get a lot of play on MDC. Thought it may be a case where they disagree with what the members want to think (like with vaccinations) and so they aren’t cited for that. Of course, a meta-analysis of RCTs of vaccination would have a lot more to work with than one of RCTs of homebirth.

    • Sue

      Cochrane reviews are sometimes useful if they answer specific questions, especially outside one’s specific specialty are of knowledge.

      But Amy and Adequate are right – you can only review what evidence is out there, and not all research questions lend themselves to RCTs.

      What many people don’t realise is that active professionals within any particular field of expertise generally know the evidence in their area intimitely – it is discussed and applied every day. CLinical practice and policy is based on the totality of the evidence – rarely does an individual paper or study change anything.

  • Renee Martin

    You know what they say, garbage in, garbage out.
    And O’Mara is an AIDS denialist. Her opinion is less than worthless. It actively deadly.

    • Guesteleh

      I will never, ever get over her support of Christine Maggiore and how Maggiore’s HIV denialism led to the death of her daughter. Fucking unconscionable.

      • attitude devant

        With any luck all the HIV deniers will die out. Darwinism in action.

        • fiftyfifty1

          It’s killed one of my family members already (although he wasn’t reproducing anyway). He was a pill of a person, so it wasn’t as sad as it might have otherwise been.

          • KarenJJ

            I was recently shocked to find that a family member had actually died of AIDS many years back. We’d all been told it was cancer. There was an awful stigma about it at the time (not helped by the “grim reaper” scare ads from the govt at the time).

          • Amy M

            There’s a guy in my husband’s family currently living with HIV…only for a while we were told it was brain cancer by other members of his family who were ashamed. Eventually, someone admitted he had “the virus” and it became clear that only HIS generation (siblings) are ashamed, all the nieces/nephews, (incl us), we are sad that he is ill, but we don’t see it as something to be ashamed of. He’s actually doing very well on his meds, and we see him from time to time. At least no one is denying he is sick.

      • Guestll

        Yes to everything you wrote and just to add, it also led to Maggiore’s own death. Everything anyone needs to know about Peggy is wrapped up in her HIV denialism and promotion of Maggiore as poster-child.

  • Trixie

    Also, speaking of garbage, here’s Rebecca Dekker yammering on about water birth at S&S: http://www.scienceandsensibility.org/?p=8487

    • Amy M

      Absolutely no mention of the infection risk or drowning risk, nor did she distinguish between laboring in water vs. actually giving birth in water. Also, her study outline sucked. Women who have a waterbirth, women who want a water birth but the tub isn’t available? wtf?

      • Amy M

        Oh, I read the other article now…I see she was trying to come up with matched groups of low-risk women. She stated it really weirdly though.

    • Young CC Prof

      I looked at her full article on Evidence Based birth. My favorite line so far:
      “Researchers found that the C-section rate for all women who labored in water and/or gave birth in the tub was only 4.4%, compared to a national Italian average of 38%.”

      So after you eliminate all the prelabor c-sections and all the high risk pregnancies, you get a c-section rate around 5%! Just like every other set of low-risk women undergoing normal labor in a first-world setting! Water birth is astonishing!

      • Amy M

        Whoever was interviewing her clearly was not a scientist.

      • Amy M

        No, Young CC Prof, Water has magical properties and laboring in it prevents Csections. And episiotomies too, according to this woman. It probably also prevents TB, vermin and rains of frogs.

        • fiftyfifty1

          well it does prevent episiotomies.

          • Young CC Prof

            She actually admitted that the reason water birth prevents episiotomies is that it’s very hard to wield scissors underwater. And that the absence of episiotomies directly increases the incidence of tears. But overall it’s good, because more intact perineums overall, not that there were any numbers presented in a way that made sense.

          • Beth S

            So it prevents episiotomies, does it prevent the infection you can get from sitting in a pool of blood, urine, feces and amniotic fluid infested waters when you tear.

          • Amy M

            I didn’t undertstand how she could say “more intact perineums” when the incidences of tears was increased. An episiotomy obviously leaves an un-intact perineum, but how can she classify a torn perineum as intact? If it is either episiotomy OR tear, then wouldn’t the number of intact perineums remain the same?

        • meg

          I have an acquaintance who is a doula and through all her attended water births, she has NEVER had ONE rain of frogs during it.

          Suck on THAT, c-section happy OBs!!

        • Zornorph

          Well, it doesn’t prevent the sea turning into blood, though. It actually causes it.

      • Elaine

        So she doesn’t know crap about comparison groups. Nice. Because of course the correct comparison group is women who would be candidates for a water birth but didn’t have one. smh. That’s like saying a population of 40-year-olds taking a statin for primary prevention had fewer CV events than a population of 80-year-old heart failure patients who weren’t on a statin. The statin may or may not be helpful, but you can’t tell from that comparison.

      • Trixie

        Well, as far as I know, there has yet to be an underwater c-section.

    • Young CC Prof

      OK, I’ve read the whole thing, and, yeah, weird. Cargo-cult meta-analysis. Pumped about trials 1,000 or fewer low-risk women in which no babies died, which is kind of like calling someone a great driver for getting to the grocery store without crashing. No math, no attempts to sort trials by quality.

      • Trixie

        We’ve been over this, YCCP. Math is HARD.

        • Beth S

          I hate math, but if a site can’t be bothered to do the math and come up with the statistics to prove what they are saying then I automatically put them in the same category as Mercola.

          • D.D.

            “From Mercola to Pharyngula* how crappy is that site”?
            *Or some other truly scientific blog.

      • Pseudonymous BC

        This follows the same logic pilots joke about in aviation: any landing you can walk away from is a good landing. So as long as baby and mother are alive, technically it was a great birth. There’s really no need to discuss meconium inhalation, group b strep infections, 3rd and 4th degree tears, prolapses, etc. I believe we are again, watching ideology trump evidence. Such is the theme in the NCB world.

      • Amy M

        She mentioned the case studies at the end, which was where the ACOG and AAP and whoever else got nervous, because there was morbidity and mortality. First she tried to downplay all of the case studies by reiterating, several times, that case studies are the lowest form of data collection. Sure, RCTs are best, but these case studies show what CAN happen during/due to a waterbirth, things which can be easily prevented by simply giving birth on land. Does that mean no one should have a waterbirth? Not necessarily, but it DOES mean that those cases should be examined and every precaution observed.

        Second, she kept pointing out that most of the babies involved didn’t actually die! That’s right! They were resuscitated, and/or sent to NICU and recovered in a matter of hours or days or weeks. So therefore, no death=waterbirth is spiffy. I have a problem with that, and I imagine many parents would, since no one wants their newborn in the NICU, potentially on death’s door, potentially sustaining brain/lung damage…even if everything turns out fine. Who wants to take that chance, just to push out a baby in the water? Again, not that waterbirth shouldn’t occur, but writing off morbidity=not cool.

        Third, and somewhat legit, was that a bunch of the waterbirths that involved mortality or morbidity took place outside the hospital. This led Ms. Dekker to completely discount them, which I think is wrong, because hey, water aspiration is water aspiration and cord avulsion is cord avulsion. Those kinds of things could happen in any setting. However, she is right to point out that waterbirth would be safer in a hospital environment with trained individuals at hand to either get the woman out of the water asap if a problem is noted, get the baby to the NICU asap, or get some cord clamps on there immediately if need be. Odds are, the issues that happened with home waterbirths would be less likely to lead to death in a hospital setting, and in some cases, might be prevented altogether (if they properly disinfect and filter their tubs). At least, that’s what I got out of it.

        She put an obvious spin on the whole thing, which was: waterbirths are great, the ACOG doesn’t know what its talking about, no babies ever died (wait what about these?–those don’t count! they are just case studies and they were ooh!), and lots of benefits, so nyah.

  • Amy M

    Ugh. I’m sure there are lots of other RCTs in the area of childbirth, like epidurals or other pain relief methods, that could be reviewed. Or how about the evidence of benefits from EFM? Sure, maybe it didn’t reduce CP significantly, but there were other effects, no? Why don’t they stick to that? Or better yet, all those great studies proving the safety and efficacy of vaccines and Vit K!

  • Trixie

    Is the Cochrane review of antibiotics in labor for group b strep garbage for similar reasons? That one is an old refrain on NCB boards.

  • Young CC Prof

    11 women. Seriously? Would you take a drug that had only been tested on 11 human beings?

    I sure wouldn’t, not unless I was dying and it was the only thing that might help. (AND I would also want to be in a real trial, so that, whether it killed me or cured me, the knowledge gained would help others.)

    • Trixie

      11 would be okay if it was like an extremely rare genetic disorder or something.

      • Amazed

        As it is, that’s half of my 8th grade class. Seriously, there were 22 of us there and out of that, only 2 were boys.

        Colour me unimpressed.

    • Anna T

      I thought the same thing. 11 women? Aren’t they ashamed of even writing that?

    • Montserrat Blanco

      I have been an investigator on trials where less people had received the trial drug before (we perform some phase I clinical trials in my group). The criteria to enter those trials are really strict, and those patients are heavily monotorized. That is exactly the reason why I would not make a medical decision based on data from 11 cases, the data is clearly insufficient. And that is why you can not sell a drug that has only been tested on 11 people.

      I would take a drug that only 11 people had received if I happen to be a good candidate for one of those trials, though. But only inside a phase I clinical trial well monitored and with the ethical commitee approval.

    • meg

      I don’t even know how the power equation would work out to gain statistical significance for 11 women. Or did it? Is that what they’re trying to say with the “we can’t find favor in either”? Failed to achieve statsig?

      • Young CC Prof

        Yes. The only possible conclusion is that there are no RTCs that say anything useful about the safety of home birth. Hence, we should rely on other evidence.

      • Beth S

        I guess they’re saying the statistics aren’t on their side but that’s okay because these 11 women made it through.

  • http://Www.awaitingjuno.blogspot.com/ Mrs. W

    Yep…garbage. I’m getting really tired of the abuse of the term “evidence based”, really, really tired of “no-evidence of benefit” being confused for “no benefit”. I’ll take a well designed study (that is not necessarily a RCT) over some of the garbage that is likely to come from the future use of “Big Data” methods….

    • The Bofa, Being of the Sofa

      I have no problem with the term “evidence based” but more with the abuse of the word “evidence”.

      A study that cannot distinguish between a strong result in favor of A, a strong result in favor of B, and no difference between them is not evidence for anything in any way, regardless of what it says.

      • Elaine

        Something that drove me absolutely bonkers when I was in school are the number of times that somebody would discuss information from a study and say something like “The data indicated X. Though it was not statistically significant blah blah blah” and I just wanted to be like “That second sentence should not exist. If it was not statistically significant, you need to stop talking about it.”

        • Amy M

          If I find something not statistically significant but close in one of my studies, I’ll put “there was a trend towards reducing the scores, but it wasn’t significant. Overall, the compound had no effect.” or something like that, in my report.

    • Pseudonymous BC

      I also share concerns about the “reviews” that will likely come about as a result of Big Data. We are already sering Big Data in education … The results equate to nothing. Throw some numbers at your readers. Add a quote from an “expert” taken conveniently out of context. Tell half the story and draw your own conclusions. Voila! You can make “evidence” say anything.