Why Cochrane Childbirth Reviews are often worse than useless


According to Peggy O’Mara, Editor of Mothering.com:

One source that does not have a conflict of interest is the Cochrane Collaboration, internationally recognized as the highest standard in evidence-based healthcare reviews. When Cochrane compared the midwifery model of care to other models of maternity care, they concluded “that most women should be offered midwife-led continuity models of care…” Cochrane also says that there is no strong evidence to favor either planned homebirth or hospital birth.

O’Mara is wrong.

O’Mara is not alone in her reliance on Cochrane Reviews. Lay people love Cochrane pregnancy and childbirth reviews. They always include easy to understand plain language summaries and are generally written by volunteers, many with an natural childbirth ax to grind.

To understand why Cochrane Reviews, particularly childbirth reviews, are often a poor standard of evidence, it helps to understand what Cochrane Reviews are and what they are not. Cochrane Reviews are systematic analysis of randomized controlled trials on a particular topic. Randomized controlled trials (RCTs) are trials in which two treatment options assigned randomly to two groups and in ideal cases neither the patient nor the scientist knows what treatment the patient received (double blind). RCTs are considered the “gold standard” in scientific research.

But what happens when RCTS are either unfeasible or unethical or both? For example, it is unethical to ignore women’s preferences and randomize them to home or hospital birth. Does that mean that it is impossible to investigate the relative safety of home and hospital birth? No, it does not. While RCT’s are at the top of the hierarchy of research methods, there are many forms of research including cohort studies, case-control studies and case series, among others.

But Cochrane Reviews do not consider these other forms of evidence, not because they are unreliable, but because they are outside the scope of Cochrane Reviews. That’s fine, but then they go ahead and do something that is not fine at all. When there are limited RCT’s or poor quality RCT’s, they analyze them anyway as if such an analysis could provide accurate data. Since it is unethical to perform an RCT on homebirth, there should be NO Cochrane Review of homebirth. Instead there is a review that is nothing more than garbage.

Here’s what the Cochrane Review of homebirth showed:

Main results

Two trials met the inclusion criteria but only one trial involving 11 women provided some outcome data and was included. The evidence from this trial was of moderate quality and too small to allow conclusions to be drawn. (my emphasis)

Authors’ conclusions

There is no strong evidence from randomised trials to favour either planned hospital birth or planned home birth for low-risk pregnant women.

No, there is no evidence PERIOD. Therefore no conclusion can be drawn PERIOD.

The bottom line is obvious. When it is unethical to perform RCTs or the RCT’s that exist are small and underpowered, a systematic review of RCTs is worthless. Hence the Cochrane Review of homebirth is useless.

Let me amend that: by publishing a “systematic review” that includes only 11 women, the Cochrane Childbirth Group makes it clear that they are willing to publish garbage if that is the alternative to publishing nothing at all.

68 Responses to “Why Cochrane Childbirth Reviews are often worse than useless”

  1. perpetual lurker
    July 11, 2014 at 10:03 am #

    Sorry for the OT, but have a question esp for those OB/CNM types out there:

    My friend’s wife just had her 2nd home birth, this one complicated by a posterior baby, and after thinking she was complete and starting pushing they realized she was only 7cm and the baby’s head was behind the cervix… so the midwife spent the next several hours on and off manually dilating her cervix with her hands during contractions (um, ouch) until the baby came out after 6 hours of pushing.

    My question – is this actually a thing? Has anyone heard of attendants doing this? I would imagine it’s pretty unheard of in the hospital since we have pitocin, but is this even a feasible alternative? I was taught on L&D that early pushing is a risk for cervical swelling and tearing/lac. Of course, now she’s written a blog novel about it and all my fb friends are talking about how amazing and powerful she is, and I’m just thinking why did you let this midwife torture you?

    • Ash
      July 11, 2014 at 10:51 am #

      On this blog, I have certainly heard of this occuring during a homebirth in the USA. There may be a story about this occurring in the Hurt by Homebirth site, actually.

    • lawyer jane
      July 11, 2014 at 11:54 am #

      My hospital-based CNM manually stretched my cervix from I think 1 to 3. OMG it was the single most painful thing I have ever experienced, much more painful than labor contractions ended up being. I don’t know why she did it – I had cryosurgery so she may have thought there was scar tissue.

      • fiftyfifty1
        July 11, 2014 at 12:34 pm #

        This happened to me with my first birth. My hospital based CNM told me that I was “paper thin” effaced and that the baby was very low “right there” but that my cervix was not dilating and asked if I had ever had cryo or any other procedure because the cervix was, in her experience, acting like a cervix that had been scarred at some point (I hadn’t ever had any instrumentation/cryo etc). Then she manually dilated it, “popped it” she said. But it was a one time thing, not something she kept doing (e.g. PowerBirth). Baby was also malpositioned.
        aside: Just thinking about this reminds me of how much I wish I would have had a CS for the first. Oh well.

    • Amy M
      July 11, 2014 at 12:34 pm #

      I have read horror stories about this, being done only by homebirth midwives. Called “powerbirthing”…several unfortunate women were basically assaulted by their midwives, when the midwife refused to stop manually stretching the cervix.

      (see Mandala Mom’s comment especially)

      The website is homebirth-friendly and there’s a nasty comment about Dr.Amy down near the bottom, but even the midwife whose blog this is can’t be down with the powerbirthing.

      • lawyer jane
        July 11, 2014 at 12:49 pm #

        It felt like an assault when it happened to me!! I’m still kind of pissed about it. I did go on to have an uncomplicated vaginal birth so maybe it worked, but I feel like it’s not something that should have been done without the availability of pain relief. Informed consent, my ass.

    • Deborah
      July 11, 2014 at 12:36 pm #

      I manually reduce cervixes all the time. But if it won’t reduce in a couple of pushes, I give up and have the patient NOT PUSH for a while before I recheck or try again. I’d never be trying to reduce a cervix for hours and hours – and the labor nurses certainly wouldn’t let me either.

    • yentavegan
      July 11, 2014 at 1:17 pm #

      six hours of PUSHING! this poor mom will be lucky if she can control her urine flow as she ages. It is backwards to withhold real medical treatment like pitocin in order to experience a pelvic damaging “natural” birth.

    • perpetual lurker
      July 12, 2014 at 7:51 pm #

      Thanks for the replies!

  2. Ducky7
    July 10, 2014 at 9:53 pm #

    Wow, seriously! 11 women??

    You are a great science communicator, Amy. You should write some sort of plain language tutorial on assessing medical research.

  3. Beth S
    July 10, 2014 at 8:08 pm #

    It seems to me as if more and more these days instead of talking out their fears and concerns with Doctors, midwives, etc.. pregnant women are turning to the internet not realizing they can find a site or study that will confirm every fear or irrational thought you have about pregnancy, motherhood, health etc… It’s part of the reason I hate Mercola so much, the man preys on people who think they’re trying to do what’s best for them, yet in the end all he sells are unicorns and fairies.
    How many women are going to go to mothering.com, see this flawed review and believe to the very depths of their soul that it proves that homebirth is safe as practiced in the United States these days? How many women are too young to remember a time when death during childbirth was not only common but the norm rather than the exception?

  4. Anonymous
    July 10, 2014 at 4:46 pm #

    Are you interested in attaining enlightenment and personal growth?

    Try doing the following two yoga exercises daily:

    Sodarshan Kriya


    Sat Nam Kriya


    Are you interested in preventing and curing cancer, Alzheimer’s, high blood pressure, and many other common diseases using an inexpensive, natural indian spice called turmeric?

    Take two teaspoons full of turmeric powder mixed with a cup of warm soy milk (or any kind of non-diary milk that you’d like). Also eat 2 or 3 black peppercorns along with it, as the pepper helps your body absorb the turmeric better. You also need fat for the turmeric to be absorbed properly, which is why drinking turmeric mixed in water won’t work. You have to drink it with milk or some fatty based liquid. Do this twice a day and you will start to feel amazing within a few days.

    Lastly, please read the following two ancient indian scriptures which talk about the divine love of God:

    Ananda Vrindavan Campu PDF


    Govinda Lilamrta PDF


    ,. .

  5. MaineJen
    July 10, 2014 at 4:19 pm #

    So…only one study involving 11 women qualified for this “review?” …WTH was the point of even publishing this thing? Am I missing something?

  6. Busbus
    July 10, 2014 at 3:31 pm #

    Publishing a trial with 11 women!? That’s ludicrous!

    • The Bofa, Being of the Sofa
      July 10, 2014 at 3:39 pm #

      No, publishing a REVIEW that only contains a trial of 11 women is ludicrous.

  7. theadequatemother
    July 10, 2014 at 2:59 pm #

    I was under the impression that the cochrane thing was a consortium. As long as you stick to their meta analysis protocol and formatting the result is included. I don’t think there is much of a review process….which is clearly a problem. As a clinician I never go to the cochrane reviews for anything. Meta analyses done to a high standard in anesthesia will be published in major anesthesia journals.

    • fiftyfifty1
      July 10, 2014 at 8:10 pm #

      “As a clinician I never go to the cochrane reviews for anything.”

      Me neither. Worthless for anything practical.

      • Beth S
        July 10, 2014 at 8:25 pm #

        I’m not a clinician just a lowly SAHM with a creative writing degree and I know if the doctors are saying it’s worthless it’s time to leave the site.

    • Elaine
      July 11, 2014 at 12:24 am #

      So they’re basically only useful for laypeople who want to think they know more than doctors/are “educated” on a subject. Good to know.

      I feel like the Cochran reviews don’t get a lot of play on MDC. Thought it may be a case where they disagree with what the members want to think (like with vaccinations) and so they aren’t cited for that. Of course, a meta-analysis of RCTs of vaccination would have a lot more to work with than one of RCTs of homebirth.

    • Sue
      July 17, 2014 at 10:21 am #

      Cochrane reviews are sometimes useful if they answer specific questions, especially outside one’s specific specialty are of knowledge.

      But Amy and Adequate are right – you can only review what evidence is out there, and not all research questions lend themselves to RCTs.

      What many people don’t realise is that active professionals within any particular field of expertise generally know the evidence in their area intimitely – it is discussed and applied every day. CLinical practice and policy is based on the totality of the evidence – rarely does an individual paper or study change anything.

  8. Renee Martin
    July 10, 2014 at 2:49 pm #

    You know what they say, garbage in, garbage out.
    And O’Mara is an AIDS denialist. Her opinion is less than worthless. It actively deadly.

    • Guesteleh
      July 10, 2014 at 4:52 pm #

      I will never, ever get over her support of Christine Maggiore and how Maggiore’s HIV denialism led to the death of her daughter. Fucking unconscionable.

      • attitude devant
        July 10, 2014 at 5:02 pm #

        With any luck all the HIV deniers will die out. Darwinism in action.

        • fiftyfifty1
          July 10, 2014 at 8:11 pm #

          It’s killed one of my family members already (although he wasn’t reproducing anyway). He was a pill of a person, so it wasn’t as sad as it might have otherwise been.

          • KarenJJ
            July 11, 2014 at 3:22 am #

            I was recently shocked to find that a family member had actually died of AIDS many years back. We’d all been told it was cancer. There was an awful stigma about it at the time (not helped by the “grim reaper” scare ads from the govt at the time).

          • Amy M
            July 11, 2014 at 6:46 am #

            There’s a guy in my husband’s family currently living with HIV…only for a while we were told it was brain cancer by other members of his family who were ashamed. Eventually, someone admitted he had “the virus” and it became clear that only HIS generation (siblings) are ashamed, all the nieces/nephews, (incl us), we are sad that he is ill, but we don’t see it as something to be ashamed of. He’s actually doing very well on his meds, and we see him from time to time. At least no one is denying he is sick.

      • Guestll
        July 11, 2014 at 1:38 am #

        Yes to everything you wrote and just to add, it also led to Maggiore’s own death. Everything anyone needs to know about Peggy is wrapped up in her HIV denialism and promotion of Maggiore as poster-child.

  9. Trixie
    July 10, 2014 at 2:12 pm #

    Also, speaking of garbage, here’s Rebecca Dekker yammering on about water birth at S&S: http://www.scienceandsensibility.org/?p=8487

    • Amy M
      July 10, 2014 at 2:20 pm #

      Absolutely no mention of the infection risk or drowning risk, nor did she distinguish between laboring in water vs. actually giving birth in water. Also, her study outline sucked. Women who have a waterbirth, women who want a water birth but the tub isn’t available? wtf?

      • Amy M
        July 10, 2014 at 2:54 pm #

        Oh, I read the other article now…I see she was trying to come up with matched groups of low-risk women. She stated it really weirdly though.

    • Young CC Prof
      July 10, 2014 at 2:23 pm #

      I looked at her full article on Evidence Based birth. My favorite line so far:
      “Researchers found that the C-section rate for all women who labored in water and/or gave birth in the tub was only 4.4%, compared to a national Italian average of 38%.”

      So after you eliminate all the prelabor c-sections and all the high risk pregnancies, you get a c-section rate around 5%! Just like every other set of low-risk women undergoing normal labor in a first-world setting! Water birth is astonishing!

      • Amy M
        July 10, 2014 at 2:30 pm #

        Whoever was interviewing her clearly was not a scientist.

      • Amy M
        July 10, 2014 at 2:32 pm #

        No, Young CC Prof, Water has magical properties and laboring in it prevents Csections. And episiotomies too, according to this woman. It probably also prevents TB, vermin and rains of frogs.

        • fiftyfifty1
          July 10, 2014 at 8:13 pm #

          well it does prevent episiotomies.

          • Young CC Prof
            July 10, 2014 at 8:15 pm #

            She actually admitted that the reason water birth prevents episiotomies is that it’s very hard to wield scissors underwater. And that the absence of episiotomies directly increases the incidence of tears. But overall it’s good, because more intact perineums overall, not that there were any numbers presented in a way that made sense.

          • Beth S
            July 10, 2014 at 8:29 pm #

            So it prevents episiotomies, does it prevent the infection you can get from sitting in a pool of blood, urine, feces and amniotic fluid infested waters when you tear.

          • Amy M
            July 11, 2014 at 6:50 am #

            I didn’t undertstand how she could say “more intact perineums” when the incidences of tears was increased. An episiotomy obviously leaves an un-intact perineum, but how can she classify a torn perineum as intact? If it is either episiotomy OR tear, then wouldn’t the number of intact perineums remain the same?

        • meg
          July 10, 2014 at 8:26 pm #

          I have an acquaintance who is a doula and through all her attended water births, she has NEVER had ONE rain of frogs during it.

          Suck on THAT, c-section happy OBs!!

        • Zornorph
          July 10, 2014 at 11:01 pm #

          Well, it doesn’t prevent the sea turning into blood, though. It actually causes it.

      • Elaine
        July 11, 2014 at 12:29 am #

        So she doesn’t know crap about comparison groups. Nice. Because of course the correct comparison group is women who would be candidates for a water birth but didn’t have one. smh. That’s like saying a population of 40-year-olds taking a statin for primary prevention had fewer CV events than a population of 80-year-old heart failure patients who weren’t on a statin. The statin may or may not be helpful, but you can’t tell from that comparison.

      • Trixie
        July 11, 2014 at 9:28 am #

        Well, as far as I know, there has yet to be an underwater c-section.

    • Young CC Prof
      July 10, 2014 at 5:49 pm #

      OK, I’ve read the whole thing, and, yeah, weird. Cargo-cult meta-analysis. Pumped about trials 1,000 or fewer low-risk women in which no babies died, which is kind of like calling someone a great driver for getting to the grocery store without crashing. No math, no attempts to sort trials by quality.

      • Trixie
        July 10, 2014 at 5:52 pm #

        We’ve been over this, YCCP. Math is HARD.

        • Beth S
          July 10, 2014 at 8:23 pm #

          I hate math, but if a site can’t be bothered to do the math and come up with the statistics to prove what they are saying then I automatically put them in the same category as Mercola.

          • D.D.
            July 11, 2014 at 4:41 am #

            “From Mercola to Pharyngula* how crappy is that site”?
            *Or some other truly scientific blog.

      • Pseudonymous BC
        July 10, 2014 at 11:05 pm #

        This follows the same logic pilots joke about in aviation: any landing you can walk away from is a good landing. So as long as baby and mother are alive, technically it was a great birth. There’s really no need to discuss meconium inhalation, group b strep infections, 3rd and 4th degree tears, prolapses, etc. I believe we are again, watching ideology trump evidence. Such is the theme in the NCB world.

      • Amy M
        July 11, 2014 at 7:09 am #

        She mentioned the case studies at the end, which was where the ACOG and AAP and whoever else got nervous, because there was morbidity and mortality. First she tried to downplay all of the case studies by reiterating, several times, that case studies are the lowest form of data collection. Sure, RCTs are best, but these case studies show what CAN happen during/due to a waterbirth, things which can be easily prevented by simply giving birth on land. Does that mean no one should have a waterbirth? Not necessarily, but it DOES mean that those cases should be examined and every precaution observed.

        Second, she kept pointing out that most of the babies involved didn’t actually die! That’s right! They were resuscitated, and/or sent to NICU and recovered in a matter of hours or days or weeks. So therefore, no death=waterbirth is spiffy. I have a problem with that, and I imagine many parents would, since no one wants their newborn in the NICU, potentially on death’s door, potentially sustaining brain/lung damage…even if everything turns out fine. Who wants to take that chance, just to push out a baby in the water? Again, not that waterbirth shouldn’t occur, but writing off morbidity=not cool.

        Third, and somewhat legit, was that a bunch of the waterbirths that involved mortality or morbidity took place outside the hospital. This led Ms. Dekker to completely discount them, which I think is wrong, because hey, water aspiration is water aspiration and cord avulsion is cord avulsion. Those kinds of things could happen in any setting. However, she is right to point out that waterbirth would be safer in a hospital environment with trained individuals at hand to either get the woman out of the water asap if a problem is noted, get the baby to the NICU asap, or get some cord clamps on there immediately if need be. Odds are, the issues that happened with home waterbirths would be less likely to lead to death in a hospital setting, and in some cases, might be prevented altogether (if they properly disinfect and filter their tubs). At least, that’s what I got out of it.

        She put an obvious spin on the whole thing, which was: waterbirths are great, the ACOG doesn’t know what its talking about, no babies ever died (wait what about these?–those don’t count! they are just case studies and they were ooh!), and lots of benefits, so nyah.

  10. Amy M
    July 10, 2014 at 2:09 pm #

    Ugh. I’m sure there are lots of other RCTs in the area of childbirth, like epidurals or other pain relief methods, that could be reviewed. Or how about the evidence of benefits from EFM? Sure, maybe it didn’t reduce CP significantly, but there were other effects, no? Why don’t they stick to that? Or better yet, all those great studies proving the safety and efficacy of vaccines and Vit K!

  11. Trixie
    July 10, 2014 at 2:06 pm #

    Is the Cochrane review of antibiotics in labor for group b strep garbage for similar reasons? That one is an old refrain on NCB boards.

  12. Young CC Prof
    July 10, 2014 at 1:54 pm #

    11 women. Seriously? Would you take a drug that had only been tested on 11 human beings?

    I sure wouldn’t, not unless I was dying and it was the only thing that might help. (AND I would also want to be in a real trial, so that, whether it killed me or cured me, the knowledge gained would help others.)

    • Trixie
      July 10, 2014 at 2:13 pm #

      11 would be okay if it was like an extremely rare genetic disorder or something.

      • Amazed
        July 10, 2014 at 3:31 pm #

        As it is, that’s half of my 8th grade class. Seriously, there were 22 of us there and out of that, only 2 were boys.

        Colour me unimpressed.

    • Anna T
      July 10, 2014 at 3:21 pm #

      I thought the same thing. 11 women? Aren’t they ashamed of even writing that?

    • Montserrat Blanco
      July 10, 2014 at 5:28 pm #

      I have been an investigator on trials where less people had received the trial drug before (we perform some phase I clinical trials in my group). The criteria to enter those trials are really strict, and those patients are heavily monotorized. That is exactly the reason why I would not make a medical decision based on data from 11 cases, the data is clearly insufficient. And that is why you can not sell a drug that has only been tested on 11 people.

      I would take a drug that only 11 people had received if I happen to be a good candidate for one of those trials, though. But only inside a phase I clinical trial well monitored and with the ethical commitee approval.

    • meg
      July 10, 2014 at 8:28 pm #

      I don’t even know how the power equation would work out to gain statistical significance for 11 women. Or did it? Is that what they’re trying to say with the “we can’t find favor in either”? Failed to achieve statsig?

      • Young CC Prof
        July 10, 2014 at 8:31 pm #

        Yes. The only possible conclusion is that there are no RTCs that say anything useful about the safety of home birth. Hence, we should rely on other evidence.

      • Beth S
        July 10, 2014 at 8:31 pm #

        I guess they’re saying the statistics aren’t on their side but that’s okay because these 11 women made it through.

  13. July 10, 2014 at 1:49 pm #

    Yep…garbage. I’m getting really tired of the abuse of the term “evidence based”, really, really tired of “no-evidence of benefit” being confused for “no benefit”. I’ll take a well designed study (that is not necessarily a RCT) over some of the garbage that is likely to come from the future use of “Big Data” methods….

    • The Bofa, Being of the Sofa
      July 10, 2014 at 1:58 pm #

      I have no problem with the term “evidence based” but more with the abuse of the word “evidence”.

      A study that cannot distinguish between a strong result in favor of A, a strong result in favor of B, and no difference between them is not evidence for anything in any way, regardless of what it says.

      • Elaine
        July 11, 2014 at 6:58 am #

        Something that drove me absolutely bonkers when I was in school are the number of times that somebody would discuss information from a study and say something like “The data indicated X. Though it was not statistically significant blah blah blah” and I just wanted to be like “That second sentence should not exist. If it was not statistically significant, you need to stop talking about it.”

        • Amy M
          July 11, 2014 at 7:12 am #

          If I find something not statistically significant but close in one of my studies, I’ll put “there was a trend towards reducing the scores, but it wasn’t significant. Overall, the compound had no effect.” or something like that, in my report.

    • Pseudonymous BC
      July 10, 2014 at 10:54 pm #

      I also share concerns about the “reviews” that will likely come about as a result of Big Data. We are already sering Big Data in education … The results equate to nothing. Throw some numbers at your readers. Add a quote from an “expert” taken conveniently out of context. Tell half the story and draw your own conclusions. Voila! You can make “evidence” say anything.

Leave a Reply

You must be logged in to post a comment.