Another day, another crappy breastfeeding study.
A new study was recently released by the Journal of Pediatrics purporting to show the benefit of breastmilk of neurocognitive development in premature infants. Unfortunately the study, Breast Milk Feeding, Brain Development, and Neurocognitive Outcomes: A 7-Year Longitudinal Study in Infants Born at Less Than 30 Weeks’ Gestation, is fundamentally disingenuous and deeply misleading.
[pullquote align=”right” cite=”” link=”” color=”” class=”” size=””]There is no remotely plausible reason to use number of days on which infants received >50% breast milk as a measure of breastmilk intake.[/pullquote]
Here’s what the authors claimed they found:
We studied 180 infants born at < 30 weeks gestation or < 1250 grams birth weight enrolled in the Victorian Infant Brain Studies cohort from 2001-2003…
Predominant breast milk feeding in the first 28 days of life was associated with a greater deep nuclear gray matter volume at term equivalent age and better IQ, academic achievement, working memory, and motor function at 7 years of age in very preterm infants.
Before we look at the actual data, let’s consider what we would need to show to be sure that increased breastmilk intake caused improved neurocognitive development. As I explained recently in the post How we know that cigarettes cause lung cancer and vaccines DON’T cause autism, we’d need to be sure that there was a a dose-response relationship and consideration and rejection of alternative explanations among other things.
In other words, we’d need to show that the improvement in neurocognitive development was closely tied to the amount of breastmilk an infant received (ml/kg) and that there were no other plausible alternative explanations for the findings.
What did the authors find?
… [W]e found favorable associations of maternal breast milk intake in the first 28 days of life with neurodevelopmental outcomes at 7 years of age. For example, IQ was 0.5 points higher per additional day that breast milk intake was >50% of total enteral intake, and 0.7 points higher per additional 10 mL/kg/d breast milk ingested…We also saw associations of breast milk intake with size of the deep nuclear gray matter and hippocampus at term equivalent age, although this effect was not present on brain volumes assessed at 7 years of age.
Why is this data a bunch of crap?
First, as I noted above, in order to show that breastmilk improves neurocognitive outcomes, you MUST show a correlation between breastmilk VOLUME (ml/kg) and neurocognitive outcomes. The authors did not do that, almost certainly because there was no association.
Instead, it appears they engaged in “p-hacking,” slicing and dicing the data in different ways until they found a bizarre parameter (days breast milk intake was >50% of total enteral intake) that was associated with outcomes.
Wikipedia defines p-hacking as:
Data dredging (also data fishing, data snooping, equation fitting and p-hacking) is the use of data mining to uncover patterns in data that can be presented as statistically significant, without first devising a specific hypothesis as to the underlying causality.
I’d be willing to bet serious money that the authors NEVER hypothesized a relationship between number of days on which infants received >50% breast milk and neurocognitive outcomes. There is no remotely plausible reason to use number of days on which infants received >50% breast milk as a measure of breastmilk intake.
We know that every data set contains patterns due entirely to chance. Slicing and dicing the data until you find a chance pattern (number of days on which infants received >50% breast milk is associated with neurocognitive outcomes!) is not scientific evidence of anything at all.
Second, the authors claim to find an association with breastmilk intake and neurocognitive outcomes at 7 years of age but NOT at 2 years of age. In addition they claim to find an association of breastmilk intake and brain volume at term equivalent age, but NOT at 7 years. That suggests that neither is a real finding; merely a chance result.
According to the authors:
These results provide support for national and international recommendations to provide breast milk as the primary diet for preterm infants.
Actually, the results suggest precisely the opposite. Since there is NO association between breastmilk volume and neurocognitive outcomes, this study offers NO support for recommendations to provided breastmilk as the primary diet for preterm infants.
Breastmilk should be provided to preterm infants because it prevents necrotizing enterocolitis, but there is no evidence that it improves neurocognitive outcomes, nor any reason to suppose it would.
OT, but on the topic of medical studies, anyone know what became of the studies into using sildenafil to treat early-onset IUGR? I came across http://www.ncbi.nlm.nih.gov/pubmed/21392225 in my googlings, but am not au fait on interpreting the data contained therein.
Any parent could point out the fallacy in your logic regarding the measuring of how much milk was ingested. In this particular instance, the volume of milk consumed is irrelevant, because it’s not an amount that can be universally controlled and monitored. No two infants will consume the same amount of nutrients at a given time. They eat when they’re hungry, and they’ll only eat as much as they want to eat. Attempting to do otherwise would undermine the study and produce false results, because you would end up starving some of the children and force feeding others. Not to mention the amount of milk that inevitably would be regurgitated at random times, and therefore can not be recorded as a portion of the total volume consumed.
The second point you tried to make was regarding inconsistent data during various stages of the experiment. The author of the study noticed a dramatic increase in grey matter in the children’s brains after 28 days of ingesting breast milk. The thing is, there is a finite amount of space in the brain, and an individual can only have so much grey matter-regardless of the situation. So, after a certain amount of time, there’s no point in measuring that grey matter. Instead, it’s becomes far more prudent to test the effect the increased grey matter has on the intelligence and neurocognitive functions of all of the children and compare the milk fed to the formula fed. That’s how the author found the correlations between breastfeeding and neurocognitive function at age 7 but not at age 2. At age 2, they were still to young for standard intelligence tests to yield any results.
To be clear, I’m not trying to undermine you or insult your intelligence-let alone experience, which far outweighs my own. I am simply attempting to provide an alternative opinion of the matter at hand by approaching the conundrum as neutral and scientifically as possible. In my limited experience, when a group performs a scientific study and presents their data to the scientific community, it takes an equal or greater amount of opposing data to properly discount the study in question. Your argument-informed and articulate as it was-lacks the cohesion that comes through the presentation of facts to counter the findings of the original study. Instead, your argument is primarily comprised of assumptions and opinions with very little proof to support your thesis.
You seem to have misunderstood. We measure volume in ml/kg. That way we correct for the different amounts of breastmilk that different babies eat.
I think he’s saying that we have no way of knowing how many ml a baby takes in, because it’s only breastfeeding if they get it from the breast. But you can either use pumped milk, or, less accurately, weigh a baby right before and right after it is breastfed, so I don’t get how it’s “impossible.”
Babies born at less than 30 weeks are going to be nearly all ng fed for the first 28 days. They actually will first be fed via IV and weaned to ng and then slowly most will be weaned to a bottle. As they get closer to term some can get the hang of nursing but it’s very common for preemies to need to be bottlefed permanently.
They will know exactly how much they’re consuming unless you have a miracle and they can nurse.
Most babies in the NICU before 30 weeks getting breastmilk are getting pumped milk.
I don’t believe Andy Walker has any idea how the NICU works.
“So, after a certain amount of time, there’s no point in measuring that grey matter.”
But the paper says they did measure it at 7 years. It’s right there in table III. The difference in deep grey matter was not significant then. There was a difference in overall brain size, but that was not significant earlier.
You can’t measure 20 different things and then get excited when one of them show something. That’s going to happen by chance if you take enough tries. See the link below for a good example.
https://xkcd.com/882/
Preemies in the NICU eat at a very set schedule, they absolutely do not eat when they’re hungry. It’s typically every 3 hours. Most of these babies would be ng fed as well.
Thanks for the article.
It’s so hard to keep up with all the crappy, pseudoscientific “evidence” that is thrown around in so many areas – not just lactation and birth but also nutrition, alt med etc etc. The problem is, proclaiming that there is now “evidence” for (insert pet topic here) because you read about it on some blog is much easier than REFUTING it, because, to refute it, you have to
(a) have the time and access to read the full paper, and
(b) have the knowledge and ability to critically analyse it.
I often find myself coming over here for references.
Great post –thank you. I saw the headlines and the abstract to this study, but couldn’t parse though all the BS
OT: has anyone taken birth control for PCOS type symptoms? Something like Yasmin or Diane35? I’ve never been diagnosed with PCOS but I struggle with oily skin and hair, acne, and hair in places I don’t want. It seems like BCP is the standard option, but I’ve read reviews of it making acne worse. Yaz didn’t help much, but I only took it for a month, so I probably didn’t give it a fair shot.
I took both Yasmin and Yaz at various points and found them extremely helpful.
I was on Yasmin for a while and it agreed with me very well compared to other birth control pills that I’ve used. It was great for my acne and the side effects were relatively mild. My GP described it to me as a “very nice” pill. In the early 2000s, I was on Dianette, which was probably the best option for acne but, in my case, had some really nasty side effects (I’m pretty sure it made my depression a lot worse, although my GP refused to accept that there could be a link).
I was on Yaz for years (probably close to 10) and loved it. Only came off it to go on hrt. It really helped a lot with my pre-menstrual dysphoria, long and painful periods and terrible mid-cycle pain.
I am the same, but Diane35 made me feel emotionally unstable.
I was on Yasmin in the past for PCOS and am on it again for PCOS/PMDD. I take it continuously (without the placebo week) and it really helps my acne and hirsutism. The PMDD symptoms are marginally better but I think that is going to require tweaking my other meds to fix and some time for my postpartum hormones to calm down.
So taken at face value, this is support for combo feeding of premies. You can stretch out the donor milk pool, and give moms a bit of a break, because there is no benefit to 100% BF as opposed to 50/50.
OT: Thank God, scientists, doctors, and anyone else applicable for Zoloft. I’m almost in tears right now because I had no idea that the postpartum phase could be this non-hellacious.
I’m 6 weeks out. Minimal anxiety, occasional depressing/obsessive thoughts (which haven’t shown up for a couple of weeks now), and an overall smooth, higher-level degree of functioning. No sitting around being stressed and depressed. I got dressed up and went out with friends for the evening this weekend, and had a great time! I’m exercising my butt off–hopefully literally. 😉 I’m eating really well. (There’s a massive chunk left of the chocolate pecan cheesecake our neighbor brought over a week ago…enough said.)
The biggest thing is that I can ask DH for help and communicate with him about what’s going on with me. This has been a huge boon to our marriage. HUGE. DS’s birth has brought us much closer together because we’re working together as a team, rather than DH trying to deal with a suicidal, raging wife who won’t let him help at all with the baby but resents him for not helping. Sorry if I sound like an advertisement, but I’m hoping that someone out there who feels like I felt when my PPD wasn’t treated will come across this and get help.
So glad that you’re doing so well!
Congratulations on the baby, and your recovery and being brave enough to tell your story.
That is such wonderful news! Congratulations on the new baby, and I am *so* glad that things are going well.
So happy for you. I wish I could rewind the clock and take the meds I was offered with my son instead of believing that I deserved to feel the way I did. I’m tempted to print your post out, stick in it my purse and re-read it repeatedly over the next so many months.
*hugs* Lord knows where we’d be without the stuff. Post partum hit hard and I was already on it. The increase has helped me a lot, too.
I am so happy for you. You don’t really realize how bad you are until you start feeling better. Thanks for sharing with us how you are doing.
I used to joke around that my corporate sponsors were Eli Lily & Co (makers of prozac, before I switched back to wellbutrin) and Juan Valdez (because the world doesn’t work well for me without coffee). It really is amazing how much better life is with the right anti-anxiety/anti-depressant.
So glad you’re feeling better!
I am feeling a renewed sense of hope knowing that there does exist the right medicinal response to post partum depression. Your experience inspires mothers and others to bravely seek out help rather than suffer in agony alone.
That’s so wonderful. PPD is no joke. Thank god (science) for anti- depressants. I put off taking Prozac with my second child to give breastfeeding a try, even knowing that I had pretty bad PPD with my first. It was a huge mistake. By 3 months pp I was angry and afraid I would hurt my kids. If there is a third baby, there will be no breastfeeding and I’ll start Prozac right after delivery.
I’m so sorry that you went though all of this and I’m so glad you are feeling better. PPD really (I cant think of a better word) sucks. I had it with my first (it wasn’t that bad of a case), but I wish I had taken something for it..
So glad to hear you’re doing well.
I distinctly remember not even realizing how terrible I felt until I started finally feeling better. It was like seeing in color again … or something. Meds definitely made the difference for me too with my second.
Thanks for sharing your story. What better advertisement could there be! 🙂
Me too! I felt somthing along the lines of “Oh! This is the person I remember being! Welcome back!”
That’s wonderful news! Please keep us updated.
Thanks for sharing your insight – so valuable when you have lived through something like this.
BTW – SSRIs like Zoloft are great for anxiety, which is a prominent part of PPD.
This reminds me – prior to my discovery/admission to myself that breastfeeding wasn’t working for us, I recall coming across a study on breastfeeding and IQ that some lactivist forum kept linking to as evidence that breastfeeding increased IQ. I read through it, and indeed, the authors found a higher IQ in their breastfed sample than in their formula-fed sample.
The thing that the lactivists were clearly NOT reading, was the the authors’ discussion that 1) they were not convinced that their findings were statistically significant and 2) they had studied the same group of children at age 3 and found no difference in IQ between the two groups of children. So in fact, the authors had concluded that any evidence of a link between IQ and breastfeeding disappeared as the children grew up. It completely undermined the agenda in that particular forum. I wish I could remember the study title, but I stupidly didn’t save it.
Anyway, my long winded point here, is that it is really important to look at these studies carefully! So thanks Dr. Amy, and everyone here, for scrutinizing the science. It’s becoming increasingly important in a world where science is increasingly being ignored.
Also important to realise that conflicts of interest in research are not just about “greedy doctors” and Big Pharma, but also about ideology that drives what we desperately want to find.
In model 3 we included neonatal weight gain (a proxy for nutritional adequacy) to explore its mediating effect, hypothesizing that adjusting for weight z-score change from birth to term would strengthen associations of breast milk intake with outcomes, given associations of breast milk intake with slower weight gain and slower weight gain with poorer developmental outcomes”
Am I reading this wrong, or are they saying that they had to correct for the slower weight gain and the associated issues in the breast-fed infants in order to make breast-feeding look better?
Also, lordy. What a broad net they cast for possible outcomes. At best, this is a hypothesis-generating exercise that requires an independent test dataset to test only pre-specified hypotheses from this trial…
That makes no sense. A mediator is a variable on the causal pathway. If low weight gain is associated with lower IQ then it can’t be a cause of high IQ.
My problem with this study is that their statistically significant findings are not actually significant in terms of response.
For example, the WASI Full Scale IQ mean score was 97 with a SD of 14 which is pretty average to start with. The adjusted test points for Full Scale IQ was 0.5 IQ points per day on >50% breast milk with a range of 0.3-0.8. This means that the “lucky” (and probably healthier) babies who get 28 days of breast milk pick up ~14 IQ points maximum. That’s not an appreciable difference in most cases.
Baby A and Baby B are both starting at a “base” IQ of 90. Baby A gets 28 days of breast milk; Baby B gets 0. The model predicts that Baby A has an IQ at age 7 of 104 and Baby B has an IQ of 90.
That sounds like a large difference, but there’s not a noticeable difference between an IQ of 104 and 90 since they are both “average” in the SB5 classifications.
And then there’s the obvious confounder. Moms who have more time and energy to pump longer tend to be moms higher in SES and with more flexible jobs with better benefits. Let’s just assume that has no effect on the highly-SES-dependent IQ tests…
They said they had a social risk score that included ‘educational level of baby’s primary caregiver’ and ’employment status and income of primary income earner,’ which seems to me a very clunky and uninformative way to get at the above. And if they talked about how they applied that ‘score’ other than saying they ‘adjusted’ for it, I missed it.
Particularly using the latter instead of ‘total household income,’ eg.
My guess is that the score was used as an independent variable within the model where IQ gain = days breast-milk + social risk score + whatever other variables they wanted to play with.
I’m not anti-modeling, per se, but this isn’t a great example of using modeling.
“Breastmilk should be provided to preterm infants because it prevents necrotizing enterocolitis”
Not to be too picky, but is it most accurate to say “reduces the risk of necrotizing enterocolitis”?
Absolutely true. The last kid I saw with NEC was exclusively breastfed.
Yes in our NICU, we had some actual negative results with breast milk with premies, and the problem of necrotizing enterocolitis, which surprised everyone.
Can someone explain to me how the part where the study says the effect was “0.7 points higher per additional 10 mL/kg/d breast milk ingested” does not show a greater effect with greater volume of breast milk? To me, it looks like they are saying that the more days an infant had breastmilk, the greater effect they saw, and that this effect also increased when the volume of breastmilk intake increased (so it was at least 50% of total intake to see any improvement, but those infants who got more than 50% breastmilk had greater improvement). I am well aware that I can’t read statistics well, so I’m honestly looking for an explanation here.
Here’s the chart that shows that relationship. As you can see, the authors subjected children to a large number of tests and only a few have statistically significant associations.
Keep in mind that every data sets contains associations that are due to chance. The fact that there is no improvement in neurocognitive scores at 2 years of age and only few associations found at 7 years of age makes it likely that these associations are purely random.
In order to conclude that breastmilk leads to improved neurocognitive outcomes, you’d need to show a consistent relationship between volume of intake and neurocognitive parameters over time.
Thanks!
What I don’t like about the data is they don’t show the baseline (i assume formula fed, they don’t really talk about it, nor do they discuss reasons for not breastfeeding ) or any real numbers, only the change. To me that rings alarm bells about trying to spruce up some pretty uncompelling data. something you might do for an undergrad or honours project so there are points to discuss. One of my stat teachers liked to call his class “lies, lies and statistics”.
I’m reluctant to accuse anyone of p-hacking without any evidence. As a scientist who does not roll that way, I would be furious if someone tried to discredit my work with that claim. What we can say definitively about these findings is that there were no adjustments for the >100 comparisons in the paper, so the p-threshold of 0.05 was not appropriate.
I’m not sure what you mean by “without any evidence.”
The authors did measure volume of breastmilk intake but did not mention any correlations with neurocognitive outcomes. Moreover, what possible justification is there for using number of days an infant received breastmilk >50% of enteral intake as a proxy for breastmilk volume?
I don’t know, why don’t you ask them?
The rule of thumb I’ve learned is that you need ~30 subjects per hypothesis test performed to avoid p-hacking. Since there were 180 subjects, running more than 5 hypothesis tests could be problematic.
P-hacking and sample size are unrelated phenomena. You can run as many statistical tests as you like on the same sample of N = 20 or whatever you have, as long as you correct for multiple comparisons.