New study inadvertently shows why mathematical models of breastfeeding benefits are wrong

Grunge red invalid word round rubber seal stamp on white background

Add this new study to the long list of papers that claim to show the benefits of breastfeeding through mathematical modeling.

According to The cost of not breastfeeding: global results from a new tool:

The results of the analysis using the tool show that 595 379 childhood deaths (6 to 59 months) from diarrhoea and pneumonia each year can be attributed to not breastfeeding according to global recommendations from WHO and UNICEF. It also estimates that 974 956 cases of childhood obesity can be attributed to not breastfeeding according to recommendations each year. For women, breastfeeding is estimated to have the potential to prevent 98 243 deaths from breast and ovarian cancers as well as type II diabetes each year. This level of avoidable morbidity and mortality translates into global health system treatment costs of US$1.1 billion annually. The economic losses of premature child and women’s mortality are estimated to equal US$53.7 billion in future lost earnings each year. The largest component of economic losses, however, is the cognitive losses, which are estimated to equal US$285.4 billion annually. Aggregating these costs, the total global economic losses are estimated to be US$341.3 billion, or 0.70% of global gross national income.

There’s just one problem. These claims are nonsense for two simple reasons:

In the language of computer modeling, the model was never verified or validated.

  • The authors inexplicably failed to include the costs of breastfeeding itself
  • The authors never compared the model to the real world

In the language of computer modeling, the model was never verified or validated. Both problems afflict all current mathematical models of breastfeeding benefits, so they’re worth exploring in detail.

Wikipedia has an excellent, relatively simple explanation of verification and validation:

…The developers and users of these models, the decision makers using information obtained from the results of these models, and the individuals affected by decisions based on such models are all rightly concerned with whether a model and its results are “correct”. This concern is addressed through verification and validation of the simulation model.

Verification is essentially checking the math.

As far as I can determine, the authors did not verify their model, but for the sake of this discussion we will assume that errors of implementation of the model are small.

Validation is concerned with whether or not the model reflects the real world.

Validation checks the accuracy of the model’s representation of the real system…

There are many approaches that can be used to validate a computer model. The approaches range from subjective reviews to objective statistical tests. One approach that is commonly used is to have the model builders determine validity of the model through a series of tests.

One of the key components of validation is determining whether the predictions of the model match what actually happens in the real world:

Naylor and Finger formulated a three-step approach to model validation that has been widely followed:

Step 1. Build a model that has high face validity.

Step 2. Validate model assumptions.

Step 3. Compare the model input-output transformations to corresponding input-output transformations for the real system.

Existing modeling of the benefits of breastfeeding do have face validity. Lactation professionals and organizations have great faith in these models because, the models confirm their pre-existing beliefs that breastfeeding has major benefits and that increasing breastfeeding rates will save massive numbers of lives and money.

The problems start with the model assumptions.

Assumptions made about a model generally fall into two categories: structural assumptions about how system works and data assumptions.

Models of breastfeeding benefits fail on both these counts.

First, these models assume causation whenever correlation exists. But many studies that claim to show that breastfeeding has a specific benefit are riddled with confounding variables. Although the initial data seems to show that increased breastfeeding rates lead to increase in that benefit, correcting for confounding variables makes it clear that it was maternal education, socio-economic status or IQ that was responsible for the observed benefit. A model that rests almost entirely on correlations is bound to be inaccurate. That’s because, as everyone knows, correlation is NOT causation.

Second, the authors also made poor data assumptions.

The authors explain what they included:

The cost of not breastfeeding tool incorporates three categories of indicators for human and economic costs attributed to not breastfeeding according to recommendations, including (1) women’s and child morbidity and mortality, (2)for health system and household formula costs and (3) the future economic costs due to mortality and cognitive losses.

But what they failed to mention (and probably never considered) is that breastfeeding has costs as well as benefits. This failure is catastrophic for the utility of the model.

The authors include savings accrued by assumed lower child morbidity and mortality, but failed to include the costs of tens of thousands of hospitalizations per year for dehydration, jaundice and starvation as a result of insufficient breastmilk at a price tag of hundreds of millions of dollar in the US alone. They failed to include the costs of infants who suffer permanent brain injuries and infants who die, smothered in maternal hospital beds or killed by falling from those same beds.

The authors include savings accrued due to household formula costs, but inexplicably, fail to include lost wages of mothers who can no longer work full time or work at all because they are breastfeeding.

They include savings due to assumed mortality and cognitive losses of formula fed babies, but, inexplicably, fail to include mortality and cognitive losses of babies harmed by breastfeeding, and cognitive losses to mothers who have to give up education and career at least temporarily and possibly permanently.

But the biggest failure in validation of the model is that the authors never compared it to the real world.

According to the Wikipedia article:

The validation test consists of comparing outputs from the system under consideration to model outputs for the same set of input conditions. The model output that is of primary interest should be used as the measure of performance. For example, if system under consideration is a fast food drive through where input to model is customer arrival time and the output measure of performance is average customer time in line, then the actual arrival time and time spent in line for customers at the drive through would be recorded. The model would be run with the actual arrival times and the model average time in line would be compared with the actual average time spent in line using one or more tests.

In the case of statistical models of the benefits of breastfeeding, the input is breastfeeding rate and the output is lives and money saved. The model should be run with actual breastfeeding rates and then predicted lives and dollars saved would be compared to actual lives and dollars saved.

For example, the new tool purports to be able to tell us how many lives and dollars would be saved if the breastfeeding rate increased by 20% by 2030. How do we know if those predictions are valid? By putting in data from the past:

What was the change in breastfeeding rates over the last 40 years? What was the predicted benefit in terms of lives and dollars saved? Is that what actually happened? No, not even close; that means that the model itself is invalid and that makes it useless for predicting future benefits.

This is why, if given the opportunity, I ask every lactation professional to show me the real world benefits of breastfeeding as opposed to extrapolations from small studies. Most recently, both Maureen Minchin, self-proclaimed breastfeeding “researcher” and Mike Woolridge, former head of Baby Friendly UK, were completely unable to do so.

Thus far, NO lactation professional has been able to do so. That’s not surprising when you consider their claims of benefits are based on mathematical models that assume causation, fail to take confounding factors into account, and fail to include the costs of breastfeeding.

This paper makes the same mistakes. In technical terms the authors are proposing a model that has never been validated.

In lay terms, their claims are nonsense.

  • BeatriceC

    Completely OT: New baby!!!

    Earlier this week I took my elderly red lored amazon into the vet. She was acting a little odd and it was time for her annual exam anyway, so I thought it best to be checked out. Turns out she’s fine, just showing her age a little. She’s about 50 and her cataracts are starting to get bad, so the impaired vision is causing her odd behavior. This is normal in geriatric birds. When I was there my vet asked me if I’d be willing to take in a baby. This is a six week old fledgling Mexican Redheaded Amazon. He is a wild rehab failure. (He could also be a backyard breeder failure, but there are several wild flocks in my area, so that’s more likely). When I made the decision to take him in, I thought he was a little older, but I was corrected in that MRA’s wean sooner than their larger cousins, so he’s weaned, but not fledged, and my vet’s best guess to age is six weeks based on development and how long he’s been taking care of him.

    Meet Stephen Squawking! Or Stevie for short. He’s settling in well. Because the original goal was rehab back into the wild, he is not as tame as a baby bird raised by a breeder. I’ve had him since Thursday and he’s already tentatively stepping up on my hand and hanging out close to me when possible. This is excellent progress. He is living up to the amazon reputation as food lovers and eating anything and everything I put in front of him. He’s gaining weight well. Betty, my elderly red lored is fascinated and tries to mother him, and Frankie, my cockatoo seems to have a ho-hum attitude about the intruder. The little birds seem to have an “oh shit, another one” attitude. They’ll adjust eventually. He’s still little enough that he needs to be in a flat bottom cage until he’s fully fledged, but he’s working hard learning to fly and perch. So far, everything is going well. He should have a long, healthy life. https://uploads.disquscdn.com/images/ee5dce97ad965c262ee923e96f1e5007fbad8870edf55b79d7cdabb9dd789bea.jpg

    • rational thinker

      Congratulations on the new arrival. He is a lucky bird to have found such a good home.

      • BeatriceC

        Slight aside: I’ve had a bit of a shock to my self confidence with everything that happened with my youngest and so badly misjudging how bad his hip had gotten. My self confidence was basically in the toilet. This whole experience has really helped me get through that. My vet is a world renowned avian/exotic vet. Other vets list internships with him on their CV’s as major bragging points. The vet on NatGeo’s “Exotic Animal ER” is one such vet. In other words, he’s a BFD in the avian medicine world. And he trusted me to take a baby this little. That’s huge. And he follows the Instagram I have for the birds and had this to say on one of my posts about the baby. It’s such a little comment but did so much to help me through this rough patch. https://uploads.disquscdn.com/images/8dc9e1090312fe41cb03bcef1e714c18ff1e65bca4a6dcf4fddd6a55cbc5eae0.png

    • PeggySue

      Oh my goodness, he is SOOOOO CUTE! He’s a lucky baby, and I wish you many happy years together!

    • StephanieJR

      Stevie the sweetie! What a lovely addition to your family. I hope you have a great time together.

    • BeatriceC

      Slight update after this morning’s weigh in. I had to add formula back to his food, because fed is best in every species. He lost a little weight overnight. He is currently stuffing his little beak with a ‘soup” made of fruits and thick formula.

    • Mel

      He’s so handsome! I love that your older bird has the instinct to try and mother him, lol.

      • BeatriceC

        Turns out you don’t have to shove and egg through your cloaca to be a good mother bird. WHO KNEW? Actually, I think people who try to appeal to nature really need to actually look at how often other species “adopt” baby animals, sometimes not even the same species, and the babies thrive. Betty is a close cousin species (she’s a red lored amazon, he’s a Mexican redheaded amazon), but they’re still different species.

  • Kathryn Marks

    Is it just me, or are those numbers *really* specific? 595 379 deaths… Would it be better to say approximately 595000 deaths. Clearly this paper isn’t worth the electrons used to transmit it.

  • Sassafras

    My time has a lot of worth, therefore, breastfeeding would be mad expensive.

  • Bee

    Those are very low numbers, just over 500,000, doesn’t sound worth the effort or stress to me. Breast feeding doesn’t just have a cost to the parents, it also costs governments and health systems-training, facilities, promotion, education, materials, special staffing the list goes on. For the parents you also have to include the cost of extra calories eaten. I found our shopping costs about an extra $20-30 a week, a tin of formula is about the same.

  • rational thinker

    I never do see them address the issue of babies readmitted to the hospital because of breastfeeding problems (starvation, jaundice). It seems like they just pretend those babies do not exist.

    Also formula feeding is way less expensive than breastfeeding. A baby only needs formula for the first 12 months. It has been just over a decade since I have had to use formula but when I did the cost for a whole year of formula and supplies was between 1300 and 1800 dollars. A lactation consultant bills as much as 400 for each visit and usually a baby and mom goes to one at least a few times. Then you need all the breastfeeding accessories pillows,clothes,pumps,ect, and moms time IS worth something also. So in what universe is breastfeeding saving anyone any money? I dont think these researchers have an answer to that question.

    • Lurker

      I think people here tend to overestimate the cost of breastfeeding, just like lactivists tend to underestimate it. Breastfeeding can be expensive when it’s not working well, but it *is* cheap when everything is going smoothly.

      Nursing my first was more expensive than my second – I saw a lactation consultant 2 or 3 times (covered by insurance, $20 copay each visit – I know we were super lucky to find someone insurance would cover), bought a handful of supplements that did very little, regularly drank nursing teas that are not terribly expensive. My pump was covered by insurance (though I did have to pay $50 for a manual pump at a Rite Aid the time I forgot it at home!), and I had to replace some of the pieces a few times at a fairly minimal cost. AND I lost all my bottles in a move, purchased a whole new set, and then found the old ones the day after I’d opened and used the new ones. But that’s not BFing’s fault! I bought nursing bras, but no other fancy clothes (and I expect I would have needed to buy new bras postpartum anyway, because my whole body seemed to be a different shape). I still don’t think it cost me more than $1,000.

      And my second? No lactation consultant – never needed one. No supplements – never needed them. One box of nursing tea that I got just in case and never used – maybe $5? Replaced pump parts maybe once (longer maternity leave, less time pumping overall) – $25? New bras that I would have needed anyway. Pump covered by insurance. Didn’t need new bottles since I had all those extras from last time 🙂 (Plus, obviously, you can’t count bottles against the cost of BFing when they’re required for formula, too.) I had a Boppy that had been a gift, but it was easier to just rest him on a pillow from the couch. Had a nursing cover, too (also a gift) that I had used occasionally with the oldest but never with the youngest. I never would have spent money on either on my own. We supplemented briefly at the beginning, but stopped needing to before the free samples from Enfamil and my pediatrician ran out. And when you take into account his dietary sensitivities – I could eat a reasonably balanced dairy-free, soy-free diet WAY WAY WAY cheaper than I could buy hypo-allergenic formula for a year+ (because babies who can’t have milk can’t transition to milk at 12 months). BFing was a decided money-saver for us the second time, as a result.

      And yes, I do recognize the privileges that let BFing work for me – reasonable insurance, generous maternity leave the second time around (by US standards, at least – I didn’t get paid a dime), a baby shower before my first was born so that many of my supplies were gifts. But I don’t think it’s accurate to say that BFing, in general, is way more expensive than formula – I imagine that individual circumstances make one or the other come out ahead for different moms and different babies, and that at it’s most likely a wash in the grand scheme of things.

      • Lurker

        Yikes, sorry, I did not realize I was writing a novel while I was typing! I promise I wasn’t trying to go on a rant or anything.

      • fiftyfifty1

        Although you didn’t factor in the cost of extra food. A lactating woman eats about 1.5 extra days worth of food per week.

        • Lurker

          I just did some very rough math and estimate that that’s somewhere in the neighborhood of an extra $800 per year in food costs for me, which would still be less than your estimate for formula, meaning that with my first (the expensive baby to nurse), we’d probably about break even with the cost of formula, and with my second (the cheap baby to nurse who would have had expensive formula requirements), we still come out ahead by something like $2,500.

          My comment was marked as spam, though, really? I may have been wordier than necessary, and I suppose I presented a perspective that doesn’t match the dominant culture among commenters here, but that’s not what “spam” means.

          • rational thinker

            It was probably a disqus problem. Dr Amy would never do that to your comment so its probably disqus being screwy again, it just happens from time to time.

          • Lurker

            Oh, I definitely did not think it was Dr. Amy, but I thought commenters can flag a comment as spam, too. I hope it was Disqus, though.

          • fiftyfifty1

            Disqus is super screwy. Not sure why it did it this time. Not officially signed in + long comment probably. Nobody wants to censor you here. Your perspective is important. For me, breastfeeding was extremely expensive. But for a mother who will be home anyway and has a smooth go of it, breastfeeding can be a big money saver for sure.

      • Cristina B

        You forgot to factor in the fact that you wanted to. I never wanted to, so the cost and time spent washing bottles was well worth it.

        • Lurker

          What? I didn’t forget that at all. We were talking about cost. I wasn’t saying anyone *should* breastfeed because it can be cheaper – just that it’s not accurate to say it’s always more expensive. No one should have to breastfeed who doesn’t want to, regardless of cost or other factors.

          • Sarah

            Yes ultimately I think the point is that neither is inherently more or less expensive than the other. It depends on factors specific to the individuals concerned.

            What we do know though is that the costs attached to breastfeeding tend to be front loaded, no pun intended, whereas those for formula feeding are more evenly distributed. This is something that should be mentioned more and isn’t, because of the economic privilege of most of those doing the theorising and policy making.

            Too many of them don’t understand that a woman might have access to £10 a week in the long term whilst not having the couple of hundred immediately that it might take her to get feeding well established, even if formula does end up being more expensive on average over the feeding period.

          • fiftyfifty1

            I think that the biggest factor is that the cost of formula is easily visible, while the cost of breastfeeding is harder to see.

            Formula is easy: a few bottles + the price of X cans of formula per week multiplied by 52 weeks.

            For breastfeeding it’s more variable. Will you end up needing a lactation consultant or not? Will you need a pump or not? Will either of these be paid by insurance? Will your kid latch no problem, or will you require all sorts of Boppy pillows and footstools to make it happen? Small breasts and any old bra will do, or all of a sudden GGG breasts and a specialized bra? Are you home anyway, or will you be working? If working, will your employer look the other way and pay you during your pump breaks, or is it unpaid time? Or maybe will your employer pay you for the pump breaks, but unconsciously consider you to be “on the mommy track” and 2 years from now promote your colleague and not you? And then the extra food you eat: a calorie of formula is more expensive than a calorie of potatoes or beans or rice, but less expensive than a calorie of blueberries or broccoli or salmon. Lots of variables!

      • Labor RN

        Just because insurance paid doesn’t mean there wasn’t a cost you just didn’t pay it out of your pocket

      • Heidi

        You went to the drugstore to buy a pump that was manual. Most could have bought 1.5 huge canisters of formula for that and not spent so much time and hand energy to pump. That’s something.

      • Heidi

        Also, employers are absolutely not required by law to pay anyone while they are pumping. They have to be given a pumping break but that is totally allowed to be unpaid.

  • fiftyfifty1

    Are they not familiar with the discordant sibling and Belarus PROBIT findings? Those are large, well designed studies and the closest to real world we have. How can nearly a million cases of childhood obesity be attributed to not breastfeeding when it has been shown that obesity rates don’t differ?! How can “cognitive losses” be $285 billion when there is no difference in IQ?! Sheesh.

    • Sarah

      Of course they’re not familiar.

    • PeggySue

      One might think that someone writing a paper predicting such losses would be asked to show some evidence that they were actually happening. One would apparently be wrong, though.