Why is there so much crappy medical research?
The latest example, complete with press release, was just published in the British Medical Journal (BMJ). It’s entitled Early death after discharge from emergency departments: analysis of national US insurance claims data.
The authors found:
[pullquote align=”right” cite=”” link=”” color=”” class=”” size=””]It’s not clear that the death rate exceeds the background rate of death of Medicare patients.[/pullquote]
Among discharged patients, 0.12% (12 375/10 093 678, in the 20% sample over 2007-12) died within seven days, or 10 093 per year nationally. Mean age at death was 69. Leading causes of death on death certi cates were atherosclerotic heart disease (13.6%), myocardial infarction (10.3%), and chronic obstructive pulmonary disease …
Every year, a substantial number of Medicare beneficiaries die soon after discharge from emergency departments, despite no diagnosis of a life limiting illnesses recorded in their claims. Further research is needed to explore whether these deaths were preventable.
The press release is hardly judicious:
These early deaths were concentrated in hospitals that admitted few patients to the hospital from the ED, hospitals that are often viewed as models by policy makers because of their low costs. By contrast, deaths were far less frequent in large, university-affiliated EDs with higher admission rates and higher costs, even though the population served by these EDs was generally less healthy when they walked in the front door of the ED.
The lead author elaborates in The Boston Globe STAT section:
The study’s lead author said that while the data reflect a fraction of Medicare patient deaths, the finding raises questions about the adequacy of hospital resources in rural and underserved areas and whether the US government’s quest to cut costs — and reduce inpatient admissions from ERs — is also cutting out essential care.
“There’s no doubt there’s a lot of unnecessary hospital admissions, but this study suggests there’s also avoidable harm from sending people home that shouldn’t go home,” said Dr. Ziad Obermeyer, an emergency medicine physician and professor at Harvard Medical School.
The implication is that people are dying preventable deaths because they were discharged from the emergency room instead of being admitted to the hospital.
Is that what the data shows? There’s no way to be sure because the single most important piece of information necessary to reach that conclusion is MISSING from the paper. How many Medicare patients die in a typical week? Quite a few, it turns out.
That’s not surprising. Medicare patients are age 65 and older. They did because everyone dies. Does the rate of death after being discharged from the ER exceed the background rate of death? The authors don’t tell us; indeed they don’t appear to have bothered to check, an inexcusable omission in a paper of this type.
Approximately 4.5% of Medicare patients die each year, for a baseline death rate of 0.09%/week. The study patients represent a subset of Medicare patients [those aged ≥ 90, receiving palliative or hospice care, or with a diagnosis of a life limiting illnesses, either during emergency department visits (for example, myocardial infarction) or in the year before (for example, malignancy) were excluded]. Nonetheless, the baseline Medicare death rate in the group being studied represents a substantial proportion of the death rate reported in the week after discharge from the ER.
Therefore, the implication that 10,000 patients die preventable deaths each year as a result of being discharged from the emergency room is flat out false. The majority of those patients almost certainly would have died anyway.
The authors do tell us how the admitted patients fared, although they do so in a misleading manner.
This chart compares the death rates of patients admitted from the ER compared to those discharged from the ER, divided into quintiles based on the admission rate.
There’s a glaringly obvious problem. The scale from admitted patients is different from that of discharge patients, making it look as if the death rate in discharged patients is higher than admitted patients when it is actually far lower. The death rate for admitted patients was generally 20X higher than for discharged patients! The only exception is the lowest quintile that admitted the fewest patients from the emergency room; in that quintile, the death rate of admitted patients was only double that of discharged patients.
It’s hardly unexpected that getting admitted was associated with a massively increased risk of dying. These patients were sicker. But it also suggests that getting admitted did not necessarily prevent death. We’ve already seen that the majority of the purported 10,000 people who die in the week after ER discharge were going to die anyway. Now we can see that admitting them to the hospital would not necessarily have prevented their deaths, either.
The authors know, or should know this. Indeed, they admit in the abstract that they have no idea whether the deaths they observed were preventable at all, then proceed to imply the exact opposite.
What does this paper tell us? NOTHING!
It’s just another crappy paper that spins a fairy tale from an observation stripped of context. For all we know, every single one of those 10,000 people who died would have died regardless. The authors certainly haven’t demonstrated otherwise.
Many of my stats students whined “But when are we ever gonna use this!” I’d answer that this course was not designed to turn them into statisticians, but to give them a basic understanding of statistical methods so they know what they’re looking at when they read professional journals, and possibly more importantly, so they can recognize bad study designs and statistical manipulation when they see it.
Thanks for covering this.
I am part of a team that is constructing a study looking at driving factors for admission to hospital from the ED. Inpatient admission is commonly seen as the lower risk default, although it is expensive, disruptive and can lead to de-conditioning in the elderly.
It can take much more time and complex work to discharge a frail elderly person from ED safely, though it may be much more humane for the patient. Also, if discharge is back to residential aged care, it would make sense that many of these people are close to death.
The other major factor in so-called disposition decisions (whether to admit or discharge after ED care) is what safe and feasible alternatives are available, at that time of day or that day of the week.
Poor quality evidence often drives poor policy decisions. Sigh.
^^^^ Actually, they say they excluded many of these patients:
” As the focus was on generally
healthy people living in the community, patients in
nursing facilities, aged ≥90, receiving palliative or
hospice care, or with a diagnosis of a life limiting
illnesses, either during emergency department visits
(for example, myocardial infarction) or in the year
before (for example, malignancy) were excluded.”
^^^^ BUT….many patients had heart failure and COPD – so they DID have life-limiting conditions.
Here’s my spin on this:
First, there ARE widely varying ED admission rates between peer hospitals. between types of hospitals and between countries. The US has significantly lower admission rates than Aus, where I practice Emergency Medicine. That is despite the fact that we have been trying for decades to drive down admission rates due to cost implications.
There is a constant tension between the drive to minimise inpatient admissions and the reports that start with “remember that person you say the other day?…” What we don’t always know is whether hospitalisation prevents early deaths after acute illness – all we know is that it is perceived as a ‘safer’ option, at least for the provider.
In the context of relatively low admission rates from ED, and the push to lower them, I see this study as an attempt to point out the risks of not-admitting, against the risks of admitting. It;s not a great study for problem-solving, though, because it’s a retrospective data dredge without enough information.
Isn’t there also the confounding factor that ER visits typically are for very serious things that could lead to death anyway? My parotlet died within two weeks of her first vet ER visit – because she was very ill! Unfortunately we didn’t know until it was too late. I’ve had other pets die within weeks of their vet ER visits, too. I would think even non-Medicare patients would have higher rates of death within weeks of an ER visit, too.
I realized you addressed this – but why wouldn’t this occur to the researchers? I mean, it occurred to me, and I’m a layperson.
Ugh.
Aw, I used to have parrotlets. Super cute, but deceptive little powerhouses. They have very strong personalities for such little birds! Anne was never cleared in James’ death, so I took her back to the breeder, and thus ended my flirtation with parrotlets : /
Because currently some Medicare reimbursements are tied to re-admission rates.
And they want to start knocking down Medicare.
Reading your point of view on papers is always massively interesting. I’ve been looking into online statistics courses because of it.
OT: Two questions from a pregnant lady.
1. It’s a natural-birth truism that stress slows or stops labor. Is there any truth to that? And if so, how does that work? I’m having trouble finding research or medical/scientific explanations on this.
2. What are best practices for preventing long-term pelvic floor damage pre- and post-partum? I hear mixed messages about Kegels. With two kids, I’ve avoided problems so far (besides temporary post-partum SUI), and I’d like to keep it that way.
I don’t have any specific studies on the subject. But I doubt that there are any high quality studies on the subject. After all, how to do you properly measure stress during labour? And a longer labour is most likely going to be more stressful, so which caused which?
Honestly, it goes without saying that virtually 100% of women experience stress during labour, no matter where they are and what kind of birth they are having. And the stress can be either psychological or physical.
Stress possibly has some kind of effect (some animals, can indeed pause their labour if they are stressed). But it’s probably going to be extremely variable between women, and far from actually being as much of a factor or a problem as the NCB community is making it to be.
Natural birth is also not a proper general answer to this. Sure, if you have a phobia of hospital and doctors, home birth will probably be less stressful. However, there is little proof as to what effect that would have on your birth.
Pain relief will also diminish stress caused by pain. For me, having constant foetal monitoring letting me know that my baby is fine and knowing that if there is anything going wrong there is a doctor to take care of it are huge stress relievers.
So basically, stress itself is probably not such a huge factor. But that doesn’t mean we shouldn’t try to minimize it (as long as you use safe techniques) However, stress itself is not ’caused’ by medical birth and Natural birth is not stress-free labour. It’s just the NCB community blowing things way out of proportion to try and benefit themselves by demonizing medicalised childbirth.
As for pelvic floor: I’ve been told to do kegels. Haven’t heard of anything else really helpful.
Great point about pain relief. I’ve heard multiple stories, including from our pro-natural-birth childbirth instructor, about women whose labors progressed more quickly after they got an epidural and were able to relax a bit. Obviously anecdotal, but a counterpoint to the NCB idea that “interventions” are inherently stressful.
Yeah, in my experience, getting relief from severe pain is a stress reducer.
Thanks! If you have any resources for learning more about the interplay of hormones during birth or suggested search terms to use for research, do let me know!
If stress slowed labor, no one would ever deliver in the car on the way to the hospital.
No one really knows the best way to prevent long term pelvic floor damage. Kegels can’t hurt, but they’re probably not that effective, either.
Thank you! That was my thought about stress. Trying some different search terms led me to your “Labor and Catecholamines” post on the old Homebirth Debate blog, which was helpful. But even searching PubMed keeps leading me back to articles by Sarah Buckley or Michel Odent.
There was some recent information put out about how pain in labor can possibly trigger post partum depression. This is from the Daily Mail, but it cites people whose research you can likely look up on PubMed http://www.dailymail.co.uk/health/article-3874736/Mothers-epidural-childbirth-likely-suffer-postnatal-depression.html
Another article with links to better newspapers (but you have to pay so I didn’t) http://www.dailymail.co.uk/news/article-3910898/Childbirth-pain-no-gain-say-doctors-bid-bust-myth-mothers-better-shun-painkillers.html
Again mentions the more pain, there’s a greater post partum depression link. Is it the pain that caused the PPD or are people more likely to have PPD the ones who feel more pain in labour? Does it matter if getting an epidural reduces pain and reduces PPD, I say. What is true, and what the article says, that the more pain your body is in the more work it has to do to function through that pain, and that includes the work of giving birth through pain. So a really painful birth is going to be physically harder on you than one that isn’t as painful, and epidurals are very useful to reduce the stress your body is under.
Those stress hormones that your body creates also have an affect on the baby. If your baby is vulnerable during birth, the additional stress hormones you create from pain can make it just that much more difficult for baby. If mom has some conditions during birth that can make it harder on mom to give birth, the stress from pain can make it more likely that those conditions can have a negative effect.
So while stress won’t slow or stop labor, there is some research that shows it can make labor worse or have long term effects on mom.
If stress was enough to stop labor, no one would ever deliver prematurely.
Another problem: sudden, severe stress often triggers labor in a wide variety of mammals. The rapid induction of labor often causes a miscarriage which is theorized by evolutionary biologists to increase the likelihood of the dam surviving by allowing her to escape quickly or distracting the attacking animal.
I can’t think of a way that a mammal’s body would suddenly reverse that trigger so that stress stops labor – especially since labor is by definition stressful.
Depending on what the issue with the pelvic floor is, Kegels may make things worse if you do them too early. My trainer had me do hypopressives and it seemed to help (this is my trainer’s website) http://www.2thecorecalgary.com/hypopressive-training.html
As with everything though, finding a good doctor or physio that is trained in post-partum issues is a good first stop.
Um, just speaking from experience (anecdotal evidence) here, but question 1, NO and it doesn’t; question 2, a c-section will prevent pelvic floor damage. In my experience, Kegels are completely useless.
People over 65 with heart disease, COPD or who have an MI, and who are sickly enough to have recently visited an ER, are more likely to die…who would have thought?
I was about to say something similar. My mother ended up in the ER because she was short of breath. (my 88 yo father was in hospice and died the night we were sitting in the ER with Mom) She ended up being admitted for a short while, then went back to the short term rehab to try to get strong enough to walk properly again(except she had to be on oxygen all the time by then). She had asthma, COPD and was diabetic. She was on meds to control her Blood pressure and her cholesterol.
She died several weeks later. It was expected, by me at least. She was 86 and had been in gradually worsening health for several years.
A lot of people are living longer than they used to but there’s a limit to how long you can go on before things wear out…
Exactly. They might say “despite no diagnosis of a life limiting illnesses recorded in their claims” – but that seems highly unlikely. Did none of them have coronary disease, heart failure, COPD, dementia, cancer…?
This seems similar to the argument against implementing a ‘7-Day NHS’ (well, the argument against implementing it at the expense of other things that the money might have been better spent on). The argument for implementing it was “People admitted to hospital at the weekend are more likely to die,” implying that this was due to lower levels of care being available. But really, if someone is so ill that they can’t wait a day or two to see their GP during normal practice hours, then of course they’re more likely to die even if the care they receive at the hospital is top-notch.
Funny that this ended up in BMJ and not an American medical journal.
Even funnier that BMJ accepted it. Where are these reviewers when I’m up?!
Seriously. I want _them_. No reviewer comments! Great paper, recommend for publication!
If hospitals are discharging poor people who should stay put, then of course that is a problem but a study that doesn’t actually show whether it’s true is not part of any solution, except for the writers’ grants