Resident Physicians' Exam Scores Tied to Patient Survival
29 comments
·February 25, 2025crooked-v
With the absolute absurdity the residency process, and the focus entirely on new doctors just after that residency, I have to wonder how much of this just corresponds to whoever's lucky enough to be the kind of high-powered mutant who can survive multiple years of 80- to 100-hour week schedules designed by a man who was high on cocaine and morphine 24/7 (seriously, look it up, it's true). There are going to be a lot of people who need an extended sabattical to recover from that before they'll be effective at anything at all, which makes any kind of baseline of test scores really suspect to me.
lurk2
Please tell me more about the man who was high on cocaine and morphine 24/7.
femto
Probably referring to this?
dawatchusay
This doesn’t show that he was “high on cocaine and morphine 24/7” as the relevant commenter suggested; just that he struggled with addiction
derbOac
Yeah I wondered how much of this is accounted for by some general resiliency thing or circumstances during residency or something along those lines.
SpicyLemonZest
Does the difference matter in this context, though? Medicine isn't like other professions where it's no big deal to have some fraction of the workforce be bad at their jobs. I'm not so status-quo-biased that I'd support 100 hour residencies, but I'm skeptical of reform proposals that focus on the subjective experience of being a doctor rather than the overriding need to deliver people the best healthcare we possibly can.
PaulKeeble
Medicine is now absurdly complex, it's far more than a person can possibly learn especially if trying to be up to date with modern research. The more you can memorise correctly and pattern match the better. Many patients are failed in the current system, most not fatally but their lives are damaged and it's not uncommon for more complex diseases to have 90% of sufferers never getting a diagnosis until they die from the disease.
Something has to change drastically in how medicine is organised because it's not working in its current iteration as the difficulty goes up and up.
smgit
Its the environment that compounds the complexity. Go down the list of Largest companies by revenue in the US and 8 in top 20 are related to "health" - are they running hospitals? are they pharma companies? No.
They run pharmacy benefits management, health insurance and drug distribution.
The estimate is 4-5 Trillion flows throw these firms. Which is larger than the GDP of India. So this gigantic structure has emerged that doesn't really make too much profit btw (very similar to Amazon Platform Economics) but is layer upon layer upon layer of cash flow passing through middlemen.
Drastic change requires new ideas about what do we do about all these middlemen who shape the environment on top of which everything exists.
Bostonian
'Board exam performance was powerfully linked to patient risk of dying or hospital readmission. For example, there was an 8 percent reduction in the odds of dying within seven days of hospitalization in patients of physicians who scored in the top 25 percent on the exam, compared to the patients of physicians who scored in the bottom 25 percent on the exam, which was still a passing grade.'
goodluckchuck
Controlled hospital quality? I figure the best credentialed doctors go to the best hospitals, where patients receive a lot of other care aside from the MD.
helsinkiandrew
From the article:
>The researchers compared outcomes for patients within the same hospitals who were cared for by doctors with different exam scores. This allowed the researchers to eliminate, or at least minimize, the effect of differences in patient populations, hospital resources, and other variations that might influence the odds of patient death or readmission, independent of a doctor’s performance.
sebmellen
This is the most important question in the thread.
ETH_start
[flagged]
_aavaa_
I’m a skeptical of the interpretations. All we have are percentages without knowing the size of each group.
Among other things, it reeks of of Simpson’s Paradox.
derbOac
MCAT != board exam, for one thing.
There have been studies suggesting that elimination of the MCAT does little to nothing to prediction of student performance beyond the second year or so.
sagarm
So do you think that if the acceptance rate for high MCAT and GPA are below 100%, then the other bars should be zero? i.e, these are the only admissions criteria that should be considered?
colechristensen
I’m all for diversity but that admissions gap is just racism.
You can’t have separate entrances for your establishment based on what folks look like, the group you prefer getting better service doesn’t make it equality.
sagarm
I know right? Black applicants with high MCAT scores were rejected in favor of white applicants with low MCAT scores! Just unbelievable.
NotYourLawyer
Unbelievable because it’s the opposite of what the link shows?
zameerb1
its easy to hide data behind percentages and say 94% of the blacks who had a certain GPA where admitted. look at the raw numbers, study after study have shown improved care for colored patients and outcome better when treated by black physicians which indicates we have to have proportional numbers of black and hispanic physicians representative of their population. If whites and asians disproportionately apply to medical schools their admission rates are going to look different. The systemic advantage afforded to affluent kids by being brought up for 18+ years by highly educated parents is not level playing field.
yyyyz
[dead]
NotYourLawyer
Huh. Maybe in fields that matter, we should be hiring for merit instead of for DEI.
ok_dad
Have too few doctors already, we should set a bar for qualifications and let anyone over the bar become a doctor. The DEI bogeyman didn’t do any harm here, since the current system requires both to get over the bar AND to be randomly selected for one of N arbitrary spots.
NotYourLawyer
> we should set a bar for qualifications and let anyone over the bar become a doctor
Absolutely. Let’s stop using different bars for different races.
jmcgough
No one is hiring based on test scores though? The bar to even get into med school is so insanely high that most people able to get in and become doctors were already upper-middle or high SES. The only point in the entire process where "DEI" matters is feeder programs for underprivileged students, the type of people who can't afford to pay for MCAT tutors etc.
SpicyLemonZest
Resident physician hiring is strongly based on test scores, specifically the USMLE Step 1. It's true that scores in the board exams the OP discusses aren't super relevant to hiring, though.
nis0s
Incompetency comes in all types, there’s no need to assume anything. In fact, you should be especially careful if your doctor is [your favorite type of person], that’s when you know your cognitive biases are working against your better judgments.
Edit: Well, that's embarrassing. I hadn't realized that the link is to a new 2024 study on IM board scores and patient outcomes. My post is in regards to a 2023 study on USMLE scores and patient outcomes that was pretty widely discussed.
Healthcare worker here. Sheriffofsodium did a great video poking holes at this study: https://youtu.be/JKS9Y-nCnKs?si=VPsUNSoepltbg4Hu
It's 45 minutes so I don't expect people to watch it, but he makes several important points, including:
- This study was performed by USMLE insiders, the only ones with access to this private data. USMLE does not share this data publicly so it's impossible to verify.
- As the USMLE makes millions of dollars from these exams, they have a clear conflict of interest.
- The differences in patient outcome are AT BEST of marginal clinical significance, which the authors of the study even state in the paper.
There is better scientific evidence that female surgeons have better patient outcomes on average: https://pubmed.ncbi.nlm.nih.gov/37647075/