I built an open source AI tool to find my autoimmune disease
271 comments
·February 10, 2025cadamsdotcom
dax_
I'm glad it worked out for your friend, but I can't help feeling extremely sad about this. What I mean is, why is the healthcare system so broken in the first place, that you need to take charge of your own diagnosis and treatment? AI in this case is a bandaid solution, improving something you shouldn't have to do all.
pjc50
I don't know if it was once different, but this seems to be a constant both in the US for-profit model and the UK not-for-profit one. After all, you are the person who knows you best, and has your own interests most at heart. So you need to advocate for yourself. Which can be extremely difficult in some situations, so you also need family who understand and are willing to help you.
mandmandam
> and the UK not-for-profit one.
The UK "not-for-profit" system has been systematically underfunded for decades [0], in the yacht-class' hopes that it can be privatized like the American system. Yes, it was once different.
Not that these things don't happen even in well funded systems, but the NHS isn't a great example of "not-for-profit" healthcare.
0 - https://www.ft.com/content/b593116d-f948-4757-b2fa-c74adadc8...
pizzathyme
Many people think that in the futuristic year 2025, the "system" of doctors and professionals and tests will be monitoring your body and health, keeping you healthy. It is nothing of the sort and possibly never will be.
We can improve healthcare systems, sure. But everyone needs to take personal responsibility for their own health. What is daily life-altering pain for a patient is just a JIRA ticket to a health care provider. No one is more incentivized than you to dig in and fix it.
lemonberry
"everyone needs to take personal responsibility for their own health"
Absolutely. I'm a caregiver to my father. He has many health issues. Some caused and/or exacerbated by poor lifestyle choices - smoking, poor diet, too much alcohol, etc. Granted his generation knew less than we do about these things today.
There's a popular myth today that doctor's just want to push pharmaceuticals on patients. Maybe this is true in some places or with some doctors, but I've been taking my father to doctor appointments for the better part of a decade. Every one of his doctors - primary care provider, nephrologist, cardiologist, and pulmonologist - emphasized the importance of eating better, quitting smoking, exercising, and drinking more water. It always landed on deaf ears.
After living one way for most of your life is very difficult. We also found out a few years ago that he brain damage from surgery to fix a leaking aneurysm in 1985. Both of these complicated his ability to make changes. Regardless, the doctors pushed pharmaceuticals as a last option.
I'm 50 now. I row almost every day, I workout with kettlebells, have a regular mobility routine, and walk outside often. My diet is pretty clean, I don't smoke anything, and do not drink. I could die tomorrow from any number of things, but it won't be because I didn't try to take care of my health.
All of that aside, I think it's important to not blame people for their health situation. It's easy to critique from the outside, but who knows what choices we'd make if we'd walked in their shoes and lived their life.
cadamsdotcom
This is in Australia.
Worldwide, doctors aren’t given the time to work through the issues properly.
nucleardog
The doctor's doing the equivalent of trying to diagnose and fix a bug where they're only given 10 minutes once a week to try and solve it (losing all their context in between) and can't directly check anything but the most superficial things. If they need more information, they have to refer the patient to somebody that can run check for one specific kind of issue in one specific place and will then provide a two sentence report back. Then they'll get another 10 minutes with that extra data point to try and come up with next steps. They can refer them to a specialist, and that person is going to only look at one small part of the context to try and come up with recommendations.
In this sort of setup it's _very_ easy to miss relevant information. Having somebody involved who is focused on the big picture and can surface relevant information and advocate based on it is really essential to getting the best outcomes.
And generally the only person involved to take on that role is... the patient. I don't think that makes the entire system broken, but it could definitely be improved.
The best solution I can think of is essentially to have someone with a medical background designated to take that role on your behalf. And apparently that's already a thing--a "case manager". (Though apparently in the US that term's been overloaded to mean "the person that works with your insurance company to do the wallet biopsy".) Though I haven't ever really run across anyone taking that role, and I'm certain wherever they are they're completely overworked and can't fulfill the role as effectively as we like.
I'm very much _not_ on the AI bandwagon and am really looking forward to all of this falling on its face so I can stop hearing about it and we can go back to finding solutions to problems instead of problems for solutions. However this does seem like one place where a LLM and adjacent developments could provide some benefit. A large part of advocating for yourself effectively is just knowing the right questions to ask and what information might be important to surface in the current situation. And all of this is being handed off to human experts, not being directly relied on or acted on by a laymen.
As a brief example: My wife developed a heart arrhythmia. The ER doctor was following the standard framework for dealing with this and was prescribing beta blockers and a blood thinner. While the ER doctor had a _lot_ of information in front of her face, what she didn't have was time to sit and dig through my wife's entire medical history. If she did, she would have found that the out-of-wack blood pressure readings she'd written off as "your heart's kinda fucked right now" were actually an on-going and previously diagnosed issue of low blood pressure.
In that moment I didn't need to understand everything or make any concrete recommendations, just make the connection "slower heart = less output?" + "already low output" = "problem?". Surfacing that information and asking the doctor whether we should be concerned didn't set the direction of her treatment, but it did change it. The doctor went to consult with a cardiologist before prescribing anything and came back with a different treatment plan.
It definitely seems like an area playing to an LLMs strengths (digesting a bunch of information and trying to relate it to the situation at hand) while avoiding the pitfalls. Ultimately everything is still in the doctor's hands, and the information it's giving is purely _additive_ and already being treated with suspicion. Even if it misses or hallucinates something we're not much worse off than we were without it.
Groxx
Even when everything is going well, ^this is still a reason for The Medical System to miss a lot of things.
It's full of people, and things can happen in an unavoidable rush with no prior context. It's not possible to have enough context and act quickly enough in many cases, regardless of intentions or money. So people do what people do: learn from past experiences where the effort/success line lies, and sometimes over-fit for the next case.
Personal attention is needed to adjust that line. Better tooling for personal attention does a lot to make that more possible. I do wish legitimate attention were easier to distinguish from hypochondria, and more doctors were willing to listen, but it's full of people - any pattern you can come up with will be wrong in both directions.
I dunno if LLMs are the current best option or not (they do have a huge benefit of understanding vague phrasing though), but I am definitely in favor of more tools. It has almost always led to better outcomes.
mapt
We have a rather high overhead ratio in play. 1.1 million Physicians made a median salary of $239,200 in 2023, or around $263M.
U.S. health care spending grew 7.5 percent in 2023, reaching $4.9 trillion or $14,570 per person.
That's about 5% of healthcare spending going towards actual physicians. The logistic exigencies of dealing with insurance companies, seeking permission to treat, justifying, arguing, interacting with medical secretaries, is something on the order of an additional half of the typical doctor's time. They are strongly disincentivized from seeking out low probability explanations using tests which they need to seek permission to apply. They barely have the time to re-familiarize themselves with your chart.
We locked down the supply of physicians in 1997 via Congressional limits in how we fund medical residency slots. We further restrict physician time with problematically low rate setting in Medicare/Medicaid compensation, relative to market rates.
Everybody dies eventually. Everybody has healthcare needs.
A very large fraction of us will at some point suffer from some uncommon or rare condition, something that a physician doesn't see every day - a "zebra" rather than a "horse". There are enough of these conditions - tens of thousands now recognized - that the very low odds of having any particular problem adds up to high odds of suffering from at least one.
To recognize some of them, the doctor would need to synthesize an idea from half a dozen disparate pieces of information, two of which you've mentioned casually in a visit three years ago, two of which you've never told anyone, and one of which is in your chart, and one of which you mentioned casually to a specialist but who did not think it relevant enough to note down... And then they would need to do a differential against hundreds of megabytes of medical research. In five minutes allocated to your visit.
Doctors are trained with a certain degree of professional image to maintain, a sort of 'Wizard' status that must be kept to a tone of mysterious respect. Nobody wants uncertainty from a doctor, despite the fact that all of statistical reasoning and diagnostics is uncertainty, quantified. If they lose this image, it is felt, people start injecting bleach or suing them for malpractice or killing themselves over an innocuous granuloma that their doctor assures them is nothing. That is not a healthy environment to drill-down low evidence of probability outcomes.
Particularly if every test and treatment impoverishes you, and the impact of that poverty on your stress level and lifespan is dramatic.
While tripling the number of doctors would certainly help, AI and a great deal more population-scale diagnostics is just better suited to some of these limitations.
Is that what it will be used for? No. Initially, at least, it will actually be used to make everything worse, to automatically deny treatments, to argue with a doctor with the persistence of a chatbot.
And that's just for the stuff that's been medically recognized. Medical science has put comparatively little effort into researching ailments that cause chronic, non-contagious, non-fatal problems in your life, and less into resolving them. There is no agreement on how dandruff works, lower back pain was dismissed as psychogenic for decades, most headaches are undiagnosable, skincare is witchcraft.
A good doctor who sees you for a brief period every year is staring at your chart and furiously googling some of the things listed there to re-familiarize themselves for five or ten minutes before the visit, just to maintain an image of professional competency by not forgetting the words. Actually applying recent research... how?
sarchertech
You're right about most of that
>We locked down the supply of physicians in 1997 via Congressional limits
This part though is wrong. We temporarily limited the number of resident slots, but total number of residents has been growing for a while now. Not as fast as many people would like obviously.
Just looking at the numbers for recent years the total number of residents jumped from 134k in 2019 to almost 163k in 2024.
If you look at total number of physicians per 10k people, that number has been growing a steady rate since the 60s.
There are now 2x as many physicians per capita as there were in 1975.
yieldcrv
This has been my experience the last 2 years as well
In my local area’s subreddit, people are entirely dismissive of AI and will keep moving the goalpost about why you cant use it. It hallucinates! (But you can fact check it and I wrote that disclaimer) It warms the planet more than other research! (…. you’re not distinguishing between self hosted and data centers….)
But its Los Angeles and writers and actors and VFX are all getting replaced, so they have to keep saying ignorant stuff to sound relevant
52-6F-62
No. You have to ask yourself why it hasn’t been used in this capacity more, and why so many billions are spent on pursuing crushing the arts and crafts into oblivion.
Given the published and (and currently expressing) philosophies of the primary investors and shareholders of those companies, it is no longer any mystery why!
jjallen
What do you mean it's no longer a mystery? I don't understand. Wouldn't the investors and shareholders want more usage of their products, almost regardless of the subject matter?
ben_w
I have the same response to both sides of that:
The reason for both things is that the best models perform, at best, on the level of a recent graduate.
When would you hire a recent graduate in either role, if you could afford better?
These models are essentially the same models for both science and art, and it was a surprise to everyone that GPT-3 was able to turn into ChatGPT, or that Stable Diffusion was able to generalise so well with relatively few issues (even despite the occasional Cronenberg anatomy study). The flaws with the LLMs that prevent accurate science are the same flaws that cause object impermanence in written stories; the flaws that prevent image and video models from being physically plausible are the same flaws — incorrect world model — that cause them to be wrong about weather forecasts, chemistry, etc.
In both cases, increasing quality of AI raises the metaphorical water level, and in this example rising tides don't lift all boats, but instead drown (the careers of) people who can't swim. I don't have a fix for that, and I'm deeply skeptical that any of the suggestions from the AI firms will work — they're not economists, and even if they were (or even if they hired loads), if the AI companies are right, this change will be at least as big as the industrial revolution, which upended old economic models.
brookst
Can you elaborate? I haven’t seen any of these philosophies that are apparently about crushing art.
As a part time artist, AI has been great for me. I’m creating more, and spending less time on scaffolding (I am a middling-at-best programmer). I’m feeling empowered, not crushed.
SkyBelow
AI does best in places where there is plenty of data for training and where messing up when used is only a mild problem (and where the problem is quick and low cost to identify). Art fits both of these categories.
Healthcare data does not. What data does exist is much more private and extracting data from research papers is much harder than extracting data from images. When something goes wrong, the impact is much more negative if it is being used in any serious capacity.
This is also why AI is just beginning to crack into short video and does nothing for long form videos yet, and even less on games, despite these falling in arts and crafts.
CrimsonRain
The crushing you are talking about, if it is possible, it should definitely be done. Enjoy.
klaussilveira
To everyone here being dismissive of AI: you have no idea how it feels like to know you have something wrong, but receiving nothing but shrugs from the experts that are supposed to help you. I have experienced the very same thing: an orthopedist looked at my joint and wrist pain, the endocrinologist checked hormones, the rheumatologist ran their own tests. If I had to rely on my primary doctor to help, I would be taking ibuprofen and some other anti-inflammatory of their favorite big pharma.
I did a large amount of tests, including RSI and carpal tunnel, chasing what could be the source of my issue. I was only able to find out that I have Ankylosing spondylitis by talking to other people online, and asking my primary care doctor to test me specifically for the HLA-B27 antigen. I was just lucky to receive help from a random person on the internet: "why don't you check for X".
I imagine all the other people out there that were not lucky as me, or did not have access to that information. If AI can help with this, then it is great.
thinkingtoilet
I was sick for close to two years. I had excellent doctors who cared and tried their best. There were days where I was literally lying in bed because I was in so much pain. Despite that, the worse pain by far was not knowing what was wrong with me. That took it's toll more than anything. If someone said killing a puppy was the cure I would have done it in a heart beat. It's not something you can ever really understand until you live it for days, weeks, months, years...
mousetree
I was also diagnosed recently with ankylosing spondylitis after years of pain in my back (I am 37).
For years I thought it was due to a lack of exercise but after spending a year doing yoga, Pilates and physiotherapy things only worsened to the point where I could barely walk and could not bend down at all.
I had to strongly insist to my GP that this was not normal. After a referral to an orthopedic surgeon he ordered an MRI. The results indicated possible AS so he referred me to a rheumatologist. Tested positive for HLA-B27. Another MRI and an X-Ray and the diagnosis was confirmed.
AI was very useful in explaining the test results and doctors letters in plain English.
y-c-o-m-b
> you have no idea how it feels like to know you have something wrong, but receiving nothing but shrugs from the experts
I remember telling this to my dad 8 years ago and he still thought of me as just being weak or a hypochondriac. "It's all in your head son!" he would say.
2 years ago he started losing function of his left leg. A very snarky "It's all in your head dad!" was my first response when he started complaining. That was the beginning of his journey into understanding what I've been going through all these years. Today his left leg no longer works and he is also losing function of his right leg. He has abnormalities on his MRIs and EMG/Nerve conduction tests. Even with this, doctors have not been able to diagnose him with anything and all he gets is shrugs. He's extremely frustrated with the US health system and has decided to just give up and wither away instead.
It's nice to finally be validated by my dad, but I wish he didn't have to go through the same experience just to show a little empathy and understanding. I think most of the people in this thread will similarly have an impossible time understanding our predicaments until it affects them the same way.
smrq
I have an autoimmune disorder and so does my wife (getting her diagnosis, the same as yours incidentally, took years). Don't tell me I have no idea how it feels just because I don't agree with you.
TechDebtDevin
Ive basically given up on US medicine
klaussilveira
My journey was done in two very different countries: US and Brazil. The exact same treatment approach and behavior from doctors.
xtracto
I have had a similar experience. But in my case all specialists said it was IBS and "just don't get stressed". I had hundreds of tests by dozen doctors in 3 countries: Mexico, UK and Germany. At the end I found out the root of my problem by mere chance with a neurologist in Mexico.
So I welcome the use of LLMs and other technology for this! Doctors are just humans, in every country.
martius
I read so many similar stories and I'm still shocked.
It took 8 months from my first acute back pain problems to the diagnostic.
reyqn
It's interesting that you've got the same disease as OP from what I gathered.
6d6b73
My son nearly died, and my persistence was the only thing that saved him. I kept insisting on more blood tests, despite being told he was just dehydrated or going through typical teenage malaise, and that it would pass. Eventually, a blood test revealed something was seriously wrong, and we rushed him to the ER. The doctor told me that if we had waited another two or three days, my son likely wouldn’t have survived.
Now, I spend my evenings trying to learn as much as I can about basic medicine and delving into endocrinology. I don’t blame the doctors themselves—they’re overworked. But I do blame the system that pushes people toward becoming YouTubers instead of doctors.
arunabha
I'm conflicted on this. On one hand, no one can seriously argue that the healthcare system(esp in the US) is good or even adequate for the edge cases. The process is a lot like navigating tech support where the initial layers are oriented towards rapid triage and (barely) adequate resolution of the most common cases. Note that this does not mean that the individuals are not capable of more. It's just that the system is not set up to do so. Getting through to 'L2' support can be just as frustrating as in the tech world.
On the other hand, interpreting domain specific data takes a lot of training and experience. Somewhat similar to looking at logs, knowing what to ignore is often more important. LLMs can dump a bunch of possible causes, some of which are perhaps hallucinated, however knowing what next steps to take with the results isn't really something most people will be capable of.
Perhaps the most likely immediate benefit for tools like this might be for the individuals who are motivated and determined enough to put in the time and gain the knowledge necessary to be effective with such tools(like with most things in life).
One thing is for sure though. As others have pointed out in this thread, you need to be your own advocate in the healthcare system.
cjbgkagh
$100K 30 doctor visits, no answers, sounds like my experience with Ehlers Danlos Syndrome (hEDS), perhaps I missed it but did not see what the final diagnosis was. Autoimmune could be a lot of things.
If I feed in my symptoms today into AI, the same list I gave to doctors, I do get hEDS and specifically I get TNXB as a candidate gene. I don't include my WGS DNA data which came after I stoped bothering with doctors. AFAIK only specialists will diagnose hEDS and AFAIK absolutely none would predict TNXB. They call it clEDS instead and consider it 1 in a million, only if you have a specific TNXB SNP and it’s homozygous. Where I think most of hEDS is caused by presumed benign TNXB SNPs. Even the studies that specifically search for DNA candidates for hEDS skip over TNXB due to the belief that hEDS is rare and TNXB SNPs are common.
So I don’t know where AI it’s getting its info from but the TNXB theory is mostly a patient led theory held by some of us who have it. There are some fringe giftedness researchers who focus on the RCCX gene cluster - which incidentally cause a ton of autoimmune conditions.
So it’s definitely a situation where I think the answers from AI are far better than from doctors - I would have found out decades sooner. hEDS is such a perfect match for my symptoms that it becomes obviously true once I researched it so even if the AI guessed wrong it wouldn’t be worse than all of the doctors who guessed wrong. One even told me I should have thyroid surgery, thankfully I had already lost faith in them by that stage.
itchyouch
Whenever I come across folk with hEDS-like symptoms, I always like to bring up fluoroquinolone (FQ) antibiotics, as a "btw do you know about this?"
I have a suspicion that some non-trivial amount of the non-genetic EDS folks may suffer from fluoroquinolone antibiotic toxicity.
Among the many effects of FQs is how it can cause a net-loss of collagen in the body. It increases MMPs that breakdown collagen in the body. Part of the reason FQ antibiotics have a black box warning for tendon tearing.
I have a family member that was sure they had hEDS, but it turned out to be FQ toxicity. 20-30 courses over years that would degrade their health, one small bit at a time, unbeknownst to them. While safe for many, for some amount of the population, it can cause what feels like every bodily system slowly failing.
The frustrating bit of it all is how resilient the body is, and most common blood work shows up "normal."
monkburger
FQ toxicity is likey caused by SNP in a specific gene related to pharmaceutical metabolism (one of the CYP450s). I believe this was published in the literature.
Another possibility is that FQ toxicity is caused by certain genes relating to detoxifying (GSH, COMT, etc) - where the body incorrectly metabolises it, leading to toxicity against cells.
monkburger
Similarly rare is the "traditional" EDS phenotype with stretchy skin and hypermobile joints. Usually seeing them for shoulder dislocations or other orthopedic injuries.
By far the most common (and massively increased over the past few years) is the crowd that attracts all the eye rolls. These are almost exclusively women, usually white, age 16-35, presenting with a constellation of nonspecific chronic complaints including myalgias/arthralgias, fatigue, GI symptoms, etc. None of them have stretchy skin or abnormally hypermobile joints. Very high rate of fibromyalgia/POTS/CFS in this group. They're usually coming to the ED for diffuse pains, dizziness, or something else I really can't fix. Maybe there is some underlying organic disorder we haven't sorted out yet, maybe these are somatic manifestations of untreated anxiety/depression.
One thing about the hEDS diagnosis is the explosion of self-dx from social media (TikTok) which makes physicians cringe.
There's no good genetic test, and even if there was there is not much to be done other than supportive measures.
cjbgkagh
I believe that you're a doctor because your response is exactly typical for a doctor. Have you considered for a second that all these people are right and that you might be wrong? I am similarly self diagnosed. 30 years of not being able to get a diagnosis despite having hypermobility so severe I can subluxate either shoulder on demand, I also had all of those other issues you mention - it's a cluster for a reason.
Hear me out, what if hEDS was 1/50 and not 1/50K or 1/15K or 1/5K or 1/500 or whatever the current literature has it at - which as a data science person is some rather large order of magnitude error bars for a figure. The 1/50 would include those diagnosed with generalized joint hypermobility, this would allow for the more common TNXB SNPs to be causal for hEDS.
And yeah, I've been told by many doctors that it's not worth even testing for because there is no treatment. Also consider the second order effects of not testing - how would you validate the original % numbers if people keep being talked out of getting tested, would subsequent surveys of population include those being talked out of the test? It's a good thing I don't rely on people who can't detect a condition to treat the condition. It is indeed very treatable - the dysautonomia aspect especially so.
AutistiCoder
It'd be cool if people could upload their (anonymized) medical histories to a central database so that they could gain additional insights to help guide their care.
If you included some genetic information like race or national origin, this could help people learn about the genetics behind the disorders.
I have a connective tissue disorder that presents as Marfan syndrome, but I went to a geneticist at UCSF back in the 10's. He looked at Marfan's, Loeys-Dietz, Lujan-Fryns, Ehlers-Danlos, and a bunch of other disorders that would explain any part of my physical symptoms, my cardiovascular problems, and my skeletal issues - he found nothing.
amazingamazing
> The most frustrating part wasn't just the lack of answers - it was how fragmented everything was. Each doctor only saw their piece of the puzzle: the orthopedist looked at joint pain, the endocrinologist checked hormones, the rheumatologist ran their own tests. No one was looking at the whole picture. It wasn't until I visited a rheumatologist who looked at the combination of my symptoms and genetic test results that I learned I likely had an autoimmune condition
>> It wasn't until I visited a rheumatologist who looked at the combination of my symptoms and genetic test results that I learned I likely had an autoimmune condition
kubb
Yeah, he already had the information from the doctor when he prompted GPT, but I wish the project a lot of success, may it help people.
burke
Anyone doubting this really has no idea what they're talking about.
Set up a "Health & Fitness" project in Claude (or whatever). Feed it:
* Basic data: height, weight, age, sex
* Basic metric snapshots from Apple Health or whatever: HRV range, RHR, typical sleep structure - go through everything and summarize it
* Typical diet (do you track it in MFP or Cronometer? Great, upload a nutrition report)
* Any supplements and medications you take
* Typical exercise habits
* Any health records you have - bloodwork results, interpreted imaging results, etc.
* Family history like you would describe it to a doctor
* Summary of any health complaints
* Anything else that seems relevant.
Then go through a few conversation loops asking it if there's any more information you could provide that would help it be more useful.
Then ask it things like "Given <health complaint>, what should I be doing more of? Less of?"—or "Please speculate about potential causes of <thing>".
Or, even if you don't have any particular health complaints you're working with, just being able to ask it questions like "What's one supplement I should consider starting or stopping today?" (and then obviously do some follow-up research...)
This is life-changing. Anyone skeptical of this has not tried it.
zekenie
RAG + analysis on health data has huge potential. We need to tread carefully, obviously, but I have also used these models + RAG on personal health data at times when the docs were scratching my head. It was very helpful, although I approached it a bit differently. In my case it was my infant in the hospital for breathing / eating issues. I took data from the network tab in mychart and added context additional context, then asked Claude questions. My main goal was to be prepared for rounds in the morning. I wanted to understand what types of information each test was going to give me.
It was pretty good! Unsurprisingly, it did hallucinate sometimes or miss some nuance. But, I knew I was talking to something that did that. It was helpful for me, even with errors. I didn't, however try to just say "hey given all this what's going on with my baby"
On another topic:
If I were you, I'd make this thing speak FHIR or some other common format. That'd make using this thing in an ecosystem of other tools a lot easier. There are even FHIR graphql apis you could just polp on top maybe.
bfeynman
Funny enough this is almost a canonical example of data leakage and the incorrect (or extremely extrapolated) assertions that AI can solve what seems like majorly important and valuable tasks. There are 2 very pertinent observations needed. 1. The state of the healthcare system, assuming this is the US, care is based on triage and constantly requires a number of hoops to go through to get deeper analysis or specialists, an emergent property of not having value based care. They do this to save costs while ensuring you are not going to die. 2. They are using and feeding in data from the doctors visits, which means that AI is not really providing a diagnosis in a way that can replace anything necessarily. The fact there are many visits would help a flow chart decision rule out/in possibilities.
You see a lot of this in AI hoopla and even in some of the industry papers where AI is relying on data that only becomes available because of a doctor's orders, so you are not "replacing" it per se.
If anything this demonstrates that value based care would provide better experience and outcomes for people with hard to diagnose conditions.
fujinghg
This should probably be phrased as:
Why didn't the medical procession find my autoimmune disease after $100k and 30 hospital visits?
afthonos
I think this is really cool tech, but that’s not at all the takeaway from this. $100,000 and 30 hospital visits are what created the data for the tool to figure out the diagnosis. The question to ask to make your point relevant is what fragment of that data was necessary, keeping in mind that you can’t change the order, because the patient themselves went in that order.
manx
The tool should probably suggest which clinical tests should be conducted next.
cainxinth
They offer an explanation in the intro:
> The most frustrating part wasn't just the lack of answers - it was how fragmented everything was. Each doctor only saw their piece of the puzzle: the orthopedist looked at joint pain, the endocrinologist checked hormones, the rheumatologist ran their own tests. No one was looking at the whole picture.
brookst
Or just as a statement: human physiology is so complex that no one person can reliably diagnose every ailment, whereas a large AI can reason over multiple highly specialized domains holistically.
Aurornis
> Why didn't the medical procession find my autoimmune disease
The post is vague, but the doctors did find the “likely” autoimmune condition around the same time that the LLM suggested it.
Which is not surprising, because this tool was simply taking the medical records and doctors reports and putting them into the LLM.
The author has also been dodging questions about what the LLM actually said. Reading between the lines, it sounds like it gave a list of possible explanations including “maybe an autoimmune condition”, not an actual diagnosis.
Schiendelman
"Why didn't my Windows Phone do this?"
Old broken systems tend to get replaced, not improved.
null
pj1115
I'm not clear on the timeline - did the patient use a tool to help doc get to a diagnosis, or did this happen in retrospect? My read is the medical profession did, but it took longer to get there. The tool is redeemed in this case because it landed on the same (correct) diagnosis as the rheumatologist, using the test results, notes and diagnostic journey that resulted from $100K and 30 hospital visits.
spondylosaurus
Genuinely? There's probably some less-than-obvious test that no one thought to run, but that if someone did run would make all the pieces fall into place.
I've shared this story before on here a few times: I have ankylosing spondylitis, and it took me a few years to get diagnosed, but even after I got diagnosed treatment didn't seem to help. Humira, steroid shots, none of it. All I could do was eat NSAIDs like candy, which helped in the short term but kinda fucked me over in the long term.
Then I saw a gastroenterologist to ask about a hemorrhoid, and he took one look at me and my file—walking with a cane at a very young age, HLA B27+, offhand complaints about "IBS" (note: it's not officially IBS unless you've ruled out other problems!)—and recommended me for a colonoscopy. My health insurance is decent, so I agreed, and he saw a bunch of inflammation in there. Turns out I have Crohn's, and the AS is likely downstream of that; when I started taking oral medication that isn't supposed to have any effect outside the GI tract, my arthritis vanished.
But your average provider won't see a patient with joint pain and skin problems and think "well obviously we need to stick a camera up their butt."
apwell23
except it did even in this made up story
"it wasn't until I visited a rheumatologist who looked at the combination of my symptoms and genetic test results that I learned I likely had an autoimmune condition."
It should prbly be phased as
"why ppl fall for any BS on the internet"
fujinghg
I think you might have hit the nail on the head there.
enragedcacti
This type of application makes me a bit queasy, but given massive issues with access to healthcare I think this sort of use of LLMs is an inevitability. Is there anything that can be done to make it more robust?
Beyond the obvious issues with LLMs, confirmation bias seems like an especially big problem for a tool like this. It is seeing only the tests doctors ordered, only the notes doctors bothered to take, only the symptoms the patient thought were relevant, and so on.
Would it come to the same conclusions if plausible synthetic results for other tests were included? Could it reliably predict results of a removed test given its previous conclusion? Could it predict the next action (test, referral, prescription) given results of a test?
Is anyone working on this kind of adversarial approach to LLM outputs? Obviously LLMs are far from ideal for an application like this but could this type of probing for internal consistency improve the error rate and allow for some measure of confidence?
jjallen
AI told me about why eating lots of meat might make me feel dramatically better. There are genes that affect the amount of an enzyme that breaks down Folic Acid into methylfolate. I have multiple of those genes. 23andme knew I had these genes for more than a decade. Multiple doctors I told about this basically said nothing and knew nothing about this.
altgeek
You can buy 5-MTHF OTC in the U.S. if you don't feel like consuming a lot of meat.
jjallen
Exactly. It's fairly affordable. Although for me there were other co-factors for me to feel the effects in my brain (almost surely Coq10) so I felt better when I took this and ate food (which normally did also have some meat in it).
mentos
I’ve felt my best on a carnivore diet but couldn’t keep a stable weight so I fear it’s the equivalent of saying I felt my best when I was slowly dying of hunger? ha
I have done 23andMe where can I look for those genes?
jjallen
Just FYI this is not carnivore and I have mostly fixed this without eating pure meat. You can take methylated folate to fix this problem.
One of the subject genes is called MTHFR. It reduces the methylation of Folic acid by ~30-40%, depending on which paper you read. Tons of people have this. It was this AND a gluten sensitivity which resulted in me having actually low levels of usable folate.
To find out if you have this go to this link: https://you.23andme.com/tools/data/?query=MTHFR. You can enter various other SNPs there if you find any of interest.
You can download your DNA there and enter it in another site that can tell you the other genes that could affect this sort of thing.
There are other things that could affect other people in unique ways so of course this isn't the only one.
What I did is ask Claude "why would eating meat make me feel dramatically better" and it told me about these genes. And then I went to 23andme where you can see individual SNPs and then I searched for other related SNPs.
I also seem to have non-Celiac gluten sensitivity which affects my absorption of Folate (and other things if I have recently eaten gluten which was almost always until recently) which is why this was a problem for me. And zero doctors or companies have ever developed a test for this or even take it seriously, including allergists and immunologists that I saw.
citruscomputing
Similar story to you with non-celiac gluten sensitivity: I was having disabling flank pain for months, but after ruling out a couple things they shrugged and gave me an IBS diagnosis and a prescription for a drug with bad long-term side effects. Someone suggested cutting out gluten, which I tried and the pain went away. Intentional and unintentional reintroduction of gluten leads to the pain coming back within a few hours.
criddell
23andme say this about MTHFR:
> Despite lots of research – and lots of buzz – the existing scientific data doesn’t support the vast majority of claims that common MTHFR variants impact human health.
https://blog.23andme.com/articles/our-take-on-the-mthfr-gene
Is their take on MTHFR out of date?
gns24
Yeah, I'm always a bit suspicious of apparently effective dietary changes which result in a calorie reduction - often different people report completely different dietary changes helped them, but one thing they all have in common is being in calorie deficit.
There are plenty of studies showing that caloric restriction calms inflammatory responses which are responsible for the majority of common diseases. Without balancing the energy intake, I'm always slightly suspicious that any improvements with changing what I eat are actually caused by the change in how much I eat.
s1artibartfast
I dont think this is an either/or question. I think people struggle with multiple levels of explanation, which is almost always the case in biology, psychology, and behavior. There is also a huge amount of biologic variability between people.
As a result, some people may find it more pleasant and easier to balance energy intake with different dietary styles.
ddorian43
There are many people who've never been fat and felt better in keto/carnivore.
Example:me. Another example is https://reddit.com/user/MythicalStrength who's a strongman competitor.
> often different people report completely different dietary changes helped them
Nobody fixed T2D, (some types of) epilepsy, (some cases of) mental illness like bipolar/psychosis on high carb or high protein.
yuppii
Could you elaborate a bit more about the diet and its effects?
ddorian43
Should be easy to troubleshoot on a forum or with a specialist. You probably were eating low fat. It happens often. Feel free to describe exactly what you ate and what problems you had.
someothherguyy
Because doctors aren't diagnostic machines, they learn a bunch of shit, use 10% of it, develop hubris syndrome and never fill in the blanks. An expert system doesn't have these same failings, but definitely has others.
3D30497420
Did you change your diet? Did it help?
jjallen
Well yes, I am mostly supplementing with a large amount of methlyated B vitamins and yes it has helped dramatically. Took me years to figure out.
renewiltord
MTHFR? Very common error variants. My dad and wife have different broken variants. Very common to have errors here. Some quarter of the population do. Easy to get supplements these days. Some prenatal supplements will even do this one methylfolate instead of folic acid.
jjallen
For me I had this and I believe a gluten sensitivity which impacted absorption of various vitamins. So when I cut out gluten and supplement I feel better, but kit if I continue eating gluten.
Awesome. It’s great what is becoming possible in healthcare when you can keep asking questions until you’re no longer confused, and keep feeding data in, more than any overworked doctor could handle.
It won’t always be correct, and needs all the right disclaimers.
A friend of mine whose father is sadly very ill, made use of ChatGPT to interpret the oncology reports written about his dad. It helped him have deep technical conversations with multiple specialists and advocate for his dad, culminating in his advocacy for a treatment - that none of the specialists had thought of - which proved critical in extending his dad’s life. Now my buddy can chat to specialists in their own language and hands the info to ChatGPT to turn into layman’s terms for his dad - all of which my friend can check over for accuracy so he knows his dad is getting correct non-technical explanations.
That is massive empowerment.