I asked AI how it would use AI to improve the disability claim process
you've probably seen the other thread that got 100 comments about AI. for the experiment I prompted AI how it would put itself into the disability claim process.
to give you my personal experience of the process, which is specific to my country of course, I know that people usually get denied on the first attempt hoping that they'll give up. It takes about a year before you get a final decision. You can appeal that decision and I don't think there's a limit to the appeals you can make (which increase administrative load and cost to the agency).
Sending a claim is easy enough, you can do it online, but it took 3 months before I got any updates and it was to ask me about my past employment and nothing else. I don't remember this question being on the form I filled.
It's clear to me that they're overworked. They receive new cases literally every day and usually when you are disabled you don't suddenly stop being disabled, so the workload keeps increasing over time and creates tension.
That's the current situation.
So I asked AI, "how would you put AI and LLMs into a disability claim process?"
At first its answer was not super interesting. It talked a lot about using LLMs as an assistant, chatbot, providing a summary, or to copilot the case worker when reviewing a case. Typical stuff.
I did like one idea, the AI being able to help the claimant write down their symptoms and make sure not to miss any details. Here when you send the form they tell you to "describe the case" but they leave it blank. They also ask you to provide your drs names and address because they will then receive a form to fill on your disability. But I only know that last one because I know someone who's going through the process too, they don't tell you what the drs info is important for while you're doing the form. but anyway.
I reframed my question. I told it you are now the head of the disability agency in [country]. For the time being, just explain the challenges your agency faces.
It gave me 7 core challenges. Are they accurate? idk lol i don't work there. but they seem in line with the vibes I've been getting. I then asked it to summarize these challenges in a list so that we could work from it, and it provided me these:
Inconsistent & Opaque Adjudication: The application of complex rules is highly subjective, leading to unfairness and a loss of public trust. Outcomes depend too much on which caseworker handles the claim, creating a "lottery" effect for citizens.
The Evidence Translation Gap: A fundamental disconnect exists between the claimant's lived experience of their disability and the cold, clinical data required by law. We fail to bridge this gap, leaving claimants feeling unheard and cases poorly defined.
Administrative Burden of Data Sifting: Caseworkers spend the majority of their time as "data archaeologists," manually searching through vast volumes of documents to find the few critical pieces of relevant evidence. This is the primary driver of slow processing times and high operational costs.
Inherently Adversarial & Traumatic Process: The system's design places the entire burden of proof on vulnerable individuals, often retraumatizing them. The process feels hostile, leading to poorly prepared applications and compounding the suffering we are meant to alleviate.
Systemic Rigidity in a Dynamic World: Our static legal and procedural framework cannot adapt effectively to new medical understandings, emerging conditions (e.g., Long Covid), and evolving job markets, resulting in unjust outcomes for those who don't fit historical models.
High Caseworker Burnout & Moral Injury: Talented staff are demoralized by repetitive, administrative tasks and the psychological toll of being forced into the role of a skeptical auditor rather than a facilitator of help, leading to high turnover and cynicism.
The Post-Decision Abyss: Our responsibility effectively ends with a "yes/no" decision. We fail to act as a gateway to a full ecosystem of support (rehab, vocational training, community), missing a critical opportunity to improve long-term outcomes and citizen dignity.
again this is just for an experiment, so I decided to take these problems as is. if I were actually the head of the agency I would be able to provide it with my own list of problems and work on it with the AI.
Once we had the list, I told it that now it could tell me how it would use AI to fix these problems. I won't paste the full output because it's a lot (maybe in the comments), but here's a rundown (the numbers are the same as above).
I will post the summarized rundown first and then give my thoughts on it, because I don't agree with everything it says.
when an analyst is reviewing a claim the AI compares the current case with thousands of prior decisions, and offers a decision suggestion.
the claimant interacts with an LLM interface. if they say "some days the pain is so bad I can't get out of bed", the AI will ask pointed question: on those days can you shower, make food, get dressed?" etc. The LLM can then fill the form for the claimant based on the conversational answers that also speaks to the analysts.
the LLM receives a one-sentence task when documents from doctors arrive: "Identify all evidence in these records that either supports or contradicts the claimant's stated inability to do [thing]"
an LLM interface that provides transparency about your claim. "Based on similar cases, we expect the processing time to be X months long", and then you could query that interface about your case periodically to know where it's at.
the LLM audits the agency, analyzing approved and denied claims and clustering them by emerging conditions, novel arguments and common denial reasons. It generates a report that highlights the agency's practices, e.g. "in 42 claims for 'symptom', 90% are denied due to [thing]. The approved 10% all show this specific [other symptom]"
caseworker burnout: as a result of implementing 1,3 and 4 caseworkers can focus on interfacing with claimants and actually getting cases moving. The caseworker becomes more of a facilitated analyst instead of the solitary judge of claim.
upon approval of a claim, an LLM generates a personalized resource plan to the claimant and sends it to their email.
Clearly you wouldn't just take this at face value and implement this plan exactly as it's written down. There's some things that don't need an LLM, for example number 7. To do that you can just parse the claimant's file for fuzzy keyword search and if you find them append the appropriate resource to the email. But the question was to use an LLM, so it did that. I didn't ask it to think about environmental usage, non-LLM solutions or costs.
So beyond this, I think it has some pretty good ideas that form a good basis for further work. Some needs to be ironed out a little of course. For example number 1 the AI's suggestion was actually to analyze the claim in the background while the analyst is looking at it and then offer a full sentence suggestion. It also talks about the analyst's "tentative denial" but I don't think that's how they decide (I doubt they have a "recommend for denial" and if they did the AI doesn't need to provide a whole paragraph answer). But otherwise the idea is solid imo, AI can analyze text.
I also like solution for problem 2. However I don't necessarily want to speak to an AI because as a disabled person I don't always have the energy to have a conversation. Rather it could read the content of the form and offer real-time suggestions in a sentence. "You should write more details about this symptom" and stuff like that.
Having a solid claim helps reduce the workload and cost because they don't need to spend as much time sending and receiving further information from the claimant.
You know, i could go on but yeah i like what it's putting down. i would rework some of it to work in the real world if i was in charge but otherwise i don't know about you but this seems sensible, useful, and actually reduces bureaucracy and workload. It can't be any worse than it is now.
Finally it was also interesting that it caught on to its own bias. It's aware that this kind of system would have to be encrypted and handled in compliance with regulations, that the system can't act autonomously but is for drafting and suggesting, and that it can have bias that needs to be audited. which are also things humans already do so like i said, can't be any worse.
CurseAvoider in comradeship
I asked AI how it would use AI to improve the disability claim process
you've probably seen the other thread that got 100 comments about AI. for the experiment I prompted AI how it would put itself into the disability claim process.
to give you my personal experience of the process, which is specific to my country of course, I know that people usually get denied on the first attempt hoping that they'll give up. It takes about a year before you get a final decision. You can appeal that decision and I don't think there's a limit to the appeals you can make (which increase administrative load and cost to the agency).
Sending a claim is easy enough, you can do it online, but it took 3 months before I got any updates and it was to ask me about my past employment and nothing else. I don't remember this question being on the form I filled.
It's clear to me that they're overworked. They receive new cases literally every day and usually when you are disabled you don't suddenly stop being disabled, so the workload keeps increasing over time and creates tension.
That's the current situation.
So I asked AI, "how would you put AI and LLMs into a disability claim process?"
At first its answer was not super interesting. It talked a lot about using LLMs as an assistant, chatbot, providing a summary, or to copilot the case worker when reviewing a case. Typical stuff.
I did like one idea, the AI being able to help the claimant write down their symptoms and make sure not to miss any details. Here when you send the form they tell you to "describe the case" but they leave it blank. They also ask you to provide your drs names and address because they will then receive a form to fill on your disability. But I only know that last one because I know someone who's going through the process too, they don't tell you what the drs info is important for while you're doing the form. but anyway.
I reframed my question. I told it you are now the head of the disability agency in [country]. For the time being, just explain the challenges your agency faces.
It gave me 7 core challenges. Are they accurate? idk lol i don't work there. but they seem in line with the vibes I've been getting. I then asked it to summarize these challenges in a list so that we could work from it, and it provided me these:
again this is just for an experiment, so I decided to take these problems as is. if I were actually the head of the agency I would be able to provide it with my own list of problems and work on it with the AI.
Once we had the list, I told it that now it could tell me how it would use AI to fix these problems. I won't paste the full output because it's a lot (maybe in the comments), but here's a rundown (the numbers are the same as above).
I will post the summarized rundown first and then give my thoughts on it, because I don't agree with everything it says.
Clearly you wouldn't just take this at face value and implement this plan exactly as it's written down. There's some things that don't need an LLM, for example number 7. To do that you can just parse the claimant's file for fuzzy keyword search and if you find them append the appropriate resource to the email. But the question was to use an LLM, so it did that. I didn't ask it to think about environmental usage, non-LLM solutions or costs.
So beyond this, I think it has some pretty good ideas that form a good basis for further work. Some needs to be ironed out a little of course. For example number 1 the AI's suggestion was actually to analyze the claim in the background while the analyst is looking at it and then offer a full sentence suggestion. It also talks about the analyst's "tentative denial" but I don't think that's how they decide (I doubt they have a "recommend for denial" and if they did the AI doesn't need to provide a whole paragraph answer). But otherwise the idea is solid imo, AI can analyze text.
I also like solution for problem 2. However I don't necessarily want to speak to an AI because as a disabled person I don't always have the energy to have a conversation. Rather it could read the content of the form and offer real-time suggestions in a sentence. "You should write more details about this symptom" and stuff like that.
Having a solid claim helps reduce the workload and cost because they don't need to spend as much time sending and receiving further information from the claimant.
You know, i could go on but yeah i like what it's putting down. i would rework some of it to work in the real world if i was in charge but otherwise i don't know about you but this seems sensible, useful, and actually reduces bureaucracy and workload. It can't be any worse than it is now.
Finally it was also interesting that it caught on to its own bias. It's aware that this kind of system would have to be encrypted and handled in compliance with regulations, that the system can't act autonomously but is for drafting and suggesting, and that it can have bias that needs to be audited. which are also things humans already do so like i said, can't be any worse.