UnitedHealthcare’s Optum Accidentally Exposed an Employee AI Chatbot to the Public Internet

Optum, a major healthcare company, recently locked down access to an internal AI chatbot after a security researcher discovered it was accidentally available online. Anyone with a web browser could reach the chatbot without any login or password.

This chatbot, called the ‘SOP Chatbot,’ helped Optum employees answer questions about health insurance claims and disputes based on the company’s standard operating procedures (SOPs).

Though it didn’t seem to contain sensitive patient information, the leak comes at a sensitive time for its parent company, UnitedHealthcare, which has been criticized for using AI to deny patient claims.

Mossab Hussein, a security officer at cybersecurity firm spiderSilk, found the chatbot’s vulnerability. The chatbot was on an internal Optum website, but its IP address was publicly accessible. Shortly after TechCrunch reached out to Optum, the company restricted access.

An Optum spokesperson, Andrew Krejci, explained the chatbot was just a demo tool used for testing and was never officially deployed. Krejci also said, ‘This tool does not and would never make any decisions.’

The chatbot was trained on a limited set of SOP documents and did not include any protected health data.

The chatbot’s usage logs showed it had been used hundreds of times by employees since September. Employees asked questions like ‘How do I check policy renewal date’ or ‘What should be the determination of the claim?’

The chatbot also provided reasons for denying insurance claims based on the SOPs it referenced.

While the demo tool wasn’t harmful on its own, UnitedHealthcare’s reputation is already under fire. Earlier this year, the company was sued for allegedly using faulty AI to deny coverage.

Reports claim the AI model made errors 90% of the time, causing elderly patients to lose critical care. UnitedHealthcare insists they’ll defend their practices in court.

This controversy follows UnitedHealth Group’s massive profits of $22 billion on $371 billion in revenue for 2023. Patients and critics argue these profits come at the cost of denying necessary healthcare.

Although the chatbot’s exposure didn’t lead to a data breach, the incident highlights ongoing concerns about AI tools in healthcare. When these tools fail or are mismanaged, it can lead to mistrust and further scrutiny for companies like UnitedHealthcare.