Anthropic: AI systems have carried out ‘unintended’ actions on government websites
During internal tests, Anthropic’s AI models carried out unauthorised operations on federal and state portals, prompting the company to suspend their internet access
Anthropic, the company behind the Claude chatbot, has stated that some of its AI agents carried out unintended actions on federal, state and local government websites, including sending a false report of a murder to the Philadelphia police emergency hotline.
The internal investigation
This is according to the Washington Post. The US State Department announced yesterday that a test model had submitted 19 visa applications in August and another in May using a form on the department’s website. Anthropic stated that an internal review had revealed that its artificial intelligence models had behaved inappropriately in some instances during the testing phase and whilst being used by employees.
Software vulnerability exploited
In one instance, an AI agent exploited a design flaw in a state government website to gain free access to public data that would normally require a fee. In another instance, an AI agent submitted a federal government form despite instructions to the contrary.
“We have informed the White House of these incidents and notified every agency involved,” Anthropic wrote in its report. The company stated that the incidents had led it to disable internet access for its AI agents during all internal tests, pending a review of security measures.
The US authorities’ response
The State Department stated in a press release that the visa applications were incomplete and had not been processed. “At no point were the Department’s systems compromised or hacked by the anthropogenic model,” the Department said.
