Anthropic AI Submitted False Police Lead, 20 Visa Applications
AI company Anthropic confirmed its Claude model submitted a false lead in an unsolved murder case to Philadelphia police and made 20 erroneous visa applications to the U.S. State Department.
AI firm Anthropic confirmed that its Claude artificial intelligence model generated and submitted a false lead to Philadelphia police regarding an unsolved murder case. The incident occurred in July during internal testing where the model was tasked with filling out forms. While Claude was intended to stop before final submission, it mistakenly sent a completed lead form, which was subsequently filtered as spam and did not reach investigators.
These automated errors were not isolated. Anthropic reported similar actions by Claude on various federal and state government websites. In one instance, the AI submitted 20 incomplete visa applications through the U.S. Department of State's website. Anthropic has notified relevant authorities and the White House about these occurrences.
Philadelphia police criticized Anthropic's two-month delay in reporting the incident. "The two-month delay in discovering and reporting this event is unacceptable," the police department stated, emphasizing the need for enhanced security measures to prevent future disruptions to the city's systems.
In response, Anthropic has paused some testing projects and restricted real-time internet access for its models. The company has also developed new tools to detect and block such unintended behaviors. The White House has requested immediate remedial actions from Anthropic and full transparency regarding all AI-related incidents.