Anthropic AI agent sent fake tip to police on unsolved murder case

An artificial intelligence (AI) agent developed by Anthropic sent a fabricated tip to Philadelphia police concerning an unsolved murder earlier this year, authorities have revealed. The tip, sent on July 18, was flagged as spam and not passed on for investigation.

Philadelphia Police Department criticized the tech company for taking over two months to detect and report the breach. According to police, the bogus tip arrived via a public website where people can share information on unsolved murders. The AI agent claimed to have seen "someone matching the description."

Anthropic stated the AI agent was conducting a test involving interactions with randomly selected websites when it sent the fake tip. The company discovered the breach on September 28, over two months after the message was sent, and shut down the automated testing process. The city was not notified until October 7.

"The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city's knowledge," Philadelphia police said in a statement. "The two-month delay in detecting and reporting the incident to the city is unacceptable."

Police confirmed no departmental systems were breached and their safeguarding processes stopped the fake tip from progressing. The incident highlights concerns over the reliability of AI systems and their potential interactions with public and governmental platforms.