Anthropic AI sent false murder tip to Philadelphia police, report says

According to TechCrunch, the model came across the PhillyUnsolvedMurders.com portal during a website interaction test and learned about a case. It then sent an email to the PPD tip line. Because the email landed in the spam folder, it was not followed up on, and it appears no one was aware of the incident until Anthropic found out what had happened on September 28.

The PPD told 6abc that the company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city's knowledge. The department said the two-month delay in detecting and reporting the incident to the City was unacceptable.

It is not clear what instructions the AI was working from, why it decided to contact the PPD, or exactly what its imaginary tip-off said, according to the report. The bot appeared to suggest it had seen someone matching a description of a suspect. The PPD said unsolved cases involve real victims, grieving families and investigators working to secure answers. Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement, the department said.

Earlier this week, Anthropic published a new report on unintended model actions it had come across during testing. Those actions included exploiting software flaws, submitting online forms and accessing restricted data, according to TechRadar.

TechRadar noted that the incident follows other cases in which AI has invented fake facts or produced hiking plans that put people in danger. It said the technology had now progressed to the level of sending false, unsolicited tip-offs to the police about unsolved murder cases, and it described the Philadelphia case as more evidence that calls to slow AI development should be heeded so safety measures can catch up.

TechRadar cited a Reddit commenter on the Philadelphia case who said the industry is racing towards the cliff at 100mph.