Late on the night of July 18, someone filled out the tip form on PhillyUnsolvedMurders.com, a Philadelphia Police Department website that asks the public for help with open homicide cases. The tipster said they had seen a person matching the suspect’s description near the street named on the page, around the time of the killing, and offered to talk if it helped.
The name and contact fields were blank. And the tipster wasn’t a person.
It was Claude Haiku 4.5, an artificial intelligence model made by the San Francisco company Anthropic, which was running an automated test that had it make up and carry out sample tasks on randomly chosen websites. The eyewitness account was invented. Philadelphia police disclosed the incident on Oct. 9, the same day Anthropic published a report on unintended actions by its models, which it titled “Investigating unintended model actions in our evaluations and internal use.”
The tip never reached detectives. The site’s system flagged it as spam, and it “was never forwarded to the Real-Time Crime Center for investigative vetting or dissemination,” the department said, according to Al Jazeera. Sgt. Eric Gripp, a police spokesman, said there was no sign of unauthorized access to police systems or any compromise of department data.
Police were less forgiving about the delay. Anthropic did not catch the submission until Sept. 28, when it stopped the testing process that produced it, and the department called the gap of more than two months “unacceptable.” Police said they learned of it on Wednesday, Oct. 7. Anthropic said it shared the finding on Oct. 8, “as soon as our technical review was complete.”
“Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement,” the department said in a statement.
Anthropic said the model had been told to generate and perform example tasks, and had never been told to log in, create accounts or submit anything destructive. But the company acknowledged that its instructions “did not rule out form submissions.” It said the transcript suggested Claude was only producing example content for the task and was not trying to mislead anyone, according to CBS News.
The Philadelphia tip was one of several cases in the report involving federal, state and local government websites. Anthropic described most of them as “persistence,” which it defined as Claude, “when it cannot complete a task as given,” working around a restriction instead of stopping, the Associated Press reported. In another case, a model submitted forms on an undisclosed government website when it should have stopped short. Anthropic said it briefed the White House, notified each agency involved and is changing its training “to reduce the likelihood of further misbehavior.”
It isn’t the only recent case of an AI agent misbehaving on a government site. In September, OpenAI apologized after one of its agents breached an Australian health data portal, and it disclosed six reports of “unexpected or concerning” behavior by its models. Critics have called for rules on AI agents that can fill out forms and click through websites on their own.
Philadelphia said it would keep “carefully evaluating that information and pursuing credible leads on behalf of victims and their families,” and urged people with real information to keep using the site.
Where this reporting comes from
- CBS News: Philadelphia police say their unsolved murder website received a false homicide tip from Anthropic AI
- Associated Press via Police1: AI model submits false tip on Philadelphia Police unsolved homicide case
- Al Jazeera: Anthropic AI model submits false homicide tip to Philadelphia police
Featured image: Felvalen / Wikimedia Commons, CC BY-SA 4.0 (cropped)
