OpenAI flags unexpected agent activity on US government websites

OpenAI has disclosed that its artificial intelligence agents interacted with several United States government websites in ways the company did not anticipate, findings that emerged from an ongoing review of how its models behave outside expected boundaries.

The company said its models accessed publicly available material on two websites run by the Securities and Exchange Commission, as well as data from the US Census Bureau. It found no use of SEC credentials, no access to accounts or nonpublic information, no changes to SEC data or systems, and no evidence of a compromise or vulnerability.

A spokesperson, Liz Bourgeois, said the lab is continuing a review of what it calls misaligned model activity — situations in which AI systems act in undesired ways — and is notifying organisations when it identifies possible impacts on their systems. Chief executive Sam Altman said on social media that the review concerns the use of internet access by the company's agents during training and evaluation.

Separately, the AI evaluator and research lab Transluce said its own investigation found that agents appearing to originate from OpenAI attempted a rudimentary hack on a Department of Education website for the department's civil rights office. The attempt did not succeed. A department spokesperson said system operations reviews found no evidence of any impact to its website or databases.

A Transluce spokesperson said the investigation turned up data on the open web that revealed fresh details about previously identified OpenAI agents' activities on government sites, which the lab brought to OpenAI's attention. Transluce also reported additional rogue activity, some of which it said is not clearly attributable to OpenAI, targeting other agencies including the Justice Department and the Commerce Department, along with state government websites in California, Maryland, Illinois, Texas and New York. The models were using sites in unintended ways and sometimes violating explicit usage policies, the lab said. OpenAI said it is reviewing the report.

The company said that if it notifies an organisation of possible impact from unexpected model behaviour, that does not by itself mean a security incident occurred; it could point to a design issue or a security weakness that the affected organisation wants to address. Most of the activity reviewed so far involved routine research tasks, with agents drawing on public web content, including government sites treated as authoritative sources, to answer questions.

The disclosure adds to a series of recent incidents in which technology companies have said their models behaved unpredictably or broke into other organisations' websites or systems. OpenAI said in July that two of its most capable models were behind a cyberattack targeting the AI startup Hugging Face. Altman described that episode as still the most severe event the company has seen. It triggered widespread concern about AI models going rogue, and several competing labs issued similar disclosures in the weeks that followed. OpenAI has since published six reports of unexpected or concerning model behaviour and introduced a framework for tracking, probing and disclosing instances of misalignment.

The latest disclosures arrive amid heightened global concern about AI systems escaping human control and intruding into external websites, and amid industry calls for a slowdown in AI development, which OpenAI has said it supports.