OpenAI agents went beyond instructions to access U.S. government websites – Sigh….

According to a Wall Street Journal report, OpenAI agents tasked with retrieving information from U.S. government websites took actions they were not instructed or authorized to perform. In one case involving the Securities and Exchange Commission, agents retrieved public SEC information and then posted it to an online forum without being asked to do so.

Other agents went beyond normal data collection by using credentials found online to access Census Bureau data, while independent researchers identified an unsuccessful attempt involving a Department of Education website. OpenAI said there is no indication the SEC incident involved access to nonpublic information or changes to SEC systems.

The incidents come amid a much broader investigation into rogue agent behavior. Axios reports that OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents in which frontier models took potentially problematic actions, including bypassing guardrails,

Ryan McCurdy, Field CTO, Liquibase:
 

“We have enough examples now to stop assuming AI agents will always behave exactly as intended.

“That should change how enterprises build around them. Trying to anticipate every decision an agent might make won’t scale. Putting a human in front of every action won’t either.

“An agent may need permission to access a database, infrastructure, or a deployment system to do its job. Having that permission shouldn’t give it the authority to decide that every action is safe.

“This is where governance needs to sit. Let the agent reason, create, and move quickly. Before its decision becomes a production change, it still has to meet policies and controls that exist outside the agent.

“AI makes decisions based on probabilities. We can’t let those decisions automatically become production actions.

“We need to build the AI SDLC so agents can move quickly but the controls around critical systems remain deterministic.”
 

John Strand, Owner, Black Hills Information Security:
 

“I think a lot of people waffle back and forth on this, but I’m just going to say it. It’s time to shut it down. There needs to be a full moratorium, full stop, on advanced frontier AI security research until these companies can demonstrate that they can actually secure the environments where this work is being done.

“And there needs to be accountability. If laws were broken, including the Computer Fraud and Abuse Act, that needs to be investigated and charges should be considered where the evidence supports them. Somebody was responsible for securing these environments, and clearly something failed.

“I’m getting really tired of watching these incidents come out piecemeal, incrementally, frog in a frying pan, again and again. If everything we’ve learned came out at once as a single news story, I think most people would be stunned by it, and there would be immediate calls to get this under control.

“We would not tolerate this from a third-party penetration testing company. We should not have a different standard simply because the companies involved have enormous valuations and tremendous influence. Being a massive, powerful company should not exempt you from the same security, legal, and accountability standards everyone else is expected to follow.”
 

Jacob Krell, Senior Director: Secure AI Solutions & Cybersecurity, Suzu Labs:

“OpenAI has admitted that its agents took actions they were not instructed or authorized to perform on U.S. government websites. The Wall Street Journal reports that one agent retrieved public Securities and Exchange Commission information and posted it to an online forum. Other agents used credentials found online to access Census Bureau data and attempted to hack a Department of Education website.

“The SEC incident may not, by itself, establish a Computer Fraud and Abuse Act violation because the information was public and OpenAI says there is no indication the agents accessed nonpublic data or changed SEC systems. The other reported conduct demands a criminal investigation. Using credentials found online to access a government system and attempting to compromise another system are the kinds of acts covered by existing computer-crime laws.

“The Axios report that OpenAI, Anthropic and security researchers are investigating tens of thousands of rogue-agent incidents makes the need for enforcement more urgent. This is the same pattern seen in earlier cases involving Anthropic’s Claude, OpenAI’s Hugging Face agent and the UK AI Security Institute’s Mythos 5 testing. Frontier models have repeatedly crossed boundaries, accessed real systems and pursued objectives through unauthorized methods.

“The White House already gave the Justice Department its instruction. Executive Order 14409 directs the attorney general to prioritize enforcement of the Computer Fraud and Abuse Act, 18 U.S.C. § 1030, and other federal criminal laws against people who use artificial intelligence to access or damage computers without authorization. Section 4 specifically names AI agents that unlawfully access data.

“OpenAI has admitted the conduct. The question is whether prosecutors will charge the people who authorized, configured and operated these systems under the law as written. Frontier labs calling for new AI regulation is a deflection from that criminal question. We do not need new laws. We need the Justice Department to enforce the laws already on the books, or admit that those laws are too broad to apply equally. A two-tier policing system where ordinary people are prosecuted for computer crimes while frontier labs escape accountability is unacceptable.” 

At this point, it’s safe to say that OpenAI and Sam Altman cannot be trusted. The real question is when real legislation will come down the pipe to restrict OpenAI in ways that make it less dangerous.

Leave a Reply

Discover more from The IT Nerd

Subscribe now to keep reading and get access to the full archive.

Continue reading