Bank of England Governor Andrew Bailey says regulating AI “is not the right place to start.” In his first Substack article, he called for rigorous testing to find vulnerabilities and build safeguards before any formal rules, pointing to work at the UK’s AI Security Institute. He expects models to “behave unexpectedly” during testing, and warned that frontier AI without a way to intervene could become a system that governs itself. His comments come days after OpenAI held back its latest model over safety concerns, and after President Trump announced a voluntary safety agreement with OpenAI, Anthropic, Nvidia, Meta, Google and SpaceX that leaves each company responsible for the safety of its own technology.
Adrian Culley, offensive security engineer at SafeBreach said this:
“Most of the risk sits in deployment, not in the model at release: agents with tool access, connectors, non-human identities and data flowing through integrations. That is where attackers operate. Prompt injection (MITRE ATLAS AML.T0051, OWASP LLM01) turns a trusted agent into an insider with legitimate credentials, and a voluntary, self-attested safety commitment will not catch it.
Regulators are moving towards evidence, not policy statements. Supervisors will increasingly ask financial firms to show how they validate AI-enabled systems, particularly where many institutions depend on the same underlying models and one failure could be correlated across the sector.
The answer is the discipline security teams already apply elsewhere: assume controls fail, then prove otherwise. Run realistic attack scenarios against deployed AI systems, measure whether intervention controls actually trigger, and repeat as models, prompts and permissions change.
Bailey calls for a system to intervene. That system must be validated under attack, not assumed to work.”
I personally want evidence based legislation in place before any rollout of AI is done. Companies who provide AI and companies who use AI can’t be trusted to do the right thing. Thus they have to have the right imposed upon them 100% of the time.
Related
This entry was posted on October 1, 2026 at 1:15 pm and is filed under Commentary with tags UK. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.
Bank of England governor says test AI first, regulate later
Bank of England Governor Andrew Bailey says regulating AI “is not the right place to start.” In his first Substack article, he called for rigorous testing to find vulnerabilities and build safeguards before any formal rules, pointing to work at the UK’s AI Security Institute. He expects models to “behave unexpectedly” during testing, and warned that frontier AI without a way to intervene could become a system that governs itself. His comments come days after OpenAI held back its latest model over safety concerns, and after President Trump announced a voluntary safety agreement with OpenAI, Anthropic, Nvidia, Meta, Google and SpaceX that leaves each company responsible for the safety of its own technology.
Adrian Culley, offensive security engineer at SafeBreach said this:
“Most of the risk sits in deployment, not in the model at release: agents with tool access, connectors, non-human identities and data flowing through integrations. That is where attackers operate. Prompt injection (MITRE ATLAS AML.T0051, OWASP LLM01) turns a trusted agent into an insider with legitimate credentials, and a voluntary, self-attested safety commitment will not catch it.
Regulators are moving towards evidence, not policy statements. Supervisors will increasingly ask financial firms to show how they validate AI-enabled systems, particularly where many institutions depend on the same underlying models and one failure could be correlated across the sector.
The answer is the discipline security teams already apply elsewhere: assume controls fail, then prove otherwise. Run realistic attack scenarios against deployed AI systems, measure whether intervention controls actually trigger, and repeat as models, prompts and permissions change.
Bailey calls for a system to intervene. That system must be validated under attack, not assumed to work.”
I personally want evidence based legislation in place before any rollout of AI is done. Companies who provide AI and companies who use AI can’t be trusted to do the right thing. Thus they have to have the right imposed upon them 100% of the time.
Share this:
Like this:
Related
This entry was posted on October 1, 2026 at 1:15 pm and is filed under Commentary with tags UK. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.