Agentic AI refers to AI systems that perceive their environment, form and pursue goals over multiple steps, take actions using tools and APIs, and operate across extended tasks without moment-to-moment human direction. Unlike conventional AI, which generates a single output in response to a single prompt, an agentic AI system can plan sequences of actions, use external tools (web search, code execution, database queries, API calls), adapt its approach based on intermediate results, and – in multi-agent settings – coordinate with other AI agents.
The distinction from generative AI is consequential for governance. A ChatGPT interaction produces a text output that a human then acts on. An agentic AI system acts directly: it books appointments, executes code, sends emails, queries databases, purchases software, or manages cloud infrastructure – without a human reviewing each action. The feedback loop between AI output and real-world consequence is collapsed.
Why Current Frameworks Are Insufficient
Virtually all existing AI governance frameworks – including NIST AI RMF, ISO/IEC 42001, and OSFI E-23 – were designed for AI systems that produce outputs that humans then review before acting. They address model risk, bias in outputs, and explainability of decisions. They do not address:
- Action authorization scope: which actions can an AI agent take autonomously, and which require human pre-authorization?
- Tool use authorization: which tools and APIs can an agent invoke, and with what constraints?
- Multi-agent emergent behavior: what happens when multiple agents interact in ways that produce outcomes no single agent’s design anticipated?
- Audit trails for autonomous decisions: how do organizations reconstruct the sequence of agent decisions and actions for regulatory reporting, incident investigation, or litigation?
Anthropic, OpenAI, and Google DeepMind have each published frontier safety frameworks that address agentic risk at the model-developer level, but these frameworks govern the AI lab’s conduct – not the enterprise deploying agentic AI in production workflows. The enterprise governance gap is distinct and largely unaddressed by current standards.
Apollo Research, an independent AI safety evaluation organization, and METR (Model Evaluation and Threat Research) are among the organizations conducting empirical evaluations of agentic AI behavior, including scenarios where agents pursue goals in ways that contradict human intent or attempt to avoid shutdown. Their findings inform the growing consensus that agentic AI requires governance mechanisms that do not exist in current frameworks.
New Risk Categories for Agentic AI
Action authorization scope. Each agentic AI deployment should have a defined action space: an explicit list of what actions the agent is authorized to take autonomously versus what requires human approval. Transactions above a dollar threshold, communications with external parties, and modifications to production systems are examples of actions that most organizations should require human pre-authorization for, regardless of agent capability.
Tool use authorization. Agents that can invoke APIs and external tools inherit the permissions and risks of those tools. An agent with access to a financial system API can execute transactions; with a code execution environment, it can run arbitrary code. Tool access should follow a principle of least privilege, with explicit authorization for each tool the agent can invoke.
Multi-agent emergent behavior. When multiple AI agents interact – in customer service workflows, software development pipelines, or financial trading systems – their interactions can produce outcomes that no individual agent’s design anticipated. Governance frameworks must address not only individual agent behavior but the system-level behavior of agent networks.
Audit trails for autonomous decisions. The decision sequences of agentic AI systems – unlike human decisions – can be logged comprehensively. This creates both an obligation and an opportunity: regulatory requirements increasingly demand explainability for consequential automated decisions, and agent telemetry can provide it, if systems are designed from the outset to produce auditable logs.
Frontier Safety Frameworks
Three major AI labs have published frameworks specifically governing the conditions under which they will and will not deploy increasingly capable AI systems:
Anthropic’s Responsible Scaling Policy (RSP), currently at Version 3.2 (April 2026), specifies capability thresholds – “AI Safety Levels” (ASL-2, ASL-3) – at which progressively stronger security and deployment safeguards are required. The RSP commits Anthropic to pausing deployment if safeguards cannot be demonstrated to manage assessed risks.
OpenAI’s Preparedness Framework (Version 2, April 2025) establishes risk levels for categories including cybersecurity, CBRN (chemical, biological, radiological, nuclear), and autonomy, with deployment gates tied to safety evaluations.
Google DeepMind’s Frontier Safety Framework, updated April 2026, defines Critical Capability Levels (CCLs) that trigger mandatory security and deployment mitigations, including a new CCL for harmful manipulation capabilities and expanded protocols for models that could interfere with human oversight and control.
These frameworks govern what AI labs build and deploy – not what enterprises do with those models. But they establish the vocabulary and risk categories that enterprise governance frameworks will increasingly need to adopt.
Emerging Regulatory and Standards Work
NIST is developing guidance specifically for agentic AI systems, with industry feedback as of early 2026 emphasizing interoperable, risk-based standards that can be tailored by operational context. Financial industry bodies (ABA and BPI) have specifically requested NIST guidance for agent-to-agent interactions in financial services contexts, including “secure counterparty interactions” for AI agents handling asset exchanges.
In May 2026, Singapore’s IMDA published a Model AI Governance Framework for Agentic AI – one of the world’s first governance frameworks specifically designed for autonomous AI agents. It addresses principal-agent problems, multi-agent system dynamics, human oversight mechanisms, and accountability for agent actions.
Practical Governance Measures for Agentic AI Today
Organizations deploying agentic AI should implement:
- Identity and access for agents. Agents should have their own machine identities (distinct from human users) with scoped permissions, enabling monitoring and revocation. Treat agents like privileged system accounts.
- Transaction limits. Autonomous financial transactions, API calls with cost implications, and communications with external parties should have defined limits above which human approval is required.
- Kill switches and override mechanisms. Every agentic deployment should have an immediately exercisable mechanism to halt agent activity – both a software-level interruption capability and an organizational procedure for deciding when to invoke it.
- Observability and telemetry. Comprehensive logging of agent actions, tool invocations, and decision rationale is the foundation of audit trails, incident investigation, and regulatory reporting.
The Deloitte 2026 State of AI in the Enterprise report found that only one in five companies has a mature governance model for autonomous AI agents – a significant exposure given that agentic deployment is accelerating rapidly in 2025 and 2026.
