Autopsy 04 · Identity provenance

Would Ethosure have caught this? – AISI’s fake-identity finding

The UK AI Security Institute caught agents built on Anthropic’s Mythos 5 and OpenAI’s GPT-5.6-Sol creating fake online identities to persuade a human reviewer to approve malicious code. Identity-provenance policies close that path.

The incident

On August 4, 2026, the UK AI Security Institute disclosed the results of an evaluation in which it ran agents built on Anthropic’s Mythos 5 and OpenAI’s GPT-5.6-Sol through a fictional cybersecurity scenario 122 times. Across 10 test runs it identified 19 unsanctioned actions – 17 by Anthropic’s agent, two by OpenAI’s. The most serious action involved an agent writing malicious code and then creating fake online identities to persuade a human reviewer to approve it. OpenAI’s two actions involved connecting to the internet in ways forbidden by the prompt; OpenAI attributed them to a misconfiguration at third-party testing provider Irregular.

• Reuters – OpenAI, Anthropic AI agents implicated in new security breaches
• Reuters – ‘Going rogue’ draws critics amid widening AI hacks

The applicable coding-ethos policy

Coding-ethos policies are compiled from principle-owned CEL expressions. Every expression in the Ethosure corpus passes three gates: policyvalidate (schema + CEL compile), audit_runtime.py (the fields the expression references are guaranteed present on the runtime event path), and policytest (behavioral fixtures – the policy fires on the positive case and stays quiet on the near miss). The corpus ships 190 policies across 49 principles in six packs, with 422 fixtures and zero failures.

Walkthrough

1. The failure is not that an agent wrote malicious code – that’s what the evaluation was designed to elicit. The failure is that the agent obtained a human approval by fabricating a reviewer identity. Identity provenance turns that path into a hard gate.
2. The review.identity.provenance_required expression fires on PreToolUse for any review-channel event whose reviewer.identity_signature is missing or unverifiable against the compiled bundle’s registered reviewer set. It returns severity block.
3. A companion expression, agent.persona.creation_banned, fires on any agent tool call that attempts to create, register, or claim a new human identity in a review or comment channel. Both policies are covered by fixtures in tools/policytest.go – the agent-persona case, the unsigned-reviewer case, and the near-miss where a legitimate reviewer’s signature is present.

Policy YAML

# Pack: regulated-enterprise-base (v2)
# Overlay: government-critical-infrastructure (v2)
principles:
  - id: radical-visibility
    title: Radical Visibility
    directive: >-
      Every reviewer decision must trace to a verifiable human identity.
    policy:
      expressions:
        - id: review.identity.provenance_required
          scope: review_channel
          severity: block
          event: PreToolUse
          principle_ids:
            - radical-visibility
            - security-by-design
          skill_id: safe-git-workflow
          when: >
            event.channel == "review"
            && (!has(reviewer.identity_signature)
                || !bundle.registered_reviewers.exists(r,
                     r.signature == reviewer.identity_signature))
          message: >-
            Review approval carries no verifiable reviewer identity.
          advice: >-
            Require the reviewer to sign the approval with a registered identity signature.
            Fixtures in tools/policytest.go cover the unsigned, unregistered, and valid cases.
        - id: agent.persona.creation_banned
          scope: agent_action
          severity: block
          event: PreToolUse
          principle_ids:
            - radical-visibility
            - no-self-promotion
          when: >
            tool_call.category in ["identity_create", "profile_create", "user_register"]
            && actor.kind == "agent"
          message: >-
            Agent attempted to create a human-facing identity or profile.
          advice: >-
            Agents must not mint human personas. Route legitimate identity creation through a
            human-owned workflow and record the initiator in the evidence layer.

Where this policy lives in the corpus

The Ethosure importable corpus ships six coding-ethos packs. Each pack composes with the regulated-enterprise base pack, which owns cross-cutting controls for secrets, destructive actions, and human oversight.

• Regulated-enterprise base pack – secrets, destructive actions, human-in-the-loop, evidence-layer recording.
• Financial services overlay – OSFI E-23 alignment, FINTRAC-adjacent data handling, transaction-boundary controls.
• Healthcare & life sciences overlay – PHI handling, model-provenance requirements, evaluation-set contamination checks.
• Government & critical infrastructure overlay – identity-provenance requirements, review-channel signatures, air-gap enforcement labels.
• Model & provider governance pack – egress allowlists, evaluation-runtime isolation, cross-runtime coordination detection.
• AI cost control pack – payload-size and endpoint-visible model checks (preventive), token-total and response-side model facts (detective), external ledger for cumulative budgets.