Every compliance professional knows the feeling. An audit is announced, a request list arrives, and the next two weeks are spent assembling a folder: screenshots of dashboards, log files requiring manual interpretation, policy documents with no clear link to the systems they describe, email threads that sort-of document a decision made six months ago. It is not evidence; it is archaeology.
For AI agents, this problem is an order of magnitude worse. An agent taking hundreds of actions per session, across dozens of systems, over weeks of operation generates more potential audit material than any manual assembly process can manage. And with the EU AI Act, OSFI AGILE, and FINTRAC demanding verifiable evidence of AI governance controls, the screenshot folder approach is no longer acceptable.
What auditors actually need
A regulator or internal audit team is asking three questions:
What was this AI system allowed to do? The auditor needs a documented, versioned policy defining the agent’s scope, access rights, and behavioral constraints – not a description of the policy, but the policy itself, in a form that can be read and evaluated.
Did the system operate within that scope? For every material action the agent took, the auditor needs evidence that the action was evaluated against the policy *before it executed*, and that the outcome was consistent with the policy. “I believe the agent operated correctly” is not evidence. A log entry showing action, policy evaluated, decision made, and timestamp *is* evidence.
When something fell outside scope, what happened? The auditor is looking for evidence that the governance system detected and responded to violations. An enforcement layer that never blocked anything is suspicious. One that blocked known-bad actions, escalated ambiguous ones, and transformed risky operations into safer forms is evidence of a functioning control.
Why the current approach fails
Most AI governance implementations fail the audit test for the same reason: the governance function is disconnected from the runtime. Organizations document policies, run AI systems, and attempt to link the two after the fact when evidence is needed.
After-the-fact reconstruction has several failure modes:
Completeness gaps. No one captured the right data at the time of the action because no one anticipated the audit request.
Provenance breaks. The log shows what happened but not which policy provision applied, which version was in effect, or what criteria led to the decision. Without provenance, the log is a record of activity, not a record of governance.
Tamper risk. Logs that can be modified after the fact do not meet the evidence standard for a serious audit. Regulators are increasingly explicit about the requirement for append-only, tamper-evident logging.
Velocity mismatch. An agent executing hundreds of actions per session produces more evidence candidates than any post-hoc process can handle. The result is sampling – and sampled evidence argues the system worked *on average*, not on the specific transaction a regulator is asking about.
What a real evidence pack contains
A real evidence pack for an AI governance audit has four components:
- The active policy record. The versioned policy bundle in effect at the time of each audited action. Machine-readable, not prose – specifying exactly what the policy evaluated and what it permitted or prohibited.
- The action log. An append-only, structured record of every material agent action: what was attempted, when, by which agent identity, in which context.
- The enforcement decision record. For each logged action, the outcome of policy evaluation – allow, transform, escalate, or block – linked to the specific policy provision, the version in effect, and the rationale for the decision.
- The exception record. All actions escalated or blocked, with full provenance. This is the controls-effectiveness evidence demonstrating the governance system is not a rubber stamp.
These four components, assembled automatically and continuously, constitute an evidence pack auditors can work with directly.
Ethosure’s evidence architecture
Ethosure’s enforcement core – built on the `coding-ethos` open-source project – was designed with audit evidence as a first-class requirement. Every enforcement decision produces an entry in the append-only JSONL evidence ledger: the action attempted, the policy bundle version evaluated, the CEL rule, the disposition, the timestamp, and the agent identity. SARIF-formatted findings are emitted alongside raw events, providing structured output that compliance teams can ingest directly.
The evidence ledger is not a reporting add-on; it is produced by the same process that makes the enforcement decision. There is no gap between what the enforcement engine decided and what the audit record shows – they are the same artifact. Ethosure links each SARIF finding and ledger entry to relevant NIST AI RMF and ISO/IEC 42001 control identifiers, so a compliance review navigates directly from regulatory requirement to satisfying evidence.
The audit preparation that becomes a continuous state
When an audit request arrives, the evidence pack is not assembled – it is queried. The append-only ledger contains a complete, verifiable record of every governance decision since deployment. The compliance team’s job shifts from archaeology to presentation.
That shift is not a luxury; it is what the regulatory environment now requires. The EU AI Act’s logging obligations, OSFI AGILE’s oversight requirements, and FINTRAC’s enforcement history all point to an evidence standard that screenshot folders cannot meet. The organizations that build real evidence infrastructure now are the ones that answer their next audit with confidence.
