← Vetro Attest · Free Art. 12/14 checker · Free EU AI Act tools
Audit trail for AI agents: what to record and how to make it hold up
An agent that calls models and tools on its own is making decisions nobody watched in real time. When something goes wrong (a refund issued twice, a customer email that should not have gone out, a bill that tripled) the question is always the same: what exactly did it do, and can we prove it?
An audit trail answers that. Application logs usually do not.
Logs vs. audit trail
| Application logs | Audit trail | |
|---|---|---|
| Written for | Debugging, by the team that wrote the code | Someone else, later, who does not trust the team by default |
| Coverage | Whatever a developer chose to log | Every action, recorded mechanically |
| Integrity | Editable, rotatable, deletable | Edits and deletions are detectable |
| Order | Best-effort timestamps | Strict sequence per run |
| Identity | Rarely says which system wrote it | Signed by the system that produced it |
What to record for an agent
- Every model call: model name and version, parameters, prompt, response, token counts, cost.
- Every tool call: tool name, arguments, result or error. This is where money moves and data changes.
- Every policy decision: budget checks, forbidden actions, content rules, with the verdict and reason. Refusals are evidence too.
- Run boundaries: start, end, outcome, errors, so each decision has a clear scope.
Four properties an auditor checks
- Completeness: are events captured at the call site, or only where someone added a log line?
- Integrity: if one byte of a stored event changes, does anything notice? (See tamper-evident logs.)
- Attribution: is the log signed, so you can tell which system produced it?
- Retention: is it kept long enough, in a form someone can still verify later? (See Article 26 retention.)
Checklist
- [ ] Model client and tool dispatcher are wrapped once; no per-call logging code
- [ ] Each run has an id and a strict sequence number per event
- [ ] Each event includes the hash of the previous one
- [ ] The run is signed with a key the agent controls, public key published
- [ ] Policy decisions are logged as events, not only as exceptions
- [ ] Verification can be run by someone outside the team, without your infrastructure
How Vetro does it
Vetro wraps the model and tool clients and writes every action to a hash-chained, ed25519-signed JSONL ledger, with policies checked inline before the action runs. If you already emit OpenTelemetry GenAI traces, you do not need to change anything to get a first reading.
Vetro Attest verifies a ledger or OTLP trace for free and, for 49 USD, issues a counter-signed report mapping the evidence to the EU AI Act record-keeping articles, including an explicit list of what the log cannot show.
See a sample ledger verified in one click, check your own logs free or get the countersigned EU AI Act report.