← Vetro Attest · Free Art. 12/14 checker · Free EU AI Act tools

EU AI Act Article 15: accuracy, robustness and cybersecurity for agents

Article 15 requires high-risk AI systems to achieve an appropriate level of accuracy, robustness and cybersecurity, and to perform consistently in those respects throughout their lifecycle. Accuracy levels and the relevant metrics must be declared in the instructions for use (see Article 13).

Accuracy

For an agent, accuracy is rarely one number. Declare the metrics that match the intended purpose (task success rate on a fixed evaluation set, rate of wrong tool calls, rate of escalations) and measure them the same way at every release. Recorded evaluation runs make the declared number reproducible.

Robustness

The system must be as resilient as possible to errors, faults and inconsistencies, including those from the environment it interacts with. For agents that means tool failures, timeouts, malformed tool output and unexpected user input. Technical redundancy and fail-safe plans are named explicitly in the text.

Useful fail-safes for agents are inline policies: a cost ceiling per run, a maximum number of steps, an allow-list of tools, a block on outbound actions without approval. When one fires, the event should be in the log.

Systems that keep learning after deployment must reduce the risk of biased outputs feeding back into future operation.

Cybersecurity

Article 15 names the attacks: data poisoning, model poisoning, adversarial examples or model evasion, confidentiality attacks and model flaws. For LLM agents the most common in practice is prompt injection through retrieved content or tool output, which is a form of adversarial input.

Two controls work together:

Prevention at the call site: policies that inspect a tool call before it runs and block what the agent was never supposed to do, whatever the prompt said.
Evidence after the fact: a tamper-evident record of what the agent saw and did, so an attack can be reconstructed and an attacker cannot erase their trail (see tamper-evident logs).

"Consistently throughout the lifecycle"

Model updates change behaviour. Replaying recorded runs against a new model version and diffing the actions is the cheapest way to show that accuracy and robustness did not silently degrade.

Check your own logs

Paste a Vetro ledger or an OpenTelemetry GenAI trace export into the free checker: it verifies integrity and shows which record-keeping obligations your evidence covers, in your browser. When you need something to hand to an auditor or a customer, the countersigned EU AI Act report costs 49 USD, one-off: an ed25519 counter-signature over your evidence plus a mapping to the articles, including an explicit list of what the log cannot show.

See a sample ledger verified in one click, check your own logs free or get the countersigned EU AI Act report.