← Vetro Attest · Free Art. 12/14 checker · Free EU AI Act tools

Audit log for smolagents

Hugging Face's smolagents gives an agent a code interpreter as its main tool: instead of calling fixed tools with fixed arguments, the agent writes and runs small snippets of code to make progress. That is powerful and it is also the part of the system a reviewer will worry about most — code execution is where an agent can do the most damage, accidentally or otherwise.

What to record

• Every model call: the prompt built from the agent's running transcript, the response, and the code it produced.
• Every code execution: the code that ran, its output, and any error — ideally with the interpreter's sandboxing settings alongside it.
• Tool calls that are not code execution, if your agent also has fixed tools.
• Step boundaries, since smolagents runs an explicit think-act-observe loop and each step is a natural unit of evidence.

Capturing it: wrap the model and the interpreter

smolagents has no Vetro integration. The two boundaries worth wrapping are the model client the agent calls and the code interpreter it executes against — both are called from your own process, so both can be recorded at the point they run, with policy checked before the code interpreter is invoked:

import { Vetro } from '@vetro/core';

const vetro = new Vetro({ policy: myPolicy });
const run = vetro.startRun({ name: 'code-agent' });

// before running the interpreter:
run.toolCall('python_interpreter', { code });
// after it returns:
run.toolResult('python_interpreter', { stdout, error });

This is shown in TypeScript for the Vetro core API; smolagents is a Python framework, so the equivalent is calling the matching Run methods around your interpreter's execute call in Python, at the same point in the loop. An inline policy that blocks specific commands or filesystem paths before the code runs is the single highest-value control here (see tamper-evident logs for why recording alone is not enough).

If you already use tracing

smolagents supports OpenTelemetry-based tracing for inspecting agent runs. If that is already wired up, export it as OTLP JSON and treat it as a first reading rather than re-instrumenting — check whether code and outputs are captured by default, since many tracing setups only record that a step happened, not its content (see OpenTelemetry GenAI traces as evidence).

Why this matters for oversight

A code-executing agent is exactly the case Article 14 human oversight has in mind for approval gates. Recording both the policy decision and the code it applied to gives a reviewer something to check the gate against, rather than the vendor's word for it.

Check your own logs

Paste a Vetro ledger or an OpenTelemetry GenAI trace export into the free checker: it verifies integrity and shows which record-keeping obligations your evidence covers, in your browser. Unsure if a code-executing agent counts as high-risk? Use the high-risk classifier. When you need something to hand to an auditor or a customer, Vetro Attest issues a countersigned EU AI Act report for 49 USD, one-off.

See a sample ledger verified in one click, check your own logs free or get the countersigned EU AI Act report.