Audit Agent Run Evidence

Overview

Turn an end-to-end success statement into independently decidable claims. Reconstruct what happened from available records, grade each claim against the strongest witness, and keep missing evidence distinct from failure.

This is a read-only audit. Do not rerun tools, approve actions, resume workers, deploy artifacts, or modify evidence unless the user separately authorizes those actions.

When to Use

• Auditing a completed or interrupted agent run from traces and artifacts.
• Checking whether an agent's end-to-end success claim is actually supported.
• Reviewing MCP, gateway, sandbox, checkpoint, retry, memory, approval, or deployment evidence.
• Separating autonomous success from human-assisted or merely requested outcomes.

Do not use this skill to design instrumentation for a future run or to perform the missing actions. It evaluates evidence that already exists.

Establish the Contract

Record these inputs before judging the run:

• declared goal and terminal success criteria;
• run, workflow, task, and parent identifiers;
• immutable code, configuration, model, prompt, tool-schema, and artifact revisions when available;
• actors and trust boundaries: orchestrator, worker, sandbox, MCP server, gateway, human approver, CI, and deployment platform;
• retry, deadline, token, cost, concurrency, and human-escalation budgets;
• supplied evidence inventory and known collection gaps.

Do not silently strengthen the original success criteria. Do not weaken them to match the evidence that happens to exist.

Build a Claim Ledger

Split the overall claim into atomic predicates. Give every row a stable claim ID.

Typical predicates include:

• every required step reached its terminal postcondition;
• sandbox isolation held for every executing worker;
• each required MCP/tool call has a correlated response;
• retries respected idempotency and did not duplicate committed effects;
• a checkpoint was durably written, verified, and actually used for resume;
• parallel branches satisfied the declared join policy;
• memory reads cite a versioned source rather than an untracked summary;
• retry, deadline, token, cost, and escalation budgets were respected;
• approval was granted by an authorized human for the exact artifact and target;
• the platform deployed that same artifact and passed the declared health checks.

Normalize Evidence

Preserve original records and create a normalized event view with:

{
  "run_id": "run-123",
  "event_id": "evt-42",
  "sequence": 42,
  "observed_at": "RFC3339 timestamp",
  "actor": {"type": "worker", "id": "worker-2"},
  "operation": "mcp.search",
  "state_before": "researching",
  "state_after": "researching",
  "attempt": 2,
  "request_id": "req-9",
  "idempotency_key": "task-7:search:2",
  "input_digest": "sha256:...",
  "output_digest": "sha256:...",
  "checkpoint_seq": 3,
  "parent_event_id": "evt-41",
  "status": "succeeded",
  "evidence_ref": "tool-log:991"
}

Use null or unknown for absent values. Never synthesize IDs, timestamps, digests, costs, approvals, or outcomes.

Verify bundle hashes or signatures when supplied. Check duplicate IDs, broken parent links, non-monotonic per-source sequences, impossible state transitions, unaccounted clock skew, and unexplained trace gaps. Treat an integrity failure as counterevidence for claims that depend on the affected records.

Rank Witnesses

Prefer the witness closest to the effect:

…