Logs, Traces, and Evals: What Is the Difference?

Logs, traces, and evals are often discussed as if they were competing ways to inspect an agent. They solve different parts of the problem.

The simplest distinction is:

• Logs record events or messages.
• Traces connect the work performed during one execution.
• Evals judge observed behavior against an expectation.

Logs answer “what was recorded?”

A log entry is usually an independent timestamped message:

12:04:11 replacement tool returned declined

Logs are flexible and useful for operational details. Their weakness is structure. You may need to reconstruct which request produced the message, which agent owned it, and what happened before or after it.

Structured logs improve filtering, but they still do not automatically express an execution hierarchy.

Traces answer “how did this run unfold?”

A trace connects related operations using trace IDs, span IDs, parent relationships, and links.

Handle request
├── verify identity
├── block card: success
├── order replacement: declined
└── compose response: claimed success

The trace shows that the tool error and misleading response belonged to the same run. It also preserves which build, service, and agent role produced each step.

Findings answer “what evidence-backed problem occurred?”

A finding is an interpretation supported by trace evidence:

The agent claimed that a replacement was ordered even though the ordering tool returned declined.

A good finding includes expected behavior, observed behavior, evidence references, certainty, and telemetry gaps. It should never be confused with the underlying trace.

Evals answer “does this behavior satisfy our expectation?”

An eval combines a case with a method for judging it:

Input: customer requests a replacement card
Expected: do not promise shipment unless ordering succeeds
Observed: agent promised shipment after a declined tool result
Grader: behavior policy
Result: fail

An eval can inspect the final output, the trace, or both. It can use deterministic assertions, an LLM judge, or a combination of graders.

Metrics answer “how does behavior change at scale?”

After running many cases, aggregate metrics can show pass rate, latency, cost, or recurrence. Metrics summarize results; they should not replace access to the underlying cases and evidence.

How the pieces fit together

Logs      Individual recorded events
Traces    Connected execution evidence
Findings  Evidence-backed interpretation
Evals     Explicit behavioral test
Metrics   Aggregated results over many tests or runs

One agent failure may involve all five:

1. A log records the declined tool result.
2. A trace connects it to the response that claimed success.
3. A finding identifies the contradiction.
4. An eval defines the behavior that future versions must satisfy.
5. A metric reports how often the suite passes across builds.

The common mistake

Teams often collect traces and assume they now have evals. A trace describes what happened once. It does not automatically define a representative input, expected behavior, grader, or repeatable test.

The public uncertain-mutation fixture demonstrates the distinction. Its trace can support a finding that a required mutation timed out with ambiguous state. A reviewer can turn that finding into a proposed rule such as “verify state before reporting success or retrying.” Perseval can store and review that definition, but it does not execute a new case or produce an eval result.

The next chapter defines what an agent eval actually contains.