Skip to main content
An evaluation scores one workflow version against your dataset. It runs an extraction on every in-scope document, compares each result to that document’s ground truth, and records the run with an accuracy you can trust to compare against other runs. Evaluations live under Health → Evaluations.

​
Running an evaluation

Choose Run eval and set two things:
  • Scope — the full dataset, or a single sub-dataset (including Untagged). This is the set of documents that will be scored.
  • Workflow version — the version being measured.
The evaluation runs an extraction for each in-scope file and scores it against ground truth.
Running an evaluation runs extractions, which consume credits — one per in-scope document. Narrow the scope to a sub-dataset when you only need to measure a slice.
If the version has fields the dataset has no ground truth for, the run still proceeds — those fields simply aren’t scored, and you’ll see a warning naming them. Missing ground truth never blocks a run.

​
Reading the run list

Each evaluation appears as a numbered run — #1, #2, … — in a table with: Because each run is pinned to a version and a scope, the list doubles as an accuracy-over-time history — the signal for whether a workflow is improving or regressing.

​
Inside a single evaluation

Open a run to debug where accuracy comes from:
  • Accuracy, Documents (how many passed), and Fields as headline numbers, plus a vs previous run delta so you can see the change at a glance.
  • By file — per-document accuracy, to find the documents dragging the number down.
  • By field — per-field accuracy, to find the fields that consistently fail.
From there you can inspect a document’s expected vs predicted values side by side to see exactly what the workflow got wrong.

​
Comparing versions

The evaluation workflow is built for iteration:
  1. Run an evaluation on the current version — your baseline.
  2. Refine the workflow (fields, instructions, the pipeline) and save a new version.
  3. Run another evaluation on the same scope.
  4. Compare the two runs’ accuracy.
Editing ground truth or refining the workflow never changes a past evaluation — each run is immutable. That’s what makes the difference between two runs real signal rather than noise. Always run a fresh evaluation to measure a change.

​
What’s next?

Building a dataset

Add documents, edit ground truth, and tag sub-datasets.

Confidence & Accuracy

How accuracy and confidence work across anyformat.