Skip to main content
An evaluation scores one workflow version against your dataset. It runs an extraction on every in-scope document, compares each result to that document’s ground truth, and records the run with an accuracy you compare against other runs. Evaluations live under Health → Evaluations.

​
Running an evaluation

Choose Run eval and set two things:
  • Scope: the full dataset, or a single sub-dataset, including Untagged. anyformat scores this set of documents.
  • Workflow version: the version you measure.
The evaluation runs an extraction for each in-scope file and scores it against ground truth. If the version has fields the dataset has no ground truth for, the run still proceeds. Those fields are not scored, and a warning names them. Missing ground truth never blocks a run.
Running an evaluation runs extractions, which consume credits. Narrow the scope to a sub-dataset to measure one slice.

​
Reading the run list

Each evaluation appears as a numbered run, #1, #2, and so on, in a table with: Each run is pinned to a version and a scope, so the list doubles as an accuracy-over-time history. It tells you whether the workflow is improving or regressing.

​
Inside a single evaluation

Open a run to debug where accuracy comes from:
  • Accuracy, Documents, and Fields as headline numbers. Documents counts how many passed. A vs previous run delta shows the change.
  • By file: per-document accuracy, to find the documents dragging the number down.
  • By field: per-field accuracy, to find the fields that fail consistently.
From there, inspect a document’s expected vs predicted values side by side to see what the workflow got wrong.

​
Comparing versions

The evaluation workflow is built for iteration:
  1. Run an evaluation on the current version. That is your baseline.
  2. Refine the workflow’s fields, instructions, and nodes, then save a new version.
  3. Run another evaluation on the same scope.
  4. Compare the two runs’ accuracy.
Editing ground truth or refining the workflow never changes a past evaluation. Each run is immutable. That is what makes the difference between two runs real signal rather than noise. Always run a fresh evaluation to measure a change.

​
What’s next?

Building a dataset

Add documents, edit ground truth, and tag sub-datasets.

Analytics & Quality

How accuracy and confidence work across anyformat.