> ## Documentation Index
> Fetch the complete documentation index at: https://docs.anyformat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Datasets & evaluations

> The vocabulary behind measuring a workflow's quality: datasets, ground truth, sub-datasets, and evaluations

Once a workflow is running, the next question is **is it getting better or worse as I change it?** Answering that reliably needs a fixed set of documents you know the correct answers for, and a repeatable way to score the workflow against them.

anyformat gives you both, in the **Health** area of each workflow. This page defines the four terms you'll see there. For how to use them, see the [Datasets guide](/guides/health/datasets) and the [Evaluations guide](/guides/health/evaluations).

<Tip>
  **The short version:** a **dataset** is a pool of documents with known-correct answers (**ground truth**). An **evaluation** runs one workflow version over that dataset and scores it. Tag files into **sub-datasets** to see accuracy on slices like "hard cases" separately from the overall number.
</Tip>

***

## Dataset

A **dataset** is a fixed collection of documents you use to measure a workflow's quality. Each workflow has **one dataset**.

The key property is that a dataset is **independent from production**:

* Files enter a dataset two ways: promote a **processed** document with **Add to dataset** (which **duplicates** it into the dataset), or **upload** documents and their ground truth directly. Either way the dataset entry is its own independent entity.
* Editing ground truth, removing a file from the dataset, or running an evaluation **never affects your production data**, and production runs never change the dataset.

This separation is what makes an evaluation reproducible: the documents being scored don't move underneath you.

<Info>
  Datasets cover **Extract** and **Classify** workflows. Parse-only workflows aren't evaluated.
</Info>

***

## Ground truth

**Ground truth** is the set of **expected, correct values** for a dataset file — the reference an evaluation scores the workflow's output against.

* Ground truth comes from one of two places: **validating** a processed file's datapoints (the *Add to dataset* path), or **supplying it directly** as a `.json` file when you upload. Either way it's editable in **Health → Datasets**.
* On the *Add to dataset* path, a file must have **every datapoint validated** first — you're establishing the correct answer, so it has to be complete. On the *upload* path ground truth is optional: supply it in the `.json` file, or leave it blank and fill it in later by editing the dataset table.

> Accuracy is only ever measured against ground truth. A field the dataset has no ground truth for is **not scored**.

***

## Sub-dataset (slice)

A **sub-dataset** is a slice of the dataset defined by **tags**. Tag files by anything meaningful — provider, document type, "hard cases" — and you can read results **overall or per sub-dataset**.

This lets one dataset answer questions like *"95% overall, but only 60% on hard cases"* without splitting into separate datasets. Files with no tag form the **Untagged** slice.

***

## Evaluation

An **evaluation** (or **eval**) is **one scored run of one workflow version over a dataset scope** — the full dataset, or a single sub-dataset.

Running an evaluation:

1. Runs an extraction on every in-scope dataset file.
2. Scores each result against that file's ground truth.
3. Records the run with a status and an **accuracy**.

Evaluations are presented as a numbered **run list** — `#1`, `#2`, … — each with its scope, workflow version, date, status, accuracy, and file count.

<Info>
  An evaluation is an **immutable historical record**. Editing ground truth after **Eval #1** does not change **Eval #1** — you run **Eval #2** to measure the effect. This is what makes version-to-version comparison trustworthy.
</Info>

***

## Accuracy

For an evaluation, **accuracy** is the **share of graded fields that matched ground truth across the in-scope documents**.

Accuracy and confidence work the same way here as everywhere else in anyformat — confidence guides *where to look*, accuracy tells you *how often the workflow is actually right*. See [Confidence & accuracy](/guides/workflows/analytics-quality) for the full explanation of both numbers.

***

## How these fit together

<CardGroup cols={2}>
  <Card title="Build a dataset" icon="database" href="/guides/health/datasets">
    Add documents, edit ground truth, and tag files into sub-datasets.
  </Card>

  <Card title="Run an evaluation" icon="flask" href="/guides/health/evaluations">
    Score a workflow version and compare accuracy across versions.
  </Card>
</CardGroup>
