Building a dataset

A dataset is the fixed set of documents an evaluation scores against. Each workflow has one, managed under Health → Datasets. This guide covers filling it, keeping its ground truth correct, and slicing it with tags. New to the terms? See Datasets & evaluations first.

---

​

Adding documents

There are two ways to put documents in a dataset — use whichever matches where your labelled data already lives.

​

From your processed results

Promote a document you’ve already run through the workflow:

1. Open its results and confirm every value is correct — validate all of its datapoints. The correct values become the file’s ground truth.
2. Choose Add to dataset.

This duplicates the file into the dataset as an independent entity. From that point on, the dataset copy and the production file are separate — editing one never changes the other.

​

By direct upload

Already have labelled data? Upload the documents and their expected answers straight into the dataset — no need to run them through the workflow first.

1. On the Datasets subtab, choose Upload files.
2. Drop your documents. To import expected values, include a .json ground-truth file named after each document — invoice.pdf pairs with invoice.json.
3. Optionally tag the uploaded files to slot them straight into a sub-dataset.
4. Review the staged list, then choose Upload.

Ground truth is optional on upload — add documents alone and fill their values in later by editing the dataset table. Each .json is paired to its document by filename, and its values are checked against the workflow’s current fields. The review step catches problems before anything is uploaded, flagging any file that is a duplicate name (dataset names must be unique — nothing is silently renamed), has invalid JSON, or is an orphan ground-truth file matching no document.

---

​

Editing ground truth

The dataset’s ground truth is the reference every evaluation is measured against, so keeping it correct matters more than anywhere else. Edit expected values directly in Health → Datasets — no need to go back to the original run. The Ground truth grid shows the expected values for each file, which are what an evaluation compares the workflow’s output to.

---

​

Sub-datasets: slicing with tags

One overall accuracy number hides where a workflow struggles. Tag files to create sub-datasets (slices) and read accuracy for each one separately. Tag by whatever distinction matters for your documents:

• Provider — bank A vs bank B
• Document type — invoices vs receipts
• Difficulty — a “hard cases” tag for the documents that trip the workflow up

You can then read results overall or per sub-dataset — for example 95% overall but 60% on hard cases — and target the slice that needs work. Files with no tag make up the Untagged slice. When you run an evaluation, the scope you pick is either the full dataset or a single sub-dataset.

---

​

Removing documents

Select files in the Datasets grid and choose Remove to take them out of the dataset. Removal only affects the dataset — the production file is untouched. Removal is final: you can add the file to the dataset again, but that creates a brand-new dataset entry with a clean slate — it isn’t linked to the removed one, and carries none of its earlier evaluation history or ground-truth edits. To measure the new entry, you’ll need to run an evaluation over it.

---

​

What’s next?