Skip to main content
The Optimizer rewrites a workflow’s field descriptions — the instructions the extraction model reads for every field — and keeps a change only when it measurably improves accuracy against your dataset. It automates the refine step of the quality loop: propose a change, evaluate it, keep it only if it helps. The Optimizer lives under Health → Optimizer.

​
What a run does

1

Measure the baseline

The current version is evaluated against the dataset. This is iteration 0 — everything after is judged against it.
2

Propose

The Optimizer studies where the previous results went wrong and writes improved field descriptions.
3

Evaluate

The proposed descriptions are run over the dataset, like a regular evaluation.
4

Keep only improvements

An iteration is accepted only if its accuracy beats the best result so far. Propose → evaluate repeats up to the number of iterations you chose.
When the run finishes, the best iteration is promoted automatically as a new workflow version. Each iteration is also recorded as a patch version, so the full trail stays visible in the workflow’s version history.
“No improvement found” is a valid outcome. It means your current descriptions already perform as well as the proposals — the promoted version then simply matches your baseline, and nothing effectively changes.

​
What it needs

  • An Extract step — the Optimizer improves extraction field descriptions, so parse-only and split workflows are excluded.
  • A dataset with ground truth — the run scores every proposal against your documents’ known-correct values, so a dataset without ground truth can’t start one. See Building a dataset.
Every iteration runs extractions over the whole dataset, which consumes credits — a run costs roughly dataset size × (iterations + 1) extractions, baseline included. Start with a small dataset and few iterations to get a feel for it.

​
Starting a run

Choose Run optimization and set:
  • Scope — the full dataset (sub-dataset scoping is coming).
  • Metric — overall accuracy.
  • Iterations — how many propose-and-evaluate rounds to attempt, 1–10 (default 3).
A workflow allows one active optimization at a time — a second start is rejected until the current run finishes.

​
Following a run

Runs appear in the Optimizer table — status, accuracy, and rounds update live as the run progresses: Open a run for the full picture: an accuracy-per-iteration chart, the timeline of iterations, and each iteration’s proposed description changes. Open a single iteration to see its per-field accuracy — which fields improved, which still mismatch — and the exact description edits it tried.
The per-field view doubles as a work list: fields that stay inaccurate after an optimization are the ones that need more (or more varied) dataset examples.

​
Versions and rolling back

The promoted result goes live automatically — it becomes the workflow’s latest version, with no separate review step. Every change is a normal version-history entry, so if you disagree with a rewrite, restore an earlier version and you’re back where you started.
While a run is active, each iteration is appended as the workflow’s latest version — so documents processed during the run may use in-progress descriptions. Prefer optimizing at a quiet moment for the workflow.

​
Asking Annie

Annie can drive the Optimizer conversationally: “improve this workflow” starts a run, “how did the optimization go?” reports accuracy movement and the changes that were kept, and “list the optimizations” shows the history — useful when you’re already in a chat about the workflow’s results.

​
Current limits

  • A running optimization can’t be cancelled — it runs to completion. Size your first runs small.
  • Only extraction field descriptions are optimized — Classify, Split, and Validate steps aren’t touched.
  • Scope and metric are fixed for now: full dataset, overall accuracy.

​
What’s next?

Building a dataset

Add documents and author the ground truth an optimization measures against.

Running evaluations

Score a version by hand and compare accuracy across versions.