Optimizing a workflow

The Optimizer rewrites a workflow’s field descriptions, the instructions the extraction model reads for every field. It keeps a change only when that change measurably improves accuracy against your dataset. It automates the refine step of the quality loop: propose a change, evaluate it, keep it only if it helps. The Optimizer lives under Health → Optimizer.

What a run does

Measure the baseline

The Optimizer evaluates the current version against the dataset. This is iteration 0. Everything after is judged against it.

Propose

The Optimizer studies where the previous results went wrong and writes improved field descriptions.

Evaluate

The Optimizer runs the proposed descriptions over the dataset, like a regular evaluation.

Keep only improvements

An iteration is accepted only if its accuracy beats the best result so far. Propose → evaluate repeats up to the number of iterations you chose.

When the run finishes, the best iteration is promoted automatically as a new workflow version. Each iteration is also recorded as a patch version, so the full trail stays visible in the workflow’s version history.

“No improvement found” is a valid outcome. Your current descriptions already perform as well as the proposals. The promoted version matches your baseline, and nothing changes.

What it needs

An Extract node. The Optimizer improves extraction field descriptions, so parse-only and split workflows are excluded.
A dataset with ground truth. The run scores every proposal against your documents’ known-correct values, so a dataset without ground truth cannot start a measured run. See Building a dataset. A one-shot run needs no ground truth.

Every iteration runs extractions over the whole dataset, which consumes credits. A run costs roughly dataset size × (iterations + 1) extractions, baseline included. Start with a small dataset and few iterations. The credits field of the optimization says what a run has cost so far. It rises while the run works, and is null when the cost was not recorded.

Starting a run

Choose Run optimization and set:

Scope: the full dataset.
Metric: overall accuracy.
Iterations: how many propose-and-evaluate rounds to attempt, from 1 to 10. The default is 3.

A workflow allows one active optimization at a time. A second start is rejected until the current run finishes.

One-shot mode

A one-shot run makes a single suggestion and applies it without measuring it. It reflects once on the extractions already in your dataset and on an optional hint you write, for example “the total includes VAT”. It runs no baseline and no evaluation, so it needs no ground truth, and it reports no accuracy. Use it when the dataset has no ground truth yet, or when you want a fast suggestion to review. It needs at least one extraction in the dataset or a hint, and the hint can hold up to 4000 characters. The suggestion goes live as a new workflow version like any other result, so check it with an evaluation before you trust it. The one-active-run rule covers both modes: a measured run and a one-shot run block each other.

Using the API

Start, list and read optimizations through the REST API, the SDKs and the MCP server. The same calls cover both modes.

Call	What it does
Start optimization	`kind: "extraction"` starts a measured run with `max_rounds`; `kind: "extraction_oneshot"` starts a one-shot run with `user_context`
List optimizations	A workflow’s runs, newest first, filtered with `kind`
Get optimization	One run with its rounds and per-field scores

from anyformat.sdk import Client

client = Client(api_key="YOUR_API_KEY")
workflow_id = "0686bb97-8c30-70f0-8000-97669e000eb8"

run = client.start_optimization(workflow_id, "extraction", max_rounds=3)
# client.start_optimization(workflow_id, "extraction_oneshot", user_context="The total includes VAT.")
run = client.get_optimization(workflow_id, run.id)  # poll until run.status is "done" or "failed"

import { Anyformat } from '@anyformat/sdk';

const af = new Anyformat({ apiKey: 'YOUR_API_KEY' });
const workflowId = '0686bb97-8c30-70f0-8000-97669e000eb8';

let run = await af.startOptimization(workflowId, { kind: 'extraction', maxRounds: 3 });
// await af.startOptimization(workflowId, { kind: 'extraction_oneshot', userContext: 'The total includes VAT.' });
run = await af.getOptimization(workflowId, run.id); // poll until run.status is 'done' or 'failed'

A start while another run is active returns 409 OPTIMIZATION_ALREADY_RUNNING. A one-shot start with neither a dataset extraction nor a hint returns 422 ONESHOT_SIGNALS_MISSING.

Following a run

Runs appear in the Optimizer table. Status, accuracy, and rounds update live as the run progresses:

Column	What it shows
Run	The run’s start time
Status	In progress, done, or failed
Accuracy	Baseline → best accuracy so far
Rounds	Iterations completed vs planned
Date	When the run started

Open a run for the full picture: an accuracy-per-iteration chart, the timeline of iterations, and each iteration’s proposed description changes. Open a single iteration to see its per-field accuracy, which fields improved, which still mismatch, and the exact description edits it tried. The per-field view doubles as a work list. A field that stays inaccurate after an optimization needs more dataset examples, or more varied ones.

Versions and rolling back

The promoted result goes live automatically. It becomes the workflow’s latest version, with no separate review step. Every change is a normal version-history entry. If you disagree with a rewrite, restore an earlier version.

While a run is active, each iteration is appended as the workflow’s latest version. Documents processed during the run may use in-progress descriptions. Optimize at a quiet moment for the workflow.

Asking annie

annie drives the Optimizer conversationally, which helps when you are already in a chat about the workflow’s results. “improve this workflow” starts a run. “how did the optimization go?” reports accuracy movement and the changes she kept. “list the optimizations” shows the history.

Current limits

A running optimization cannot be cancelled. It runs to completion. Size your first runs small.
The Optimizer changes extraction field descriptions only. It never touches Classify, Split, or Validate nodes.
Scope and metric are fixed for now: full dataset, overall accuracy.

Get started

Nodes

Workflows

Schema & fields

Documents

Outputs & integrations

Health

Billing

Optimizing a workflow

What a run does

What it needs

Starting a run

One-shot mode

Using the API

Following a run

Versions and rolling back

Asking annie

Current limits

What’s next?

Building a dataset

Running evaluations