curl -X GET 'https://api.anyformat.ai/v3/workflows/0686bb97-8c30-70f0-8000-97669e000eb8/evals/069dcc2c-e14c-7606-8000-2ee4fb17b4f9/' \
-H 'Authorization: Bearer YOUR_API_KEY'
import time
from anyformat.sdk import Client
client = Client(api_key="YOUR_API_KEY")
workflow_id = "0686bb97-8c30-70f0-8000-97669e000eb8"
eval_id = "069dcc2c-e14c-7606-8000-2ee4fb17b4f9"
for _ in range(100):
an_eval = client.get_eval(workflow_id, eval_id)
if an_eval.status == "processed":
print(f"accuracy={an_eval.accuracy} "
f"({an_eval.matched}/{an_eval.mismatched}/{an_eval.ungraded})")
break
if an_eval.status == "error":
print("Eval failed")
break
time.sleep(5)
interface EvalDetail {
id: string;
run_number: number;
version_id: string;
status: 'in_progress' | 'processed' | 'error';
accuracy: number | null;
matched: number | null;
mismatched: number | null;
ungraded: number | null;
file_count: number;
failed_count: number;
created_at: string | null;
}
const workflowId = '0686bb97-8c30-70f0-8000-97669e000eb8';
const evalId = '069dcc2c-e14c-7606-8000-2ee4fb17b4f9';
const response = await fetch(
`https://api.anyformat.ai/v3/workflows/${workflowId}/evals/${evalId}/`,
{ headers: { 'Authorization': 'Bearer YOUR_API_KEY' } }
);
if (!response.ok) {
throw new Error(`API error: ${response.status}`);
}
const an_eval: EvalDetail = await response.json();
if (an_eval.status === 'processed') {
console.log(an_eval.accuracy);
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "processed",
"accuracy": 0.94,
"matched": 47,
"mismatched": 3,
"ungraded": 1,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "in_progress",
"accuracy": null,
"matched": null,
"mismatched": null,
"ungraded": null,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "error",
"accuracy": null,
"matched": null,
"mismatched": null,
"ungraded": null,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
Get Eval
Fetch one eval’s frozen headline: grading status plus accuracy
curl -X GET 'https://api.anyformat.ai/v3/workflows/0686bb97-8c30-70f0-8000-97669e000eb8/evals/069dcc2c-e14c-7606-8000-2ee4fb17b4f9/' \
-H 'Authorization: Bearer YOUR_API_KEY'
import time
from anyformat.sdk import Client
client = Client(api_key="YOUR_API_KEY")
workflow_id = "0686bb97-8c30-70f0-8000-97669e000eb8"
eval_id = "069dcc2c-e14c-7606-8000-2ee4fb17b4f9"
for _ in range(100):
an_eval = client.get_eval(workflow_id, eval_id)
if an_eval.status == "processed":
print(f"accuracy={an_eval.accuracy} "
f"({an_eval.matched}/{an_eval.mismatched}/{an_eval.ungraded})")
break
if an_eval.status == "error":
print("Eval failed")
break
time.sleep(5)
interface EvalDetail {
id: string;
run_number: number;
version_id: string;
status: 'in_progress' | 'processed' | 'error';
accuracy: number | null;
matched: number | null;
mismatched: number | null;
ungraded: number | null;
file_count: number;
failed_count: number;
created_at: string | null;
}
const workflowId = '0686bb97-8c30-70f0-8000-97669e000eb8';
const evalId = '069dcc2c-e14c-7606-8000-2ee4fb17b4f9';
const response = await fetch(
`https://api.anyformat.ai/v3/workflows/${workflowId}/evals/${evalId}/`,
{ headers: { 'Authorization': 'Bearer YOUR_API_KEY' } }
);
if (!response.ok) {
throw new Error(`API error: ${response.status}`);
}
const an_eval: EvalDetail = await response.json();
if (an_eval.status === 'processed') {
console.log(an_eval.accuracy);
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "processed",
"accuracy": 0.94,
"matched": 47,
"mismatched": 3,
"ungraded": 1,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "in_progress",
"accuracy": null,
"matched": null,
"mismatched": null,
"ungraded": null,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "error",
"accuracy": null,
"matched": null,
"mismatched": null,
"ungraded": null,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
eval_id. It is the poll target for a run launched with POST /v3/workflows/{workflow_id}/evals/. The id in the response is the identifier that launch returned as eval_id.
The response is the eval’s frozen headline: grading status, the accuracy fraction, the matched, mismatched and ungraded field tallies, the cohort’s file_count, and failed_count. failed_count counts the documents that failed to enqueue. The headline carries counts, never internal ids.
Status model
The endpoint always returns200 while the eval exists. Grading in flight raises no precondition error.
status | Meaning | Headline (accuracy, matched, mismatched, ungraded) |
|---|---|---|
in_progress | Extractions are running, or grading is not final | null |
processed | Terminal. The eval is graded | populated |
error | Terminal. The run failed | null |
status is terminal. On a processed eval, accuracy is the matched / (matched + mismatched) fraction. It stays null when the graded denominator is 0, which happens when nothing is gradable. Every field being ungraded for lack of ground truth produces that case.
An unknown id returns 404, and so does an eval on another workflow or organization: existence never leaks. A caller outside the workflow’s organization gets 403.
curl -X GET 'https://api.anyformat.ai/v3/workflows/0686bb97-8c30-70f0-8000-97669e000eb8/evals/069dcc2c-e14c-7606-8000-2ee4fb17b4f9/' \
-H 'Authorization: Bearer YOUR_API_KEY'
import time
from anyformat.sdk import Client
client = Client(api_key="YOUR_API_KEY")
workflow_id = "0686bb97-8c30-70f0-8000-97669e000eb8"
eval_id = "069dcc2c-e14c-7606-8000-2ee4fb17b4f9"
for _ in range(100):
an_eval = client.get_eval(workflow_id, eval_id)
if an_eval.status == "processed":
print(f"accuracy={an_eval.accuracy} "
f"({an_eval.matched}/{an_eval.mismatched}/{an_eval.ungraded})")
break
if an_eval.status == "error":
print("Eval failed")
break
time.sleep(5)
interface EvalDetail {
id: string;
run_number: number;
version_id: string;
status: 'in_progress' | 'processed' | 'error';
accuracy: number | null;
matched: number | null;
mismatched: number | null;
ungraded: number | null;
file_count: number;
failed_count: number;
created_at: string | null;
}
const workflowId = '0686bb97-8c30-70f0-8000-97669e000eb8';
const evalId = '069dcc2c-e14c-7606-8000-2ee4fb17b4f9';
const response = await fetch(
`https://api.anyformat.ai/v3/workflows/${workflowId}/evals/${evalId}/`,
{ headers: { 'Authorization': 'Bearer YOUR_API_KEY' } }
);
if (!response.ok) {
throw new Error(`API error: ${response.status}`);
}
const an_eval: EvalDetail = await response.json();
if (an_eval.status === 'processed') {
console.log(an_eval.accuracy);
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "processed",
"accuracy": 0.94,
"matched": 47,
"mismatched": 3,
"ungraded": 1,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "in_progress",
"accuracy": null,
"matched": null,
"mismatched": null,
"ungraded": null,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
{
"id": "069dcc2c-e14c-7606-8000-2ee4fb17b4f9",
"run_number": 3,
"version_id": "abcdef1234",
"status": "error",
"accuracy": null,
"matched": null,
"mismatched": null,
"ungraded": null,
"file_count": 42,
"failed_count": 0,
"created_at": "2026-07-05T10:00:00.000Z"
}
Authorizations
API key issued from app.anyformat.ai/api-key. Send as Authorization: Bearer <key>.
Response
Successful Response
One eval's frozen headline — the poll target for a launched run.
id here is the same identifier the launch returned as eval_id. The
headline fields (accuracy, matched, mismatched, ungraded)
are null while status is in_progress; accuracy stays null
on a processed eval whose graded denominator is 0.
Unique identifier of the eval (hyphenated UUID); equals the launch's eval_id.
"069dcc2c-e14c-7606-8000-2ee4fb17b4f9"
Per-workflow run counter (#1, #2, …) for this eval.
3
Workflow version the eval scored.
"abcdef1234"
Grading status. Non-terminal: in_progress. Terminal: processed, error.
in_progress, processed, error "processed"
Number of dataset documents in the eval's realized cohort.
42
Number of dataset documents that failed to enqueue for this run.
0
Headline accuracy in [0, 1]; null while in_progress and on a processed eval with a 0 graded denominator.
0.94
Fields that matched ground truth; null while in_progress.
Fields that mismatched ground truth; null while in_progress.
Fields with no ground truth to grade against; null while in_progress.
Timestamp when the eval was created (ISO 8601).

