> ## Documentation Index
> Fetch the complete documentation index at: https://docs.anyformat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Upload to Dataset

> Upload one document (+ optional ground truth) into a workflow dataset

*Rate limit tier: **submission** (60 req/min) — see [Rate limits](/api-reference-v3/introduction#rate-limits).*

Uploads 1–10 files as a single [document packet](/concepts/document-packets) in the workflow's dataset, plus an optional `ground_truth` for that document packet. Creation is all-or-nothing: any rejected file (unsupported type, disguised bytes) or a ground-truth failure fails the whole request and nothing is stored.

Send the files as multipart form data under the `files` field (repeat the field for a multi-file document). The response returns a stable `document_packet_id` for the document; the presigned upload-slot ids stay hidden.

Bulk ingestion is client-orchestrated: loop this endpoint, **one call per document**. There is no batch endpoint — each call is bounded, atomic, and independently retryable. The SDKs provide the per-document loop.

<Note>
  **Filenames are unique within a workflow.** By default (`on_conflict=error`) a document whose name collides with a live dataset member fails with `409` (a rename is never silent); the response lists each conflict and the name it would take. Pass `on_conflict=rename` to auto-rename the collision instead — the returned file's `name` is the name it landed under and `original_name` holds the name you sent (`null` when no rename happened).
</Note>

<Note>
  **Ground truth (`ground_truth`).** Optional; a JSON-encoded object — the expected document for this document packet (multipart can't carry nested objects natively, so send the JSON as a string). Keys are the workflow schema's field identifiers: advertise `persistent_id` (the API also accepts `sanitized_name`). A scalar field maps to `string | null`; a table/object field maps to an array of row objects. Ground truth attaches to the workflow's current version. On success `ground_truth_saved` is `true`.
</Note>

<Note>
  **Retries are safe with `Idempotency-Key`.** Pass any unique string; retrying the request with the same key replays the original document packet, so no duplicate document is registered. See [Idempotency](/api-reference-v3/introduction#idempotency).
</Note>

<RequestExample>
  ```bash curl theme={null}
  curl -X POST 'https://api.anyformat.ai/v3/workflows/0686bb97-8c30-70f0-8000-97669e000eb8/dataset/upload/' \
    -H 'Authorization: Bearer YOUR_API_KEY' \
    -H 'Idempotency-Key: 5f2b6c3e-8a1d-4f7e-9c0b-1a2b3c4d5e6f' \
    -F 'files=@/path/to/invoice_001.pdf' \
    -F 'ground_truth={"invoice_no": "INV-1", "total": "90.00"}'
  ```

  ```python Python (SDK) theme={null}
  from anyformat.sdk import Client, DatasetDocument

  client = Client(api_key="YOUR_API_KEY")
  workflow_id = "0686bb97-8c30-70f0-8000-97669e000eb8"

  # One document with ground truth.
  upload = client.upload_to_dataset(
      workflow_id,
      "invoice_001.pdf",
      ground_truth={"invoice_no": "INV-1", "total": "90.00"},
  )
  print(upload.document_packet_id, upload.ground_truth_saved)

  # Bulk: one atomic call per document.
  uploads = client.upload_documents_to_dataset(
      workflow_id,
      [
          DatasetDocument(file="invoice_001.pdf", ground_truth={"invoice_no": "INV-1"}),
          DatasetDocument(file="invoice_002.pdf", ground_truth={"invoice_no": "INV-2"}),
      ],
  )
  ```

  ```typescript TypeScript theme={null}
  interface DatasetUpload {
    document_packet_id: string;
    files: { id: string; name: string; original_name: string | null }[];
    ground_truth_saved: boolean;
  }

  const workflowId = '0686bb97-8c30-70f0-8000-97669e000eb8';
  const formData = new FormData();
  formData.append('files', fileInput.files[0]);
  formData.append('ground_truth', JSON.stringify({ invoice_no: 'INV-1', total: '90.00' }));

  const response = await fetch(
    `https://api.anyformat.ai/v3/workflows/${workflowId}/dataset/upload/`,
    {
      method: 'POST',
      headers: {
        'Authorization': 'Bearer YOUR_API_KEY',
        'Idempotency-Key': crypto.randomUUID(),
      },
      body: formData,
    }
  );

  if (!response.ok) {
    throw new Error(`API error: ${response.status}`);
  }

  const upload: DatasetUpload = await response.json();
  console.log(upload.document_packet_id, upload.ground_truth_saved);
  ```
</RequestExample>

<ResponseExample>
  ```json Response (201 Created) theme={null}
  {
    "document_packet_id": "069dcc2c-e14c-7606-8000-2ee4fb17b4e1",
    "files": [
      { "id": "069dcc2c-e14c-7606-8000-2ee4fb17b4e2", "name": "invoice_001.pdf", "original_name": null }
    ],
    "ground_truth_saved": true
  }
  ```
</ResponseExample>


## OpenAPI

````yaml POST /v3/workflows/{workflow_id}/dataset/upload/
openapi: 3.1.0
info:
  title: anyformat API
  description: >
    Document extraction and workflow automation API.


    AnyFormat lets you define extraction workflows that pull structured data
    from any document — PDFs, images, scanned files, or plain text. Upload a
    file, run it through a workflow, and get back structured fields with
    confidence scores and source evidence.


    ## Quick start


    See the [Quick start
    guide](https://docs.anyformat.ai/api-reference/introduction) for a
    walkthrough.


    ## Authentication


    All endpoints (except `/health/`) require a Bearer token in the
    `Authorization` header:


    ```

    Authorization: Bearer <your-api-key>

    ```


    Get your API key from
    [app.anyformat.ai/api-key](https://app.anyformat.ai/api-key).


    ## Versioning


    Endpoints are versioned by path prefix (`/v2/`, `/v3/`). Every response
    under

    a versioned prefix includes `X-API-Version`.
  version: 2.0.0
servers:
  - url: https://api.anyformat.ai
    description: API server
security: []
tags:
  - name: workflows
    description: >-
      Workflows define extraction templates — what fields to extract from
      documents. Create workflows, upload files, run extractions, and fetch
      results.
  - name: files
    description: >-
      File collections group uploaded documents and track their extraction
      progress. Upload files, check status, and retrieve extraction results.
  - name: suggestions
    description: >-
      AI-powered schema and workflow suggestion endpoints for the AnyFormat
      webapp. Authenticated with a Cognito JWT rather than an API key.
  - name: document-packets
    description: >-
      Document packets group uploaded documents for a workflow (v3). Address a
      packet directly to (re-)run its workflow.
  - name: runs
    description: >-
      Extraction runs — the v3 first-class result entity. Each run is a single
      execution of a workflow against a document packet; fetch its status and
      results flat by run id.
  - name: health
    description: Health check endpoints to verify API availability.
paths:
  /v3/workflows/{workflow_id}/dataset/upload/:
    post:
      tags:
        - datasets
      summary: Upload a document (+ ground truth) to a workflow dataset
      description: >-
        Upload one document (1..N files) plus optional ground truth as ONE

        document packet in the dataset (atomic).


        All-or-nothing: any rejected file (unsupported type, disguised bytes) or
        a

        ground-truth failure fails the whole request and nothing is stored.


        Filenames are unique within a workflow. By default (`on_conflict=error`)
        a

        file whose name collides with an existing file fails the request with
        `409`

        before anything is uploaded; pass `on_conflict=rename` to auto-rename
        the

        collision instead (`invoice.pdf` -> `invoice (1).pdf`), each returned
        file's

        ``name`` being the name it landed under and ``original_name`` holding
        the

        uploaded name when a rename happened.


        Bulk ingestion loops this endpoint, one call per document; each call is

        bounded and independently retryable.
      operationId: v3_upload_to_dataset
      parameters:
        - name: workflow_id
          in: path
          required: true
          schema:
            type: string
            title: Workflow Id
        - name: Idempotency-Key
          in: header
          required: false
          schema:
            anyOf:
              - type: string
              - type: 'null'
            description: >-
              Optional caller-supplied key (Stripe convention). Retrying the
              request with the same key replays the original document packet —
              no duplicate document is registered.
            title: Idempotency-Key
          description: >-
            Optional caller-supplied key (Stripe convention). Retrying the
            request with the same key replays the original document packet — no
            duplicate document is registered.
      requestBody:
        required: true
        content:
          multipart/form-data:
            schema:
              $ref: '#/components/schemas/Body_v3_upload_to_dataset'
      responses:
        '201':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/DatasetUploadResponseV3'
          headers:
            X-RateLimit-Limit:
              description: Requests allowed per window (60s).
              schema:
                type: integer
            X-RateLimit-Remaining:
              description: Requests remaining in the current window.
              schema:
                type: integer
            X-RateLimit-Reset:
              description: Seconds until the current window resets.
              schema:
                type: integer
        '401':
          description: Authentication failed — missing, invalid, or expired API key.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorEnvelope'
              examples:
                AUTH_FAILED:
                  value:
                    error: Authentication failed.
                    detail: Authentication failed.
                    error_code: AUTH_FAILED
                    retryable: false
                    request_id: req_abc123
        '404':
          description: >-
            Resource not found — the workflow, document packet, or run ID does
            not exist or is not visible to this API key.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorEnvelope'
              examples:
                NOT_FOUND:
                  value:
                    error: Resource not found.
                    detail: Resource not found.
                    error_code: NOT_FOUND
                    retryable: false
                    request_id: req_abc123
        '422':
          description: >-
            Ground truth failed validation (`GT_INVALID`): `detail.errors` lists
            each offending key. Other 422 reasons — unsupported bytes
            (`UNSUPPORTED_FILE_TYPE`) and an unresolved extract node
            (`EXTRACT_NODE_UNRESOLVED`) — use the standard envelope with a
            string `detail`.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/GroundTruthInvalidError'
        '429':
          description: >-
            Rate limited — retry after the `Retry-After` header. Default
            windows: 60 requests/minute for extraction submission, 600
            requests/minute for general endpoints.
          headers:
            Retry-After:
              description: Seconds until the next request is allowed.
              schema:
                type: integer
            X-RateLimit-Limit:
              description: Requests allowed per window (60s).
              schema:
                type: integer
            X-RateLimit-Remaining:
              description: Requests remaining in the current window.
              schema:
                type: integer
            X-RateLimit-Reset:
              description: Seconds until the current window resets.
              schema:
                type: integer
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorEnvelope'
              examples:
                RATE_LIMITED:
                  value:
                    error: Rate limit exceeded — retry after backoff.
                    detail: Rate limit exceeded — retry after backoff.
                    error_code: RATE_LIMITED
                    retryable: true
                    request_id: req_abc123
      security:
        - ApiKeyAuth: []
components:
  schemas:
    Body_v3_upload_to_dataset:
      properties:
        files:
          items:
            type: string
            contentMediaType: application/octet-stream
          type: array
          title: Files
          description: >-
            1..10 files forming ONE document (one document packet in the
            dataset).
        ground_truth:
          anyOf:
            - type: string
            - type: 'null'
          title: Ground Truth
          description: >-
            Optional JSON-encoded object: the expected document for this
            document packet. Keys are the workflow schema's field identifiers —
            advertise `persistent_id` (the API also accepts `sanitized_name`). A
            scalar field maps to `string | null`; a table/object field maps to
            an array of row objects. Ground truth attaches to the workflow's
            current version.
        on_conflict:
          type: string
          enum:
            - error
            - rename
          title: On Conflict
          description: >-
            How to handle an uploaded filename that already exists in the
            workflow (filenames are unique within a workflow). `error` (the
            default) rejects the whole request with `409` and lists the
            conflicting names, so a rename is never silent — the caller must opt
            in. `rename` accepts the collision and lets the server auto-rename
            the file by inserting a ` (n)` counter before the extension
            (`invoice.pdf` → `invoice (1).pdf`).
          default: error
      type: object
      required:
        - files
      title: Body_v3_upload_to_dataset
    DatasetUploadResponseV3:
      properties:
        document_packet_id:
          type: string
          title: Document Packet Id
          description: >-
            Unique identifier of the created dataset document packet (hyphenated
            UUID).
          examples:
            - 069dcc2c-e14c-7606-8000-2ee4fb17b4e1
        files:
          items:
            $ref: '#/components/schemas/DatasetUploadFileV3'
          type: array
          title: Files
          description: Files in the packet, in the order they were provided.
        ground_truth_saved:
          type: boolean
          title: Ground Truth Saved
          description: >-
            Whether ground truth was supplied and saved for this document
            packet.
      type: object
      required:
        - document_packet_id
        - files
        - ground_truth_saved
      title: DatasetUploadResponseV3
      description: |-
        Response for `POST /v3/workflows/{workflow_id}/dataset/upload/`.

        One dataset document packet, all-or-nothing. `document_packet_id` is the
        stable handle for the deferred ground-truth-edit follow-up.
    ErrorEnvelope:
      type: object
      title: ErrorEnvelope
      description: >-
        Uniform error response. `retryable=true` means the same request may
        succeed on retry (with backoff); `retryable=false` means the caller must
        change the request before retrying.
      required:
        - error
        - detail
        - error_code
        - retryable
        - request_id
      properties:
        error:
          type: string
          description: Short human-readable summary.
        detail:
          description: >-
            Machine-readable detail. String for most errors; list of field
            errors for validation failures.
          oneOf:
            - type: string
            - type: array
              items:
                type: object
        error_code:
          type: string
          description: Stable identifier for programmatic handling.
          examples:
            - VALIDATION_ERROR
            - AUTH_FAILED
            - NOT_FOUND
            - PRECONDITION_FAILED
            - RATE_LIMITED
            - INTERNAL_ERROR
            - GATEWAY_TIMEOUT
        retryable:
          type: boolean
          description: True when the caller should retry (with backoff for 429/412/5xx).
        request_id:
          type: string
          description: Correlates this response with server logs.
    GroundTruthInvalidError:
      properties:
        error:
          type: string
          title: Error
          examples:
            - Ground truth validation failed
        detail:
          $ref: '#/components/schemas/GroundTruthErrorDetail'
        error_code:
          type: string
          const: GT_INVALID
          title: Error Code
          default: GT_INVALID
        retryable:
          type: boolean
          title: Retryable
          default: false
        request_id:
          type: string
          title: Request Id
          description: Correlates this response with the server logs.
      type: object
      required:
        - error
        - detail
        - request_id
      title: GroundTruthInvalidError
      description: >-
        `422 GT_INVALID` error envelope: the supplied ground truth failed
        validation.


        `detail.errors` lists each offending key. The other 422 reasons on this
        route

        (unsupported bytes, an unresolved extract node, an invalid request)
        reuse the

        standard error envelope with a string `detail`.
    DatasetUploadFileV3:
      properties:
        id:
          type: string
          title: Id
          description: Unique identifier of the file (hyphenated UUID).
          examples:
            - 069dcc2c-e14c-7606-8000-2ee4fb17b4e2
        name:
          type: string
          title: Name
          description: Filename the file is stored under.
          examples:
            - invoice.pdf
        original_name:
          anyOf:
            - type: string
            - type: 'null'
          title: Original Name
          description: >-
            The name you uploaded, present only when the file was auto-renamed
            (`on_conflict=rename` resolving a collision). `null` when the stored
            `name` is the name you sent.
      type: object
      required:
        - id
        - name
      title: DatasetUploadFileV3
      description: Per-file entry in a dataset-upload response.
    GroundTruthErrorDetail:
      properties:
        errors:
          items:
            $ref: '#/components/schemas/GroundTruthErrorItem'
          type: array
          title: Errors
          description: One entry per offending ground-truth key.
      type: object
      required:
        - errors
      title: GroundTruthErrorDetail
      description: '`detail` body of a `422 GT_INVALID` error.'
    GroundTruthErrorItem:
      properties:
        field:
          type: string
          title: Field
          description: Dotted path to the offending ground-truth key.
          examples:
            - line_items.0.sku
        msg:
          type: string
          title: Msg
          description: Human-readable reason the value was rejected.
      type: object
      required:
        - field
        - msg
      title: GroundTruthErrorItem
      description: >-
        One ground-truth validation error.


        Stable public remap of the backend's pydantic `{type, loc, msg}`:
        `field`

        is the dotted path from `loc`, `msg` is kept, the raw pydantic `type` is

        dropped so the public contract never leaks the validator name.
  securitySchemes:
    ApiKeyAuth:
      type: http
      description: >-
        API key issued from app.anyformat.ai/api-key. Send as `Authorization:
        Bearer <key>`.
      scheme: bearer

````