Batch API Reference

Overview

Batch processing is the Runtime API surface for non-interactive, independent work such as classification, enrichment, summarization, or offline evaluation. A batch is a parent operation; each item becomes its own invocation with a stable invocation ID and its own terminal outcome.

New OpenAI batches use the Responses API, as regular chat and Economy do. Keep using the Interlocute request format; you do not choose a provider API endpoint or send OpenAI request bodies.

Successful items create their thread and messages after provider completion. At acceptance time, the response returns preallocated thread, input-message, and output-message IDs for correlation; an item that fails or is canceled does not create an empty thread. Those identifiers are not result URLs: retrieve completed output through the batch results endpoint.

Retrieval is demand-driven: an authorized request for batch details, results, or an item's Chit receipt checks for progress and collects available results. Frequent or concurrent requests may reuse the last saved status. Configured provider completion webhooks can also collect results automatically. Interlocute does not run periodic provider polling. There is no Interlocute age cutoff for uncollected results: a later request still checks upstream. Provider work and charges continue even if you never retrieve them.

Endpoints

All endpoints are relative to the Runtime API and require credentials authorized for the target node.

On a Node alias host, POST /chats accepts the same batchId and items object shown below.

Submit a batch

Submit 1–10,000 independent items. Provider request/file limits, model context limits and aggregate metadata limits also apply; split large collections into smaller batches when needed. Large input text uses the same lossless storage path as ordinary chat, and completed outputs remain available in the normal receipt shape. The /chats alias takes this Batch request object, including per-item controls; it does not take an array of ordinary /chat requests.

batchId is optional and makes a submission idempotent when you retry the same items. Each customId must be unique within the request and is the correlation key returned in results. Each item may also provide sparse, capability-validated inference controls; omitted controls inherit the node's configured execution posture.

Repeating a batchId with the same items and effective settings returns the original acceptance. Changed content, inference controls, or resolved provider/model returns 409 batch_conflict without enrolling new provider work. Use a new ID for an intentional new attempt.

curl -X POST https://api.interlocute.ai/nodes/nd_abc123/batches \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "batchId": "customers-2026-08-20",
    "items": [
      {
        "customId": "customer-1001",
        "content": "Classify this support request: please reset my account.",
        "maxOutputTokens": 120
      },
      {
        "customId": "customer-1002",
        "content": "Classify this support request: I need an invoice copy."
      }
    ]
  }'

A successful request returns 202 Accepted after the Batch and inputs are durably saved and estimated credit is reserved. Validation and credit refusal remain synchronous; insufficient credit returns 402 rather than accepted work. Preparation and provider submission continue after acceptance, so planned does not mean the provider has accepted it yet. Closing the HTTP connection does not cancel an accepted Batch. Use the explicit cancellation endpoint.

Keep the returned Batch ID as your immediate status handle. All item invocation, Thread and message IDs are allocated at acceptance, but individual Chit receipts may not yet exist while preparation runs. Read the Batch detail first if an early item lookup returns 404; do not submit duplicate work. Supply your own batchId before sending when you need to recover an acknowledgment lost to a client timeout. Repeat the identical request with that ID.

{
  "batchId": "customers-2026-08-20",
  "state": "planned",
  "itemCount": 2,
  "createdAtUtc": "2026-08-20T08:00:00Z",
  "expiresAtUtc": null,
  "reservedChargeMicros": 3400,
  "currency": "USD",
  "items": [
    {
      "customId": "customer-1001",
      "invocationId": "inv_01...",
      "threadId": "thr_01...",
      "inputMessageId": "msg_01...",
      "outputMessageId": "msg_02..."
    },
    {
      "customId": "customer-1002",
      "invocationId": "inv_02...",
      "threadId": "thr_02...",
      "inputMessageId": "msg_03...",
      "outputMessageId": "msg_04..."
    }
  ]
}

Select a model for the Batch

Set optional model at the top level of the Batch request to select one model for every item. Omit it to use the Node's default model. Item output and reasoning controls are validated against the selected model, and the accepted model is preserved for pricing, submission and receipts. You cannot select a different model inside each item; submit separate Batches when comparing models.

An explicit Node model allow-list authorizes only its listed models from the same provider, including an approved model that costs more than the default. Without an explicit list, overrides must stay with the same provider and be cheaper than the Node default. Unknown models return 400; disallowed selections return 403. Model selection does not bypass capability checks or credit reservation.

{
  "batchId": "comparison-sol-1",
  "model": "gpt-6-sol",
  "items": [
    {
      "customId": "variant-1",
      "content": "Summarize this source.",
      "maxOutputTokens": 2400
    }
  ]
}

The example requires a Node that authorizes the selected model. Changing the model when replaying an existing Batch ID conflicts; it never changes already accepted work or silently substitutes the default.

Per-item inference controls

maxOutputTokens is a per-item output ceiling, not a token budget for the whole batch. Omit it to inherit the Node's configured output setting; if no setting is supplied, Batch uses 4,096 output tokens per item. An explicit item value overrides the Node default and must be positive and supported by the selected model/provider. There is no separate 100,000-token Batch ceiling. The Node default is not a hard cap on item overrides. Input context and provider batch/file limits still apply.

Choose a realistic ceiling for each task: the credit reservation includes estimated input and each item's maximum output. A larger ceiling increases the reservation even if the model ultimately writes less; the final charge is reconciled to reported usage. Repeating a Batch ID with changed ceilings conflicts; use a new Batch ID when intentionally changing the request.

Batch items use the same public control vocabulary as Chat options. The API accepts reasoning with optional effort, mode, and thinking members, plus maxOutputTokens, temperature, and topP. Provider and model remain owned by the node/batch configuration; there is no per-item provider field here.

curl -X POST https://api.interlocute.ai/nodes/nd_abc123/batches \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "batchId": "classification-2026-08-20",
    "items": [
      {
        "customId": "customer-1001",
        "content": "Classify this support request: please reset my account.",
        "reasoning": { "effort": "medium" },
        "maxOutputTokens": 120
      },
      {
        "customId": "customer-1002",
        "content": "Classify this support request: I need an invoice copy.",
        "maxOutputTokens": 120
      }
    ]
  }'

Monitor lifecycle

Provider completion and local result readiness are separate. Large completed Batches can need several bounded ingestion passes before resultsReady becomes true. Status checks retain completed-item progress and resume unfinished ingestion; they do not purchase inference again. Configured provider completion notifications also continue this finite result recovery before the consumer completion callback is delivered.

Cognitive synthesis is off by default for API Batches. Set the root cognitiveSynthesisEnabled field to true to request optional Thread cognitive-state synthesis after each successful receipt completes. It does not gate result readiness, settlement or completion callbacks. This optional work is best-effort: a host restart can interrupt it; it does not retry automatically. Human chat retains its synthesis default. The choice is frozen at acceptance, so changing it on a retry conflicts. Historical accepted Batches retain their existing choice. Completed provider output is retained privately and reused across local ingestion passes rather than downloaded again for every pass.

Poll the batch detail while it is active. It reports normalized lifecycle state, local item counts, the provider batch ID and provider status when assigned, provider counts, a last-poll heartbeat, billing reconciliation, cancellation state, and whether final results are ready. Common states are planned, submitted, inprogress, completed, failed, canceled, and expired.

Preparation failures and safe retry

Recover after insufficient credits

An initial 402 can leave a saved, failed Batch with credit_insufficientfunds before acceptance. No provider work has started and no credit has been reserved. The 402 response and Batch summary report canRetryReservation: true and recoveryAction: "retry_reservation" when this recovery is available.

Top up, then repeat the identical create POST with the same Batch ID, items, model, controls and callback configuration. Keep the original request with your durable job. Interlocute retries reservation using the retained Batch and item/invocation identities. Success returns 202; insufficient funds still returns 402. Concurrent duplicate retries cannot reserve twice or submit accepted work again. Changed requests return a conflict.

A balance top-up or status read alone does not resume inference. Keep your existing job paused until the explicit POST succeeds, then monitor that same Batch and receive its normal completion callback if configured. This recovery is separate from safeToResubmit, which permits a new Batch identity after a refunded preparation failure. Never infer recovery safety from failed or pending alone. Accepted, canceled, submitted and uncertain provider work cannot be restarted by credit recovery.

{
  "status": 402,
  "batchId": "your-original-batch-id",
  "canRetryReservation": true,
  "recoveryAction": "retry_reservation"
}

Preparation and provider submission

During planned, local preparation is still running and provider submission has not begun. submitting means a provider request may already be underway; submissionuncertain means its acceptance cannot be confirmed. Interlocute does not automatically submit those uncertain requests again.

If local preparation fails before provider submission, the Batch reports its failure and refunds the reservation. A newly accepted Batch that never starts preparation within ten minutes, or whose preparation owner is subsequently lost, can be failed by reading that specific Batch or replaying its identical create request. Interlocute reports preparation_interrupted and refunds reserved credit. Active preparation is not failed merely because ten minutes have passed since acceptance. If preparation cannot retain its ownership before submission, it reports preparation_ownership_lost and refunds the reservation. Recovery happens on demand; no scheduled check resumes interrupted work. Preparation recovery is separate from the provider's execution window and result retention.

Read safeToResubmit on the Batch summary/detail instead of inferring retry safety from a failed status. True confirms that no provider submission was attempted and reserved credit was refunded. Submit an intentional retry with a new Batch ID; the failed identity is not reopened. False is not permission to replay, including for submitting, uncertain or already submitted work. A definitive provider rejection may refund credit while still reporting false. Batch lists show saved summaries; open the specific Batch for recovery.

A completed batch means provider processing has ended; individual items can still have errors. Inspect each item's terminal receipt for success, failure or cancellation. Expired work also exposes an item failure reason. Unknown provider progress does not become a successful completion. The batch list shows saved summaries; request a batch's details, results or an item's Chit for fresh progress.

curl https://api.interlocute.ai/nodes/nd_abc123/batches/customers-2026-08-20 \
  -H "Authorization: Bearer $API_KEY"

Cancel a batch or Economy invocation

Cancellation is a request, not an assertion that every item stopped instantly. Items already accepted by a provider can finish and be billed; continue polling until the batch becomes terminal and final results are ingested.

Canceling while the Batch is still planned can prevent provider submission and refund the reservation. A client disconnect, a timeout or stopping status polling is not an explicit cancellation.

curl -X POST https://api.interlocute.ai/nodes/nd_abc123/batches/customers-2026-08-20/cancel \
  -H "Authorization: Bearer $API_KEY"

Economy chat acceptance includes a batchId; use it on this parent batch route. POST /chit/{invocationId}/cancel applies to direct Async chat and returns 409 for batch item receipts. See direct Async cancellation when your request uses responseMode: "Async".

Retrieve all results

Every accepted item can also be checked individually through GET /chit/{invocationId} on the Node alias host, or GET /nodes/{nodeId}/chit/{invocationId}. Once the item's receipt exists, an authorized item read refreshes its parent batch and returns the normal receipt, with batchId. During initial preparation use the Batch detail as the status handle. Bulk retries use the parent batchId and per-item customId; bulk items do not accept externalInvocationId.

Call the results endpoint after resultsReady is true. Before then it returns 409 Batch Results Not Ready. Results are paged with pageSize and continuationToken; follow the token until it is absent. This prevents a large completed batch from being silently truncated. Each item contains its normal invocation receipt. For a completed item, read or parse the assistant response from invocation.outputs[].content where kind is assistantMessage.

curl "https://api.interlocute.ai/nodes/nd_abc123/batches/customers-2026-08-20/results?pageSize=50" \
  -H "Authorization: Bearer $API_KEY"

{
  "batchId": "customers-2026-08-20",
  "state": "completed",
  "items": [
    {
      "customId": "customer-1001",
      "invocation": {
        "invocationId": "inv_01...",
        "batchId": "customers-2026-08-20",
        "status": "completed",
        "threadId": "thr_01...",
        "executionSettings": {
          "provider": "OpenAIResponses",
          "model": "gpt-5-mini",
          "reasoning": { "effort": "medium" },
          "maxOutputTokens": 120
        },
        "outputs": [{ "kind": "assistantMessage", "outputId": "msg_02...", "threadId": "thr_01...", "content": "account_access" }],
        "usage": { "inputTokens": 18, "outputTokens": 4, "totalTokens": 22, "estimated": false }
      }
    }
  ],
  "continuationToken": null,
  "hasMore": false
}

Optional completion callbacks

Add callbackUrl to a batch request, the identical /chats request, or an Economy chat request. Supply your public HTTPS endpoint directly; no callback registration is needed. The destination applies only to this batch and is saved privately with it. Optionally supply callbackSigningSecret to authenticate delivery. Changing the URL or signing secret on an idempotent retry returns 409. Omit the URL for no callback; ordinary Chit retrieval remains available either way. HTTP, IP literals, private network destinations, URL user information, fragments and redirects are rejected.

{
  "batchId": "customers-2026-10-09",
  "callbackUrl": "https://your-app.example/interlocute/receipts",
  "callbackSigningSecret": "your-random-shared-secret",
  "items": [{ "customId": "customer-1001", "content": "Classify this support request." }]
}

When all item receipts are terminal, Interlocute POSTs one or more JSON pages to your supplied destination. Each page wraps the same receipt collection returned by POST /chits: type: "batch.receipts", deliveryId, batchId, offset, and receipts. Receipt status, invocation IDs, external invocation IDs when supplied, outputs, errors and usage keep their normal meaning. A failed item remains a failed item in the callback.

Pages contain at most 100 receipts and the full envelope is at most 1 MiB. No output is truncated. A receipt too large for a page has per-item 413 / receipt_too_large; retrieve it through individual Chit. The page offset and receipts.nextOffset refer to the original ordered batch items.

Every page carries webhook-id and webhook-timestamp. If you supplied callbackSigningSecret, verify the webhook-signature header before accepting a page. The signature is v1, followed by base64 HMAC-SHA256, using your supplied signing secret as UTF-8 bytes over webhook-id.webhook-timestamp.raw-body. Check timestamp freshness and deduplicate by delivery ID. Save the page durably, then acknowledge with 2xx. Without a signing secret, the signature header is omitted. URLs and secrets are not echoed in receipts. Retries preserve the same delivery ID and body; delivery is at least once and retries are bounded. A delivery failure does not cancel execution, erase receipts or change charges. Chit remains available.

Batch support does not imply provider webhook support. OpenAI completion webhooks require operator registration in the upstream OpenAI project and can cover every Node using it. Anthropic Message Batches currently uses on-demand retrieval; its consumer callback starts after a Chit or batch read collects the terminal results. Direct Async completion callbacks are not supported by this batch feature.

Callback status and redelivery

The Node Batches detail page always indicates whether a completion callback is configured. If none exists, enter a public HTTPS URL and optional signing secret, confirm that your endpoint is ready, and choose Save callback. For an already completed Batch this starts delivery of its existing receipts; it never repeats inference or changes billing. An existing destination cannot be replaced. Signing secrets are cleared from the form after submission.

API clients can attach a missing callback with PUT /nodes/{nodeId}/batches/{batchId}/callback, sending callbackUrl and optional callbackSigningSecret. Repeating the same configuration is safe; a different destination or secret returns 409. Your original Batch-create request remains idempotent after this separate configuration operation. Failed delivery shows a Retry callback action on the detail page, with confirmation and the same cooldown and quota as the API. While the page is open, active callback progress refreshes.

Read GET /nodes/{nodeId}/batches/{batchId}/callback or the callback member of a Batch summary to see delivery state, acknowledged item count, page attempts, next retry and recent HTTP outcomes. Callback failure does not change completed results or billing. The Node Batches detail page shows this status too. Only the most recent 32 attempts are retained; historical deliveries may have no attempt records. Destination URLs, secrets and response bodies are not exposed.

After fixing your endpoint, POST /nodes/{nodeId}/batches/{batchId}/callback/redeliver with your normal bearer credentials. A failed callback resumes at its retained page with the same payload and delivery ID; already acknowledged pages are not sent again. This never reruns inference. Pending requests coalesce and return 202; delivered callbacks return 200. Non-ready callbacks return 409. New redelivery requests have a five-minute cooldown and a limit of three per Batch in 24 hours; a 429 response includes Retry-After. Wait for your endpoint repair before triggering.

Automatic delivery makes at most eight attempts per page: one initial attempt and seven retries. After consecutive retryable failures, the delays are 30 seconds, 1 minute, 2 minutes, 4 minutes, 8 minutes, 16 minutes and 32 minutes. This totals 63 minutes 30 seconds of waiting, plus HTTP request and queue time. A consumer's Retry-After can extend a delay up to one hour; scheduling or outages can postpone attempts, so this is not a guaranteed wall-clock deadline. Transient HTTP 408, 409, 425, 429 and 5xx failures retry with persisted backoff; other HTTP failures stop for manual intervention. A retry retains the delivery ID, so deduplicate delivery and durably save each page before 2xx. An acknowledged delivery proves HTTP acceptance, not completion of your downstream processing.

After the eighth unsuccessful attempt, callback state becomes failed and automatic delivery stops. Interlocute does not keep retrying forever. Each acknowledged page advances the cursor and gives the next page its own attempt budget. Stopping callback delivery does not delete the Batch's results or change charges; ordinary Chit and Batch result reads remain available.

Manually resume after fixing your endpoint

Manual retry is available after automatic attempts are exhausted or a permanent HTTP failure is repaired. It gives the failed page a fresh, bounded eight-attempt budget and continues the remaining pages. It keeps the retained payload and delivery ID, and does not resend acknowledged pages. A callback already marked delivered is a no-op; this route does not replay a successfully acknowledged Batch. No callback is sent merely by reading status.

curl -X POST https://api.interlocute.ai/nodes/nd_abc123/batches/customers-2026-08-20/callback/redeliver \
  -H "Authorization: Bearer $API_KEY"

After HTTP 202, use GET /nodes/{nodeId}/batches/{batchId}/callback to monitor delivery. delivered means every page was acknowledged; failed means intervention is required. Check canRedeliver and redeliveryAvailableAtUtc before retrying. The five-minute cooldown and three-request rolling 24-hour limit apply across clients sharing the Batch.

Completion, expiry and result availability

A provider's completion window is separate from result retention. Interlocute does not cancel or expire submitted, uncollected provider work merely because time has passed. The local preparation limit described above applies before submission. A request can retrieve an available result even a year later; provider retention is not guaranteed. expiresAtUtc is a provider-reported timestamp when known, otherwise null, and is not a receipt deletion deadline.

Once Interlocute collects a result, it stays available locally without an age-based receipt expiry or another provider read. If upstream has deleted an uncollected job or result, unresolved items fail with provider_batch_unavailable or provider_results_unavailable. Already collected items remain available. These errors report retrieval loss; they do not imply that provider execution was canceled or free. Temporary upstream errors can be retried by reading the same receipt or batch.

Economy chat and multi-item batches

responseMode: "economy" on /chat is a single-item convenience path for an isolated, new first turn. It returns a batch ID and invocation ID, but it is not the API for sending a collection. Use POST /nodes/{nodeId}/batches for multi-item work.

Provider support, completion windows, pricing, and request-size limits vary by provider and model. Treat the runtime's accepted response and lifecycle state as the operational contract rather than assuming a universal duration or discount.

Economy is available for supported batch-capable models. An allowed options.model selects the model; omission uses the Node default. Supply externalInvocationId for safe same-request POST retries, then poll the returned item URL. Changed request intent returns 409; conversational externalCorrelationId remains unsupported. See the Economy Chat contract and example.