Chat Completions

Create completions with OpenAI-compatible request/response shapes plus optional Xantly orchestration controls.

• POST /v1/chat/completions
• HEAD /v1/chat/completions (returns 204 No Content; useful as a lightweight probe)
• Auth: Authorization: Bearer <token>

Recommended rollout: start with standard fields only (model, messages, temperature, max_tokens), then add orchestration controls incrementally.

Quick start example

curl -sS https://api.xantly.com/v1/chat/completions \
  -H "Authorization: Bearer $XANTLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      {"role": "user", "content": "Explain vector databases in 2 bullets."}
    ]
  }'

Request body

Standard parameters (OpenAI-compatible)

ChatMessage object

Proprietary parameters (Xantly)

These are optional. If omitted, the gateway uses defaults and automatic policy.

routing_hints (soft preferences)

routing_override (harder overrides)

xantly orchestration block

Legacy request headers (still supported)

The gateway maps selected request headers into metadata for backward compatibility.

Response body

Non-stream (200 OK)

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1741400000,
  "model": "deepseek-chat",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 31,
    "completion_tokens": 18,
    "total_tokens": 49
  },
  "xantly_metadata": {
    "request_id": "req_01abc...",
    "routing_decision": "Lane: Turbo (Source: BarpRouter)",
    "provider": "deepseek",
    "tier": "T2",
    "provider_tier": "T2",
    "latency_ms": 142,
    "requested_model": "auto",
    "decision_source": "BarpRouter",
    "task_family": "analysis",
    "cost_usd": 0.00032,
    "baseline_cost_usd": 0.00120,
    "savings_usd": 0.00088,
    "savings_pct": 73.3,
    "cost_attribution": "xantly",
    "estimated_cost_usd": 0.00032,
    "healer_report": null
  }
}

xantly_metadata fields

Notes

• choices[].logprobs is part of schema, but may be omitted in normalized API responses.
• For n > 1, if upstream returns fewer choices, the gateway may replicate the first choice to satisfy requested cardinality.

Stream (text/event-stream)

• All stream responses emit chat.completion.chunk events and terminate with data: [DONE].
• Non-voice streaming currently emits a compact sequence (role chunk + content chunk + [DONE]) instead of token-by-token chunking.
• stream_options.include_usage is forwarded to providers where supported; do not assume a terminal usage chunk is always present.

Response headers

Common observability headers

Cache-path headers

Usage / cost headers

These are included when usage is available:

• x-xantly-input-tokens
• x-xantly-output-tokens
• x-xantly-cost-usd

Errors

Error payloads use OpenAI-style shape:

{
  "error": {
    "message": "temperature (2.5) must be between 0 and 2",
    "type": "invalid_request_error",
    "code": "validation_error",
    "param": null
  }
}

Error headers

• x-error-id is returned on error responses for support correlation.
• retry-after is returned on eligible rate-limit responses.

Edge cases and implementation notes

1. Model validation is catalog-aware

   • model validation is strict when the model catalog is loaded.
   • During catalog cold-start/empty states, strict rejection may be relaxed.

2. top_logprobs validation

   • Server enforces top_logprobs <= 20.
   • Provider-specific logprobs behavior may vary.

3. Chain limit overrides are conditional

   • xantly.max_chain_steps and xantly.chain_timeout_secs apply only when workflow resolves to long_horizon_autonomous.

4. Streaming semantics

   • Non-voice stream mode is SSE-compatible but not guaranteed token-by-token.

5. Reserved fields

   • Some accepted proprietary fields are currently advisory/reserved for compatibility (force_provider, cache_ttl_secs, compress_context, enable_tool_reranking, and certain routing_hints fields).

Practical rollout checklist

1. Start with model: "auto" + standard fields.
2. Enable response_format for structured outputs.
3. Add one routing/orchestration control at a time.
4. Track x-xantly-* response headers in observability.
5. Add override fields only for test/debug traffic.

See also

• Benchmark Results — Every parameter on this page is individually validated. See the full 252-test scorecard including boundary and invalid-input cases.
• Streaming Responses — Complete guide to SSE format, chunk handling, and SDK examples.
• Rate Limits — RPM and TPM limits that apply to this endpoint.
• Billing & Quotas — Token quotas, budget caps, and cost visibility via xantly_metadata.