Tool calling

Pass tools and tool_choice exactly as you would to the OpenAI API. Whichever provider actually serves the request — including ones with a completely different native tool-call wire format, like Anthropic's Messages API — gets an adapted call, and the response comes back to you in the standard OpenAI tool_calls shape.

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "parameters": {"type": "object", "properties": {"city": {"type": "string"}}},
        },
    }],
)

Capability-aware routing

When you route with model:"auto" and pass tools, the router already knows this request needs tool support and only considers models the catalog marks as tool-capable — you never get auto-routed to a text-only model that would 400 on your own tools array. Pinning an explicit model id skips this check; set provider.require_parameters: true (see Provider selection) to enforce the same capability check even on an explicit pin, degrading to the next candidate rather than a raw upstream 400.

Structured outputs

response_format is normalized across providers — the same body works whether the request lands on OpenAI, Anthropic, or Gemini. The simplest form, { "type": "json_object" } (JSON mode, valid JSON with no fixed shape), is forwarded as-is. For a specific schema, use "type": "json_schema":

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Extract the invoice total and due date"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "invoice",
            "schema": {
                "type": "object",
                "properties": {
                    "total_usd": {"type": "number"},
                    "due_date": {"type": "string"},
                },
                "required": ["total_usd", "due_date"],
            },
        },
    },
)

Under model:"auto", either form of response_format only routes to a model the catalog marks as JSON-mode capable — browse which models support it (badged "JSON mode") at /models.

Other forwarded params

Also passed straight through: temperature, top_p, max_tokens, stop, n, frequency_penalty, presence_penalty, seed, and logprobs — a parameter a chosen provider doesn't support surfaces as that provider's own validation error, same as calling it directly.