OpenAI-compatible providers

Any provider that speaks the OpenAI Chat Completions dialect works with Solwyn: point an openai.OpenAI client at the endpoint via base_url and wrap it as usual. Solwyn detects the real provider from the URL, so budgets, per-agent attribution, failover, and the cost dashboard all see the actual provider (e.g. groq) -- not "openai". (The Responses API is a separate story: it is metered on Azure OpenAI only; on every other compatible endpoint its leaves are untracked and follow your on_unmetered posture — see Azure OpenAI.)

from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(base_url="https://api.groq.com/openai/v1", api_key="gsk_..."),
    api_key="sk_proj_...",
)
response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Hello!"}],
)

Available from SDK v0.1.7. No new install extras -- compat providers use the same openai client package as the OpenAI provider, installed via pip install "solwyn[openai]".

Using Together's native Together or AsyncTogether client instead? See the dedicated Together AI provider guide. The OpenAI-client route below remains supported when you intentionally use Together's compatible endpoint.

Auto-detected providers

Detection runs at construction, from the client's base_url (or, for Azure, the client class). Each provider also carries its own streaming-usage policy -- Solwyn injects stream_options={"include_usage": True} only where that is documented-safe:

Notes on detection:

• Local servers (Ollama, vLLM, LM Studio) are recognized by their conventional default ports on a local host (localhost, 127.0.0.1, 0.0.0.0, ::1). When identity is guessed from a port, the SDK logs a one-time INFO message naming the detected provider (never the URL) and suggesting provider= if the guess is wrong. A non-default port still works via the generic catch-all -- use the explicit override to name it.
• The catch-all claims any http(s) base URL that is not OpenAI's own (api.openai.com, including regional eu. / us. subdomains) and not a named provider above. A client with no custom base_url remains plain openai.
• Model prefixes supplement URL detection for providers with distinctive naming (grok-, deepseek-, the Mistral family, qwen / qwq- / qvq-, glm-, accounts/fireworks/, sonar). Providers that serve shared open-model catalogs (Groq, Together, OpenRouter, local servers) deliberately have no model prefixes -- a bare llama-3-8b is never silently claimed for any of them.

Explicit provider identity

For endpoints auto-detection cannot name -- e.g. vLLM on a non-default port, or a private gateway -- pass the provider explicitly. On the constructor for the primary client, or as the 4th element of a fallback spec:

client = Solwyn(
    OpenAI(base_url="http://gpu-box:8080/v1", api_key="-"),
    api_key="sk_proj_...",
    provider="vllm",
    fallback=[
        (OpenAI(base_url="https://openrouter.ai/api/v1", api_key="sk-or-..."), "openrouter/auto"),
        (other_client, "my-model", {}, "ollama"),
    ],
)

Fallback specs accept (client, model), (client, model, default_params), or (client, model, default_params, provider).

SDK v0.6.0+: a pin is an identity assertion, not a label applied after detection. It bypasses type and base_url detection entirely and selects the named adapter. That is what lets native OpenAI behind a corporate gateway or local proxy keep native metering — including the Responses API — instead of being detected as the generic openai_compatible catch-all:

client = Solwyn(
    OpenAI(base_url="https://llm-gateway.internal/v1", api_key="..."),
    api_key="sk_proj_...",
    provider="openai",       # native OpenAI adapter, Responses metered
)

The pin does not translate dialects, rewrite the endpoint, or synthesize a different SDK client — it cannot make an Anthropic client speak the OpenAI wire shape. Construction still validates the actual client family and sync/async mode against the pinned adapter, and fails loud with ConfigurationError:

• field="provider" — an unknown provider name (the message lists the nineteen known values), or a malformed fallback spec (field="fallback_specs").
• field="client" — a pin whose family does not match the client object (provider="anthropic" on an openai.OpenAI client), a sync client under AsyncSolwyn or vice versa (the message names the wrapper to use), a bedrock pin on anything but a bedrock-runtime client, or a client that is not a recognized provider SDK client.

Health, latency, price hints, and failover labeling all key off the provider name, so two different endpoints pinned to the same name share one breaker and one identity. See the exceptions reference for details.

Z.AI

Z.AI's OpenAI-compatible endpoint serves the GLM model family. Point the client at it and wrap as usual — detection keys off the api.z.ai host, with glm- model prefixes as a supplement:

client = Solwyn(
    OpenAI(base_url="https://api.z.ai/api/paas/v4", api_key="..."),
    api_key="sk_proj_...",
)
response = client.chat.completions.create(
    model="glm-4.6",
    messages=[{"role": "user", "content": "Hello!"}],
)

Tracked calls are attributed as provider="zai". Z.AI is priced for text only — the provider × modality matrix has the full picture. Streamed calls settle on exact usage, not estimates: Z.AI supports stream_options.include_usage, so Solwyn injects it and reads usage from the final chunk.

Z.AI detection and streaming-usage injection require Solwyn 0.1.11 or newer.

Azure OpenAI

Azure clients are detected by class (AzureOpenAI / AsyncAzureOpenAI) or by host suffix, and attributed as provider="azure_openai".

Azure pricing follows the one thing an Azure request actually carries: your deployment name, an arbitrary string you chose — not a model catalog id. A deployment named after the OpenAI catalog model it serves (the common convention — a deployment called gpt-4o serving gpt-4o) prices at OpenAI catalog rates. Any other deployment name is accepted and recorded unpriced — real spend the platform declined to guess at, visible on the dashboard's unpriced lane, never rejected and never a silent $0. Name deployments after their catalog models if you want priced events.

SDK v0.6.0+: Azure is the one compatible profile with a metered Responses API. responses.create, responses.parse, and the responses.stream() helper on an AzureOpenAI / AsyncAzureOpenAI client are budget-checked and settled exactly as on native OpenAI — attributed as azure_openai, with the same refusals (background, streaming parse, metering-critical extra_body), the same primary-only rule, and the deployment-name pricing rule above. Responses streams read usage from the terminal response.completed event and never send stream_options, so the data_sources caveat is Chat-Completions-only. On every other compatible endpoint the Responses leaves — including create / parse / stream — are untracked and follow your on_unmetered posture (warn once and forward by default); they are not budget-checked and produce no cost event. Azure Responses coverage is unit-tested against fakes; the end-to-end lane exercises the native OpenAI pin. See The Responses API.

Streaming usage and estimation

Sync and async streaming are fully supported (AsyncSolwyn mirrors Solwyn). Token accounting for a stream resolves in three tiers:

1. Standard usage block -- the last chunk whose usage parses to non-zero counts wins; zeroed placeholder blocks never latch.
2. Groq legacy x_groq.usage -- the older final-chunk shape is read when the standard block is absent.
3. Flagged estimation -- if the provider reports no usage at all, the SDK falls back to a length-based estimate: input tokens from the pre-call length estimate, output tokens from accumulated delta character lengths at 4.0 characters per token.

Estimation is explicitly marked: the wire payload carries token_details.is_estimated = true, and the SDK logs a one-time WARNING per provider. Budgets still enforce, but per-call costs are approximate. Degraded accounting is loud and flagged, never silently zero.

Non-streaming responses behave the same way: missing usage, a zeroed usage block alongside real content, or garbage counts (negative, non-integer) fall back to flagged estimation. Provider-reported non-zero usage is never overridden, and an all-zero block on a genuinely empty response is taken as provider truth, not estimated. Usage extraction never raises -- a deliverable response is never settled as a failure over broken usage metadata.

is_estimated is serialized only when true, so payloads from providers that always report usage are unchanged.

For an Azure Responses stream the same tiers apply with two differences: tier 1 reads the usage nested under the terminal response.completed event, and tier 3's output estimate is structurally zero — the delta-length walker is Chat-Completions-shaped — so a Responses stream that ends without usage settles as a marked input-only estimate and keeps its lease reservation.

stream_options drop-in contract

The injection policy in the table above is Solwyn's own behavior. A stream_options you pass explicitly always reaches your configured provider untouched. It is stripped only when a cross-provider failover hop lands on a provider known to reject it (and the data_sources exception applies on hops onto Azure).

Token detail fields

Compat adapters reuse the full OpenAI Chat Completions extractor: input_tokens, output_tokens, cached_input_tokens (from prompt_tokens_details.cached_tokens), reasoning_tokens, audio input/output tokens, and accepted/rejected prediction tokens -- see the OpenAI token detail table for the exact source fields. Azure Responses calls use the Responses-shape extractor from the same table. A returned service_tier is captured (bounded and string-only), as on OpenAI.

Compat providers carry no per-region pricing contract, so no provider region is reported.

Pricing

The SDK never computes cost. It reports the served (provider, model) pair verbatim -- for OpenRouter that is the full model slug (e.g. anthropic/claude-sonnet-4.5) -- and the Solwyn Cloud API prices it server-side. Models unknown to the pricing catalog surface as unpriced on the dashboard, never silently costed at $0.

Failover

Compat providers participate fully in provider failover:

• Same-dialect hops (e.g. Groq → OpenRouter) are native passthrough -- tools, JSON mode, and streaming all survive, with no canonical-subset restriction. max_completion_tokens is rewritten to max_tokens for targets that need the legacy key (and inversely max_tokens → max_completion_tokens for OpenAI/Azure o1 / o3 / o4 / gpt-5 targets).
• Endpoint-scoped params (extra_headers / extra_query / extra_body) are stripped on cross-provider hops -- they carry gateway credentials authored for the original endpoint -- and the target entry's own default_params versions re-apply. They are untouched on the primary and on same-provider model swaps.
• Cross-dialect hops (e.g. Groq → Anthropic) use the standard canonical translation subset. Tool-using streams across dialects fail loud with UntranslatableRequestError before any network call.

Known limitations

Circuit-breaker health, latency signals, and failover labeling key off the provider name. Two chain entries that resolve to the same name -- two Azure resources, or two unnamed gateways both detected as openai_compatible -- share one health domain, are reported as model fallbacks of each other, and skip cross-provider request sanitization (stream_options stripping, the max_completion_tokens rewrite, endpoint-scoped param stripping). A header authored for the first endpoint then reaches the second untouched and can fail there. Give distinct endpoints distinct provider= identities -- on the constructor, or as the 4th element of a fallback spec.

chat.completions.create, embeddings.create, images.generate, audio.transcriptions.create, and audio.speech.create are intercepted on the OpenAI dialect. Embedding calls are budget-checked before the request and priced server-side from the response's usage.prompt_tokens, recorded as a cost event with modality embedding; when a compat endpoint returns no usable usage, the SDK falls back to a length-based estimate flagged is_estimated = true -- never a silent $0. Image generation is budget-checked before the request and recorded as a cost event with modality image; because compat image endpoints return no usage, its billing quantities are request-derived (image count × the model's per-image rate) -- measured inside the privacy firewall from config values (the n, size, and quality you pass), never the prompt and never image bytes. Audio calls are budget-checked and recorded as a cost event with modality audio: transcriptions price from their usage token buckets (token-billed models such as gpt-4o-transcribe) or from the whole-second audio duration on the provider's usage block (Whisper-style models -- present only on a JSON response_format; a text/srt/vtt call is recorded but unpriced and warns once), and text-to-speech prices from the input character count measured inside the privacy firewall, never the text itself. Groq is fully covered here -- its Whisper transcription prices per hour of audio and its Orpheus TTS per input character. Token-billed TTS models that report no usage (gpt-4o-mini-tts) and audio.translations are untracked and follow your on_unmetered posture — warn once per process by default. Video generation (videos.create, Sora) is OpenAI-native only, so on an OpenAI-compatible endpoint it is unsupported — videos.create fails loud with UnsupportedSurfaceError rather than passing through. The Responses API is metered on Azure OpenAI only; on every other compatible endpoint the Responses leaves — including create / parse / stream — are untracked and follow your on_unmetered posture (warn once and forward by default); they are not budget-checked and produce no cost event. Every other client surface (files, batches, models.list, ...) is untracked and follows on_unmetered the same way.

Privacy

The same privacy posture applies as for every provider: Solwyn is a wrapper, not a proxy -- calls go direct to the provider, and prompts and responses are never captured, logged, or transmitted. Compat-specific guarantees:

• Length-based estimation sums string lengths of message text, reasoning content, and tool-call arguments without concatenating, storing, or logging any content -- only an irreversible character count is used.
• The streaming accumulator extracts usage and service_tier as chunks pass through and never retains chunk objects (deltas carry content).
• The local-port detection log names the detected provider only, never the base_url, which may carry credentials.
• Provider credentials live only on your client objects -- Solwyn's provider configuration cannot carry an api_key or base_url.