Provider Failover

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[
        (Anthropic(), "claude-sonnet-4-5"),
    ],
)

# If the OpenAI call fails (or its circuit is open), Solwyn translates the
# request and retries it on Anthropic — automatically, in-process.
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(response.choices[0].message.content)
client.close()

Solwyn routes each call through an ordered failover chain: the wrapped client first, then each configured fallback in turn. Failover is automatic and in-process — no second deployment, no proxy. The same chain handles two cases:

• Same-provider model fallback — retry on the same client with a different model (e.g. gpt-4o to gpt-4o-mini).
• Cross-provider failover — retry on a different provider (e.g. OpenAI to Anthropic), with the request translated to the target's dialect.

The fallback chain

Pass fallback= a list of provider entries. Each entry is a tuple:

(client, model)                              # provider client + model name
(client, model, default_params)              # ... plus fill-absent default request params
(client, model, default_params, provider)    # ... plus an explicit provider identity

The optional 4th element names the entry's provider when auto-detection cannot — e.g. (other_client, "my-model", {}, "ollama") for a local server on a non-default port. It follows the same rules as the constructor's provider= argument: SDK v0.6.0+, a pin bypasses type and base_url detection entirely and selects the named adapter; it does not translate dialects, rewrite the endpoint, or synthesize a different client, and construction still validates the client's family and sync/async mode against the pinned adapter. An unknown name raises ConfigurationError(field="provider"); a mismatched client raises ConfigurationError(field="client"). See Explicit provider identity.

The chain is [primary, *fallbacks]. providers[0] is always the wrapped client; the rest are attempted in the order you list them.

import os
from anthropic import Anthropic
from google import genai
from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(),                                   # primary
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[
        (OpenAI(), "gpt-4o-mini"),                                  # same provider, cheaper model
        (Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024}),  # cross-provider
        (genai.Client(api_key=os.environ["GEMINI_API_KEY"]), "gemini-2.0-flash"),
    ],
)

Note: Each entry carries no credentials. Provider API keys live only on the client objects you construct. ProviderEntry rejects api_key / base_url with a hard error — Solwyn never accepts, stores, or logs a provider credential.

The model constructor argument

model= names the model for the primary entry. It is optional: for a single-provider client with no fallback, the per-call model= you pass to .create() is authoritative and the constructor model= can be omitted. When you configure a fallback chain, set it so the primary entry is fully described.

default_params

default_params are fill-absent request parameters — applied only when the caller did not pass that key on the call. Precedence is:

per-call kwargs  >  per-entry default_params  >  global default_params

Set a global default for every entry via the constructor, and per-entry defaults inside the tuple. This is how you supply a target provider's required fields — for example Anthropic requires max_tokens, which OpenAI callers do not send:

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    default_params={"temperature": 0.7},        # applied to every entry
    fallback=[
        (Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024}),  # entry-only
    ],
)

Same-provider model fallback

When a fallback entry uses the same provider as the primary, only the model is swapped — same HTTP client, same API key, native passthrough (no translation). A success on such a hop is reported with is_model_fallback=true.

import os
from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(OpenAI(), "gpt-4o-mini")],
)
# A failing gpt-4o call retries on gpt-4o-mini before the chain is exhausted.

Cross-provider failover

When a fallback entry targets a different provider, the hop's behavior depends on whether the two providers speak the same API dialect. A success served by a different provider is reported with is_provider_fallback=true either way.

Same-dialect hops (native passthrough)

When source and target both speak the OpenAI Chat Completions dialect — e.g. Groq → OpenRouter, or OpenAI → an OpenAI-compatible provider — the request passes through natively: tools, JSON mode, and streaming all survive, with no canonical-subset restriction. Instead of translating, Solwyn sanitizes the request for the new endpoint:

• max_completion_tokens is rewritten to max_tokens for targets that need the legacy key, and inversely max_tokens → max_completion_tokens for OpenAI/Azure o1 / o3 / o4 / gpt-5 targets (when both keys are present in one source, the modern key wins).
• A caller-supplied stream_options is stripped when the hop lands on a provider known to reject it. On your configured primary it always reaches the provider untouched.
• Endpoint-scoped extra_headers / extra_query / extra_body are stripped on cross-provider hops — they carry gateway credentials authored for the original endpoint — and the target entry's own default_params versions re-apply. They are untouched on the primary and on same-provider model swaps.

Cross-dialect hops (canonical translation)

When the dialects differ — e.g. OpenAI → Anthropic, or Bedrock → Anthropic — Solwyn translates the request from the source dialect into a canonical subset, then into the target's dialect, before the hop. The response is reshaped to the caller's dialect: a Bedrock caller served by Anthropic still gets a Converse-shaped dict, and an OpenAI caller served by Bedrock gets OpenAI-shaped objects and chunks.

Translation is a structural operation only — it maps request shape (messages, system prompt, tool definitions, common parameters). It never inspects, logs, or rewrites prompt or response content. See Privacy.

Amazon Bedrock participates in both directions via translation — e.g. Bedrock-hosted Claude failing over to direct Anthropic, or an OpenAI primary failing over to Bedrock. Bedrock-specific hop rules (native Bedrock primary calls pass kwargs straight through to boto3, untouched):

• inferenceConfig.maxTokens is required for a cross-provider hop from Bedrock — the SDK will not invent an output bound and fails loud with the structural label missing_max_tokens.
• guardrailConfig fails loud on a hop (bedrock.guardrail_config) — a safety feature a hop would otherwise silently strip — as do cachePoint blocks, s3Location image sources, and other shapes outside the canonical subset. See Amazon Bedrock.
• boto3 has no per-call timeout override, so the chain's failover_total_timeout cannot shorten an in-flight Bedrock hop. Bound it on the client with botocore.config.Config(retries={"total_max_attempts": 1}, read_timeout=60) so Solwyn owns retries and failover instead of stacking botocore's legacy retry layer. See Timeouts and retries.

Distinct endpoints need distinct provider identities

Circuit-breaker health, latency signals, price hints, and failover labeling key off the provider name. Two chain entries that resolve to the same name — two Azure resources, or two unnamed gateways both detected as openai_compatible — share one health domain, are reported as model fallbacks of each other, and skip cross-provider request sanitization (the stream_options strip, the max_completion_tokens rewrite, the endpoint-scoped param strip). A header authored for the first endpoint then reaches the second untouched and can fail there. Give distinct endpoints distinct identities via provider= or the 4th fallback-tuple element — and, conversely, do not pin two different endpoints to the same name, which creates the same collision.

Responses API calls never fail over at all: they are served by the primary or fail. See The Responses API.

When a request cannot be translated

On a cross-dialect hop, some request shapes have no equivalent on the target provider. Rather than silently dropping a field, Solwyn fails loud before any network call and aborts the whole chain (tool-using streams across dialects also fail loud, pre-dispatch):

Both carry structural labels only — source, target, and a feature token like "response_format" — never the offending value and never prompt content.

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import Solwyn, UntranslatableRequestError

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024})],
)

try:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )
except UntranslatableRequestError as e:
    print(f"Cannot translate {e.feature} from {e.source} to {e.target}")

client.close()

Aborting the chain on an untranslatable feature is deliberate: a translated request that quietly differs from what you asked for is worse than a clear failure. Keep your fallback chain to providers that can serve the same request shape, or scope the untranslatable feature out of calls that may fail over.

Selection policies

A selection policy decides the order in which healthy candidates are attempted. A policy is pure and side-effect-free; it only reorders the usable set, dropping any provider whose circuit is open and not recovery-eligible. Pass one with selection_policy=:

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import Solwyn, LatencyPolicy

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024})],
    selection_policy=LatencyPolicy(),
)

All three apply the same health filter first, so a policy can never promote an unhealthy provider ahead of a healthy one, nor attempt a provider whose circuit is open. Selection policies are deliberately outside the plan-scoped tuning entitlement: no directive ever swaps or suppresses your policy.

Note: LatencyPolicy orders by observed latency, so it needs samples before it can act. Solwyn records the latency of each successful call in a rolling window and reports a provider's p50 only after at least 3 successes. Until a provider reaches that threshold it sorts after providers with a known p50, so the chain behaves like HealthBasedPolicy while latencies warm up.

Cost-aware routing

SDK v0.6.0+. Every per-call budget check opts into price hints, and Solwyn Cloud answers with a map of relative token rates for the providers in the request's failover chain: the cheapest positively-priced provider is 1.0, a provider twice as expensive is 2.0, zero-cost local lanes (ollama, vllm, lmstudio) are 0.0, and a provider Solwyn cannot price is omitted. The numbers are dimensionless ratios computed server-side from the standard-tier input-plus-output rate at the request's context bracket, and the fold is deliberately asymmetric: the primary is priced from its own model, every other provider from its priciest declared model. The SDK never computes, derives, or combines prices — CostPolicy only sorts by the server's numbers.

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import CostPolicy, Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024})],
    selection_policy=CostPolicy(),
)

Hints are request-scoped: they are priced for the chain and estimate on that check and applied to exactly that call's dispatch. A check that changes the primary model gets new hints. A cached allow replays its own entry's hints, never another's. There is no client-wide hint store — Solwyn.update_price_hints was removed in 0.6.0 — and nothing to configure: hints are available on every tier.

When a healthy, cheaper provider serves a call ahead of the healthy primary, the metadata event reports failover_reason: "cost_routed" with is_provider_fallback: true and the requested provider and model. The label is attribution only — it never changes the order the policy chose — and it is applied only when the served hint is strictly below the primary's (or the primary carries no hint), both breakers are CLOSED, and the walk never reached the primary. A cheaper hop that fails, after which the primary also errors and a later provider serves, reports primary_error — if the primary itself serves, the call carries no failover_reason at all; a custom policy that fronts a pricier provider reports circuit_open, never cost_routed; a same-provider model swap stays model_fallback. The hint values themselves are never reported.

Three server answers are worth recognizing:

Known limitation: lease-backed solwyn.run() calls carry no price hints — lease grants and renewals do not price the chain yet — so CostPolicy keeps configured order for those calls, and the warning above fires once even though nothing is misconfigured. Tagged calls always take a live check; untagged non-run calls may be served from the allow cache, which replays that entry's own hints. Also note that a zero-cost local lane sorts strictly first under CostPolicy, so a chain with an ollama fallback will route every healthy call to it and label the call cost_routed; the openai_compatible catch-all is deliberately excluded from the zero-cost lane so paid gateways are unaffected.

Failover tuning

Note: These failover knobs are constructor-only — they have no SOLWYN_* environment variable. Only the fields listed in SolwynConfig can be set from the environment.

Failover timeouts

New in 0.4.0. Failover is bounded by two independent timeouts, and understanding the split matters if you run reasoning models or large max_tokens values.

Before 0.4.0, failover_total_timeout capped everything, including an in-flight hop's read. A slow-but-connected provider was cut at 30 seconds and re-raised as APITimeoutError. That cut bought nothing: a read timeout is post-send ambiguous — the request may already have run and been billed — so under the default failover_idempotency="safe" it re-raises without failing over. All it did was convert legitimately slow generations into ambiguous spend.

So the two bounds now do different jobs:

A pre-send hang (connect, pool wait) is provably failover-safe, so it must fail inside the window. A post-send read is not, so it gets its own generous bound.

600.0 matches the openai and anthropic SDKs' own read/write default, so a wrapped call's read bound never fires earlier than the unwrapped SDK's would. Because window expiry still gates advancement between hops, at most one hop per call can consume the full read bound:

worst-case wall clock  ≈  one failover window  +  one hop read timeout

Lower failover_hop_read_timeout if you would rather fail fast than wait out a slow generation — remembering that the fast failure is an ambiguous re-raise, not a failover:

client = Solwyn(OpenAI(), api_key="sk_proj_...", failover_hop_read_timeout=120.0)

Breaking in 0.6.0: both bounds must be finite numbers. Booleans, NaN, +inf, and -inf are rejected at construction with ConfigurationError naming the field; float("inf") was previously accepted and produced an unbounded window or hop read. failover_hop_read_timeout must also be greater than zero. A zero failover_total_timeout remains legal: no candidate is dispatched and ProviderUnavailableError carries the full attempted chain.

Anthropic 1.x ships on the separate httpx2 HTTP stack. The per-hop bound is delivered to such a client as a native httpx2.Timeout when the SDK can prove the client's own timeout class by identity, and as a granular four-tuple otherwise — the transport bound is correct either way; only Anthropic's informational x-stainless-read-timeout header degrades in the fallback. Solwyn never imports httpx2. See Anthropic.

Clients exposing with_options — openai, anthropic, every OpenAI-compatible provider, and together — receive a granular per-hop timeout in their own HTTP stack (an httpx.Timeout, or the httpx2.Timeout above for anthropic 1.x): connect and pool from the shrinking window, read and write from the hop bound. Two providers cannot be bounded that way:

• Google Gemini. google-genai supports only a single whole-request timeout — it cannot split connect from read. Solwyn gives a google hop the read bound as its whole-request timeout, so a google pre-send hang is not bounded by the failover window and can block up to failover_hop_read_timeout without ever failing over. See Google Gemini.
• Amazon Bedrock. boto3 has no per-call timeout override, so Solwyn cannot bound a Bedrock hop at all — your botocore Config(read_timeout=...) governs. Building a Bedrock client with read_timeout=None now logs a warning. See Bedrock.

Also fixed in 0.4.0: a connect or pool timeout that the provider SDK wrapped in APITimeoutError now fails over correctly. Both openai and anthropic wrap the entire httpx/httpx2 TimeoutException family in one class, so a provably pre-send ConnectTimeout used to reach classification wearing the same class name as a post-send read timeout — and was treated as ambiguous, taking the whole call down instead of failing over. APITimeoutError is now classified by its chained cause.

Plan-scoped tuning entitlement

New in 0.3.0, widened in 0.4.0. Custom failover and breaker tuning is an entitlement — it is available on plans that carry the failover-tuning entitlement. Each pre-flight budget check opts into a versioned failover directive, and Solwyn Cloud may answer with one. The directive governs a closed set of eight fields — the five failover knobs above and the three circuit-breaker parameters:

No other configuration field is touched — provider entries, the selection policy, and price hints are deliberately outside this boundary, so CostPolicy and cost-aware routing are never suppressed by plan. The directive has three states:

The transition is reversible in both directions — it is not a one-way ratchet. A false directive replaces your values with the defaults above; a later true restores your original constructor values.

Changes are applied in place to the breakers already running:

• Provider order never changes. The directive tunes timing and thresholds only — it cannot reorder, add, or remove chain entries.
• Breakers keep their identity and their accumulated health and probe state. A CLOSED or HALF_OPEN breaker is unaffected mid-episode; an OPEN breaker re-samples its recovery window from the new timeout and jitter.
• The chain deadline preserves the original call start. A directive-driven failover_total_timeout change replaces the total but never restarts the clock, so the remaining window can shrink below the time already elapsed. This holds across the sync and async chat and media paths alike.
• Each call reads one coherent snapshot. Since 0.4.0, a call captures the tuning once at dispatch and consumes only that snapshot. A directive landing mid-call can no longer leave it running on a torn mix of old and new values — for example the new total timeout paired with the old idempotency mode.

When a false directive suppresses tuning you actually asked for, the SDK logs one WARNING — Custom failover tuning is unavailable for this plan; SDK defaults applied — at most once per client instance, and only when your requested tuning genuinely differs from the defaults. Clients running default configuration see nothing. See Logging.

Idempotency and duplicate-call safety

A failure after the request reached the provider (a read timeout, a dropped connection mid-response) is ambiguous: the call may already have run and been billed. Failing over would risk a duplicate provider charge. failover_idempotency controls that trade-off:

Override per call with the solwyn_idempotent keyword (stripped before the request reaches the provider):

# This specific call is safe to retry anywhere — fail over aggressively.
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    solwyn_idempotent=True,   # -> "always" for this call
)

solwyn_idempotent=True maps to "always", False maps to "safe"; omit it to use the configured failover_idempotency.

Circuit breaker

The circuit breaker tracks the health of each provider in the chain independently. Enforcement is process-local — no breaker state is shared between SDK instances, and no snapshot ever drives admission. From SDK v0.3.0+ that state is also reported to Solwyn Cloud for dashboard visibility; see Breaker state reporting. When a provider's circuit is open, the router skips it and moves to the next candidate without attempting a call.

Note: The dashboard, homepage, and alert messages show these states under plain names: CLOSED appears as Healthy, OPEN as Down, and HALF_OPEN as Recovering. SDK code, logs, and wire payloads always use the technical terms documented on this page.

CLOSED  --[failure_threshold failures]--> OPEN
OPEN    --[recovery_timeout elapsed]----> HALF_OPEN
HALF_OPEN --[success_threshold OKs]----> CLOSED
HALF_OPEN --[any failure]--------------> OPEN

State transitions

1. CLOSED to OPEN: After failure_threshold consecutive failures (default: 3), the circuit opens. Request-shaped failures — a 4xx other than 429 and 529 — never count toward this threshold. The router stops sending to this provider until the recovery timeout elapses.
2. OPEN to HALF_OPEN: After recovery_timeout seconds (default: 60, with jitter), the circuit allows probe requests.
3. HALF_OPEN to CLOSED: After success_threshold consecutive successes (default: 2), the circuit closes.
4. HALF_OPEN to OPEN: Any failure during probing immediately re-opens the circuit.

Tuning the circuit breaker

These three can also be set via SOLWYN_CIRCUIT_BREAKER_FAILURE_THRESHOLD, SOLWYN_CIRCUIT_BREAKER_RECOVERY_TIMEOUT, and SOLWYN_CIRCUIT_BREAKER_SUCCESS_THRESHOLD.

All three are part of the eight-field closed set a plan-scoped failover directive can govern (SDK v0.3.0+) — on a plan without the failover-tuning entitlement they revert to the defaults above.

Note: A same-provider model fallback that fails on both the primary model and the fallback model records only one failure against that provider's breaker — breaker failures are deduplicated per provider within a single request, so one failing request can never burn more than one count toward failure_threshold per provider.

Breaker state reporting

New in 0.3.0. The SDK reports each provider's breaker state to Solwyn Cloud, giving the dashboard fleet-wide provider-health visibility. Each report carries structural labels and quantities only — never content:

Reporting is advisory and one-way. Cloud snapshots never drive admission, and breaker state is never shared between instances — every failover decision stays 100% in-process. Turning reporting off changes what the dashboard sees, never how your chain behaves.

Beyond the dashboard, these snapshots drive circuit_breaker_opened and circuit_breaker_closed webhook events and notifications on paid plans, per project, provider, and SDK instance — so a provider tripping in your fleet can page you rather than wait to be noticed. Disabling reporting, or never establishing a project id, means no events fire.

Behavior notes:

• Never blocks a call. Snapshots ride the background reporter on its own worker, off the request path. A send failure is dropped with a warning — see Logging.

Changed in 0.4.0. Reporting got substantially quieter:

• Reports are sent when a breaker's snapshot changes, not on every cycle. "Snapshot" means the state plus its failure and success counts — so a provider failing intermittently while its breaker stays closed still reports, because its counters moved. A full refresh of every provider goes out periodically regardless, governed by breaker_report_heartbeat (default 60.0 seconds, SOLWYN_BREAKER_REPORT_HEARTBEAT).
• A failed send stays due rather than being forgotten, so the next cycle retries it. An idle reporter with nothing due does no work at all.
• close() attempts one final forced snapshot within the shared shutdown deadline, so a clean exit leaves the dashboard current.
• A read-only key is recognized on this path. It previously produced one reporter.breaker_send_failed warning per provider on every 5-second cycle — exactly the noise the read-only diagnostic exists to collapse. A read-only refusal now logs the once-per-process diagnostic and ends the cycle instead of posting the remaining doomed snapshots. See Troubleshooting.

The control-plane breaker that guards Solwyn Cloud itself is a separate health domain and is deliberately excluded from these reports — they describe your LLM providers' health, not Solwyn's.

• Needs a project first. Reports are sent only after a successful budget check has established the project id. Until that happens, there are silently no reports.
• One instance id per client. The id is a fresh UUID for each client instance, so client-per-request patterns fragment fleet snapshots into single-use identities. Construct the client once and reuse it.

Off switch — the only new configuration field in 0.3.0:

import os
from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    breaker_reporting_enabled=False,   # or SOLWYN_BREAKER_REPORTING_ENABLED=false
)

ProviderUnavailableError

ProviderUnavailableError is raised when no candidate could be dispatched — every circuit open, or the failover window expired. If every candidate was attempted and failed, the last provider's own exception propagates instead:

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import ProviderUnavailableError, Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024})],
)

try:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )
except ProviderUnavailableError as e:
    print(f"No provider could be dispatched. Candidates: {e.attempted}")

client.close()

Attributes

What the dashboard sees

Every call reports a metadata event tagged with how the chain resolved, so the dashboard can show how often failover saves a request:

These fields carry structural labels only — never prompts, responses, or error strings. See Privacy.

Related

• SolwynConfig — every failover field and its precedence
• Error Handling — UntranslatableRequestError, UntranslatableModelError, ProviderUnavailableError
• Logging — look for Circuit breaker opened to confirm the circuit tripped
• Troubleshooting — failover that fires more (or less) than expected