Provider Failover

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[
        (Anthropic(), "claude-sonnet-4-5"),
    ],
)

# If the OpenAI call fails (or its circuit is open), Solwyn translates the
# request and retries it on Anthropic — automatically, in-process.
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(response.choices[0].message.content)
client.close()

Solwyn routes each call through an ordered failover chain: the wrapped client first, then each configured fallback in turn. Failover is automatic and in-process — no second deployment, no proxy. The same chain handles two cases:

• Same-provider model fallback — retry on the same client with a different model (e.g. gpt-4o to gpt-4o-mini).
• Cross-provider failover — retry on a different provider (e.g. OpenAI to Anthropic), with the request translated to the target's dialect.

The fallback chain

Pass fallback= a list of provider entries. Each entry is a tuple:

(client, model)                              # provider client + model name
(client, model, default_params)              # ... plus fill-absent default request params
(client, model, default_params, provider)    # ... plus an explicit provider identity

The optional 4th element names the entry's provider when auto-detection cannot — e.g. (other_client, "my-model", {}, "ollama") for a local server on a non-default port. It follows the same rules as the constructor's provider= argument: it relabels within the client's API dialect only, and an unknown or mismatched value raises ConfigurationError at construction. See OpenAI-compatible providers.

The chain is [primary, *fallbacks]. providers[0] is always the wrapped client; the rest are attempted in the order you list them.

import os
from anthropic import Anthropic
from google import genai
from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(),                                   # primary
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[
        (OpenAI(), "gpt-4o-mini"),                                  # same provider, cheaper model
        (Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024}),  # cross-provider
        (genai.Client(api_key=os.environ["GEMINI_API_KEY"]), "gemini-2.0-flash"),
    ],
)

Note: Each entry carries no credentials. Provider API keys live only on the client objects you construct. ProviderEntry rejects api_key / base_url with a hard error — Solwyn never accepts, stores, or logs a provider credential.

The model constructor argument

model= names the model for the primary entry. It is optional: for a single-provider client with no fallback, the per-call model= you pass to .create() is authoritative and the constructor model= can be omitted. When you configure a fallback chain, set it so the primary entry is fully described.

default_params

default_params are fill-absent request parameters — applied only when the caller did not pass that key on the call. Precedence is:

per-call kwargs  >  per-entry default_params  >  global default_params

Set a global default for every entry via the constructor, and per-entry defaults inside the tuple. This is how you supply a target provider's required fields — for example Anthropic requires max_tokens, which OpenAI callers do not send:

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    default_params={"temperature": 0.7},        # applied to every entry
    fallback=[
        (Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024}),  # entry-only
    ],
)

Same-provider model fallback

When a fallback entry uses the same provider as the primary, only the model is swapped — same HTTP client, same API key, native passthrough (no translation). A success on such a hop is reported with is_model_fallback=true.

import os
from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(OpenAI(), "gpt-4o-mini")],
)
# A failing gpt-4o call retries on gpt-4o-mini before the chain is exhausted.

Cross-provider failover

When a fallback entry targets a different provider, the hop's behavior depends on whether the two providers speak the same API dialect. A success served by a different provider is reported with is_provider_fallback=true either way.

Same-dialect hops (native passthrough)

When source and target both speak the OpenAI Chat Completions dialect — e.g. Groq → OpenRouter, or OpenAI → an OpenAI-compatible provider — the request passes through natively: tools, JSON mode, and streaming all survive, with no canonical-subset restriction. Instead of translating, Solwyn sanitizes the request for the new endpoint:

• max_completion_tokens is rewritten to max_tokens for targets that need the legacy key, and inversely max_tokens → max_completion_tokens for OpenAI/Azure o1 / o3 / o4 / gpt-5 targets (when both keys are present in one source, the modern key wins).
• A caller-supplied stream_options is stripped when the hop lands on a provider known to reject it. On your configured primary it always reaches the provider untouched.
• Endpoint-scoped extra_headers / extra_query / extra_body are stripped on cross-provider hops — they carry gateway credentials authored for the original endpoint — and the target entry's own default_params versions re-apply. They are untouched on the primary and on same-provider model swaps.

Cross-dialect hops (canonical translation)

When the dialects differ — e.g. OpenAI → Anthropic, or Bedrock → Anthropic — Solwyn translates the request from the source dialect into a canonical subset, then into the target's dialect, before the hop. The response is reshaped to the caller's dialect: a Bedrock caller served by Anthropic still gets a Converse-shaped dict, and an OpenAI caller served by Bedrock gets OpenAI-shaped objects and chunks.

Translation is a structural operation only — it maps request shape (messages, system prompt, tool definitions, common parameters). It never inspects, logs, or rewrites prompt or response content. See Privacy.

Amazon Bedrock participates in both directions via translation — e.g. Bedrock-hosted Claude failing over to direct Anthropic, or an OpenAI primary failing over to Bedrock. Bedrock-specific hop rules (native Bedrock primary calls pass kwargs straight through to boto3, untouched):

• inferenceConfig.maxTokens is required for a cross-provider hop from Bedrock — the SDK will not invent an output bound and fails loud with the structural label missing_max_tokens.
• guardrailConfig fails loud on a hop (bedrock.guardrail_config) — a safety feature a hop would otherwise silently strip — as do cachePoint blocks, s3Location image sources, and other shapes outside the canonical subset. See Amazon Bedrock.
• boto3 has no per-call timeout override, so the chain's failover_total_timeout cannot shorten an in-flight Bedrock hop. Bound it on the client with botocore.config.Config(retries={"total_max_attempts": 1}, read_timeout=60) so Solwyn owns retries and failover instead of stacking botocore's legacy retry layer. See Timeouts and retries.

Distinct endpoints need distinct provider identities

Circuit-breaker health, latency signals, and failover labeling key off the provider name. Two chain entries that resolve to the same name — two Azure resources, or two unnamed gateways both detected as openai_compatible — share one health domain, are reported as model fallbacks of each other, and skip cross-provider request sanitization (the stream_options strip, the max_completion_tokens rewrite, the endpoint-scoped param strip). A header authored for the first endpoint then reaches the second untouched and can fail there. Give distinct endpoints distinct identities via provider= or the 4th fallback-tuple element.

When a request cannot be translated

On a cross-dialect hop, some request shapes have no equivalent on the target provider. Rather than silently dropping a field, Solwyn fails loud before any network call and aborts the whole chain (tool-using streams across dialects also fail loud, pre-dispatch):

Both carry structural labels only — source, target, and a feature token like "response_format" — never the offending value and never prompt content.

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import Solwyn, UntranslatableRequestError

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024})],
)

try:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )
except UntranslatableRequestError as e:
    print(f"Cannot translate {e.feature} from {e.source} to {e.target}")

client.close()

Aborting the chain on an untranslatable feature is deliberate: a translated request that quietly differs from what you asked for is worse than a clear failure. Keep your fallback chain to providers that can serve the same request shape, or scope the untranslatable feature out of calls that may fail over.

Selection policies

A selection policy decides the order in which healthy candidates are attempted. A policy is pure and side-effect-free; it only reorders the usable set, dropping any provider whose circuit is open and not recovery-eligible. Pass one with selection_policy=:

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import Solwyn, LatencyPolicy

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024})],
    selection_policy=LatencyPolicy(),
)

Both apply the same health filter first, so a policy can never promote an unhealthy provider ahead of a healthy one, nor attempt a provider whose circuit is open.

Note: LatencyPolicy orders by observed latency, so it needs samples before it can act. Solwyn records the latency of each successful call in a rolling window and reports a provider's p50 only after at least 3 successes. Until a provider reaches that threshold it sorts after providers with a known p50, so the chain behaves like HealthBasedPolicy while latencies warm up.

Failover tuning

Note: These failover knobs are constructor-only — they have no SOLWYN_* environment variable. Only the fields listed in SolwynConfig can be set from the environment.

Plan-scoped tuning entitlement

New in 0.3.0. Custom failover and breaker tuning is an entitlement — it is available on plans that carry the failover-tuning entitlement. Each pre-flight budget check opts into a versioned failover directive, and Solwyn Cloud may answer with one. The directive governs a closed set of seven fields — the four failover knobs above and the three circuit-breaker parameters:

No other configuration field is touched. The directive has three states:

The transition is reversible in both directions — it is not a one-way ratchet. A false directive replaces your values with the defaults above; a later true restores your original constructor values.

Changes are applied in place to the breakers already running:

• Provider order never changes. The directive tunes timing and thresholds only — it cannot reorder, add, or remove chain entries.
• Breakers keep their identity and their accumulated health and probe state. A CLOSED or HALF_OPEN breaker is unaffected mid-episode; an OPEN breaker re-samples its recovery window from the new timeout and jitter.
• The chain deadline preserves the original call start. A directive-driven failover_total_timeout change replaces the total but never restarts the clock, so the remaining window can shrink below the time already elapsed. This holds across the sync and async chat and media paths alike.

When a false directive suppresses tuning you actually asked for, the SDK logs one WARNING — Custom failover tuning is unavailable for this plan; SDK defaults applied — at most once per client instance, and only when your requested tuning genuinely differs from the defaults. Clients running default configuration see nothing. See Logging.

Idempotency and duplicate-call safety

A failure after the request reached the provider (a read timeout, a dropped connection mid-response) is ambiguous: the call may already have run and been billed. Failing over would risk a duplicate provider charge. failover_idempotency controls that trade-off:

Override per call with the solwyn_idempotent keyword (stripped before the request reaches the provider):

# This specific call is safe to retry anywhere — fail over aggressively.
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    solwyn_idempotent=True,   # -> "always" for this call
)

solwyn_idempotent=True maps to "always", False maps to "safe"; omit it to use the configured failover_idempotency.

Circuit breaker

The circuit breaker tracks the health of each provider in the chain independently. Enforcement is process-local — no breaker state is shared between SDK instances, and no snapshot ever drives admission. From SDK v0.3.0+ that state is also reported to Solwyn Cloud for dashboard visibility; see Breaker state reporting. When a provider's circuit is open, the router skips it and moves to the next candidate without attempting a call.

CLOSED  --[failure_threshold failures]--> OPEN
OPEN    --[recovery_timeout elapsed]----> HALF_OPEN
HALF_OPEN --[success_threshold OKs]----> CLOSED
HALF_OPEN --[any failure]--------------> OPEN

State transitions

1. CLOSED to OPEN: After failure_threshold consecutive failures (default: 3), the circuit opens. The router stops sending to this provider until the recovery timeout elapses.
2. OPEN to HALF_OPEN: After recovery_timeout seconds (default: 60, with jitter), the circuit allows probe requests.
3. HALF_OPEN to CLOSED: After success_threshold consecutive successes (default: 2), the circuit closes.
4. HALF_OPEN to OPEN: Any failure during probing immediately re-opens the circuit.

Tuning the circuit breaker

These three can also be set via SOLWYN_CIRCUIT_BREAKER_FAILURE_THRESHOLD, SOLWYN_CIRCUIT_BREAKER_RECOVERY_TIMEOUT, and SOLWYN_CIRCUIT_BREAKER_SUCCESS_THRESHOLD.

All three are part of the seven-field closed set a plan-scoped failover directive can govern (SDK v0.3.0+) — on a plan without the failover-tuning entitlement they revert to the defaults above.

Note: A same-provider model fallback that fails on both the primary model and the fallback model records only one failure against that provider's breaker — breaker failures are deduplicated per provider within a single request, so one failing request can never burn more than one count toward failure_threshold per provider.

Breaker state reporting

New in 0.3.0. The SDK periodically snapshots each provider's breaker state and reports it to Solwyn Cloud, giving the dashboard fleet-wide provider-health visibility. Each report carries structural labels and quantities only — never content:

Reporting is advisory and one-way. Cloud snapshots never drive admission, and breaker state is never shared between instances — every failover decision stays 100% in-process. Turning reporting off changes what the dashboard sees, never how your chain behaves.

Beyond the dashboard, these snapshots drive circuit_breaker_opened and circuit_breaker_closed webhook events and notifications on paid plans, per project, provider, and SDK instance — so a provider tripping in your fleet can page you rather than wait to be noticed. Disabling reporting, or never establishing a project id, means no events fire.

Behavior notes:

• Never blocks a call. Snapshots ride the background reporter on its own worker, off the request path. Every send failure is dropped with a warning — see Logging.
• Needs a project first. Reports are sent only after a successful budget check has established the project id. Until that happens, there are silently no reports.
• One instance id per client. The id is a fresh UUID for each client instance, so client-per-request patterns fragment fleet snapshots into single-use identities. Construct the client once and reuse it.

Off switch — the only new configuration field in 0.3.0:

import os
from openai import OpenAI
from solwyn import Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    breaker_reporting_enabled=False,   # or SOLWYN_BREAKER_REPORTING_ENABLED=false
)

ProviderUnavailableError

When every candidate in the chain is unavailable — all circuits open, or the chain is exhausted without a success — the SDK raises ProviderUnavailableError:

import os
from anthropic import Anthropic
from openai import OpenAI
from solwyn import ProviderUnavailableError, Solwyn

client = Solwyn(
    OpenAI(),
    model="gpt-4o",
    api_key=os.environ["SOLWYN_API_KEY"],
    fallback=[(Anthropic(), "claude-sonnet-4-5", {"max_tokens": 1024})],
)

try:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )
except ProviderUnavailableError as e:
    print(f"Chain exhausted. Tried: {e.attempted}")
    print(f"Provider: {e.provider}, circuit: {e.circuit_state}")

client.close()

Attributes

What the dashboard sees

Every call reports a metadata event tagged with how the chain resolved, so the dashboard can show how often failover saves a request:

These fields carry structural labels only — never prompts, responses, or error strings. See Privacy.

Related

• SolwynConfig — every failover field and its precedence
• Error Handling — UntranslatableRequestError, UntranslatableModelError, ProviderUnavailableError
• Logging — look for Circuit breaker opened to confirm the circuit tripped
• Troubleshooting — failover that fires more (or less) than expected