TypeScript SDK Failover

The TypeScript SDK can route a text call through an ordered provider/model chain when the requested target is unhealthy or fails in a way that is safe to retry. The routing happens in your process; provider requests still go directly to the selected provider.

Configure a fallback chain

Each fallback entry is a positional tuple: [client, model], [client, model, defaultParams], or [client, model, defaultParams, providerOverride].

import { Solwyn } from "@solwyn/sdk";
import Anthropic from "@anthropic-ai/sdk";
import OpenAI from "openai";

const solwynApiKey = process.env.SOLWYN_API_KEY;
if (!solwynApiKey) throw new Error("SOLWYN_API_KEY is required");

const groq = new OpenAI({
  baseURL: "https://api.groq.com/openai/v1",
  apiKey: process.env.GROQ_API_KEY,
});

const client = new Solwyn(new OpenAI(), {
  apiKey: solwynApiKey,
  model: "gpt-4o",
  fallback: [
    [groq, "llama-3.3-70b-versatile"],
    [new Anthropic(), "claude-sonnet-4-20250514", { max_tokens: 1024 }],
  ],
});

try {
  const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Explain graceful degradation briefly." }],
  });
  console.log(response.choices[0]?.message.content);
} finally {
  await client.close();
}

The primary call uses its per-call model when present. A fallback always uses the model configured on its own tuple. Request defaults merge in this order: global defaultParams, fallback-entry defaults, then per-call values.

The fallback chain applies to tracked chat/text dispatch. Native OpenAI and Azure Responses calls are primary-only, as are embeddings and media surfaces. See providers for the exact intercepted methods.

Provider name and dialect are different

Every adapter has two identities:

• Name identifies attribution, budgets, health, and breaker state—for example groq, openrouter, openai, or anthropic.
• Dialect identifies the request and response wire shape—for example OpenAI Chat Completions, Anthropic Messages, Google Gemini, or Bedrock Converse.

Two providers can have different names but the same dialect. Groq to OpenRouter is therefore a same-dialect hop: the SDK preserves the native request shape, swaps the model, removes source-endpoint transport fields, and applies the target entry's endpoint defaults. A same-provider model fallback is native passthrough as well.

Circuit breakers and routing signals key by provider name. If two endpoints need independent health state, give them distinct provider identities with provider on the primary or the fourth fallback tuple element.

Cross-dialect translation is intentionally small

A hop between different dialects passes through the SDK's minimal canonical translation subset. It supports the common chat/message shape and a bounded set of generation controls; it is not a general compatibility layer for every provider feature.

Unsupported request features fail before the target provider is called:

• UntranslatableRequestError means the target dialect cannot represent the request. Its source, target, and feature fields are structural labels only.
• UntranslatableModelError means the fallback has no usable target model. Its fields are model and provider.

Both errors abort the entire chain. They are never converted to a retry and never include prompt or response content. Cross-dialect streaming with tool declarations is refused before opening the foreign stream because tool-call deltas are outside the streaming translation subset.

Bedrock-to-Bedrock failover is supported. A cross-dialect hop into Bedrock fails with UntranslatableRequestError because the generic translation path cannot construct the target client's ConverseCommand; use another Bedrock client as the fallback for a Bedrock primary.

See typed errors for fields and catch patterns, and privacy for the translation boundary.

Idempotency controls cross-provider retries

failoverIdempotency determines whether the router may cross provider names after an attempt fails:

Request-shaped failures fail fast and preserve the provider SDK's original error. Transport and provider exceptions that escape the chain also remain the original values; Solwyn does not wrap them just because failover was configured.

sameProviderRetries is separate. It defaults to 0 and, when positive, can retry the same target after an HTTP 429 with a parseable Retry-After value that still fits inside the failover window. An intermediate retry is not a new fallback hop. Aborting the caller's signal during a Retry-After sleep rejects immediately and does not fail over.

A per-call solwynIdempotent request field can only tighten the configured policy: false limits that call to "safe" while preserving a configured "never", and true keeps the configured policy. Solwyn strips the field from every intercepted provider request, including media and Bedrock command input, without mutating the caller's object.

Time bounds

The total window does not interrupt a provider request already in flight. Approximate worst-case wall time until a response starts is therefore the failover window plus one provider request bound. A total timeout of 0 is valid and prevents new work after call entry; failoverHopReadTimeout must be a finite positive number.

JavaScript provider SDKs expose different timeout carriers. OpenAI, compatible clients, and Anthropic receive their request timeout, which covers only connecting and receiving response headers: a response body or stream that stalls after its first bytes can outlast it. Pass an AbortSignal in the request options, or apply your own idle timeout while consuming a stream, to bound that phase. Google receives config.httpOptions.timeout; on a @google/genai version whose request-signal support Solwyn has verified (2.20.x and 2.30.x), Solwyn disables that native timeout and arms its own AbortSignal timer, which stays armed until the buffered response or the stream finishes. Bedrock receives requestTimeout. Bedrock's carrier is not a universal hard abort for every Converse stream handler, so configure a finite socket timeout on its AWS request handler as well.

Solwyn Cloud can return whether custom tuning is available for the project. If it is unavailable, the SDK applies the defaults for the eight failover and breaker fields while preserving a stricter requested idempotency policy, a lower sameProviderRetries, and a smaller failoverTotalTimeout, and warns once; the fallback chain itself remains configured.

Provider circuit breakers

Each provider name has an in-process circuit breaker:

The defaults are:

Provably retryable failures and ambiguous post-send failures count as health signals. Request-shaped fail-fast errors do not. On an established stream, non-429 HTTP 4xx errors are health-neutral, and errors without an HTTP status count as provider failures unless the caller cancelled. Mid-stream errors never trigger failover or record a latency sample. A half-open streaming probe is held for at most circuitBreakerRecoveryTimeout seconds after setup, with a one-second minimum, so an unread stream cannot block recovery indefinitely. A provider counts at most one breaker failure per logical call, even when the chain contains multiple models for that provider.

If no candidate can be dispatched because breakers refuse admission or the chain window has expired, the SDK throws ProviderUnavailableError and exposes an attempted provider-name list when available. If one or more provider attempts actually run and all fail, the final provider's original exception is re-thrown instead.

Breaker decisions stay local. Content-free breaker snapshots can be reported asynchronously to Solwyn Cloud for fleet visibility; breakerReportingEnabled: false disables that advisory channel.

Selection policies

The default HealthBasedPolicy orders usable candidates by health tier while preserving configured order inside a tier. Two exported alternatives use local or Cloud-supplied signals inside the same health ordering:

import { LatencyPolicy, Solwyn } from "@solwyn/sdk";
import OpenAI from "openai";

const solwynApiKey = process.env.SOLWYN_API_KEY;
if (!solwynApiKey) throw new Error("SOLWYN_API_KEY is required");

const openRouter = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
});

const client = new Solwyn(new OpenAI(), {
  apiKey: solwynApiKey,
  fallback: [[openRouter, "openai/gpt-4o"]],
  selectionPolicy: new LatencyPolicy(),
});

await client.close();

• LatencyPolicy prefers lower observed p50 latency inside a health tier after enough local samples.
• CostPolicy compares request-scoped relative price hints supplied by Cloud. The SDK does not calculate provider costs or combine hints with token counts.

Lease-backed run calls currently carry no price hints, so CostPolicy falls back to health ordering for those calls. See budgets for run-lease eligibility and streaming for failover stream settlement.