TypeScript SDK Streaming

The TypeScript SDK preserves each provider's async-iterable streaming contract. Chunks remain on the direct provider connection; the wrapper observes usage metadata, settles the call once, and forwards caller-visible chunks in the requested dialect.

OpenAI Chat Completions

Pass stream: true as you normally would and consume the result with for await:

import { Solwyn } from "@solwyn/sdk";
import OpenAI from "openai";

const solwynApiKey = process.env.SOLWYN_API_KEY;
if (!solwynApiKey) throw new Error("SOLWYN_API_KEY is required");

const client = new Solwyn(new OpenAI(), { apiKey: solwynApiKey });

try {
  const stream = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Write a two-line poem about latency." }],
    stream: true,
  });

  for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  }
} finally {
  await client.close();
}

For native OpenAI chat, the SDK requests stream_options.include_usage: true on the copied request so it can settle from the terminal usage chunk. Caller request objects are not mutated.

Where terminal usage comes from

Compatible endpoints receive include_usage only when the detected provider profile documents it as safe. If a compatible stream ends without usable usage, Solwyn never silently records zero: it uses the local length estimate, marks token_details.is_estimated: true, and warns once for that provider.

Additional Anthropic streaming cache writes are attributed to the served request's cache TTL when it agrees with the initial usage split. Mixed-TTL requests mark the inferred cache-write growth as estimated.

Provider-reported usage remains settlement truth whenever it is available. See providers for supported client packages and compatible-host profiles.

Settlement happens exactly once

The wrapper settles at terminal disposition, not between chunks:

• Full iteration settles when the provider's terminal usage event or stream end is observed.
• Breaking out of a for await loop invokes the iterator's return() and settles a partial success with usage metadata observed before the break. That can be zero when a provider emits usage only in a terminal event that the caller did not consume.
• An exception thrown into the iterator with throw() settles the same way as an early return().
• A mid-stream provider error on a stream that used a per-call budget check settles the provider-reported usage observed before the error, and the error event carries those tokens. Lease-funded streams, the default for untagged run-scoped text calls, still report zero usage and release the local lease claim. A deferred responses.stream helper reports a zero-token error event with no confirmation.
• client.close() settles each open stream once from the usage observed so far; later pulls reject with SolwynError("client is closed"). Finish or cancel streams before closing to include terminal usage.
• A stream abandoned without cleanup is settled from observed usage when it is garbage-collected. Finalization timing is runtime-dependent, so explicit cleanup is the deterministic path.
• Repeated iterator cleanup does not settle the same stream twice.

Settlement is queued through the reporter and does not add a Cloud round trip between streamed chunks. Await client.close() during application shutdown so pending confirmations and events get a bounded final flush.

If you manually call next() rather than using for await, finish the iterator with return() when you stop early. Ordinary for await control flow, including break and an exception thrown from the loop body, performs that iterator cleanup automatically.

Native provider shapes remain native

Anthropic and Google express streaming with their own APIs:

import { Solwyn } from "@solwyn/sdk";
import Anthropic from "@anthropic-ai/sdk";

const solwynApiKey = process.env.SOLWYN_API_KEY;
if (!solwynApiKey) throw new Error("SOLWYN_API_KEY is required");

const client = new Solwyn(new Anthropic(), { apiKey: solwynApiKey });

try {
  const stream = await client.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 256,
    messages: [{ role: "user", content: "Explain terminal usage in one paragraph." }],
    stream: true,
  });

  for await (const event of stream) {
    if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
      process.stdout.write(event.delta.text);
    }
  }
} finally {
  await client.close();
}

import { GoogleGenAI } from "@google/genai";
import { Solwyn } from "@solwyn/sdk";

const solwynApiKey = process.env.SOLWYN_API_KEY;
if (!solwynApiKey) throw new Error("SOLWYN_API_KEY is required");

const google = new GoogleGenAI({ apiKey: process.env.GOOGLE_API_KEY });
const client = new Solwyn(google, { apiKey: solwynApiKey });

try {
  const stream = await client.models.generateContentStream({
    model: "gemini-2.0-flash",
    contents: "Explain terminal usage in one paragraph.",
  });

  for await (const chunk of stream) {
    process.stdout.write(chunk.text ?? "");
  }
} finally {
  await client.close();
}

On @google/genai versions with automatic continuation, such as 2.30.0, Solwyn checks the logical call once and, on success, settles the cumulative usage across all continuation hops. failoverHopReadTimeout bounds the whole continuation sequence. If a later streaming hop fails, the per-call and lease-funded error rules under settlement apply. Set config.automaticContinuation: false when your application needs separate admission and settlement for each hop.

Bedrock preserves the AWS SDK v3 response object; the wrapped async iterable remains at response.stream:

import { BedrockRuntimeClient, ConverseStreamCommand } from "@aws-sdk/client-bedrock-runtime";
import { Solwyn } from "@solwyn/sdk";

const solwynApiKey = process.env.SOLWYN_API_KEY;
if (!solwynApiKey) throw new Error("SOLWYN_API_KEY is required");

const bedrock = new BedrockRuntimeClient({ region: "us-east-1" });
const client = new Solwyn(bedrock, { apiKey: solwynApiKey });

try {
  const response = await client.send(
    new ConverseStreamCommand({
      modelId: "amazon.nova-pro-v1:0",
      messages: [{ role: "user", content: [{ text: "Write a latency haiku." }] }],
      inferenceConfig: { maxTokens: 128 },
    }),
  );

  for await (const event of response.stream ?? []) {
    if (event.contentBlockDelta?.delta?.text) {
      process.stdout.write(event.contentBlockDelta.delta.text);
    }
  }
} finally {
  await client.close();
}

Raw Bedrock InvokeModelWithResponseStreamCommand and InvokeModelWithBidirectionalStreamCommand are not metered and fail before dispatch with ConfigurationError; use ConverseStream for tracked streaming.

OpenAI Responses streaming is deferred

On native OpenAI and Azure clients, responses.stream() returns a deferred helper synchronously:

import { Solwyn } from "@solwyn/sdk";
import OpenAI from "openai";

const solwynApiKey = process.env.SOLWYN_API_KEY;
if (!solwynApiKey) throw new Error("SOLWYN_API_KEY is required");

const client = new Solwyn(new OpenAI(), { apiKey: solwynApiKey });

try {
  const stream = client.responses.stream({
    model: "gpt-4o",
    input: "Explain terminal usage in one paragraph.",
  });

  for await (const event of stream) {
    if (event.type === "response.output_text.delta") {
      process.stdout.write(event.delta);
    }
  }
} finally {
  await client.close();
}

The budget check and provider request begin only when iteration or an activating helper first needs the stream. finalResponse(), done(), on(), once(), and emitted() activate it; off() does not. Calling abort() before activation dispatches neither the budget check nor the provider request.

Responses streaming is primary-only and settles from Responses usage, not Chat Completions stream_options. Streaming responses.parse(...) is rejected with ConfigurationError because that shape cannot be metered consistently.

A stopped run aborts at the next provider-chunk boundary

A stream created inside a Node run cooperatively observes exact-ID run termination. After a server stop or a deny-eligible local velocity stop:

1. The next call into the raw provider iterator pulls one provider chunk.
2. The SDK records any usage carried by that chunk, detects the stop, discards the chunk before exposing or translating it, and closes the provider stream.
3. It settles exactly once as a partial success using the usage observed through that chunk.
4. Iteration throws RunStoppedError with agentRunId, reason, and source.

The stream is terminal after that point. Later iteration rethrows the same RunStoppedError. clearRunTermination(runId) is forward-looking and cannot revive a stream that has already latched the stop; start a new stream after clearing.

The boundary is deliberately between chunks, never in the middle of one. See runs for stop and velocity controls and typed errors for handling RunStoppedError separately from budget denials.

Streaming with failover

Same-dialect fallback streams preserve the requested native shape. Cross-dialect fallback streams translate a small text/finish-reason subset back into the caller's dialect while accumulating usage from the raw provider stream that actually served the call. Cross-dialect tool streaming fails before provider dispatch with UntranslatableRequestError.

See failover for the idempotency and translation boundary and budgets for reservation and lease behavior.