Run control

import os
import solwyn
from openai import OpenAI
from solwyn import BudgetExceededError, RunStoppedError

client = solwyn.Solwyn(OpenAI(), api_key=os.environ["SOLWYN_API_KEY"])

with solwyn.run("nightly-report") as run_id:
    for step in plan:
        try:
            client.chat.completions.create(model="gpt-4o", messages=[...])
        except RunStoppedError as stopped:
            # Terminal for this run. Do not retry.
            print(f"run {stopped.agent_run_id} stopped ({stopped.source}: {stopped.reason})")
            break
        except BudgetExceededError:
            # Over a cap. Back off, or wait for the next period.
            break

client.close()

New in 0.6.0. A run can be stopped from either side of the wire. Solwyn Cloud pushes a stop when an operator presses Stop on a run in the dashboard's Agents tab and confirms in the Stop this run? dialog; the SDK stops a run on its own when a content-free velocity rule you have enabled for denial fires. Both surface the same way in your code — RunStoppedError before the next provider call — and both leave a content-free denial receipt behind so the dashboard can show what was refused.

A stop is not a budget denial. It is a decision that this run must not make another call, and it stays in force through control-plane outages, past alert_only, and past fail_open.

RunStoppedError is not a BudgetExceededError

RunStoppedError inherits directly from SolwynError, not from BudgetExceededError. That is deliberate: an agent loop's except BudgetExceededError handler — the one that backs off and retries next period — cannot accidentally swallow an explicit stop and keep the run alive.

The message is Agent run <id> was stopped (<source>: <reason>). None of the three attributes is derived from prompt or response content.

Handle the two exceptions separately, and treat RunStoppedError as terminal for that run id. See Exceptions.

Where a stop is enforced

Every intercepted call inside a solwyn.run(...) scope — chat, Responses, and every media surface, sync and async — is gated on the run's stop state: before the budget check when the SDK's velocity detector stopped the run, and after the check for an operator stop, where a live check is still issued and comes back as a run_stopped denial. A stopped run never reaches the provider again, and a reservation taken for the call is released before the error is raised.

Two boundaries are worth knowing precisely:

• A non-streaming request already in flight is not preempted. The stop takes effect at the run's next dispatch. Spend already in flight settles normally.
• A stream is stopped at its next raw provider-chunk boundary. The next chunk is pulled from the provider and discarded — never observed, never yielded — the usage seen so far is settled exactly once as a partial success, the provider's stream is closed, and RunStoppedError is raised from the iterator. Every later next() re-raises it; the stop is terminal for that stream. This holds for every provider's chat stream, the OpenAI Responses stream helper, Google's generate_content_stream, and Bedrock's converse_stream (where the boundary is the raw Converse event inside result["stream"], so a stopped Bedrock stream settles without ever seeing its terminal metadata event).

No with block is needed for a stop to settle a stream — the abort settles it. The with form still matters for your own early exits; see Streaming.

Stops from the dashboard

Operators stop a run from the project's Agents tab (or from a runaway-run alert in the Alerts feed). Solwyn Cloud then denies every later budget check, lease grant, and lease renewal that names that run, and attaches a versioned run_control directive the SDK opted into on every one of those requests. The SDK records the stop, raises RunStoppedError, and refuses lease authority for the run from then on — an in-flight grant or renewal that lands on a stopped run is surrendered rather than installed. A run holding a live lease is reached through its next renewal, so a stop is never delayed by a full lease.

Two properties follow from the directive being authoritative rather than advisory:

• alert_only does not soften it. A stop is normalized to a hard denial even on an alert-only project, and even if a drifted response body says allowed: true while carrying a terminate directive.
• It survives a control-plane outage. A stop the SDK has seen is preserved for that run id while Solwyn is unreachable — regardless of fail_open, and ahead of every stage of the lease outage ladder. Each such call raises with deny_source="sticky_replay" and logs Cloud API unreachable; preserving prior hard deny: $X/$Y used, the retained denial the directive rode in on; Cloud API unreachable; preserving retained run stop is logged instead when only the local stop registry still holds the run — after a local velocity stop, or once the retained denial has been evicted.

What a stop cannot do is reach an agent that never contacts Solwyn: enforcement happens at the run's next budget check or lease renewal, so a run that cannot reach the control plane at all keeps its current posture until connectivity returns. The dashboard states the expected bound when you confirm a stop — see Agent runs in the dashboard.

Only the run you stopped is affected. A stop is remembered per run id, so an outage cannot replay a dashboard stop against unrelated runs, and it does not create project-wide denial authority the way a project-period hard deny does. A stop cannot be undone for that run id.

Contract drift is handled on one call, not fleet-wide

If Solwyn ever echoed a terminate directive for the wrong run id, the SDK treats it as server contract drift rather than obeying it: that one call degrades to the unreachable posture, the shared control-plane breaker is credited a success (three drifted bodies must not open the breaker fleet-wide), and a distinct ERROR names the channel — budget.check_directive_misrouted, lease.grant_directive_misrouted, or lease.renew_directive_misrouted. Nothing is marked stopped. See Logging.

Local velocity detection

The SDK also watches each run's shape — sizes, timestamps, and model identifiers, never content — for the signatures of a loop that has stopped making progress. Three rules run over the last 64 observations per run:

velocity_mode decides what a firing rule does:

client = solwyn.Solwyn(
    OpenAI(),
    api_key=os.environ["SOLWYN_API_KEY"],
    velocity_mode="deny",          # or SOLWYN_VELOCITY_MODE=deny
    velocity_repeat_count=8,       # tolerate a few more near-identical calls
)

The seven settings — velocity_mode, velocity_repeat_count, velocity_repeat_window_s, velocity_growth_streak, velocity_growth_factor, velocity_accel_floor_per_min, and velocity_accel_factor — each read a SOLWYN_VELOCITY_* environment variable; bounds and defaults are in SolwynConfig. Count-style thresholds cap at 64, the retained window, so every accepted configuration is achievable.

Three things the detector is careful about:

• It is per run. Calls outside a solwyn.run(...) scope are never observed. History is kept for the 128 most recently active runs, 64 observations each, in fixed memory.
• It is content-free. Each observation is a monotonic timestamp, an estimated input-token count, and a model identifier. The module performs no I/O and no logging of its own.
• It never invents a signal. When identity churn evicts history inside the rate window, rate_acceleration — the one rule whose verdict depends on a complete count — is suppressed for the affected run rather than evaluated over incomplete data; if the eviction bookkeeping itself overflows, that suppression applies process-wide for one window. Losing an identifier can silence a warning but can never produce a false one.

Reading and clearing stop state

Cooperative run code can consult the SDK's stop registry directly. All three are exported from the package root:

import solwyn

with solwyn.run("crawler") as run_id:
    for page in pages:
        if solwyn.current_run_terminated():
            break                      # exit cleanly instead of raising on the next call
        client.chat.completions.create(...)

    termination = solwyn.run_termination(run_id)
    if termination is not None:
        log.info("stopped: %s via %s", termination.reason, termination.source)

The registry keeps exact answers for the 256 most recently touched run ids. It is keyed on the raw run id and never guesses from fingerprints, so identity churn cannot false-stop an unrelated or new run. The trade-off is that a stopped run with no active stream can be forgotten after eviction if the control plane does not reaffirm the stop — a fixed-memory choice that preserves exact answers. An active stream keeps its own immutable copy of the first stop it saw, independent of eviction, until it settles.

There is no callback or hook for stops: the surfaces are the exception, the three functions above, and the receipts described next.

Denial receipts

Every refused call — a stop, a budget cap, a lease exhausted, local fail-closed enforcement — still produces a metadata event, with status="budget_denied" and a handful of optional, content-free attribution fields:

The receipt is reported before the exception is raised; a failure to report it is logged and never changes the outcome. On the dashboard, receipts drive each run's Lifetime denied cost, the Flagged and Stopped badges on the Agents tab, and the blocked-call savings on the Costs tab. If a receipt cannot be delivered, it is folded and replayed as an aggregate rather than lost — see Spend delivery.

Testing a stop

solwyn.testing.FakeControlPlane reproduces both stop sources without a network: plane.stop_run(run_id) is the dashboard button, the solwyn-test/kill magic model scripts the same kill from a model name, solwyn-test/deny-stopped returns the denial without the directive, and velocity_mode="deny" on a wrapped client exercises the local path. See Testing budget enforcement.

What leaves your process

A stop adds no new content to the wire. Outbound, every budget check, lease grant, and lease renewal opts into the run_control directive with one version field; inbound, the directive carries a run id and a bounded reason. The velocity detector's state never leaves your process at all. The receipt fields above are structural labels and integer counts. See Privacy.

Related

• Agent Runs -- the solwyn.run(...) scope a stop is scoped to
• Budget Enforcement -- caps, leases, and the outage ladder a stop sits ahead of
• Agent runs in the dashboard -- the Stop button, the Flagged and Stopped badges, and the runaway-run alert rule
• Exceptions -- RunStoppedError and RunTermination
• SolwynConfig -- the seven velocity_* settings
• Logging -- velocity.flagged and the directive-drift messages