Rensei docs

Routing Durability & On-Prem Posture

The four decision-class failure postures (fail-closed, fail-static, conservative-cap, fail-open-deterministic), the signed ruleset snapshot that makes fail-static possible, and what a control-plane outage actually does to admitted work.

An evaluator asking "what happens when the control plane goes down" is really asking four separate questions - the platform does not answer all of them the same way, and it should not. This page names the four decision classes that make up a placement or selection decision, the posture each one commits to, and the mechanism (a signed, cached ruleset snapshot) that makes the non-authorization postures possible without widening what is permitted.

Four decision classes, four postures

Every placement or selection decision asks some mix of four questions, and each question fails a different way when its input cannot be freshly read:

ClassQuestionPostureOn a read failure
AuthorizationMay this run there?fail_closedDenies - exactly like an explicit forbid. Never widens.
PlacementWhere should this land?fail_staticServes the last known-good value, bounded and flagged. Never silently nulls out.
BudgetIs this within limits?conservative_capThe authoritative counter fails closed; only its own live-delivery signal (not the counter itself) falls back to a separate, bounded, conservative local cap.
Bandit explorationWhich arm should be sampled?fail_open_deterministicFalls back to the deterministic, non-learned choice. Dispatch is never blocked by a sampler that cannot run.

A decision class is not the same thing as one of the six composition-law stages: a stage is where in the law a decision happens, a class is what kind of fact it decides. The permission stage's Cedar check and the interactive lane's degraded-mode admission check are both authorization-shaped, even though they sit at different points in the pipeline; a resolver's viability and preference sub-stages are both placement-shaped, not budget-shaped, even though a capacity ceiling is itself budget-shaped.

Only one class is intentionally soft. Bandit exploration is the sole class where falling back to a deterministic choice on failure is the correct behavior - a stochastic ranker that cannot sample is not a security or budget question, and blocking every dispatch because Thompson sampling could not run would turn routing intelligence into a new single point of failure for dispatch itself. The other three classes each commit to either denying or serving a bounded, visibly-flagged stale value - never a silent widen.

Authorization: fail-closed

A Cedar policy evaluation that throws - an unreachable policy source, a malformed bundle, a missing snapshot section - is treated as an explicit deny, the same as a named forbid rule. There is exactly one sanctioned way to get an allow with zero authored policy: a workspace's declared default-permit fallback, which is a deliberate, named steady state, not a swallowed error. A read failure never becomes a permit.

Placement: fail-static

Viability, preference, and ranking reads - the ruleset snapshot itself, and any resolver that consults it - serve the last known-good value on a refresh failure instead of erroring or returning null. Staleness is bounded and exposed, never hidden: every decision record that resolved through a snapshot-backed read carries rulesetRev, snapshotAgeMs, and degraded. A resolver that still reads Postgres directly rather than through a snapshot - true of most placement resolvers today, with the permission stage first to cut over - stamps degraded: false with rulesetRev: null. That is documented live-database semantics ("this read the freshest possible truth"), not an unset or ambiguous field; the two shapes are distinguishable on every decision record, never conflated.

Budget: conservative-cap

The org-level inflight and period counters that are the actual source of truth for a spend or concurrency cap fail closed on a read failure - a launch is rejected outright, because nothing else could have admitted a session without going through that counter. The softer posture only applies one layer down: when the signal that would normally bound live delivery (not the authoritative counter) becomes unavailable, a separate, bounded, per-project local cap takes over as a circuit breaker for admission - never an unbounded fallback, and never a substitute for the authoritative counter itself.

Bandit exploration: fail-open-deterministic

A stochastic exploration policy that cannot sample - the posterior store is down, a prior does not exist yet, or exploration is disabled outright - falls back to the deterministic static or top-capability-match choice a non-learning dispatch would have made. This is the platform's existing, correct behavior for both bandits in the provider-selection and agent-card-selection paths; nothing about it changes here.

The ruleset snapshot: the durability unit

Fail-static placement needs something concrete to fail static to. That something is the ruleset snapshot: a versioned, signed bundle covering five sections - the active policy bundle, capacity profiles and their grants, the pool and host inventory, the execution-cell matrix, and a posterior summary - compiled from the org's live data and persisted as one row per revision.

  • Versioned. Every compile allocates the next monotonic revision for the org; the identifier that shows up on a decision record and in the CLI is {orgId}@{revision} (for example org_a1b2c3@42).
  • Ed25519-signed. The bundle's content hash is computed deterministically (every section is sorted by a stable key, so two compiles over identical data agree byte-for-byte) and signed with the org's audit-signing key - the same signing infrastructure the audit chain uses, not a parallel one. A consumer recomputes the hash and verifies the signature against the recorded signing key before trusting anything in the bundle.
  • Validator-gated. A compile that fails validation never publishes: the distribution endpoint returns a 503 naming the failing validator rather than serving a bundle nobody checked.
  • Distributed on demand and on mutation. GET /api/daemon/ruleset-snapshot serves an org's current snapshot to any authenticated worker, compiling one on the fly for an org that has never had one rather than 404ing. A capacity-profile, grant, pool, or policy write invalidates the in-process cache and kicks a background recompile; nothing has to wait for a periodic tick to see its own change reflected, though nothing blocks on that recompile either - the previous cached value keeps serving until the new one lands. The response carries the same staleness signal as headers (X-Ruleset-Rev, Age, X-Ruleset-Degraded, X-Ruleset-Compiled-At) that the decision record fields mirror.

On the daemon side, the same bundle is verified again independently - a daemon does not simply trust the platform's own claim that a signature is valid, it checks the content hash and the Ed25519 signature itself against a trusted key - and then persisted to disk with a temp-file-then-rename write, so a crash mid-write can never leave a half-written snapshot in place of a good one. That on-disk copy is what "last known-good" means concretely: a daemon that restarts mid-outage reloads its own verified copy from disk rather than starting from nothing.

Three ages, one posture

A cached snapshot's age determines what a daemon does with it:

AgeWhat happens
Below DegradedAfter (default 5 minutes)Served normally. Fresh.
Between DegradedAfter and RefuseAfter (default up to 30 minutes)Still served - claims keep succeeding - but every decision it produces is flagged degraded: true with its age, visibly, never silently.
Past RefuseAfterA loud, typed refusal (*rulesetsnapshot.ExpiredError, carrying the revision, age, and configured bound) rather than a guess that a very stale bundle is still good enough.

The claim path additionally re-checks permission against the cached snapshot before admitting a claim - is this claim's target pool still granted, per the freshest cached data - so an org revoking a pool grant takes effect for a fail-static claim exactly as it would for a live one; the only thing caching changes is "is this still true" replacing "is this true right now." When no live claim-gate provider is wired at all, the snapshot-backed default answers claim-gate questions on its own within the same bounds.

Visibility

rensei host status prints a Ruleset snapshot: line whenever the daemon has a snapshot source configured, showing the cached revision and its current age (with a (degraded) suffix once the age crosses the threshold); see Host / Daemon - Status for the full sample output. The line is absent, not wrong, on a daemon with no snapshot source configured - self-hosted, single-machine installs are unaffected. The same signal is queryable directly against the local daemon's HTTP API (GET /api/daemon/routing/config) without going through the CLI.

Hosted, every decision record that resolved through a snapshot-backed read carries the identical fields (rulesetRev, snapshotAgeMs, degraded) - see Decision records & explain for the full shape and how to read one.

Verified live

A live rehearsal against an isolated daemon - deliberately not the operator's real running service - exercised the outage/degrade/restore cycle end to end and confirmed, against a real platform and a real daemon:

  • The cached revision survives a daemon restart mid-outage: pointing the snapshot endpoint at an unreachable address and restarting the daemon still leaves it serving the same revision it had cached before the restart, read back from its on-disk persisted copy - the fail-static claim holds across a process restart, not just within one process's lifetime.
  • A claim stays degraded but serving once the cached snapshot's age crosses DegradedAfter: the daemon does not stop accepting claims, it flags them.
  • Recompiling on the control-plane side and letting the daemon's next poll land advances the cached revision - restore is not a manual intervention, it is the daemon catching up on its next successful fetch.

What that rehearsal did not reach. The claim-time invariants that need a real admission-receipt-bearing dispatch in flight during the outage - an in-flight session continuing uninterrupted, a permission-denied claim still refusing against the cached snapshot, and RefuseAfter's typed error actually firing past the bound - are implemented in code (the typed ExpiredError, the claim-gate's permission re-check) but were not exercised end to end by an automated live rehearsal; reaching them needs a synthetic dispatch path that does not exist yet. This is a scope boundary in how far the rehearsal could reach, not a gap in the mechanism itself, and it is called out here rather than left to look like a completed proof.

On-prem and air-gapped posture

Rensei's control plane itself ships as a hosted service; a fully self-hosted control plane is not a shipped product today (see the on-prem portability row on the worker-fleet migration page for that roadmap item's actual status). What is already true, and already shipped, is the execution side: worker hosts can run on a customer's own infrastructure - local, Docker, or Kubernetes capacity pools

  • while the fail-static evaluator described on this page is exactly what keeps them working through a control-plane outage.

Concretely, for a regulated or on-prem evaluator: a control-plane outage does not stop admitted work. A daemon holding a cached, verified snapshot within its bounded TTL keeps evaluating claims against that snapshot locally - identical logic, degraded flag and all, whether the control plane is briefly unreachable over the network or the daemon is deliberately isolated from it. Authorization is the one deliberate exception to "outage does not stop work": a policy bundle the daemon cannot freshly evaluate still denies by default, so a control-plane outage is never mistaken for a control-plane outage plus a standing permit. Nothing about the evaluator's own logic changes for an air-gapped deployment shape - it only ever reads its own last verified local copy - but that specific deployment shape has not itself been delivered or certified end to end; treat it as an architectural property that falls out of the design, not as a shipped on-prem SKU.

On this page