Skip to content

Operator Task Store

Issue2104 adds the durable store for the operator task policy: the Core port IOperatorTaskStore and the pure stream fold OperatorTaskHistory. It is wired (plan C8b): a scoped DI registration whose only callers are the signed task routes. There is still no Marten schema change. Confirm (C10b) creates the one GitHub issue, only after its own Committed claim. The Marten adapter is MartenOperatorTaskStore.

Core contract

  • One stream per request, cockpit-task:<lowercase canonical UUID>, in the owner's tenant database. Each policy transition appends exactly one immutable event (Proposed, Approved, AttemptStarted, OutcomeUnknown, Submitted, and after Submitted the rc#2271 delivery), so Revision == event count == Marten stream version. The pure fold rejects every event the policy cannot append in each state — including a SECOND delivery, since re-recording a terminal success is the bug that fact exists to prevent. A stream therefore holds at most six events, which is why ReadAsync gives up after seven fetches rather than six.
  • The owner is trusted and server-constructed. Every operation checks it against the immutable stored owner with exact ordinal equality. A different tenant, operator or conversation gets NotFound or null.
  • Only Committed from TryBeginAttempt may authorize a future GitHub issue create (OperatorTaskStoreResult.AuthorizesIssueCreate). Existing, Conflict, Uncertain, a restart, an expired lease or an empty search never do. After Uncertain or OutcomeUnknown, the caller rereads and never retries.
  • The cockpit-task: namespace is reserved (OperatorTaskStreams, prefix match ignoring case). POST /api/events answers 400 for a body streamId or agentId in it, before dedup, tenant resolution or any append. SubmitEvent refuses any computed stream id in it for every caller: it appends nothing and returns the suppression reason reserved_stream.
  • The owner's tenant is the repo's tenant, from OperatorTaskTenancy.RepoTenant. It coalesces the resolver's answer the way the webhook does: an unresolved repo (null) is tenant-0 invotek, and a canonical tenant id is itself. A blank or non-canonical answer is refused, never coalesced to a default.

Marten adapter

  • Fresh session per operation. Every call opens its own LightweightSession(owner.TenantId) and stamps the tenant_id header, never the request-scoped session. rc#1062 saw a shared session's FetchStreamAsync return a stale or empty view. A claim decided on such a view could authorize a second issue create.
  • Propose runs the Core policy, then requires RepoTenant of the ITenantResolver answer for dashecorp/dashe-website to be exactly the owner's tenant. In v1 every target repo is hosted in the dashecorp GitHub org, which resolves to tenant-0 invotek today. An unresolved repo is also invotek, so an invotek owner commits in invotek's database. The task routes build the owner from that repo tenant. A different, blank, non-canonical or throwing resolution is Forbidden and writes nothing. StartStream either commits or collides. On a collision the adapter rereads in a new session: the same owner and intent get Existing, and anyone else gets NotFound with no data.
  • Transitions (Confirm, TryBeginAttempt, MarkUnknown, Reconcile, MarkVerified) use FetchForWriting, so the fetched version is enforced at SaveChanges. Confirm and TryBeginAttempt first re-resolve the fixed repo (Forbidden, with no read or write, on any mismatch), so a claim never commits after a remap. MarkUnknown, Reconcile and MarkVerified only record an outcome, so a remap or resolver outage does not block them, and Reconcile binds the trusted caller, never evidence.Owner.
  • The live-aggregation view folds every event. OperatorTaskStreamView has one Apply per event type and Marten aggregates it live (no registered projection, no document table). An event the view does not fold is SKIPPED, so the folded Revision falls behind mt_streams.version and the next fetch throws the poisoned-stream exception — adding an event without its Apply bricks every stream that carries it. A non-Applied policy outcome maps to its result with no write. A lost race gets Conflict plus a fresh reread. Any other save or commit failure is Uncertain with no snapshot. No write is retried, and there is no last-writer-wins.
  • Torn reads are lost races. FetchForWriting reads mt_streams.version and then all events, as two READ COMMITTED statements. A commit between them yields Revision > version. A transition then decides nothing and returns Conflict plus a fresh reread; Read refetches in a new session. Each tear needs a new legitimate event, and a stream holds at most five, so a read tears at most five times.
  • Foreign events fail closed. POST /api/events and SubmitEvent now refuse the key, but this check stays as defense in depth: any other writer (a direct IEventStore append) could still land an event on the key in the same tenant database. Live aggregation skips types without an Apply. Commits only add events, so Revision < version means the fold skipped one. The verified owner then gets an exception and nothing is written. A foreign event committed during one fetch can hide from that fetch, but any write it leads to fails the version check. Ownership is checked first, so a non-owner gets the same null or NotFound for a missing, clean or poisoned stream.
  • The live aggregate is not registered: FetchForWriting aggregates it live and creates no document table, so production wiring needs no aggregate registration or schema change.

Evidence

OperatorTaskStoreRaceTests run against Testcontainers postgres:16-alpine through OperatorTaskStoreHarness. The store mirrors Program.cs options, and TenantProvisioner registers the dashecorp and other databases. Every case asserts event rows and mt_streams.version with raw SQL. Gated races hold all contenders before SaveChangesAsync, then release them together or one at a time. The one-at-a-time mode is required because Marten resolves an unversioned append's version before listeners run. An ungated, jittered burst lets fetches overlap commits. A lock on mt_events commits a transition between the two fetch statements deterministically. MarkUnknown and Reconcile are raced too (8-way and mixed). Each of these mutants fails the suite: last-writer-wins or a retry at a fresh revision on any transition; ignoring the caller's revision on any transition; a torn view that throws, decides, or is returned without a refetch; deleting the foreign-event or owner check; case-insensitive owners; Reconcile trusting evidence.Owner; a re-resolve on MarkUnknown or Reconcile; Confirm ignoring the clock; a Conflict without a reread or carrying the loser's own claim; Uncertain as committed; a resolver that fails open; and Propose faults treated as committed or as a collision.

Remaining enablement gates

Done in C8b: the signed Worker-to-Conductor transport, DI and route wiring, a server-constructed owner built from RepoTenant (tenant-0 invotek today), and a check that the tenant database is registered before any session opens (task routes). Release consent through the enforced hold is done too: MergeGate holds a task PR (C9b) until the signed release route approves its exact head (C9c). The confirm route and its status reconciliation (C10b) call the App-token issue creator (C10a). Still open: the provisioned key (plan G2). Real submission stays blocked until it is live.

Known limit: one issue per requestId holds within one tenant database

Each tenant has its own database, and the owner tenant is re-derived from the repo on every call, so StartStream only collides with a requestId inside the current tenant's database. If the repo's resolved tenant moves while a task is in flight (the plan's OA10 canary activates a dashecorp tenant row), the old task reads 404, its held PR can no longer be released through the route, and a Worker that re-proposes the same requestId (only while its D1 row is still pending, so after a lost D1 advance) would start a fresh stream and a confirm would POST a second issue. Operating rule until this is designed (rc#2104 review, with infra#456's owner-binding item): never remap dashecorp/dashe-website's tenant while a task is in flight. A cross-tenant existence check at propose was not added: it would couple every proposal to the health and schema of every tenant database (with the schema fence armed, a tenant database without event storage would fail every proposal). The durable fix is to record requestId → owner tenant in the control database at propose and bind every later call to it.