Operator task lifecycle walk¶
Tracking: rig-conductor#2228, gap 1 of the end-to-end plan written after #2104 shipped.
What was missing¶
The Rig-E operator-task feature shipped with 23 route-level e2e cases
(CockpitTaskEndpointsE2eTests), 39 MergeGate cases
and a cross-repo contract harness. Between them they cover every transition — and every one of them builds its own
state and asserts one transition. Nothing drove one task through the whole pipeline, so:
- an ordering or state-machine defect between two stages was invisible;
- the 15-key status contract was asserted per stage, not on every read of one task's life;
- "a stage reached stays in the timeline" was never asserted across more than two stages;
- nothing proved a refused release leaves the task exactly as releasable as it was.
What the walk adds¶
CockpitTaskLifecycleWalkE2eTests — three walks
over the real Program.cs pipeline and Testcontainers Postgres, each one task, each read asserting all 15 keys of
CockpitTaskStatus:
| Step | Driven | Asserted |
|---|---|---|
| 1 | signed propose |
proposed; issue/pr/release/deploy/verify all null; no GitHub call before Submitted |
| 2 | a second status | unchanged, still no call (the card polls this route) |
| 3 | confirm, then a replayed confirm |
one issue POSTed as the App, the fixed template, a marker naming this request; the replay is existing with no second POST |
| 4 | — | queued, "Waiting for Planner-E" |
| 5 | Planner-E's TaskSpec comment + tier:t1 |
queued, "Planned (tier:t1), waiting for agent-ready" |
| 6 | the agent-ready label and its event |
implementing, dated from the label event, not from the read |
| 7 | a draft PR, then ready for review | implementing (a draft is not in review) → in_review; release null while nothing is held |
| 8 | Review-E approves the head; the check run completes | MergeGate itself holds the task PR instead of merging → approval_needed, release.heldSha at the head |
| 9 | a release signed for another head | 409 head_moved, nothing merged, nothing approved — and the task is still approval_needed |
| 10 | the release at the approved head | merged pinned to that head, one audit comment naming the request, and the approval's MAC verifies under the task key |
| 11-13 | the deploy run, then the new href | merged → deployed → verified, terminal, whole timeline in order |
| 14 | two more status reads | terminal: GitHub and the probe are never called again |
Two further walks cover the branches that only mean something on a task with real history: a release deferred by a
pending check (the approval survives; the status retries the pinned merge 2 min later), and a create whose outcome is
unknown (never retried, reconciled from the one issue the App filed, and the task then walks on to in_review).
The fake is driven, not replayed¶
WalkWorld serves GitHub, Valkey and dashecorp.com from mutable state that the test moves the way Planner-E, Dev-E and
Review-E would: the issue body every read serves is the body the confirm route posted, and the merge PUT actually
marks the PR merged and closes the issue. That is what makes the order load-bearing — a replayed fixture would answer
the same thing whatever order the reads came in.
Two things this cost, worth knowing before extending it:
- MergeGate records its own conditions with the single-field
HashSetAsyncoverload. Substituting only theHashEntry[]overload makes the gate not see theci-passedit just wrote, take the "review approved, CI silent" branch, and hold nothing — a green-looking drive that asserts nothing. - A pending check defers the merge before the hold is decided, so a walk that wants
approval_neededmust reach it with the check green and only then make one pending.
Proof it is not vacuous¶
Each of these mutations of shipped code was applied, run and reverted; every one fails the walk:
| Mutation | Result |
|---|---|
Terminal renamed on the status record (the wire contract drifts) |
3 of 3 walks fail |
a draft PR projected as in_review (a stage-machine defect) |
the walk fails |
releasing listed before approval_needed in the timeline |
2 of 3 walks fail |
Running it¶
CI needs no new workflow: test.yml already runs the whole Api project on every PR
and on main. Locally, pin LC_ALL=en_US.UTF-8 for the full suite — six formatting assertions elsewhere fail without
it and look like a regression.
Still open¶
The walk closes gap 1 of five. The rest, in the order they are worth doing: the cross-repo contract harness is in no
repo and no CI; the Worker leg is replayed rather than driven; nothing runs
check-rig-e-persona.py in rig-gitops; there is no scheduled read-only live smoke; and the feature has never once run
for real — that last one is operator-gated, because it wakes the GPU pod and files a real issue.