feat(ocap-kernel): carry out a vat restart on the run loop by sirtimid · Pull Request #1096 · Consensys-Incorporated/ocap-kernel

sirtimid · 2026-09-15T19:02:24Z

Stacked on #1093. Diff: git diff sirtimid/retire-vat-synchronously...sirtimid/restart-vat-run-queue-item.

A restart takes a vat out of the kernel's reach while a new worker launches. Done outside the run loop, a crank landing in that window reads a live vat as dead. restartVat now queues a restartVat run queue item and waits, and the run loop carries it out in a crank of its own, like SwingSet's upgrade-vat.

Waiters are kept in RAM per vat. Every request queues its own item; the first crank to run settles all waiters for that vat, and later items find none and are dropped.
A failed relaunch returns { abort, terminate }, like a failed delivery. The run loop rolls back what the failed initVat buffered, then retires the vat. Callers are answered once the crank ends.

Known gaps

Writes made from outside the run loop (restartVat, queueMessage) during a crank that aborts are rolled away, and their callers wait until the run loop dies. This is pre-existing for any aborting delivery; a failed restart is a longer such crank, since it waits out the handshake. It closes when control-plane writes move onto the run loop.
reset, clearStorage and stop can abandon a queued restart request. Left for the PR that stops the run loop first.

Testing

VatManager.restart-crank.test.ts: real run loop and store. A failed relaunch's buffered send or notify is discarded and the vat terminated. Fails without the abort.
VatManager.test.ts: waiter coalescing, stale and leftover items, a vat gone before its crank, relaunch failure, and an old worker that will not shut down cleanly.
kernel-test: a vat restarted twice reaches start count: 3 in baggage.
Full @metamask/ocap-kernel suite green.

Carries the findings from #1065, #1068 and #1081.

🤖 Generated with Claude Code

Note

Medium Risk
Changes vat lifecycle and run-loop crank/commit semantics; failed restarts now terminate vats, but behavior is heavily tested and aligns with existing abort/terminate paths.

Overview
restartVat no longer stops and relaunches a worker inline. It enqueues a new restartVat run-queue item, registers per-vat waiters, and resolves only after the run loop finishes the restart crank—so no delivery can see a persisted vat with no live worker in between.

The run loop routes that item through KernelRouter to VatManager.performVatRestart, which terminates the old worker, relaunches from config (including vats already between workers), and coalesces multiple concurrent restart callers on one crank. KernelQueue adds enqueueRestartVat, defers enqueue while a crank is open (so an aborted crank does not roll away the request), and onRunLoopDeath rejects restart waiters if the loop dies first.

If relaunch fails, performVatRestart returns { abort, terminate } so the crank rolls back buffered initVat output before the vat is retired—matching failed-delivery semantics instead of leaving a persisted vat with no worker.

Integration and unit coverage include a kernel-test double-restart baggage check, VatManager.restart-crank.test.ts with a real store/queue, and expanded VatManager / KernelQueue tests; Kernel.test now asserts enqueue + run-loop-death behavior rather than full inline restart.

^{Reviewed by Cursor Bugbot for commit b2477a9. Bugbot is set up for automated code reviews on this repo. Configure here.}

cursor

Stale Bugbot comment from a previous run.

github-actions · 2026-10-01T12:26:49Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	73.45% ⬆️ +0.12%	10034 / 13660
🔵	Statements	73.24% ⬆️ +0.12%	10163 / 13876
🔵	Functions	73.81% ⬆️ +0.06%	2342 / 3173
🔵	Branches	67.84% ⬆️ +0.15%	4140 / 6102

File Coverage

File	Stmts	Branches	Functions	Lines	Uncovered Lines
Changed Files
packages/ocap-kernel/src/Kernel.ts	89.84% 🟰 ±0%	79.54% 🟰 ±0%	85.41% 🟰 ±0%	89.84% 🟰 ±0%	323-325, 396, 420, 495-505, 593, 661, 737-740, 753, 763-764, 817, 840
packages/ocap-kernel/src/KernelQueue.ts	98.38% ⬇️ -0.37%	91.3% ⬆️ +0.61%	96% ⬇️ -4.00%	98.91% ⬆️ +0.16%	181, 235, 680
packages/ocap-kernel/src/KernelRouter.ts	94.55% ⬆️ +0.07%	81.94% ⬆️ +0.25%	100% 🟰 ±0%	94.55% ⬆️ +0.07%	122, 185, 202, 304, 357, 417, 435, 438
packages/ocap-kernel/src/types.ts	100% 🟰 ±0%	100% 🟰 ±0%	100% 🟰 ±0%	100% 🟰 ±0%
packages/ocap-kernel/src/vats/VatManager.ts	98.7% ⬆️ +0.37%	96.87% ⬇️ -0.74%	96.87% ⬆️ +0.45%	98.7% ⬆️ +0.37%	226-229, 256

Generated in workflow #5113 for commit b2477a9 by the Vitest Coverage Report Action

cursor · 2026-10-01T16:38:58Z

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

cursor

Stale Bugbot comment from a previous run.

cursor

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c9b7cb9. Configure here.}

A restart keeps the vat's c-list while taking the vat itself out of the kernel's reach for as long as launching a worker and negotiating with it takes. Done where it is asked for, with the run loop free to run cranks throughout, a crank landing in that window reads a live vat as a dead one. `restartVat` now queues a `restartVat` run queue item and waits on a RAM waiter keyed by the vat; the run loop carries the request out in a crank of its own, where nothing else can reach the vat. Two callers for one vat share one item, because the first one's crank produces exactly what the second asked for. A request nobody is waiting for outlived the process that queued it, and is dropped. A relaunch that fails retires the vat and kills the worker it left behind, without throwing: the run loop's catch would roll the crank back, undoing those records and restoring the request, so every later start would replay the same failing restart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Skipping the enqueue on the strength of someone else's waiter made a lost item permanent: a request rolled away by an aborting crank left a waiter nothing would consume, and every later request for that vat then skipped the queue too and hung with a healthy run loop. Every request queues an item; the crank that arrives first settles the whole list and the rest are dropped as stale, which the same path already did for an item that outlived its process. The enqueue also moves ahead of the waiter. A run loop already dead rejects the waiter the moment it is registered and then refuses the enqueue, so the throw left by way of the `try` and nothing ever awaited the rejected promise — an unhandled rejection, which Node's default turns into a dead process. An old worker whose channel will not close no longer costs the vat its relaunch: the handle is off the books and the worker killed either way, so retiring it for that was retiring a vat that was merely untidy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…e test #1084 landed with a router built positionally, so the logger took the slot the new parameter added and the `already settled` assertions saw nothing. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

The crank completed without aborting, so whatever the failed `initVat` buffered was flushed after the vat had been retired: sends from a vat that no longer existed went out, and a notify addressed to it could kill the run loop. `performVatRestart` now reports an abort and a termination, the shape a failed delivery already has. The run loop rolls the crank back and then retires the vat; the rollback restores the request, which finds no waiters and is dropped. Callers are answered once the crank ends, so they wake to a committed termination. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…not say Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…nk ends Written into an open crank, the request was rolled back with it if the crank aborted, and restartVat waited on an item that no longer existed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

A caller now wakes after the crank ends, by which time the next crank may be another restart that has already taken the handle away, and looking the vat up then threw VatNotFoundError for a restart that succeeded. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

sirtimid requested a review from a team as a code owner September 15, 2026 19:02

cursor Bot reviewed Sep 15, 2026

View reviewed changes

Comment thread packages/ocap-kernel/src/vats/VatManager.ts Outdated

sirtimid added this pull request to stack #1108 September 15, 2026 22:38

sirtimid force-pushed the sirtimid/restart-vat-run-queue-item branch from 1f752ce to 2b78778 Compare October 1, 2026 12:17

Base automatically changed from sirtimid/retire-vat-synchronously to main October 1, 2026 16:38

sirtimid force-pushed the sirtimid/restart-vat-run-queue-item branch from 2b78778 to ec916be Compare October 1, 2026 16:38

ci-belphegor mentioned this pull request Oct 1, 2026

terminate during restart creates zombie vat #1133

Open

sirtimid force-pushed the sirtimid/restart-vat-run-queue-item branch from 59c175f to 3f26187 Compare October 2, 2026 09:03

cursor Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread packages/ocap-kernel/src/vats/VatManager.ts

cursor Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread packages/ocap-kernel/src/vats/VatManager.ts

sirtimid and others added 8 commits October 2, 2026 20:07

docs(ocap-kernel): cut the restart comments down to what the code can…

06ef6a7

…not say Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(ocap-kernel): answer a restart's callers once its crank ends

b97c877

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

sirtimid force-pushed the sirtimid/restart-vat-run-queue-item branch from c9b7cb9 to b2477a9 Compare October 2, 2026 18:43

Navigation Menu

feat(ocap-kernel): carry out a vat restart on the run loop - #1096

feat(ocap-kernel): carry out a vat restart on the run loop#1096
sirtimid wants to merge 8 commits into
mainfrom
sirtimid/restart-vat-run-queue-item

sirtimid commented Sep 15, 2026 •

edited by cursor Bot

Loading

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot commented Oct 1, 2026 •

edited

Loading

Uh oh!

cursor Bot commented Oct 1, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

sirtimid commented Sep 15, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Known gaps

Testing

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report

Uh oh!

cursor Bot commented Oct 1, 2026

Bugbot needs on-demand usage enabled

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sirtimid commented Sep 15, 2026 •

edited by cursor Bot

Loading

cursor Bot left a comment •

edited

Loading

github-actions Bot commented Oct 1, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading