Repository navigation
design: Real-time co-editing - #4267
khoinguyenpham04 wants to merge 13 commits into
Conversation
Add an RFC for real-time co-editing in the admin editor. A server-side room per entry (a Durable Object on Workers, an in-process object on Node.js) holds the entry's Yjs document, logs every update before sharing it, and checkpoints the document to the draft revision through the existing save path. REST, CLI and MCP writes keep getting ENTRY_LOCKED; plugins, scheduled publishing, the visual editor and overrideLock requests write through the room. Amend the readable revision history plan for entries edited in a room: multi-editor versions, recovery versions, private viewing and restore.
Rework the co-editing RFC around rooms that only sync and store the Yjs document. Clients save checkpoints through today's autosave request with `_rev`, one at a time under a room save grant, so rooms no longer run the EmDash runtime, hooks or a Portable Text converter. - WebSockets on both hosts: a Durable Object on Workers, and a new `emdash start` command on Node.js. - Coverage uses Yjs snapshots, so delete-only edits count. - Out-of-band writes merge field by field at the next checkpoint. - Stale rooms save uncovered edits as recovery revisions. - Drop fallback locks, the operation queue, field leases, the room registry, signed tickets and server-side three-way merges. Amend readable revision history for the new revision rules.
Stale rooms close old sessions, the plain lock route refuses room leases, initializing clients decline oversized entries and unsupported marks, the connection lifetime is checked on every message, and merge updates count as their sender's edits.
|
github-actions
Bot
added
the
query-count changed
Query-count snapshot changes4 routes changed, total Δ +16 queries. SQLite
D1
Comparing snapshot files between base and head. Updated automatically on each push. |
github-actions
Bot
added
review/needs-review
Rework the co-editing RFC so the room saves. It initializes from the stored entry, saves checkpoints through the content PUT's steps as the latest co-editor, and merges every other write live, so REST, MCP, the CLI, the visual editor and plugins are no longer locked out. - On by default wherever the host can run rooms: `emdash start` on Node.js, or the `EmDashRoom` Durable Object on Workers. Core code stays vendor-agnostic. - A merge base of `_rev` and per-field hashes decides each merge, and saves send only changed fields. - Operations hold the room while they run. - A shared, DOM-free editor schema and converter, with derived keys, so serialization is deterministic. - Recovery revisions are marked, and restoring one runs validation and hooks. Amend readable revision history for room checkpoints and recovery versions.
Cut the co-editing RFC from about 8.9k to 7.5k words without changing its design, and close the findings of an adversarial review of the whole branch. - A move to Trash reaches the room, which keeps unsaved edits as a recovery revision before it closes. - In a room, SEO, author, date, byline and reference saves send no `_rev`, and operations run without a form save. - An operation caller's own `_rev`, as MCP sends, is checked before the room saves anything. - The room merges before falling back to a recovery revision when it drops its state, and drops a document that outgrows 2 MB. - Values a publish copies into sibling locales notify their rooms. - Drawbacks name several Node.js processes and the lock race. Amend readable revision history so the room keeps the current draft only before operations a person runs.
- Take over through the join route needs edit access, as the lock route does today. - A drop forgets unused tokens, and token requests wait for it, so no client joins state that is being dropped. - Operations wait for a checkpoint that covers the client's edits, and show a refused save's error. - An operation caller's `_rev` is compared with the stored entry's. - The room ignores the slug in save responses. - Name the restore and discard refusal for unsupported marks, and the four MCP tools that require `_rev`.
Readability: add a Yjs primer and a sequence diagram of the example, give each guarantee its reason, and rewrite the checkpoint, kept checkpoint and merge base terms. Review fixes: - Record the merge base from the save's own write, not a later read. - A room nobody is in saves conflicting or leftover edits as recovery revisions, never over newer work. - Tokens carry the status and author that access was checked against. Rooms refuse stale tokens and tokens from other builds. - Reset a room when a loaded entry has another manifest hash, and read from the primary database. - Broadcast checkpoints with nothing to send, keep editedBy until covered, and name Take over's window and MCP content_update's partial apply.
|
Thanks @khoinguyenpham04, for this solid and in-depth write-up! I also liked how the design learns from WordPress's co-editing attempt. I understand co-editing won't work reliably when multiple Node processes are running, as each process has its own separate memory. So scaling would cause silent data loss, and the fix is to set However, I think it would be great to have a way to detect an additional process. For example, a lease table (room ID, owner ID, A lighter alternative is to implement a heartbeat at startup that disables co-editing when it finds more than one process. Would love to hear your thoughts on whether this detection would be beneficial in a multi-process Node setup. |
|
Thanks @CacheMeOwside, great catch! With co-editing on by default, telling people to set I've added your first idea to the RFC ([Claims on Node.js](https://github.com/emdash-cms/emdash/blob/noah/emash-realtime-collab-research-cb0d11/proposals/real-time-co-editing.md#claims-on-nodejs)). A process claims an entry's room with one conditional write on the existing I left out the startup check because one extra process would turn co-editing off for every entry, and it would also fire during deploys, when the new process starts before the old one stops. |
Add claims, so several Node.js processes sharing a database never store one entry's room twice. Joins that reach another process get ROOM_BUSY and retry. Also close gaps found in a full review: fall back to single-editor mode when WebSockets are blocked, open the socket on the join path, keep editor-draft plugins working in a room, and have the shared save function return its own write.
MA2153
left a comment
MA2153
left a comment
There was a problem hiding this comment.
This looks good so far for v1 and I am willing to sponsor it. My only complaint is about the text of the RFC itself - it felt too procedural and could benefit from some abstraction and re-organization to make it easier to read.
After v1 I am hoping revisions could show all those who contributed not just the last one (I could see that being confusing / misleading to normal users).
emdashbot
Bot
added
review/needs-review
github-actions
Bot
added
review/approved
The detailed design now opens with how the parts fit: the parts, the last checkpoint and merge base that tie the shared document to the stored entry, and three fallbacks. The rules follow by subject, with tables for join outcomes, save results and the room lifecycle, and the exact mechanics of documents, saves and merges move to a new Appendix B. The design is unchanged.
Outside a room, autosaves replace each other, so a version showed only the last person who saved it. Autosaves now store their author in edited_by and carry the names of the revision they replace, and a null edited_by means the author alone. The revision-history amendment applies this to every collection with revisions.
|
Overall this is good, and I am certainly in favour of the proposal itself. My concern is the Node.js implementation. I particularly do not like the requirement to create the new CLI command. It's changing the way Astro is run, which is excessive for a single feature. The ideal would be to add WebSocket support upstream to Astro, which we can propose. However failing that, it looks like Node does offer access to the raw request using |
Drop the planned emdash start command, which changed how Astro runs. EmDash now finds the server that serves Astro through node:diagnostics_channel and attaches an upgrade listener, so Node.js sites start as they do today, in middleware mode too. It listens to http.server.response.finish, because Astro loads EmDash during a process's first request, after Node.js has published that request's start. Where Node.js has shouldUpgradeCallback, other upgrade requests are served as before. Cluster workers keep single-editor mode, and a process handles SIGTERM and SIGINT while it holds claims, so its rooms save and release them before it stops. Also correct facts found in a full review: Hocuspocus 4 runs on both hosts, Wrangler's declarative exports field replaces migrations, older versions can't export a migrated site, the join route needs a site-write fence exemption, notifications come from the repositories, and adding table rows doesn't replace the table.
github-actions
Bot
added
review/needs-rereview
Design proposal
Proposal: proposals/real-time-co-editing.md
Only one person can edit an EmDash entry at a time today. This RFC lets several people edit one entry in the admin editor at once, with merging, presence, cursors and per-person undo.
emdash startcommand), without a third-party service or Cloudflare code inemdash. On Node.js, a claim in the database keeps each entry's room in one process, so sites that run several processes never split a room.Proposal type
proposals/readable-revision-history.mdfor versions that several people edited and for entries edited in a co-editing room.Related work
Discussion: #4173
Readiness
accepted(check only when ready to merge)Open before acceptance: the prototype results for the acceptance gates listed under "Verification" in the RFC.
AI assistance