Skip to content

layout-rendering.md

Latest commit

 

History

History
218 lines (141 loc) · 21.8 KB

layout-rendering.md

File metadata and controls

218 lines (141 loc) · 21.8 KB
Raw

Layout Rendering Algorithm

Thoughts are not rendered as a nested DOM tree. Instead they are a flat list of absolutely-positioned siblings — every visible thought is a sibling <div> of every other visible thought, positioned via transform: translate(x, y). The visual hierarchy is constructed by the x/y math, not by nesting.

This decoupling is what makes em's signature animation possible: a thought can move smoothly between any two states (different parent, different depth, different position in a context view) because no DOM hierarchy needs to be re-parented mid-animation. The same algorithm renders Normal view, Context view, and Table view.

The whole pipeline lives in src/components/LayoutTree.tsx and the hooks it composes.

Two-pass render

Thoughts have variable heights — they wrap, they have notes, they are taller at cliffs. The browser is the only authority on their measured height. So rendering happens in two passes:

  1. Estimate pass. The first render places every thought at a y position computed from singleLineHeight (an estimate based on fontSize, refined to the measured single-line height once any single-line thought has been rendered).
  2. Measurement pass. Each VirtualThought measures its rendered height in a useLayoutAnimationFrameEffect and calls onResize → setSize with the actual value. useSizeTracking updates a heights map.
  3. Reposition. The state change in heights triggers a re-render. usePositionedThoughts recomputes y positions using the measured heights. CSS transitions animate the y deltas into place smoothly.

The estimate is rarely far enough off to be visible, so the user almost never sees the intermediate position. The two-pass model is what lets the layout converge without snapping.

Two lists, one ordering

Each render produces two parallel lists, both indexed by the same key:

  • treeThoughts: TreeThought[] — produced by the linearizeTree selector. An in-order traversal of all currently visible thoughts. Each entry carries depth, table-cell flags, autofocus state, the Path and SimplePath, accumulated styles, etc. Recalculated whenever Redux state changes.
  • treeThoughtsPositioned: TreeThoughtPositioned[] — produced by usePositionedThoughts. Same shape but with x, y, width, height, cliff, isLastVisible. Recalculated when the heights or widths map changes.

The render walks treeThoughtsPositioned, wrapping each entry in a TreeNode (which mounts a VirtualThought plus the appropriate drop targets) inside a TransitionGroup so removed thoughts can fade out.

Keys: crossContextualKey

A thought can be visible in multiple places simultaneously when context views are active — e.g. cat under both Animals/m~/cat and Pets/m~/cat. The same ThoughtId must produce different React keys at each occurrence, otherwise React will reuse the DOM node and animations break.

linearizeTree builds keys via:

const crossContextualKey = (contextChain, id) =>
  `${(contextChain || []).map(head).join('')}|${id}`

So the key is the concatenation of every context-view boundary's id plus the thought's own id. The heights map and TreeThought entries are both keyed this way.

linearizeTree (the in-order traversal)

linearizeTree recursively walks the visible tree. For each visit it produces a TreeThought with everything the second pass needs.

Important behaviors:

  • Sibling order comes from rank. Children are read with getChildrenRanked, never re-sorted during the walk. A context's =sort preference is materialized into its children's ranks when the preference is set — see data-model.md → Visibility and sorting.
  • Visibility gating. A subtree is skipped entirely unless state.expanded[hashPath(path)] is set, so collapsed branches don't appear in treeThoughts at all. (The expansion model itself lives in expandThoughts.)
  • Context-view pivot. When a thought has its context view active, the recursion pivots from "render this thought's children" to "render the contexts in which this thought appears" via getContextsSortedAndRanked. The contextChain accumulator is updated, which feeds crossContextualKey. A special early-return: if the context view has only one context, the NoOtherContexts placeholder is rendered instead. Subthought renders the placeholder outside of linearizeTree, so it applies the same visibility gating itself: a collapsed thought with its context view active shows nothing beneath it, like any other collapsed thought.
  • belowCursor propagation. Once the cursor's Path is encountered during the walk, every subsequent TreeThought gets belowCursor: true. LayoutTree later uses this flag to exclude hidden-below-cursor thoughts from totalHeight so the document doesn't have a giant trailing dead zone.
  • Style inheritance. =children/=style and =grandchildren/=style are merged into styleAccum (current level) and styleFromGrandparent (skips one level). Specific positioning properties (marginLeft, paddingLeft) accumulate down the tree so descendants stay aligned with their ancestors.
  • =let environment accumulation. A context's =let bindings (parseLet) are merged into the env that is passed down the recursion and carried on every TreeThought, so a descendant can name a binding defined by any ancestor and a nearer =let shadows an outer one. A level that defines no bindings passes the inherited env through by reference, so trees without =let never allocate one and TreeNode's memoization is unaffected. See metaprogramming.md → =let.
  • Table cell flags. Each thought is tagged with isTableCol1 / isTableCol2 / isTableCol2Child based on =view/Table on its parent / grandparent / great-grandparent, plus visibleChildrenKeys is populated on table parents so col1 width can later be computed from the children's measured widths.

The result is a flat array, in document order, with one entry per visible thought.

usePositionedThoughts (x and y)

usePositionedThoughts walks treeThoughts once and produces the positioned list. It runs in a useMemo keyed on the inputs (heights, single-line height, viewport height, etc.).

Per thought:

const cliff = next ? Math.min(0, next.depth - node.depth) : -node.depth - 1

cliff is the number of levels dropped after this thought (negative). cliff = 0 means the next thought is at the same depth or deeper; cliff = -3 means the next thought is three levels shallower (the visible structure has fallen off a "cliff"). cliff drives both the cliff-padding (cliffPadding = fontSize / 4 of extra height to make room for the trailing DropCliff) and how many DropEnds to render between this thought and the next.

const height = heights[node.key]?.height ?? singleLineHeight + (cliff < 0 ? cliffPadding : 0)

Use the measured height; fall back to the estimate.

const x = fontSize * node.depth + ancestorTableWidths

x indents by 1em per level of depth, plus any table-col1 widths above this thought (so col2 is offset to clear col1).

y is accumulated from a running yaccum, with three special cases:

  • Table col1: do not increment yaccum, so col2 starts at the same y as col1. Push { depth, y: yaccum + height } onto ycol1Ancestors. This stack remembers the bottom of each col1 thought so a sibling that follows a tall col1 can clear it.
  • Returning out of a col2: if the top of ycol1Ancestors is at or above the current depth, pop it and bump yaccum to its bottom y if needed (so the next sibling clears col1, even when col2 was shorter).
  • New-cliff anticipation: when a brand-new thought (!heights[node.key]) appears at a cliff, the previous sibling is about to lose its cliff padding but hasn't been re-measured yet. Subtract cliffPadding from the new thought's y to anticipate the impending shift; otherwise the new thought renders at the wrong y for one frame and visibly slides up.
y = yaccum - (isNewCliff ? cliffPadding : 0)
yaccum += height (unless table col1)

After the loop:

  • tableCol1Widths: Map<ThoughtId, number> — the widths of every table's col1, used as inputs to ancestorTableWidths further down the loop. A col1 is as wide as its widest thought, and a col1 thought that virtualization has unmounted still counts at its last measured width, since its width is kept until it leaves the tree (see useSizeTracking). Without that, the column would narrow whenever its widest thought fell below the cutoff, the col2 thoughts above it would unwrap and shorten, and the widest thought would move back above the cutoff and remount, widening the column again. That cycle never settles.
  • indentCursorAncestorTables — captured when the cursor node is visited; drives the global horizontal autocrop (see Indent).
  • hoverArrowVisibility — 'above' | 'below' | null, set when the user is dragging into a sorted context and the drop position is outside the currently-rendered range. Drives the HoverArrow component.

useAutocrop (vertical autocrop)

When the cursor is deep, many ancestors and ancestor-siblings are hidden by autofocus. Their hidden heights still occupy the document's y-coordinate space, leaving a tall blank region above the cursor. If you scroll up, the entire viewport is empty.

useAutocrop crops this empty space and counter-scrolls so the visible thoughts stay put under the user's finger:

  1. LayoutTree sums the heights of every thought above the cursor where heights[key].isVisible === false && !belowCursor. Result: spaceAbove.
  2. useAutocrop extends that to at least one viewport height (so there's still room to scroll up): spaceAboveExtended = max(spaceAbove, viewportHeight).
  3. The hook returns autocrop = -spaceAboveExtended + viewportHeight, applied as transform: translateY(...) on the outer container.
  4. When autocrop changes, window.scrollTo({ top: window.scrollY + delta }) keeps visible thoughts positionally stable. The counter-scroll tracks the translation, not spaceAboveExtended: a viewport height change (e.g. rotating the device) moves spaceAboveExtended without shifting the thoughts when spaceAbove is less than the viewport, and scrolling by it would push the cursor off screen (#3990).

Net effect: the outer container is shifted up off-screen by exactly enough that one viewport's worth of empty space sits above the cursor. The user can scroll into that empty space; visible thoughts don't jump.

History: the original autocrop work is issue #1751; the uncle-handling refinement is issue #3055.

Indent (horizontal autocrop)

The same idea applied horizontally. As the cursor descends, the entire tree slides left so the cursor stays roughly center-screen, and table-col1 widths are absorbed.

indent = indentDepth * 0.9 + indentCursorAncestorTables / fontSize

indentDepth is cursor.length minus 2 or 3 (depending on whether the cursor is on a leaf), so the indent only kicks in once the cursor has descended past the second level. The 0.9 multiplier is intentionally less than 1 — it under-shifts the tree by 10% per level so the user keeps a visual sense of depth. (Going to 1.0 makes deep navigation feel completely flat.)

indentCursorAncestorTables was captured during the position pass; it adjusts the indent based on whether the cursor is in col1 (extra 1em), col2 (offset by col1 width minus 3em), or a col2 child (offset by col1 width minus 4em).

The indent is applied as transform: translateX(${1.5 - indent}em) on the inner container with a slow CSS transition (durations.layoutSlowShift). A matching negative marginRight keeps thought widths from being clipped during the shift.

Virtualization

LayoutTree computes viewportBottomOffset = spaceAbove + singleLineHeight * 5 and passes it with innerHeight. Each TreeNode combines those stable values with the current scrollTop to get the bottom virtualization boundary:

viewportBottom = max(scrollTop, 0)
               + innerHeight
               + spaceAbove
               + (singleLineHeight * 5) // overshoot, so a small scroll doesn't reveal blanks

Each TreeNode subscribes to scrollTopStore with a selector that returns only whether that thought is beyond the boundary. Most scroll updates leave this boolean unchanged, so they do not rerender LayoutTree, TransitionGroup, or the full thought list. A TreeNode returns null only when it is below the cursor, is not the cursor itself, and its y is more than one estimated thought height beyond viewportBottom. A thought that was virtualized and then rendered again while scrollTop and innerHeight stayed the same is not virtualized again until one of them changes. That guard keeps a layout feedback loop from forming: when the layout above a thought depends on whether that thought is rendered, as a table's col1 width once did, unmounting it can move it back above the cutoff, and remounting it can push it back below, indefinitely, until React aborts with "Maximum update depth exceeded" (error #185). Every such cycle needs a second unmount at the same viewport, so the guard breaks it whatever closes it. It remains in treeThoughtsPositioned so crossing the boundary can render it without rebuilding the list, while the fixed container height keeps the document height stable. (Above the cursor, autocrop already handles the blank space.)

useSizeTracking, heights and widths

useSizeTracking keeps two state maps keyed by crossContextualKey: heights holds { height, isVisible, cliff }, and widths holds the measured width of each thought. Widths are kept apart because only table col1 reads them, to size the column (see tableCol1Widths above), while every other consumer reads heights and treats an entry in heights as a measured thought. setSize is the entry point — it's passed into every VirtualThought as onResize.

Two details:

  • Height clipping by fontSize / 8. Each measured height is reduced by fontSize / 8 before storage. Why: thoughts use a clipPath to avoid accidental caret-snap to the line above/below when a click lands on a 1px boundary. The clipPath leaves a gap; the small height reduction overlaps thoughts by exactly that gap, eliminating it visually.
  • Unmount cleanup. When a VirtualThought unmounts, it calls onResize with height: null, and useSizeTracking.removeSize deletes its entry from heights. Its width stays in widths until its TreeNode unmounts and calls removeWidth, which happens only when the thought leaves the tree: a virtualized thought keeps its TreeNode and only unmounts the VirtualThought inside it. An unmounted ref prevents racing setState after the parent has already gone.

useSingleLineHeight (in LayoutTree.tsx) finds the smallest measured height in heights that's close to fontSize * 2 and uses that as the canonical single-line height for unmeasured thoughts. Cached in a ref so it doesn't reset to the estimate when single-line thoughts scroll out of view.

VirtualThought — when does it re-measure?

VirtualThought re-measures its height when:

  • The thought first mounts.
  • The value prop changes.
  • The cursor moves (subscribed via useSelectorEffect(updateSize, selectCursor, isEqual)).
  • The thought's style changes (subscribed via useSelectorEffect).
  • The viewport's innerWidth changes.
  • autofocus, crossContextualKey, id, or isVisible change.

offsetHeight is used on touch (avoids transform-induced fractional pixels); getBoundingClientRect().height on desktop.

Render tree

div                                              (outer; translateY for autocrop)
  HoverArrow                                     (drop arrow when dragging into a sorted context off-screen)
  div                                            (inner; translateX for indent, slow CSS transition)
    BulletCursorOverlay                          (rendered separately so cursor moves don't re-render every thought)
    TransitionGroup
      for each thought in treeThoughtsPositioned
        TreeNode                                 (absolute positioning via inline style)
          VirtualThought                         (measures height, calls onResize)
          DropChild | DropEnd | DropCliff | DropUncle  (drag-only)

Table view

Tables are rendered by the same algorithm; the differences live in three places:

  1. linearizeTree flags each thought. isTableCol1 if the parent has =view/Table; isTableCol2 if the grandparent does; isTableCol2Child for one level deeper. visibleChildrenKeys is populated on the table parent so col1 widths can later be computed from measured children.
  2. usePositionedThoughts does the column placement. As above: col1 doesn't advance yaccum (so col2 starts at the same y); ycol1Ancestors stack keeps track of col1's bottom y so the next row clears it. tableCol1Widths is built per-table and consulted by descendants for ancestorTableWidths (which feeds x).
  3. indentCursorAncestorTables provides the cursor-aware horizontal shift. When the cursor is in col1, col2, or a col2 child, the global indent is adjusted so the user stays oriented: a tap into col2 shifts the tree left by col1's width minus 3em, so col1 stays visible but col2 moves toward center.

Worked example:

- a
  - =view
    - Table
  - b
    - b1
    - b2
  - c
    - d
      - e

isTable: a; isTableCol1: b, c; isTableCol2: b1, b2, d; isTableCol2Child: e.

When b is positioned, yaccum is not incremented; b1 is placed at (x = fontSize * 2 + b.width, y = b.y); b2 at (x = fontSize * 2 + b.width, y = b1.bottom). When the loop reaches c, ycol1Ancestors.pop() returns b's { y: b.bottom, depth: b.depth }; if b.bottom > yaccum (i.e. col1 was taller than col2), yaccum is bumped to b.bottom so c clears the previous row. c is then positioned at (x = c.depth * fontSize, y = yaccum). Nested col2 rendering (e.g. d/e) accumulates col1 widths via tableCol1Widths, allowing arbitrarily nested tables.

useNavAndFooterHeight and spaceBelow

useNavAndFooterHeight measures the toolbar/breadcrumb nav and footer heights via getBoundingClientRect (throttled to 16.66 ms). The container reserves spaceBelow = viewportHeight - navAndFooterHeight - CONTENT_PADDING_BOTTOM (153px) - singleLineHeight so that closing the virtual keyboard doesn't force a scroll-position change — the document still has at least one viewport's worth of slack at the bottom.

useLayoutTreeTop

A small layout effect that writes the LayoutTree's top y (offset + autocrop) into viewportStore.layoutTreeTop. Used by scrollCursorIntoView to figure out where thoughts actually start on the page (vs. the toolbar above). It is a layout effect so that the value is set before the cursor overlay, a child of LayoutTree, checks the cursor on mount: React runs a child's passive effects before its parent's, so a passive effect here would leave layoutTreeTop at 0 for that check, and a cursor in plain view would be measured as behind the toolbar and scrolled.

Scrolling the cursor into view

scrollCursorIntoView scrolls the minimum amount needed to clear the toolbar above and the navbar (or virtual keyboard) below. useScrollCursorIntoView calls it two ways: directly when the cursor's y or height changes, and through scheduleScrollCursorIntoView when the editing value changes — the cursor's rank can move as it is edited, e.g. toggling bold in a long, sorted context.

The scheduled path defers to the next tick before reading the cursor's size, because editingValueStore subscribers run synchronously and would otherwise close over a size from before the render (#3083), and then throttles to 400 ms. A cursor scroll can therefore be waiting in three places at once: the tick before it reaches the throttle, the throttle's trailing call, and the 10 ms retry that scrollIntoViewIfNeeded arms while preventAutoscroll is in progress (see cursor-and-caret.md → preventAutoscroll.ts). scheduleScrollCursorIntoView.cancel() clears all three; cancelling the throttle alone leaves a timer that has not fired yet free to re-arm it.

scrollTo cancels before it scrolls. Its callers — Escape, Home, opening a modal, the footer, the tutorial's scroll-up button — are all deliberate moves of the viewport, so a cursor scroll queued before one of them is stale; without the cancel it lands up to 400 ms later and undoes the scroll that was just asked for. Escape and Home are the sharp cases, since clearing the cursor is itself what schedules the pending scroll. New code that repositions the viewport on purpose should go through scrollTo for the same reason. The autocrop compensation in LayoutTree is not such a case and correctly bypasses it: it holds the viewport still while content shifts underneath, so it has no pending cursor scroll to supersede.