fix: bound the search for repeated moved array elements and type maybe-async recipes of primitive states

[unadlib]

Part of #168.

Summary

This PR fixes a performance regression that #200 introduced and a typing gap that #201 left, documents two limits, and refreshes the published bundle sizes, the API reference, the size limit of all ESM exports and the size baseline.

#	Issue	main	npm 1.3.0 and Immer 11.1.18	This PR
1	Reads, removals and searches of moved elements of an array that holds a few objects many times, after shift(), unshift() or splice()	since #200, the lookup of a moved element's original index follows a repeated element forward copy by copy, one search per later copy: shift() and eight reads of 10,000 rows of two objects in turn take 581 µs, against 10.3 µs before #200	1.3.0 moves elements through the proxy, 8.4 ms for these updates at 10,000 rows; Immer drafts them on its proxy path	follows 16 later copies one after another, then searches forward with doubling steps, and backward only from the first step past the last copy: 13.6 µs
2	An explicit primitive state type with a recipe that may return a Promise, as in create<number>(1, (n): number | Promise<number> => …)	typed number, or as the patches tuple, although the result can be a Promise; also in curried producers and creators from makeCreator()	1.3.0 the same	number | Promise<number>, as object state types have since #201; synchronous and async recipes keep their types
3	Options whose enablePatches is typed true but may be missing at run time: an optional options parameter of a wrapper, options that may be undefined, or { enablePatches?: true }	typed as the patches tuple, while create() returns the state alone	1.3.0 the same; Immer returns patches from a separate produceWithPatches()	documented, with boolean as the type that gives either shape, also as the type argument next to an explicit state type
4	apply() with a prototype that other code has already changed	path segments other than the __proto__ and constructor of objects and arrays and a function's prototype are read as properties	the same	documented, including that Map keys of these names are ordinary keys

Changes, one commit per item

1. perf: bound the forward search for later copies of a moved array element (item 1): the first eight lookups of an array check the index that the offset of the last move gives, and perf: find a moved array element's original index from the offset of its move #200 then searched forward from it for each later copy, to find the last index of a repeated element, which the map of original indices holds for the later lookups. An element repeated throughout the array took one indexOf() call per later copy, 39,984 for eight reads after shift() of 10,000 rows of two objects in turn, where the backward search before perf: find a moved array element's original index from the offset of its move #200 found its last copy near the end at once. This commit follows at most 16 later copies forward and otherwise searches once from the end; commit 9 replaces the search from the end. It returns the same index as before. A test counts the searches on the original array (3,984 on main for 1,000 rows), and the moved-element tests also run on an array of two objects in turn, frozen or not, with 1, 8, 9 and 40 edits.
2. fix: type a recipe that may return a Promise for a primitive state (item 2): the first explicit-state overload only gives the async recipes of a primitive state their context, so fix: harden apply() for Set positions and mutable root replacement, and fix development checks and recipe types #201 kept the maybe-async overloads away from primitive state types, whose synchronous recipes would otherwise have matched them. A primitive state now has an overload that takes only synchronous recipes, for direct and curried calls, before the maybe-async ones, which apply to every state type again. The migration guide in the README and on the website no longer lists primitive state types as an exception.
3. docs: describe how the type of the options decides the result type (item 3): under enablePatches in the README and on the create() page.
4. docs: state that apply() does not guard against prototypes changed elsewhere (item 4): the README and the apply() page.
5. docs: measure the bounded search for repeated moved elements in the performance summary (item 1): a paragraph and a table in "Moved array elements", the case that still searches from the end under "Tradeoffs and limits", and an entry in the history; commit 12 replaces them.
6. docs: refresh the bundle sizes in the README, the installation guide and the performance summary: the README still gave the sizes measured for perf: rerun the full benchmark on main and refresh the published results #199, 7.8 and 8.4 kB; with perf: find a moved array element's original index from the offset of its move #200, fix: harden apply() for Set positions and mutable root replacement, and fix development checks and recipe types #201 and this PR they are 7.9 and 8.5 kB (7,912 and 8,524 B Brotli at the head, measured as the README describes). The size-limit figures of the summary dated from perf: read the type of a draft from its state when applying patches #196.
7. docs: regenerate the API reference: the overloads of commit 2 and the line numbers that changed after fix: harden apply() for Set positions and mutable root replacement, and fix development checks and recipe types #201 generated it; the links point to the commit before it.
8. build: refresh the size baseline for the bounded search of moved array elements: commit 1 grows the development builds by 96 B raw, over the allowance of 64 B.
9. perf: search backward from the first probe past the last copy of a repeated moved element (item 1): after commit 1, an element with more than 16 later copies was searched for from the end of the array, as before perf: find a moved array element's original index from the offset of its move #200, which V8 runs about 15 times as slowly on frozen arrays. When the first 100 of 10,000 rows hold two objects in turn, shift() and eight reads of a frozen base took 1,895 µs, against 62 µs on main. After 16 copies, each forward search now starts twice as far ahead as the one before; the last copy then lies before the start of the first search that finds none, and the backward search starts there. It returns the same index as before. The counting test now allows 17 forward searches one copy after another and about one per doubling of the step (at most 8 × 28 for 1,000 rows; 208 now), with one backward search per read; a new test checks that the backward searches for an array whose first 100 of 1,000 rows repeat two objects start before index 200, which fails on commit 8; and the moved-element tests also run on 60 rows whose first 40 hold two objects in turn.
10. docs: limit the prototype note of apply() to objects and arrays (item 4): apply() checks __proto__ and constructor on the paths of objects and arrays; Map keys of these names are ordinary keys, which it reads and writes.
11. docs: show the patches type argument to pass with an explicit state type (item 3): with an explicit state type, TypeScript infers no other type argument, so options typed { enablePatches: boolean } need create<State, false, boolean>(…), or create<State, [], false, boolean>(…) for a curried producer; without it, the call does not compile.
12. docs: measure the doubling search for repeated moved elements in the performance summary (item 1): the paragraph, the table, the "Tradeoffs and limits" bullet, the size-limit figures and the history entry of commits 5 and 6, measured again with the lookup of commit 9.
13. build: raise the size limit of all ESM exports and refresh the baseline for the doubling search: commit 9 brings all ESM exports to 8,515 B, over their limit of 8.5 kB, which is now 8.6 kB, and grows the development builds by 53 B raw.

Behavior changes to review

• Item 1: none in results: the lookup returns the same original index as on main. 20,000 random recipes gave the same states, return values, patches, inverse patches and replays as main with the head's production build, and 100,000 with commit 1.
• Item 2: these calls now get a result type that includes a Promise, so code that used the result as a number has to handle the Promise, as for object state types since fix: harden apply() for Set positions and mutable root replacement, and fix development checks and recipe types #201.

Not covered

• Item 1: the backward search covers at most the last doubled step of the forward searches, which a long run of copies makes long. When the first half of 10,000 rows holds two objects in turn, shift() and eight reads take 655 µs on a frozen base, against 330 µs on main and 985 µs before perf: find a moved array element's original index from the offset of its move #200. Arrays of objects repeated throughout take 3–4 µs more than before perf: find a moved array element's original index from the offset of its move #200, whose single backward search found their last copies at once (13.6 against 10.3 µs at 10,000 rows). Other lookups were measured:
  • Searching from the end after 16 copies, commit 1 alone: see commit 9.
  • Halving the range of the last copy with forward searches (+14 B over this PR) was faster in V8 for the first half of the rows (90 µs), but up to 3.7 times slower than commit 1 in WebKit on a frozen base, and 1.2–2.1 times slower in Firefox.
  • A backward loop in JavaScript instead of lastIndexOf() (+6 B over this PR) was fast in a process that sees one kind of array, but became up to 7.5 times slower once its lookups had seen frozen, sealed, non-extensible and holey arrays, then slower than lastIndexOf().
  • A single forward check followed by one backward search saves 11 B over commit 1, but brought arrays that hold each object twice back to their cost before perf: find a moved array element's original index from the offset of its move #200, 1,108 µs instead of 176 µs at 10,000 frozen rows.
  • Taking a repeated element's first index instead of its last needs a single forward search and was the fastest in every layout, and saves 64 B. It changed the patches of 478 of 40,000 random recipes and broke the replay of 17, where a removed copy took the key of an unmoved, changed draft and its change was lost.
• Item 3: telling these calls apart needs the options as a type parameter. TypeScript drops undefined from the inferred type of an optional parameter, so only overloads with a required options parameter, for every form of create(), could do it.
• Item 4: an own-property check on every path segment would close it for objects, arrays and Sets; on Set positions alone, it made mutable applications through a Set 3–6% slower in fix: harden apply() for Set positions and mutable root replacement, and fix development checks and recipe types #201.
• A state type that includes Promises, such as object or unknown, cannot tell an async recipe from one that returns a Promise as the new state, since TypeScript infers no recipe type once the call gives the state type. With patches, the result is then typed as the tuple.
• An explicit state type that is a type parameter, as in create<T>(base, async (draft) => { … }) in a generic function, still gives T, as in 1.3.0; leaving out the type argument gives Promise<T>. docs: name the state types that keep async results typed as the state, fix the JSON Patch note and state the TypeScript versions #203 documents it. Two changes to the overloads were tried: async overloads with a plain base type before the overloads that infer the state type fix those calls, but also changed 10 of the other 152 types that the probes check in TypeScript 5.0.4–5.9.3, such as create(state, (draft) => { draft.count++; }), typed DraftedObject<State> without strict mode, and create(0, (n) => n + 1), rejected in strict mode; giving the inferring overloads a never return by default broke five existing type checks.

Size

The production CJS artifact grows from 27,375 B to 27,437 B raw and from 8,449 B to 8,472 B Brotli; the UMD and ESM production artifacts grow by 28 B and 31 B Brotli. size-limit measures 8,640 B for it (8,615 B on main), 7,575 B for an ESM bundle of create (7,546 B) and 8,515 B for all ESM exports (8,485 B), within the limits of 8.7, 7.7 and 8.6 kB. Consumer bundles built with esbuild grow by 27–73 B Brotli, and the development builds by 149–153 B raw. Commit 2 changes types only: the declaration of makeCreator grows from 4,064 B to 4,554 B.

Performance

Production CJS builds of 9f93f16, before #200, of main and of this PR, outside the suite, each in processes of its own, three rounds in rotated order, µs per update of shift() and eight reads near the start; a frozen input is a frozen base state, and a frozen result adds auto-freeze:

Rows	Layout	Frozen	9f93f16	main	This PR
10,000	no repeated object	no	138	27.9	27.9
10,000	no repeated object	input and result	2,049	198	199
10,000	two objects in turn	no	10.3	581	13.6
10,000	two objects in turn	input and result	182	745	187
100,000	two objects in turn	no	142	5,840	145
100,000	two objects in turn	input and result	1,905	7,539	1,897
10,000	16 objects in turn	no	10.5	93.5	14.3
100,000	16 objects in turn	no	141	973	145
10,000	each object twice	input and result	1,120	191	191
10,000	two objects in turn in the first 20 rows	input and result	2,060	201	199
10,000	two objects in turn in the first 100 rows	no	137	33.2	30.9
10,000	two objects in turn in the first 100 rows	input	1,897	62.7	64.4
10,000	two objects in turn in the first 100 rows	input and result	2,041	205	206
100,000	two objects in turn in the first 100 rows	input	19,181	617	623
100,000	two objects in turn in the first 100 rows	input and result	20,793	2,091	2,086
10,000	two objects in turn in the first half	no	74.6	303	58.0
10,000	two objects in turn in the first half	input	985	330	655
10,000	two objects in turn in the first half	input and result	1,128	472	797

At 10,000 rows with auto-freeze off, in Chromium 148, Firefox 150 and WebKit 26.4, this PR took at most about 1 µs more than commit 1 with no repeated object, two objects in turn, each object twice, and two objects in turn in the first 100 rows or the first half, frozen or not, and 44 µs instead of 1,553 µs in Chromium for the first 100 rows on a frozen base. The suite has no scenario with repeated objects; its moved-row cells, such as shift-and-update, take the code path of unique elements.

The paired performance budgets of CI on the head compared 146 latency cells in five groups: geometric mean 0.999, the slowest cells 1.055 (return-replace with patches and without freezing, the slowest cell on commit 8 too, whose code this PR does not change) and 1.046 (search-current-shifted with freezing and patches, 0.968–1.003 in its other modes), shift-and-update 0.983–1.015, the array move cells 0.972–1.018, and the apply-* cells 0.978–1.009.

Type-checking the built declarations with TypeScript 5.8, 150 calls with distinct state types per shape: direct and curried calls with an object state type take the same number of instantiations as on main, and inferred calls 8 more. A recipe that may return a Promise takes 3% more with an object state type (30,126 instead of 29,223); with a primitive state type, synchronous recipes take 13% more (20,455 instead of 18,064), curried ones 22% more (16,552 instead of 13,548), and recipes that may return a Promise, now typed correctly, 29,154 instead of 18,064. Recipes that return a literal, as in create<'a' | 'b'>('a', () => 'b'), take 58% more (28,577 instead of 18,054), and curried ones 73% more (23,461 instead of 13,538); checking 150 such calls took 0.08–0.09 s instead of 0.07 s.

Verification

• The local CI sequence passes on the head: lint, format, build, the benchmark checks, test:benchmarks, size, test:package, test:build-watch, type-check and 5,040 tests with full coverage of the 28 source files. Commits 1, 2 and 9 pass type-check, lint, format and the tests on their own.
• Item 1: the counting test fails on main, and the test of commit 9 on commit 8. 20,000 random recipes of shift, unshift, splice, reverse, push and pop with reads, edits, assignments and searches over arrays of up to 70 elements drawn from one to four repeated objects, frozen or not, with or without auto-freeze and with both path formats, gave the same results, return values, patches, inverse patches and replays as main; 907 of them searched after the 16th copy.
• Item 2: the four new type assertions fail on main. 76 call shapes, checked on TypeScript 4.8.4, 5.0.4, 5.8.3 and 6.0.3 with and without strict mode, change only for the seven shapes with a primitive state type and a recipe that may return a Promise; TypeScript 7.0.2 gives the same types for them. 47 other call shapes, on TypeScript 5.1.6, 5.8.3 and 5.9.3, change only for the 12 with a primitive state type and a recipe that may return a Promise.
• Items 3 and 4: with TypeScript 5.8.3, options typed { enablePatches: boolean } | undefined give the union with create<State, false, boolean> and create<State, [], false, boolean>, and create<State> with them does not compile. A patch through a Map key __proto__ or constructor applies; one through an object's __proto__ throws.

[github-actions]

Coverage after merging fix/repeated-elements-and-primitive-types into main will be

100.00%

Coverage Report

File	Stmts	Branches	Funcs	Lines	Uncovered Lines
src
apply.ts	100%	100%	100%	100%
array.ts	100%	100%	100%	100%
constant.ts	100%	100%	100%	100%
create.ts	100%	100%	100%	100%
current.ts	100%	100%	100%	100%
draft.ts	100%	100%	100%	100%
draftify.ts	100%	100%	100%	100%
error.ts	100%	100%	100%	100%
index.ts	100%	100%	100%	100%
interface.ts	100%	100%	100%	100%
internal.ts	100%	100%	100%	100%
makeCreator.ts	100%	100%	100%	100%
map.ts	100%	100%	100%	100%
original.ts	100%	100%	100%	100%
patch.ts	100%	100%	100%	100%
rawReturn.ts	100%	100%	100%	100%
set.ts	100%	100%	100%	100%
unsafe.ts	100%	100%	100%	100%
src/utils
cast.ts	100%	100%	100%	100%
copy.ts	100%	100%	100%	100%
deepFreeze.ts	100%	100%	100%	100%
draft.ts	100%	100%	100%	100%
finalize.ts	100%	100%	100%	100%
forEach.ts	100%	100%	100%	100%
index.ts	100%	100%	100%	100%
mark.ts	100%	100%	100%	100%
marker.ts	100%	100%	100%	100%
proto.ts	100%	100%	100%	100%

[github-actions]

Coverage after merging fix/repeated-elements-and-primitive-types into main will be

100.00%

Coverage Report

File	Stmts	Branches	Funcs	Lines	Uncovered Lines
src
apply.ts	100%	100%	100%	100%
array.ts	100%	100%	100%	100%
constant.ts	100%	100%	100%	100%
create.ts	100%	100%	100%	100%
current.ts	100%	100%	100%	100%
draft.ts	100%	100%	100%	100%
draftify.ts	100%	100%	100%	100%
error.ts	100%	100%	100%	100%
index.ts	100%	100%	100%	100%
interface.ts	100%	100%	100%	100%
internal.ts	100%	100%	100%	100%
makeCreator.ts	100%	100%	100%	100%
map.ts	100%	100%	100%	100%
original.ts	100%	100%	100%	100%
patch.ts	100%	100%	100%	100%
rawReturn.ts	100%	100%	100%	100%
set.ts	100%	100%	100%	100%
unsafe.ts	100%	100%	100%	100%
src/utils
cast.ts	100%	100%	100%	100%
copy.ts	100%	100%	100%	100%
deepFreeze.ts	100%	100%	100%	100%
draft.ts	100%	100%	100%	100%
finalize.ts	100%	100%	100%	100%
forEach.ts	100%	100%	100%	100%
index.ts	100%	100%	100%	100%
mark.ts	100%	100%	100%	100%
marker.ts	100%	100%	100%	100%
proto.ts	100%	100%	100%	100%