Skip to content

Reduce the size of the Windows shim#148

Open
ofek wants to merge 1 commit into
facebook:mainfrom
ofek:shim-optimize-size
Open

Reduce the size of the Windows shim#148
ofek wants to merge 1 commit into
facebook:mainfrom
ofek:shim-optimize-size

Conversation

@ofek

@ofek ofek commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Reduce both checked-in Windows shim binaries from 4,096 to 2,560 bytes.

This also adds a checked-in, generated 69-byte PE stub and exact artifact-size regression tests.

Size impact

Changes are cumulative; each delta is relative to the preceding row.

Checkpoint x86-64 ARM64 Delta per binary
Latest release 4,096 4,096 N/A
API/import cleanups, including temporary HeapFree/CloseHandle removal 4,096 4,096 0
Bounded local UTF-16 copy loop 3,584 3,584 -512
Restore explicit HeapFree before waiting 3,584 3,584 0
strip = "symbols" 3,584 3,584 0
Bundled rust-lld 3,584 3,584 0
Merge .pdata into read-only .rdata 3,072 3,072 -512
Compact PE stub 2,560 2,560 -512

Overall reduction: 1,536 bytes per binary, or 37.5%.

Changes that reduced file size

Bounded local UTF-16 copying: -512 bytes

Replace lstrlenW and lstrcatW with a single bounded copy loop.

This removes two imports and, importantly, brings .rdata below a 512-byte file-alignment boundary. It also fixes the previous terminator-boundary overflow: the command buffer now explicitly reserves one UTF-16 code unit for the terminating null.

Merge .pdata into .rdata: -512 bytes

Use /MERGE:.pdata=.rdata with LLD.

LLD's smaller read-only data layout leaves enough room for the unwind records in the existing .rdata allocation. The PE exception directory still points to all five runtime-function records on both architectures.

Unlike the rejected .pdata=.text experiment, the resulting metadata remains read-only and non-executable.

Compact PE stub: -512 bytes

Generate and check in a minimal valid 69-byte stub and pass it through /STUB.

This moves the PE header to offset 72 and allows all PE headers to fit in one 512-byte file-alignment block, reducing SizeOfHeaders from 1,024 to 512 bytes.

Changes retained despite no immediate file-size reduction

These changes reduced complexity and imports but did not individually cross a file-alignment boundary:

  • Pass NULL directly to GetModuleFileNameW, removing GetModuleHandleW.

  • Truncate the final four .exe code units under the documented shim naming contract, removing PathCchRemoveExtension.

    Extensionless shims were already nonfunctional: the previous implementation passed the PE executable itself to DotSlash as though it were the manifest, and the removed test asserted that this failed. Unconditional truncation therefore changes only how an incorrectly named shim fails, not any working behavior under the documented <DotSlash-file>.exe contract.

  • Quote the manifest path directly, removing PathQuoteSpacesW.

  • Locate the raw argument tail with a local quote-state parser, removing PathGetArgsW.

  • Free the command buffer immediately after CreateProcessW. On failure, capture GetLastError before freeing it so the existing diagnostic remains unchanged. Restoring HeapFree adds one import but does not change the aligned artifact size.

  • Let process teardown close the child handles, removing CloseHandle.

  • Retain strip = "symbols"; it tied without changing the aligned artifact size.

  • Use Rust's bundled rust-lld; switching linkers alone tied at 3,584 bytes, but its smaller .rdata layout enabled the subsequent safe section merge.

  • Ignore the unused default msvcrt request with /NODEFAULTLIB:msvcrt, allowing the CRT-free LLD cross-build.

Together, the source cleanups reduce imports from 19 functions across three DLLs to 12 functions from kernel32.dll.

Resource-cleanup tradeoff

All three options produced 2,560-byte binaries:

Option Imports Runtime effect Decision
Rely entirely on process teardown 11 Keeps the 65,534-byte command buffer and both child handles until the child exits. Rejected because long-running children retain avoidable memory.
Restore only HeapFree 12 Releases the command buffer immediately; process teardown still closes the handles shortly after the required wait and exit-code query. Selected as the meaningful memory improvement with less code and import surface.
Restore HeapFree and CloseHandle 13 Also closes the thread handle promptly and the process handle after querying the exit code. Rejected because it adds code and an import while providing negligible RSS benefit; the process handle must remain open while waiting anyway.

Although restoring both cleanup APIs currently fits within the same aligned file size, restoring only HeapFree leaves more section headroom for future changes.

The section impact is measurable even though every option retains 1,024-byte raw .text and .rdata sections:

Option x86-64 .text / .rdata virtual bytes ARM64 .text / .rdata virtual bytes
Process teardown only 726 / 772 832 / 768
Restore only HeapFree 754 / 804 (+28 / +32) 872 / 796 (+40 / +28)
Restore HeapFree and CloseHandle 776 / 836 (+50 / +64) 896 / 828 (+64 / +60)

The selected option therefore recovers 22 bytes of x86-64 .text, 24 bytes of ARM64 .text, and 32 bytes of .rdata on both architectures compared with restoring both APIs.

You can run the following command to confirm:

llvm-readobj --file-headers --sections --coff-imports <shim>.exe

Experiments not retained

  • Emitting each complete diagnostic with one WriteFile call did not improve the final size.
  • Silent or generic diagnostics, removing differentiated missing-DotSlash handling, and weakening wait-result checks were measured only as lower-bound experiments and reverted to preserve behavior.
  • Leaving the command buffer allocated until process exit removed the HeapFree import but retained a 65,534-byte heap allocation for the lifetime of every waiting shim. Restoring explicit cleanup did not change the 2,560-byte artifact size, so the removal was reverted.
  • A static command buffer removed two imports but tied in total size while adding a writable section.
  • opt-level = "s" and the other strip/profile combinations were not smaller than the selected "z" profile.
  • Disabling LLD tool-version information had no effect.
  • Forced helper inlining increased both binaries from 2,560 to 3,072 bytes.
  • A stack command buffer failed to link because it introduced an unresolved architecture-specific __chkstk dependency.
  • Merging .rdata or .pdata into executable .text worsened section permissions. .pdata=.text reached 3,072 bytes, but executable unwind metadata was rejected.
  • /FIXED /DYNAMICBASE:NO was measured on x86-64 and reverted because disabling ASLR was outside the acceptable security contract and it had no effect on size.

Considered but deliberately not attempted

I didn't pursue changes likely to violate user-visible or platform contracts:

  • Stop waiting for the child or stop propagating its exit status.
  • Disable handle inheritance or alter stdin/stdout/stderr forwarding.
  • Reconstruct the argument list instead of forwarding the raw command-line tail.
  • Switch to lossy ANSI APIs or reduce Unicode support.
  • Reduce the command buffer below the documented CreateProcessW maximum or truncate arguments.
  • Remove required x86-64 or ARM64 unwind metadata.
  • Use nonstandard PE alignment or writable/executable section combinations.
  • Supply custom architecture-specific stack probing merely to support a stack buffer.

Testing

  • All Windows shim tests pass without skips.
  • The final x86-64 shim successfully executes a DotSlash file for jq 1.8.2.
  • Both architectures were structurally inspected for headers, sections, imports, unwind metadata, and security characteristics.
  • ARM64 was cross-built and structurally validated; runtime testing was performed on x86-64.

@meta-cla meta-cla Bot added the cla signed label Jul 15, 2026
@meta-codesync

meta-codesync Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D112049927. (Because this pull request was imported automatically, there will not be any future comments.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant