Skip to content
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions docs/guides/native/go-binaries.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,13 @@ against the table. The result keeps the universal file as `artifact` (its
SHA-256 and size), adds `selected_slice` with the slice offset, length, raw CPU
type/subtype and the FAT table digest, and reports every source range as a
file offset inside that slice. Without `architecture`, or when the file has no
such slice or several, the request fails on `architecture` and lists the slices
present. `architecture` on any other image is rejected rather than ignored.
such slice, the request fails on `architecture` and lists the slices present.
When several slices share the CPU family, REA reads the one `inspect_macho`
would select: `arm64` before `arm64e` before `arm64v8`, the canonical `i386`
and `x86_64` subtypes before variants such as `x86_64h`, and FAT table order
among subtypes with no listed preference. `limitations` names the same-family slices it did not read.
Slices that share a base subtype still fail on `architecture`.
Comment thread
chordpli marked this conversation as resolved.
Resolved
`architecture` on any other image is rejected rather than ignored.

For ELF images without a named build-info section, discovery checks each
nonempty writable, non-executable load segment in program-header order. This
Expand Down
9 changes: 8 additions & 1 deletion docs/guides/native/native-initializers.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,14 @@ requires an absolute filesystem path:
}
```

A universal Mach-O requires `architecture` to select one slice. ELF files and
A universal Mach-O requires `architecture` to select one slice. When several
slices share the CPU family, such as the `arm64` and `arm64e` slices of stock
macOS tools, REA reads the one `inspect_macho` would select (`arm64` before
`arm64e` before `arm64v8`, the canonical `i386` and `x86_64` subtypes before
variants such as `x86_64h`, and FAT table order among subtypes with no listed
preference) and reports it in `slice`. `limitations`
names the same-family slices it did not read. Slices that share a base subtype
fail on `architecture`. ELF files and
thin Mach-O files do not take it.

## Sources
Expand Down
9 changes: 7 additions & 2 deletions docs/rust-binaries.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,8 +168,13 @@ against the table. The result keeps the universal file as `artifact` (its
SHA-256 and size), adds `selected_slice` with the slice offset, length, raw CPU
type/subtype and the FAT table digest, and reports the section and rustc-path offsets as a
file offset inside that slice. Without `architecture`, or when the file has no
such slice or several, the request fails on `architecture` and lists the slices
present. `architecture` on any other image is rejected rather than ignored.
such slice, the request fails on `architecture` and lists the slices present.
When several slices share the CPU family, REA reads the one `inspect_macho`
would select: `arm64` before `arm64e` before `arm64v8`, the canonical `i386`
and `x86_64` subtypes before variants such as `x86_64h`, and FAT table order
among subtypes with no listed preference. `limitations` names the same-family slices it did not read.
Slices that share a base subtype still fail on `architecture`.
`architecture` on any other image is rejected rather than ignored.

Package and dependency counts are checked against the minimum required JSON
representation before graph allocation or schema cloning. Oversized graphs
Expand Down
2 changes: 1 addition & 1 deletion src/domain/apple/machoTargetSelection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ export const MACHO_CPU_TYPES = {
export const machoSliceArchitectureSchema = z
.enum(["x86", "x86_64", "arm", "arm64"])
.describe(
"CPU family of the slice to read from a universal (FAT) Mach-O. Required for universal files and rejected for every other image.",
"CPU family of the slice to read from a universal (FAT) Mach-O. Required for universal files and rejected for every other image. When several slices share the family, the preferred subtype is read (arm64 before arm64e).",
) satisfies z.ZodType<keyof typeof MACHO_CPU_TYPES>;
export type MachoSliceArchitecture = z.infer<
typeof machoSliceArchitectureSchema
Expand Down
15 changes: 9 additions & 6 deletions src/go/GoBinaryProvider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -331,12 +331,15 @@ export class GoBinaryProvider implements GoBinaryPort {
...(stringLiterals === undefined
? {}
: { string_literals: stringLiterals }),
limitations: limitationsFor(
info,
sectionNamesLimitation,
stringLiterals,
machoChainedFixups,
),
limitations: [
...limitationsFor(
info,
sectionNamesLimitation,
stringLiterals,
machoChainedFixups,
),
...selected.limitations,
],
};
if (Buffer.byteLength(JSON.stringify(report), "utf8") > OUTPUT_BYTES)
throw outputLimit(input.path);
Expand Down
36 changes: 12 additions & 24 deletions src/native/AppleMachoSelection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,8 @@ import type { FixupCommands } from "./AppleMachoFixups.js";
import { decodeMachoFixedName, decodeMachoName } from "./AppleMachoNames.js";
import {
MACHO_ARCHITECTURE_POLICIES,
machoSubtypeRank,
selectPreferredMachoSlice,
} from "./MachoArchitectureSelection.js";
import { machoCpuSubtypeBase } from "../domain/apple/machoCpuIdentity.js";
import {
readFatSliceDeclarations,
readFatHeaderPrefix,
Expand Down Expand Up @@ -100,39 +99,28 @@ export const selectMachoSlice = (
const { count, tableEnd: headerEnd } = header;
if (count > 128 || headerEnd > bytes.length)
throw malformed("Malformed FAT architecture table");
const policy = MACHO_ARCHITECTURE_POLICIES[architecture];
if (policy === undefined)
// An unknown architecture is refused before the declarations are parsed.
if (MACHO_ARCHITECTURE_POLICIES[architecture] === undefined)
throw invalid("Requested FAT architecture is absent");
const declarations = readFatSliceDeclarations(bytes.subarray(8, headerEnd), {
count,
wide,
littleEndian,
});
if (declarations.status !== "parsed") throw malformed(declarations.reason);
let best: FatSliceDeclaration | undefined;
let bestRank = Number.POSITIVE_INFINITY;
let ambiguous = false;
for (const declaration of declarations.slices) {
const issue = validateFatSlice(declaration, bytes.length, headerEnd);
if (issue !== null) throw malformed(issue);
if (declaration.cpuType !== policy.cpuType) continue;
const rank = machoSubtypeRank(policy, declaration.cpuSubtype);
if (best !== undefined && rank >= bestRank) {
// Equal ranks keep FAT table order unless the slices share a subtype.
if (
rank === bestRank &&
machoCpuSubtypeBase(declaration.cpuSubtype) ===
machoCpuSubtypeBase(best.cpuSubtype)
)
ambiguous = true;
continue;
}
best = declaration;
bestRank = rank;
ambiguous = false;
}
if (best === undefined) throw invalid("Requested FAT architecture is absent");
if (ambiguous) throw invalid("Ambiguous FAT architecture slice");
const preferred = selectPreferredMachoSlice(
declarations.slices,
architecture,
);
if (preferred.status === "absent")
throw invalid("Requested FAT architecture is absent");
if (preferred.status === "ambiguous")
throw invalid("Ambiguous FAT architecture slice");
const best = preferred.slice;
const selected: MachoSlice = {
slice: best.offset,
sliceEnd: best.offset + best.size,
Expand Down
49 changes: 49 additions & 0 deletions src/native/MachoArchitectureSelection.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import { describe, expect, it } from "vitest";

import { selectPreferredMachoSlice } from "./MachoArchitectureSelection.js";

const CPU_ARM = 12;
const CPU_X86_64 = 0x01000007;
const CPU_ARM64 = 0x0100000c;
const slice = (index: number, cpuType: number, cpuSubtype: number) => ({
index,
cpuType,
cpuSubtype,
});

describe("selectPreferredMachoSlice", () => {
it("prefers listed subtypes over FAT table order", () => {
const x86 = [slice(0, CPU_X86_64, 8), slice(1, CPU_X86_64, 3)];
expect(selectPreferredMachoSlice(x86, "x86_64")).toEqual({
status: "selected",
slice: x86[1],
});
const arm64 = [slice(0, CPU_ARM64, 0x80000002), slice(1, CPU_ARM64, 0)];
expect(selectPreferredMachoSlice(arm64, "arm64")).toEqual({
status: "selected",
slice: arm64[1],
});
});

it("keeps FAT table order among subtypes with no listed preference", () => {
// armv7s (11) before armv7 (9): the arm policy lists no subtype order.
const arm = [slice(0, CPU_ARM, 11), slice(1, CPU_ARM, 9)];
expect(selectPreferredMachoSlice(arm, "arm")).toEqual({
status: "selected",
slice: arm[0],
});
});

it("refuses slices that share a base subtype and reports absent families", () => {
const duplicate = [slice(0, CPU_ARM64, 0), slice(1, CPU_ARM64, 0)];
expect(selectPreferredMachoSlice(duplicate, "arm64")).toEqual({
status: "ambiguous",
});
expect(selectPreferredMachoSlice(duplicate, "x86_64")).toEqual({
status: "absent",
});
expect(selectPreferredMachoSlice(duplicate, "sparc")).toEqual({
status: "absent",
});
});
});
46 changes: 45 additions & 1 deletion src/native/MachoArchitectureSelection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ export const MACHO_ARCHITECTURE_POLICIES: Readonly<
};

/** Unlisted subtypes follow the preferred slices in FAT table order. */
export const machoSubtypeRank = (
const machoSubtypeRank = (
policy: MachoArchitecturePolicy,
subtype: number,
): number => {
Expand All @@ -46,6 +46,50 @@ export const machoSubtypeRank = (
return rank < 0 ? policy.subtypes.length : rank;
};

/** The FAT slice one CPU family resolves to, or why none can be chosen. */
export type PreferredMachoSlice<Slice> =
| { readonly status: "selected"; readonly slice: Slice }
| { readonly status: "absent" }
| { readonly status: "ambiguous" };

/**
* Choose the one FAT slice an architecture selects. Lower subtype ranks win;
* equal ranks keep FAT table order unless the slices share a base subtype,
* which leaves no single slice to choose.
*/
export const selectPreferredMachoSlice = <
Slice extends { readonly cpuType: number; readonly cpuSubtype: number },
>(
slices: readonly Slice[],
architecture: string,
): PreferredMachoSlice<Slice> => {
const policy = MACHO_ARCHITECTURE_POLICIES[architecture];
if (policy === undefined) return { status: "absent" };
let best: Slice | undefined;
let bestRank = Number.POSITIVE_INFINITY;
let ambiguous = false;
for (const slice of slices) {
if (slice.cpuType !== policy.cpuType) continue;
const rank = machoSubtypeRank(policy, slice.cpuSubtype);
if (best !== undefined && rank >= bestRank) {
if (
rank === bestRank &&
machoCpuSubtypeBase(slice.cpuSubtype) ===
machoCpuSubtypeBase(best.cpuSubtype)
)
ambiguous = true;
continue;
}
best = slice;
bestRank = rank;
ambiguous = false;
}
if (best === undefined) return { status: "absent" };
return ambiguous
? { status: "ambiguous" }
: { status: "selected", slice: best };
};

/** codesign's architecture name for one FAT CPU identity, when known. */
const codeSignArchitectureName = (
cpuType: number,
Expand Down
6 changes: 5 additions & 1 deletion src/native/initializers/NativeInitializersProvider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,11 @@ export const readNativeInitializers = async (
);
const page = { offset: input.offset, limit: input.limit };
const inspection = await inspectSelected(bytes, selected, page, signal);
return { artifact, ...inspection };
return {
artifact,
...inspection,
limitations: [...inspection.limitations, ...selected.limitations],
};
};

const inspectSelected = async (
Expand Down
33 changes: 24 additions & 9 deletions src/native/metadata/MachoSliceSelection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
} from "../../domain/apple/machoTargetSelection.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { err, ok, type Result } from "../../domain/result.js";
import { selectPreferredMachoSlice } from "../MachoArchitectureSelection.js";

const cpuName = (cpuType: number, cpuSubtype: number): string => {
const name = Object.entries(MACHO_CPU_TYPES).find(
Expand All @@ -29,6 +30,8 @@ export interface SelectedNativeImage {
readonly fileOffset: number;
/** Selected universal slice identity; null when the file is not a universal Mach-O. */
readonly selection: MachoTargetSelection | null;
/** Other slices of the selected CPU family that this inspection does not read. */
readonly limitations: readonly string[];
}

/** Why no image could be selected; `architecture` issues are the caller's to correct. */
Expand All @@ -39,8 +42,10 @@ export interface MachoSliceSelectionIssue {

/**
* Select the thin image to inspect without lipo. A universal Mach-O needs the
* caller's architecture and yields exactly one validated slice; any other file
* is returned whole and rejects an architecture rather than ignoring it.
* caller's architecture and yields the one validated slice picked by the
* subtype preference order (`MACHO_ARCHITECTURE_POLICIES`) that
* `inspect_macho` also uses; any other file is returned whole and rejects an
* architecture rather than ignoring it.
*/
export const selectNativeImage = (
bytes: Buffer,
Expand All @@ -51,7 +56,7 @@ export const selectNativeImage = (
const fat = container.value;
if (fat === null)
return architecture === undefined
? ok({ bytes, fileOffset: 0, selection: null })
? ok({ bytes, fileOffset: 0, selection: null, limitations: [] })
: err({
kind: "architecture",
message: `architecture ${architecture} was selected, but the file is not a universal Mach-O; omit architecture.`,
Expand All @@ -64,20 +69,24 @@ export const selectNativeImage = (
kind: "architecture",
message: `Universal Mach-O holds ${available}; select one architecture.`,
});
const matches = fat.slices.filter(
({ cpuType }) => cpuType === MACHO_CPU_TYPES[architecture],
);
const [slice] = matches;
if (slice === undefined || matches.length > 1)
const preferred = selectPreferredMachoSlice(fat.slices, architecture);
Comment thread
chordpli marked this conversation as resolved.
Resolved
if (preferred.status !== "selected")
return err({
kind: "architecture",
message:
slice === undefined
preferred.status === "absent"
? `Universal Mach-O holds ${available}, not ${architecture}.`
: `Universal Mach-O holds several ${architecture} slices (${available}); no slice was selected.`,
});
const { slice } = preferred;
const selected = readUniversalMachoSlice(bytes, fat, slice);
if (!selected.ok) return selected;
const unread = fat.slices
.filter((other) => other !== slice && other.cpuType === slice.cpuType)
.map(
(other) =>
`${cpuName(other.cpuType, other.cpuSubtype)} at file offset ${String(other.offset)}`,
);
return ok({
bytes: selected.value,
fileOffset: slice.offset,
Expand All @@ -92,6 +101,12 @@ export const selectNativeImage = (
.update(bytes.subarray(0, fat.tableEnd))
.digest("hex"),
},
limitations:
unread.length === 0
? []
: [
`Only the preferred ${architecture} slice (${cpuName(slice.cpuType, slice.cpuSubtype)} at file offset ${String(slice.offset)}) was read; this universal Mach-O also holds ${unread.join(", ")}, which no architecture value selects.`,
],
});
};

Expand Down
2 changes: 1 addition & 1 deletion src/rust/RustBinaryProvider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ export class RustBinaryProvider implements RustBinaryPort {
...(selected.selection === null
? {}
: { selected_slice: selected.selection }),
limitations: limitationsFor(image),
limitations: [...limitationsFor(image), ...selected.limitations],
};
if (
Buffer.byteLength(JSON.stringify(report), "utf8") > RUST_OUTPUT_BYTES
Expand Down
Loading
Loading