feat(labels): demangle the MSVC names a PDB carries - #257
Draft
r0ny123 wants to merge 48 commits into
Draft
Conversation
PdbSymbolProvider was gated on an optional pdbparse import. pdbparse is
GPL-licensed against a BSD-2-Clause project, unmaintained since 2020,
and was never a declared dependency, so a default install had no PDB
support at all. The import failure was reported at debug level, which
the default logging configuration hides, so a PE with a usable PDB
beside it came back with no names and nothing said why.
purepdb replaces it as a declared, permissively-licensed, pure-Python
dependency with no native code. Names now come from three merged record
sources rather than the global symbol stream alone - S_GPROC32/S_LPROC32
procedures, publics, and S_THUNK32 thunks - and each address is resolved
through the section table the symbols are expressed against, including
OMAP-translated images, instead of by recomputing it here.
Most of what remains unnamed after that is not anonymous functions but
fragments. MSVC-style codegen outlines cold paths and EH funclets into
separate chunks, and SMDA reports each chunk as a function of its own. A
function with no symbol at its entry that falls inside some procedure's
[rva, rva + code size) is now labelled from that procedure as
<parent>$+0x<delta>. The label is parent-relative rather than absolute
so that it survives a rebuild and a different load address, which is
what MCRIT's (pic_hash, function_name) dedup depends on. Import thunks
are excluded from being fragment parents, identified by the linker input
the PDB records for the address rather than by the shape of the name.
Names are reported as the PDB stores them. purepdb does not demangle, by
design, so an MSVC-decorated name now reaches the report decorated where
undname would have expanded it. Rust and current MSVC toolchains already
write undecorated names into the PDB, and there is no maintained,
permissively-licensed MSVC demangler to substitute; a decorated name is
also stable, which an approximate expansion would not be.
Silence is the other half of the bug. A .pdb sitting next to the input
while no PDB has been loaded now warns, as does a file supplied as a PDB
that is not one, and as does a PDB that fails to parse.
Measured with disassembleFile(exe, pdb_path=pdb), naming counted over
all reported functions: a Rust x86_64-pc-windows-msvc binary linked by
rust-lld goes from 1 of 477 functions named to 442 (92.7%), 46 of those
from fragment attribution; sqlite3 x64 built by link.exe from 273 of
4045 to 3698 (91.4%), 97 fragments; sqlite3 x86 from 273 of 4103 to 3730
(90.9%), 111 fragments. The function totals move because recovered
symbols also seed candidates. On the Rust binary the 35 that stay
unnamed all lie past the last procedure record, in ranges the PDB
carries no proc, public or contribution for.
_parseOep is dropped. It only ever saw the first 16 bytes of its target
while PE entry-point extraction needs at least 0x40, so it could not
produce a symbol under any input; PeSymbolProvider supplies
original_entry_point.
tests/rust_pe_msvc_i686{,_pdb}_xored is a new fixture pair for this, a
no_std i686 build linked by rust-lld, small enough to carry and paired
so the PDB can be checked against the image it describes.
Closes danielplohmann#230
Every failure mode purepdb has on a real file is an empty result rather than an exception, which reintroduced the silent no-op through a door the pdbparse version never had. A PDB with no section-header stream parses cleanly and resolves every rva to None, so the loop skips all of them and the provider reports zero symbols; because the parse succeeded it also records the path, which suppresses the sidecar warning. A managed (.NET) PDB behaves the same way - its records are keyed by metadata token and have no address to resolve. Both now warn, quoting the reasons purepdb's own diagnose() gives, so the message says which of the two it was rather than only that nothing came back. A PDB carrying publics but no procedure records is the third case, and it degrades rather than fails: names still resolve at entry points, but with no code sizes there is nothing to attribute a fragment to. That is what /DEBUG:FASTLINK and pre-2010 toolchains produce, and it is worth saying out loud because the difference it makes to naming is large. A PDB with both symbols and procedure records stays silent; verified on the x64 Rust and both sqlite fixtures, whose naming is unchanged.
0.3.0 is the first release carrying the API this provider is written against: Function.module and .aliases, PDB.functions(code_publics=...), and diagnose().warnings. 0.2.0 installs cleanly and shares the version number with a much smaller API, so a floor is the only thing that separates them. Verified against the release from PyPI rather than a source checkout. Naming is unchanged on every fixture: 442 of 477 functions on the x64 Rust binary, 3698 of 4045 and 3730 of 4103 on sqlite x64 and x86, 6 of 6 on the i686 Rust build, with 46, 97, 111 and 0 of those from fragment attribution. Runs remain reproducible and no label carries an address.
A PDB stores Rust names mangled, and this provider reported them verbatim, so a function whose only name lives in the PDB was labelled _RNvCs8JfRDQSkzJ8_15rust_pe_symbols4main. The demangler this codebase already applies to ELF, Mach-O and PE symbol tables reaches none of those names: it is driven from a parsed binary, and a PDB never becomes one -- it arrives as a separate file path. Route each recovered name through the same evidence gate the other providers use, so a name is parsed before it is rewritten. A "_R" prefix alone also matches _RTC_Initialize, _ReadFile@20 and _RoInitialize@4, all real names carrying the function flag in a sqlite3 PDB, and corrupting a good name is worse than leaving a mangled one alone. A name that fails to parse, demangles to an empty path, or raises keeps the spelling the PDB gave it rather than costing the remaining symbols. Fragment labels follow the entry point they are attributed to, so a fragment inside a Rust procedure reads rust_pe_symbols::main$+0x60 instead of carrying the mangled parent. MSVC decoration stays untouched: ?mangled@@yaxxz still comes back as the PDB spells it, since there is no maintained pure-Python undname to substitute. Validated against real PDBs. Every name this rewrites is byte-identical to rustc-demangle 0.1.28, and none of the 7221 names in the sqlite3 x86/x64 PDBs are altered. On a rust-lld/MSVC x64 image only 20 of 399 functions arrive mangled, because rustc writes readable names into its proc records and the mangled spelling survives as a public that merges into an alias; a PDB carrying only publics sees all of them.
Demangled names were passed through a port of Ghidra's CondensedString before being stored, so <Error as core::fmt::Display>::fmt reached the report as <Error_as_core::fmt::Display>::fmt and A, B became A,B. Ghidra needs that because a Ghidra symbol name cannot contain spaces; SMDA has no such constraint, and already stores names with spaces from another source -- 39 of the 399 names recovered from a PDB on a rust-lld x64 image contain them, because MSVC proc records spell them that way. The condensation was applying one spelling rule to names from a symbol table and another to names from a PDB, in the same report. Removing it exposed a printer bug it had been masking: a function pointer's ABI ran into its fn, giving unsafe extern "C"fn(*mut u8). The closing quote was emitted without the trailing space, and comparisons against the reference implementation could not see it because the condensation deleted that space from both sides. One existing test asserted the wrong spelling and is corrected. Measured against rustc-demangle 0.1.28 on 147 real symbols read out of a rust-lld x64 image: 93 of 147 matched byte for byte before, 146 after dropping the condensation, and 147 after the ABI fix. Cross-checked against the 18 mangled symbols in the reference implementation's own test corpus: 8 match, 9 raise and so keep the name the binary gave them, and 1 differs -- a punycode identifier this port renders as a placeholder, which is pre-existing and left alone here.
src/smda/common/labelprovider/rust_demangler/ is vendored third-party code and carries no attribution. The commit that introduced it says so directly -- it vendors the rust_demangler package by Team bi0s, MIT licensed, with robustness behaviour reimplemented from Ghidra, which is Apache-2.0. Neither is recorded: LICENSE covers only this project's own BSD 2-Clause, there is no NOTICE, and the package has no header. The MIT licence asks that its copyright and permission notice travel with copies and substantial portions, and this code is redistributed in the wheel published to PyPI. Add a NOTICE carrying the MIT copyright line and permission text and a pointer to Apache-2.0 for the Ghidra-derived parts, a docstring on the vendored package pointing at it, and a line in the README credits. The in-code comments this attribution was drawn from named "Ghidra's rust-demangle.c", and no such file exists: Ghidra's Rust demanglers are Java, at Ghidra/Features/Rust/src/main/java/ghidra/app/plugin/core/ analysis/rust/demangler/RustDemanglerV0.java and RustDemanglerLegacy.java. Name those instead, in the NOTICE and in the two comments that were the source of the error, and record that Ghidra's own V0 demangler is a port of the rustc-demangle crate, which is the real upstream of the behaviour. The NOTICE lists every vendored component rather than only the one that prompted it, so Tarjan.py and DominatorTree.py -- both already credited in the README paragraph the new line joins -- appear alongside it. No pyproject change is needed: setuptools' default license-file glob already matches NOTICE*, and a build confirms it reaches both the wheel (dist-info/licenses/NOTICE) and the sdist.
Two reasons a PE reached the report with mangled names. RustSymbolProvider recovered nothing at all from a PE. Its COFF loop skipped a symbol when Symbol.section was None, and lief never populates that attribute for PE -- PeSymbolProvider already says so in a comment and resolves through section_idx instead. On the bundled mingw-linked Rust fixture, 0 of 4818 symbols carry a section while 4366 carry a usable section_idx, so the guard rejected every one, the provider contributed nothing, and PeSymbolProvider's raw spellings won by default: 2098 names reached the report spelled _RNv... Resolving through section_idx recovers those 2098 and none of them stay mangled. PeSymbolProvider never demangled anything. ElfSymbolProvider and MachoSymbolProvider both route recovered names through a demangler, while this one stored them verbatim, so a mingw-built C++ PE reported _ZN12FileExplorerC2Ev rather than a signature. It now applies the same Itanium helper ElfSymbolProvider uses, which leaves every non-Itanium name untouched. Making that COFF loop live also made the gate in front of it matter, and it was the wrong one. _is_rust_symbol tested the prefixes alone, but legacy Rust mangling shares _ZN with the C++ Itanium ABI, and this provider is consulted before the format providers -- so on a Rust binary that also carries C++ symbols it would have claimed names like _ZN4test4funcEv and replaced the full Itanium signature "test::func()" with the "test::func" the Rust legacy demangler degrades it to. It now uses the shared evidence gate, which parses a name before claiming it. Recovery on the fixture is unchanged at 2098 names. No bundled PE carried Itanium C++ symbols to show the second fix on, so add one: a small C++ translation unit compiled for x86_64-w64-mingw32 by g++ 16.2.0. Before the change it yields three mangled names and nothing readable; after it, three readable signatures and nothing mangled. The mocks in testRustSymbolProvider modelled a PE symbol through Symbol.section, which no real PE symbol has, so they exercised a path that could not work on a real binary; they now carry section_idx, and the provider reads it directly rather than through a getattr default that would turn a future lief rename back into "skip every symbol". Several tests used _ZN3foo3barE as their Rust fixture, which has no 17h<hash> suffix and so is Itanium C++ rather than legacy Rust; they now use a name that really is Rust-legacy, and the PE provider test expects foo::bar from the C++ demangler.
MSVC decoration was the one mangling scheme with no demangler here, so a binary built with the Microsoft toolchain reported ?__crt_rotate_pointer_value@@YAIIH@Z where a GCC-built one reported a signature. Add one, and let the PE provider pick a demangler by the decoration a name carries. The contract is that a name comes back either as the reference implementation spells it or exactly as it went in, never as a third spelling. That matters more than coverage does: a decorated name is still a usable identity, matchable against a symbol server or another report, while a confidently wrong expansion matches neither the decorated form nor the real one. Every construct the parser does not fully model therefore declines. A type is built as a tree and spelled around a declarator rather than rendered eagerly, because C nests the two: the name belongs inside its own type. That is what makes int (__stdcall *j)(signed char) and int (* __cdecl f(void))[2] come out right, and it is why a function pointer used as a data declaration, a return type, or the pointee of another pointer are all ordinary cases here instead of refusals. Measured against llvm-undname on the corpus LLVM tests its own demangler with -- names from llvm/test/Demangle/ms-*.test, a compiler stress suite rather than a sample of ordinary binaries -- 363 of 609 are spelled identically, 246 come back untouched, and none get a third spelling. On the MSVC names carried by real PDBs: 10 exact of 14, none wrong. The corpus ships as test data so the demangler is measured against a reference rather than against its own output. Understood: qualified and nested names, name and argument back-references, the basic and extended type set, pointers and references with their own and their pointee's qualifiers, __restrict, arrays, function pointers, declarator composition over all of those, variadic lists, tagged types, templates, data symbols with their storage class, constructors, destructors, operators and the vftable family. Declined: back-references resolved against a table holding a template render, whose numbering does not follow the obvious rule; qualified back-references, which the reference implementation does not form; local-scope and lambda names; RTTI descriptors. The grammar comes from Microsoft's documented format, with llvm-undname settling the cases the documentation leaves ambiguous -- pointer qualifier composition, where PBQAD and PAQAD both spell char *const *; the storage classes 3 and 4, which both render with no prefix; and the spacing around a declarator, where a pointer abuts a name but is separated from a nested function declarator. Both resources a hostile name can spend are bounded. Depth is capped below the point where CPython's own recursion limit could be what stops a parse, so the answer cannot depend on how deep the caller already is. Each rendered type is capped too: argument back-references can reuse an earlier rendering repeatedly, which grew a 132-byte name into half a gigabyte of output before the bound. A symbol table holds whatever bytes were written into it, so every prefix of every corpus name is fed through in the tests and none may raise, malformed shapes each have a case asserting they come back untouched, and a hypothesis target checks that an expansion is printable and bounded rather than merely a string.
Differential fuzzing against llvm-undname 22.1.7 - 15044 names derived by mutating and splicing the reference corpus - found 256 shapes this demangler expanded confidently while the reference refused them. A decorated name is a usable identity, so answering where the grammar does not is the one failure this demangler is built to avoid. Five tightenings take that to 4. The largest class was a missing terminator. A parameter list is closed by the throw specification, so a fixed list ends "@z" and a variadic one "ZZ", but the terminator was consumed only when the parser was not already at end of input. Any name truncated just past its parameters therefore got a plausible answer: "?a2@@Yahx" read as "int __cdecl a2(void)" and "?f@@YAXHZ" as a variadic call whose trailing byte was never there. The extended integer table carried _D through _I. No Microsoft-compatible mangler emits those - __int8, __int16 and __int32 are spelled with the plain char, short and int codes - so a name containing one is not MSVC-decorated. _L and _M stay: clang's Microsoft mangler does emit them for __int128, which is also why they are spelled from the mangler's table rather than the reference's, the reference having no reader for them. The remaining three each restore a rule the grammar states and the parser did not check: a template name is an identifier, not a digit or an operator code; a special name takes a signature or a storage class by which code it is, never both, so a vftable with a parameter list and an operator with a vftable's storage class are equally malformed; and a reference cannot carry a qualifier from the type enclosing it, which had been rendering as "char const &const volatile *". Every change refuses more and expands nothing new. The reference corpus is unchanged at 363 of 609 exact with none given a different spelling, and the MSVC names carried by real PDBs are unchanged at 11 of 14.
A return type is the one position that carries a qualifier of its own, and that qualifier is not optional decoration: every function returning a class by value is spelled with it, "?A" being the unqualified case rather than an absent one. Nothing here parsed the prefix, so those names were refused - "?combine@geometry@@ya?AUMatrix@@aebu2@NI@Z" among them. A return type is parsed at three places: a plain function, a function pointer, and a "$$A6" function type. All three take the prefix, so the three now share one reader rather than each calling type() directly. This came out of building a fixture with the toolchain the demangler is meant to read. No PE available here carries MSVC-decorated names - sqlite3's 277 exports are all C - so the MSVC arm of the PE provider had only synthetic symbols behind it, and its first real export was one it could not read. tests/msvc_cxx_pe_xored is that binary: a small translation unit built for x86_64-pc-windows-msvc by clang-cl 22.1.7, exporting free functions, a namespace, a class returned by value, and an extern "C" name. All four of its decorated exports now come back as signatures. Worth twenty names on the reference corpus, 363 to 383 of 609, none given a different spelling. Twenty-one corpus names used the form the whole time; they sat in the declined bucket, where a single aggregate count kept them from reading as a gap.
The PDB provider had a Rust arm and nothing else, so an MSVC-decorated
name reached the report decorated. That was accurate when the provider
was written -- there was no MSVC demangler here to route it through --
and it no longer is.
Where this bites is narrower than it sounds, and worth stating so the
fixture makes sense. purepdb prefers a procedure record's name, which
the compiler already writes undecorated, so a fully /Z7-built image
needs none of this. The decorated form reaches the name slot when a
function has a public record and no procedure record: a translation
unit built without debug info, a static library shipped without its
PDB, /DEBUG:FASTLINK. Those are the names that arrived as
?measure@text@@YAHPEBDI@Z.
Dispatch keys on the leading "?" ahead of the Rust evidence gate. The
two schemes cannot collide -- no Rust mangling begins with "?" -- and a
name the demangler declines comes back byte-identical, so a local-scope
name, a lambda, an RTTI descriptor or a truncated name keeps the
spelling the PDB holds, and a fragment label built from a declined
parent stays decorated with it.
tests/msvc_cxx_pdb_{pe,}_xored is a clang-cl/lld-link x64 DLL built from
two translation units, one compiled with /Z7 and one without, so the PDB
carries both shapes at once: procedure records that already spell the
name, and publics that do not. Its two decorated publics now reach the
report as signatures, and its procedure records are untouched.
The NUL guard was written because "an expansion holding a control character would travel into the report as a symbol name", and then tested only for NUL. Every other control byte went straight through: "?ctrl\x01@@yaxxz" expanded to "void __cdecl ctrl\x01(void)". An identifier is copied into the answer verbatim, so a control character in the input is a control character in a reported symbol name, and from there in the serialized report. The parser has no reason to reject one on its own -- it reads them as ordinary identifier bytes -- so testing the input is what keeps the answer clean. A decorated name is read from a NUL-terminated string of source-legal characters and cannot hold any of these, so refusing them costs no real name: the reference corpus is unchanged at 381 names expanded.
A PDB string table holds whatever bytes were written into it, and every name in it travelled straight into the report: "bell\x07name" was stored and reported as-is, and as a fragment parent it would have carried the byte into every "$+0x..." label derived from it. The PE provider already refuses a name it cannot vouch for. This applies the same idea, testing for control characters rather than for non-ASCII: a demangled Rust identifier is legitimately non-ASCII, so the printable- ASCII rule the PE provider uses on raw mangled names would throw away real ones here. Empty names are skipped with them, which they always should have been -- an address labelled "" is an address with no label.
Contributor
Author
|
Two additions from auditing the demangling paths end to end. A PDB name now has to be free of control characters to be reported. A PDB string table holds whatever bytes were written into it, and every name travelled straight into the report — The MSVC control-character guard this branch picks up from #255, where the NUL check turned out to cover only NUL. Also measured, while looking for recall gaps: on Re-validated: ruff clean, |
Back-references were the largest thing this demangler could not read. A name table entry was appended for every name met, so any name that used one after a repeat resolved to the wrong entry, and a template made the table unusable altogether: every back-reference inside or after one declined. The mangler's rules are three, and clang's own Microsoft mangler is where they are stated rather than the demangler that reads it. A name is recorded only when the table does not already hold it and while it holds fewer than ten. A template instantiation is written in a fresh scope, which opens before the template's own name, so that name takes index 0 and its arguments are numbered from 1. The rendered template belongs to the enclosing scope instead -- except for the symbol's own name, which is not recorded at all, so "??$f@H@N@@YAXV0@@z" resolves 0 to N. Two shapes that are not names also stopped being answered. "B" was read as a reference introducer, which would make it a volatile-qualified reference -- something C++ cannot write and no mangler emits. And an argument back-reference was read as a type anywhere a type belongs, when it stands for a whole argument and is only one where a whole argument is: "?h@@YAXPAHPA0@Z" got an answer for a name that cannot exist. Measured against llvm-undname 22.1.8. The reference corpus goes from 383 of 609 spelled identically to 415, with none given a different spelling. Differential fuzzing over 15785 names derived from that corpus, on three seeds: wrong answers fall from 4, 5 and 6 to 0, 3 and 2, and the names read rise about nine per cent on each. The five that remain are shapes this demangler already answered before this change, in three families it still models incorrectly; each is recorded with the name that reproduces it. One corpus pair asserted a spelling the reference does not produce -- "?g@@yaxus@@pa0@Z" was recorded as a demangled name although llvm-undname refuses it, which the argument back-reference rule now surfaces. It moves to the declined list.
Differential fuzzing against llvm-undname 22.1.8, widened from three seeds to seven, kept finding names answered where the grammar has no answer, and one class spelled differently from the reference. Four rules close them. The "6" storage form belongs to the vftable family. Every data-special operator was reading it, so "??_Bx@@6b@" got an answer; vcall, typeof and the local static guard take a storage class this parser does not model, and now decline rather than borrow the vftable's. No mangler writes __ptr64 in front of a function type. "P6A" and "R6A" are names, "PE6A" and "RE6A" are not. "$$C" qualifies an array element or a template argument. Written as a parameter of its own, or as the pointee of a pointer or reference, it is not a type: "?f@@Yax$$CBH@Z" was answered. An earlier attempt to require a template scope cost eight real names, because an array element is the other place it belongs. A data symbol's trailing qualifier belongs to what its outermost pointer points at, not to the pointer, and was dropped whenever the type was not a plain one: "?s@@3PADB" is "char const *s", and "?s@@3PAPADB" is "char *const *s". A qualifier the pointee already spells is not spelled twice. Spacing follows from the same source: a pointer or reference sigil abuts a type ending in neither an alphanumeric character nor ">", while a named declarator is spaced off whatever precedes it. "struct S_*" and "enum <unnamed-type-*", but "struct S *" and "enum E$ __cdecl fooE(void)". Across seven fuzz seeds, 110495 derived names: wrong answers fall from 31 to 6, three of them one spacing corner on mutant lambda data names, two the divergences this demangler keeps deliberately -- __int128, which clang's mangler emits and the reference cannot read, and the vector deleting destructor family, which the reference spells with an empty operator name. The reference corpus is unchanged at 415 of 609 exact with none spelled differently.
Every anonymous namespace in an MSVC-built binary is spelled "?A" with a discriminator, and none of them were read: a name qualified by one declined outright, which is why the LLVM sources in the reference corpus were among the names this could not spell. Two details decide it. The discriminator tells two unnamed namespaces apart inside one binary and the reference does not spell it, so both render as `anonymous namespace' - which is what the source looks like too. But it is the discriminator, not that spelling, that a later back-reference resolves to: "?f@?A0x1@@YAXV1@@z" names its parameter "class 0x1". Recording the rendering instead shifts every later index and spells a different name, which is how the mutated LLVM symbols in the fuzz corpus caught it. The fragment is read only where a namespace can appear. A leading "??A" is operator[], and taking it as a namespace turned "??AFoo@@QAGXXZ" into "Foo::`anonymous namespace'(void)". Corpus 415 to 417 of 609 exact, none spelled differently, and about 25 more names read per fuzz seed.
Two constructs ordinary C++ produces, and neither was read. A pointer to member function is how any callback into a class is spelled, and an integer template argument is what std::array<int, 3> and every fixed-size container carry. Between them they account for most of what the reference corpus still declined. A member pointer qualifies its declarator with the class rather than its type -- "void (__thiscall S::*)(void)" -- and the member's own cv follows the parameter list, where a member function keeps it. The pointer's own qualifiers stay on the pointer: "Q8" is "S::*const". The __ptr64 modifier is no more written here than in front of a plain function. An integer is a single digit standing for itself plus one, or nibbles "A" to "P" ended by "@", with a leading "?" for negative. The accumulator is 64 bits and wraps, which the corpus shows: the eighteen nibbles of "$0HPPPPPPPPPPPPPPPPPP@" are 18446744073709551615, and a magnitude that wraps to zero under a minus sign is spelled "-0". Parenthesising the declarator needed an anchored test rather than a substring one. A rendered parameter may itself hold "::*", and treating that as a member pointer put brackets around "std::forward" -- caught on the corpus, which is why the test now only matches a declarator whose own prefix is an owner. Corpus 417 to 457 of 609 exact, none spelled differently, and about 250 more names read per fuzz seed.
A lambda, a function-local static and a local class are all named by the function they sit in, and none of them were read: any name qualified by one declined. They are the largest thing left that an ordinary build emits. The enclosing function is a complete decorated name in its own right, so a second cursor reads it over the same text, and the fragment stops where that name stops - the "@" after it terminates the qualified name it belongs to, not the fragment. The scope number is one less than the one spelled: "?1??f@@yaxxz@" is the second scope of "void __cdecl f(void)". Two rules decide whether the answer is right, and both cost wrong spellings before they were settled against the reference. The enclosing name continues the outer back-reference table rather than opening its own, so "?N@?1??SN@?$NS@H@0@QEAAHXZ@4HA" resolves its "0" to the outer N - reading it in a fresh table spelled that "SN::NS<int>::SN". And the enclosing name is a symbol, so its own leading template is not recorded either, the same exception the outer name gets. Corpus 457 to 484 of 609 exact, none spelled differently. Across seven fuzz seeds the count of wrong answers is unchanged at 7, all in shapes this already answered: the lambda-data spacing corner, the two divergences kept deliberately, and one spliced mutant.
The provider's declined list was written when a name scoped inside a function had no reading, and used one as its example. The demangler reads them now, so the example moves to the demangled side - which is the more useful assertion anyway, since a PDB carries these by the hundred: every function-local static and every lambda.
… scope Three more forms an ordinary build emits, and the largest of what the reference corpus still declined. "__unaligned" qualifies what a pointer points at, and is spelled after that pointee's own const and volatile: "int const __unaligned *". It also travels with them, so a pointer pointing at an unaligned pointer keeps it -- "int __unaligned *__unaligned *" -- which needed the qualifier merge to carry more than const and volatile. A user-defined literal is spelled "operator ""suffix", and its suffix is the identifier following the code, so "??__K_deg@@yaho@Z" is operator ""_deg. Any other double-underscore code still declines. A scope inside a function can be written "@" rather than a digit, and that one is spelled zero -- a digit is spelled one higher than written, so the two forms meet at nothing. Corpus 484 to 491 of 609 exact, none spelled differently, and the count of wrong answers across seven fuzz seeds is unchanged at 7.
"const MyClass instance" reached the report as "class MyClass inst": the trailing qualifier was applied only when the declared type was a plain one, so every class, struct and enum lost it. That is an ordinary declaration, not an exotic one. An array had it worse. The qualifier was appended to the array node's own text, which holds the dimensions rather than a type name, so "?arr@@3QAY01HB" came out as "[2] const *const arr" - a spelling of nothing. That one arrived with the earlier fix for pointers and is the reason this replaces the special cases with one rule. The rule the reference follows: the qualifier belongs to what the symbol declares. A pointer passes it one level in, to what it points at, which is why "?s@@3PADB" is "char const *s" while "?s@@3PAPADB" keeps it on the inner pointer. An array passes it on to its element, the way C spells one, so "?arr@@3QAY01HB" is "int const (*const arr)[2]". Neither the reference corpus nor the fuzz corpus moved: they carry no name of these shapes, which is why the nine spellings this settles are pinned directly instead.
Two forms left over from the ones already read, and between them most of what the reference corpus still declined. A function type written as a template argument may carry what only a member function carries. "$$A8" is that form: it names no class -- the reference refuses one -- so it reads as an ordinary function type with the qualifier appended, and the qualifier is more than cv. A reference qualifier and __restrict are written in front of it and spelled after it, so "GB" is " const &" and "IA" is " __restrict". Each is written at most once, so "HH" is not a name. A scope inside a function carries the same number a template argument does, and it was read as a single digit. Nibbles are just as legal: "?L@" is the eleventh scope, and reading only digits declined every name past the ninth. "A" stays out of that set, because "?A" is the unnamed namespace. Corpus 493 to 509 of 609 exact, none spelled differently, and the count of wrong answers across seven fuzz seeds is unchanged at 7.
The code standing where a pointee's qualifier would be says both that a pointer points into a class and what the member is qualified by, so "PRfoo@@d" is "char const foo::*". As a data symbol the same pointer repeats that qualifier and names its class again by back-reference, which is the form "?m@@3PRfoo@@dr1@" takes. Two shapes stay refused rather than guessed. C++ has no reference to member, so "AT..." is not a name however much it parses like one. And when the member type opens with a qualified pointer the reference drops qualifiers this would keep -- "PQfoo@@SAPEAX" is "void **foo::*" there, not "void *const volatile *foo::*" -- with nothing on the producer side to settle which is right, so that one declines. Corpus 509 to 516 of 609 exact, none spelled differently. Two more fuzz divergences appear, both the declarator-spacing corner already recorded, now reachable through a member pointer written as a data symbol.
Three of them, and between them most of what the reference corpus still declined. __ptr64 stands in front of a data symbol's qualifier, where something is pointed at: "?s@@3PEAHEA" is a name and "?s@@3HEA" is not, since nothing there is pointed at to carry it. A pointer into a class spells its storage the long way even so - the member's qualifier and its class again by back-reference - so "?m@@3PEFRfoo@@der1@" carries both that and the __unaligned in front of it, and the short form the other types use is not a name for it. And a name may carry no signature at all, in which case the linkage is what is being spelled: "?extern_c_func@@9" is extern "C". Nothing follows the marker. Corpus 516 to 524 of 609 exact, none spelled differently. __unaligned was also being dropped when it stood in front of a pointer into a class, which the fuzz corpus caught as three wrong spellings.
A pack separator and an empty pack stand between template arguments
without being arguments themselves, so "f<int, int>" and "f<>" are what
"H$$ZH" and "$$$V" spell.
An array's extent is written the way a template argument's number is,
which reading only digits declined past the ninth: "Y0BE@" is [20].
The two dynamic-initialisation codes name what they run around, and that
name is recorded for later reference, unlike a literal operator's suffix.
Both take a signature rather than a storage class, since they run code.
A name may be mangled although it is extern "C", which "$$J" marks; the
digit after it counts characters of the original mangling and is not
spelled, but it is written, so a marker without one is not a name.
A vftable or vbtable may say which base it is the table for, spelled
{for `D::C'} after it.
A member function pointer written as a data symbol keeps that symbol's
qualifier after its parameters, where a member function keeps it, rather
than on what the pointer points at - which had spelled a name with no
return type at all.
And the declarator spacing had one case the other way round: a
parenthesised pointer declarator abuts the sigil while a function
declarator is separated from it, so "int (__cdecl *(*a)[20])(int, int)"
and "int * (__cdecl *)(int)" are both right.
Corpus 534 to 548 of 609 exact, none spelled differently.
An empty pack has a second spelling, "$$V", and an alias template is
named rather than described: "$$YAliasA@PR20047@@" is the name itself.
A vftable or vbtable may name more than one base, and the reference
strings them together: {for `D::C's `F::E'}. The terminator closes the
list rather than each name in it.
__restrict appears in two more places. On a member function it sits with
the reference qualifier, which is the same set a member function type
carries, so the two now read through one reader. On a data symbol it
qualifies the pointer rather than what is pointed at, and is written
once however many times it is spelled: "?h3@@3QIAHIA" and "?h3@@3QAHIA"
are both "int *const __restrict h3".
Corpus 548 to 558 of 609 exact, none spelled differently.
…f the conventions An operator may be a template, which the name reader refused outright: "??$?HH@S@@QEAAAEAU0@H@Z" is S::operator+<int>. It reads through the same path an identifier template does, since only the name differs. An operator may also leave its return slot empty, the way a constructor does -- every one of them but the conversion operator, whose return is the type it converts to. "$$B" introduces a type as written rather than as a parameter would decay it, so a template argument keeps its extent: "int[2]", and "int[]" where the extent is nothing. Every letter names a calling convention, and most of them are spelled with nothing at all. A convention spelled with nothing still leaves the parentheses a pointer needs, and the space it would have filled: "int ( *)(void)". This had been reading only nine of the twenty-six, declining the rest. Two spacing rules came out of the same names: an array declarator abuts the qualifier before it, so "int *const[5]" rather than "int *const [5]". Corpus 558 to 574 of 609 exact, none spelled differently.
"??R<lambda_1>@@qeba@XZ" was listed as a shape the demangler does not model, and it now reads: an operator may leave its return slot empty. The list keeps a name that is still declined in its place, so what the test pins is unchanged.
A template argument may be the address of a symbol, or the symbol itself. What follows "$1" or "$E" is a complete decorated name, and it is read in the template's own back-reference scope rather than a fresh one or the enclosing one: "??$f@VBar@@$1?x@0@3HA@@yaxxz" resolves its 0 to f, the template's own entry, and not to Bar. Two earlier attempts at this guessed the scope instead of asking for it, and each cost wrong answers. A thunk stands in for a member function and adjusts "this" on the way through. Three forms: an adjustor, whose access letter says how much and which access it carries; a vtordisp, which writes two signed displacements through a virtual base; and a vcall, which names no access at all and carries no parameters, the slot being the whole of it. A vcall name and a vcall thunk require each other, so neither reads alone. A member data pointer may point at a function type as well as at a qualified pointer, which the guard against the second was refusing along with it. Corpus 574 to 592 of 609 exact, none spelled differently.
The last of the forms, and with them the corpus is understood end to end: all 609 names come back spelled as llvm-undname spells them, and the test now pins each one rather than counting how many pass. A string literal is spelled by its contents, not by a placeholder. The length counts the bytes with their terminator, eight characters of hash follow it, and a byte is written plainly, as one of ten punctuation escapes, or as "$" and two nibbles. The wider encodings spell their bytes differently and are left alone. The rest of the RTTI family names a class rather than a type, each with its own storage: the base class descriptor says where the base sits, the array and hierarchy descriptors carry nothing, and the complete object locator is written like a vftable. A conversion operator is named by the type it converts to, which is written in the return slot and so is only known once the signature has been read. Also here: a placeholder type the compiler writes where a deduced one belongs, a name replaced by a hash of itself along with whatever decorated name follows it, the guards around a function-local static and the thread-local form, and a dynamic initialiser whose object is a data symbol written without its leading "?". Two displacements were being read too wide: a vtordisp field is 32 bits, so the value is masked before its sign is taken.
Each rule below was established by probing llvm-undname directly, because
the reference corpus carries no name that exercises it:
- an attribute-spelled calling convention carries a space of its own in
front of a declarator, where __cdecl and __vectorcall carry only the
separator, and a convention spelled with nothing leaves no gap at all
- a member function's modifiers are written __ptr64, __restrict,
__unaligned then a reference qualifier, each at most once, and are
spelled back in that same order
- a vcall is the slot it dispatches through: it carries neither a
qualifier nor a convention of its own, and is never spelled with storage
- "$$B" stands where an argument stands and nowhere a type may nest, and
never over a function type
- an integer is an argument, so it is not an array's element either, and
an array of no dimensions is not a type
- no modifier stands in front of a function type or a member function
pointer
- how far a base class table reaches, and what it is flagged with, are
not negative
- what a dynamic initialiser runs for is a name, and a digit there stands
for an earlier name that does not exist
- a conversion operator reads what it converts to from its return slot,
so it is spelled with a signature and never with storage
A rendering fix comes with them. Whether a declarator was a nested
function was decided by looking for "(" in the rendered text, which
mis-fires on a name that contains one of its own - an operator() name, or
a local scope embedding a signature. The producing site now says so
explicitly. The same class of test appeared once more in the file, an
unanchored "::*" search where the anchored expression six lines below
asks the same question; both now use it.
Corpus stays at 609 of 609 spelled exactly as the reference spells them,
0 given a different spelling. Differential fuzzing over seven seeds and
110495 derived names goes from 57 wrong answers to 2, both of them the
divergences kept deliberately: __int128, which clang's own Microsoft
mangler emits and the reference cannot read back, and the
vector-deleting-destructor family, which it spells with an empty operator
name.
A PDB carries one of these per polymorphic class, and the demangler this provider calls now reads them, so the name moves from the list asserting it comes back decorated to one asserting the spelling.
Contributor
Author
|
Correcting something I asserted above. That comment says the I had the reasoning inverted for several turns and wrote it down before re-checking it against the reference. The demangler now reads string literals properly, in the stack this branch sits on: the length counts the terminator, the eight characters after it are a hash of the bytes rather than part of them, and the bytes themselves are written plainly or as an escape — a digit standing for one of ten punctuation characters, or The part of that paragraph that still holds: they are data publics with no code size, and What changed on this branch itself. With the demangler underneath now reading the whole reference corpus (609 of 609, 0 spelled differently — see #255), an RTTI type descriptor reaching this provider demangles rather than coming back decorated. Re-validated on the merged branch: ruff clean, |
This was referenced Aug 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this exists
#245 rebuilt the PDB provider on purepdb and left MSVC-decorated names decorated, saying so explicitly: "there is no maintained, permissively-licensed MSVC demangler to substitute." #255 then wrote one, and wired it into
PeSymbolProvideronly. This connects the two.Where it actually bites
Narrower than it sounds, and worth stating because it dictates the fixture. purepdb prefers a procedure record's name, which the compiler already writes undecorated — a fully
/Z7-built image needs none of this. The decorated form reaches the name slot when a function has a public record and no procedure record: a translation unit built without debug info, a static library shipped without its PDB,/DEBUG:FASTLINK.Analyzing the new fixture with
disassembleFile(exe, pdb_path=pdb):0x180001000geometry::combinegeometry::combine(proc record, untouched)0x180001064?measure@text@@YAHPEBDI@Zint __cdecl text::measure(char const *, unsigned int)0x18000107f?measure_export@@YAHPEBD@Zint __cdecl measure_export(char const *)How it dispatches
On the leading
?, ahead of the Rust evidence gate — the same keyPeSymbolProvideruses. The two schemes cannot collide, since no Rust mangling begins with?. A name the demangler declines comes back byte-identical, and a fragment label built from a declined parent stays decorated with it. That set has shrunk as #255 progressed: local-scope names, lambdas and RTTI descriptors all read now, so what the tests here pin as declined is a function-local name with no type and a name truncated past its parameter list. Rust names, the undecorated-name path and thegetApicontract are unchanged.Fixture
tests/msvc_cxx_pdb_pe_xored+tests/msvc_cxx_pdb_xored: a clang-cl / lld-linkx86_64-pc-windows-msvcDLL built from two translation units, one compiled with/Z7and one without, so the PDB carries both name shapes at once. Built with/GS- /GR- /O1and linked/DLL /NOENTRY /NODEFAULTLIB /DEBUG, so it links against no CRT and no import library and involves no Microsoft-licensed material; the only stub needed isextern "C" int _fltused = 0;. Xored like the other binary fixtures. 80 KB for the pair, paired so the PDB can be checked against the image it describes.Coverage added: the demangled public and the untouched procedure name off the real PDB, an assertion that no reported name stays decorated, report integration through
Disassembler, and synthetic cases for the fragment labels and the four declined shapes.Also worth knowing
A whole-tree sweep of the class this belongs to — a recovered name reaching a report slot without the demangler for its scheme — found the function-label views consistent after this change, and one genuine defect in the opposite direction on the import side, filed separately as #256.
Validation
ruff check .andruff format --check .clean;make typecheckexits 0 with no new diagnostics;pytest tests/1298 passed, 1 skipped, 2452 subtests;diff-cover --fail-under=100reports 100% of changed lines covered.