perf(pypangraph): parse graphs into typed pydantic models

[ivan-aksamentov]

⚠️ AI-generated contribution. This pull request was implemented by an AI agent. Review the code, tests, and benchmark methodology before merging.

Alternative PRs (mutually exclusive, merge one):

• jsonschema-rs #201: swap the validator, keep dicts; smallest change
• msgspec #202: typed decode; fastest
• -> pydantic #203 (you are here): typed models; most common library

Problem

Pangraph.from_json validates every loaded graph against the JSON schema generated from the Rust types before building the object. On real graphs that validation, not parsing, dominates load time. Issue #200 reports the slowdown.

On the benchmark graph below, parsing the JSON takes about 40 ms while the pure-Python jsonschema pass takes about 2 seconds: the validator walks every node, edit and position in interpreted Python.

This change

Parse and validate the JSON in one call with PangraphData.model_validate_json, building the typed model in pypangraph/model.py in pydantic's compiled core. A malformed document (invalid JSON) is reported as a read failure, distinct from a schema violation. The models carry the schema's constraints (non-negative integers, single-character alt, the strand enum), and the path, block and node collections read the typed models. Pangraph still accepts a plain dict, which is validated with PangraphData.model_validate, so an in-memory dict enforces the same schema as a file.

Why this is the one to merge

• Typed internals on a foundation reviewers already know. pydantic is one of the most widely used Python libraries; the model style, the Field constraints and the error format are familiar, so the code is approachable and easy to extend. The load still drops from about 2 seconds to about 92 ms, a 22x speedup.
• Best-in-class validation errors. When a graph is malformed, pydantic reports the exact field path, the expected type and the offending value, which makes debugging a bad graph far easier than a schema-walker's message.
• The project is already set up for it. pydantic and datamodel-code-generator are already in requirements.txt, and the Makefile already generates a Python model from the schema. This PR fits the direction the repository is already pointed.
• Typed data model, static checks. Attribute access replaces string keys, so a wrong field name is caught statically and editors autocomplete the model, which the dict-based option cannot offer.

If the goal is typed internals with the most familiar, best-supported validation library, this is the strongest choice.

Benchmark

Fixture: packages/pypangraph/tests/data/staph.json.gz (664 blocks, 6817 nodes, 15 paths; 1.81 MB compressed, 9.72 MB decoded). Median of 7 runs on one machine in the project Python container, measured by packages/pypangraph/benchmarks/bench_load. Correctness parity was verified for every engine: each accepts the valid graph and rejects missing-field, wrong-type, negative-value and bad-strand mutations.

Full load (parse and validate together):

loader	parse + validate	speedup
baseline (json + jsonschema)	1989 ms	1x
jsonschema-rs	47 ms	42x
msgspec	17 ms	117x
pydantic (this PR)	92 ms	22x

Per-phase breakdown. Decompression is shared; this PR replaces JSON parsing and validation with a single typed parse.

Shared (unchanged by this PR):

phase	ms
gzip decompress	23

Parse and validate (what this PR changes):

step	ms
json parse + jsonschema validate (baseline)	1988
pydantic parse + validate (this PR)	92

Methodology notes:

• Precompiling the pure-Python jsonschema validator does not help; the cost is the interpreted traversal, not validator construction.
• fastjsonschema was rejected: it errors on the schema's format: uint annotation.
• format: uint is a decorative annotation; the non-negative range is enforced by minimum: 0. No engine asserts on the format string.

Conclusion: validation is about 98% of load time, and every candidate removes it. pydantic folds parsing and validation into one compiled call that emits typed models, trading raw speed against msgspec for a far more common library and richer error messages.

The three alternatives

All three PRs branch from feat/merge and rewrite the same loader; they cannot be combined.

• jsonschema-rs. Swap the validator, keep dicts. One-line loader change, no downstream impact, about 42x. The minimal, lowest-risk fix; keeps the untyped dict data model.
• msgspec. Decode JSON bytes straight into typed structs; validation is part of decoding. Fastest overall (about 117x) and typed, using a smaller, less common library.
• pydantic (this PR). Parse and validate into typed models with model_validate_json. Typed internals from the most widely used validation library, with the clearest error messages, about 22x.

Pick pydantic for typed internals on the most familiar foundation; pick msgspec for maximum speed, or jsonschema-rs for the smallest diff.

Work items

• Add typed pydantic models mirroring the schema in pypangraph/model.py.
• Parse graphs into the models in pypangraph/class_graph.py; accept a dict via model_validate.
• Keep malformed JSON distinct from schema violations in the load error.
• Read the typed models in the path, block, alignment and node collections.
• Declare pydantic in pyproject.toml and the Python container (already in requirements.txt).
• Add parameterized tests locking in the accept/reject contract.
• Add benchmarks/bench_load and a graph-loading doc.

Possible improvements

• Generate the models from the schema with datamodel-codegen (as the Makefile already does for the dataclass example) so they cannot drift from the Rust types.

Verify

./dev/docker/python bash -c 'pip install -e packages/pypangraph pytest && cd packages/pypangraph && python3 -m pytest -q'
./dev/docker/python bash -c 'cd packages/pypangraph && python3 benchmarks/bench_load'