perf: store constraint sign as int8 codes (8x smaller sign array, lossless)

[FBumann]

TODO (author): replace this line with your own intent/motivation before un-drafting.

Note

The following content was generated by AI.

Second follow-up to the int32-labels win (#566). The constraint sign is stored as a numpy <U2 unicode array — 8 bytes/entry for a field with only three canonical values (=, <=, >=). Storing it as int8 category codes (1 byte) is an 8× reduction on an array as large as the vars array, and — unlike the float32 change in #839 — it is exact, with no precision trade-off.

What this does. The sign is stored internally as int8 codes and decoded back to canonical string DataArrays at the public constraint.sign boundary, so existing string consumers are unaffected. Encoding happens at constraint construction; decoding at the .sign accessor and the LP/MPS + indicator export paths. Default is on (opt-out via Model(dtypes={"sign": np.str_}), which restores the exact legacy <U2 storage — kept mainly to A/B measure the saving in CI).

Because it's lossless, this one is a candidate to stay default-on (no opt-in flip planned, unlike #839).

Implementation, raw-sign reader audit & verification

Files

• linopy/constants.py: canonical SIGN_TO_CODE (=→0, <=→1, >=→2) + CODE_TO_SIGN, marked a stable netcdf/pickle format (never renumber).
• linopy/common.py: encode_signs/decode_signs (+ numpy kernels). Encode is idempotent and passes degenerate dtypes through (so Dataset.shift's int8→float+NaN upcast doesn't crash).
• linopy/model.py: DtypeKey = Literal["labels", "sign"], "sign" ∈ {np.int8, np.str_} (default int8), per-key validation, docstrings; decodes the infinite-RHS sign check in add_constraints.
• linopy/constraints.py: encode in Constraint.__init__ and _update_data; decode in Constraint.sign; dtype-aware reindex/reindex_like fill.
• linopy/io.py: indicator LP writer decodes via con.sign.

Raw-sign reader audit (every direct reader of the stored array, and how handled)

• Constraint.sign → decode_signs(self.data.sign).
• Constraint._update_data(sign=…) → encodes before assign.
• Constraint.to_matrix_with_rhs / .flat (deprecated) → read via decoded self.sign.
• io.py indicator writer → con.sign.values. Critical path, verified.
• add_constraints infinite-RHS check → decodes before string compare.
• CSRConstraint._sign intentionally stays strings (built from decoded con.sign, transient at solve), so all solver interfaces, dualization, persistent snapshot/diff (<U1 sense buffer unchanged), and the Constraints collection accessors already consume strings — no change needed.

Verification

• Full suite: 3859 passed, 45 skipped, 0 failed. Targeted: test_dtypes 24, test_io 47/3-skip, test_optimization -k "milp/simple/basic/lp" 944/10-skip (incl. direct-solver LP + polars paths), constraints/dualization/piecewise/indicator/sos all pass.
• mypy + ruff clean.
• Memory sanity (100k-row constraint): stored sign int8, 100000 bytes vs 800000 for <U2 → exactly 8.0×.
• Manual: LP file shows correct <=/>=/=; mixed-sense model solves correctly; netcdf round-trips int8 → correct decoded strings (old <U2 files still load).

On-disk note. Saved netcdf now stores int8 sign; old <U2 files still load (re-encoded on read).

🤖 Generated with Claude Code

[codspeed-hq]

Merging this PR will degrade performance by 21.54%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 1 regressed benchmark
✅ 172 untouched benchmarks
⏩ 173 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Memory	test_build[sos-n=10]	4.5 KB	5.7 KB	-21.54%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing fluxopt:perf/sign-int8 (ba58b0d) with master (c2607b6)}

Footnotes

1. 173 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[FBumann]

Closing this: after profiling it, the win is too small to justify carrying the change. The real memory lever is float coefficients (O(nnz)), tracked in #839 — not the sign.

Note

The following analysis was generated by AI.

Why the payoff doesn't hold up. The constraint sign is O(n_constraints) — one element per row — whereas coeffs/vars are O(nnz) and grow with terms-per-row. So the int8 saving, while an 8× cut on the sign array, is a shrinking fraction of the model:

• coefficient-light model (2 terms/constraint): ~11% of resident array bytes
• realistic model (many terms/row): low single-digit %, trending to negligible

And no benchmark can see it. The saving lives in the resident model. Every current memory benchmark is peak-based — test_build measures construction peak (transient-dominated), test_to_lp/test_to_solver measure export peak — none isolate held-model footprint. Measured with memray (n=400): build peak 121.5 MB (master) vs 121.8 MB (branch), total-allocated even +5 MB from the encode transients.

It also fights the grain. Sign must be strings at every boundary (LP/MPS writer, solver senses), so it needs decode-on-read everywhere. The first cut regressed to_lp peak ~27–38% by materialising a full <U2 array at export on top of the resident codes; that was fixable (feed codes straight into polars, decode only the scalar / map via replace_strict), but it left the change net-neutral in CI rather than a win.

Numbers (basic n=250, one constraint dataset)

array	master	branch	share
coeffs float64	1000 KB	1000 KB	36–43% ← the real target (#839)
rhs float64	500 KB	500 KB	18–22% ← also #839
vars int32	500 KB	500 KB	(#566)
sign	500 KB <U2	62.5 KB int8	this PR
labels int32	250 KB	250 KB	(#566)
dataset total	2.754 MB	2.317 MB	−16%

Whole-model array bytes (n=400): 8.02 → 7.14 MB (−11%). Build peak (memray): neutral.

The branch stays on the fork if anyone wants to revive it (e.g. bundled with a resident-footprint benchmark that would actually show it). For now, focusing memory effort on #839 (float32 coefficients).