perf: add Arrow-native WKB and EWKB output

[mailletf]

Problem

For 500k precision-7 geohashes, the pure Rust core of decode_many_to_wkb runs in under 4 ms, while the function called from Python takes 45 ms. Over 90% of the wall time is the boundary: list[str] → Vec<String> on the way in, and half a million Python bytes objects on the way out.

That is also why the num_threads knob does nothing here — it parallelises the 4 ms.

Fix

Add decode_many_to_wkb_arrow and decode_many_to_ewkb_arrow, following the existing expand_geohash_mapping_arrow. Strings are read straight from the input's Arrow buffers and the output is built into Arrow buffers, so no Python object exists per row.

Every row serializes to the same width — 93 bytes for WKB, 97 for EWKB — so the values buffer is allocated once for the whole column and filled across threads in place via par_chunks_exact_mut, with offsets computed from the row width. That removes the per-row Vec the list API still pays.

Benchmarks

N = 500,000, medians of 15 runs:

	list API	Arrow
decode_many_to_wkb	45.0 ms	3.1 ms	14.5×
decode_many_to_ewkb	51.8 ms	3.2 ms	16.2×

Even when the data starts as a Python list and has to be converted, list → pa.array → wkb_arrow is 10.5 ms — still 4.3×. Output is byte-identical to the list API in every case.

Notes

• Inputs accept Utf8 or LargeUtf8, so callers need not match whichever offset width the library happens to prefer.
• Null inputs produce null outputs rather than being rejected or silently decoded as an empty string.
• serialize_bbox now wraps write_bbox, which writes into a caller-provided slice. The allocating form stays for the existing list API.
• The list-returning functions are untouched. These are additions, not replacements.

[mailletf]

This change is part of the following stack:

• fix: don't drop multipolygon parts whose seed lands in an earlier part #30
  • perf: cover polygons by descending the geohash tree #31
    • perf: expand geohash sets on packed integers instead of strings #32
      • fix: cap the expansion hop count instead of hanging near the poles #33
        • fix: release the GIL while covering a polygon #34
          • chore: delete the dead handbrake cover #35
            • chore: cargo fmt #36
              • perf: add Arrow-native WKB and EWKB output #37 ◀
                • perf: add Arrow-native encode and coordinate decode #38
                  • perf: optionally dictionary-encode geog_id in the Arrow mapping output #39
                  • perf: cover large polygons across Rayon workers #40

_{Change managed by git-spice.}

[Copilot]

Pull request overview

This PR adds Arrow-native bulk decoding APIs to output fixed-width WKB/EWKB polygon bounding boxes as Arrow LargeBinary, minimizing Python boundary overhead for large columns.

Changes:

• Added decode_many_to_wkb_arrow / decode_many_to_ewkb_arrow PyO3 entry points that accept Utf8 or LargeUtf8 Arrow arrays and produce LargeBinary output.
• Refactored bbox serialization into an in-place writer (write_bbox) and reused it for the existing list-based API via serialize_bbox.
• Added Rust + Python tests and updated Python type stubs to cover the new Arrow APIs.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/lib.rs	Implements Arrow string-column abstraction, fixed-width parallel WKB/EWKB column builder, new PyO3 functions, and in-place bbox writer.
tests/test_arrow_ops.py	Adds PyArrow-focused tests ensuring Arrow output matches the list API, preserves nulls, supports slicing, and validates inputs.
geohash_polygon/__init__.pyi	Adds stubs/docs for the new Arrow decoding functions.
Cargo.toml	Adds arrow-buffer dependency to construct buffers/offsets/nulls for Arrow output.
Cargo.lock	Locks the new arrow-buffer dependency.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[mailletf]

Not fixing this one. width is 93 or 97, so rows * width needs rows above ~1.9e17 to wrap a 64-bit usize — an Arrow array of that length cannot exist, since its own offset buffer would already be orders of magnitude past addressable memory. The vec! below would OOM long before arithmetic became the problem.

[mailletf]

Agreed, added in de54668 (now 0dcf5a8). write_bbox asserts its final write position against out.len(), so adding a field without bumping wkb_width fails a test instead of silently truncating or bleeding into the next row's slot.