perf: add Arrow-native encode and coordinate decode

[mailletf]

Problem

Same boundary cost as #37, for the rest of the bulk API. encode_many, decode_many and decode_many_exactly all spend most of their time building Python objects — 500k strings, or 500k tuples — rather than doing geohash work.

Fix

Arrow twins for all three. encode_many_arrow reuses the fixed-width column trick: every hash is exactly precision characters, so the buffer is allocated once and filled across threads. The decoders write one row-major cell buffer, so each row is touched by exactly one thread, then split it into columns.

Benchmarks

N = 500,000 at precision 7, medians of 15 runs:

	list API	Arrow
encode_many	24.8 ms	3.4 ms	7.3×
decode_many	52.8 ms	1.9 ms	27.8×
decode_many_exactly	68.8 ms	2.8 ms	24.6×

decode_many gains most because a tuple per row is the most expensive thing any of these functions does.

Notes

• The decoders return a RecordBatch rather than an array of tuples — (lng, lat) and (lng, lat, lng_err, lat_err) — which is the shape a dataframe or a DuckDB table actually wants, and keeps each column contiguous.
• Nulls propagate: a null geohash gives null coordinates, and a row is null if either input coordinate is null.
• Precision is validated up front, so encode_many_arrow reports a bad precision as such rather than failing per row.
• The list-returning functions are untouched.

[mailletf]

This change is part of the following stack:

• fix: don't drop multipolygon parts whose seed lands in an earlier part #30
  • perf: cover polygons by descending the geohash tree #31
    • perf: expand geohash sets on packed integers instead of strings #32
      • fix: cap the expansion hop count instead of hanging near the poles #33
        • fix: release the GIL while covering a polygon #34
          • chore: delete the dead handbrake cover #35
            • chore: cargo fmt #36
              • perf: add Arrow-native WKB and EWKB output #37
                • perf: add Arrow-native encode and coordinate decode #38 ◀
                  • perf: optionally dictionary-encode geog_id in the Arrow mapping output #39
                  • perf: cover large polygons across Rayon workers #40

_{Change managed by git-spice.}

[Copilot]

Pull request overview

Adds Arrow-native bulk encode/decode APIs to reduce Python object materialization overhead in the hot path, extending the existing Arrow-focused approach used for WKB/EWKB output.

Changes:

• Added encode_many_arrow producing a LargeUtf8 geohash column from Float64 coordinate arrays.
• Added decode_many_arrow / decode_many_exactly_arrow returning Arrow RecordBatches with decoded coordinate columns.
• Extended Python typing stubs and added PyArrow-based tests to validate parity with list-based APIs, null handling, and input validation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
src/lib.rs	Implements Arrow-native encode/decode functions and shared helpers; registers new PyO3 functions; adds Rust unit tests for the new Arrow column logic.
tests/test_arrow_ops.py	Adds PyArrow integration tests for the new Arrow encode/decode APIs (parity with list APIs, null propagation, and validation).
geohash_polygon/__init__.pyi	Updates the Python type stub to include the new Arrow API entry points and their expected shapes.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[mailletf]

Good call — fixed in f53ff32. The single buffer is now laid out column-major instead of row-major, so each output column is already contiguous and goes to Arrow as a ScalarBuffer window onto it. That removes both the gather pass and the per-column Vec, so peak memory is one copy rather than two.

Threads still split by row: each column is cut into 4096-row windows up front and the windows regrouped by index, which hands every task one disjoint slice of every column.