Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
146 commits
Select commit Hold shift + click to select a range
6ab036b
chore(polars-core): Expose AsyncRuntime::handle()
lwwmanning May 13, 2026
d5dbe4d
feat(polars-vortex): Add crate skeleton for Vortex file-format integr…
lwwmanning May 13, 2026
40f9647
feat(polars-vortex): Wire Vortex variant through IR, DSL, and streami…
lwwmanning May 13, 2026
adf5e21
feat(polars-vortex): DType→Arrow converter, IO decorator, working fil…
lwwmanning May 13, 2026
9c82aad
feat(polars-vortex): end-to-end scan_vortex().collect() — C-ABI zero-…
lwwmanning May 13, 2026
09ed099
feat(polars-vortex): filter pushdown via ColumnPredicates → Vortex Ex…
lwwmanning May 13, 2026
f716ee5
feat(polars-vortex): LIKE-based pushdown for StartsWith / EndsWith
lwwmanning May 13, 2026
e946a89
feat(polars-vortex): cloud reads via polars_io::cloud + ObjectStoreRe…
lwwmanning May 13, 2026
f97ca32
feat(polars-vortex): Python bindings — pl.scan_vortex / pl.read_vortex
lwwmanning May 13, 2026
c00ef36
feat(polars-vortex): eager writer — reverse C-ABI bridge + write_vortex
lwwmanning May 13, 2026
8577792
feat(polars-vortex): streaming sink — LazyFrame::sink_vortex via File…
lwwmanning May 14, 2026
65bf0c5
feat(polars-vortex): pl.set_vortex_cache_bytes(N) — Python segment ca…
lwwmanning May 14, 2026
a0d7a4e
feat(polars-vortex): negative slice pushdown
lwwmanning May 14, 2026
35b4e3f
feat(polars-vortex): df.write_vortex() — eager Python write API
lwwmanning May 14, 2026
24e0890
docs(polars-vortex): detailed README
lwwmanning May 14, 2026
bbe16b3
feat(polars-vortex): cloud sink — lf.sink_vortex("s3://...")
lwwmanning May 14, 2026
73f7075
docs(polars-vortex): cloud sink now supported
lwwmanning May 14, 2026
c607b7e
feat(polars-vortex): surface write options (compression/layout/chunk_…
lwwmanning May 14, 2026
8178016
fix(polars-vortex): write options now actually take effect + roundtri…
lwwmanning May 14, 2026
f73007a
test(polars-vortex): expand roundtrip coverage
lwwmanning May 14, 2026
61570a9
test(polars-vortex): Python-level roundtrip smoke tests
lwwmanning May 14, 2026
287543a
docs(polars-vortex): sync README with current write-options + tests
lwwmanning May 14, 2026
d0ecc82
test(polars-vortex): unit tests for predicate convertor + schema mapping
lwwmanning May 14, 2026
2cd9e4e
chore(polars-vortex): refresh capability comment
lwwmanning May 14, 2026
3fb50da
ci: add polars-vortex to the rust-test matrix
lwwmanning May 14, 2026
33b56a8
refactor(polars-vortex): review cleanups, scalar convertor, transmute…
lwwmanning May 14, 2026
7f34d0a
docs(polars-vortex): refresh README after cleanup wave
lwwmanning May 14, 2026
c124312
fix(polars-vortex): review-surfaced bugs (decimal wraparound, dead ca…
lwwmanning May 14, 2026
3388e10
test(polars-vortex): expand dtype + scalar coverage
lwwmanning May 14, 2026
c2ad43c
fix(polars-vortex): wire deferred review items — mem-engine guard, cl…
lwwmanning May 14, 2026
9a899dc
docs(polars-vortex): refresh README crate layout
lwwmanning May 14, 2026
a6d307d
plan: initial draft from Phase 1.2 exploration
lwwmanning May 15, 2026
4d339c8
plan: resolve Step 1.4 design-tree decisions
lwwmanning May 15, 2026
61ddaf8
plan: transition to executing (start phase 1)
lwwmanning May 15, 2026
ce2a2b9
plan: record phase_entry_sha for phase 1
lwwmanning May 15, 2026
b323ecf
plan: revert executing transition; re-open Step 1.4 decisions for use…
lwwmanning May 15, 2026
622fe01
plan: resolve 4 re-opened Step 1.4 design-tree decisions
lwwmanning May 15, 2026
657c78c
plan: transition to executing (start phase 1)
lwwmanning May 15, 2026
6a0773f
plan: record phase_entry_sha for phase 1
lwwmanning May 15, 2026
83cb927
chore(polars-vortex): migrate workspace dep to crates.io vortex 0.70.0
lwwmanning May 15, 2026
7911984
plan: PR-1.1 awaiting review
lwwmanning May 15, 2026
a21aab4
plan: PR-1.1 gauntlet cycle 1 accepted (zero must-fix)
lwwmanning May 15, 2026
018f2ce
test(polars-vortex): cover DType::Variant bail-arm (review finding)
lwwmanning May 15, 2026
79a9ecd
plan: drop GIT_CONFIG_GLOBAL env-shim from BAN + Verification commands
lwwmanning May 15, 2026
04751d0
plan: defer PR-1.1 Python local verification to CI (review finding)
lwwmanning May 15, 2026
91cf37e
plan: PR-1.1 complete (confidence: high, deferred: 1)
lwwmanning May 15, 2026
e176d1c
chore: exclude .big-plans/ from dprint
lwwmanning May 15, 2026
136021e
plan: start PR-1.2
lwwmanning May 15, 2026
9456412
fix(py-polars): ruff errors in vortex-integration files (4 lints)
lwwmanning May 15, 2026
b897f0b
feat(py-polars): expose VortexCacheMode via cache_mode= parameter
lwwmanning May 15, 2026
6f0fe06
fix(polars-python): make serde_json non-optional to unblock latent cf…
lwwmanning May 15, 2026
2c9abdf
plan: defer in-memory buffer zero-copy from PR-1.2 (not low-effort)
lwwmanning May 15, 2026
69dad1e
style: cargo fmt + dprint markdown emphasis on polars-vortex README
lwwmanning May 15, 2026
57b5660
fix(py-polars): mypy stubs for vortex pyo3 methods + cache_mode resolver
lwwmanning May 15, 2026
343d02a
fix: clippy approx_constant + deny 0BSD license + dprint README align…
lwwmanning May 15, 2026
50c3159
fix(polars-plan): wire polars-vortex's dsl-schema feature + regen sch…
lwwmanning May 15, 2026
530c963
plan: PR-1.2 awaiting review
lwwmanning May 15, 2026
ed36542
plan: PR-1.2 gauntlet cycle 1 rejected (1 must-fix item)
lwwmanning May 15, 2026
b2aeb2b
fix(py-polars): cover bool/float/u64-overflow cache_mode rejection pa…
lwwmanning May 15, 2026
8945bd2
plan: PR-1.2 must-fix #1 resolved — outstanding count 0
lwwmanning May 15, 2026
88e48d7
plan: defer PR-1.2 should-fix #3 (Rust dispatch tighten) to follow-up
lwwmanning May 15, 2026
c42274a
plan: correct PR-1.2 row to reflect actual visitor-fix location (revi…
lwwmanning May 15, 2026
1530bc6
plan: PR-1.2 gauntlet cycle 2 accepted (zero must-fix)
lwwmanning May 15, 2026
bcc35fe
plan: PR-1.2 complete (confidence: high, deferred: 1)
lwwmanning May 15, 2026
9dbb23c
fix: dprint auto-format + add RUSTSEC-2024-0436 (paste unmaintained) …
lwwmanning May 15, 2026
c08765e
fix: clippy lints + typo + unused imports (CI greenup chain continued)
lwwmanning May 15, 2026
e6cdcba
fix: rustfmt blank-line drift from last edit
lwwmanning May 15, 2026
c7740a4
plan: phase-end gauntlet cycle 1 — 6 must-fix items
lwwmanning May 16, 2026
db53a50
phase-Ratify + crates.io transition: end-of-phase review cycle 1 (rej…
lwwmanning May 16, 2026
28cbe64
plan: phase-end reject (Phase 1 cycle 1) — implicated PRs derived
lwwmanning May 16, 2026
3244097
plan: phase-end reject — re-open PR-1.3 (Phase 1 cycle 1)
lwwmanning May 16, 2026
ce79cd3
fix(polars-stream): wrap Vortex sink writer task in AbortOnDropHandle…
lwwmanning May 16, 2026
71a1d7e
plan: PR-1.3 must-fix item resolved — outstanding count 5 (MF-001 Abo…
lwwmanning May 16, 2026
64a444b
fix(polars-stream): forward Vortex sink producer errors via the chunk…
lwwmanning May 16, 2026
55a48b4
plan: PR-1.3 must-fix item resolved — outstanding count 4 (MF-002 pro…
lwwmanning May 16, 2026
33e6494
fix(polars-plan): replace u64 as usize cast on row_count with BAN-com…
lwwmanning May 16, 2026
3a25826
plan: PR-1.3 must-fix item resolved — outstanding count 3 (MF-003 row…
lwwmanning May 16, 2026
5475448
docs(polars-vortex): correct README field name `cache` → `segment_cac…
lwwmanning May 16, 2026
06a861e
plan: PR-1.3 must-fix item resolved — outstanding count 2 (MF-004 REA…
lwwmanning May 16, 2026
75e7db5
docs(polars-vortex): add cache_mode= to README scan_vortex signature …
lwwmanning May 16, 2026
1517de9
plan: PR-1.3 must-fix item resolved — outstanding count 1 (MF-005 REA…
lwwmanning May 16, 2026
31baf0e
fix(polars-stream): document why OnceLock::set Err is discarded in vo…
lwwmanning May 16, 2026
850c06d
plan: PR-1.3 must-fix item resolved — outstanding count 0 (MF-006 Onc…
lwwmanning May 16, 2026
e873e51
plan: PR-1.3 awaiting review
lwwmanning May 16, 2026
e1688a5
fix(polars-stream): resolve `vortex` module path via polars_vortex re…
lwwmanning May 16, 2026
a708bbe
plan: PR-1.3 gauntlet cycle 1 — 2 must-fix items
lwwmanning May 16, 2026
0fd381f
plan: PR-1.3 must-fix item resolved — outstanding count 1 (cycle-1 vo…
lwwmanning May 16, 2026
f408469
fix(polars-stream): align Vortex set_io_metrics with CSV/IPC/NDJSON c…
lwwmanning May 16, 2026
0204474
plan: PR-1.3 must-fix item resolved — outstanding count 0 (cycle-1 MF…
lwwmanning May 16, 2026
297fb7c
fix(polars-vortex): clamp row_block_size u64→usize cast at strategy.r…
lwwmanning May 16, 2026
7018c45
style(polars-stream): cargo fmt vortex sink import-group order (revie…
lwwmanning May 16, 2026
66be601
plan: defer PR-1.3 cycle-1 should-fix items #5 + #6 — total 2 new def…
lwwmanning May 16, 2026
400f3e1
plan: PR-1.3 gauntlet cycle 2 accepted (zero must-fix)
lwwmanning May 16, 2026
4bb97b0
plan: defer PR-1.3 cycle-2 should-fix (pre-existing E0004 build break…
lwwmanning May 16, 2026
791827d
plan: PR-1.3 complete (confidence: high, deferred: 3)
lwwmanning May 16, 2026
8ac3aca
plan: phase-end reject — PR-1.3 must-fix items resolved (Phase 1 cycl…
lwwmanning May 16, 2026
42377c0
plan: re-enter phase 1 end-review (after cycle 1 reject)
lwwmanning May 16, 2026
484c223
plan: phase-end gauntlet cycle 2 accepted (zero must-fix)
lwwmanning May 16, 2026
1d55e10
phase-Ratify + crates.io transition: end-of-phase review cycle 2 (acc…
lwwmanning May 16, 2026
8f96e0a
plan: begin amend phase 1: add PR-1.4 (cycle-2 cleanup of top should-…
lwwmanning May 16, 2026
6a6ccfa
plan: amend phase 1 complete
lwwmanning May 16, 2026
68f9d5c
fix(polars-plan): thread VortexScanOptions::segment_cache through vor…
lwwmanning May 16, 2026
38035b2
refactor(polars-vortex): narrow `pub use ::vortex;` re-export to 5 su…
lwwmanning May 16, 2026
3479c6b
refactor(polars-vortex): inline read::metadata back-compat shim (cycl…
lwwmanning May 16, 2026
2eb4544
plan: correct Deferred work entry on producer/writer error-path deter…
lwwmanning May 16, 2026
e360d70
plan: PR-1.4 awaiting review
lwwmanning May 16, 2026
24f8f5d
plan: PR-1.4 gauntlet cycle 1 — 1 must-fix item
lwwmanning May 16, 2026
5c634f2
fix(polars-plan): drop redundant .clone() on Copy VortexCacheMode at …
lwwmanning May 16, 2026
1e16e1c
plan: PR-1.4 must-fix item resolved — outstanding count 0 (clippy::cl…
lwwmanning May 16, 2026
3fc5ee4
plan: PR-1.4 gauntlet cycle 2 accepted (zero must-fix)
lwwmanning May 16, 2026
7eaf4e2
plan: PR-1.4 complete (confidence: high, deferred: 0)
lwwmanning May 16, 2026
f0e99e4
plan: re-open PR-1.4 for CI fix (rustfmt + typos at phase-boundary)
lwwmanning May 16, 2026
0a71bc2
style(polars-plan,polars-vortex): apply cargo fmt (rustfmt CI green-up)
lwwmanning May 16, 2026
c1dac4b
plan: PR-1.4 awaiting review (CI-reopen)
lwwmanning May 16, 2026
0677ffc
plan: PR-1.4 gauntlet cycle 1 accepted (zero must-fix)
lwwmanning May 16, 2026
1a244a0
plan: PR-1.4 complete (confidence: high, deferred: 0) [CI-reopen]
lwwmanning May 16, 2026
d687da0
plan: phase-Ratify + crates.io transition gauntlet cycle 3 raw JSON a…
lwwmanning May 16, 2026
88560e6
phase-Ratify + crates.io transition: end-of-phase review cycle 3 (rej…
lwwmanning May 16, 2026
00aac72
plan: phase-end reject (Phase 1 cycle 3) — implicated PRs derived
lwwmanning May 16, 2026
c1aa921
plan: phase-end reject — re-open PR-1.4 (Phase 1 cycle 3)
lwwmanning May 16, 2026
d71911f
plan: drop hyphenated form of 'mischaracterizes' from PR-1.4 CI-reope…
lwwmanning May 16, 2026
919ab6d
plan: PR-1.4 must-fix item resolved — outstanding count 1 (plan:404+4…
lwwmanning May 16, 2026
522f413
docs(polars-vortex): drop stale back-compat metadata re-export phrase…
lwwmanning May 16, 2026
da042c1
plan: PR-1.4 must-fix item resolved — outstanding count 0 (README:244…
lwwmanning May 16, 2026
fbd3706
plan: PR-1.4 awaiting review (cycle-3 reject-fix iteration)
lwwmanning May 16, 2026
5f6a008
plan: PR-1.4 gauntlet cycle 1 accepted (zero must-fix) [reject-fix it…
lwwmanning May 16, 2026
6917bef
plan: phase-end reject — PR-1.4 must-fix items resolved (Phase 1 cycl…
lwwmanning May 16, 2026
080e2a0
plan: re-enter phase 1 end-review (after cycle 3 reject)
lwwmanning May 16, 2026
6b808c1
plan: drop hyphenated 'mis-' compound forms (typos CI green-up; mis-s…
lwwmanning May 16, 2026
6848e09
plan: phase-Ratify + crates.io transition gauntlet cycle 4 raw JSON a…
lwwmanning May 16, 2026
652faf7
phase-Ratify + crates.io transition: end-of-phase review cycle 4 (acc…
lwwmanning May 16, 2026
93643dd
feat(polars): PR-1.5 — Vortex Criterion bench harness (Phase 1 ↔ Phas…
lwwmanning May 18, 2026
b451526
style(polars-vortex): dprint fmt README (paragraph reflow)
lwwmanning May 18, 2026
e730c6e
ci(typos): exclude .big-plans/ (workspace-internal plan files, delete…
lwwmanning May 18, 2026
0a0aa5f
feat(polars-vortex,polars-plan,polars-mem-engine,py-polars): absorb f…
lwwmanning May 19, 2026
8b66a9a
docs(polars-stream,polars-vortex): PR-2.0 commit 2 — code-doc carry-f…
lwwmanning May 16, 2026
7eb3172
refactor(polars-vortex,polars-plan,polars-stream): PR-2.0 commit 3 — …
lwwmanning May 16, 2026
31979c9
fix(polars-plan): C-001+C-002 — schema-supplied Dedicated single-cach…
lwwmanning May 16, 2026
ec023cf
test(polars-vortex): C-003 — VortexSegmentCacheRef identity-preservat…
lwwmanning May 16, 2026
89063ca
fix(polars-plan): C2-001 — add segment_cache: None to expand_datasets…
lwwmanning May 16, 2026
3aabb76
style(polars-vortex): apply cargo fmt on C-003 test assert_eq (cycle-…
lwwmanning May 16, 2026
deeaeb1
plan: re-stack restructure — Phase 1++ absorbs Phase 3 + PR-2.0 cleanups
lwwmanning May 19, 2026
d5ad381
plan: refresh Phases and PRs + Implementation status for the 2026-05-…
lwwmanning May 19, 2026
089f771
plan: update PR references after #1 → #3 swap (Phase 2's PR closed an…
lwwmanning May 19, 2026
f4fd0f4
plan: session-wrap Current State refresh (last_user_touchpoint + last…
lwwmanning May 19, 2026
fb7a146
plan: update references after PR #3 → #4 swap + branch rename to vort…
lwwmanning May 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,939 changes: 1,939 additions & 0 deletions .big-plans/vortex-integration.md

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions .github/workflows/test-rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ jobs:
-p polars-sql
-p polars-time
-p polars-utils
-p polars-vortex

- name: Run tests
if: github.ref_name != 'main'
Expand All @@ -77,6 +78,7 @@ jobs:
-p polars-sql
-p polars-time
-p polars-utils
-p polars-vortex

integration-test:
runs-on: ${{ matrix.os }}
Expand Down
1 change: 1 addition & 0 deletions .typos.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
[files]
extend-exclude = [
".git/",
".big-plans/",
"*.csv",
"*.gz",
"dists.dss",
Expand Down
Loading