Add built-in NumPy support by trevorprater · Pull Request #248 · pydantic/monty

trevorprater · 2026-03-07T06:33:34Z

Summary

Adds built-in NumPy support to Monty for sandbox-safe numeric array workloads.

This is implemented inside Monty's runtime, so user code can run ordinary import numpy as np snippets without loading CPython's C-backed NumPy package, exposing host filesystem/network access, or adding general third-party package support. The PR closes the source-derived pure-core backlog without adding host-boundary APIs, unsafe memory access, or out-of-scope Python modules.

This PR now covers:

Core NdArray runtime support with shape, dtype, indexing, conversion, display, arithmetic, comparison, aggregation, manipulation, selection, and iterator-style helpers.
NumPy-style broadcasting infrastructure and public helpers including broadcast_shapes, broadcast_to, broadcast_arrays, and a sandbox-safe materialized broadcast subset.
Real-valued ufunc-like math, comparison, logical, integer, bitwise, tuple-returning, and predicate helpers, with np.ufunc type compatibility for implemented ufunc-like callables.
Dtype aliases, dtype metadata predicates, scalar limit metadata, module-level astype, scalar float formatting, and public np.ndarray / np.dtype compatibility.
Boolean scalar constants, dtype hierarchy predicate markers (generic, number, signedinteger, unsignedinteger, complexfloating, flexible, character), metadata-only dtype/scalar names (complex64, complex128, cdouble, csingle, clongdouble, str_, bytes_, void, object_, datetime64, timedelta64, ScalarType), and pure numeric index-trick helpers (index_exp, s_, mgrid, ogrid, r_, c_).
Shape, indexing, sorting, histogram, quantile, polynomial, unique, display, runtime/config, construction, and memory-overlap helpers that fit Monty's current ndarray model.
Generalized core helpers: block, tensordot, einsum, einsum_path, apply_along_axis, apply_over_axes, piecewise, and pad.
ndarray.flat now has a distinct np.flatiter type marker while remaining backed by Monty's existing materialized 1-D ndarray behavior.
Parser startup optimization for arbitrary filename labels so the enlarged static NumPy string table does not add fixed overhead to tiny programs.
User-facing README examples and support-boundary docs for Monty's built-in NumPy subset.

NumPy Surface Audit

Local audit against NumPy 2.4.2 on PR head ec5da8b:

implemented_in_monty: 445
missing_but_feasible_safe: 0
missing_core_numpy_runtime_infrastructure: 0
missing_marker_or_metadata_only: 0
missing_requires_unsupported_runtime_features: 19
missing_unsafe_or_sandbox_out_of_scope: 15
submodule_api_family_requires_separate_implementation: 16

That means the audit has no remaining names in the categories we can honestly implement inside Monty's current safe core runtime. Remaining public NumPy names are excluded because they require one of:

Host/filesystem or external-memory access, such as loaders/savers, include-path discovery, DLPack, memory maps, or file-backed parsers.
Runtime features Monty does not currently support as real array storage or scalar classes, such as complex, string/bytes, datetime/business-day, matrix/subclass, object-array, callable-wrapper, or polynomial-object semantics.
Separate API families/submodules rather than single core ndarray helpers.

Performance

The first CodSpeed regression was traced to fixed parser startup overhead from probing the enlarged StaticStrings table for arbitrary filename labels, not to NumPy execution itself. Commit 66e1466 routes filename labels through dynamic-only interning while preserving static interning for identifiers, attributes, imports, and literals.

CodSpeed passed on pushed head 1f7d54d, then failed on 5b155a9 with a regression in non-NumPy hot-path benchmarks (dict_comp__monty, fib__monty, and list_comp__monty). Commit 7c0d15e removes the new Value::FlatIter enum arm from the hot value path and represents ndarray.flat as a marked NdArray instead.

Code head 7c0d15e passed the CodSpeed performance gate against main (ffd29f0): 17 benchmarks unchanged and 15 skipped benchmarks using baseline results. Heads 331da18 and 3e7447b were green on GitHub CI/CodSpeed, including Cubic. Current head ec5da8b is green on GitHub CI/CodSpeed, including Cubic.

Latest Commits

ec5da8b Implement NumPy metadata dtype markers
3e7447b Fix NumPy ndarray type display
cd7bcb7 Implement NumPy compatibility markers
331da18 Document built-in NumPy support
7c0d15e Keep flatiter off the Value hot path
5b155a9 Complete NumPy core runtime coverage

Test Plan

Local verification completed on head ec5da8b:

make format-rs
RUSTUP_TOOLCHAIN=nightly make lint-rs
uvx ruff==0.14.7 format --check
uvx ruff==0.14.7 check
RUSTUP_TOOLCHAIN=nightly cargo run -p monty-datatest --features memory-model-checks numpy__aliases
RUSTUP_TOOLCHAIN=nightly cargo run -p monty-datatest --features memory-model-checks numpy__parity
RUSTUP_TOOLCHAIN=nightly RUST_TEST_THREADS=1 make test-cases (954 passed)
python3 playground/numpy_surface_audit.py
git diff --check

Review feedback:

Cubic comments reviewed; valid comments have fixes and replies. The remaining ndarray type-display thread is addressed in 3e7447b while preserving np.ndarray.__name__ == "ndarray".

GitHub verification:

Previous head 1f7d54d: required checks green, including CodSpeed Performance Analysis.
Previous head 5b155a9: functionally green locally, but CodSpeed Performance Analysis failed with a non-NumPy hot-path regression.
Previous head 7c0d15e: CodSpeed Performance Analysis passed against main (ffd29f0) and GitHub CI checks were green.
Previous head 331da18: GitHub CI/CodSpeed checks were green.
Previous head 3e7447b: GitHub CI/CodSpeed checks green.
Current head ec5da8b: GitHub CI/CodSpeed checks green.
External Cubic reviewer passed on the latest pushed head.

Working Document

Missing NumPy feature assessment

codspeed-hq · 2026-03-07T06:38:17Z

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 15 skipped benchmarks¹

_{Comparing trevorprater:feat/numpy-and-pandas (ec5da8b) with main (ffd29f0)}

15 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

codecov · 2026-03-07T06:40:07Z

Codecov Report

❌ Patch coverage is 83.85502% with 98 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/monty/src/bytecode/vm/binary.rs	88.45%	28 Missing and 16 partials ⚠️
crates/monty/src/bytecode/vm/mod.rs	45.94%	15 Missing and 5 partials ⚠️
crates/monty/src/bytecode/vm/compare.rs	81.63%	14 Missing and 4 partials ⚠️
crates/monty/src/value.rs	62.06%	10 Missing and 1 partial ⚠️
crates/monty/src/types/iter.rs	87.50%	1 Missing and 1 partial ⚠️
crates/monty/src/heap_data.rs	90.00%	1 Missing ⚠️
crates/monty/src/intern.rs	80.00%	1 Missing ⚠️
crates/monty/src/modules/mod.rs	75.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

davidhewitt · 2026-03-09T15:32:21Z

Thank you for this PR. As you note, it's absolutely massive. I can understand the use case however I'm also reluctant to add such a huge feature set to the codebase, it's also not a complete feature set of numpy / pandas so I would imagine that merging this will set precedent for:

adding further numpy / pandas functionality not implmented in this PR
adding further popular third-party modules

I worry this will quickly become unsustainable to maintain.

I wonder if there's a few different approaches to explore here:

Do you think implementing the standard library array type will offer any value as a more constrained alternative?
Similarly, is there value in merging the numpy portion without the pandas portion? (My gut tells me pandas is a much more complex project than numpy, however I could be wrong)
As a more hand-wavy alternative, I wonder, is there a way to make it easier for users to customise their Monty with components that they want to add? I don't think we'll ever support loading external Python packages into Monty (?), however I could imagine that users implementing useful reusable pieces for Monty could add a lot of value for their own use cases.

dheerapat · 2026-03-16T16:22:04Z

I certainly would like to help here but my rust knowledge is next to nothing, I done my own sandbox project but it use pyodide which is different approach to monty here https://github.com/auto-medica-labs/vivarium

samuelcolvin · 2026-03-24T00:19:42Z

Hi thanks so much for this @trevorprater I agree with one of @davidhewitt's suggestions: we should add numpy to Monty, but not pandas.

Rationale:

numpy is (AFAIK) in memory only, there's no debate about number writing to file or reading files (if there is some filesystem support, the fact I've never heard of it suggests agents won't want to use it)
it's the only well known such library in the python ecosystem (unlike pandas where there's also polars)
A really good pandas implementation would inevitably need to use a query engine (e.g. datafusion, polars or duckdb) for completeness and performance and that would enormously bloated the binary size - e.g. pydantic-monty is around 6MB today, while polars is from memory around 40MB

LMK what you think?

trevorprater · 2026-03-25T09:39:00Z

Thanks @samuelcolvin, that makes sense. I agree – numpy is a much cleaner fit for Monty’s scope. The pandas surface area is large and would inevitably pull toward needing a real query engine for completeness, which isn’t worth the binary bloat.

I’ll split the PR to numpy-only. Plan is:

Drop all pandas types (DataFrame, Series, GroupBy, string/datetime accessors) and associated tests
Untangle the shared plumbing in heap, heap_data, intern, value, and the VM binary/compare ops
Rebase on current main
Should have the updated PR ready in the next few days.

@davidhewitt – apologies for the slow response, things have been hectic on my end. I really appreciate the thoughtful review and the suggestions around a plugin/extension mechanism (great idea).

davidhewitt · 2026-03-26T12:44:27Z

Great, no worries and no rush. FYI, I recently merged #265 which was a huge refactoring that'll likely hit this PR hard (but makes it MUCH easier to avoid copying data, satisfy borrow checker etc).

Happy to take pings if you need any comments about how to get this PR up-to-date with main for the numpy bindings.

Implements a built-in numpy module with ndarray type for running LLM-generated numeric Python code in the Monty sandbox. Module functions: array, zeros, ones, arange, linspace, sum, mean, min, max, abs, sqrt, log, exp, round, clip, where, maximum, minimum, sort, unique, concatenate, cumsum, dot, ceil, floor, log10, std. NdArray methods: sum, mean, min, max, std, flatten, tolist, copy, sort, argsort, argmin, argmax, all, any, cumsum, reshape, round, clip, dot, astype. Element-wise binary ops (+, -, *, /, //, %, **) and comparisons (==, !=, >, <, >=, <=) between arrays and scalars. All tests verified against real CPython + numpy.

Parity test with ~200 assertions covering every numpy function, method, attribute, binary op, comparison, and edge case — verified against both real CPython+numpy and Monty. Bugs found and fixed: - argmax() returned last max on ties instead of first (numpy returns first) - .T attribute not resolved (single-char "T" uses ASCII interning, not StaticStrings) - Float repr wrote "1.0" instead of numpy's "1." - len() on 2D arrays returned total elements instead of shape[0]

… functions - Pre-check allocation size in np.zeros, np.ones, np.arange, np.linspace before allocating Vec to prevent memory exhaustion from user-controlled sizes - Accept plain list arguments in call_elementwise (np.abs, np.sqrt, etc.) matching real NumPy behavior - Improve reshape error message to include actual size and requested shape - Add tests for elementwise functions on plain lists

- Fix `~` on int arrays to use bitwise NOT (e.g. ~1 = -2) instead of logical NOT - Fix `~` on float arrays to raise TypeError, matching NumPy - Fix `~` on bool arrays to correctly flip True/False with Bool dtype - Track bool vs int dtype in array creation so np.array([True, False]) gets dtype='bool' instead of 'int64' - Validate that np.where x/y array lengths match condition length, raising ValueError on mismatch instead of silently producing inconsistent arrays

NaN/Inf correctness: - Fix min()/max() to propagate NaN (was silently ignoring NaN values) - Fix sort/argsort/unique to put NaN values last, matching NumPy - Fix float repr to use lowercase 'nan'/'inf' instead of Rust's 'NaN'/'inf' Empty array handling: - Fix empty np.array([]) to default to float64 dtype (was int64) 2D array correctness: - Fix tolist() to produce nested lists for 2D+ arrays (was flattening) Dtype promotion: - Track scalar_is_float through binary operations so int_arr * 1.0 correctly promotes to float64 (was staying int64 because 1.0.fract() == 0) - Pass is_float flag from Value::Float through value_to_f64() and all scalar operation dispatch Resource safety: - Add check_array_alloc_size() to np.concatenate (was unchecked) - Use checked_mul in reshape to prevent usize overflow from user input Tests: - Add ~60 new assertions for NaN/Inf, empty arrays, dtype, and 2D ops

…overage Add comprehensive numpy ndarray support covering all operations commonly generated by LLMs. Implementation spans 8 categories: Phase 1 - Math: np.sin, np.cos, np.tan, np.log2, np.power, np.diff Phase 2 - Creation: np.full, np.eye, np.copy, np.empty, np.zeros((m,n)), np.ones((m,n)), np.zeros_like, np.ones_like Phase 3 - Testing: np.isnan, np.isinf, np.isfinite, np.array_equal, np.count_nonzero, np.all, np.any (module-level) Phase 4 - Aggregation: .prod(), np.prod, .var(), np.var, np.median, np.argmin, np.argmax (module-level) Phase 5 - Manipulation: np.reshape, np.transpose (module-level), np.append, np.vstack, np.hstack, np.stack, .ravel() Phase 6 - Indexing: np.nonzero, np.argwhere, fancy indexing with integer arrays, slice indexing (arr[1:3], arr[::2], arr[::-1]) Phase 7 - Utilities: np.tile, np.repeat, np.split, .astype("int32"/"float32"/"int"/"float") Phase 8 - Validation: 560 assertions verified against NumPy 2.x, edge cases for empty arrays, NaN/Inf, single elements All 918 integration tests pass, clippy clean, ref-count-panic clean.

- Validate negative arguments for np.linspace (num), np.tile (reps), np.repeat (repeats), and np.split (sections) — previously these would silently wrap to huge usize values via cast_sign_loss - Fix np.split error message for sections=0 to match NumPy's wording - Use defer_drop! for reps_val in tile/repeat for cleaner ref counting - Update module-level docstring to list all ~40 supported functions - Add doc comments about np.stack/np.hstack 1D-only limitation - Add doc comment about np.array_equal NaN behavior - Add comment about ref-count leak acceptability in call_split (resource exhaustion is terminal per project convention) - Replace temporal "New functions" section comment with descriptive label

Phase 1: Constants (np.pi, np.e, np.inf, np.nan, np.newaxis) and dtype type objects (np.float64, np.int64, np.bool_, np.float32, np.int32). Phase 3: Inverse trig (arcsin, arccos, arctan, arctan2), hyperbolic (sinh, cosh, tanh, arcsinh, arccosh, arctanh), and remaining element-wise math (sign, square, cbrt, reciprocal, log1p, exp2, expm1, deg2rad, rad2deg, degrees, radians, hypot, nan_to_num, fmin, fmax, fmod, rint, fabs, positive, negative). Phase 4: NaN-aware aggregations (nansum, nanmean, nanmin, nanmax, nanstd, nanvar, nanprod, nanmedian, nanargmin, nanargmax, nancumsum, nancumprod) and statistics (ptp, cumprod, percentile, quantile, average). Phase 5: Logical functions (logical_and, logical_or, logical_not, logical_xor, allclose, isclose, isin). Phase 6: Array manipulation (flip, fliplr, flipud, roll, expand_dims, squeeze, ravel, delete, insert, diag, diagonal, trace, flatnonzero, asarray, column_stack, array_split, full_like, empty_like). Phase 7: Sorting/searching/set ops (argsort, searchsorted, extract, intersect1d, union1d, setdiff1d, setxor1d, bincount, digitize). Phase 8: Linear algebra (outer, cross). Phase 9-10: Creation functions (logspace, geomspace, tri, tril, triu, identity, meshgrid, gradient, convolve, correlate, interp, select). Also fixes ref-counting bugs in functions using into_pos_only + .next() pattern, and fixes np.sign to return 0.0 for zero (matching NumPy). Test assertions: 560 → 735 (+175 new).

… ops, matmul Implements the most critical missing operators for ndarray parity: - Bitwise operators (&, |, ^) on bool and int arrays - __setitem__ with int index, bool mask, and slice assignment - __iter__ for iterating over array elements in for loops - __contains__ for 'val in arr' membership testing - In-place operators (+=, -=, *=, /=) for scalar and array operands - @ (matmul) operator and np.matmul function for dot/matrix products Updates the matmul test expectation from NotImplementedError to TypeError since @ is now implemented for ndarray but not for plain int/float. Test assertions: 735 → 779 (+44 new).

… etc.) and attributes (nbytes, itemsize) Adds missing ndarray methods and attributes: - .item() - extract single element as scalar - .cumprod() - cumulative product - .squeeze() - remove size-1 dimensions - .take(indices) - take elements at given indices - .diagonal() - extract diagonal of 2D array - .trace() - sum of diagonal elements - .fill(value) - fill array in-place - .compress(condition) - select elements by bool mask - .swapaxes(a, b) - swap two axes - .nbytes - total bytes (size * 8) - .itemsize - bytes per element (8) Test assertions: 779 → 803 (+24 new).

…+ test assertions - Fix ndarray.sort() to mutate in-place and return None (was creating new array) - Fix slice setitem to handle array RHS values, not just scalars - Add comprehensive test assertions (total now 1002) covering: - In-place sort, setitem with slices/masks/arrays - ndarray methods: item, cumprod, squeeze, take, diagonal, trace, fill, compress, swapaxes - ndarray attributes: nbytes, itemsize - Additional edge cases for existing functions

…nzero() methods - Add FloorDiv, Mod, Pow variants to NdArrayInplaceOp for true in-place mutation - Wire InplaceFloorDiv/InplaceMod/InplacePow to new inplace methods - Add .flat attribute returning flattened 1D copy of the array - Add ndarray.repeat(n) method for element-wise repetition - Add ndarray.nonzero() method returning tuple of index arrays - Total assertions: 1014

# Conflicts: # crates/monty-datatest/src/main.rs # crates/monty/src/bytecode/vm/binary.rs # crates/monty/src/bytecode/vm/compare.rs # crates/monty/src/bytecode/vm/mod.rs # crates/monty/src/heap.rs # crates/monty/src/heap_data.rs # crates/monty/src/intern.rs # crates/monty/src/modules/mod.rs # crates/monty/src/object.rs

cubic-dev-ai

No issues found across 28 files

cubic-dev-ai

No issues found across 29 files

cubic-dev-ai

16 issues found across 29 files

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/monty/test_cases/numpy__arithmetic.py">

<violation number="1" location="crates/monty/test_cases/numpy__arithmetic.py:75">
P2: The test claims to cover in-place addition, but it uses normal addition and reassignment instead, so `__iadd__`/mutation semantics are never tested.</violation>
</file>

<file name="crates/monty/src/types/iter.rs">

<violation number="1" location="crates/monty/src/types/iter.rs:602">
P2: 0-D ndarrays are treated as empty iterables instead of raising `TypeError`, which diverges from NumPy/Python iteration semantics.</violation>
</file>

<file name="crates/monty/src/modules/mod.rs">

<violation number="1" location="crates/monty/src/modules/mod.rs:53">
P2: Adding `Numpy` here renumbers `StandardLib`'s `u8` discriminants, which breaks the bytecode/module-ID ABI for previously compiled or serialized code.</violation>
</file>

<file name="crates/monty/test_cases/numpy__methods.py">

<violation number="1" location="crates/monty/test_cases/numpy__methods.py:62">
P3: `np.abs` on a list of ints should not produce floats; this expected value conflicts with NumPy semantics and will mis-test correct behavior.</violation>
</file>

<file name="crates/monty/src/types/ndarray.rs">

<violation number="1" location="crates/monty/src/types/ndarray.rs:222">
P2: Broadcast allocation accounting omits the result vector that callers allocate after broadcasting, so large broadcasted ops can pass the precheck even when peak memory is ~3x the element count.</violation>

<violation number="2" location="crates/monty/src/types/ndarray.rs:426">
P1: Large ndarray arithmetic results are materialized without a pre-allocation resource check, so broadcasting/scalar ops can allocate untracked Rust Vecs beyond the intended sandbox limits.</violation>

<violation number="3" location="crates/monty/src/types/ndarray.rs:1331">
P1: repeat_array allocates and fills the repeated Vec before checking resource limits, so large inputs can OOM or overflow before enforcement.</violation>

<violation number="4" location="crates/monty/src/types/ndarray.rs:1768">
P2: Fancy indexing accepts float ndarrays via truncation and indexes flattened storage instead of NumPy-style advanced indexing.</violation>

<violation number="5" location="crates/monty/src/types/ndarray.rs:1855">
P1: ndarray slice assignment from another ndarray silently pads/truncates instead of validating the RHS length, which can corrupt data without raising an error.</violation>

<violation number="6" location="crates/monty/src/types/ndarray.rs:2260">
P2: Unsupported numeric arguments are silently coerced to `0.0` instead of raising a type error, so invalid inputs can be accepted and produce incorrect ndarray results.</violation>
</file>

<file name="crates/monty/src/bytecode/vm/binary.rs">

<violation number="1" location="crates/monty/src/bytecode/vm/binary.rs:886">
P2: In-place ndarray ops only compare flat data length, so arrays with different shapes but the same element count are incorrectly treated as compatible and updated elementwise in storage order.</violation>

<violation number="2" location="crates/monty/src/bytecode/vm/binary.rs:917">
P2: In-place ndarray ops mutate element storage without promoting/casting `dtype`, so arrays can hold fractional data while still reporting an integer/bool dtype.</violation>
</file>

<file name="crates/monty/test_cases/numpy__aliases.py">

<violation number="1" location="crates/monty/test_cases/numpy__aliases.py:388">
P2: The test over-asserts NumPy partition behavior by requiring a fully sorted result for `kth=-1`, but partition only guarantees the kth element's position and leaves the rest unordered.</violation>
</file>

<file name="crates/monty/src/types/type.rs">

<violation number="1" location="crates/monty/src/types/type.rs:118">
P2: `Type::NdArray` formats as `ndarray` instead of the documented `numpy.ndarray`.</violation>
</file>

<file name="crates/monty-datatest/src/main.rs">

<violation number="1" location="crates/monty-datatest/src/main.rs:182">
P1: `skip-cpython-windows` is accidentally parsed as `skip-cpython`, causing CPython tests to be skipped on all platforms.</violation>
</file>

<file name="crates/monty/src/value.rs">

<violation number="1" location="crates/monty/src/value.rs:1652">
P2: NdArray membership only accepts Int/Float/Bool needles and returns false for numeric LongInt values, causing false negatives for numerically equal inputs.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

cubic-dev-ai

2 issues found across 16 files (changes from recent commits).

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/monty/test_cases/numpy__parity.py">

<violation number="1" location="crates/monty/test_cases/numpy__parity.py:2079">
P1: These `+=` tests expect int arrays to upcast to float64, but NumPy in-place ufuncs preserve the original dtype and raise on float RHS instead.</violation>
</file>

<file name="crates/monty/src/bytecode/vm/binary.rs">

<violation number="1" location="crates/monty/src/bytecode/vm/binary.rs:1012">
P2: In-place pow still leaves integer dtype unchanged for negative integer exponents, so fractional f64 results can be stored under Int64 and later truncated on readback.</violation>
</file>

_{Tip: Review your code locally with the cubic CLI to iterate faster.}

trevorprater · 2026-05-08T06:56:42Z

@samuelcolvin @davidhewitt, I think this PR is now at the point where I should stop expanding it and ask for project-owner direction on what you want next.

Since the earlier review discussion, I have reshaped this to the scope Sam suggested: built-in NumPy support only, no pandas. The current PR is intended to be a constrained Monty-native NumPy subset, not a general third-party package mechanism and not a bridge to CPython's C-backed NumPy.

Current state:

The implementation runs inside Monty's runtime and does not add host filesystem/network access, external package loading, FFI, unsafe memory access, or a new sandbox escape surface.
It covers the core ndarray/runtime surface: construction, dtype handling, shape/indexing, broadcasting, arithmetic/comparison/logical helpers, reductions, sorting/selection, histogram/quantile helpers, polynomial helpers, unique/display/config helpers, and related compatibility markers.
It now includes dtype/scalar metadata markers such as complex64, complex128, str_, bytes_, void, object_, datetime64, timedelta64, and ScalarType, but it keeps those honest: they are metadata/category tokens, not fake support for complex/string/object/datetime array storage.
The source-derived audit against NumPy 2.4.2 currently reports:

implemented_in_monty: 445
missing_but_feasible_safe: 0
missing_core_numpy_runtime_infrastructure: 0
missing_marker_or_metadata_only: 0
missing_requires_unsupported_runtime_features: 19
missing_unsafe_or_sandbox_out_of_scope: 15
submodule_api_family_requires_separate_implementation: 16

The working assessment is here: https://gist.github.com/trevorprater/ebaec1df6aec69d8c9c8f2579de244eb
All current GitHub checks are green on head ec5da8b, including CodSpeed. Cubic is green. All Cubic inline review comments have either been fixed or explicitly answered, and there are currently zero unresolved review threads.

My opinion: this is the right place to freeze the PR's scope. I do not think it is wise to keep adding more NumPy surface area here. If the project is comfortable owning a constrained built-in NumPy runtime, I think this is now a reasonable "NumPy core support" PR. Further work should be separate PRs, probably in this order:

numpy.exceptions / numpy.typing as low-risk import compatibility.
VFS-only text I/O, if you want loadtxt / savetxt style APIs, with every path going through Monty's virtual filesystem boundary.
A deterministic/snapshot-safe numpy.random policy.
A narrow numpy.linalg subset.

What I need from you at this stage:

Is this constrained NumPy-only scope acceptable in principle for Monty?
If yes, what review path would you prefer from here: review/merge this PR as the core NumPy PR, or split it further before review?
If no, what boundary would you like drawn differently before I do more work?

I am intentionally pausing feature expansion until I know which direction you want.

Thank you

trevorprater force-pushed the feat/numpy-and-pandas branch 2 times, most recently from bf87cce to 1dffa8a Compare March 7, 2026 06:38

trevorprater force-pushed the feat/numpy-and-pandas branch 4 times, most recently from 0b57170 to 6bfc6ce Compare March 7, 2026 07:03

davidhewitt mentioned this pull request Mar 16, 2026

at least should bundled data science tools, numpy, pandas/polars, matplotlib #222

Open

trevorprater added 2 commits March 26, 2026 18:40

trevorprater force-pushed the feat/numpy-and-pandas branch from 6bfc6ce to 529777b Compare March 26, 2026 22:58

trevorprater changed the title ~~Add pandas and numpy support for LLM-generated Python~~ Add numpy ndarray support for LLM-generated Python Mar 26, 2026

trevorprater added 11 commits March 26, 2026 19:16

Fix codespell hits in numpy CI

1252f29

trevorprater added 7 commits May 7, 2026 02:44

Add NumPy dtype category predicates

bddbc7d

Add NumPy polynomial division helpers

c155bbf

Add NumPy iterator index helpers

3797946

Add NumPy dtype info helpers

6b06a18

Add NumPy callable array constructors

13ba23c

Add NumPy scalar metadata exports

9fc607c

Add NumPy fromstring text parser

71a6c82

trevorprater changed the title ~~Add numpy ndarray support for LLM-generated Python~~ Add NumPy ndarray support and complete safe surface audit May 7, 2026

Optimize parser filename interning

66e1466

cubic-dev-ai Bot reviewed May 7, 2026

View reviewed changes

Implement NumPy broadcasting support

649cb7d

trevorprater changed the title ~~Add NumPy ndarray support and complete safe surface audit~~ Add NumPy ndarray and broadcasting support May 7, 2026

cubic-dev-ai Bot reviewed May 7, 2026

View reviewed changes

trevorprater added 4 commits May 7, 2026 19:53

Implement NumPy dtype and float formatting helpers

1481608

Merge remote-tracking branch 'origin/main' into feat/numpy-only

9c9f3fe

Implement NumPy tensor contraction helpers

0575c7b

Implement NumPy block assembly

1f7d54d

cubic-dev-ai Bot reviewed May 8, 2026

View reviewed changes

Complete NumPy core runtime coverage

5b155a9

trevorprater changed the title ~~Add NumPy ndarray and broadcasting support~~ Complete sandbox-safe NumPy core subset May 8, 2026

trevorprater added 2 commits May 7, 2026 21:06

Keep flatiter off the Value hot path

7c0d15e

Document built-in NumPy support

331da18

trevorprater changed the title ~~Complete sandbox-safe NumPy core subset~~ Add built-in NumPy support May 8, 2026

cubic-dev-ai Bot reviewed May 8, 2026

View reviewed changes

Comment thread crates/monty/test_cases/numpy__parity.py

Comment thread crates/monty/src/bytecode/vm/binary.rs

Address NumPy review feedback

31be82e

trevorprater force-pushed the feat/numpy-and-pandas branch from 09ca97e to 31be82e Compare May 8, 2026 03:36

trevorprater added 3 commits May 8, 2026 01:22

Implement NumPy compatibility markers

cd7bcb7

Fix NumPy ndarray type display

3e7447b

Implement NumPy metadata dtype markers

ec5da8b

Conversation

trevorprater commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

NumPy Surface Audit

Performance

Latest Commits

Test Plan

Working Document

Uh oh!

codspeed-hq Bot commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

codecov Bot commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

davidhewitt commented Mar 9, 2026

Uh oh!

dheerapat commented Mar 16, 2026

Uh oh!

samuelcolvin commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trevorprater commented Mar 25, 2026

Uh oh!

davidhewitt commented Mar 26, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

trevorprater commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

trevorprater commented Mar 7, 2026 •

edited

Loading

codspeed-hq Bot commented Mar 7, 2026 •

edited

Loading

codecov Bot commented Mar 7, 2026 •

edited

Loading

samuelcolvin commented Mar 24, 2026 •

edited

Loading

trevorprater commented May 8, 2026 •

edited

Loading