monty-js: allows maxMemory, allocation, recursion depth, and gc interval limits above u32::MAX#344
monty-js: allows maxMemory, allocation, recursion depth, and gc interval limits above u32::MAX#344sathish-t wants to merge 6 commits intopydantic:mainfrom
Conversation
Was limited to < approx 4 GB due to u32 datatype.
Merging this PR will not alter performance
Comparing Footnotes
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
davidhewitt
left a comment
There was a problem hiding this comment.
Thanks, happy to proceed with this, would like to see the unhappy paths adjusted first.
- convert JS-facing allocation, recursion depth, and gc interval limits to f64-backed fields - make js_number_to_usize fallible and validate non-finite, fractional, negative, and out-of-range values - add extract_limits as the fallible JS -> ResourceLimits conversion helper - replace the From<JsResourceLimits> impl with TryFrom<JsResourceLimits> based on extract_limits - switch call sites from limits.into() to limits.try_into()? - remove the redundant below-u32::MAX memory test - add above-u32::MAX coverage for allocation and memory limits - did not write tests for gc interval, recursion depth above u32::MAX
|
I've also made max allocation, recursion depth and gc intervals able to go above |
davidhewitt
left a comment
There was a problem hiding this comment.
Super, thank you. It'd be great to fix the pre-existing panics please.
- use Duration::try_from_secs_f64 for CLI, Python, and JS duration limits - make CLI limit parsing fallible for invalid --max-duration values - report JS duration limit errors as napi InvalidArg - report Python duration limit errors as ValueError - update doc comments for the new error behavior
- restore mount state before returning on failed limit conversion - handle the error path in both run() and start()
|
Looks like I bit off more than I could chew! I fixed the two pre-existing panics (the python side and the javascript side |
On the javascript side, max memory was limited to approx 4GB (u32::MAX) whereas the Rust and Python limits were larger I believe. I am interested in HPC applications, so wanted to push this higher. I wrote the code with an agent but I reviewed and am happy with it.
Had to convert max_memory:
Option<u32>toOption<f64>as f64s are needed for a clean conversion from thenumberdatatype on the Javascript side. I decided to panic on invalid inputs to maxMemory as that was similar in behaviour toDuration::from_secs_f64in this part of the codebase. I can change toTryInto,Erretc. and avoid a panic if you want.What changed
Ran successfully
This includes the new JS coverage for values below and above u32::MAX.