Support semver version ranges for dependencies by heatonmatthew · Pull Request #311 · helsing-ai/buffrs

heatonmatthew · 2026-04-18T01:27:04Z

Summary

This addresses and implements #205

Removes the exact-pin-only restriction — dependencies can now be specified with any semver range (^1.0, ~1.2.0, >=1.5.0, =1.0.0, etc.), not just =x.y.z
Adds list_versions() and resolve_version() to Artifactory — queries the registry for all available versions and selects the highest one satisfying the requirement; get_latest_version() is reimplemented on top of this
download() now takes an explicit &Version — resolution is decoupled from downloading; both Artifactory and LocalRegistry receive the resolved version directly
Adds find_satisfying() helpers to lockfile types — PackageLockfile, WorkspaceLockfile, and the Lockfile enum can now look up the best locked version matching a VersionReq, enabling the installer to skip the registry for already-locked ranges
Removes dependency_version_string() from src/registry/mod.rs along with the VersionNotPinned error — the old pin-enforcement code is gone entirely
Documentation — new reference pages for semver range syntax (docs/src/reference/semver.md) and the resolver (docs/src/reference/resolver.md); updated specifying-dependencies.md and buffrs-add.md
Tests — five new workspace-level integration test scenarios: caret/tilde resolution, compatible diamond, incompatible diamond, and multi-level tree; mock registry extended with query-string support to back these tests

What a reviewer should know

The resolver remains single-pass (interleaves graph walking and downloading). Version resolution now works as:

Check the lockfile for a version satisfying the requirement → use it if found
Check the local cache for a matching version → use it if found
Call Artifactory::resolve_version() to query the registry → download the resolved exact version and cache it

Conflict detection (incompatible diamond) is handled in validate_version_compatibility() in src/resolver.rs — unchanged in structure, now exercised by the new test cases.

The query axum feature was enabled in Cargo.toml for the test mock registry to parse search query parameters.

Test plan

cargo build passes (requires protoc)
cargo test --workspace passes (requires protoc + Git LFS)
cargo clippy --all-targets --workspace -- -D warnings -D clippy::all passes
cargo fmt --check --all passes
New integration tests: cargo test --test e2e install covers the five new range scenarios

- Impl is in place but more thorough tests needed to validate

LinaUr · 2026-04-22T00:06:34Z

+Local dependencies do not have a version requirement — the package at that
+path is used as-is. They cannot be mixed with a remote entry for the same
+package name.


Can we keep the removed lines:

Suggested change

Local dependencies do not have a version requirement — the package at that

path is used as-is. They cannot be mixed with a remote entry for the same

package name.

The `path` field is a relative path from the manifest file to the dependency's

root directory (the directory containing the dependency's `Proto.toml`).

Local dependencies do not have a version requirement — the package at that

path is used as-is. They cannot be mixed with a remote entry for the same

package name.

See [Local Dependencies](../guide/local-dependencies.md) for more information.

Good catch. Fixed.

Please check you're happy.

LinaUr · 2026-04-22T01:02:36Z

+        };
+
+        // Phase 1: resolve the requirement to a concrete version.
+        // Try the lockfile first (avoids a registry round-trip when the lock is fresh).


The existing tests/cmd/install/workspace/lockfile/stale/mod.rs test already executes two consecutive installs and verifies the lockfile is reused and extended. However, afaik, it uses exact pins (=0.1.0), so it tests the old Lockfile::get() path with exact version matching, not explicitly covering the new find_satisfying() path with range matching.

Could you add a test for that path please?
For example: run install with ^1.0.0, get 1.2.0 locked, run install again, verify it reuses 1.2.0 from the lockfile without hitting the registry

I've now added tests/cmd/install/workspace/lockfile/range_pin_reuse/mod.rs to test this path.

LinaUr · 2026-04-22T02:05:13Z

            self.validate_compatibility(dependency, existing)
                .wrap_err_with(|| format!("conflicting dependency on {}", package_name))?;
            return Ok(());
        }


I think there might be a bug here.
IIUC, when a package is encountered for the second time (via a different path in the dependency tree), the resolver doesn't re-resolve.

For example, given:

pkg-1 requires lib-a ^1.0.0

pkg-2 requires lib-a <=1.2.0

highest version of lib-a is 1.5.0

If pkg-1 resolves first, the version is 1.5.0.
When pkg-2 tries to resolve, 1.5.0 doesn't satisfy <=1.2.0, so install fails.
However, in this case I'd expect it to resolve to 1.2.0 across the requirements.

If pkg-2 would resolves first, the version would satisfy pkg-1's requirement.

CMIIW, but since DependencyMap is HashMap<PackageName, DependencyManifest>, I believe the dependency iteration order is also non-deterministic. That would make an integration test for the order-dependency inherently flaky.

It's worth adding a test for these cases.

Thanks for pointing this out. It's likely that I'll need to change the resolver algorithm away from the current single-pass recursive descent. It will need to keep track of aggregated package/requirement information, then fetch recursive transitives and flag any changes implied on already resolved package/requirements. Then this process would be looped until either 1. all requirements are met or 2. a subset of the requirements are irresolvable.

I'm working on it and will get back to you.

My apologies for the delay, I've had an extended illness. I'll try to get back to finishing this soon.

OK, I've made a bunch more changes and added new tests that should resolve this issue. Ready for another review, thanks.

The range_cascade_resolution and range_downgrade_diamond tests should exercise this now.

Stop relying on HashMap iteration order when materializing the dep Vec. Required for the resolver merge-and-resolve work to produce a stable graph across runs.

…leStale Reshape the error enum to express the cases the upcoming merge-and-resolve algorithm produces. VersionConflict is deleted; validate_version_compatibility is stubbed with Ok(()) until Task 3 removes it entirely. Also renames Offline.version to Offline.requirements: Vec<VersionReq> and updates both construction sites. Temporarily ignores range_compatible_diamond and range_incompatible_diamond — the new resolver behaviour is not yet in place.

…degen

The workspace lockfile is a pool keyed by (name, version) that is shared across independently-resolved members, so a pin that fails the merged requirements of one member is indistinguishable from another member's legitimate entry. Treating that as a hard error is a false positive; the resolver re-resolves against the registry instead.

…erging GraphBuilder now collects every requirement edge in a package's dependency graph into a per-package set. Resolution picks the highest version satisfying every accumulated requirement at once, instead of locking in the first encountered requirement's match, and retracts the edges a superseded version contributed. Merging is a within-graph concern: workspace members are resolved independently by design, so the diamond that motivates this change is expressed transitively. pkg1 -> lib-x -> lib-a ^1.0.0 pkg1 -> lib-y -> lib-a <=1.2.0 -> resolves to 1.2.0 DependencyNode::version now carries the exact resolved version so the installer cannot re-resolve upward past the resolver's pick. Cycle detection moves from a mid-descent visiting set (which cannot fire under BFS) to a topological check on the finished graph.

Covers the case where a transitive edge tightens an already-resolved package's requirement set so the previous version no longer fits: the edges that version contributed are retracted, newly-unreachable packages are dropped, and the new version's transitives are walked. Also re-enables the incompatible-diamond test against the NoCompatibleVersion variant, which now reports the full requirement set rather than the first pairwise conflict.

Adds a fixture where a second transitive path tightens a package's requirements past the existing pin, and asserts the resolver re-resolves and rewrites the lockfile rather than failing. Also pins down that the pooled workspace lockfile is unaffected: entries belonging to other members are skipped, not mistaken for stale pins.

range_cascade_resolution exercises retraction end-to-end, but only one level of it: the dropped package contributes no further edges. Add unit tests over retract_contributions_from for the branches it cannot reach — transitive cascade through a dropped package, rescheduling a survivor whose pick no longer fits the shrunken requirement set, and leaving a still-satisfied survivor at its current version.

range_pin_reuse validates that once a package version enters the lockfile, if it still satisfies all range criteria, that version is reused without hitting the registry.

heatonmatthew added 5 commits April 18, 2026 11:18

feat(cli): [WIP] Support semantic version ranges.

de6d421

- Impl is in place but more thorough tests needed to validate

feat(cli): [WIP] Additional unit tests for semantic version ranges.

ce2b42f

feat(cli): Update documentation

99ed6e7

fix(cli): Incomplete handling of parts in "name version" Deserialize

95326c5

fix(cli): Incorrect handling of CWD when installing a workspace.

a9ae96c

heatonmatthew marked this pull request as draft April 18, 2026 01:27

chore: fix lint fmt

65ce199

heatonmatthew marked this pull request as ready for review April 18, 2026 05:46

heatonmatthew changed the title ~~feat: support semver version ranges for dependencies~~ Support semver version ranges for dependencies Apr 18, 2026

chore(cli): re-add debug traces for Artifactory calls

46604d8

LinaUr reviewed Apr 22, 2026

View reviewed changes

mara-schulke added component::cli Everything related to the buffrs cli complexity::high Issues or ideas that are highly complex. require discussion and may affect backwards compatibility type::epic An epic change. This is going to make a big difference to buffrs as a product. labels Apr 22, 2026

LinaUr assigned heatonmatthew Apr 23, 2026

heatonmatthew added 15 commits April 28, 2026 17:43

doc: Re-add incorrectly removed content

d33ebc7

Remove unused function.

19596ee

Tidy-up imports

09e53cf

chore: ignore .worktrees/ directory

5f14c66

manifest: sort dependencies_as_vec output deterministically

8f3db35

Stop relying on HashMap iteration order when materializing the dep Vec. Required for the resolver merge-and-resolve work to produce a stable graph across runs.

style: remove redundant type annotation in dependencies_as_vec

7964fae

fix: scope unused_assignments allow to file level for thiserror v2 co…

7f7238e

…degen

docs: describe merge-and-resolve version selection

7c4fcd1

Merge remote-tracking branch 'upstream/main' into versioning

c1c8e41

heatonmatthew added 2 commits August 28, 2026 20:07

test: lockfile pins are not re-resolved for range criteria

4d4a09f

range_pin_reuse validates that once a package version enters the lockfile, if it still satisfies all range criteria, that version is reused without hitting the registry.

doc: keep examples in original single line format

ff08773

Navigation Menu

Uh oh!

Support semver version ranges for dependencies - #311

Support semver version ranges for dependencies#311
heatonmatthew wants to merge 24 commits into
helsing-ai:mainfrom
heatonmatthew:versioning

heatonmatthew commented Apr 18, 2026 •

edited

Loading

Uh oh!

LinaUr Apr 22, 2026

Uh oh!

heatonmatthew Apr 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

LinaUr Apr 22, 2026

Uh oh!

heatonmatthew Aug 28, 2026

Uh oh!

Uh oh!

LinaUr Apr 22, 2026 •

edited

Loading

Uh oh!

LinaUr Apr 22, 2026

Uh oh!

heatonmatthew Apr 28, 2026 •

edited

Loading

Uh oh!

heatonmatthew Aug 18, 2026

Uh oh!

heatonmatthew Aug 28, 2026

Uh oh!

heatonmatthew Aug 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

heatonmatthew commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What a reviewer should know

Test plan

Uh oh!

LinaUr Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

heatonmatthew Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LinaUr Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

heatonmatthew Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

LinaUr Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LinaUr Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

heatonmatthew Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

heatonmatthew Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

heatonmatthew Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

heatonmatthew Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

heatonmatthew commented Apr 18, 2026 •

edited

Loading

heatonmatthew Apr 28, 2026 •

edited

Loading

LinaUr Apr 22, 2026 •

edited

Loading

heatonmatthew Apr 28, 2026 •

edited

Loading