Conversation
Networks are merged into a new network under a new scope with a given name. Task and ProtocolDAGResultRef support is not implemented here.
There is hardly any difference in performance.
Add the client + API surface for the merge_networks database operation introduced in this branch. Users can now combine multiple existing AlchemicalNetworks into a single new AlchemicalNetwork through AlchemiscaleClient, with completed and errored Tasks (and their ProtocolDAGResultRefs) carried over so previously-computed results do not need to be re-run. - POST /networks/merge endpoint validates the destination Scope and each source network's Scope against the caller's token, then defers to Neo4jStore.merge_networks. - AlchemiscaleClient.merge_networks wraps the endpoint and guards against wildcard Scopes, empty input, and non-AlchemicalNetwork ScopedKeys client-side. - Integration tests cover the happy path, scope authorization (both bad destination and bad source), and that Tasks + PDRRs in complete/error state are cloned into the new scope and reachable through get_network_tasks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the inline kc_to_gufe helper and manual chain-slicing loop with a call to KeyedChain.decode_subchains, paired with a list comprehension that captures the original database GufeKeys in the same order. The decode_subchains helper (gufe >=1.8.0) reuses a shared tokenizable_map across yields, so common dependencies in a source network are decoded only once. Lift the previously nested TransformationData dataclass to a private module-level _TransformationData so merge_networks reads top-to-bottom without chasing an inline class definition, and so the helper can be referenced by future tests in isolation. The dropped key_decode_dependencies import goes with kc_to_gufe. subchain_cache is preserved with the same shape but now populated via KeyedChain.from_gufe(transformation); whether the cache (and the unconnected tf_node it feeds) is needed at all is a separate question to revisit once the PERFORMS-wiring question is settled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
merge_networks now uses KeyedChain.decode_subchains, which was added in gufe 1.8.0. Bump the pin in: - devtools/conda-envs/test.yml (was =1.7.1) so the integration tests exercise the new code path. - devtools/conda-envs/alchemiscale-server.yml (was =1.6.1) so server deployments running merge_networks pick up the required gufe API. Client and compute env files are left alone since neither imports the new code path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code review revealed that cloned Tasks on the merged network were
written but never wired back to their Transformations, leaving every
PERFORMS-based traversal (get_network_tasks, get_task_transformation,
set_tasks_status, etc.) blind to them. The new
test_merge_networks_preserves_tasks_and_results check via
get_network_tasks asserts this exact thing and would have failed in
CI.
Changes
-------
- Add the PERFORMS edge in _TransformationData.to_subgraph so cloned
Tasks are reachable from the standard
(:AlchemicalNetwork)-[:DEPENDS_ON]->(:Transformation)<-[:PERFORMS]-(:Task)
traversal. The pre-existing tf_node construction is now meaningful,
resolving the open question about subchain_cache.
- Add a `state` parameter to Neo4jStore.merge_networks,
/networks/merge, and AlchemiscaleClient.merge_networks, bringing
merge_networks to feature parity with assemble_network /
create_network. Defaults to NetworkStateEnum.active.
- Document the "cloned Tasks are not actioned to the merged TaskHub"
semantics in the store-level and client docstrings, including the
remediation path (call action_tasks after merging).
- Replace the O(N^2) `transformation_data.index(transformation)`
dedup with a `dict[GufeKey, _TransformationData]` keyed on
transformation.key, avoiding repeated GufeTokenizable equality
checks on large networks.
- Improve the missing-ScopedKey error to collect and report all
missing source SKs in a single ValueError, rather than only the
one that first triggered KeyError.
- Add a chain-order contract comment at the
zip(database_keys, transformations) site pointing at the
decode_subchains yield-order guarantee.
- Make /networks/merge an `async def` endpoint, matching the style
of the neighboring create_network endpoint.
- Add a parametrized client test (state in {active, inactive}) that
exercises the new state parameter end-to-end.
- Drop the unused network_tyk2 fixture from the two API-level
authorization tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Member
Author
|
Thanks for the careful review. Pushed Blocking
Should-fix
Nits picked up
Deferred
Sanity check the reviewer asked for Static analysis confirms the PERFORMS edge is now emitted inside the per-task loop with the correct direction ( |
merge_networks now depends on KeyedChain.decode_subchains (gufe 1.8.0). Pin all five env files to an exact 1.8.0 release, matching the repo's convention of pinning gufe exactly rather than with a lower bound. Touches the test, server, compute, client, and docs envs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Revives the work originally proposed in #433 (now unreachable since the contributing fork was deleted) and finishes it.
Neo4jStore.merge_networksfor combining multiple existingAlchemicalNetwork\s into a newAlchemicalNetworkin a targetScope, with completed and erroredTask\s (and theirProtocolDAGResultRef\s) cloned into the new scope so previously-computed results do not need to be re-run. (Originally authored by @ianmkenney; preserved here as-is, then refactored — see commits.)POST /networks/mergeonAlchemiscaleAPI, which validates the destinationScopeand each source network'sScopeagainst the caller's token before delegating to the store.AlchemiscaleClient.merge_networks(networks, name, scope), with client-side guards for wildcard scopes, empty input, and non-AlchemicalNetworkScopedKey\s.KeyedChain.decode_subchains(gufe ≥1.8.0), which replaces a hand-rolledkc_to_gufehelper plus manual chain-slicing loop, and lifts the inlineTransformationDatadataclass to a private module-level_TransformationDatafor readability.gufepin to>=1.8.0indevtools/conda-envs/test.ymlanddevtools/conda-envs/alchemiscale-server.yml(the only env files that actually exercise this code path).complete/errorTask\s with theirProtocolDAGResultRef\s — including a reachability check viaget_network_tasks(merged_sk).Closes #221.
Notes for reviewers
merge_networksbody is unchanged in behavior from the original PR; the diff against Allow server-side copying and merging of AlchemicalNetworks #433's last revision is thedecode_subchainsrefactor plus the lift ofTransformationData.get_network_tasksassertion intest_merge_networks_preserves_tasks_and_resultsexercises the standard(:AlchemicalNetwork)-[:DEPENDS_ON]->(:Transformation)<-[:PERFORMS]-(:Task)traversal on the merged network. If it fails, that indicates a missingPERFORMSedge in_TransformationData.to_subgraph— flagged here for follow-up rather than fixed in this PR, since the original implementation has the same behavior.subchain_cacheis preserved with cleaner contents (KeyedChain.from_gufe(transformation)) but its necessity is an open question tied to thePERFORMSfollow-up above.Test plan
pytest -n auto -v alchemiscale/tests/integration/storage/test_statestore.py -k merge_networkspytest -n auto -v alchemiscale/tests/integration/interface/test_api.py -k merge_networkspytest -n auto -v alchemiscale/tests/integration/interface/client/test_client.py -k merge_networksblack --check --diff alchemiscalemypy alchemiscaletest_merge_networks_preserves_tasks_and_resultsreveals the suspected missingPERFORMSwiring; open a follow-up if so.🤖 Generated with Claude Code