Prevent path traversal in control server patch applier (#2532) - #2557
Conversation
apply_code_patch joined the patch path onto the target directory without any containment check, so a model-code or package patch with a path like ../../../etc/cron.d/x (or an absolute path) could write or delete files outside the model/packages directory via the control server's POST /control/patch endpoint. Resolve the joined path and reject any result that escapes the target directory. Legitimate nested paths within the target are unaffected. Fixes basetenlabs#2532
| from truss.templates.control.control.helpers.custom_types import Action, Patch | ||
|
|
||
|
|
||
| def _resolve_within(base_dir: Path, relative_path: str) -> Path: |
There was a problem hiding this comment.
Thanks for the contribution! Can you give me some background on how you came across this? Was this an issue you hit or just something you saw browsing open issues?
There was a problem hiding this comment.
Thanks for taking a look! Just something I came across browsing the open issues — I wasn't hitting it in production. The security label on #2532 caught my eye, and since the fix was nicely scoped to a single sink (apply_code_patch) with a clear containment invariant to enforce, it seemed like a good self-contained hardening PR to pick up. Happy to adjust scope or approach however you'd prefer.
There was a problem hiding this comment.
I am a bit concerned without having tested this against code patching against the backend (i.e. truss push --watch/truss watch) any side effects it may have giving a qualified path here
There was a problem hiding this comment.
Good catch — the concern was valid: the check previously handed the fully-resolved path to the file operations, which changes what gets written/logged if the app dir sits behind a symlink. Fixed in 2db3c64:
_validate_withinnow uses resolved copies only for the containment check and returns the plainrelative_dir / patch.pathjoin, so the path handed tomkdir/write/unlink(and the log lines) is byte-for-byte identical to pre-PR behavior for every legitimate patch.- Added a regression test that applies an UPDATE patch through a symlinked app dir and asserts it lands in the original location (and that traversal is still rejected through the symlink).
To verify against the actual live-reload flow, I ran test_control_truss_apply_patch (the Docker integration test that builds a control truss from the local templates, POSTs a real PatchRequest to /control/patch — the same server-side path truss push --watch/truss watch drives — and asserts the model's predict output changes): 1 passed in 3:46. Also green locally: the patch-applier unit tests (19), the control server endpoint tests (35), and the client-side dir-patch applier tests (68).
There was a problem hiding this comment.
Are you a Baseten user and were you able to verify this against the Baseten server side platform by deploying/watching a model?
There was a problem hiding this comment.
Yes — I'm a Baseten user, and I just verified this against the platform. I deployed a dev model from this branch's checkout (truss push --watch) and exercised the live-reload flow end to end:
- UPDATE: edited
model.py→Created patch to update model code file: model/model.py→patched successfully→ next predict returned the new output (no rebuild, same deployment) - ADD (nested new dir): created
model/helpers/nested_util.py→ patched successfully, and the model imported it on reload - REMOVE (incl. empty-dir cleanup): deleted the file and dir → patched successfully → predict reflected the revert
All three round-tripped cleanly through POST /control/patch on the deployed dev container.
One caveat for full transparency: the dev image's control server is built by the Baseten backend from released truss, so the platform run pins down the exact behavior the released applier has today. As of 2db3c64 the containment check returns the identical unresolved relative_dir / patch.path join for every accepted patch — so for the entire flow verified above, the patched code is behavior-identical by construction. The only behavioral delta is rejecting paths that escape the target dir, and that path is covered by the unit tests plus the local Docker integration test (test_control_truss_apply_patch), which does bake this branch's control server into the container image.
…havior Address review concern about side effects of handing a fully-resolved (qualified) path to the file operations: the containment check now uses resolved copies internally but returns the plain relative_dir / patch.path join, so write/delete behavior and log output during live reload (truss push --watch / truss watch) are byte-for-byte identical to before for legitimate patches, including when the app dir sits behind a symlink. Adds a regression test applying a patch through a symlinked app dir. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thanks for the contribution! |
What
Fixes #2532. The control server's
apply_code_patchjoined the patch path directly onto the target directory with no containment check:patch.pathcomes straight from the JSON body ofPOST /control/patch, so a model-code or package patch with a path like../../../etc/cron.d/x— or an absolute path, which/-join leaves unchanged — could write (ADD/UPDATE) or delete (REMOVE) files anywhere the process can reach, outside the model/packages directory. BothModelCodePatchandPackagePatchflow through this function, so both were affected.As noted on the issue, this endpoint is dev/internal, so this is a defense-in-depth hardening: a patch applier should never touch files outside its target directory, whether the offending path comes from a malicious sender on a reachable control port or simply from buggy tooling that emits an unexpected
... Today the latter silently corrupts or deletes unrelated files instead of failing cleanly.How
Added
_resolve_within(base_dir, relative_path), which resolves the joined path and raisesValueErrorif the result escapesbase_dir(viaPath.is_relative_to, available on the project's minimum Python 3.9).apply_code_patchnow routes through it. This also rejects absolute-path injection, since an absolutepatch.pathresolves outside the target. Legitimate nested paths within the target directory are unaffected.The change is scoped to the reported
apply_code_patchsink. The external-data path is already covered by #2486; other sinks inModelContainerPatchApplier(config write) are left untouched to keep this focused.Testing
New regression tests in
test_model_container_patch_applier.py(all fail on the pre-fix code, pass after):../traversal on write is rejected and nothing is written — parametrized overModelCodePatch/PackagePatch×ADD/UPDATE../traversal onREMOVEis rejected and the outside file survivesnested/dir/new.py) still writes correctlyuv run ruff check/ruff formatare clean on both files.