Repository navigation
Fail migrations when named Azure Blob sources are missing - #262
Merged
Mark Brown (markjbrown) merged 2 commits intoOct 31, 2026
Merged
Conversation
Fail named missing blobs instead of completing an empty migration. Preserve safe source context across access and streaming errors, and add offline JSON pipeline and CLI regressions. Relates to #250 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Credential redaction remains incomplete because malformed or non-HTTP URLs can expose user-info, while aggregate transport failures can bypass sanitization and leak secrets through exception logging.
2 open issues
What changed in this PR
This PR addresses #250 by making missing named Azure Blob sources fail migrations instead of appearing as successful empty input. Destination atomicity and rollback remain separate under #251.
Changes:
- Adds source-failure propagation and sanitized diagnostics for initialization, access, opening, and streaming.
- Adds 30 offline regression cases covering pipeline outcomes, CLI reporting, diagnostics, cancellation, and stream behavior.
- Documents the source-failure contract and preserves valid empty JSON input.
| File | Description |
|---|---|
| Extensions/AzureBlob/README.md | Documents failure behavior and rollback limitations. |
| Extensions/AzureBlob/Cosmos.DataTransfer.AzureBlobStorage/Cosmos.DataTransfer.AzureBlobStorage.csproj | Exposes internals to the test assembly. |
| Extensions/AzureBlob/Cosmos.DataTransfer.AzureBlobStorage/AzureBlobDataSource.cs | Adds explicit failures, diagnostic sanitization, and a read-stream wrapper. |
| Extensions/AzureBlob/Cosmos.DataTransfer.AzureBlobStorage.UnitTests/Cosmos.DataTransfer.AzureBlobStorage.UnitTests.csproj | Defines the offline test project and dependencies. |
| Extensions/AzureBlob/Cosmos.DataTransfer.AzureBlobStorage.UnitTests/AzureBlobSourceFailureTests.cs | Adds source, pipeline, CLI, and stream regressions. |
| CosmosDbDataMigrationTool.sln | Registers the Azure Blob test project. |
Review effort: Balanced
Redact possible URL user-info in unsafe location fallbacks and sanitize nested aggregates of read failures. Preserve cancellation and unexpected aggregate exceptions; add 20 offline regressions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Fixes #250.
Data transfer complete.[]input, cancellation, stream seeking, and disposal.Destination-file atomicity and rollback remain separate in #251. This PR does not change Core, JSON, or Common production code, and does not guarantee preservation of destination bytes after a source failure.
Copilot review follow-up
Both findings were reproduced and fixed in
2858d2ababa5321773bba8a0c1c712c1c4bf22b8:Added 20 regression cases; all 50 Azure Blob tests pass locally and in CI. Both review threads are resolved.
Verification
Current validated head:
2858d2ababa5321773bba8a0c1c712c1c4bf22b8, based on81a625be548effbce3ce1d1b4074581fcc9ecaffcontaining the merged .NET 10 upgrade.548e22b): before the fix, the missing-blob source threw no exception and the real JSON/file-sink + command-handler pipeline returned 0; after the fix, it fails with exit code 1. The Windows whole-solution run passed 1,384 tests with 8 unconfigured-service skips. Initial Build/Test and Validate runs also passed.548e22b: the coordinating session tested the self-contained Windows CLI and real Azure SDK against a localhost-only HTTP fixture. All 12 cases passed: missing 404, denied 403, disappearance after HEAD, and malformed JSON fail nonzero; valid JSON and empty arrays succeed. Checks include new/existing destinations and a SAS sentinel that never appears in logs. No live Azure account or credentials were used.Limits
No live-cloud or emulator integration run is claimed. Existing package vulnerability warnings (
System.Security.Cryptography.Pkcs9.0.0, NU1902/NU1903) and pre-existing nullable/analyzer warnings remain unsuppressed. No test exclusions were broadened. No release, deploy, or publication workflow was run.