[Mark Brown (markjbrown)]

Summary

Closes #223.

Return a fresh HttpClient for each Cosmos SDK initialization instead of reusing a process-wide mutable wrapper. Reuse process-owned SocketsHttpHandler connection pools keyed by proxy URL, proxy credentials, pre-authentication and TLS validation settings. SDK-owned wrappers use disposeHandler: false; source and sink Cosmos clients are disposed at their operation/enumeration boundary, including encryption wrappers.

The custom handler explicitly applies proxy and TLS options: SDK 3.49 rejects WebProxy together with HttpClientFactory and skips its HTTP-handler certificate setup when a factory is supplied. Default certificate validation remains enabled; the opt-out warning and Direct-mode callback remain intact. Default network credentials are distinct from proxy credentials and not required for RBAC. No production auth-flow, retry or partition-key rewrite is included.

Review fixes and emulator investigation — October 31

Preserved the contributor's main-branch merge (335ce22). Corrected the remaining sink proxy/pre-authentication guidance and resolved the Copilot review thread.

The merged integration suite allocated twelve containers against the current Linux emulator's ten-partition limit; the original failed run returned 503/1007 on container creation. The Operations fixture now reuses two destinations across both pre-authentication variants, keeping the suite at ten containers. All sequential operations and read-back assertions remain; upsert ensures both variants perform real writes. Command failures now log their underlying errors instead of only reporting exit code 1.

A separate native crash then occurred during the earlier partition-key tests on three attempts of head 418c450: cosmosdb-emulator reported a fatal error in sqlpal.dll, exited with code 1, and subsequent requests failed with connection refused. Docker did not report an OOM kill. The repeated partition migration fixture was exercising fresh SDK-owned connection pools for each disposed sink client.

Commit 2186d16 runs that repeated migration fixture through this PR's shared HTTP transport (UseDefaultCredentials=true) while retaining scoped CosmosClient disposal and every partition-key assertion. With the same emulator image and partition configuration, the full current Linux suite now passes on its first attempt. This supports connection-pool teardown/churn as a trigger; it does not identify or repair the emulator's internal native defect. The separate JSON migration still exercises SDK-managed transport without default credentials, and offline lifecycle tests retain no-custom-factory controls. No test cases, assertions, required checks, or failure guards were removed or weakened.

Current exact-head validation

All PR checks and the separate Windows emulator run passed on 2186d16888217d937a79c3cad1dfa8ef03a424eb.

Validation	Result
Build/test all projects	Passed
All six desktop packages	Passed
CodeQL C# / Actions	Passed
Windows emulator	291 passed, zero skipped
Linux current emulator	291 passed, zero skipped
Linux vNext emulator	289 passed, two existing preview missing/null-capability skips
Integration failure guards	Passed

Downloaded emulator TRX artifacts confirm these counts and successful Operations coverage. Local validation before the final two-line fixture adjustment passed 284 tests with seven emulator tests skipped; the final-head build and emulator runs above compiled and exercised that adjustment.

Root cause and ownership evidence

The SDK invokes the factory and assigns HttpClient.Timeout and default headers during construction. Reusing a started wrapper causes the reported InvalidOperationException, with either RBAC or key authentication. The SDK also disposes factory-returned clients.

• SDK 3.49 CosmosHttpClientCore setup/disposal
• .NET 8 HttpMessageInvoker handler ownership

Pools survive individual client disposal; connections recycle after five minutes for DNS refresh. Cookies are disabled to avoid cross-client state. Keys/tokens and HTTP headers are not cached in handler configuration.

Regression coverage

• Baseline: eight lifecycle cases failed and two no-custom-factory controls passed. The baseline TRX captures the exact HttpClient.set_Timeout -> CosmosHttpClientCore.CreateHelper -> DMT.CreateClient exception after a pre-cancelled loopback request.
• Twenty-two lifecycle cases cover key/RBAC paths, both pre-auth values, same/different account settings, source/sink reuse, concurrent construction and HTTP use, SDK/encryption disposal, fresh wrappers, pooled socket reuse, explicit proxy routing, TLS rejection/opt-out, and preserved client options.
• The mandatory emulator round-trip test executes actual sequential Cosmos-to-Cosmos Operations with both pre-authentication settings.
• RBAC constructor tests inject an offline credential through a narrow internal seam. SDK constructors eagerly acquire tokens; a dedicated test proves the endpoint scope is requested only through the fake. Existing RBAC diagnostic-construction tests were isolated too. Pure credential-selection tests remain intact; production credential selection is unchanged.

Historical validation on pre-main-merge head 480e3097f291abf1e80ff8035854047d398a39a6: all eleven jobs passed, with 131/131 emulator tests on each platform: build/tests, packaging, Linux, Windows.

Limits and merge context

These tests do not validate live Entra roles/token acquisition or enterprise proxy credential negotiation. Emulator migrations use keys; offline RBAC tests validate construction/transport lifetime, not live authorization. Bulk execution is not exercised by the emulator round trips. The existing Cosmos Encryption 2.0.4 / Cosmos SDK 3.49 NU1608 warning remains. Native current-emulator stability under rapid SDK-managed pool creation/disposal remains a limitation, not a claimed vendor-engine fix.

Upstream retry and partition-key changes incorporated by the contributor's main merge are preserved. No cloud provisioning, credential-state changes, release/deployment/publishing, approval, merge or manual issue-closure actions were performed.

[Mark Brown (markjbrown)]

Phil Nachreiner (@philnach) This PR is ready to merge once the required approval is recorded. I reviewed head 480e3097, including HTTP client ownership/disposal, handler isolation, TLS/proxy configuration, and the offline credential test seam. No additional code changes were needed and there are no outstanding review discussions.

Today's local Cosmos suite passed 129 tests, including 22 lifecycle cases, with only the two optional local-emulator tests skipped. All ten PR checks are successful at this head; the existing Linux emulator/guard and Windows emulator runs also pass. CI was rechecked, not newly rerun today.

The documented limits remain: no live Entra authorization or enterprise proxy negotiation was exercised. This PR shares Cosmos sink/test/docs files with #254 and #255, so preserve the independent fixes and recheck CI as those changes land.

[Mark Brown (markjbrown)]

Copilot resolve the merge conflicts in this pull request

[Copilot]

Copilot resolve the merge conflicts in this pull request

Resolved the conflicts and merged main in commit 335ce22. The Cosmos extension test project passed (284 passed; 7 emulator tests skipped without emulator configuration).

[Copilot]

🟡 Changes recommended

The sink documentation still gives contradictory proxy authentication guidance and incorrectly suggests pre-authentication avoids the first challenge.

1 open issue

• Update sink proxy authentication guidance · New

What changed in this PR

Fixes the Cosmos HTTP client lifecycle issue reported in #223 by separating SDK-owned clients from shared connection pools, allowing sequential Operations to initialize clients safely.

Changes:

• Creates fresh HttpClient wrappers over shared handlers configured for proxy, credentials, and TLS settings.
• Disposes source and sink Cosmos clients at their enumeration or operation boundaries.
• Adds offline lifecycle coverage and sequential emulator migration tests, and clarifies connection-setting documentation.

File	Description
Extensions/​Cosmos/​README.md	Documents transport ownership, authentication settings, and regression coverage.
Extensions/​Cosmos/​Cosmos.DataTransfer.CosmosExtension/​CosmosExtensionServices.cs	Separates client wrappers from pooled handlers and adds an internal credential-injection seam.
Extensions/​Cosmos/​Cosmos.DataTransfer.CosmosExtension/​CosmosDataSourceExtension.cs	Disposes the Cosmos client when enumeration ends.
Extensions/​Cosmos/​Cosmos.DataTransfer.CosmosExtension/​CosmosDataSinkExtension.cs	Scopes Cosmos client disposal to the write operation.
Extensions/​Cosmos/​Cosmos.DataTransfer.CosmosExtension.UnitTests/​OfflineTokenCredential.cs	Provides a credential fake that rejects token acquisition locally.
Extensions/​Cosmos/​Cosmos.DataTransfer.CosmosExtension.UnitTests/​CosmosHttpClientLifecycleTests.cs	Covers lifecycle isolation, pooling, disposal, proxy routing, and TLS behavior.
Extensions/​Cosmos/​Cosmos.DataTransfer.CosmosExtension.UnitTests/​CosmosDataSourceExtensionTests.cs	Extends emulator coverage with sequential Cosmos-to-Cosmos operations.
Extensions/​Cosmos/​Cosmos.DataTransfer.CosmosExtension.UnitTests/​CertificateConfigurationTests.cs	Disposes test clients and isolates RBAC construction from live credentials.

_{Review effort: Balanced}