[jeffspahr]

Problem

By-name SDK and CLI calls do not expose the pipeline namespace even though explicit namespace lookup already exists. Private pipelines can be missed or a same-named shared pipeline selected, forcing callers to resolve IDs manually.

Summary

Make SDK/CLI by-name operations usable for private pipelines without requiring a separate lookup followed by an ID-based call. This addresses an existing client usability gap, not a newly introduced security regression.

Motivation and why now

The SDK already supports get_pipeline_id(name, namespace=...), but the version-upload helpers and CLI run/version commands call it without exposing a pipeline-namespace argument. Those operations therefore search shared pipelines, even when the client or CLI has a default experiment namespace. A private pipeline may not be found; a same-named shared pipeline may be selected instead. Users can already select a private pipeline by resolving its ID with an explicit namespace and then using the ID-based operation. This PR makes that existing workflow available directly in the convenience methods and commands.

Reviewing the namespace-lookup security fixes (#14052 and #14522) and 2.18 upgrade guidance surfaced this gap. These SDK/CLI paths use filtered ListPipelines, not the affected GetPipelineByName endpoint, and did not rely on its unsafe defaults. This PR is not needed to preserve ordinary SDK/CLI compatibility with those backend fixes and is not an additional authorization fix.

Changes

• Add optional namespace= to SDK upload_pipeline_version and its from_pipeline_func wrapper.
• Add --pipeline-namespace to CLI pipeline create-version and run create when resolving --pipeline-name.
• Keep omitted scope shared; never inherit the client/global experiment namespace or try other namespaces.
• Fail a missing parent lookup before upload or experiment/run creation, naming the searched scope and suggesting an explicit namespace or pipeline ID.
• Reject combining the new namespace selector with an explicit parent ID. Existing ID-based calls remain unchanged; versions inherit the parent's namespace. Server-side namespace/permission checks are unchanged and permission errors are not retried in another scope.

For example:

client.upload_pipeline_version(
    "pipeline.yaml", "revision-2",
    pipeline_name="training", namespace="team-a",
)

kfp pipeline create-version pipeline.yaml --pipeline-version revision-2 \
  --pipeline-name training --pipeline-namespace team-a
kfp --namespace team-a run create --experiment-name experiment \
  --pipeline-name training --pipeline-namespace team-a --version VERSION_ID

The global CLI --namespace continues to scope the experiment; it does not select the pipeline lookup scope. A private pipeline/version still needs an experiment in its owning namespace and normal permissions.

Compatibility and release tracking

Related to the caller review in #14421 and documentation in #14703. Target master; any release-2.18 backport is a separate usability decision tracked in #14409, not a prerequisite for the namespace security fixes or a security release gate. Older clients can continue using the existing explicitly scoped lookup followed by an ID-based call. Installing a backend release does not update SDK/CLI installations.

The missing-parent behavior is deliberately earlier and clearer: the old SDK proceeded to upload with pipelineid=None; the new SDK raises ValueError before making that request. get_pipeline_id itself retains its Optional return contract. No automatic namespace fallback, authorization relaxation, or cluster configuration change is introduced.

Verification

• Full SDK client and CLI suites: 305 tests and 8 subtests passed on Python 3.11, including explicit/shared/empty scope, same-named pipelines in two namespaces, non-inheritance of client/global defaults, no mutation on lookup miss, permission-error propagation, ID behavior and wrapper forwarding.
• Reproduced the missing-parent upload against the pre-change implementation.
• Applicable changed-file pre-commit hooks and whitespace checks passed.
• Independent review found no blocking implementation issue; corrected its test-contract finding by supplying/asserting the required version ID for CLI template runs.
• Live multi-tenant API execution is not claimed by these mocked-client regressions; backend authorization behavior is unchanged.

Checklist

• DCO-signed commits.
• Conventional PR title.

Review follow-up: fbe674f10 adds invalid namespace-selector CLI coverage and SDK release guidance, and wraps long expressions. 205 targeted CLI/client tests plus 8 subtests and applicable hooks pass; hosted CI must validate the new head. The prior full-suite results above remain scoped to the preceding revision.

CI prerequisite

Includes the focused runner compatibility fix from #14845 after a confirmed missing-libpython failure before SDK client tests. The #14845 prerequisite has merged; the namespace implementation is unchanged.

Current validation

After the current-base refresh, 206 client/CLI tests and eight subtests pass; eight Docker mirror regression tests also pass. Hosted CI is running on the updated branch; earlier green runs do not establish current-head completion.

Shared MLflow CI prerequisite

The compatible immutable MLflow runtime/operator pins are now inherited from merged #14861. The independent fixture pin regression is retained. Database migration and serving assertions remain intact; no migration or replica-readiness bypass is introduced.

Documentation CI prerequisite

Includes #14863 to build SDK documentation with system fonts and no Google Fonts downloads. The no-network regression builds both actual font configurations; documentation content checks remain intact.

Includes the GitHub CLI provisioning prerequisite consolidated into #14845 for managed runners. Seven setup regressions and twelve artifact-polling tests pass. Shared #14845 runner compatibility fixes are retained; these prerequisites do not change product behavior.

Conflict-resolution validation

Rebased onto master 2ca202210. The resulting tree exactly matches an independently resolved merge, preserving the product implementation. CI guidance retains upstream Istio instructions and the applicable PR guidance. Whitespace checks passed. Eight Docker mirror tests and seven GitHub CLI provisioning tests passed for the inherited runner setup. Earlier application-test results above describe the preceding revisions; fresh hosted CI is pending for fee4e865a.

E2E runner contention

The TLS E2E lane timed out while ten Ginkgo workers shared a four-CPU runner with Kind and pipeline pods: CPU averaged 97%, load averaged 43.3 (peak 60.1), and CPU pressure stalled work for 91.4% of the test window. Automatic single-user E2E concurrency is now capped at half the available CPUs (minimum one), retaining any stricter lane cap. Explicit workflow-dispatch overrides remain authoritative. This changes neither selected specs nor suite timeouts.

flowchart LR
  A[Four-CPU runner] --> B[Before: ten concurrent test workers plus Kind]
  A --> C[After: two concurrent test workers plus Kind]

Loading

The configure-shell regression fails against the previous workflow and passes with the cap, including one/four/32-CPU runners, stricter failure/nested lane caps, label preservation, and an explicit seven-worker override. Both concurrency tests and five existing API shard tests pass; Ruff and whitespace validation pass. Hosted validation on 7d40b8a67 passes every substantive check. The previously timed-out TLS lane passed all 33 selected specs in 1,407 seconds (~23.5 minutes), with unchanged labels and timeout. On its four-CPU runner, CPU averaged 15%, load averaged 1.6 (peak 3.6), and CPU-pressure stalls fell to 5.42% of the test window.

[github-actions]

Contributor Report for @jeffspahr

Metric	Value
Kubeflow org member	Yes
Issues opened in kubeflow/pipelines	379
Merged PRs in kubeflow/pipelines	309
GitHub account age	3472 days (created 2017-04-21)
PR comments in kubeflow/pipelines	1525

[Copilot]

Copilot review overview

🟢 Approval recommended

The implementation is consistent across SDK and CLI surfaces and includes comprehensive regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Adds explicit namespace selection for pipeline-by-name SDK and CLI operations while preserving shared-pipeline defaults.

Changes:

• Adds namespace-aware pipeline version uploads and CLI lookups.
• Improves missing-parent errors and rejects incompatible selectors.
• Adds coverage for scoped lookup, failures, permissions, and ID behavior.

🤖 LGTM 🤖

File	Description
sdk/​python/​kfp/​client/​client.py	Adds namespace-aware version upload APIs.
sdk/​python/​kfp/​client/​client_test.py	Tests SDK lookup and validation behavior.
sdk/​python/​kfp/​cli/​run.py	Adds namespace selection to run creation.
sdk/​python/​kfp/​cli/​pipeline.py	Adds namespace selection to version creation.
sdk/​python/​kfp/​cli/​cli_test.py	Tests CLI scope and failure behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟢 Approval recommended

The namespace behavior is explicit, backward-compatible, validated before mutation, and well covered by tests.

0 open findings

🧠 Review effort: Balanced

[droctothorpe]

AP agent review — #14712: fix(sdk): support explicit pipeline namespaces in by-name callers

Reviewed head: 06d638078020357f924645c0aaa456fc72bef469

Classification: rhr (241 additions + deletions). Reviewers identified product/end-user impact or could not rule it out.
Result: Passed — no blocking findings.
Human review: Required; this agent review is not sufficient.

Two independent read-only agent reviewers assessed this head. Tests were inspected but not executed; this does not certify CI or live deployment behavior.

Correctness / Risk

Adds an optional pipeline-namespace selector to the SDK upload_pipeline_version helpers and to the CLI create-version and run create commands, and fails early with an actionable error when a by-name lookup finds nothing. The control flow is correct: omitted namespace still searches shared pipelines only, the client/global default namespace is never inherited, namespace with an explicit ID is rejected before any API call, and the ID-only path is byte-for-byte unchanged. The early failure in run create is an improvement over the base, where a failed name lookup with --version created the experiment and then hit a ValueError in _create_job_config. Test mocks match the real call signatures and the CI environment (requires-python >=3.11, Click 8.3.1) supports the enterContext and result.output usage. Only non-blocking notes remain.

Standards / Tests

Adds an optional namespace argument to Client.upload_pipeline_version and its from_pipeline_func wrapper, plus --pipeline-namespace on kfp pipeline create-version and kfp run create, with fail-fast errors when a by-name lookup misses and rejection of namespace+ID combinations. All five changed files were read in full at the pinned head. Test additions are well-targeted: the SDK tests patch the real service layer (pipeline_service_list_pipelines / upload_pipeline_version) and verify that the explicit scope (None, '', team-a, team-b) is what reaches the API with a client whose default namespace is ns1, that misses raise before upload, that 403s propagate untouched, and that ID calls never look up. The CLI tests drive the real click group with a mocked Client, assert the exact get_pipeline_id(name=, namespace=) call shape matching production, and check exit codes (1 for ClickException, 2 for UsageError) and no-mutation on failure. The from_pipeline_func test fix removes a previously invalid id+name call shape. TestCase.enterContext is new to the SDK test suite but valid: pyproject requires Python >=3.11 and the SDK CI matrix is 3.11/3.13. No blocking issues; four optional suggestions (duplicated error-message construction, one untested UsageError branch in run.py, missing sdk/RELEASE.md entry for a user-visible behavior change, one over-length line).

[droctothorpe]

AA review summary for #14712: fix(sdk): support explicit pipeline namespaces in by-name callers

• Reviewed head SHA: 06d638078020357f924645c0aaa456fc72bef469
• Eyeballs verdict: eyeballs required (ER)
• AA review result: passed
• Correctness / risk: Explicit namespace lookup is only applied for by-name callers, preserves the shared default, and surfaces actionable misses; no concrete regression was spotted in the inspected CLI/SDK paths.
• Standards / tests: CLI and client tests cover explicit/shared scope, missing parents, ID bypass, and permission propagation; no standards or test-coverage blocker was found.
• Blocking findings: 0
• Human review still required: yes

[jeffspahr]

@droctothorpe Addressed the optional coverage/documentation notes in #14712 (comment) in fbe674f.

Added CLI rejection tests for --pipeline-namespace with neither name nor ID and with both name and ID, verifying exit 2 and no lookup/upload/experiment/run mutation. Added sdk/RELEASE.md guidance for explicit scope, the shared-only default, and early lookup failure; wrapped the long conditional expressions.

I kept the short error messages local to each CLI command: sharing them would add cross-command coupling solely for text reuse without changing the contract. 205 targeted CLI/client tests plus 8 subtests and applicable hooks pass. Hosted CI must validate the new head.

[droctothorpe]

AA review summary for #14712: fix(sdk): support explicit pipeline namespaces in by-name callers

• Reviewed head SHA: fbe674f104622554174ca8978dfaaf9f6d9bb31e
• Eyeballs verdict: eyeballs required (ER)
• AA review result: passed
• Correctness / risk: Explicit pipeline-namespace lookup is limited to by-name CLI/SDK paths, preserves shared-default behavior when omitted, rejects invalid selector mixes, and fails missing-parent lookups before upload or run side effects.
• Standards / tests: Targeted CLI/client tests cover explicit and shared scope, invalid namespace selectors, missing-parent failures, permission propagation, ID bypass, and wrapper forwarding; release notes/help/docstrings were updated consistently.
• Blocking findings: 0
• Human review still required: yes

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jeffspahr. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• .github/OWNERS
• docs/OWNERS
• kubernetes_platform/OWNERS
• sdk/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[droctothorpe]

AA review summary for #14712: fix(sdk): support explicit pipeline namespaces in by-name callers

• Reviewed head SHA: 5e07197d3d1d0a8f8d4c5f937aa3101851171df5
• Eyeballs verdict: eyeballs required (ER)
• AA review result: passed
• Correctness / risk: Full static diff review found no blocker. Explicit namespace forwarding is confined to by-name lookup, preserves shared-only scope when omitted, rejects incompatible selectors and fails before mutations. Prior coverage/release-note suggestions addressed. No local tests or live multi-tenant validation.
• Standards / tests: Tests check exact namespace arguments, permission propagation, ID bypass and no side effects after lookup failure. Nonblocking inherited gap: readthedocs-builds.yml PR filters omit docs/sdk/**, so SDK-docs-only changes miss the new font regression. Exact-head SDK/script/docs checks passed; fresh-base aggregate CI and test: bound frontend image builds and recover stalled builders #14845/test: build SDK documentation without external font downloads #14863 coordination remain prerequisites.
• Blocking findings: 0
• Human review still required: yes