Skip to content

docs: make 2.18 upgrade requirements explicit - #14703

Open
jeffspahr wants to merge 1 commit into
kubeflow:release-2.18from
jeffspahr:codex/218-upgrade-guidance
Open

jeffspahr wants to merge 1 commit into
kubeflow:release-2.18from
jeffspahr:codex/218-upgrade-guidance

Conversation

@jeffspahr

@jeffspahr jeffspahr commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Cluster operators need a concrete 2.18 upgrade contract: what happens automatically, which users are affected, and what they must configure or sequence. The release notes, operator guide and agent assessment now describe the behavior that ships with the runtime prerequisites below.

Changes

  • Explain automatic schedule adoption and writer discovery, bounded schedule-write retries, and the exceptions for inconsistent records, CR-only schedules and external/shared-database writers.

  • Describe default auto artifact authorization: supported managed storage automatically uses verified namespace identities, preserving isolated existing roots; legacy managed artifacts/ retains ownership checks. Administrator-approved external S3 brokers using rotating credentials in a named file are also supported; unapproved custom configurations retain prefix checks; explicit key-prefix and reviewed storage remain available. SeaweedFS stays central.

  • Document reusable external adoption policies, protected UID/principal inventories and receiver templates, existing/new enrollment, cutoff semantics, missing-source recovery and denied tenant self-approval. Explicit per-namespace approvals remain available for exceptions. Distinguish artifact-read enrollment from provisioning pipeline roots/write grants. Preserve the credential-file contract, renewal without receiver restart, revocation limits and external IAM responsibilities; this is not universal IRSA discovery or independent policy certification.

  • Explain automatic proxy creation, readiness gating, namespace/credential binding, rotation and old/new receiver compatibility. No UI/controller ordering or storage opt-in is required for supported upstream installations. Document automatic recovery of missing stock bucket-policy statements without credential replacement/Secret deletion, transient retries, custom-service exceptions and per-namespace pod capacity (10m CPU/70 MiB requests, 100m CPU/500 MiB limits, plus sidecars/surge). Standard NetworkPolicy must be enforced.

  • Keep HTTP artifacts, archived logs, endpoint trust, the 2.18 environment/workload-identity credential adaptation, and temporary artifact audit separate. Enforced storage mode remains relevant to 3.0; removal of audit does not mandate moving every historical object.

  • Cover merged online JobUUID preparation and automatic retention indexes, prepare-only staging, recovery and later catch-up without describing all startup database work as nonblocking. Retain Python, size-limit, RBAC, pagination, reporting and TensorBoard impacts.

  • Make missing-record namespace routing, malformed shared-configuration scope and automatic provisional-record recovery explicit. Previously adopted/revoked and ambiguous older records remain fail-closed.

  • Warn that switching a namespace from bundled MinIO to external S3 replaces the receiver: old run history requires migration of both objects and persisted references, or postponing cutover. Same-store historical roots remain supported.

  • Provide the agent assessment guide for read-only evidence collection and explicitly authorized live checks, with unknowns recorded rather than inferred compatibility.

  • Identify pre-SeaweedFS V1 tenant outputs under artifacts/<workflow>/.... Managed auto retains ownership/prefix checks for that broadly granted legacy root; historical object/reference migration may still be required. Do not mistake the stock shared grant for public data.

  • Document persistent HTTP 503 on custom stores without the controller's IAM API despite pre-created Secrets. Select key-prefix before rollout or review genuinely isolated storage delegation; retain working receiver configuration and the custom namespace provisioner.

  • Document approved dotted-bucket support and require agent acceptance to distinguish direct storage access from authenticated UI ownership enforcement on real old-layout objects.

  • Require pre-upgrade review of effective storage grants and namespace viewers. ready does not certify least privilege; shared/ is tenant-writable, and retained broad administrator grants can expand viewer access. Narrow unintended grants or select key-prefix before rollout; verify using a read-only viewer.

  • Distinguish plain retryable receiver-upgrade/pending/transient errors from malformed readiness records and authentication/RBAC failures requiring correction, without a misleading retry header.

  • Explain managed IAM verification failure recovery: the controller requests a 10-second resync and retains credentials; pending remains fail-closed and timing is not an outage guarantee. Name the external receiver configuration setting in startup troubleshooting.

Merge dependency

Merge master #14883, then its 2.18 backport #14884, then this guide. The backport includes automatic authorization selection, protected receivers and order-independent component rollout documented here.

Managed writer discovery #14868, online JobUUID #14874, retention automation #14877, artifact audit standardization #14866, size diagnostics #14867, scheduling retries #14869/#14870, sub-DAG #14864 and the Python minimum #14865 are already merged. Master retention PR #14876 remains separate from its delivered 2.18 adaptation.

Validation

Single DCO-signed commit d62977187 rebased on release-2.18 cd1832663. Standalone master/release/guide and combined runtime/guide Sphinx builds pass with warnings treated as errors and zero warnings; scoped documentation hooks pass without source changes. Exact combined tree: f850b5d591a425e004df220dfa0a193efb841d65.

Runtime candidate acceptance passed on master 0d891201e (44 managed checks, one expected old-UI/V1-API skip, 25 external checks) and 2.18 da4b84c66 (45 managed, 25 external, no skips). Both suites verify current source/runtime image identities and clean cleanup. Current-head hosted frontend CI also passed on both branches. Earlier failed fixture revisions are not relabeled as successes.

No runtime changes in this PR. Required checks, approvals and merges remain; final merged-release acceptance and advisory approval are tracked in #14421, delivery in #14409, and the separate 3.0 migration in #14029.

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Contributor Report for @jeffspahr

Metric Value
Kubeflow org member Yes
Issues opened in kubeflow/pipelines 360
Merged PRs in kubeflow/pipelines 300
GitHub account age 3470 days (created 2017-04-21)
PR comments in kubeflow/pipelines 1520

@github-actions github-actions Bot added the ci-passed All CI tests on a pull request have passed label Oct 6, 2026
@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 7, 2026
@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 7, 2026
@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 8, 2026
@jeffspahr
jeffspahr requested a balanced review from Copilot October 11, 2026 00:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The guide omits the retained TaskService authorization limitation required by the PR description.

1 open finding
What changed in this PR

Adds centralized KFP 2.18 upgrade guidance covering schedule adoption, RBAC, pipeline lookup compatibility, and UI signing-key setup.

Changes:

  • Adds the 2.18 operator checklist and navigation links.
  • Clarifies client/RBAC migration requirements.
  • Expands signing-key configuration and recovery guidance.
File Description
docs/​user-guides/​migration.md Links SDK migration readers to the operator checklist.
docs/​operator-guides/​upgrade-2.18.md Adds the central upgrade checklist.
docs/​operator-guides/​server-config.md Clarifies signing-key setup and recovery.
docs/​operator-guides/​rbac-migration-2.18.md Adds rollout and private-pipeline guidance.
docs/​operator-guides/​installation.md Warns upgrading operators to consult the checklist.
docs/​operator-guides/​index.md Adds the checklist to navigation.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/operator-guides/upgrade-2.18.md
@jeffspahr
jeffspahr force-pushed the codex/218-upgrade-guidance branch from b8ad7ad to 107140d Compare October 11, 2026 00:51
@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 11, 2026
@github-actions github-actions Bot added the ci-passed All CI tests on a pull request have passed label Oct 14, 2026
@google-oss-prow google-oss-prow Bot added size/XL and removed size/L labels Oct 14, 2026
@google-oss-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jeffspahr. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 14, 2026
@github-actions github-actions Bot removed the ci-passed All CI tests on a pull request have passed label Oct 14, 2026
@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 14, 2026
@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 14, 2026
@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 14, 2026
@github-actions github-actions Bot added ci-passed All CI tests on a pull request have passed and removed ci-passed All CI tests on a pull request have passed labels Oct 14, 2026
@droctothorpe

Copy link
Copy Markdown
Collaborator

AA review summary for #14703: docs: make 2.18 upgrade requirements explicit

Document the release's security defaults, automated migration paths and
remaining operator responsibilities, including external artifact trust,
online database indexes and schedule adoption.

Provide an agent assessment guide that distinguishes read-only inspection
from live validation in an explicitly authorized disposable installation.

Signed-off-by: Jeff Spahr <spahrj@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-passed All CI tests on a pull request have passed size/XXL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants