Skip to content

ci: standardize change-impact validation across CUBRID Lab repositories #74

Closed as not planned

Description

@yeongseon

Problem

sqlalchemy-cubrid PR #1152 includes documentation, release-note Python tooling, and CI configuration. Broad scripts/** and workflow risk filters classify it as runtime-risk, invoking offline, typing, Alembic, packaging, and live CUBRID jobs even though the changed release-note scripts do not modify the dialect, connection behavior, or database test infrastructure.

An initial 18-repository workflow audit on 2026-11-07 found that change-impact routing is implemented unevenly. This issue is the organization-wide prevention and audit tracker; repository-specific fixes should stay in their own repositories.

Organization-wide policy

  1. Define mutually understandable impact categories: documentation, documentation build, release automation/tooling, build/dependency metadata, runtime/unit, database integration/compatibility, and CI impact classifier itself.
  2. For PRs, run the minimum sufficient tests by affected component. Pure Markdown must not launch database services. Release-note tooling should run targeted script/format/attribution tests, not a dialect integration matrix.
  3. CI classifier and gate changes must self-test with a full/fail-closed validation where required; unknown paths, incomplete diffs, API failures, rename/delete changes, and ambiguous impact must fail closed. Keep main/release/scheduled/full validation policy and security boundaries intact.
  4. Keep an always-reported required aggregate gate, so intentionally skipped matrix lanes do not strand branch protection in Pending. Verify selected-job failures/cancellations cannot be hidden. Avoid workflow-level paths filters for required checks unless required-status behavior is proven.
  5. Add table-driven classifier regression tests covering docs-only, release-note scripts, docs-build scripts, runtime, DB tests, dependency manifests, changes to each workflow, mixed changes, renames/deletions and unknown paths. Ensure the tests exercise the actual matcher used by CI.
  6. Show a PR job summary with changed paths, chosen category, reasons, selected/skipped lanes and any fail-closed fallback. Measure CI job-minutes and queue latency before/after (not only the displayed count of skipped checks).
  7. Where practical, use a shared policy/contract test from this repository without forcing language-specific projects into one generic workflow. Repo owners keep their own Go/Rust/TS/Python test implementation and release gates.

Initial static audit (2026-11-07)

Repository Initial finding Priority
sqlalchemy-cubrid Broad scripts/workflow risk expands release-note changes to DB tests (PR #1152) P1
pycubrid Broad scripts and ci.yml risk; refine only safe release/tooling paths while preserving protocol, test setup and CI-policy guards P1
cubrid-client Every PR runs Node 18/20/22/24/26 build/test/coverage (no path classification) P1
cubrid-go, cubrid-rs Every PR reaches integration job (no path classification) P2
gorm-cubrid All PRs run Go matrix; main pushes run DB integration even for docs-only changes P2
drizzle-cubrid, sea-orm-cubrid No docs-only exclusion; full TypeScript/Rust tests run on all PRs P2
terraform-zadara-k8s Terraform plan triggers on all PRs and non-main branch pushes, even documentation-only diffs P1
cubrid-kubernetes-operator Has per-workflow impact classifier, fail-closed fallback and inline regression cases — reference pattern Reference
cubrid-mcp-server Has explicit docs/tooling/runtime scopes and classifier tests Reference
cubrid-cookbook-python Has explicit docs/recipe/DB tier selection and classifier tests; some unconditional lightweight checks are intentional Reference
cubrid-benchmark Bench workflow has path-scoped PR/push triggers No immediate finding
showcase Docs/layout and Pages workflows use path-scoped triggers No immediate finding
brand Single brand-specific validation job; no independent DB matrix No immediate finding
cubrid-lab.github.io Deployment on main changes; documentation-only site repository No immediate finding
.github Shared workflows; define contract and central audit record here Owner
cubrid Workflow is changed-file aware for C++ policy checks; perform targeted follow-up if job-minute evidence indicates waste Monitor

Acceptance criteria

  • Publish shared written policy and a review checklist with job impact examples.
  • Open repository-local issues for confirmed over-selection; link them here.
  • Each changed classifier has automated regression cases, safe missing-diff handling, and a required aggregate gate.
  • Verify doc-only PRs skip DB/live and broad matrix jobs, with valid required-status conclusions.
  • Keep full validation for runtime-risk and main/release operations.
  • Record real before/after job-minutes and runtime of representative PRs.
  • Audit all 18 repositories' workflow changes against this standard and record exceptions.

Scope: CI selection and safety, not removal of test coverage or weakening publication checks.

Activity

  1. yeongseon commented on Nov 7, 2026

    @yeongseon
    ContributorAuthor

    Repository-local implementation tracking (2026-11-07)

    The initial audit covered 18 organization repositories; the following repository-owned issues and implementation PRs have been opened. These are not merged yet. A CI-policy PR is expected to execute the broad self-test; a separate docs-only canary must prove selective skipping and stable branch-protection behavior.

    Repository Issue Implementation
    sqlalchemy-cubrid #1154 PR #1155
    pycubrid #1157 PR #1158
    cubrid-client #215 PR #216
    cubrid-go #58 PR #59
    cubrid-rs #31 PR #32
    gorm-cubrid #41 PR #42
    drizzle-cubrid #152 PR #153
    sea-orm-cubrid #24 PR #25
    terraform-zadara-k8s #70 PR #71

    Organization standard: PR #75.

    Open validation caveats: drizzle-cubrid CI currently fails its unchanged 95% coverage gate (54 tests pass; reported line coverage ~87%, branch coverage ~67%); tracked separately in #154 without reducing thresholds. Both Python CI-policy PRs required follow-up to honor existing setup-uv, workflow-install and format contracts; they have been updated and require passing final-head CI. Do not merge with red checks or unverified required-check migration.

  2. yeongseon commented on Nov 7, 2026

    @yeongseon
    ContributorAuthor

    Organization CI policy rollout status (2026-11-07)

    The shared written policy and contributor link are merged: PR #75, including fail-closed rules, required-status gates, targeted release tooling tests, documentation examples and rollout checklist. The 18-repository static workflow audit and repository-local issues are recorded in this issue's existing comment.

    Implementation verification is not finished, so the umbrella issue remains open. At this check:

    Remaining acceptance: demonstrate docs-only vs runtime-risk selected lanes, confirm required statuses remain satisfiable, preserve full main/release validation, and record actual CI job-minutes before/after. Do not substitute green on a ci.yml-changing PR for a docs-only canary, and do not merge a failing gate.

  3. yeongseon commented on Nov 9, 2026

    @yeongseon
    ContributorAuthor

    Closing the organization-wide CI policy/tracker scope as not planned under the repository-local process decision. GitHub does not inherit engineering policy Markdown into individual repositories, and each repository already owns its CI impact classifier, test matrix, required status checks and release safeguards. We should not install a second central contract or mandate uniform impact categories.

    Existing local tasks (for example sqlalchemy-cubrid #1154, pycubrid #1157, cubrid-go #58 and cubrid-rs #31) retain their independent ownership and are not being cancelled. Repo-specific CI optimization remains valid only when motivated and tested against that repository's actual failure modes.

    The central document retirement is being tracked separately in #84; no CI gates were weakened by this decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions