Skip to content

feat: reduce ci job count #2104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
thesimplekid wants to merge 1 commit into
base: main
Choose a base branch
from
+70 −114
Open

feat: reduce ci job count #2104

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
182 changes: 68 additions & 114 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ env:

jobs:
pre-commit-checks:
name: "Cargo fmt, typos"
name: "Cargo fmt, Nix fmt, typos"
runs-on: self-hosted
timeout-minutes: 30
steps:
Expand All @@ -36,11 +36,13 @@ jobs:
continue-on-error: true
- name: Cargo fmt
run: nix develop -i -L .#stable --command cargo fmt --check
- name: Nix fmt
run: nix develop -i -L .#stable --command bash -c "find . -name '*.nix' -not -path './.git/*' -not -path './.jj/*' -print0 | xargs -0 nixpkgs-fmt --check"
- name: typos
run: nix develop -i -L .#stable --command typos

quick-check:
name: "Quick Check (lint, clippy, unit tests)"
name: "Quick Check (workspace clippy)"
runs-on: self-hosted
timeout-minutes: 30
needs: pre-commit-checks
Expand All @@ -53,36 +55,45 @@ jobs:
authToken: ${{ secrets.CACHIX_AUTH_TOKEN }}
useDaemon: false
continue-on-error: true
- name: Run quick-check
run: nix develop -i -L .#stable --command just quick-check
- name: Run workspace clippy
run: nix develop -i -L .#stable --command cargo clippy --workspace --all-targets -- -D warnings

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore unit test execution in quick-check

This now only runs clippy. I checked the rest of this workflow and the flake-discovered check matrix: no remaining job runs the removed just quick-check/test-units workspace unit suite, and crates such as cdk-bdk and cdk-http-client are not included in clippyAndTestChecks, so their existing unit tests would no longer execute in CI. Please keep the unit-test portion here or add those crates to the Nix test matrix before dropping just quick-check.

Useful? React with 👍 / 👎.


# Discover example checks from flake - single source of truth
discover-examples:
name: "Discover examples"
# Discover CI check groups from flake - single source of truth
discover_ci_groups:
name: "Discover CI groups"
runs-on: self-hosted
timeout-minutes: 5
outputs:
examples: ${{ steps.examples.outputs.examples }}
examples: ${{ steps.groups.outputs.examples }}
checks: ${{ steps.groups.outputs.checks }}
msrv_checks: ${{ steps.groups.outputs.msrv_checks }}
wasm_checks: ${{ steps.groups.outputs.wasm_checks }}
steps:
- name: checkout
uses: actions/checkout@v5
- name: Get example check names
id: examples
- name: Get check groups
id: groups
run: |
# Get all example check names (prefixed with "example-")
examples=$(nix eval .#checks.x86_64-linux --apply 'attrs: builtins.filter (n: builtins.substring 0 8 n == "example-") (builtins.attrNames attrs)' --json)
echo "examples=$examples" >> $GITHUB_OUTPUT
checks=$(nix eval .#checks.x86_64-linux --apply 'attrs: builtins.filter (n: n != "pre-commit-check" && n != "doc-tests" && n != "strict-docs" && n != "ffi-tests" && builtins.substring 0 8 n != "example-" && builtins.substring 0 5 n != "msrv-" && builtins.substring 0 5 n != "wasm-") (builtins.attrNames attrs)' --json)
msrv_checks=$(nix eval .#checks.x86_64-linux --apply 'attrs: builtins.filter (n: builtins.substring 0 5 n == "msrv-") (builtins.attrNames attrs)' --json)
wasm_checks=$(nix eval .#checks.x86_64-linux --apply 'attrs: builtins.filter (n: builtins.substring 0 5 n == "wasm-") (builtins.attrNames attrs)' --json)
{
echo "examples=$examples"
echo "checks=$checks"
echo "msrv_checks=$msrv_checks"
echo "wasm_checks=$wasm_checks"
} >> "$GITHUB_OUTPUT"
echo "Found examples: $examples"
echo "Found checks: $checks"
echo "Found MSRV checks: $msrv_checks"
echo "Found WASM checks: $wasm_checks"

examples:
name: "Example: ${{ matrix.example }}"
name: "Examples"
runs-on: self-hosted
timeout-minutes: 30
needs: [pre-commit-checks, discover-examples]
strategy:
fail-fast: true
matrix:
example: ${{ fromJson(needs.discover-examples.outputs.examples) }}
needs: [pre-commit-checks, discover_ci_groups]
steps:
- name: checkout
uses: actions/checkout@v5
Expand All @@ -92,45 +103,33 @@ jobs:
authToken: ${{ secrets.CACHIX_AUTH_TOKEN }}
useDaemon: false
continue-on-error: true
- name: Build example
run: nix build -L .#${{ matrix.example }}
- name: Run example
- name: Build and run examples
run: |
# Extract binary name by removing "example-" prefix
BINARY_NAME="${{ matrix.example }}"
BINARY_NAME="${BINARY_NAME#example-}"
./result/bin/$BINARY_NAME

# Discover clippy + test checks from flake - single source of truth
discover-checks:
name: "Discover checks"
runs-on: self-hosted
timeout-minutes: 5
outputs:
checks: ${{ steps.checks.outputs.checks }}
steps:
- name: checkout
uses: actions/checkout@v5
- name: Get check names
id: checks
run: |
# Get all check names except pre-commit-check, example-*, msrv-*, wasm-*, doc-tests, strict-docs, ffi-tests
# Those have their own dedicated CI jobs
checks=$(nix eval .#checks.x86_64-linux --apply 'attrs: builtins.filter (n: n != "pre-commit-check" && n != "doc-tests" && n != "strict-docs" && n != "ffi-tests" && builtins.substring 0 8 n != "example-" && builtins.substring 0 5 n != "msrv-" && builtins.substring 0 5 n != "wasm-") (builtins.attrNames attrs)' --json)
echo "checks=$checks" >> $GITHUB_OUTPUT
echo "Found checks: $checks"
set -euo pipefail
examples='${{ needs.discover_ci_groups.outputs.examples }}'
examples=$(printf '%s' "$examples" | sed 's/[][]//g; s/"//g; s/,/ /g')
for example in $examples; do
echo "::group::Build $example"
nix build -L ".#$example"
echo "::endgroup::"

binary_name="${example#example-}"
echo "::group::Run $binary_name"
./result/bin/"$binary_name"
echo "::endgroup::"
done

# Dynamic clippy + test matrix - uses cached deps from Cachix
# Each check runs both clippy and unit tests for that crate/feature combination
clippy-and-test:
name: "Check: ${{ matrix.check }}"
runs-on: self-hosted
timeout-minutes: 30
needs: [pre-commit-checks, discover-checks]
needs: [pre-commit-checks, discover_ci_groups]
strategy:
fail-fast: false
matrix:
check: ${{ fromJson(needs.discover-checks.outputs.checks) }}
check: ${{ fromJson(needs.discover_ci_groups.outputs.checks) }}
steps:
- name: checkout
uses: actions/checkout@v5
Expand Down Expand Up @@ -160,17 +159,17 @@ jobs:
authToken: ${{ secrets.CACHIX_AUTH_TOKEN }}
useDaemon: false
continue-on-error: true
- name: Build nextest archive
run: nix build -L .#itest-archive
- name: Build harness binaries (warm cache)
- name: Build nextest archive and harness binaries
run: |
nix build -L .#start-fake-mint
nix build -L .#start-regtest-mints
nix build -L .#start-fake-auth-mint
nix build -L .#start-regtest
nix build -L .#signatory
nix build -L .#cdk-payment-processor
nix build -L .#cdk-mintd-grpc
nix build -L \
.#itest-archive \
.#start-fake-mint \
.#start-regtest-mints \
.#start-fake-auth-mint \
.#start-regtest \
.#signatory \
.#cdk-payment-processor \
.#cdk-mintd-grpc

# Tests that require a running PostgreSQL instance
postgres-tests:
Expand Down Expand Up @@ -366,9 +365,15 @@ jobs:
run: echo "path=$(nix build .#itest-archive --print-out-paths --no-link)/itest-archive.tar.zst" >> $GITHUB_OUTPUT
- name: Test fake mint
run: nix develop -i -L .#regtest --command bash -c "export TMPDIR=$(mktemp -d -p $PWD) && export CDK_ITEST_ARCHIVE='${{ steps.archive.outputs.path }}' && just test-pure ${{ matrix.database }}"
- name: Unit tests and basic mint test
- name: Basic mint integration test
if: matrix.database == 'memory'
run: nix develop -i -L .#regtest --command bash -c "export TMPDIR=$(mktemp -d -p $PWD) && export CDK_ITEST_ARCHIVE='${{ steps.archive.outputs.path }}' && just test"
run: |
nix develop -i -L .#regtest --command bash -c "
set -euo pipefail
export TMPDIR=\$(mktemp -d -p \"\$PWD\")
export CDK_ITEST_ARCHIVE='${{ steps.archive.outputs.path }}'
cargo nextest run --archive-file \"\$CDK_ITEST_ARCHIVE\" --workspace-remap . -E \"binary(/^mint$/)\"
"

payment-processor-itests:
name: "Payment processor tests"
Expand All @@ -394,33 +399,15 @@ jobs:
- name: Test
run: nix develop -i -L .#regtest --command bash -c "export TMPDIR=$(mktemp -d -p $PWD) && export CDK_ITEST_ARCHIVE='${{ steps.archive.outputs.path }}' && just itest-payment-processor ${{matrix.ln}}"

# Discover MSRV checks from flake - single source of truth
discover-msrv-checks:
name: "Discover MSRV checks"
runs-on: self-hosted
timeout-minutes: 5
outputs:
checks: ${{ steps.checks.outputs.checks }}
steps:
- name: checkout
uses: actions/checkout@v5
- name: Get MSRV check names
id: checks
run: |
# Get all MSRV check names (prefixed with "msrv-")
checks=$(nix eval .#checks.x86_64-linux --apply 'attrs: builtins.filter (n: builtins.substring 0 5 n == "msrv-") (builtins.attrNames attrs)' --json)
echo "checks=$checks" >> $GITHUB_OUTPUT
echo "Found MSRV checks: $checks"

msrv-build:
name: "MSRV: ${{ matrix.check }}"
runs-on: self-hosted
timeout-minutes: 30
needs: [pre-commit-checks, discover-msrv-checks]
needs: [pre-commit-checks, discover_ci_groups]
strategy:
fail-fast: true
matrix:
check: ${{ fromJson(needs.discover-msrv-checks.outputs.checks) }}
check: ${{ fromJson(needs.discover_ci_groups.outputs.msrv_checks) }}
steps:
- name: checkout
uses: actions/checkout@v5
Expand All @@ -433,33 +420,15 @@ jobs:
- name: Build
run: nix build -L .#checks.x86_64-linux.${{ matrix.check }}

# Discover WASM checks from flake - single source of truth
discover-wasm-checks:
name: "Discover WASM checks"
runs-on: self-hosted
timeout-minutes: 5
outputs:
checks: ${{ steps.checks.outputs.checks }}
steps:
- name: checkout
uses: actions/checkout@v5
- name: Get WASM check names
id: checks
run: |
# Get all WASM check names (prefixed with "wasm-")
checks=$(nix eval .#checks.x86_64-linux --apply 'attrs: builtins.filter (n: builtins.substring 0 5 n == "wasm-") (builtins.attrNames attrs)' --json)
echo "checks=$checks" >> $GITHUB_OUTPUT
echo "Found WASM checks: $checks"

check-wasm:
name: "WASM: ${{ matrix.check }}"
runs-on: self-hosted
timeout-minutes: 30
needs: [pre-commit-checks, discover-wasm-checks]
needs: [pre-commit-checks, discover_ci_groups]
strategy:
fail-fast: true
matrix:
check: ${{ fromJson(needs.discover-wasm-checks.outputs.checks) }}
check: ${{ fromJson(needs.discover_ci_groups.outputs.wasm_checks) }}
steps:
- name: checkout
uses: actions/checkout@v5
Expand Down Expand Up @@ -510,7 +479,7 @@ jobs:
docker compose -f misc/keycloak/docker-compose-recover.yml down

docs:
name: "Documentation tests"
name: "Documentation checks"
runs-on: self-hosted
timeout-minutes: 30
needs: pre-commit-checks
Expand All @@ -525,21 +494,6 @@ jobs:
continue-on-error: true
- name: Run doc tests
run: nix build -L .#checks.x86_64-linux.doc-tests

strict-docs:
name: "Strict Documentation Check"
runs-on: self-hosted
timeout-minutes: 30
needs: docs
steps:
- name: checkout
uses: actions/checkout@v5
- uses: cachix/cachix-action@v17
with:
name: cashudevkit
authToken: ${{ secrets.CACHIX_AUTH_TOKEN }}
useDaemon: false
continue-on-error: true
- name: Check docs with strict warnings
run: nix build -L .#checks.x86_64-linux.strict-docs

Expand Down
2 changes: 2 additions & 0 deletions flake.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -750,9 +750,11 @@
];

"lightning-and-api" = [
"-p cdk-http-client"
"-p cdk-axum"
"-p cdk-axum --no-default-features"
"-p cdk-axum --no-default-features --features redis"
"-p cdk-bdk"
"-p cdk-cln"
"-p cdk-lnd"
"-p cdk-lnbits"
Expand Down
Loading