Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 35 additions & 39 deletions .github/workflows/musl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,52 +28,48 @@ jobs:
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/runner=amd64-large/image=ubuntu24-full-x64-pre-v2/tag=rust-build-musl', github.run_id)
|| 'ubuntu-latest' }}
container:
image: ghcr.io/rust-lang/rust:1-alpine3.21
credentials:
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
env:
# -crt-static makes build-script binaries dynamically linked so bindgen
# (custom-labels, a transitive dep of vortex-io) can dlopen libclang; a
# fully static musl build script panics with "Dynamic loading not supported".
RUSTFLAGS: "-A warnings -C target-feature=-crt-static"
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
# Build and test natively inside Alpine so the musl C/C++ toolchain (gcc, g++) and
# GNU ld are used. Cross-compiling from the glibc host is not viable: musl-tools
# ships no musl-g++ for C++ deps such as custom-labels, and zig's linker rejects the
# --dynamic-list arg those deps emit. Checkout runs on the host because JS actions
# cannot run inside an Alpine (musl) container, so this job provisions rustup +
# nextest itself rather than using the prebuilt toolchain / sccache setup that the
# glibc test runners use.
#
# Exclusions:
# - CUDA crates have no CUDA toolkit / musl target (as in the wasm build).
# - vortex-duckdb and its dependents can't build standalone for musl: its
# build.rs has no prebuilt DuckDB for musl. The DuckDB extension builds it
# against a musl DuckDB it compiles itself, so it is exercised there instead.
- name: Build & test workspace for x86_64-unknown-linux-musl
- name: Install Alpine packages
# tzdata provides /usr/share/zoneinfo; without it Alpine has no timezone
# database and datetime tests fail with "failed to find time zone `UTC`".
run: |
apk add --no-cache build-base clang clang-dev llvm-dev cmake make perl \
pkgconf protobuf protobuf-dev openssl-dev zstd-dev git curl ca-certificates \
python3 python3-dev tar tzdata bash

- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6

- name: Install nextest
shell: bash
# Prebuilt static musl nextest binary; building it from source would
# roughly double the cold-cache compile time of this job.
run: |
curl -LsSf https://get.nexte.st/latest/linux-musl | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin

- name: Run workspace tests for x86_64-unknown-linux-musl
shell: bash
run: |
docker run --rm -v "${GITHUB_WORKSPACE}:/work" -w /work alpine:3.21 sh -euc '
# tzdata provides /usr/share/zoneinfo; without it Alpine has no timezone
# database and datetime tests fail with "failed to find time zone `UTC`".
apk add --no-cache build-base clang clang-dev llvm-dev cmake make perl \
pkgconf protobuf protobuf-dev openssl-dev zstd-dev bash git curl ca-certificates \
python3 python3-dev tar tzdata
# -crt-static makes build-script binaries dynamically linked so bindgen
# (custom-labels, a transitive dep of vortex-io) can dlopen libclang; a
# fully static musl build script panics with "Dynamic loading not supported".
export CARGO_HOME=/root/.cargo CARGO_TARGET_DIR=/tmp/target RUSTFLAGS="-A warnings -C target-feature=-crt-static"
export PATH="$CARGO_HOME/bin:$PATH"
curl --proto =https --tlsv1.2 -sSf https://sh.rustup.rs \
| sh -s -- -y --profile minimal --default-toolchain none
rustup show
# Prebuilt static musl nextest binary; building it from source would
# roughly double the cold-cache compile time of this job.
curl -LsSf https://get.nexte.st/latest/linux-musl \
| tar zxf - -C "$CARGO_HOME/bin"
# The workspace is bind-mounted from the host runner and owned by a
# different uid than the container root, so git rejects it as "dubious
# ownership" and `git rev-parse HEAD` returns empty. vortex-bench and the
# benchmarks website embed that SHA in snapshots (redacted to <commit-sha> /
# <build-sha>); an empty SHA breaks the redaction and fails 10 snapshot tests.
git config --global --add safe.directory /work
cargo nextest run --cargo-profile ci --locked --workspace --no-fail-fast \
--exclude vortex-cuda --exclude vortex-cub --exclude vortex-nvcomp \
--exclude gpu-scan-cli --exclude vortex-test-e2e-cuda \
--exclude vortex-duckdb --exclude duckdb-bench --exclude vortex-sqllogictest
'
cargo nextest run --cargo-profile ci --locked --workspace --no-fail-fast \
--exclude vortex-cuda --exclude vortex-cub --exclude vortex-nvcomp \
--exclude gpu-scan-cli --exclude vortex-test-e2e-cuda \
--exclude vortex-duckdb --exclude duckdb-bench --exclude vortex-sqllogictest \
--exclude vortex-bench --exclude lance-bench --exclude datafusion-bench --exclude vortex-datafusion \
--exclude compress-bench --exclude random-access-bench --exclude vortex-bench-server

- name: Alert incident.io
if: failure() && github.event_name == 'push' && github.ref == 'refs/heads/develop'
Expand Down
Loading