feat: add riscv64 Docker images for cryptography ecosystem#749
feat: add riscv64 Docker images for cryptography ecosystem#749
Conversation
Add riscv64 entries to the Docker image build matrix: - cryptography-runner-ubuntu-rolling:riscv64 - cryptography-manylinux_2_39:riscv64 Uses manylinux_2_39 (first manylinux with riscv64 support) and RISE native riscv64 runners (ubuntu-24.04-riscv). Also adds riscv64 to the binutils install block in the Dockerfile (same as x86_64/ppc64le). This unblocks riscv64 wheel builds for cryptography, bcrypt, and pynacl — the entire pyca ecosystem. Signed-off-by: Bruno Verachten <gounthar@gmail.com>
|
Successful riscv64 build of cryptography on native RISE runner (~10 min): https://github.com/gounthar/cryptography/actions/runs/23718958244 Uses uvx + maturin on ubuntu-24.04-riscv. Build, wheel verification, and release all passed. |
|
10 minutes just to build cryptography without even running the tests will blow way through the time we're willing to allocate for tests. |
|
Fair point, 10 minutes is a lot. I will look into whether cross-compilation from x86_64 can bring that down, or if there is a way to avoid the full Rust build in CI. |
|
We really do not want cross-compilation, it introduces additional mainenance burden, failure modes, and generally makes other platforms more difficult for us to manage. I know this is a high burden, but so far we've managed to clear it with x86-64, ARM64, ARMv7, and PPC64le |
|
As Alex said, the bar here isn't trivial but quite a few arches have done so. If you look at a recent cryptography run (https://github.com/pyca/cryptography/actions/runs/23682974586/usage) you can see our runners typically take 2-4 min (that's total runtime, including setup, build, and test). We do have two slower runners (32-bit windows and intel macOS), both of which we're dropping in an upcoming release because they're both deprecated architectures and slow enough that they inhibit our CI. So, the performance requirement is that it must be approximately 4 minutes end to end at worst from a performance perspective. |
|
Thanks for the context. 2-4 minutes total is a tight budget. The RISE runners are bare-metal but the hardware is not as fast as the GitHub-hosted runners for other architectures. I will keep the PR open in case the runner hardware improves, but I understand if it does not meet the bar right now. |
|
FWIW, I think the best place to start would probably be bcrypt: it's much lower volume of PRs, and we're less performance sensitive there, so we can probably land something there. If we can see that performance for an end to end CI run there isn't too atrocious, I think we can probably land this (image builder) PR. |
|
Good call. bcrypt is a much smaller Rust build than cryptography, so there is a real chance it fits within the time budget. I will fork bcrypt, add riscv64 CI using the RISE runners, and report build and test times here before doing anything else with this PR. If the numbers are acceptable, we can move forward on the image builder side. |
|
bcrypt results on RISE native runner (ubuntu-24.04-riscv): https://github.com/gounthar/bcrypt/actions/runs/23785173905 Cold build (no cache):
With Cargo cache (warm run):
The Rust build itself is fast; bcrypt has only a handful of dependencies (pyo3, bcrypt, bcrypt-pbkdf). The cold 3m53s is dominated by that first compile. Subsequent runs with a populated Cargo cache should fit comfortably within the 2-4 minute budget. Happy to add caching and run again to confirm the warm numbers if that would help. |
|
Warm run (Cargo cache hit) results: https://github.com/gounthar/bcrypt/actions/runs/23785675309
The Cargo cache saves about a minute on the compile step. Total job is still just over 4 minutes because rustup installs fresh each run and pip installs nox/uv each run. Adding nox virtualenv caching and rustup caching would bring the total closer to 2-3 minutes. That said, if the requirement is specifically the |
|
I think we’re fine with that speed for bcrypt. |
Add riscv64 entries to the Docker image build matrix using
manylinux_2_39(first manylinux with riscv64 support) and RISE native runners (ubuntu-24.04-riscv).Changes
.github/workflows/build-docker-images.yml: Add riscv64 runner and manylinux entriescryptography-linux/Dockerfile: Add riscv64 to binutils install blockImages added
cryptography-runner-ubuntu-rolling:riscv64cryptography-manylinux_2_39:riscv64Evidence
All pyca packages build from source on native riscv64 (BananaPi F3, SpacemiT K1). Native runners provided by RISE.
Fixes #748