-
Notifications
You must be signed in to change notification settings - Fork 995
Add RISC-V smoke test on QEMU #19399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
7a5889c
04fd660
3f9eaa1
af79b03
27c02de
10a6f58
f98a5f6
d0cbb51
6fbf6d7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| #!/usr/bin/env bash | ||
| # Copyright 2026 The ExecuTorch Authors. | ||
| # | ||
| # This source code is licensed under the BSD-style license found in the | ||
| # LICENSE file in the root directory of this source tree. | ||
|
|
||
| # CI wrapper: install RISC-V cross-compile + qemu-user tooling, then run the | ||
| # RISC-V Phase 1 smoke test (export, cross-compile, qemu-user execution) via | ||
| # examples/riscv/run.sh. The bundled-IO comparison and Test_result: PASS | ||
| # check are done by run.sh. | ||
|
|
||
| set -eu | ||
|
|
||
| script_dir=$(realpath "$(dirname "${BASH_SOURCE[0]}")") | ||
| et_root_dir=$(realpath "${script_dir}/../..") | ||
|
|
||
| bash "${et_root_dir}/examples/riscv/setup.sh" | ||
| bash "${et_root_dir}/examples/riscv/run.sh" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| name: Test RISC-V QEMU smoke | ||
|
|
||
| permissions: | ||
| id-token: write | ||
| contents: read | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| timeout: | ||
| description: 'Per-job timeout in minutes' | ||
| required: false | ||
| type: number | ||
| default: 30 | ||
|
|
||
| jobs: | ||
| run: | ||
| uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main | ||
| with: | ||
| runner: linux.2xlarge | ||
| docker-image: ci-image:executorch-ubuntu-22.04-gcc11 | ||
| submodules: 'recursive' | ||
| ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }} | ||
| timeout: ${{ inputs.timeout }} | ||
| script: | | ||
| CONDA_ENV=$(conda env list --json | jq -r ".envs | .[-1]") | ||
| conda activate "${CONDA_ENV}" | ||
|
|
||
| source .ci/scripts/utils.sh | ||
| install_executorch "--use-pt-pinned-commit" | ||
|
|
||
| bash .ci/scripts/test_riscv_qemu.sh |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| name: RISC-V | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| - release/* | ||
| tags: | ||
| - ciflow/trunk/* | ||
| pull_request: | ||
| paths: | ||
| - .github/workflows/riscv64.yml | ||
| - .ci/scripts/test_riscv_qemu.sh | ||
| - tools/cmake/preset/riscv64_linux.cmake | ||
| - examples/riscv/** | ||
| workflow_dispatch: | ||
| schedule: | ||
| - cron: '0 10 * * *' # Runs daily at 2 AM PST | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}-${{ github.event_name == 'workflow_dispatch' }}-${{ github.event_name == 'schedule' }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| test-riscv: | ||
| name: test-riscv | ||
| uses: ./.github/workflows/_test_riscv.yml | ||
| permissions: | ||
| id-token: write | ||
| contents: read |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,6 +9,7 @@ __pycache__/ | |
|
|
||
| # Build and tool-generated files | ||
| arm_test/ | ||
| riscv_test/ | ||
| buck-out/ | ||
| buck2-bin/ | ||
| build/ | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # RISC-V | ||
|
|
||
| Cross-compile `executor_runner` for `riscv64-linux-gnu` and run it under | ||
| `qemu-user-static` against a small bundled program. The end-to-end check | ||
| mirrors the Arm Cortex-M e2e flow: a `Test_result: PASS` line in stdout from | ||
| the bundled-IO comparison path is the pass criterion. | ||
|
|
||
| This is the Phase 1 deliverable for the RISC-V Support RFC at | ||
| [pytorch/executorch#18991][rfc]. The cross-compile and runner artifacts | ||
| (toolchain file, preset, AOT script) are designed to carry over unchanged | ||
| to a hardware-runner job once one becomes available; only the invocation | ||
| step (qemu-user vs. native) would change. | ||
|
|
||
| [rfc]: https://github.com/pytorch/executorch/issues/18991 | ||
|
|
||
| ## Quick start (Ubuntu / Debian) | ||
|
|
||
| ```bash | ||
| examples/riscv/setup.sh # apt: gcc-riscv64-linux-gnu, qemu-user-static | ||
| examples/riscv/run.sh # export, cross-compile, run under qemu-user | ||
| ``` | ||
|
|
||
| The driver does three steps: | ||
|
|
||
| 1. `python examples/riscv/aot_riscv.py` exports a `torch.add` module to | ||
| `riscv_test/add_riscv.bpte` (a BundledProgram with reference outputs | ||
| embedded for two test cases). | ||
| 2. `cmake --preset riscv64-linux` configures the cross-build using | ||
| `examples/riscv/riscv64-linux-gnu-toolchain.cmake` and | ||
| `tools/cmake/preset/riscv64_linux.cmake`. `executor_runner` is built | ||
| against portable kernels with `ET_BUNDLE_IO_ENABLED` defined. | ||
| 3. `qemu-riscv64-static` invokes the runner with `--model_path` pointing at | ||
| the `.bpte`. The runner detects the bundle, runs every embedded test case, | ||
| and emits `Test_result: PASS` (or `FAIL`) per case. | ||
|
|
||
| ## CI | ||
|
|
||
| `.github/workflows/_test_riscv_qemu.yml` is a reusable `workflow_call` | ||
| job (mirroring `_test_cortex_m_e2e.yml`) invoked from `pull.yml` to run on | ||
| every PR. It runs on the standard `linux.2xlarge` x86_64 runner using the | ||
| `executorch-ubuntu-22.04-gcc11` docker image. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Copyright 2026 The ExecuTorch Authors. | ||
| # | ||
| # This source code is licensed under the BSD-style license found in the | ||
| # LICENSE file in the root directory of this source tree. | ||
|
|
||
| """AOT export for the RISC-V Phase 1.0 smoke test. | ||
|
|
||
| Exports a trivial ``torch.add`` module to a BundledProgram (.bpte) that the | ||
| portable executor_runner can load on a riscv64 target and verify against the | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit, perhaps we should plan to add riscv32 support in the future, even if we don't add it now, and not assume march too tightly. I am thinking about embedded uscases.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Absolutely. I don’t have direct input on customers asking for it, but the current scripts can easily be extended to test on qemu-static-riscv32.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| embedded reference output, emitting ``Test_result: PASS`` on success. | ||
| """ | ||
|
|
||
| import argparse | ||
| from pathlib import Path | ||
|
|
||
| import torch | ||
| from executorch.devtools import BundledProgram | ||
| from executorch.devtools.bundled_program.config import MethodTestCase, MethodTestSuite | ||
| from executorch.devtools.bundled_program.serialize import ( | ||
| serialize_from_bundled_program_to_flatbuffer, | ||
| ) | ||
| from executorch.exir import to_edge_transform_and_lower | ||
| from torch.export import export | ||
|
|
||
|
|
||
| class AddModule(torch.nn.Module): | ||
| def forward(self, x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: | ||
| return x + y | ||
|
|
||
|
|
||
| def main() -> None: | ||
| parser = argparse.ArgumentParser(description=__doc__) | ||
| parser.add_argument( | ||
| "--output", | ||
| type=Path, | ||
| default=Path("add_riscv.bpte"), | ||
| help="Output .bpte path", | ||
| ) | ||
| args = parser.parse_args() | ||
|
|
||
| model = AddModule().eval() | ||
| example_inputs = (torch.ones(1, 4), torch.full((1, 4), 2.0)) | ||
|
|
||
| exported = export(model, example_inputs) | ||
| et_program = to_edge_transform_and_lower(exported).to_executorch() | ||
|
|
||
| test_inputs = [ | ||
| (torch.ones(1, 4), torch.full((1, 4), 2.0)), | ||
| (torch.full((1, 4), 3.0), torch.full((1, 4), 4.0)), | ||
| ] | ||
| test_suite = MethodTestSuite( | ||
| method_name="forward", | ||
| test_cases=[ | ||
| MethodTestCase(inputs=inp, expected_outputs=(model(*inp),)) | ||
| for inp in test_inputs | ||
| ], | ||
| ) | ||
|
|
||
| bundled = BundledProgram(et_program, [test_suite]) | ||
| serialized = serialize_from_bundled_program_to_flatbuffer(bundled) | ||
|
|
||
| args.output.parent.mkdir(parents=True, exist_ok=True) | ||
| args.output.write_bytes(serialized) | ||
| print(f"Wrote {args.output} ({len(serialized)} bytes)") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # Copyright 2026 The ExecuTorch Authors. | ||
| # | ||
| # This source code is licensed under the BSD-style license found in the | ||
| # LICENSE file in the root directory of this source tree. | ||
|
|
||
| # CMake toolchain file for cross-compiling to riscv64 Linux glibc using the | ||
| # Ubuntu / Debian gcc-riscv64-linux-gnu and g++-riscv64-linux-gnu packages. | ||
| # Resulting binaries can be executed under qemu-user-static (qemu-riscv64) or | ||
| # directly on a riscv64 Linux host. | ||
|
|
||
| set(CMAKE_SYSTEM_NAME Linux) | ||
| set(CMAKE_SYSTEM_PROCESSOR riscv64) | ||
|
|
||
| set(CMAKE_C_COMPILER | ||
| "riscv64-linux-gnu-gcc" | ||
| CACHE FILEPATH "RISC-V cross C compiler" | ||
| ) | ||
| set(CMAKE_CXX_COMPILER | ||
| "riscv64-linux-gnu-g++" | ||
| CACHE FILEPATH "RISC-V cross C++ compiler" | ||
| ) | ||
| set(CMAKE_AR | ||
| "riscv64-linux-gnu-ar" | ||
| CACHE FILEPATH "RISC-V archiver" | ||
| ) | ||
| set(CMAKE_RANLIB | ||
| "riscv64-linux-gnu-ranlib" | ||
| CACHE FILEPATH "RISC-V ranlib" | ||
| ) | ||
| set(CMAKE_STRIP | ||
| "riscv64-linux-gnu-strip" | ||
| CACHE FILEPATH "RISC-V strip" | ||
| ) | ||
|
|
||
| # Sysroot installed by the apt package gcc-riscv64-linux-gnu. | ||
| set(CMAKE_FIND_ROOT_PATH "/usr/riscv64-linux-gnu") | ||
| set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) | ||
| set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) | ||
| set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) | ||
| set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| #!/usr/bin/env bash | ||
| # Copyright 2026 The ExecuTorch Authors. | ||
| # | ||
| # This source code is licensed under the BSD-style license found in the | ||
| # LICENSE file in the root directory of this source tree. | ||
|
|
||
| # RISC-V Phase 1 smoke test driver (pytorch/executorch#18991): | ||
| # 1. Export a tiny model to a BundledProgram (.bpte) on the x86_64 host. | ||
| # 2. Cross-compile executor_runner for riscv64 Linux glibc. | ||
| # 3. Invoke the runner under qemu-user-static and grep its stdout for the | ||
| # Test_result: PASS marker emitted by the bundled-IO comparison path. | ||
|
|
||
| set -eu | ||
|
|
||
| script_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &> /dev/null && pwd) | ||
| et_root_dir=$(realpath "${script_dir}/../..") | ||
|
|
||
| build_only=false | ||
| build_dir="${et_root_dir}/cmake-out-riscv" | ||
| output_dir="${et_root_dir}/riscv_test" | ||
| qemu="qemu-riscv64-static" | ||
| qemu_timeout="600" | ||
|
|
||
| usage() { | ||
| cat <<EOF | ||
| Usage: $(basename "$0") [options] | ||
| Options: | ||
| --build_only Only export and cross-compile; do not invoke QEMU | ||
| --build_dir=<DIR> CMake build directory (default: ${build_dir}) | ||
| --output_dir=<DIR> Directory for the exported .bpte (default: ${output_dir}) | ||
| --qemu=<BIN> qemu-user binary (default: ${qemu}) | ||
| --timeout=<SECONDS> Maximum QEMU runtime; matches run_fvp.sh --timelimit (default: ${qemu_timeout}) | ||
| -h, --help Show this help | ||
| EOF | ||
| } | ||
|
|
||
| for arg in "$@"; do | ||
| case $arg in | ||
| --build_only) build_only=true ;; | ||
| --build_dir=*) build_dir="${arg#*=}" ;; | ||
| --output_dir=*) output_dir="${arg#*=}" ;; | ||
| --qemu=*) qemu="${arg#*=}" ;; | ||
| --timeout=*) qemu_timeout="${arg#*=}" ;; | ||
| -h|--help) usage; exit 0 ;; | ||
| *) echo "Unknown option: $arg" >&2; usage; exit 1 ;; | ||
| esac | ||
| done | ||
|
|
||
| mkdir -p "${output_dir}" | ||
| bpte_path="${output_dir}/add_riscv.bpte" | ||
|
|
||
| echo "[run.sh] Step 1/3: AOT export on host" | ||
| python "${script_dir}/aot_riscv.py" --output "${bpte_path}" | ||
|
|
||
| echo "[run.sh] Step 2/3: cross-compile executor_runner for riscv64-linux" | ||
| cmake -S "${et_root_dir}" -B "${build_dir}" \ | ||
| --preset riscv64-linux \ | ||
| -DCMAKE_BUILD_TYPE=Release | ||
| cmake --build "${build_dir}" -j"$(nproc)" --target executor_runner | ||
|
|
||
| runner="${build_dir}/executor_runner" | ||
| [[ -x "${runner}" ]] || { echo "[run.sh] runner not found at ${runner}" >&2; exit 1; } | ||
|
|
||
| if file "${runner}" | grep -q "RISC-V"; then | ||
| echo "[run.sh] runner is a RISC-V ELF: $(file -b "${runner}")" | ||
| else | ||
| echo "[run.sh] ERROR: ${runner} does not look like a RISC-V ELF" | ||
| file "${runner}" | ||
| exit 1 | ||
| fi | ||
|
|
||
| if ${build_only}; then | ||
| echo "[run.sh] --build_only set, skipping QEMU invocation" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "[run.sh] Step 3/3: run under ${qemu}" | ||
| hash "${qemu}" 2>/dev/null || { | ||
| echo "[run.sh] ERROR: ${qemu} not found on PATH; install with examples/riscv/setup.sh" >&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| # QEMU_LD_PREFIX points qemu-user at the riscv64 sysroot so the dynamic | ||
| # linker (ld-linux-riscv64-lp64d.so.1) referenced in the ELF resolves. | ||
| export QEMU_LD_PREFIX="${QEMU_LD_PREFIX:-/usr/riscv64-linux-gnu}" | ||
|
|
||
| log_file=$(mktemp) | ||
| trap 'rm -f "${log_file}"' EXIT | ||
|
|
||
| set +e | ||
| timeout --signal=KILL "${qemu_timeout}" "${qemu}" "${runner}" \ | ||
| --model_path="${bpte_path}" \ | ||
| 2>&1 | tee "${log_file}" | ||
| qemu_status=${PIPESTATUS[0]} | ||
| set -e | ||
|
|
||
| echo "[run.sh] qemu exit status: ${qemu_status}" | ||
|
|
||
| if grep -q "Test_result: PASS" "${log_file}"; then | ||
| echo "[run.sh] Bundled I/O check PASSED" | ||
| exit 0 | ||
| elif grep -q "Test_result: FAIL" "${log_file}"; then | ||
| echo "[run.sh] ERROR: Bundled I/O check FAILED" | ||
| exit 1 | ||
| else | ||
| echo "[run.sh] ERROR: No Test_result line found in QEMU output" | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice