GreenThread Benchmarking System

Hardware capability verification tool for Linux systems. Run directly on target machines to determine if the hardware can support vLLM-based inferencing workloads. Targets NVIDIA GPU (CUDA) environments on x86_64 and ARM64 architectures, with Ubuntu 24.04 as the primary tested platform.

Build

Requires Go and Task.

task build              # build for the current platform
task vet                # run go vet
task run                # build and run (pass flags with: task run -- --json)

Cross-compilation via Docker

Standard GOOS=linux cross-compilation from macOS does not work because go-nvml uses cgo. Instead, use Docker to produce Linux binaries:

task build-linux        # Linux amd64 binary → dist/gt-benchy
task build-linux-arm64  # Linux arm64 binary → dist/gt-benchy-arm64

These use Dockerfile.build with a golang:1.25 base image. The resulting binary only needs libnvidia-ml.so (from the NVIDIA driver) at runtime on the target machine.

Alternatively, copy the source to the target and build directly:

scp -r . user@target:/tmp/gt-benchy
ssh user@target 'cd /tmp/gt-benchy && go build -o gt-benchy .'

Usage

./gt-benchy          # human-readable output
./gt-benchy --json   # JSON output

CLI Flags

Flag	Description	Default
--json	Output results as JSON	off
--disk-path <path>	Path to check for disk space	/

Exit code 0 = vLLM ready, exit code 1 = not ready.

Installing Benchmark Tools on the Target

gt-benchy can run GPU bandwidth benchmarks if nvbandwidth or the CUDA bandwidthTest demo is installed on the target machine. Without either tool, detection and readiness checks still work — benchmarks are simply skipped.

nvbandwidth (recommended)

sudo apt update
sudo apt install git build-essential cmake libboost-program-options-dev

git clone https://github.com/NVIDIA/nvbandwidth.git /tmp/nvbandwidth
cd /tmp/nvbandwidth
cmake . && make
sudo cp nvbandwidth /usr/local/bin/

bandwidthTest (alternative)

The CUDA demo suite may already be installed with the CUDA toolkit:

ls /usr/local/cuda*/extras/demo_suite/bandwidthTest

If not, install the CUDA samples package for your toolkit version.

What gt-benchy measures

When a benchmark tool is available, gt-benchy runs:

Metric	Description
Host -> Device	CPU to GPU memory transfer bandwidth (GB/s)
Device -> Host	GPU to CPU memory transfer bandwidth (GB/s)
Device -> Device	GPU internal memory copy bandwidth (GB/s)

Unified Memory (DGX Spark / GB10)

On unified memory architectures like the NVIDIA GB10 (DGX Spark), the GPU shares system RAM rather than having dedicated vRAM. NVML does not report memory for these devices — gt-benchy detects this and reports the system RAM pool as GPU-accessible unified memory.

The nvidia_fs DKMS module was built locally but isn't signed with a key that Secure Boot trusts. This is a common issue on Ubuntu with Secure Boot enabled.

GDS Activation and setup

sudo apt-get install nvidia-fs

Check the status with sudo modprobe nvidia_fs if you get

modprobe: ERROR: could not insert 'nvidia_fs': Key was rejected by service

You may need to fix up the module:

If you install the nvidia_fs DKMS module, you have two options:

1. Sign the module (keeps Secure Boot on):

   # Check if MOK (Machine Owner Key) is set up
   sudo mokutil --list-enrolled
   
   # If DKMS was configured with a signing key during nvidia driver install,
   # you may just need to enroll it:
   sudo mokutil --import /var/lib/shim-signed/mok/MOK.der
   # Reboot, follow the blue MOK enrollment screen

2. Disable Secure Boot (simpler but less secure):

   sudo mokutil --disable-validation
   # Reboot, confirm at the blue screen
   

Target Environment

• OS: Linux (Ubuntu 24.04 tested)
• CPU: x86_64 or ARM64
• GPU: NVIDIA with CUDA drivers (compute capability >= 7.0 / Volta+)

vLLM Readiness Criteria

Required (fail if not met):

• Linux OS
• amd64 or arm64 architecture
• NVIDIA driver present
• At least 1 GPU
• Compute capability >= 7.0 (Volta+)

Warnings:

• vRAM < 16 GB per GPU (or unified memory < 16 GB)
• System RAM < 64 GB
• Untested distro