Skip to content

fix(trt_llm): warn when python_version is ignored - #2598

Open
blockgroot wants to merge 1 commit into
basetenlabs:mainfrom
blockgroot:fix/2168-trt-llm-python-version-warning
Open

fix(trt_llm): warn when python_version is ignored#2598
blockgroot wants to merge 1 commit into
basetenlabs:mainfrom
blockgroot:fix/2168-trt-llm-python-version-warning

Conversation

@blockgroot

Copy link
Copy Markdown
Contributor

🚀 What

Adds a warning when python_version is set on a TRT-LLM model, since it currently has no effect there.

Fixes #2168

Problem: for any TRT-LLM model (decoder/Briton, encoder/BEI, encoder-bert, or v2), _fill_trt_llm_versions (truss/contexts/docker_build_setup.py:34-84) forces config.base_image + a fixed python_executable_path/usr/local/briton/venv/bin/python for decoder, /usr/bin/python3 for the others. Once config.base_image is set this way, serving_image_builder.py:970-975 skips the python_version-derived image tag entirely, and server.Dockerfile.jinja:32-34 symlinks python to that fixed path. So python_version is accepted, validated, and stored — but has zero effect on the running container, with no indication to the user. Reproduced directly against TrussConfig:

$ uv run python -c "
from truss.base.truss_config import TrussConfig
from truss.base.constants import TRTLLM_PYTHON_EXECUTABLE
cfg = TrussConfig.from_dict({
    'model_name': 'repro', 'python_version': 'py39',
    'model_metadata': {'tags': ['openai-compatible']},
    'resources': {'accelerator': 'A100', 'use_gpu': True},
    'trt_llm': {'build': {'max_seq_len': 1000, 'max_batch_size': 1, 'base_model': 'decoder',
        'checkpoint_repository': {'repo': 'TinyLlama/TinyLlama-1.1B-Chat-v1.0', 'source': 'HF'}}},
})
print('accepted python_version =', cfg.python_version)
print('but forced to run on:', TRTLLM_PYTHON_EXECUTABLE)
"
accepted python_version = py39
but forced to run on: /usr/local/briton/venv/bin/python

No warning was logged for this mismatch on main.

This override is intentional (traced to #1548 — TRT-LLM base images are backend-supplied with their own bundled Python/TensorRT-LLM runtime), so this PR doesn't change that behavior. It only surfaces it, since nothing currently does.

💻 How

Added a logger.warning in trt_llm_common_validation() (truss/base/trt_llm_config.py), which already runs for every TRT-LLM config (both v1 and v2, via trt_llm_validation) and already contains a similar warning for another misconfiguration (WEIGHTS_ONLY_INT8 quantization on A100). The new check fires when config.python_version differs from the field's default, using the same idiom already established in this function.

Considered raising a ValueError instead, since a few other checks in this function do raise. Went with a warning because setting python_version alongside trt_llm isn't actually invalid — just inert — so hard-failing would break any existing config that harmlessly sets both. Happy to switch to a hard error if you'd prefer that here.

🔬 Testing

Added 3 tests to truss/tests/trt_llm/test_trt_llm_config.py, reusing the existing trtllm_config/trtllm_config_v2 fixtures and the caplog.at_level("WARNING") pattern already used in truss/tests/test_config.py:

  • test_trt_llm_non_default_python_version_warns — fails on main (no warning exists), passes with this change.
  • test_trt_llm_default_python_version_no_warning — the default (py313) config produces no warning; guards against noise on the common case.
  • test_trt_llm_v2_non_default_python_version_warns — same check against a v2 config, since trt_llm_common_validation is shared between both inference stacks.
truss/tests/trt_llm/ -q → 24 passed, 1 skipped

Also ran the full non-integration suite: 1651 passed, 19 skipped, 0 failed (no regressions vs. the main baseline of 1648 passed, 19 skipped), plus ruff check ., ruff format . --check, and pre-commit run --all-files (mypy-local, uv-lock, config-schema) — all clean.

Out of scope: the original issue also reported a torch.compile/Triton InductorError crash on newer PyTorch/Triton versions — per the issue thread itself, that's a deployment-environment issue, not something fixable in truss, so this PR only addresses the python_version part.

TRT-LLM base images ship their own fixed Python interpreter, and
docker_build_setup.py overrides config.base_image + python_executable_path
unconditionally for every TRT-LLM v1 and v2 model. As a result,
python_version was accepted and validated but had no effect on TRT-LLM
deployments, with nothing surfacing that to the user.

Warn in trt_llm_common_validation() when python_version differs from its
default alongside a TRT-LLM config, matching the existing warning idiom
in the same function. This is additive only: the override behavior is
unchanged, existing configs just get visibility into it.

Fixes basetenlabs#2168
@CLAassistant

CLAassistant commented Aug 6, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deployment Regression: torch.compile Triton Compilation Failures on TensorRT-LLM with Python 3.12 / PyTorch 2.5+

2 participants