Make docker_server nginx max payload size configurable

[FredLiu876]

Problem

The nginx reverse proxy Truss generates in front of a docker_server (custom server) container hardcodes client_max_body_size to 64M:

• truss/base/constants.py:34 — TRUSSLESS_MAX_PAYLOAD_SIZE = "64M"
• truss/contexts/image_builder/serving_image_builder.py:461 — passed into proxy.conf.jinja

Custom-server users get a 413 at exactly 64 MiB with no way to raise it. It reads like a platform ingress limit (our docs say the ingress cap is 100 MB), but it is generated into the customer's own image, so nothing on their side or in their Truss config controls it.

This came from a customer who bracketed it precisely: highest accepted body 66,837,864 B (63.74 MiB), lowest rejected 67,850,445 B (64.71 MiB). 64M = 67,108,864 B sits inside that bracket. A request trace confirmed every Baseten hop passed the full body through and the 413 was generated after dispatch into the model pod.

Change

Adds an optional docker_server.max_payload_size, validated against nginx size syntax (integer with optional k/m/g suffix), threaded into the nginx template. Falls back to TRUSSLESS_MAX_PAYLOAD_SIZE when unset.

docker_server:
  start_command: python main.py
  server_port: 8000
  predict_endpoint: /predict
  readiness_endpoint: /health
  liveness_endpoint: /health
  max_payload_size: 128M

Default behavior is unchanged — a config without the field renders byte-identical nginx output.

Sizing caveat for reviewers

This is why the field is opt-in and the default is untouched.

proxy.conf.jinja sets client_body_buffer_size 4M and points client_body_temp_path at /dev/shm, so any body over 4 MB spills into pod memory. /dev/shm is an explicit memory-backed emptyDir (operator model.service.template.yaml.jinja:283-286, mounted at :906-907) and its sizeLimit scales with GPU count, not instance RAM (operator/core/utils/workload_utils.py:120-138):

tier	shm per GPU
CPU-only	1 GiB flat, regardless of RAM
T4 / A10G / L4 / L40S / A100 / V100	1 GiB x N
H100 / H200 / RTX-PRO-6000	8 GiB x N
B200 / B300 / GB300	32 GiB x N

The tmpfs counts against the container memory limit — actions.py:1687 says so directly ("the tempfs usage is limited by the container memory limit"). So oversizing this gives you ENOSPC past sizeLimit, or an OOMKill if shm growth blows the cgroup. The template also sets error_log /dev/null, so that failure is silent.

That is the argument against simply raising TRUSSLESS_MAX_PAYLOAD_SIZE. The constant is baked into every docker_server image: on the 1 GiB tier, 128 MiB bodies cap out around 8 concurrent, and there are ~6,400 CPU-only docker_server models on 1x2 (1 GiB shm inside a 2 GiB cgroup) plus ~1,800 on single-GPU L4/A10G/T4. An opt-in field leaves all of them untouched.

Open question for reviewers: should the validator enforce an upper bound instead of accepting 1G? A safer long-term fix may be moving client_body_temp_path off /dev/shm onto a small disk-backed emptyDir — #2163 moved it to /dev/shm only to dodge read-only filesystems, which a dedicated emptyDir also solves.

Note this does not by itself raise the end-to-end ceiling: the WP ingress controller is proxy-body-size: 100m, so 100 MiB is the most that is reachable over the public path today.

Test plan

• truss/tests/test_config.py::test_docker_server_invalid_max_payload_size — rejects 64MB
• truss/tests/contexts/image_builder/test_serving_image_builder.py::test_nginx_config_disables_disk_writes — parametrized: default renders 64M, explicit 128M renders 128M

uv run pytest truss/tests/test_config.py truss/tests/contexts/image_builder/test_serving_image_builder.py -m 'not integration' — 287 passed, 3 skipped. ruff check / ruff format --check clean. bin/generate_truss_config_schema.py --check up to date.

Draft — opening for direction on the sizing questions above before marking ready.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}