refactor(api-proxy): runner as pure nginx config, client streams raw bytes

[rickstaa]

Replaces the Python runner with a stock nginx — the example's thesis ("no Livepeer code in the app") taken to its endpoint: there is no app code at all. Credit to @j0sh for the observation on #41 that this runner could be a static nginx config.

What

• nginx.conf.template is the whole runner (21 lines): /health returns 200 for the orchestrator's poll; /proxy proxies to one pinned Hugging Face model URL and injects the operator's token. MODEL and HF_TOKEN come from the environment via the stock nginx image's template mechanism.
• runner.py and the Dockerfile are deleted; the compose app service is image: nginx:1.27-alpine + a config mount. api-proxy is removed from the CI image matrix (nothing to build).
• The pinned URL is the security model: callers choose nothing but the payload, so the operator's credential can only be spent on the offered model (resolves the confused-deputy concern from feat(api-proxy): modernize to single-shot fixed pricing with HF text-to-image demo #41's review at the config level). A second model = one more runners.json entry + one more nginx service.
• Client speaks the HF payload natively ({"inputs": "<prompt>"}) and receives raw JPEG bytes via call_runner(..., stream=True) → aiter_bytes() (fix(transcode): multi-GPU dispatch, nvenc rate-control bug, GPU H264/H265 #25's streaming path) — no base64 envelope anywhere. Once livepeer-python-gateway#51 lands, the buffered form (result.raw) becomes an alternative.

Tested

• nginx runner standalone: /health 200; /proxy → HF returned 200 image/jpeg (119 KB) with the operator token injected.
• SDK streaming path against the nginx runner: call_runner(stream=True) pulled a valid 1024×1024 JPEG (content_type=image/jpeg).
• Compose config validates; full offchain stack + on-chain path to be exercised in review (host port 8935 was occupied).

[Copilot]

Pull request overview

Refactors the api-proxy example to remove all application code by replacing the Python runner with a stock nginx container driven entirely by a templated config, while updating the client/docs to send the Hugging Face payload directly and receive raw JPEG bytes.

Changes:

• Replace api-proxy’s Python aiohttp runner with an nginx.conf.template-only runner and use nginx:1.27-alpine in compose.
• Update the api-proxy client to call /proxy with the HF payload and read the image via call_runner(..., stream=True) + aiter_bytes().
• Remove api-proxy from the CI image build matrix (nothing custom to build anymore) and update docs to match the new transport.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
README.md	Updates the examples table to reflect api-proxy transport as raw JPEG bytes.
api-proxy/runner.py	Deleted; removes the Python runner implementation.
api-proxy/README.md	Rewrites documentation to describe the nginx-only runner and raw-byte transport.
api-proxy/nginx.conf.template	Adds the nginx-only runner config (health + pinned HF proxy).
api-proxy/Dockerfile	Deleted; no longer building a custom image.
api-proxy/compose.yml	Switches the app service to nginx:1.27-alpine with templated config mount + env vars.
api-proxy/client.py	Sends HF payload directly and streams raw JPEG bytes back.
.github/workflows/images.yml	Removes api-proxy from the image build workflow matrix and path filters.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

api-proxy/client.py:70

• The client writes whatever bytes come back from the runner to a .jpg without checking the HTTP status/content-type. If Hugging Face returns an error (non-200 or JSON error body), this will silently produce an invalid image file instead of failing fast with a helpful message.

        async with stream:
            image = b"".join([chunk async for chunk in stream.aiter_bytes()])

        out_path = Path(args.output).expanduser()
        out_path.write_bytes(image)

api-proxy/README.md:22

• This section claims callers "choose nothing but the prompt", but the nginx runner forwards the request body verbatim to Hugging Face. Callers can include additional JSON fields (e.g., model parameters) beyond just inputs, so the documentation should describe the constraint accurately (pinned URL/method/args/auth), without implying payload-shape enforcement that doesn't exist.

Everything is operator-side config. `runners.json` names the capability and sets the **fixed per-image price**; the nginx config pins the model URL and holds the credential (`HF_TOKEN`, from `.env`). The pinned URL is also the security model: the operator's credential can only be spent on exactly the offered model. The config pins the method to `POST` and drops the caller's query string too, so the body is the only thing a caller controls: they choose nothing but the prompt, and never see an API key. They discover the capability and pay **per image through Livepeer**, while the operator pays the upstream and prices above the per-image upstream cost.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

api-proxy/README.md:18

• This sentence has an unbalanced inline-code span: it opens a backtick before the Authorization: … header example but never closes it, which breaks Markdown rendering for the rest of the paragraph. Please add the closing backtick after the header snippet (and keep the token placeholder non-secret, e.g. <HF_TOKEN>).

The app is attached as a **static runner**: the orchestrator reads [runners.json](runners.json) via `-liveRunnerConfig` — app id, runner URL, single-shot mode, and the fixed price — and health-polls `/health` (an nginx `return 200`). The `/proxy` location proxies to the pinned model URL (`MODEL` in [compose.yml](compose.yml)) with `Authorization: Bearer <HF_TOKEN>` added. The caller's body is the [Hugging Face text-to-image payload](https://huggingface.co/docs/inference-providers/tasks/text-to-image) forwarded verbatim — `{"inputs": "<prompt>"}` — and the image comes back as **raw JPEG bytes**. The client calls it with `runner_selector` → `call_runner(..., stream=True)` ([client.py](client.py)) — discover, then one **single-shot** call per image, reading the bytes with `aiter_bytes()`; the orchestrator reserves a session per call and releases it when the response returns. Grep `# Livepeer:` in client.py to see the exact calls.

api-proxy/client.py:67

• In streaming mode the code currently buffers the entire response into memory and also never checks stream.status before writing bytes to disk. If Hugging Face returns a non-200 (often JSON) you’ll still write it as api-proxy-out.jpg, and the missing explicit timeout may cause diffusion calls to fail earlier than the runner’s 120s upstream timeout.

Consider using the same streaming pattern as vllm/gateway.py (check stream.status and write chunks as they arrive), and pass an explicit timeout aligned with the nginx proxy_read_timeout.

            runner_url=runner.url.rstrip("/") + "/proxy",
            payload={"inputs": args.prompt},  # the HF payload, forwarded as-is
            signer_url=args.signer.strip() or None,
            stream=True,  # the image comes back as raw bytes, not JSON
        )

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

api-proxy/client.py:73

• The streaming path currently writes whatever bytes come back to a .jpg without validating the HTTP status or content-type. If Hugging Face returns an error (e.g., JSON with non-200 status), the client will still write it to the output file and report success. Consider buffering bytes into a bytearray, then raising a LivepeerGatewayError on non-200 (and optionally rejecting non-image content-types) before writing the file.

        async with stream:
            image = b"".join([chunk async for chunk in stream.aiter_bytes()])

        out_path = Path(args.output).expanduser()
        out_path.write_bytes(image)
        log.info("wrote %s (%d bytes, %s)", out_path, len(image), stream.content_type)

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

api-proxy/client.py:70

• The streaming path writes whatever comes back to a .jpg without checking stream.status (or even that the content type is an image). If HF returns an error JSON/HTML (non-200), this will silently write an invalid JPEG and hide the real failure.

        async with stream:
            image = b"".join([chunk async for chunk in stream.aiter_bytes()])

api-proxy/compose.yml:39

• These comments imply the runners.json app id should exactly “match” MODEL, but the current example uses a shortened app id (livepeer-example/stable-diffusion-3-medium) vs a full HF model id (stabilityai/stable-diffusion-3-medium-diffusers). Reword to avoid misleading readers, and mention the client constant that also needs updating when the app id changes.

      # The one model this runner offers; swap it here, and rename the app id
      # in runners.json to match (the app id is what callers discover).
      - MODEL=stabilityai/stable-diffusion-3-medium-diffusers

api-proxy/README.md:18

• Markdown inline code formatting is broken here: the backtick before the Authorization: snippet is never closed, so the rest of the paragraph renders as code.

The app is attached as a **static runner**: the orchestrator reads [runners.json](runners.json) via `-liveRunnerConfig` — app id, runner URL, single-shot mode, and the fixed price — and health-polls `/health` (an nginx `return 200`). The `/proxy` location proxies to the pinned model URL (`MODEL` in [compose.yml](compose.yml)) with `Authorization: Bearer <HF_TOKEN>` added. The caller's body is the [Hugging Face text-to-image payload](https://huggingface.co/docs/inference-providers/tasks/text-to-image) forwarded verbatim — `{"inputs": "<prompt>"}` — and the image comes back as **raw JPEG bytes**. The client calls it with `runner_selector` → `call_runner(..., stream=True)` ([client.py](client.py)) — discover, then one **single-shot** call per image, reading the bytes with `aiter_bytes()`; the orchestrator reserves a session per call and releases it when the response returns. Grep `# Livepeer:` in client.py to see the exact calls.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

api-proxy/client.py:72

• The streaming path buffers the entire response into memory and writes it to disk even if the runner returns a non-200 status (e.g., an upstream error JSON/HTML would be saved as a .jpg). Since call_runner(stream=True) exposes stream.status/aiter_bytes(), it’s safer to gate on status and stream bytes directly to the output file.

        async with stream:
            image = b"".join([chunk async for chunk in stream.aiter_bytes()])

        out_path = Path(args.output).expanduser()
        out_path.write_bytes(image)

api-proxy/compose.yml:38

• These comments say the MODEL value and the app id in runners.json should “match”, but the current defaults intentionally use different strings (MODEL is the full HF repo; app id is a shorter discovery name). Rewording avoids confusing operators about what must be kept in sync.

      # The one model this runner offers; swap it here, and rename the app id
      # in runners.json to match (the app id is what callers discover).

api-proxy/README.md:24

• This sentence implies the app id must be renamed to literally match MODEL. In this example the app id is a human-facing discovery identifier (and may be a simplified alias), while MODEL is the upstream HF repo path. Clarify the relationship so readers don’t infer a strict 1:1 string requirement.

The app id names the model, not the proxy, because that is what callers discover: they match it exactly, so it has to say what they get. Swapping `MODEL` means renaming the app id with it.