feat(api-proxy): modernize to single-shot fixed pricing with HF text-to-image demo

[rickstaa]

Brings the api-proxy example (previously uncommitted) to mergeable state: a minimal passthrough that lets an orchestrator operator offer an API running somewhere else as a paid capability. Demonstrated against the Hugging Face text-to-image inference API.

Design

• Static registration: the operator attaches the proxy via runners.json (-liveRunnerConfig), the orchestrator health-polls /health — and runner.py contains zero Livepeer code (plain aiohttp, only dep is aiohttp). Everything the operator sets lives operator-side: app id, URL, mode, fixed price (runners.json) and the upstream credential (UPSTREAM_TOKEN env).
• Single-shot + fixed pricing: one call = one bounded unit of work = one flat payment. "mode": "single-shot" + "unit": "fixed" in runners.json (both verified supported in go-livepeer's live runner config). This fills the static + single-shot cell of the axis matrix.
• Operator-held credential: the app injects UPSTREAM_TOKEN as a Bearer header on every forward and drops any caller-sent Authorization — callers pay per call through Livepeer and never see an API key.
• Binary envelope support: text-ish upstream bodies return "body", everything else "body_b64".
• README notes the dynamic counterpart: register_runner can attach several API endpoints at runtime — pointing to livepeer/api-proxy (note: repo currently private).

Model note

FLUX.1-schnell is deprecated on the hf-inference provider (410) — stabilityai/stable-diffusion-3-medium-diffusers is currently the only text-to-image model the provider serves, so it is the default (--model to swap). Output is JPEG (api-proxy-out.jpg).

Tested

• HF route verified live: 200, image/jpeg, valid image; deprecated-model error passes through as "body" with upstream 410.
• Static runner run locally: /health 200; /proxy forwards the envelope with the operator token injected (verified via postman-echo /headers: caller's bogus Authorization dropped, UPSTREAM_TOKEN seen upstream); binary path previously verified end-to-end (131 KB image decoded).
• runners.json schema (mode single-shot, price_info.unit fixed) checked against go-livepeer's ai/runner/live_runner.go.
• Both compose configs validate. Not run: full docker compose stack (host port 8935 occupied) and the on-chain payment path.

[Copilot]

Pull request overview

Adds a mergeable api-proxy example runner that demonstrates single-shot fixed pricing by proxying requests to a real upstream (Hugging Face text-to-image), with binary-safe JSON envelope support and operator-held upstream credentials. This aligns api-proxy with the repo’s post-#39 single-shot client flow and includes CI image-build wiring and docs updates.

Changes:

• Introduces the api-proxy example (runner + client + Docker/compose + env templates) using mode="single-shot" and unit="fixed".
• Implements a generic proxy envelope with text vs binary response handling (body vs body_b64) and server-side Bearer token injection.
• Updates root docs and the images workflow to include/build the new example.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
README.md	Adds api-proxy to supported transports, examples table, and single-shot flow docs.
api-proxy/runner.py	New aiohttp proxy runner registering as single-shot/fixed and forwarding envelope requests upstream.
api-proxy/client.py	New client that discovers the runner and demos HF text-to-image via the envelope, saving the output image.
api-proxy/README.md	New example documentation and run instructions (offchain/on-chain, Docker/non-Docker).
api-proxy/pyproject.toml	New Python project metadata/deps for the example, including SDK source pin to git branch.
api-proxy/Dockerfile	Container build for the example runner.
api-proxy/compose.yml	Offchain compose stack wiring (orchestrator + app) with upstream token injection.
api-proxy/compose.onchain.yml	On-chain overlay enabling signer + priced registration.
api-proxy/.env.example	On-chain/offchain environment template including pricing caps.
api-proxy/.gitignore	Ignores generated demo output image.
.github/workflows/images.yml	Adds api-proxy to the image build matrix and path filters.

Comments suppressed due to low confidence (1)

api-proxy/runner.py:88

• Passing json=body to aiohttp will serialize None to the literal JSON body null (and typically set Content-Type: application/json) when the envelope omits json. That can break upstream GETs and POSTs that expect no body.

Only include the json parameter when the envelope actually provides one.

        async with session.request(
            method,
            upstream,
            headers=headers,
            json=body,

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

api-proxy/runner.py:74

• await request.json() and the subsequent payload.get(...)/(payload.get("headers") or {}).items() will throw and return a 500 if the request body is not valid JSON, not a JSON object, or if headers isn’t an object. Since /proxy is a public entrypoint, return a clear 400 for malformed envelopes and validate headers before iterating.

async def _handle_proxy(request: web.Request) -> web.Response:
    payload = await request.json()
    method = str(payload.get("method", "GET")).upper()
    path = str(payload.get("path", "/"))
    body = payload.get("json")

    # The operator's credential, not the caller's: drop any Authorization the
    # caller sent and inject the stored token instead.
    headers = {
        k: v
        for k, v in (payload.get("headers") or {}).items()
        if k.lower() != "authorization"
    }

api-proxy/runner.py:94

• aiohttp request timeouts are not subclasses of aiohttp.ClientError, so an upstream timeout will currently bubble up as an unhandled exception (500). Handle timeouts explicitly and return an appropriate gateway status code.

            content_type = resp.headers.get("Content-Type", "")
            raw = await resp.read()
    except aiohttp.ClientError as exc:
        return web.json_response({"error": str(exc)}, status=502)

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (4)

api-proxy/runner.py:63

• Invalid/malformed JSON bodies will currently raise out of await request.json() and turn into a 500. For a public proxy endpoint, it’s better to return a 400 with a clear error message.

    payload = await request.json()

api-proxy/runner.py:90

• aiohttp will send a JSON body even when payload["json"] is omitted (it will serialize None to null). That can unintentionally add a request body + Content-Type: application/json to GET/HEAD requests and potentially change upstream behavior. Build the request kwargs and only pass json= when a JSON body was provided.

        async with session.request(
            method,
            upstream,
            headers=headers,
            json=body,
            timeout=aiohttp.ClientTimeout(total=UPSTREAM_TIMEOUT),
        ) as resp:

api-proxy/client.py:65

• If discovery returns no candidates, cursor.candidates[0] raises an IndexError and the user gets a stack trace instead of a clear error. Handle the empty list and raise a LivepeerGatewayError with context.

        cursor = await runner_selector(  # Livepeer: 1
            discovery_url=args.discovery, app=APP_ID
        )
        runner = cursor.candidates[0]
        log.info("app_url=%s", runner.url)

api-proxy/README.md:13

• The README refers to the Hugging Face token as HF_TOKEN, but the app reads UPSTREAM_TOKEN (and compose maps HF_TOKEN → UPSTREAM_TOKEN). Clarify this in the prerequisites sentence so users don’t set the wrong env var when running without Docker.

Prerequisites (Docker, `uv`, and the not-yet-released `livepeer-gateway` SDK — pinned in `pyproject.toml`) and the shared on-chain/payment setup live in the [repo README](../README.md). The demo upstream additionally needs a **Hugging Face API token** (`HF_TOKEN`, from [huggingface.co → settings → tokens](https://huggingface.co/settings/tokens)) with inference-provider credits.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

api-proxy/runner.py:74

• /proxy forwards caller-controlled method, path, and headers without validating their types/shape. Non-object JSON, non-dict headers, or unexpected methods can currently trigger server-side exceptions (500) instead of a clean 400, and it’s safer to explicitly restrict methods and require path to be a rooted path.

    payload = await request.json()
    method = str(payload.get("method", "GET")).upper()
    path = str(payload.get("path", "/"))
    body = payload.get("json")

    # The operator's credential, not the caller's: drop any Authorization the
    # caller sent and inject the stored token instead.
    headers = {
        k: v
        for k, v in (payload.get("headers") or {}).items()
        if k.lower() != "authorization"
    }

api-proxy/runner.py:94

• Upstream timeouts aren’t handled: aiohttp can raise TimeoutError when ClientTimeout(total=...) elapses, but the handler only catches aiohttp.ClientError, which can lead to a 500 instead of a clean 502/504-style proxy error. Also, normalizing Content-Type to lowercase makes the text/binary decision robust to unusual casing.

    try:
        async with session.request(
            method,
            upstream,
            headers=headers,
            json=body,
            timeout=aiohttp.ClientTimeout(total=UPSTREAM_TIMEOUT),
        ) as resp:
            content_type = resp.headers.get("Content-Type", "")
            raw = await resp.read()
    except aiohttp.ClientError as exc:
        return web.json_response({"error": str(exc)}, status=502)

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

api-proxy/runner.py:72

• Authorization injection is currently hardcoded to asterisks, so upstream calls won’t be authenticated even when UPSTREAM_TOKEN is set. This breaks the core proxy behavior (operator-held credential).

    token = request.app["token"]
    if token:
        headers["Authorization"] = f"Bearer {token}"

api-proxy/runner.py:60

• await request.json() and subsequent .get(...) assume the body is valid JSON and a JSON object; malformed JSON or a non-object payload will currently raise and return a 500. Consider returning a 400 for invalid envelopes to keep the proxy robust.

async def _handle_proxy(request: web.Request) -> web.Response:
    payload = await request.json()
    method = str(payload.get("method", "GET")).upper()
    path = str(payload.get("path", "/"))
    body = payload.get("json")

api-proxy/runner.py:14

• The module docstring says the upstream token is injected "as a ******", which is unclear and doesn’t match the intended Bearer auth behavior described elsewhere. This should explicitly say it injects a Bearer token (and the code below should do the same).

This issue also appears on line 69 of the same file.

({"status", "headers", "body"} for text, {"status", "headers", "body_b64"} for
binary). The upstream credential stays server-side: set UPSTREAM_TOKEN and the
app injects it as a Bearer token on every forward — callers pay Livepeer per
call and never see an API key.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

api-proxy/runner.py:84

• json=body is always passed to aiohttp even when the envelope omits json (or when using GET). In aiohttp this will still send a JSON payload (often null) and set a JSON content-type, which can break upstream endpoints that expect an empty-body GET/HEAD.

        async with session.request(
            method,
            upstream,
            headers=headers,
            json=body,
            timeout=aiohttp.ClientTimeout(total=UPSTREAM_TIMEOUT),
        ) as resp:

api-proxy/runner.py:68

• The proxy forwards caller-supplied headers verbatim (other than Authorization). Forwarding hop-by-hop headers like Host, Content-Length, Connection, or Transfer-Encoding can produce invalid upstream requests or conflict with aiohttp's own header handling.

    # The operator's credential, not the caller's: drop any Authorization the
    # caller sent and inject the stored token instead.
    headers = {
        k: v
        for k, v in (payload.get("headers") or {}).items()
        if k.lower() != "authorization"
    }

api-proxy/runner.py:88

• Upstream timeouts from aiohttp.ClientTimeout(...) raise TimeoutError (not aiohttp.ClientError), so they currently surface as a 500 instead of a controlled proxy error. Catch timeouts explicitly and return 504 (or 502) with an error payload.

    except aiohttp.ClientError as exc:
        return web.json_response({"error": str(exc)}, status=502)

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

api-proxy/runner.py:72

• Authorization header injection is currently setting a literal "******" string, so the upstream never receives the operator token and authenticated upstream calls will fail. This should send a Bearer token using the UPSTREAM_TOKEN value.

    token = request.app["token"]
    if token:
        headers["Authorization"] = f"Bearer {token}"

api-proxy/runner.py:14

• The docstring currently says the app injects the upstream credential as "******", which is unclear/incorrect. Update it to explicitly say it injects a Bearer token so the documentation matches the behavior.

binary). The upstream credential stays server-side: set UPSTREAM_TOKEN and the
app injects it as a Bearer token on every forward — callers pay Livepeer per
call and never see an API key.

api-proxy/README.md:21

• This paragraph says the app injects the upstream credential as "******"; it should state that it injects it as a Bearer token so operators understand what header is sent upstream.

Everything the operator sets lives on the operator's side. `runners.json` names the capability, the proxy's URL, and the **fixed per-call price**; the upstream credential (`UPSTREAM_TOKEN`, fed from `HF_TOKEN` by the compose files) sits in the app's environment, and the app injects it as a Bearer header on every forward — any `Authorization` a caller sends is dropped. Callers need no API key of their own: they discover the capability and pay **per call through Livepeer**, while the operator pays the upstream and prices above the per-call upstream cost.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

api-proxy/runner.py:60

• _handle_proxy assumes the request body is valid JSON and a JSON object; if the caller sends invalid JSON or a non-object (e.g. a list), await request.json() / .get() can raise and return a 500 instead of a clear 400. It also assumes headers is an object (.items()), which can similarly crash on bad input.

async def _handle_proxy(request: web.Request) -> web.Response:
    payload = await request.json()
    method = str(payload.get("method", "GET")).upper()
    path = str(payload.get("path", "/"))
    body = payload.get("json")

api-proxy/runner.py:84

• The proxy forwards json=body unconditionally. In aiohttp, passing json=None still sends a JSON body (null) and sets a JSON content-type, which can change semantics for methods like GET and for requests that intentionally have no body. Only include the json parameter when the envelope actually contains a json field.

        async with session.request(
            method,
            upstream,
            headers=headers,
            json=body,
            timeout=aiohttp.ClientTimeout(total=UPSTREAM_TIMEOUT),

api-proxy/client.py:66

• runner_selector() can legally return zero candidates; indexing cursor.candidates[0] will raise IndexError and produce a traceback (it isn't caught by the LivepeerGatewayError handler). Handle the empty case and raise a user-facing error instead.

        cursor = await runner_selector(  # Livepeer: 1
            discovery_url=args.discovery, app=APP_ID
        )
        runner = cursor.candidates[0]
        log.info("app_url=%s", runner.url)

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

api-proxy/runner.py:60

• /proxy assumes a valid JSON object body; malformed JSON or a non-object payload will currently raise and return a 500. It’s better to return a 400 with a clear error and validate the basic envelope fields (including allowed HTTP methods).

async def _handle_proxy(request: web.Request) -> web.Response:
    payload = await request.json()
    method = str(payload.get("method", "GET")).upper()
    path = str(payload.get("path", "/"))
    body = payload.get("json")

api-proxy/client.py:66

• runner_selector() can return an empty candidate list; indexing [0] will raise IndexError and produce a confusing stack trace. Handle the no-runner case and return a clear LivepeerGatewayError instead.

        cursor = await runner_selector(  # Livepeer: 1
            discovery_url=args.discovery, app=APP_ID
        )
        runner = cursor.candidates[0]
        log.info("app_url=%s", runner.url)

api-proxy/client.py:85

• status = result.data.get("status") may be missing or non-integer; comparing directly to 200 can misclassify a successful response (e.g., "200") or produce unclear errors. Coerce to int (with a helpful failure) before checking.

        status = result.data.get("status")
        if status != 200:
            body = result.data.get("body") or result.data.get("error")
            raise LivepeerGatewayError(f"upstream returned {status}: {body}")

api-proxy/runner.py:88

• The upstream request path can raise TimeoutError (from ClientTimeout) and ValueError (invalid URL/headers), and await resp.read() will buffer the entire response in memory. As written, timeouts/misconfig can bubble up as 500s, and large upstream bodies can cause high memory usage. Consider catching these errors and reading with a hard cap + case-insensitive Content-Type normalization.

            content_type = resp.headers.get("Content-Type", "")
            raw = await resp.read()
    except aiohttp.ClientError as exc: