diff --git a/README.md b/README.md index 1b1cdba..cd78360 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ Example **apps that run on** the Livepeer **live runner** — [go-livepeer](http The point is to **swap the compute without changing your app — permissionlessly, no lock-in**. Your app stays a plain service with little or no Livepeer-specific code, so you're never tied to us. And the network is permissionless: anyone can run or extend it, no one gatekeeps what you deploy, and no single party can take your app down. Write the app once; **move the compute freely**. > [!NOTE] -> Live runners aren't on go-livepeer `main` yet — they live on the [`ja/live-runner`](https://github.com/livepeer/go-livepeer/tree/ja/live-runner) branch. Until it merges, both the orchestrator image and the SDK come from that branch. +> Live runners ship in mainline go-livepeer since [v0.9.0](https://github.com/livepeer/go-livepeer/releases/tag/v0.9.0). The Python SDK still comes from the [`ja/live-runner`](https://github.com/livepeer/livepeer-python-gateway/tree/ja/live-runner) branch until it is published to PyPI. ## How it works @@ -18,7 +18,7 @@ flowchart LR app["Your app
HTTP / WebSocket / trickle"] signer["Remote signer
(on-chain)"] - client -->|"discover → reserve → call → release"| orch + client -->|"single-shot: discover → call · persistent: + reserve/release"| orch orch -->|"forwards your endpoints, unchanged"| app app -.->|"dynamic: register_runner · static: runners.json"| orch signer <-.->|"micropayment tickets"| orch @@ -39,8 +39,8 @@ Need a schema that isn't here? [Open an issue](https://github.com/livepeer/live- | Example | Goal | Registration | Mode | Transport | | ------------------------------ | ----------------------------------------------- | ------------ | ---------------------------------- | ----------------- | -| [`hello-world`](./hello-world) | The simplest app: one request, one response | dynamic | persistent (single-shot by nature) | HTTP (JSON) | -| [`tiles`](./tiles) | Capacity fan-out — one session per tile | dynamic | persistent (single-shot by nature) | HTTP (base64 PNG) | +| [`hello-world`](./hello-world) | The simplest app: one request, one response | dynamic | single-shot | HTTP (JSON) | +| [`tiles`](./tiles) | Capacity fan-out — one call per tile | dynamic | single-shot | HTTP (base64 PNG) | | [`echo`](./echo) | Realtime video, transformed and echoed back | dynamic | persistent | trickle | | [`vllm`](./vllm) | Drop-in OpenAI API; the client stays unmodified | static | persistent (single-shot by nature) | HTTP + SSE | @@ -71,26 +71,21 @@ flowchart LR Chosen _at_ registration (above); **defaults to `persistent`** — set on both `register_runner(...)` and in `runners.json`. The examples set it explicitly. -- **Persistent** — a held-open session billed per second of wall-clock. Best for realtime / streaming. (`echo`) -- **Single-shot** — one request in, one response out. Best for batch / request-response. (`hello-world`, `vllm` are single-shot by nature.) +- **Persistent** — a held-open session the client reserves and releases, billed per second of wall-clock (or once, with fixed pricing). Best for realtime / streaming. (`echo`, `vllm`) +- **Single-shot** — one request in, one response out; the orchestrator reserves a session per call and releases it when the response returns, so the client manages no session at all. Best for batch / request-response. (`hello-world`, `tiles`) -> [!IMPORTANT] -> Single-shot payment isn't implemented yet ([go-livepeer#3955](https://github.com/livepeer/go-livepeer/issues/3955)), so the single-shot-by-nature apps above register as **persistent**. On-chain that bills per second for the whole open session and overbills short calls — keep them **offchain-only** until #3955 lands ([#5](https://github.com/livepeer/live-runner-example-apps/issues/5)). +> [!NOTE] +> The `vllm` example is single-shot by nature but stays **persistent** for now: it meters per second across a reserved session, and true per-token billing is brokerage for the gateway/signer layer. ## Calling your app -The client side is the same shape for every app — **discover → reserve → call → release**: +The client side depends on the runner's mode: -1. **Discover** the app via the orchestrator's `/discovery`. -2. **Reserve** a session (`reserve_session`). -3. **Call** it — one `call_runner`, streamed frames, or a WebSocket, depending on transport. -4. **Release** the session (`stop_runner_session`), which settles payment on-chain. +- **Single-shot** — **discover → call**: find the app via `runner_selector`, then one `call_runner`. The orchestrator reserves a session for the call and releases it when the response returns; on the paid path `call_runner` answers the 402 payment challenge inline. (`hello-world`, `tiles`) +- **Persistent** — **discover → reserve → call → release**: reserve a session (`reserve_session`), call it — `call_runner`, streamed frames, or a WebSocket, depending on transport — then release it (`stop_runner_session`), which settles payment on-chain. (`echo`, `vllm`) Each example's `client.py` shows its exact calls — grep `# Livepeer:` to find them. -> [!NOTE] -> This is the flow today. Once single-shot lands ([#5](https://github.com/livepeer/live-runner-example-apps/issues/5)), we intend to abstract it into a single call — exact design TBD. - ## External examples Apps that integrate the live runner and live in their own repos — production deployments and standalone examples alike: @@ -128,7 +123,7 @@ On-chain runs add a **remote signer** that holds the payer wallet and mints [pro - **Wallets stay outside the repo** — `*_KEYSTORE_DIR` points at go-livepeer keystores (mounted read-only); only the address + password come from `.env`. - **`.env` is per example and gitignored** — copy `.env.example` and fill in RPC, network, keystore paths, accounts, and pricing (it holds the keystore password). -- **Runner price is a plain USD amount**: the app advertises `PRICE` (e.g. `0.01`). `currency` and `unit` default to `usd` / `hour`, so only `price` is required. With `hour` the orchestrator converts it to wei via the price feed and meters the session per second. Apps with bounded per-call work can register `unit="fixed"` to bill the price once per session (see tiles). The signer caps what it pays at `MAX_PRICE_PER_UNIT`, compared per billing unit: one second of runtime when metered (0.000111USD is about 0.40 USD/hour), the whole session price when fixed. +- **Runner price is a plain USD amount**: the app advertises `PRICE` (e.g. `0.01`). `currency` and `unit` default to `usd` / `hour`, so only `price` is required. With `hour` the orchestrator converts it to wei via the price feed and meters the session per second. Apps with bounded per-call work can register `unit="fixed"` to bill the price once per session (see hello-world, tiles). The signer caps what it pays at `MAX_PRICE_PER_UNIT`, compared per billing unit: one second of runtime when metered (0.000111USD is about 0.40 USD/hour), the whole session price when fixed. - **Payments are probabilistic** — on a short run you'll rarely see a redemption; that's expected. ### Verifying discovery diff --git a/hello-world/.env.example b/hello-world/.env.example index 4e34c47..91be315 100644 --- a/hello-world/.env.example +++ b/hello-world/.env.example @@ -16,8 +16,10 @@ ORCH_ETH_PASSWORD=your-operator-keystore-password # Registered orch = ticket recipient (-ethOrchAddr); empty = use the operating key. ORCH_ONCHAIN_ADDR=0xYourRegisteredOrchestrator -# Runner price (on-chain): USD per hour, converted to wei via the price feed. -PRICE=0.01 -# Signer's max-price cap (payer side), applied per second of runtime -# (0.000111USD is about 0.40 USD/hour). +# Runner price (on-chain): USD billed once per call (fixed pricing). +# Keep under ~0.00019: the signer signs at most 100 tickets per payment, +# and the demo orchestrator runs -ticketEV=1e9 (numTickets = fee / ticketEV). +PRICE=0.0001 +# Signer's max-price cap (payer side), compared per billing unit. With fixed +# pricing the unit is one call, so this must exceed PRICE. MAX_PRICE_PER_UNIT=0.000111USD diff --git a/hello-world/README.md b/hello-world/README.md index 1be0e73..560db2f 100644 --- a/hello-world/README.md +++ b/hello-world/README.md @@ -5,19 +5,16 @@ The smallest possible app on the Livepeer network: a synchronous request/respons | | | | ------------ | ------------------------------------ | | App id | `livepeer-example/hello-world` | -| Runner mode | persistent (single-shot by nature) | +| Runner mode | single-shot | | Registration | dynamic (self-registers via the SDK) | | Transport | HTTP (JSON request/response) | | Port | 8989 | Prerequisites (Docker, `uv`, and the not-yet-released `livepeer-gateway` SDK — pinned in `pyproject.toml`) and the shared on-chain/payment setup live in the [repo README](../README.md). -> [!NOTE] -> This app currently runs in **persistent** mode. It will switch to **single-shot** once [#5](https://github.com/livepeer/live-runner-example-apps/issues/5) ships. - ## How it's wired -The app is **dynamically registered**: it self-registers with the orchestrator via `register_runner` ([runner.py](runner.py)) and exposes a single `POST /hello`, reverse-proxied through the orchestrator. The client calls it with `reserve_session` → `call_runner` → `stop_runner_session` ([client.py](client.py)) — discover, reserve, call, release; one request, one response. Grep `# Livepeer:` in either file to see the exact calls. This is the base flow every other example builds on. +The app is **dynamically registered**: it self-registers with the orchestrator via `register_runner` ([runner.py](runner.py)) and exposes a single `POST /hello`, reverse-proxied through the orchestrator. The client calls it with `runner_selector` → `call_runner` ([client.py](client.py)) — discover, then one call. Because the runner is **single-shot**, there is no session to manage: the orchestrator reserves one for the call and releases it when the response returns, and on the paid path `call_runner` answers the 402 payment challenge inline. Grep `# Livepeer:` in either file to see the exact calls. This is the base flow every other example builds on. ## Run offchain (free) @@ -45,11 +42,11 @@ uv run client.py --name livepeer \ docker compose -f compose.yml -f compose.onchain.yml down ``` -The app registers with a price (`--price` in USD/hour from `.env`) and the orchestrator advertises it in `/discovery`. The SDK client does discovery, the session, the `/hello` call, and payment itself — paying through the remote signer with **no gateway in between**. So this is the full paid stack end to end: **app + orchestrator + remote signer + SDK client**. +The app registers with a price (`--price` from `.env`, in USD billed once per call — **fixed pricing**, the natural fit for bounded request/response work) and the orchestrator advertises it in `/discovery`. The SDK client does discovery, the `/hello` call, and payment itself — paying through the remote signer with **no gateway in between**. So this is the full paid stack end to end: **app + orchestrator + remote signer + SDK client**. ## Run without Docker -Start an orchestrator built from `ja/live-runner` (see [Build from source](https://docs.livepeer.org/v1/orchestrators/guides/install-go-livepeer#build-from-source)), then the app and client directly: +Start an orchestrator built from go-livepeer `v0.9.0` or newer (see [Build from source](https://docs.livepeer.org/v1/orchestrators/guides/install-go-livepeer#build-from-source)), then the app and client directly: ```sh ./livepeer -orchestrator -useLiveRunners -serviceAddr localhost:8935 -orchSecret abcdef -v 6 diff --git a/hello-world/client.py b/hello-world/client.py index 86ceb8c..19b9f6a 100644 --- a/hello-world/client.py +++ b/hello-world/client.py @@ -1,11 +1,12 @@ #!/usr/bin/env python3 -"""hello-world client: reserve an orchestrator session, call the app, settle up. +"""hello-world client: discover a runner, call the app, pay inline. Livepeer integration (grep `# Livepeer:`): - 1. reserve_session() — discover orchestrators advertising the app, reserve one - (to be removed once #4 lands) - 2. call_runner() — invoke the app through the orchestrator - 3. stop_runner_session() — end the session (settles payment on-chain) + 1. runner_selector() — discover orchestrators advertising the app + 2. call_runner() — call the app through the orchestrator; on the paid path it + answers the 402 payment challenge inline. Single-shot needs + no reserve/stop: the orchestrator reserves the session for + this one request and releases it when the response returns. """ from __future__ import annotations @@ -13,11 +14,10 @@ import argparse import asyncio import logging -from contextlib import suppress from livepeer_gateway.errors import LivepeerGatewayError -from livepeer_gateway.live_runner import call_runner, stop_runner_session -from livepeer_gateway.selection import reserve_session +from livepeer_gateway.live_runner import call_runner +from livepeer_gateway.selection import runner_selector DEFAULT_DISCOVERY = "https://localhost:8935/discovery" APP_ID = "livepeer-example/hello-world" @@ -43,27 +43,22 @@ async def main() -> None: ) args = _parse_args() signer_url = args.signer.strip() or None - session = None try: - session = await reserve_session( # Livepeer: 1 (to be removed once #4 lands) - discovery_url=args.discovery, - app=APP_ID, - signer_url=signer_url, + cursor = await runner_selector( # Livepeer: 1 + discovery_url=args.discovery, app=APP_ID ) - log.info("session_id=%s app_url=%s", session.session_id, session.app_url) + runner = cursor.candidates[0] + log.info("app_url=%s", runner.url) result = await call_runner( # Livepeer: 2 - runner_url=session.app_url.rstrip("/") + "/hello", + runner=runner, # discovery metadata tells call_runner the price unit + runner_url=runner.url.rstrip("/") + "/hello", payload={"name": args.name}, signer_url=signer_url, ) print(result.data) except LivepeerGatewayError as exc: raise SystemExit(f"ERROR: {exc}") from exc - finally: - if session is not None: - with suppress(Exception): - await stop_runner_session(session) # Livepeer: 3 if __name__ == "__main__": diff --git a/hello-world/compose.onchain.yml b/hello-world/compose.onchain.yml index aedc271..9017c85 100644 --- a/hello-world/compose.onchain.yml +++ b/hello-world/compose.onchain.yml @@ -29,4 +29,5 @@ services: - --orchestrator=https://orchestrator:8935 - --orchSecret=abcdef - --runner-url=http://app:8989 + # Billed once per call (fixed pricing); price cap in .env.example. - --price=${PRICE} diff --git a/hello-world/runner.py b/hello-world/runner.py index 274a777..70896e0 100644 --- a/hello-world/runner.py +++ b/hello-world/runner.py @@ -38,7 +38,7 @@ def _parse_args() -> argparse.Namespace: "--price", type=float, default=0, - help="Runner price in USD per hour (0 = free, the offchain default).", + help="Runner price in USD per call (0 = free, the offchain default).", ) return parser.parse_args() @@ -61,10 +61,10 @@ async def _on_startup(app: web.Application) -> None: secret=args.orchSecret, runner_url=args.runner_url, app=APP_ID, - # single-shot by nature; stays persistent until single-shot payment lands - # (go-livepeer#3955) - mode="persistent", - price=args.price, # USD/hour + mode="single-shot", + price=args.price, # USD per call + # one flat payment per call instead of per-second metering + unit="fixed", ) log.info( "registered runner_id=%s orchestrator=%s", diff --git a/tiles/.env.example b/tiles/.env.example index ba102d0..2d70a87 100644 --- a/tiles/.env.example +++ b/tiles/.env.example @@ -16,9 +16,10 @@ ORCH_ETH_PASSWORD=your-operator-keystore-password # Registered orch = ticket recipient (-ethOrchAddr); empty = use the operating key. ORCH_ONCHAIN_ADDR=0xYourRegisteredOrchestrator -# Runner price (on-chain): USD billed once per tile session (fixed pricing). -# Must stay under ~0.00019 so the fee fits 100 tickets at ticketEV=1e9. +# Runner price (on-chain): USD billed once per tile (fixed pricing). +# Keep under ~0.00019: the signer signs at most 100 tickets per payment, +# and the demo orchestrator runs -ticketEV=1e9 (numTickets = fee / ticketEV). PRICE=0.0001 # Signer's max-price cap (payer side), compared per billing unit. With fixed -# pricing the unit is one tile session, so this must exceed PRICE. +# pricing the unit is one tile call, so this must exceed PRICE. MAX_PRICE_PER_UNIT=0.000111USD diff --git a/tiles/README.md b/tiles/README.md index 2264e78..829ac1c 100644 --- a/tiles/README.md +++ b/tiles/README.md @@ -1,37 +1,34 @@ # Tiles app (capacity fan-out) -An image processor on the Livepeer network that shows what **capacity** does. It exposes `POST /tile`, which stylizes one image tile (a deliberately CPU-heavy transform). The client splits an image into a grid and opens **one session per tile at once**, so the runner's capacity — the number of sessions it serves concurrently — decides how many tiles process in parallel. +An image processor on the Livepeer network that shows what **capacity** does. It exposes `POST /tile`, which stylizes one image tile (a deliberately CPU-heavy transform). The client splits an image into a grid and fires **one call per tile at once**, so the runner's capacity — the number of sessions it serves concurrently — decides how many tiles process in parallel. | | | | ------------ | ------------------------------------ | | App id | `livepeer-example/tiles` | -| Runner mode | persistent (single-shot by nature) | +| Runner mode | single-shot | | Registration | dynamic (self-registers via the SDK) | | Transport | HTTP (base64 PNG in/out) | | Port | 8989 | Prerequisites (Docker, `uv`, and the not-yet-released `livepeer-gateway` SDK — pinned in `pyproject.toml`) and the shared on-chain/payment setup live in the [repo README](../README.md). -> [!NOTE] -> This app currently runs in **persistent** mode. It will switch to **single-shot** once [#5](https://github.com/livepeer/live-runner-example-apps/issues/5) ships. - ## How it's wired -The app is **dynamically registered**: it self-registers with the orchestrator via `register_runner` ([runner.py](runner.py)) — advertising its **`capacity`** — and exposes `POST /tile`, reverse-proxied through the orchestrator. The client calls it with `reserve_session` → `call_runner` → `stop_runner_session` ([client.py](client.py)) — discover, reserve, call, release — but for the whole grid at once, one session per tile. Grep `# Livepeer:` in either file to see the exact calls. `/tile` is an ordinary stateless handler; its CPU work runs in a thread so tiles process in parallel. +The app is **dynamically registered**: it self-registers with the orchestrator via `register_runner` ([runner.py](runner.py)) — advertising its **`capacity`** — and exposes `POST /tile`, reverse-proxied through the orchestrator. The client calls it with `runner_selector` → `call_runner` ([client.py](client.py)) — discover, then one **single-shot** call per tile, for the whole grid at once. There is no session to manage: the orchestrator reserves one per call and releases it when the response returns. Grep `# Livepeer:` in either file to see the exact calls. `/tile` is an ordinary stateless handler; its CPU work runs in a thread so tiles process in parallel. ## Capacity — what this shows -**Capacity is the maximum number of sessions the orchestrator will route to one runner at the same time.** The runner advertises it at registration (`register_runner(capacity=N)`, or the `capacity` field in a static `runners.json`); the orchestrator tracks the runner's live sessions and, once `capacity` are open, stops routing new ones there — a further reserve is refused until a session ends. +**Capacity is the maximum number of sessions the orchestrator will route to one runner at the same time.** The runner advertises it at registration (`register_runner(capacity=N)`, or the `capacity` field in a static `runners.json`); the orchestrator tracks the runner's live sessions and, once `capacity` are open, stops routing new ones there. Each in-flight single-shot call holds one session slot, so a further call is refused until one finishes. -This example makes that visible. The client fans out one session per tile: +This example makes that visible. The client fans out one call per tile: -- **`capacity=1`** — the runner serves one tile at a time. The other tiles' reserves are refused, so the client waits and retries; tiles process **one after another**. -- **`capacity=9`** (for a 3×3 grid) — all nine sessions open at once and the tiles process **in parallel**. +- **`capacity=1`** — the runner serves one tile at a time. The other tiles' calls are refused, so the client waits and retries; tiles process **one after another**. +- **`capacity=9`** (for a 3×3 grid) — all nine calls run at once and the tiles process **in parallel**. The output image is identical either way. **Capacity changes throughput, not the result** — which is the whole point: it is how an operator sizes a runner to the concurrency it can actually handle (GPU memory, CPU, model instances), and how the network spreads load once a runner is full. > [!NOTE] -> When a reserve hits a full runner the SDK raises `NoRunnerAvailableError`. The client treats that as "wait for a slot," retrying with backoff until one frees. That wait _is_ the capacity limit doing its job. +> When a call hits a full runner the orchestrator refuses it with HTTP 503 (insufficient capacity). The client treats that as "wait for a slot," retrying with backoff until one frees. That wait _is_ the capacity limit doing its job. ## Run offchain (free) @@ -46,11 +43,11 @@ uv run client.py sample.png --discovery https://localhost:8935/discovery docker compose down ``` -`compose.yml` brings up an orchestrator (`-useLiveRunners`) and the app; `CAPACITY` (default 4) sets the runner's advertised capacity. The client splits `sample.png` into a 3×3 grid (`--grid N` to change), processes every tile through the orchestrator, and writes `tiles-out.png`. Watch the client log: at `capacity=1` the `reserved` lines appear one at a time; at `capacity=9` they land together. +`compose.yml` brings up an orchestrator (`-useLiveRunners`) and the app; `CAPACITY` (default 4) sets the runner's advertised capacity. The client splits `sample.png` into a 3×3 grid (`--grid N` to change), processes every tile through the orchestrator, and writes `tiles-out.png`. Watch the client log: at `capacity=1` the `done` lines appear one at a time; at `capacity=9` they land together. ## Run on-chain (paid) -Layer `compose.onchain.yml` to run the orchestrator on-chain with a remote signer paying each tile call. This example showcases **fixed pricing**: `PRICE` is billed once per tile session instead of metered per second, the natural fit for bounded work. For the required RPC and wallets see [On-chain (paid) setup](../README.md#on-chain-paid-setup) in the repo README. +Layer `compose.onchain.yml` to run the orchestrator on-chain with a remote signer paying each tile call. This example showcases **fixed pricing**: `PRICE` is billed once per tile instead of metered per second, the natural fit for bounded work. For the required RPC and wallets see [On-chain (paid) setup](../README.md#on-chain-paid-setup) in the repo README. ```sh cp .env.example .env # fill in RPC, network, keystore paths, accounts, pricing @@ -61,11 +58,11 @@ uv run client.py sample.png \ docker compose -f compose.yml -f compose.onchain.yml down ``` -Each tile is its own reserve → pay → call → release, so a paid run mints a payment per tile. Keep the grid small on-chain. +Each tile is one paid single-shot call — the orchestrator reserves a session for it, takes one fixed payment, and releases it when the response returns — so a paid run mints a payment per tile. Keep the grid small on-chain. ## Run without Docker -Start an orchestrator built from `ja/live-runner` (see [Build from source](https://docs.livepeer.org/v1/orchestrators/guides/install-go-livepeer#build-from-source)), then the app and client directly: +Start an orchestrator built from go-livepeer `v0.9.0` or newer (see [Build from source](https://docs.livepeer.org/v1/orchestrators/guides/install-go-livepeer#build-from-source)), then the app and client directly: ```sh ./livepeer -orchestrator -useLiveRunners -serviceAddr localhost:8935 -orchSecret abcdef -v 6 diff --git a/tiles/client.py b/tiles/client.py index 5743756..47c7a3b 100644 --- a/tiles/client.py +++ b/tiles/client.py @@ -1,18 +1,19 @@ #!/usr/bin/env python3 -"""tiles client: split an image, fan out one session per tile, stitch the results. +"""tiles client: split an image, fan out one single-shot call per tile, stitch the results. -Opens all tile sessions concurrently; the runner's `capacity` gates how many are live -at once. At capacity=1 tiles serialize; at capacity=N they process in parallel and the -whole image finishes far faster. Same output either way — capacity changes speed, not -result. +Fires all tile calls concurrently; the runner's `capacity` gates how many run at once. +At capacity=1 tiles serialize; at capacity=N they process in parallel and the whole +image finishes far faster. Same output either way — capacity changes speed, not result. Livepeer integration (grep `# Livepeer:`): - 1. reserve_session() — discover orchestrators advertising the app, reserve one - 2. call_runner() — invoke /tile through the orchestrator - 3. stop_runner_session() — end the session (settles payment on-chain) + 1. runner_selector() — discover orchestrators advertising the app + 2. call_runner() — run one tile through the orchestrator; on the paid path it + answers the 402 payment challenge inline (one fixed payment + per tile). Single-shot needs no reserve/stop: the orchestrator + reserves a session for the call and releases it on return. -A reserve is refused while the runner is at capacity, so it retries with backoff until a -slot frees; that wait is exactly the capacity limit doing its job. +A call is refused with HTTP 503 while the runner is at capacity, so it retries with +backoff until a slot frees; that wait is exactly the capacity limit doing its job. """ from __future__ import annotations @@ -23,7 +24,6 @@ import logging import random import time -from contextlib import suppress from pathlib import Path from typing import Iterator @@ -32,11 +32,12 @@ from livepeer_gateway.errors import ( LivepeerGatewayError, + LivepeerHTTPError, NoOrchestratorAvailableError, NoRunnerAvailableError, ) -from livepeer_gateway.live_runner import call_runner, stop_runner_session -from livepeer_gateway.selection import LiveRunnerSession, reserve_session +from livepeer_gateway.live_runner import LiveRunnerCallResult, call_runner +from livepeer_gateway.selection import runner_selector DEFAULT_DISCOVERY = "https://localhost:8935/discovery" APP_ID = "livepeer-example/tiles" @@ -62,7 +63,7 @@ def _parse_args() -> argparse.Namespace: "--signer", default="", help="Remote signer base URL (on-chain/paid path)." ) parser.add_argument( - "--reserve-timeout", + "--slot-timeout", type=float, default=120.0, help="Max seconds to wait for a capacity slot per tile.", @@ -91,24 +92,47 @@ def _split( y0 += row.shape[0] -async def _reserve_with_retry( - *, r: int, c: int, discovery_url: str, signer_url: str | None, deadline: float -) -> LiveRunnerSession: - # A full runner refuses the reserve; wait for a slot instead of failing the tile. +async def _call_tile_with_retry( + *, + r: int, + c: int, + payload: dict[str, str], + discovery_url: str, + signer_url: str | None, + deadline: float, +) -> LiveRunnerCallResult: + # A full runner refuses the call with 503; wait for a slot instead of failing the + # tile. Re-discover each attempt so a restarted runner is picked up. cap = 0.25 waiting = False while True: try: - return await reserve_session( - discovery_url=discovery_url, app=APP_ID, signer_url=signer_url - ) # Livepeer: 1 - except (NoRunnerAvailableError, NoOrchestratorAvailableError) as exc: + cursor = await runner_selector( # Livepeer: 1 + discovery_url=discovery_url, app=APP_ID + ) + runner = cursor.candidates[0] + return await call_runner( # Livepeer: 2 + runner=runner, # discovery metadata tells call_runner the price unit + runner_url=runner.url.rstrip("/") + "/tile", + payload=payload, + signer_url=signer_url, + timeout=60.0, + ) + except ( + NoRunnerAvailableError, + NoOrchestratorAvailableError, + LivepeerHTTPError, + ) as exc: + # Anything but "insufficient capacity" (503) is a real error, not a + # full runner. + if isinstance(exc, LivepeerHTTPError) and exc.status_code != 503: + raise if time.monotonic() >= deadline: raise if not waiting: # Warn once per tile; the reason tells a full runner from a dead one. log.warning( - "tile (%d,%d) no slot yet, retrying until --reserve-timeout: %s", + "tile (%d,%d) no slot yet, retrying until --slot-timeout: %s", r, c, exc, @@ -128,7 +152,7 @@ async def _process_tile( discovery_url: str, signer_url: str | None, t0: float, - reserve_timeout: float, + slot_timeout: float, dump_dir: Path | None, ) -> np.ndarray: # Encode the tile as PNG for the JSON payload. @@ -136,50 +160,37 @@ async def _process_tile( if not ok: raise LivepeerGatewayError(f"tile ({r},{c}): could not encode PNG") - deadline = time.monotonic() + reserve_timeout - session = None - try: - # Reserve a slot (waits out capacity), then run the tile through the runner. - session = await _reserve_with_retry( - r=r, - c=c, - discovery_url=discovery_url, - signer_url=signer_url, - deadline=deadline, - ) - log.info("tile (%d,%d) reserved (+%.1fs)", r, c, time.monotonic() - t0) - result = await call_runner( # Livepeer: 2 - runner_url=session.app_url.rstrip("/") + "/tile", - payload={"tile": base64.b64encode(png.tobytes()).decode()}, - signer_url=signer_url, - timeout=60.0, - ) + # One single-shot call per tile (waits out capacity); the orchestrator reserves + # and releases the session around the call. + result = await _call_tile_with_retry( + r=r, + c=c, + payload={"tile": base64.b64encode(png.tobytes()).decode()}, + discovery_url=discovery_url, + signer_url=signer_url, + deadline=time.monotonic() + slot_timeout, + ) - # Pull the processed tile out of the response. - out_b64 = result.data.get("tile") - if not isinstance(out_b64, str) or not out_b64: - raise LivepeerGatewayError(f"tile ({r},{c}): response missing 'tile'") - log.info("tile (%d,%d) done (+%.1fs)", r, c, time.monotonic() - t0) + # Pull the processed tile out of the response. + out_b64 = result.data.get("tile") + if not isinstance(out_b64, str) or not out_b64: + raise LivepeerGatewayError(f"tile ({r},{c}): response missing 'tile'") + log.info("tile (%d,%d) done (+%.1fs)", r, c, time.monotonic() - t0) - # Decode base64 PNG back to pixels, refitting to the tile's exact size. - out = cv2.imdecode( - np.frombuffer(base64.b64decode(out_b64), dtype=np.uint8), cv2.IMREAD_COLOR - ) - if out is None: - raise LivepeerGatewayError(f"tile ({r},{c}): undecodable image from runner") - h, w = tile.shape[:2] - if out.shape[:2] != (h, w): - out = cv2.resize(out, (w, h)) + # Decode base64 PNG back to pixels, refitting to the tile's exact size. + out = cv2.imdecode( + np.frombuffer(base64.b64decode(out_b64), dtype=np.uint8), cv2.IMREAD_COLOR + ) + if out is None: + raise LivepeerGatewayError(f"tile ({r},{c}): undecodable image from runner") + h, w = tile.shape[:2] + if out.shape[:2] != (h, w): + out = cv2.resize(out, (w, h)) - if dump_dir is not None: - cv2.imwrite(str(dump_dir / f"tile_r{r}_c{c}_in.png"), tile) - cv2.imwrite(str(dump_dir / f"tile_r{r}_c{c}_out.png"), out) - return out - finally: - # Always release the session so the slot frees for a waiting tile. - if session is not None: - with suppress(Exception): - await stop_runner_session(session) # Livepeer: 3 + if dump_dir is not None: + cv2.imwrite(str(dump_dir / f"tile_r{r}_c{c}_in.png"), tile) + cv2.imwrite(str(dump_dir / f"tile_r{r}_c{c}_out.png"), out) + return out async def main() -> None: @@ -198,7 +209,7 @@ async def main() -> None: pieces = list(_split(img, grid)) log.info( - "split %s into %d tiles (%dx%d grid); fanning out one session per tile", + "split %s into %d tiles (%dx%d grid); fanning out one call per tile", input_path.name, len(pieces), grid, @@ -220,7 +231,7 @@ async def main() -> None: discovery_url=args.discovery, signer_url=args.signer.strip() or None, t0=t0, - reserve_timeout=args.reserve_timeout, + slot_timeout=args.slot_timeout, dump_dir=dump_dir, ) for (r, c, _y0, _x0, tile) in pieces diff --git a/tiles/compose.onchain.yml b/tiles/compose.onchain.yml index 9f7363b..2bf0808 100644 --- a/tiles/compose.onchain.yml +++ b/tiles/compose.onchain.yml @@ -4,8 +4,8 @@ # Adds the shared remote signer, re-points the orchestrator on-chain (see # ../compose.onchain.yml), and registers the app with a price so the orchestrator # issues a payment challenge. Requires a local .env (gitignored); copy .env.example -# and fill it in. Each tile is its own reserve -> pay -> call -> release, so keep the -# grid small on-chain. Then pay through the signer: +# and fill it in. Each tile is one paid single-shot call, so keep the grid small +# on-chain. Then pay through the signer: # uv run client.py photo.jpg \ # --discovery https://localhost:8935/discovery \ # --signer http://localhost:7936 @@ -31,7 +31,5 @@ services: - --orchSecret=abcdef - --runner-url=http://app:8989 - --capacity=${CAPACITY:-4} - # Fixed pricing: PRICE is billed once per tile session, not metered. - # Keep it under ~0.00019 USD: the fee must fit in 100 tickets at the - # orchestrator's -ticketEV=1e9 (numTickets = fee / ticketEV). + # Billed once per tile (fixed pricing); price cap in .env.example. - --price=${PRICE} diff --git a/tiles/runner.py b/tiles/runner.py index fb10e74..499ad63 100644 --- a/tiles/runner.py +++ b/tiles/runner.py @@ -2,7 +2,7 @@ """tiles app: a CPU-bound image processor, made callable on the Livepeer network. Each POST /tile stylizes one image tile (a deliberately CPU-heavy transform). The -client splits an image into a grid and fans out one session per tile, so `capacity` +client splits an image into a grid and fans out one call per tile, so `capacity` — the number of sessions the orchestrator routes here at once — decides how many tiles process in parallel. See the README for the capacity demo. @@ -66,7 +66,7 @@ def _parse_args() -> argparse.Namespace: "--price", type=float, default=0, - help="Runner price in USD per tile session (0 = free, the offchain default).", + help="Runner price in USD per tile (0 = free, the offchain default).", ) return parser.parse_args() @@ -120,13 +120,10 @@ async def _on_startup(app: web.Application) -> None: secret=args.orchSecret, runner_url=args.runner_url, app=APP_ID, - # single-shot by nature; stays persistent until single-shot payment lands - # (go-livepeer#3955) - mode="persistent", + mode="single-shot", capacity=args.capacity, # the knob this example showcases - price=args.price, # USD per tile session - # Fixed pricing: bill once per session, not per second; tile work is - # bounded, so the billing model is part of the app, not a deploy knob. + price=args.price, # USD per tile + # one flat payment per tile instead of per-second metering unit="fixed", ) log.info(