Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 12 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Example **apps that run on** the Livepeer **live runner** — [go-livepeer](http
The point is to **swap the compute without changing your app — permissionlessly, no lock-in**. Your app stays a plain service with little or no Livepeer-specific code, so you're never tied to us. And the network is permissionless: anyone can run or extend it, no one gatekeeps what you deploy, and no single party can take your app down. Write the app once; **move the compute freely**.

> [!NOTE]
> Live runners aren't on go-livepeer `main` yet — they live on the [`ja/live-runner`](https://github.com/livepeer/go-livepeer/tree/ja/live-runner) branch. Until it merges, both the orchestrator image and the SDK come from that branch.
> Live runners ship in mainline go-livepeer since [v0.9.0](https://github.com/livepeer/go-livepeer/releases/tag/v0.9.0). The Python SDK still comes from the [`ja/live-runner`](https://github.com/livepeer/livepeer-python-gateway/tree/ja/live-runner) branch until it is published to PyPI.

## How it works

Expand All @@ -18,7 +18,7 @@ flowchart LR
app["Your app<br/>HTTP / WebSocket / trickle"]
signer["Remote signer<br/>(on-chain)"]

client -->|"discover → reserve → call → release"| orch
client -->|"single-shot: discover → call · persistent: + reserve/release"| orch
orch -->|"forwards your endpoints, unchanged"| app
app -.->|"dynamic: register_runner · static: runners.json"| orch
signer <-.->|"micropayment tickets"| orch
Expand All @@ -39,8 +39,8 @@ Need a schema that isn't here? [Open an issue](https://github.com/livepeer/live-

| Example | Goal | Registration | Mode | Transport |
| ------------------------------ | ----------------------------------------------- | ------------ | ---------------------------------- | ----------------- |
| [`hello-world`](./hello-world) | The simplest app: one request, one response | dynamic | persistent (single-shot by nature) | HTTP (JSON) |
| [`tiles`](./tiles) | Capacity fan-out — one session per tile | dynamic | persistent (single-shot by nature) | HTTP (base64 PNG) |
| [`hello-world`](./hello-world) | The simplest app: one request, one response | dynamic | single-shot | HTTP (JSON) |
| [`tiles`](./tiles) | Capacity fan-out — one call per tile | dynamic | single-shot | HTTP (base64 PNG) |
| [`echo`](./echo) | Realtime video, transformed and echoed back | dynamic | persistent | trickle |
| [`vllm`](./vllm) | Drop-in OpenAI API; the client stays unmodified | static | persistent (single-shot by nature) | HTTP + SSE |

Expand Down Expand Up @@ -71,26 +71,21 @@ flowchart LR

Chosen _at_ registration (above); **defaults to `persistent`** — set on both `register_runner(...)` and in `runners.json`. The examples set it explicitly.

- **Persistent** — a held-open session billed per second of wall-clock. Best for realtime / streaming. (`echo`)
- **Single-shot** — one request in, one response out. Best for batch / request-response. (`hello-world`, `vllm` are single-shot by nature.)
- **Persistent** — a held-open session the client reserves and releases, billed per second of wall-clock (or once, with fixed pricing). Best for realtime / streaming. (`echo`, `vllm`)
- **Single-shot** — one request in, one response out; the orchestrator reserves a session per call and releases it when the response returns, so the client manages no session at all. Best for batch / request-response. (`hello-world`, `tiles`)

> [!IMPORTANT]
> Single-shot payment isn't implemented yet ([go-livepeer#3955](https://github.com/livepeer/go-livepeer/issues/3955)), so the single-shot-by-nature apps above register as **persistent**. On-chain that bills per second for the whole open session and overbills short calls — keep them **offchain-only** until #3955 lands ([#5](https://github.com/livepeer/live-runner-example-apps/issues/5)).
> [!NOTE]
> The `vllm` example is single-shot by nature but stays **persistent** for now: it meters per second across a reserved session, and true per-token billing is brokerage for the gateway/signer layer.

## Calling your app

The client side is the same shape for every app — **discover → reserve → call → release**:
The client side depends on the runner's mode:

1. **Discover** the app via the orchestrator's `/discovery`.
2. **Reserve** a session (`reserve_session`).
3. **Call** it — one `call_runner`, streamed frames, or a WebSocket, depending on transport.
4. **Release** the session (`stop_runner_session`), which settles payment on-chain.
- **Single-shot** — **discover → call**: find the app via `runner_selector`, then one `call_runner`. The orchestrator reserves a session for the call and releases it when the response returns; on the paid path `call_runner` answers the 402 payment challenge inline. (`hello-world`, `tiles`)
- **Persistent** — **discover → reserve → call → release**: reserve a session (`reserve_session`), call it — `call_runner`, streamed frames, or a WebSocket, depending on transport — then release it (`stop_runner_session`), which settles payment on-chain. (`echo`, `vllm`)

Each example's `client.py` shows its exact calls — grep `# Livepeer:` to find them.

> [!NOTE]
> This is the flow today. Once single-shot lands ([#5](https://github.com/livepeer/live-runner-example-apps/issues/5)), we intend to abstract it into a single call — exact design TBD.

## External examples

Apps that integrate the live runner and live in their own repos — production deployments and standalone examples alike:
Expand Down Expand Up @@ -128,7 +123,7 @@ On-chain runs add a **remote signer** that holds the payer wallet and mints [pro

- **Wallets stay outside the repo** — `*_KEYSTORE_DIR` points at go-livepeer keystores (mounted read-only); only the address + password come from `.env`.
- **`.env` is per example and gitignored** — copy `.env.example` and fill in RPC, network, keystore paths, accounts, and pricing (it holds the keystore password).
- **Runner price is a plain USD amount**: the app advertises `PRICE` (e.g. `0.01`). `currency` and `unit` default to `usd` / `hour`, so only `price` is required. With `hour` the orchestrator converts it to wei via the price feed and meters the session per second. Apps with bounded per-call work can register `unit="fixed"` to bill the price once per session (see tiles). The signer caps what it pays at `MAX_PRICE_PER_UNIT`, compared per billing unit: one second of runtime when metered (0.000111USD is about 0.40 USD/hour), the whole session price when fixed.
- **Runner price is a plain USD amount**: the app advertises `PRICE` (e.g. `0.01`). `currency` and `unit` default to `usd` / `hour`, so only `price` is required. With `hour` the orchestrator converts it to wei via the price feed and meters the session per second. Apps with bounded per-call work can register `unit="fixed"` to bill the price once per session (see hello-world, tiles). The signer caps what it pays at `MAX_PRICE_PER_UNIT`, compared per billing unit: one second of runtime when metered (0.000111USD is about 0.40 USD/hour), the whole session price when fixed.
- **Payments are probabilistic** — on a short run you'll rarely see a redemption; that's expected.

### Verifying discovery
Expand Down
10 changes: 6 additions & 4 deletions hello-world/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ ORCH_ETH_PASSWORD=your-operator-keystore-password
# Registered orch = ticket recipient (-ethOrchAddr); empty = use the operating key.
ORCH_ONCHAIN_ADDR=0xYourRegisteredOrchestrator

# Runner price (on-chain): USD per hour, converted to wei via the price feed.
PRICE=0.01
# Signer's max-price cap (payer side), applied per second of runtime
# (0.000111USD is about 0.40 USD/hour).
# Runner price (on-chain): USD billed once per call (fixed pricing).
# Keep under ~0.00019: the signer signs at most 100 tickets per payment,
# and the demo orchestrator runs -ticketEV=1e9 (numTickets = fee / ticketEV).
PRICE=0.0001
# Signer's max-price cap (payer side), compared per billing unit. With fixed
# pricing the unit is one call, so this must exceed PRICE.
MAX_PRICE_PER_UNIT=0.000111USD
11 changes: 4 additions & 7 deletions hello-world/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,16 @@ The smallest possible app on the Livepeer network: a synchronous request/respons
| | |
| ------------ | ------------------------------------ |
| App id | `livepeer-example/hello-world` |
| Runner mode | persistent (single-shot by nature) |
| Runner mode | single-shot |
| Registration | dynamic (self-registers via the SDK) |
| Transport | HTTP (JSON request/response) |
| Port | 8989 |

Prerequisites (Docker, `uv`, and the not-yet-released `livepeer-gateway` SDK — pinned in `pyproject.toml`) and the shared on-chain/payment setup live in the [repo README](../README.md).

> [!NOTE]
> This app currently runs in **persistent** mode. It will switch to **single-shot** once [#5](https://github.com/livepeer/live-runner-example-apps/issues/5) ships.

## How it's wired

The app is **dynamically registered**: it self-registers with the orchestrator via `register_runner` ([runner.py](runner.py)) and exposes a single `POST /hello`, reverse-proxied through the orchestrator. The client calls it with `reserve_session` → `call_runner` → `stop_runner_session` ([client.py](client.py)) — discover, reserve, call, release; one request, one response. Grep `# Livepeer:` in either file to see the exact calls. This is the base flow every other example builds on.
The app is **dynamically registered**: it self-registers with the orchestrator via `register_runner` ([runner.py](runner.py)) and exposes a single `POST /hello`, reverse-proxied through the orchestrator. The client calls it with `runner_selector` → `call_runner` ([client.py](client.py)) — discover, then one call. Because the runner is **single-shot**, there is no session to manage: the orchestrator reserves one for the call and releases it when the response returns, and on the paid path `call_runner` answers the 402 payment challenge inline. Grep `# Livepeer:` in either file to see the exact calls. This is the base flow every other example builds on.

## Run offchain (free)

Expand Down Expand Up @@ -45,11 +42,11 @@ uv run client.py --name livepeer \
docker compose -f compose.yml -f compose.onchain.yml down
```

The app registers with a price (`--price` in USD/hour from `.env`) and the orchestrator advertises it in `/discovery`. The SDK client does discovery, the session, the `/hello` call, and payment itself — paying through the remote signer with **no gateway in between**. So this is the full paid stack end to end: **app + orchestrator + remote signer + SDK client**.
The app registers with a price (`--price` from `.env`, in USD billed once per call — **fixed pricing**, the natural fit for bounded request/response work) and the orchestrator advertises it in `/discovery`. The SDK client does discovery, the `/hello` call, and payment itself — paying through the remote signer with **no gateway in between**. So this is the full paid stack end to end: **app + orchestrator + remote signer + SDK client**.

## Run without Docker

Start an orchestrator built from `ja/live-runner` (see [Build from source](https://docs.livepeer.org/v1/orchestrators/guides/install-go-livepeer#build-from-source)), then the app and client directly:
Start an orchestrator built from go-livepeer `v0.9.0` or newer (see [Build from source](https://docs.livepeer.org/v1/orchestrators/guides/install-go-livepeer#build-from-source)), then the app and client directly:

```sh
./livepeer -orchestrator -useLiveRunners -serviceAddr localhost:8935 -orchSecret abcdef -v 6
Expand Down
33 changes: 14 additions & 19 deletions hello-world/client.py
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
#!/usr/bin/env python3
"""hello-world client: reserve an orchestrator session, call the app, settle up.
"""hello-world client: discover a runner, call the app, pay inline.

Livepeer integration (grep `# Livepeer:`):
1. reserve_session() — discover orchestrators advertising the app, reserve one
(to be removed once #4 lands)
2. call_runner() — invoke the app through the orchestrator
3. stop_runner_session() — end the session (settles payment on-chain)
1. runner_selector() — discover orchestrators advertising the app
2. call_runner() — call the app through the orchestrator; on the paid path it
answers the 402 payment challenge inline. Single-shot needs
no reserve/stop: the orchestrator reserves the session for
this one request and releases it when the response returns.
"""

from __future__ import annotations

import argparse
import asyncio
import logging
from contextlib import suppress

from livepeer_gateway.errors import LivepeerGatewayError
from livepeer_gateway.live_runner import call_runner, stop_runner_session
from livepeer_gateway.selection import reserve_session
from livepeer_gateway.live_runner import call_runner
from livepeer_gateway.selection import runner_selector

DEFAULT_DISCOVERY = "https://localhost:8935/discovery"
APP_ID = "livepeer-example/hello-world"
Expand All @@ -43,27 +43,22 @@ async def main() -> None:
)
args = _parse_args()
signer_url = args.signer.strip() or None
session = None
try:
session = await reserve_session( # Livepeer: 1 (to be removed once #4 lands)
discovery_url=args.discovery,
app=APP_ID,
signer_url=signer_url,
cursor = await runner_selector( # Livepeer: 1
discovery_url=args.discovery, app=APP_ID
)
log.info("session_id=%s app_url=%s", session.session_id, session.app_url)
runner = cursor.candidates[0]
log.info("app_url=%s", runner.url)

result = await call_runner( # Livepeer: 2
runner_url=session.app_url.rstrip("/") + "/hello",
runner=runner, # discovery metadata tells call_runner the price unit
runner_url=runner.url.rstrip("/") + "/hello",
payload={"name": args.name},
signer_url=signer_url,
)
print(result.data)
except LivepeerGatewayError as exc:
raise SystemExit(f"ERROR: {exc}") from exc
finally:
if session is not None:
with suppress(Exception):
await stop_runner_session(session) # Livepeer: 3


if __name__ == "__main__":
Expand Down
1 change: 1 addition & 0 deletions hello-world/compose.onchain.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,5 @@ services:
- --orchestrator=https://orchestrator:8935
- --orchSecret=abcdef
- --runner-url=http://app:8989
# Billed once per call (fixed pricing); price cap in .env.example.
- --price=${PRICE}
10 changes: 5 additions & 5 deletions hello-world/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def _parse_args() -> argparse.Namespace:
"--price",
type=float,
default=0,
help="Runner price in USD per hour (0 = free, the offchain default).",
help="Runner price in USD per call (0 = free, the offchain default).",
)
return parser.parse_args()

Expand All @@ -61,10 +61,10 @@ async def _on_startup(app: web.Application) -> None:
secret=args.orchSecret,
runner_url=args.runner_url,
app=APP_ID,
# single-shot by nature; stays persistent until single-shot payment lands
# (go-livepeer#3955)
mode="persistent",
price=args.price, # USD/hour
mode="single-shot",
price=args.price, # USD per call
# one flat payment per call instead of per-second metering
unit="fixed",
)
log.info(
"registered runner_id=%s orchestrator=%s",
Expand Down
7 changes: 4 additions & 3 deletions tiles/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ ORCH_ETH_PASSWORD=your-operator-keystore-password
# Registered orch = ticket recipient (-ethOrchAddr); empty = use the operating key.
ORCH_ONCHAIN_ADDR=0xYourRegisteredOrchestrator

# Runner price (on-chain): USD billed once per tile session (fixed pricing).
# Must stay under ~0.00019 so the fee fits 100 tickets at ticketEV=1e9.
# Runner price (on-chain): USD billed once per tile (fixed pricing).
# Keep under ~0.00019: the signer signs at most 100 tickets per payment,
# and the demo orchestrator runs -ticketEV=1e9 (numTickets = fee / ticketEV).
PRICE=0.0001
# Signer's max-price cap (payer side), compared per billing unit. With fixed
# pricing the unit is one tile session, so this must exceed PRICE.
# pricing the unit is one tile call, so this must exceed PRICE.
MAX_PRICE_PER_UNIT=0.000111USD
Loading
Loading