Skip to content

Commit 4dcbcda

Browse files
feat(bridge): record prompt-cache breakdown on llm_inference events (bridge-v0.3.3) (#3)
* feat(bridge): record prompt-cache breakdown on llm_inference events The PyO3 bridge transport dropped the prompt-cache breakdown — korgex's KorgBridgeClient already forwarded cache_read_tokens / cache_creation_tokens / uncached_input_tokens (with a TypeError fallback), but record_llm_call didn't accept them, so a cache hit was unprovable from a bridge-written journal while the local-journal and HTTP transports carried it. record_llm_call now takes the three optional cache params and folds them onto the event args ONLY when caching is active (a read or a write happened), mirroring src/korg_ledger.py::_llm_call_args. A cold turn keeps the legacy two-field args, so older readers and the hash-chain over historical events are undisturbed; field order is irrelevant since args are canonicalised with sorted keys at hash time. Bump 0.3.2 -> 0.3.3. Tests: +warm-breakdown / +creation-only cases; 19 bridge tests pass (incl. the chain-integrity suite, so the new field verifies in-chain). * docs(bridge): changelog + README for the cache breakdown bridge-v0.3.3 entry (matching the existing per-bridge-version convention) and a note in the korg-bridge README that record_llm_call carries the optional prompt-cache breakdown, so a cache hit is provable from a bridge-written journal at parity with korgex's local-journal and HTTP transports. --------- Co-authored-by: ares <285551516+New1Direction@users.noreply.github.com>
1 parent 8be5850 commit 4dcbcda

7 files changed

Lines changed: 112 additions & 8 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,13 @@ versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
3434

3535
---
3636

37+
## [bridge-v0.3.3] — 2026-06-04
38+
39+
### Added
40+
- `record_llm_call` accepts the optional prompt-cache breakdown (`cache_read_tokens` / `cache_creation_tokens` / `uncached_input_tokens`) and folds it onto the event `args` — but only when caching is active. A prompt-cache hit is now provable from a bridge-written journal too, at parity with korgex's local-journal and HTTP transports. A cold turn keeps the legacy two-field `args`, so older readers and the hash-chain over historical events are undisturbed (field order is irrelevant — `args` canonicalize with sorted keys at hash time).
41+
42+
---
43+
3744
## [bridge-v0.3.2] — 2026-05-27
3845

3946
### Added
@@ -117,3 +124,4 @@ See [ROADMAP.md](ROADMAP.md) for planned features.
117124
[bridge-v0.3.0]: https://github.com/New1Direction/korg/releases/tag/bridge-v0.3.0
118125
[bridge-v0.3.1]: https://github.com/New1Direction/korg/releases/tag/bridge-v0.3.1
119126
[bridge-v0.3.2]: https://github.com/New1Direction/korg/releases/tag/bridge-v0.3.2
127+
[bridge-v0.3.3]: https://github.com/New1Direction/korg/releases/tag/bridge-v0.3.3

‎Cargo.lock‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎crates/korg-bridge/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "korg-bridge"
3-
version = "0.3.2"
3+
version = "0.3.3"
44
edition = "2021"
55
description = "PyO3 bridge — exposes Korg's in-process WAL (CapabilityJournal) to Python so korgex / KorgChat can write directly without HTTP."
66
license = "MIT OR Apache-2.0"

‎crates/korg-bridge/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,12 @@ byte-for-byte under both `korg_registry::ledger_chain::verify_chain` (Rust) and
1717
korgex's `src/ledger_spec.verify_chain` (Python): the two ledger paths
1818
(korgchat-via-bridge, korgex) collapse onto one chained substrate.
1919

20+
`record_llm_call` also carries an optional prompt-cache breakdown
21+
(`cache_read_tokens` / `cache_creation_tokens` / `uncached_input_tokens`), folded
22+
onto the event `args` only when caching is active — so a cache hit is provable
23+
from the journal too, at parity with korgex's local-journal and HTTP transports.
24+
A cold turn keeps the legacy two-field `args`.
25+
2026
Build the wheel and verify:
2127

2228
```bash

‎crates/korg-bridge/pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "maturin"
44

55
[project]
66
name = "korg_bridge"
7-
version = "0.3.2"
7+
version = "0.3.3"
88
description = "In-process Python ↔ Korg WAL bridge (PyO3). Replaces the HTTP loop that korg_ledger.py used to use."
99
requires-python = ">=3.8"
1010
readme = "README.md"

‎crates/korg-bridge/src/lib.rs‎

Lines changed: 40 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,11 @@ impl Bridge {
136136
/// consumers (search, audit, replay) can find it. Token counts stay in
137137
/// the same places as before. Callers responsible for content-addressing
138138
/// large replies — the bridge writes whatever text it's given.
139+
///
140+
/// v0.3.3: optional prompt-cache breakdown (`cache_read_tokens` /
141+
/// `cache_creation_tokens` / `uncached_input_tokens`). Folded onto `args`
142+
/// only when caching is active, so a cache hit is provable from the
143+
/// bridge-written journal too. A cold turn keeps the legacy `args` shape.
139144
#[pyo3(signature = (
140145
model,
141146
prompt_tokens,
@@ -144,6 +149,9 @@ impl Bridge {
144149
triggered_by,
145150
source_agent = "agent:korgex@0.3.0",
146151
assistant_text = None,
152+
cache_read_tokens = 0,
153+
cache_creation_tokens = 0,
154+
uncached_input_tokens = None,
147155
))]
148156
fn record_llm_call(
149157
&self,
@@ -154,11 +162,39 @@ impl Bridge {
154162
triggered_by: Option<u64>,
155163
source_agent: &str,
156164
assistant_text: Option<&str>,
165+
cache_read_tokens: u64,
166+
cache_creation_tokens: u64,
167+
uncached_input_tokens: Option<u64>,
157168
) -> PyResult<u64> {
158-
let args = serde_json::json!({
159-
"model": model,
160-
"prompt_tokens": prompt_tokens,
161-
});
169+
// Prompt-cache breakdown — mirrors src/korg_ledger.py::_llm_call_args so the
170+
// bridge transport carries the same provable cache data as the local-journal
171+
// and HTTP paths. Folded in ONLY when caching is active (a read or a write
172+
// happened); a cold turn keeps the legacy two-field shape, so older readers
173+
// and the hash-chain over historical events are undisturbed. Field order is
174+
// irrelevant — args are canonicalised with sorted keys at hash time.
175+
let mut args_map = serde_json::Map::new();
176+
args_map.insert("model".to_string(), serde_json::Value::from(model));
177+
args_map.insert(
178+
"prompt_tokens".to_string(),
179+
serde_json::Value::from(prompt_tokens),
180+
);
181+
if cache_read_tokens != 0 || cache_creation_tokens != 0 {
182+
args_map.insert(
183+
"cache_read_tokens".to_string(),
184+
serde_json::Value::from(cache_read_tokens),
185+
);
186+
args_map.insert(
187+
"cache_creation_tokens".to_string(),
188+
serde_json::Value::from(cache_creation_tokens),
189+
);
190+
if let Some(uncached) = uncached_input_tokens {
191+
args_map.insert(
192+
"uncached_input_tokens".to_string(),
193+
serde_json::Value::from(uncached),
194+
);
195+
}
196+
}
197+
let args = serde_json::Value::Object(args_map);
162198
// result carries completion_tokens + (since v0.3.2) optional text.
163199
// Keep the field name "text" — short, obvious, and matches what a
164200
// future content-addressed variant would also use as a key.

‎crates/korg-bridge/tests/test_bridge.py‎

Lines changed: 55 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ def _read_events(journal_path: Path) -> list[dict]:
2727

2828

2929
def test_module_version_present():
30-
assert korg_bridge.__version__ == "0.3.2"
30+
assert korg_bridge.__version__ == "0.3.3"
3131

3232

3333
def test_record_llm_call_assistant_text_optional(tmp_journal):
@@ -59,6 +59,60 @@ def test_record_llm_call_assistant_text_optional(tmp_journal):
5959
}
6060

6161

62+
def test_record_llm_call_cache_breakdown(tmp_journal):
63+
"""v0.3.3: the prompt-cache breakdown lands on the event args when caching is
64+
active, so a cache hit is provable from the bridge-written journal too — parity
65+
with the local-journal and HTTP transports."""
66+
bridge = korg_bridge.Bridge(str(tmp_journal))
67+
# 1. cold turn (no cache) → legacy two-field args, byte-identical to before
68+
bridge.record_llm_call(
69+
model="claude-sonnet-4-6",
70+
prompt_tokens=500,
71+
completion_tokens=5,
72+
duration_ms=100,
73+
triggered_by=None,
74+
)
75+
# 2. warm turn → disjoint breakdown folded in
76+
bridge.record_llm_call(
77+
model="claude-sonnet-4-6",
78+
prompt_tokens=400,
79+
completion_tokens=8,
80+
duration_ms=120,
81+
triggered_by=1,
82+
cache_read_tokens=800,
83+
cache_creation_tokens=50,
84+
uncached_input_tokens=400,
85+
)
86+
events = _read_events(tmp_journal)
87+
assert events[0]["event"]["args"] == {"model": "claude-sonnet-4-6", "prompt_tokens": 500}
88+
assert events[1]["event"]["args"] == {
89+
"model": "claude-sonnet-4-6",
90+
"prompt_tokens": 400,
91+
"cache_read_tokens": 800,
92+
"cache_creation_tokens": 50,
93+
"uncached_input_tokens": 400,
94+
}
95+
96+
97+
def test_record_llm_call_cache_creation_only(tmp_journal):
98+
"""A cold turn that WRITES the cache (creation>0, read==0) is still a cache
99+
event — the breakdown is recorded."""
100+
bridge = korg_bridge.Bridge(str(tmp_journal))
101+
bridge.record_llm_call(
102+
model="claude-sonnet-4-6",
103+
prompt_tokens=1000,
104+
completion_tokens=8,
105+
duration_ms=120,
106+
triggered_by=None,
107+
cache_read_tokens=0,
108+
cache_creation_tokens=1000,
109+
uncached_input_tokens=1000,
110+
)
111+
events = _read_events(tmp_journal)
112+
assert events[0]["event"]["args"]["cache_creation_tokens"] == 1000
113+
assert events[0]["event"]["args"]["uncached_input_tokens"] == 1000
114+
115+
62116
def test_repr_initial_state(tmp_journal):
63117
bridge = korg_bridge.Bridge(str(tmp_journal))
64118
rep = repr(bridge)

0 commit comments

Comments
 (0)