Skip to content

feat: publish the liq_pnl_* windows from the bot every 5 minutes [RAI-3093] - #1685

Closed
JuaniRios wants to merge 1 commit into
rai-2998-liq-metrics-pnlfrom
rai-2998-liq-metrics-pnl-series
Closed

JuaniRios wants to merge 1 commit into
rai-2998-liq-metrics-pnlfrom
rai-2998-liq-metrics-pnl-series

Conversation

@JuaniRios

@JuaniRios JuaniRios commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Part of RAI-3093.

Publishes all 25 liq_pnl_* series from the bot for the 6 PnL windows, built from the same report as GET /pnl. Item 4 of the RAI-2998 stack, part 2 of 3.

Live effect: a new supervised task, liq-pnl-refresh, runs 6 PnL reports every 5 min and publishes the liq_pnl_* names, with liq_collector_last_success_ts_seconds{collector="pnl_<window>"} per window · Risk: medium (6 extra ledger replays and 6 Alpaca activity fetches every 5 min while the exporter still runs) · Ships: on the next bot release

Why

The TVL report, the board and the shareholder numbers read liq_pnl_*. Until now only the exporter sidecar published them. No P&L series is dropped.

Decisions

  • Windows copy the exporter: 1d, 1w, 1m (anchor minus 30), ytd, 1y (minus 364) and all. All end on the last day with fills, and no start is before the first day with fills. A cold range cache costs one probe report. Order 1w, 1d, all, 1m, ytd, 1y, inside a 120 s budget.
  • Each window is its own family, LiqFamily::Pnl(window), and owns only samples with its window label.
  • J5: a window whose report fails or does not fit the budget keeps its last value. The exporter dropped it until the next cycle. Its pnl_<window> collector time stops advancing, so staleness is visible. This is a recorded difference: compare.py has a kept_window rule for it.
  • The task has its own admission of 1 permit, so live /pnl keeps both of its permits. Each window runs its own ledger catch-up, as each exporter call did.
  • No outer timeout around a report. A cancelled report leaves its replay on a blocking thread that holds the task's only permit, so the next windows would fail. Each report is bounded anyway: 10 s catch-up, Alpaca HTTP timeouts, finite replay.
  • Day buckets (J1): the last 90 buckets per window, running totals over all buckets first, stripped symbols summed (tAAPL and wtAAPL). An unparseable summary decimal leaves its series absent. All sums use exact Float.
  • An unparseable day component no longer counts as 0, cause a 0 reads as a real day. The bot leaves out the day value it feeds and every later running total, and logs an error. pnl.rs
  • The exporter still counts that component as 0 and publishes the day and its running totals (t0.devops 8dca7be, pnl_day_samples). So compare.py --known-diffs gets a kept_window_bot_absent rule for the four liq_pnl_day_* names: a day series that only the bot is missing is not a finding. Values that both publish are still compared. The cost: the compare no longer catches any other day series the bot drops. compare.py
  • Every attempt is timed in metrics_refresh_duration_seconds{collector="pnl_<window>"}, failed ones too.
  • 6 replays per cycle is the exporter's cost too. One shared replay would cost about a sixth, but needs the report split. That is a follow-up, written in the module doc.

Proof

  • clippy, fmt, and pre-commit clean. golden.py --check (12 files), test_compare.py, and gen-t0-liquidity.py --check pass.
  • Goldens from the exporter at t0.devops 226c029: pnl runs collect_pnl over all 6 windows; pnl-days has 95 day buckets to check the 90 cut and the running totals. Fixtures use fake values only.
  • Tests: the window table (clamp, ytd, 1m = A-30), one probe then 6 windows in order, a failed window keeps its samples and timestamp, the budget skips windows and keeps them, no fills publishes nothing, a later fill day moves the next cycle, the metrics task runs while live requests hold both permits, and an unparseable day component leaves its series out.
  • Full st0x-hedge suite on the top of the stack, before the latest fix push: 4814 passed.
  • Multi-model review loop over the 3 PRs (sol, grok, composer, flash, and opus) converged. Pass 1 found and fixed the cancelled-report permit leak, the histogram that skipped failures, and the missing cost note.
  • Not verified: values against the exporter on staging (compare.py, plan section 13), the /pnl 503 rate and the window p95 during Stage 3. Those need a release.

Merge conditions

Same as the stack below.

@linear-code

linear-code Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

RAI-2998

RAI-3061

RAI-3093

JuaniRios commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more


How to use the Graphite Merge Queue

Add the label add-to-gt-merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@JuaniRios

Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: 2665fcd2-f386-4e56-ac0a-8a1c1f2c744f

📥 Commits

Reviewing files that changed from the base of the PR and between 2bc9e20 and f2bb5f2.


📒 Files selected for processing (4)
  • SPEC.md
  • scripts/liq-parity/compare.py
  • scripts/liq-parity/test_compare.py
  • src/metrics/liquidity/pnl.rs

Included review availability: This review used your included allowance. 6 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 8 reviews per hour.



Walkthrough

The change adds 25 P&L metric series for six windows and refreshes them every five minutes. Each window is generated and replaced independently. Failed or skipped reports leave its previous samples and collector timestamp unchanged. Daily metrics aggregate normalized symbols, calculate cumulative values across the full range, and emit up to 90 days. The change also updates P&L parity rules, golden cases, tests, and observability documentation.

Priority: ➖ Normal

Merge Risk: 🟡 Moderate · up to f2bb5

Some per-symbol P&L metrics can be absent, and the staging comparison can miss absent day values. Resolve or explicitly accept these gaps before merging.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 41.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 108 functions across 8 files. (1 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: publishing the bot's liq_pnl_* windows every five minutes.
Description check Passed The description directly explains the new liq-pnl-refresh task, its six windows, timing, behavior, tests, and known verification limits.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 41.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 108 functions across 8 files. (1 skipped: 1 unsupported.)



  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR

🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR


  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@JuaniRios

Copy link
Copy Markdown
Collaborator Author

@rain-marvin review

@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rain-marvin

rain-marvin Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

🔎 Reviewing f2bb5f2, started by @JuaniRios. The review will appear here when it's done.

@rain-marvin rain-marvin Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adds the supervised liq-pnl-refresh task. Every 5 minutes it runs one PnL report per window (1w, 1d, all, 1m, ytd, 1y) through the same path as GET /pnl, and publishes all 25 liq_pnl_* series from the bot. Each window is its own family with its own collector timestamp. The task has its own one-permit admission, so the live /pnl permits stay free.

The latest round (f2bb5f2) fixes both CodeRabbit threads. compare.py now excuses a missing liq_pnl_day_* series only while the bot still publishes that name for the same window, so a whole dropped day name is a finding again, and a new test covers it. Symbol rows that strip to the same label are left out with an error log, so no row silently replaces another. Both changes do what the replies say.

The goal holds: the window table, the probe, the order and the 120 s budget match the exporter. A failed or skipped window keeps its last value, and its timestamp shows that it is stale. Nothing adds state, schema or config, so an upgrade or rollback cannot fail to start. The deferred items in the open threads stay as agreed follow-ups.

One minor point is new: the sample build does a lot of Float work on an async worker, while /pnl runs the same kind of work in spawn_blocking. Two doc nits are also new. No blocker.

I also agree with 1 open thread and replied +1 in each.

claude-opus-5-5 · high · 12 min

Comment thread src/metrics/liquidity/pnl_refresh.rs
Comment thread src/metrics/liquidity/pnl.rs
Comment thread docs/observability.md
This was referenced Oct 10, 2026
@rain-marvin

rain-marvin Bot commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

@JuaniRios 🔔 Follow-up: a day after my review

@JuaniRios, the bots are done with #1685: CodeRabbit and I both approved head f2bb5f2, CI is green, and all 12 review threads are resolved. It still needs a human review: human/reviewed is failing, and none of the requested reviewers (agryaznov, ueco-jb, rouzwelt, findolor) has approved it yet. It also can't merge until the stack below it does. rouzwelt has approved the bottom four PRs (#1673, #1674, #1665, #1678), but #1684, the PR directly below, has no human approval either, so ask a reviewer to take #1684 and #1685 together.

claude-opus-5-5 · high · 2 h 31 min

@agryaznov agryaznov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, the deferred minors are tracked in the threads.

@ueco-jb ueco-jb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at f2bb5f2. The per-window family replacement, independent admission, date ranges, failure/freshness handling, exact PnL conversion, and exporter parity are sound. Two non-blocking follow-ups remain: make independent cost fields distinct in the golden fixture so each mapping is pinned, and state the 90-bucket cut on all four day-series catalog rows. No blockers.

@graphite-app

graphite-app Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Merge activity

  • Oct 12, 4:04 PM UTC: JuaniRios added this pull request to the Graphite merge queue.
  • Oct 12, 4:04 PM UTC: CI is running for this pull request on a draft pull request (#1727) due to your merge queue CI optimization settings.
  • Oct 12, 4:20 PM UTC: Merged by the Graphite merge queue via draft PR: #1727.

@graphite-app graphite-app Bot closed this Oct 12, 2026
@graphite-app
graphite-app Bot deleted the rai-2998-liq-metrics-pnl-series branch October 12, 2026 16:21
@github-actions github-actions Bot added the externally-merged Added to a PR that appears closed but is actually merged by an external actor like Graphite label Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

externally-merged Added to a PR that appears closed but is actually merged by an external actor like Graphite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants