Skip to content

perf: harden gateway, billing, and ops paths under high concurrency#4337

Open
DR-lin-eng wants to merge 140 commits into
Wei-Shaw:mainfrom
DR-lin-eng:main
Open

perf: harden gateway, billing, and ops paths under high concurrency#4337
DR-lin-eng wants to merge 140 commits into
Wei-Shaw:mainfrom
DR-lin-eng:main

Conversation

@DR-lin-eng

Copy link
Copy Markdown

Summary

  • bound OpenAI request and response buffering while improving channel-aware failover behavior
  • make usage billing durable under load with queued persistence, bounded fallback, and mutation-version cache guards
  • consolidate dashboard reads and add Redis-backed operational and runtime controls

Key changes

  • add durable usage billing queueing with synchronous fallback when critical work cannot be enqueued
  • prevent stale cache repopulation after billing and balance mutations
  • bound request bodies, SSE scanning, session state, OAuth refresh locks, and account runtime state
  • improve channel-aware scheduling, retry handling, and temporary account exclusion
  • add shared dashboard snapshots and merged usage aggregations
  • add Redis-only system logs, runtime retention overrides, user usage statistics, and explicit clear-all operations
  • add a global temporary-unschedulable switch with cross-layer cleanup
  • expose the new scheduling, logging, retention, and usage controls in the admin UI

Testing

  • go test ./...
  • golangci-lint v2.9.0 run --timeout=30m ./...
  • make test-frontend
  • pnpm --dir frontend run build

I have read the CLA Document and I hereby sign the CLA

@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Before we can merge this PR, we need you all to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.


I have read the CLA Document and I hereby sign the CLA


13 out of 14 committers have signed the CLA.
✅ (DR-lin-eng)[https://github.com/DR-lin-eng]
✅ (BenjaminAaron196)[https://github.com/BenjaminAaron196]
✅ (yardbirds0)[https://github.com/yardbirds0]
✅ (Eyre921)[https://github.com/Eyre921]
✅ (dftian478)[https://github.com/dftian478]
✅ (SatellaPoi)[https://github.com/SatellaPoi]
✅ (wucm667)[https://github.com/wucm667]
✅ (aomsir)[https://github.com/aomsir]
✅ (adamglin0)[https://github.com/adamglin0]
✅ (feitianbubu)[https://github.com/feitianbubu]
✅ (DylanChiang-Dev)[https://github.com/DylanChiang-Dev]
✅ (AixLau)[https://github.com/AixLau]
✅ (heathermhuang)[https://github.com/heathermhuang]
@CRAZYShimakaze
You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@DR-lin-eng

Copy link
Copy Markdown
Author

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.

I have read the CLA Document and I hereby sign the CLA

You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

I have read the CLA Document and I hereby sign the CLA

@DR-lin-eng

Copy link
Copy Markdown
Author

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).
To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.
I have read the CLA Document and I hereby sign the CLA
You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

I have read the CLA Document and I hereby sign the CLA

recheck

@DR-lin-eng

Copy link
Copy Markdown
Author

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.

I have read the CLA Document and I hereby sign the CLA

You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

recheck

@DR-lin-eng

Copy link
Copy Markdown
Author

recheck

@DR-lin-eng

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@DR-lin-eng

Copy link
Copy Markdown
Author

recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.