Skip to content

feat(worker): add generic Prometheus metrics to WorkerRunner + wire workers - #176

Open
extreme4all wants to merge 1 commit into
developfrom
feat/worker-generic-metrics
Open

feat(worker): add generic Prometheus metrics to WorkerRunner + wire workers#176
extreme4all wants to merge 1 commit into
developfrom
feat/worker-generic-metrics

Conversation

@extreme4all

Copy link
Copy Markdown
Contributor

Summary

Adds generic Prometheus metrics to the shared WorkerRunner framework and wires a metrics server into the framework-based workers.

Changes

Framework (components/bot_detector/worker/)

  • New metrics.py: 7 generic metrics + start_metrics_server(port) (idempotent, no-op under ENVIRONMENT=test)
  • core.py: WorkerRunner gains a required worker_name param; _consume() emits the metrics
  • Exported start_metrics_server from the component

Generic metrics (all labeled by worker):

Metric Type What
worker_messages_consumed_total Counter messages pulled from queue
worker_messages_committed_total Counter messages committed (processed)
worker_messages_requeued_total Counter requeued msgs by reason (worker_error/requeue/error/cancelled)
worker_batch_size Histogram batch size distribution
worker_handle_duration_seconds Histogram time in handle()
worker_batch_errors_total Counter exceptions in handle() by error_type
worker_poll_idle_total Counter empty polls

Workers (each: start_metrics_server() at startup + worker_name)**

  • worker_hiscore"hiscore"
  • worker_report"report"
  • worker_ban_migration"ban_migration" (also refactored its base metrics.py to drop the duplicate start_http_server, now centralized)

Projects

  • Added prometheus-client==0.22.0 to worker_hiscore and worker_report (+ re-locked). worker_ml skipped (doesn't use the worker component); worker_ban_migration already had it.

Validation

  • ruff check / ruff format --check: clean
  • pytest: 127 passed
  • Lockfiles: only functional change is +prometheus-client (rest is uv lock reformatting)

Related

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant