From 3cd0f6dcd89592c87e1e082cec07f1fb42b761f0 Mon Sep 17 00:00:00 2001 From: Adam Grare Date: Fri, 2 Oct 2026 09:17:58 -0400 Subject: [PATCH] Add AGENTS.md documentation on MiqQueue/Kafka There was a line in the product.md about the miq_queue table but nothing around MiqQueue and its uses, or Kafka and where that is used. Also specifically there was nothing indicating that e.g. new callers to `put_unless_exist` should be avoided. --- AGENTS.md | 2 +- docs/agents/architecture.md | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index f063cf59cd0..b922935b1c8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -77,7 +77,7 @@ Load these when the task requires it — do not load them speculatively: - **Product features / model names**: [`docs/agents/product.md`](docs/agents/product.md) — providers, resource management, service catalog, policy, automation, RBAC, deployment - **Making code changes**: [`docs/agents/coding.md`](docs/agents/coding.md) — non-obvious patterns, loggers, RuboCop, factory helpers -- **Architecture / design questions**: [`docs/agents/architecture.md`](docs/agents/architecture.md) — RBAC, Settings, ID regions, workers, plugin model +- **Architecture / design questions**: [`docs/agents/architecture.md`](docs/agents/architecture.md) — RBAC, Settings, ID regions, workers, queue, plugin model - **Working with RBAC or authorization**: [`docs/agents/rbac.md`](docs/agents/rbac.md) — data model, filter types, tenancy, feature flags, spec helpers - **Writing or modifying specs**: [`docs/agents/testing.md`](docs/agents/testing.md) — RSpec setup, FactoryBot, VCR, spec helpers - **Working in a plugin gem**: [`docs/agents/plugins.md`](docs/agents/plugins.md) — repo layout, bin/setup, engine, settings, spec structure diff --git a/docs/agents/architecture.md b/docs/agents/architecture.md index 0f56bf09413..2ae5bb5fb2e 100644 --- a/docs/agents/architecture.md +++ b/docs/agents/architecture.md @@ -32,6 +32,29 @@ lib/workers/bin/run_single_worker.rb Architecture must not rely on in-process shared state between workers. +## Queue + +As ManageIQ is a distributed and scalable product a queue system is used to distribute work to the various workers. There are at present two queue backend types used: + +- **MiqQueue**: A database backed work distribution system +- **Kafka**: A highly scalable event streaming platform + +### MiqQueue + +MiqQueue is the queue system used by ManageIQ from the beginning. It uses a table called `miq_queue` and combined with a number of columns (`queue_name`, `class_name`, `instance_id`, `args`, `zone`, and `role`) is used by Queue workers to get their work items. + +The MiqQueue interface is simple: `MiqQueue.put` and `MiqQueue.get` form the core of the MiqQueue interface and are used for submitting work for a worker and for a worker to dequeue a work item respectively. +Additionally there are some methods which are specific to `MiqQueue` and are not commonly provided by standard queue systems: `put_or_update`, `put_unless_exists`, and `unqueue`. New invocations of these methods should be avoided at all cost as they make it more difficult to move to a more standard queue backend. + +### Kafka + +Kafka is a highly scalable event streaming system, allowing for both "queue" and "topic" modes based on the configuration of broadcast groups. + +In "queue" mode it operates similar to `MiqQueue` where multiple workers can be listening to a queue but only 1 will receive the message. +In "topic" mode all workers listening to a queue will receive a message published to the queue. + +Currently ManageIQ uses kafka for the ems_events topic, so calls to `EmsEvent.add_queue` will publish the event to a kafka topic if kafka is configured. + ## Plugin / provider model Provider code (AWS, VMware, OpenStack, etc.) lives in separate gems (e.g. `manageiq-providers-vmware`) loaded as Rails engines at boot. Core models define abstract base classes (e.g. `ExtManagementSystem`, `VmOrTemplate`); providers subclass them. New features added to base classes automatically propagate to all providers.