kafka.md

Apache Kafka Provider

Apache Kafka support for LibreDB Studio, read-only over the Kafka protocol through @platformatic/kafka. The editor text is a JSON read request, declared queryLanguage: "json" with queryDialect: "kafka": one object that names a topic and where to start reading. Studio browses topics, partitions, consumer groups with their lag, brokers and configs, and reads messages by partition, offset or timestamp; it never produces, commits, joins a group or creates a topic. This document is the single reference for the Kafka provider: design, architecture, usage and tests.

Status	Implemented & shipped
Database type id	kafka
Family	Stream (src/lib/db/providers/stream/kafka/), the first provider in that family
Driver	@platformatic/kafka 2.14.0, pinned exactly, pure TypeScript, loaded under Bun and Node alike (§2.5)
Query language	json with queryDialect: "kafka": a JSON read request of this product's own schema, not MongoDB's JSON (§5.1)
Default port	9092, the port a stock broker listens on; the same number is the default under TLS, because a secured listener serves on whatever port its operator chose
Connection pooling	One Admin, one Consumer and one fetch ConnectionPool per connection, all closed by disconnect() (§3.4)
Connection string	Not supported: a Kafka client takes a bootstrap address, and no URI convention is read
SSH tunnel	Refused: a tunnel forwards one address, and a Kafka client reaches every broker at the address the broker advertises (§4.5)
EXPLAIN	None
Write reach	none: the client sends only the read APIs of its API key set KAFKA_READ_APIS, the editor's text is a read request parseReadRequest admits or refuses, and there is no object edit, maintenance or transaction route, so every connection is read-only and a read-only one is accepted (§11.5)
Transactions	Not applicable; reads are read-committed (§5.2)
Maintenance	None
Query cancellation	None: a read is bounded by its limit, its budgets and its timeout, and there is no server-side evaluation to stop (§5.5)
Verified against	Apache Kafka 4.3.1, the official apache/kafka:4.3.1 image, as a single KRaft node (kafka), a three-node KRaft cluster (kafka-cluster) and a TLS, SCRAM and authorizer broker (kafka-auth), all in database-compose.yml; the captures of tests/fixtures/kafka/README.md were taken on 2026-09-24
Source	src/lib/db/providers/stream/kafka/
Tests	tests/integration/db/kafka-provider.test.ts + tests/unit/db/kafka/
Tracking issue	#1088, the approved design, under #424

---

1. Overview

Apache Kafka is a distributed log: producers append records to the partitions of a topic, and consumers read them back by offset. It is not SQL-shaped: there are no tables, no rows a client updates, and no query language on the broker. What a user browses is what the cluster holds: topics and their partitions, consumer groups and how far behind they are, and brokers and their configs. The teams this product is built for run a message log next to their databases, which is why it ships here (#1088, section 1).

Concept mapping

Studio concept	Kafka concept
Relation (a row in the tree with columns)	A topic; its columns are the fixed fields of a read result (§6.1)
Row	A message (rowName "Message")
Statement	One JSON read request (§5.1)
Grouping folder	A consumer group, as a leaf with its lag in its source
Configuration object	A broker, with its configs in its source
Object source	The broker's own answers for that object, re-serialised with JSON.stringify
Container	None: containerLevels: [], the Elasticsearch and SQLite shape, because one connection is one cluster

---

2. Architecture

2.1 Where it sits

src/lib/db/providers/stream/kafka/
├── index.ts               KafkaProvider extends BaseDatabaseProvider: lifecycle and composition only
├── client.ts              THE SEAM: the KafkaReadClient interface (read methods only), its types, KafkaError. No I/O
├── platformatic-client.ts The only file that imports @platformatic/kafka; holds the two raw calls
├── connection-options.ts  DatabaseConnection to the bootstrap address, tls and sasl, after validation
├── request.ts             Pure: the editor JSON to a ReadRequest
├── read.ts                Start offsets and the bounded fetch loop
├── decode.ts              Pure: key, value and header decoding
├── results.ts             Pure: decoded records to a QueryResult
├── groups.ts              Group listing, description and the pure lag function
├── objects.ts             Kind declarations and the object surface
├── monitoring.ts          Pure mappings from metadata, config and log-dir answers to the monitoring types
└── errors.ts              KafkaError category to the repository's error classes

Each module has one reason to change (#1088, section 3.5). index.ts holds no wire vocabulary, no request parsing, no decoding, no result shaping, no lag arithmetic and no error table; it delegates each to the module that owns it. read.ts, groups.ts, objects.ts and monitoring.ts receive a Pick<KafkaReadClient, ...> slice, never the whole client or the provider, and every high-level module depends on the KafkaReadClient interface, never on platformatic-client.ts. Nothing outside this directory is imported from another provider: the Couchbase TLS mapping is the reference for connection-options.ts and not a dependency. The host and port validators are the shared validateHost and validatePort of src/lib/db/http/endpoint.ts, not a copy.

2.2 Class hierarchy

DatabaseProvider (interface)
└── BaseDatabaseProvider
    └── KafkaProvider

BaseDatabaseProvider directly, as for MongoDB, Redis, Couchbase, Prometheus and LibreDB: a read request is not SQL, so nothing SQLBaseProvider adds would be true here. A separate StreamProvider root beside DatabaseProvider was considered and set aside: every route under src/app/api/db/ reaches one DatabaseProvider, and the v1 surfaces fit that contract (#1088, section 3.1).

2.3 What the base class gives for free

Connection-state bookkeeping, the configured query timeout, and the error logging every provider shares. Nothing Kafka-specific lives in the base class.

2.4 Registration & lifecycle

The factory builds it through createDatabaseProvider() (factory.ts) with a dynamic import, so no other engine loads this module, and the client library itself loads only when the first Kafka connection connects. Its DB_UI_CONFIG entry draws it in text-hue-green, because Kafka's own mark is black, which is no identity hue, and SHOWCASE_RANK places it on the login showcase behind Prometheus and ahead of libSQL. The constructor validates nothing and opens nothing, so a provider built only to read its declarations touches no socket. connect() validates the connection (§4) before any client exists, builds the client, and reads the brokers once, forced past the client's cache, which proves a broker answers the protocol at that address with that credential. That read runs under the connection's query timeout as a whole, retries included: the client retries it once, and the ApiVersions read inside it once, each after a one-second delay, and its connect and request timeouts bound only one attempt. Unbounded, a host that accepts and never answers held a connect with a 2 s query timeout for about 11 s, and a Windows runner, where a refused connect takes about 2 s of SYN retries, held a connect to a closed port for over 20 s. A connect still unanswered when the query timeout runs out fails as a ConnectionError with connect-timeout (§10), on every platform. A connect that fails after the client was built closes it again, which also stops the retries the client had scheduled. disconnect() closes the Admin, the Consumer and the fetch pool, and so does the factory's idle eviction.

2.5 The client, and why

@platformatic/kafka, used fetch-only, was chosen at 2.11.0 by a broker-side measurement before any code was written (#1088, section 3.2 and Appendix B). 2.12.0, 2.13.0 and then 2.14.0 were each taken only after the live read-only check passed on them (§11.4). Reading by partition, offset and timestamp through its listOffsets, listOffsetsWithTimestamps and a fetch registered no consumer group and never created __consumer_offsets, under Bun and Node. Its consume() in MANUAL mode, with explicit offsets and autocommit: false, joins the group and leaves an Empty group registered after close(), so consume() is never called. All four codecs, gzip, snappy, lz4 and zstd, decode with no native addon: snappy and lz4 come from WebAssembly, gzip and zstd from node:zlib, and the optional @node-rs/crc32 falls back to WebAssembly. @confluentinc/kafka-javascript was ruled out because its group-free assign() is a librdkafka native addon that fails under Bun, which runs this repository's tests, and kafkajs because it cannot read without joining a group and has had no functional commit since 2023. The Next.js build keeps the package external (serverExternalPackages in next.config.ts), because its optional native CRC32C addon stops Turbopack; external, the library resolves the addon at run time and falls back to WebAssembly without it. The client loads ajv-draft-04 at import time, and ajv-draft-04 requires ajv/dist/core, which ajv 6 does not have, from where it is installed. bun hoists ajv-draft-04 to the top of node_modules, so the ajv there must be 8. In this repository the direct ajv dependency puts ajv 8 there (tests/unit/db/kafka/dependency-resolution.test.ts), which fixes this repository's own install only. A host that installs the published package with bun, and whose own tree leaves an ajv 6 at the top of its node_modules (its own, or the one eslint 9 brings), needs ajv ^8 resolvable at its own root, for example as a dependency of its own; a host that keeps ajv 6 as its own dependency can install with bun install --linker isolated instead. npm nests ajv-draft-04 beside ajv 8 under the client, so a host that installs with npm needs neither (measured with bun 1.4.2 and npm 11.9.0). Where the installation does not resolve a package the client requires, connect() refuses with a DatabaseConfigError naming it, never the runtime's own text, which carries the server's paths: The Kafka client library could not be loaded: the module "ajv/dist/core" it requires does not resolve in this installation. See docs/providers/kafka.md section 2.5. The refusal reads the runtime's error where that error names the package, which this failure's does under Node and Bun, since ajv-draft-04 requires ajv/dist/core from CommonJS. Node's ES module loader names a subpath missing from an installed package by the file's absolute path instead, so where an ES module among the client's dependencies imports a subpath from a package installed at a major without it, a connect fails with the runtime's own error.

Supported brokers: a read sends Fetch v13, the first version that names a topic by id (KIP-516), which Apache Kafka answers from 3.1 on. The client's own README states Apache Kafka 3.5.0 to 4.2.0 as its supported range; this provider was verified against 4.3.1. A broker whose Fetch range does not hold version 13 is refused at the first read with a QueryError naming its range, such as "This broker answers Fetch versions 0 to 12; a read sends Fetch 13, which Apache Kafka answers from 3.1 on".

---

3. Design decisions

3.1 A JSON dialect, not a third query language

The editor text is JSON, so the provider declares queryLanguage: "json" and names its kind of JSON with queryDialect: "kafka", the rule Prometheus's design stated: queryDialect distinguishes kinds of JSON, and a new queryLanguage is for text that is not JSON (#1088, section 3.3). The cost is the #427 bug class: a reader keyed on "json" alone treats the text as MongoDB. Every reader of either field has a Kafka arm or a test pinning that its branch is correct for Kafka: a Kafka tab type rendered in Monaco's built-in json mode, a Kafka record in DIALECT_GENERATORS, read before the MongoDB arm in both generators, the MongoDB completion provider registered only where no JSON dialect is declared, and the gates below. On a topic row both row menus withhold Profile, through offersColumnProfiling, Code Generator, through offersCodeGeneration, and Generate Test Data, because no kind declares acceptsRowWrites. Code generation is withheld because the models it writes over a topic's fixed columns reject the rows a read returns: a Date timestamp against the ISO string, a record-typed value against text, base64 and the Confluent label. Both menus also withhold Generate Count Query, through offersCountQuery (#702): a read request is JSON of its own dialect, with no count grammar. The execution confirmation gate never asks on a Kafka connection: a read request cannot write, so the kafka row of NON_SQL_DESTRUCTIVE_VOCABULARY (src/lib/db/providers/stream/kafka/destructive.ts, collected in src/lib/db/destructive-commands.ts) names no operation and is the gate's whole answer, and a topic named delete or drop is never read as SQL. No Monaco language and no JSON schema for the request are registered; a schema with completion and validation is recorded in docs/BACKLOG.md. Widening the published queryDialect union breaks an external consumer's exhaustive switch over it, and that ships with a release note rather than a compatibility layer.

3.2 Read-only by construction (K4)

Two checks hold the guarantee, and both are required. The seam guard, tests/unit/db/kafka/seam-guard.test.ts, fails the build if @platformatic/kafka is loaded anywhere in the repository but platformatic-client.ts and the fixture seed docker/kafka/seed-binary.ts, or if the provider calls a member of the library's objects outside this allowlist: metadata, listTopics, describeConfigs, listGroups, describeGroups, listConsumerGroupOffsets, listOffsets, listOffsetsWithTimestamps, describeLogDirs, findCoordinator, listApis, close, connect on the one-off Connection of a consumer-protocol group's description, get on the fetch ConnectionPool, and the two raw calls consumerGroupDescribeV0 and fetchV13. So consume, commit, joinGroup, leaveGroup, Producer, createTopics, deleteTopics, alterConfigs, incrementalAlterConfigs, alterConsumerGroupOffsets and deleteRecords cannot be written in the provider, and neither can an autocreateTopics other than the literal false or a Consumer groupProtocol other than "classic". Every member of the library classes the adapter holds is classified against the installed package as allowed, forbidden or unused, so a client version that adds a member fails the guard until someone classifies it. The guard also pins the agent read path (§15.1): KafkaAgentReadClient picks the client's metadata, listTopics and offsets alone, every agent-mode module names no other client type, imports only the modules and names it lists, and calls nothing else on its client, and the provider hands an agent-mode surface a new object holding those three reads and nothing more. The guard is syntactic: it catches an ordinary edit that reaches the library outside the adapter or calls a member outside the allowlist, and its docblock states the ways code written to hide a load or a call can pass it, which are stated rather than chased. The second check is behavioural: the live harness snapshots the broker before and after its whole run (the topic and group lists, every partition's earliest and latest offsets, every group's committed offsets, the topic and broker configs, and on kafka-auth the ACLs) and fails on any difference, and it asserts that the broker log names no group-coordinator entry for the Consumer's group id, libredb-studio-never-joined, which the constructor requires and the broker never receives, and no automatic topic creation. The diff catches a future client version whose read path starts joining a group, committing or writing, which the seam guard cannot see.

3.3 Never creating a topic

The broker's default auto.create.topics.enable=true is live: a metadata read of a missing topic with autocreateTopics: true created it in the measurement. So platformatic-client.ts passes autocreateTopics: false on every metadata read, explicitly rather than by the client default, and a missing topic answers a QueryError saying the topic does not exist. A missing internal topic is the exception on a broker before Apache Kafka 2.8, which creates it for any Metadata request that names it, whatever allowAutoTopicCreation and auto.create.topics.enable say, so no request the provider sends names one (§6.1).

3.4 One client per connection, and no fetch session

One Admin, one Consumer and one fetch ConnectionPool are built from one options object per connection, and nothing else is built later but the one-off Connection a consumer-protocol group's description opens and closes in a finally. A read never goes through the client's Consumer.fetch, for two measured reasons. Its READ_COMMITTED filter reads a control batch's first record and an aborted range's end unguarded, inside the socket handler, so an answer that holds an empty control batch, which Kafka's log cleaner keeps of a producer's last marker, or an ABORT marker of a producer the answer does not list, beside a listed aborted transaction, throws there, and that fetch never settles. And it keeps one fetch session per broker, which two fetches in flight at once collide on (INVALID_FETCH_SESSION_EPOCH, about a second a round with the client's retry delay). So the adapter sends Fetch v13 itself, through the client's exported fetchV13 module, on a connection its ConnectionPool holds to the leader's advertised address, READ_COMMITTED and sessionless (session id 0, epoch -1), with currentLeaderEpoch: -1 so a read across a leader failover is not fenced. The client parses that answer inside its own try and settles the call either way, so a compacted transactional topic reads in full; with no fetch session, reads that overlap run side by side; and the answer reaches only the adapter's own filter, the Java consumer's rule of §5.2. The consumer-protocol group description is the other raw call, ConsumerGroupDescribe (API 69) through the exported consumerGroupDescribeV0, because the client has no Admin method for it. Both depend on exports the client does not document as public API; the seam guard pins them, and an Admin method for ConsumerGroupDescribe is requested upstream as a draft recorded in docs/BACKLOG.md.

3.5 A host that embeds the provider factory

The client handles some answers inside its socket handler, where an exception is uncaught. The adapter keeps every input known to throw there away from the client: it refuses an internal topic (§6.1) and a SASL user that is not a string (§4.2) before the call, and a read's answer never reaches the client's READ_COMMITTED filter (§3.4). A malformed answer, which a hostile broker can send (§4.4), can still throw there on the client's other calls. describeGroups is no longer one of them: since @platformatic/kafka 2.13.0 it resolves a classic group whose member's subscription or assignment it cannot decode with the group's error set and only the members before that one, and the provider refuses that group its whole description instead (§6.1). In the standalone server, whose Next.js router server registers an uncaughtException handler, the exception is logged and that one call never settles until its timeout. A host that embeds the package's provider factory, createDatabaseProvider or getOrCreateProvider from src/exports/providers.ts, in a Node process that installs no uncaughtException handler, ends the process instead. Such a host should install a handler before it connects a Kafka provider to a broker it does not trust.

3.6 Measurements

Every number below is a constant the code uses, read back by tests/unit/db/kafka/provider-doc.test.ts, so a row that disagrees with the code fails the build.

Constant	Value	What it bounds
DEFAULT_QUERY_LIMIT	500	The largest limit a read request takes, counted across its partitions; the default is 50
KAFKA_RESULT_BYTE_BUDGET	8388608	Bytes (8 MiB) of record data the rows of a result hold: keys, values, header names and values, as the client hands them over after decompression (§5.4)
KAFKA_CELL_LIMIT	65536	Characters (64 Ki) of rendered text in one cell (§5.3)
KAFKA_TOPIC_LIST_CAP	2000	Topic names the tree and the inventory hold; past it the Topics count is a floor (§6.1)
KAFKA_DEFAULT_PORT	9092	The default bootstrap port

A fetch waits at most 250 ms on the broker for new records, so an empty partition does not hold a request open.

---

4. Connection

4.1 Configuration fields

Field	Used	Notes
host	Yes	The bootstrap broker: a hostname or an IP literal; anything carrying :, /, @, % or whitespace is refused before any socket opens, with a DatabaseConfigError that does not echo the value
port	Yes	Default 9092; outside 1 to 65535 is refused the same way
saslMechanism	Optional	Labelled "SASL mechanism" in the dialog, a select offering None, PLAIN, SCRAM-SHA-256 and SCRAM-SHA-512, with the hint "PLAIN and SCRAM require TLS"
user	With a mechanism	Sent only when a mechanism is chosen
password	With a mechanism	Sent only when a mechanism is chosen
database	No	No Database box renders: one connection is one cluster, so there is nothing to select
TLS panel	Yes	§4.3
SSH tunnel	No	The panel is hidden and a tunnel is refused (§4.5)

The select is declared on the kafka entry of CONNECTION_FORM in src/lib/db/connection-form.ts (fieldOptions, fieldLabels, fieldHints) rather than chosen by a type test in the dialog. saslMechanism is an optional, non-secret field of DatabaseConnection: it names how the password is checked, not a credential, so the connection store keeps it in the clear and a managed seed returns it while it withholds the password. It is part of the provider cache key's credential frame (credentialDigest in src/lib/db/provider-cache-key.ts), because Kafka keeps SCRAM credentials per mechanism, so two connections that differ only in mechanism never share a provider. A seed file takes it as a literal name (docs/SEED_CONNECTIONS.md).

One bootstrap address is enough: the client learns every broker from the metadata it answers, so a second address would help only when the first is down at connect time, and v1 takes one.

4.2 Authentication, and never in the clear (K3)

saslMechanism	user and password	What happens
none	empty	No authentication
none	either set	Refused with a DatabaseConfigError naming the missing mechanism, never a silent drop of the credential
PLAIN, SCRAM-SHA-256 or SCRAM-SHA-512	set	SASL with that mechanism, over TLS only
anything else, null and "" included	set or empty	Refused with a DatabaseConfigError that names the three mechanisms and never the value: only an absent saslMechanism means none
any mechanism	TLS mode disable	Refused with a DatabaseConfigError before any socket opens

PLAIN sends the password in the clear, and a Kafka credential is usually cluster-wide, so SASL of any mechanism over plaintext is refused. That is stricter than the Prometheus provider, which sends a credential over plain HTTP. A CR, LF or NUL in user or password is refused with a DatabaseConfigError that names the field and never contains the value, and nothing is trimmed silently. A user or password that is not a string, such as a number or an array in a connection sent to the API, which reaches the provider as it came, is refused the same way, whatever the mechanism, and is never coerced: the client's SCRAM step reads the user as a string inside its socket handler, where a number throws and ends a Node process that installs no uncaughtException handler (§3.5), and its PLAIN step joins the credential into text, so ["reader"] would authenticate as reader. A null in either reads as an absent field. The client's protocol logger prints the first bytes of every request frame when DEBUG names it (plt:kafka:protocol, or DEBUG=*), and a SASL PLAIN frame carries the password, so platformatic-client.ts mutes that logger when it loads the client, while the client's own logger keeps its lines. OAUTHBEARER, GSSAPI and AWS MSK IAM are not offered: a pasted bearer token expires within hours and v1 has no refresh flow (docs/BACKLOG.md).

Accepted limitation: the broker chooses how much work a SCRAM exchange takes. A SCRAM exchange runs PBKDF2 over the password as many times as the broker's first SCRAM answer asks, and the client checks only the lower bound, 4,096. Apache Kafka 4.3.1 stores no SCRAM credential above 16,384 iterations (ScramMechanism's maximum, which its controller and its storage tool enforce), so a larger count comes only from a hostile broker (§4.4) or from one in the middle of a connection whose certificate is not verified, as TLS mode require leaves it. Each exchange holds a thread of the runtime's pool for as long as its count takes, measured at about 2.9 million SHA-512 iterations a second under Node 24.14.0 and 3.3 million under Bun 1.4.2, so the largest count the runtime's PBKDF2 accepts, 2,147,483,647, takes about 11 to 12 minutes, and a larger one fails the authentication at once. The client retries a failed connect once, so a connect asks for the work twice, and the work goes on after the connect has timed out and after the client is closed. Node's pool has four threads unless UV_THREADPOOL_SIZE says otherwise, and every file read, DNS lookup and crypto call of the one Studio process every user shares waits for a free thread: with four such exchanges running under Node, a file read took 4,490 ms instead of 1 ms. The client offers no hook that runs before that work short of replacing its SCRAM implementation, which this provider does not own, so the exposure is stated rather than bounded; TLS verify-full keeps it to brokers whose certificate the configured CA vouches for, and a cap on the iteration count is requested upstream as a draft recorded in docs/BACKLOG.md.

Accepted limitation: the broker chooses how often a connection authenticates again. A broker that answers a SASL authentication with a session lifetime (KIP-368; on Apache Kafka connections.max.reauth.ms, which defaults to 0, no lifetime) has the client authenticate again at 80% of it, and again after each time, for as long as the connection is open, with no floor on the lifetime. Measured on one connection over five idle seconds with a lifetime of 1 ms: 3,926 re-authentications under Node 24.14.0 and 3,005 under Bun 1.4.2 with PLAIN, 11 to 14% of a core, and 1,559 and 1,609 with SCRAM-SHA-512 at 4,096 iterations, about 60% of a core, where a lifetime of 0 or of one hour made none. Every connection the provider holds carries the connection's SASL options, the Admin's, the Consumer's, the fetch pool's and a consumer-protocol group description's alike, and a cached provider keeps its connections until it is disconnected or evicted, which the factory's sweep, every five minutes, does once the provider has been idle for 30 minutes, so a hostile or misconfigured broker can keep that much of the one Studio process busy per connection for as long; closing a connection stops its loop. The client applies the lifetime inside its connection, where the adapter's only hook, authBytesValidator, sees an answer's bytes and never its lifetime, so the exposure is stated rather than bounded, and a floor on the session lifetime the client honours is requested upstream as a draft recorded in docs/BACKLOG.md.

4.3 TLS

ssl.mode	What happens
disable, or no TLS panel	Plaintext; SASL is refused (§4.2)
require	Encrypted, certificate not verified unless rejectUnauthorized is set
verify-system, verify-ca, verify-full	Certificate verified against the supplied CA or, without one, the runtime's trust store

ssl.caCert, ssl.clientCert and ssl.clientKey reach the client as ca, cert and key, and rejectUnauthorized = ssl.rejectUnauthorized ?? ssl.mode !== "require", the Couchbase rule written again here by the isolation rule (docs/BACKLOG.md D37). The panel is checked whole, whatever its mode, before any of it is read: an ssl that is not an object, a mode the table does not name, a caCert, clientCert or clientKey that is not a string, or a rejectUnauthorized that is not a boolean is refused with a DatabaseConfigError naming the field and never its value. Such a panel is never guessed at: read as it came, ssl: false would turn TLS on, and a certificate that is not a string would reach Node, whose TypeError can quote the value. A null reads as an absent field, and a panel with no mode, which a seed file may carry, is read as a verifying one. With TLS on, the adapter sets the client's tlsServerName option, so each broker connection sends its own host as its TLS server name (SNI), which SNI-routed listeners need. The option is set rather than left to the client's default, which turned on only in 2.14.0; Node itself sends no server name unasked. An IP literal is no legal server name, and Node 26, the production runtime, throws on one; since 2.14.0 the client decides per connection and sends none for an IP literal, so a broker advertised by IP is checked against the certificate's IP names while the brokers the same cluster advertises by name still get their server name. A handshake or verification failure is a ConnectionError carrying the Node error code, and no connection is retried without TLS. A verification failure is known by any name Node gives a refused certificate, such as UNABLE_TO_VERIFY_LEAF_SIGNATURE, DEPTH_ZERO_SELF_SIGNED_CERT or INVALID_PURPOSE. A client certificate the broker refuses, or none where the broker requires one, reads as a lost connection, not as a TLS failure: under TLS 1.3, which the kafka-auth fixture negotiates, a broker refuses a client certificate after the client's half of the handshake, and the client reports only that the connection closed, handing the socket's error, which carries the alert, to no caller. A request upstream for the client to keep that error is drafted in docs/BACKLOG.md. Each outcome is exercised by a real handshake against a local TLS server in tests/unit/db/kafka/tls-handshake.test.ts, with certificates generated at test time and never committed, and the server-name rule under Node in a child process, because Bun sends a server name unasked.

4.4 The broker chooses where Studio connects next

A Kafka client connects to the addresses the broker advertises in its metadata, and authenticates to each with the connection's SASL credentials. A hostile or misconfigured broker can therefore direct Studio, with its credentials, to any host and port. This is stated, not mitigated: restricting advertised hosts to the bootstrap host's domain breaks legitimate clusters, since managed services advertise per-broker names. Use TLS with verify-full, under which credentials reach only hosts whose certificate the configured CA vouches for; docs/SECURITY.md records it with the limitation it widens. An advertised address Studio cannot reach after the bootstrap succeeded is the most common Kafka connection failure behind Docker and NAT, and it is a ConnectionError naming the advertised host and port: "The broker advertised :, which this server cannot reach; the cluster's advertised listeners must be reachable from where Studio runs".

4.5 No SSH tunnel

A tunnel forwards one address, and the client reaches every broker at the address that broker advertises, which the tunnel does not carry, so the tunnel would carry the bootstrap connection only and every read would go to the advertised addresses directly. The dialog hides the SSH panel for Kafka through showSshTunnel: false, while it keeps the TLS panel, and buildConnection writes sshTunnel only for a type that offers the panel (offersSshTunnel), so a tunnel left in the dialog's state by another connection is dropped. A tunnelled connection that arrives another way, from a seed or the API, is refused before any broker request with a DatabaseConfigError: "Kafka does not run through an SSH tunnel: the tunnel forwards one address, and a Kafka client reads from every broker the cluster advertises, at the address the broker advertises. Connect to the brokers directly". There the server opens the tunnel before the provider runs, so an unreachable bastion answers with its SSH error first. The server opens a tunnel for any sshTunnel.enabled that JavaScript reads as true, such as the string "true", so an enabled that is not a boolean is refused too, naming the field, rather than read as no tunnel. An sshTunnel that is not an object, such as true, is refused the same way, as an ssl that is not one is, though the server opens no tunnel for it.

Connection string

kafka has no connection string scheme, no readTls, no paste and no seed load check: every field of its row in CONNECTION_STRING is null (connection-string.ts).

Execution UI

The rolled-back notice names COMMIT as the keyword that keeps an open transaction, and kafka has no consent deadline, so the agent's consent card shows no deadline chip or sentence for it. Its row in EXECUTION_UI is declared in execution-ui.ts. What a tree click and Generate Query write is KAFKA_GENERATORS, the kafka row of DIALECT_GENERATORS, declared in generators.ts and read by the engine-agnostic dispatcher in src/lib/query-generators.ts.

---

5. Query interface

5.1 The read request

The editor text is one JSON object with a strict schema: an unknown key, a wrong type or a missing required key is a QueryError naming the key, before any request.

{ "topic": "orders", "from": "latest", "limit": 50 }
{ "topic": "orders", "partition": 0, "from": "earliest", "limit": 50 }
{ "topic": "orders", "partition": 0, "from": { "offset": "120" } }
{ "topic": "orders", "from": { "timestamp": "2026-09-23T00:00:00Z" } }

Each line above is one request; the editor sends its whole buffer as one.

Key	Required	Meaning
topic	Yes	A string matching ^[a-zA-Z0-9._-]{1,249}$, other than . and ..: Kafka's legal topic name
partition	No	A whole number from 0 to 2147483647, Kafka's INT32; absent means every partition of the topic
from	No	Where each partition starts; default "latest" (below)
limit	No	A whole number from 1 to 500 (DEFAULT_QUERY_LIMIT), default 50, counted across all partitions; "limit": null is a wrong type, never the default

The from forms:

• "earliest": each partition from its earliest offset.
• "latest": the last limit messages, each partition starting at the larger of its earliest offset and its latest offset minus limit.
• { "offset": n }: requires partition; n is a non-negative whole number, or a digit string, because offsets are 64-bit, at most 9223372036854775807, Kafka's INT64. An offset written as a JSON number above 2^53 is refused with advice to write it as a digit string, because JSON.parse has already rounded it. An offset outside the range a read reaches, from the partition's earliest offset to its last stable offset, is a QueryError naming that range; while a transaction is open on the partition, the last stable offset lies below the log end the topic's source shows, so an offset the source lists can lie past it.
• { "timestamp": "<ISO-8601>" }: each partition from the first offset whose timestamp is at or after the instant, through listOffsetsWithTimestamps. The instant must carry a zone, because the server's zone and the browser's would read it differently; it must name a day its month has, because Date.parse rolls 2026-02-30 into March; and it must fall at or after 1970-01-01T00:00:00Z, because ListOffsets reads -1 and -2 ms as its latest and earliest sentinels. It is written as YYYY-MM-DDThh:mm, with optional seconds of at most three decimals, then Z, +hh:mm or -hh:mm; a finer fraction is refused rather than cut, because Date.parse cuts it to the millisecond, and a record less than a millisecond before the instant would then read as at or after it. A partition with no message at or after the instant below its last stable offset contributes no rows, and the result carries a warning naming each such partition with that offset; while a transaction is open there, messages at or after the instant can lie past it, below the log end, where a read-committed read does not reach until the transaction ends.

Bound params are refused with a DatabaseConfigError: there is no binding, and ignoring them would run a different read from the one the caller built. Text that is empty once whitespace is removed is refused with a QueryError, before the request is parsed. Invalid JSON is refused with a QueryError carrying the fixed text "The read request is not valid JSON", never the parser's message, which under Node quotes the start of the text. A partition the topic does not have is refused with a QueryError naming the partitions the topic has, such as Topic "orders" has partitions 0 to 2; partition 3 does not exist, before any offset is read. supportsResultPagination and supportsExternalQueryLimiting are both false, as on Redis and MongoDB, and prepareQuery is the base pass-through: the request carries its own limit.

A topic click in the tree writes and runs {"topic": <name>, "from": "latest", "limit": 50}, the action the labels name "Read Latest 50". Generate Read Request opens {"topic": <name>, "partition": 0, "from": "earliest", "limit": 50} in a new tab: an offset of 0 lies before the earliest offset of almost every topic retention has trimmed, and the result's offset column then shows the offsets the {"offset": n} form can use. It is one object, not a set of snippets: the tab's whole buffer is sent as one read request, and JSON has no comments to hold alternatives. Format rewrites the request as indented JSON and reads the same request back: a digit-string offset stays a string.

Where a read starts and ends:

• Every partition a read covers must be named in each offsets answer it uses, the earliest and latest offsets and, for a timestamp read, the offsets at the timestamp; a partition an answer leaves out is refused with a QueryError naming the topic, the partition and the answer, before any fetch, and never started from an invented offset.
• A read stops at each partition's last stable offset, the end a read-committed fetch reaches, while a group's lag is measured to the high watermark (§6.1); on a partition with an open transaction the two ends differ.
• A fetch the broker refuses as out of range, because the partition's log moved after the read began, trimmed by retention or a deletion or truncated, is refused with a QueryError naming the partition, the offset and the range the partition holds now, read again once; the fetch is never sent again, even when that range holds the offset, so the read has to be run again. When an offsets answer read again leaves the partition out, the refusal says which answer did and names no range.
• The offsets are read one answer at a time, the earliest, then those at the timestamp, then the latest, so a record written while the read is positioned lies below the end it stops at. A timestamp read looks each partition up once, before it reads the end, so a message written after that lookup is not in the read even when it lies below the end.
• A transaction's COMMIT and ABORT markers and an aborted transaction's records use offsets but are never rows, so a "latest" read of a transactional topic, whose window is limit offsets per partition, can answer fewer than limit rows.
• Which messages a read answers: of each partition's first limit messages from its start in log order, or for "latest" those of its last limit offsets, the first limit in result order, or the last for "latest". Result order is by timestamp, and a producer's CreateTime can go back along the log, so a message the read never fetched, beyond a partition's window, can have a timestamp that would have placed it among the rows.

5.2 Result shape

One row per record, with the fields partition, offset, timestamp, key, key_encoding, value, value_encoding and headers, in that order.

• Across partitions, rows are ordered by timestamp, then partition, then offset; within one partition that is log order only while timestamps rise with offsets, as under LogAppendTime or one producer's clock.
• offset is a decimal string, because offsets are 64-bit and a JavaScript number loses precision past 2^53; partition is a number.
• timestamp is an ISO-8601 UTC string with millisecond precision, the form the chart tab recognises as a date. It is the record's Kafka timestamp: the producer's CreateTime, or on a topic whose message.timestamp.type is LogAppendTime, the broker's append time, which the broker writes only into the batch's maxTimestamp, so the provider reads it from there as the Java client does. A record whose timestamp is the protocol's no-timestamp value, -1, has a null timestamp, never the instant one millisecond before the epoch. A timestamp outside the range a JavaScript Date holds is refused with a QueryError naming the partition, the offset and the timestamp.
• headers is an object from header name to decoded value, each name an own key, so a header named __proto__ or toString is a header like any other; a name repeated in one record becomes an array in arrival order, so no header the cell holds is lost. A header value of null stays null. The headers are one cell under KAFKA_CELL_LIMIT, like the key and the value: they join it in arrival order while the JSON the grid renders for it stays within the limit, and the first header that would pass it ends the cell, with every header after it, as a cut cell, so a record's headers never answer more than one cell's worth of text, however many it carries.
• Reads are read-committed: a COMMIT or ABORT marker is never a row, a record of an aborted transaction is dropped by the Java consumer's rule, including when the ABORT marker lies in a later response, and a marker still advances the read past its offset. An empty control batch, which the log cleaner keeps of a producer's last marker, marks nothing, and a marker whose key is not an int16 version and an int16 type, or whose version is negative, is refused with a QueryError naming its offset, never read as no marker.
• A result the provider bounded reports wasLimited: true with hasMore: false on its pagination: no offset can page a Kafka read.

5.3 Decoding

Applied in order to the key, to the value and to each header value; the first rule that matches wins.

1. null stays null, encoding null.
2. At least 5 bytes whose byte 0 is 0x00 is the Confluent wire format (magic byte 0, then a 4-byte big-endian schema id): the cell is schema id <N>, not decoded, encoding confluent.
3. Valid UTF-8 that JSON.parse accepts as an object or an array, and whose parse shows every value sent: the parsed value, encoding json.
4. Other valid UTF-8: the string, encoding text.
5. Anything else: base64 of the value's first bytes, as many whole 3-byte groups as fit in the cell limit beside the suffix, written <base64> (<n> bytes), encoding base64; the byte length is always in the cell, and a cut value ends on a whole group.

An integer past Number.MAX_SAFE_INTEGER either side of zero, which JSON.parse would round, is quoted before the parse, the rule of docs/ADDING_A_PROVIDER.md for 64-bit integers, so a 64-bit id keeps every digit, as a string. A document that repeats a member name, which JSON.parse answers with its last value alone, or that holds a number no double shows as sent, is answered by rule 4 as its text, exactly as sent: a number past the double's range, which the query route would send as null, one that underflows or loses digits, and -0, which it would send as 0. Another spelling of the same number, such as 1.10 or 1e2, is the same value, and so is another member order, since a JSON object is unordered. Rule 2 comes before rule 3 because a Confluent-framed payload can itself be valid UTF-8, so a text value that happens to start with 0x00 is labelled confluent: a false positive, stated here rather than guessed around. No Schema Registry is read and no Avro, Protobuf or JSON Schema payload is decoded; a framed value is labelled with its schema id, never guessed (docs/BACKLOG.md). UTF-8 is checked with a fatal decoder, so invalid bytes are never shown as a replacement character, and with the byte order mark kept, so a value is shown as sent and two header names differing only by the mark stay two keys. A value longer than four times KAFKA_CELL_LIMIT is judged on its first bytes only: rule 3 never applies to it, rule 4 checks that prefix, and rule 5 encodes a prefix, so no whole large value is turned into text before it is cut. A header name is decoded as text, or as base64 when it is not valid UTF-8, never as JSON and never as a Confluent frame, so two names that look like JSON never collapse into one. A cell whose text exceeds KAFKA_CELL_LIMIT is cut, never inside a UTF-16 surrogate pair, and the result carries a warning naming how many cells were cut and the limit.

5.4 Bounds

A fetch's maxBytes does not bound its answer: the broker always returns the first batch whole, and with maxBytes: 1024 it returned a 900,000-byte record. So the bounds are the provider's own, and each sets wasLimited when it cuts a read; the budget, the cell limit and a fetch that makes no progress also carry a warning naming what was cut.

• The row limit: a partition stops at limit of its records or at its end, and the result holds at most limit rows across partitions; a limit that leaves records unread sets wasLimited with no warning, because asking for limit rows is asking for that cut. A "latest" window that starts above its partition's earliest offset is such a cut, since the limit placed its start, so it sets wasLimited even when a transactional topic's markers leave its offsets fewer than limit rows.
• The result budget, KAFKA_RESULT_BYTE_BUDGET, counts the record bytes of the rows the result holds, never a row the merge dropped: at every point of the read the result holds only the rows it would answer if the read ended there, a row that falls out of them is dropped as it falls out, and a record that sorts past them is never shaped. Each record is shaped once, as it arrives, and only its row is kept. The first record is always kept, so a read never answers nothing because one record is larger than the budget; such a record is decoded from its prefix only and cut to the cell limit.
• Where the budget stops a read: the partitions are read one after another, in the order the metadata lists them, each forward from its start, and the budget stops the whole read. So a read the budget stops answers the rows it held when it stopped, from the partitions read before the stop and the first rows of the one it stopped in, and none from a later partition, whose records are never fetched. A "latest" read can therefore miss the newest rows of the topic, even when those rows alone would fit the budget, and one the budget stops in the first partition it reads holds the oldest rows of that partition's window. Its warning names the offset it stopped before and each later partition with offsets left to read, a partition whose window holds only transaction markers included, since only reading it could tell. Answering the newest rows that fit instead would read every partition's window whatever the record size; that trade is recorded in docs/BACKLOG.md.
• No progress: a fetch that makes no progress below a partition's end stops that partition rather than spin, and the result names the partition, the offset it stopped at and its end.
• The cell limit, KAFKA_CELL_LIMIT, bounds each of a row's three decoded cells, the key, the value and the headers (§5.2, §5.3).

Accepted limitation: decompression before any bound (K5). The client decompresses every batch of a fetch answer whole, synchronously and with no output cap, before any provider bound applies, and the broker bounds only the compressed batch (max.message.bytes is "after compression"). So one answer can hold its compressed size times the codec's ratio in the one Studio process every user shares, measured at about 1,029 to 1 for gzip and 32,692 to 1 for zstd, and a hostile broker (§4.4) is bounded by neither, because the client buffers whatever frame length the broker announces. A decompressed-size cap is requested upstream as a draft recorded in docs/BACKLOG.md; docs/SECURITY.md records the limitation beside the advertised-address one.

5.5 Timeout and cancellation

A read runs under the connection's query timeout through an AbortSignal, and every other client call is bounded by the client's connect and request timeouts, set to the same value; a timeout is a TimeoutError. The connect's broker read is the exception: it runs under the query timeout as a whole, the client's retries included, and fails as a ConnectionError with connect-timeout when that runs out (§2.4). Each step of a read also waits on the read's signal, the metadata and offsets it plans from as well as each fetch, so a read its timeout stops is answered at once, whatever it waited on, and makes no further call; a read whose timeout ran out before it started makes none at all. There is no cancelQuery: both routes detect cancellation by presence, a presence-detected method exists only where it does its job, and a read is bounded by its limit, its budgets and its timeout, with no long server-side evaluation to stop.

5.6 EXPLAIN

None: supportsExplain is false, and the plan view is not offered.

---

5.7 SQL grammar

How this engine's statement text is read is declared once, in src/lib/db/providers/stream/kafka/sql-grammar.ts, the kafka row of the SQL grammar registry in src/lib/db/sql-grammar/. readsSqlText is false: kafka takes a JSON read request (#1088), one object naming a topic and where to read it from, whose strings escape with a backslash as a MongoDB document's do; the provider extends BaseDatabaseProvider and parses the text itself. The confirmation gate therefore answers from this engine's row of NON_SQL_DESTRUCTIVE_VOCABULARY (src/lib/db/providers/stream/kafka/destructive.ts, collected in src/lib/db/destructive-commands.ts) instead of a SQL reading. Its grammar is DEFAULT_SQL_GRAMMAR, the compatibility default, since no SQL grammar of its own applies to this text. Redaction is none: a redacted statement is stored as ?, because a reader guessing where a literal ends in a language it does not read can store the literal.

6. Schema introspection

6.1 The object surface (#789)

Kinds, folders and identity

No container level: the three kind folders hang directly under the connection row.

Kind	Role	Path	Row name	Columns	Source	status
topic	relation	[topic]	the topic name	yes, fixed	partitions and configs	offline if any partition has no leader, else under-replicated if any partition's in-sync replicas are fewer than its replicas
consumer_group	group	[groupId]	the group id	no	the group and its lag	never
broker	config	[nodeId]	<nodeId> <host>:<port>	no	the broker's configs	never

• Partitions are not a kind: the tree cannot nest children under an object, so a partition kind would be one flat folder holding every partition of every topic; partitions live in the topic's source, and a partition worth acting on surfaces as the topic's status.
• Lag is not a status: any threshold would be the product's invention, so lag lives in the group's source.
• Internal topics (__consumer_offsets, __transaction_state, __share_group_state, the set Apache Kafka's own Topic.isInternal reads) are neither listed nor readable. The listing leaves them out by name, even where the broker lists one as an ordinary topic: a broker before Apache Kafka 3.9 marks only the first two internal and lets a user create a __share_group_state topic, which it lists as ordinary, and such a topic is left out and refused here like the internal one. The client's metadata cache answers their names with nothing, and its listOffsets on one throws inside its socket handler. A read, a description or a source that names one is refused by that name before any request names it, because a broker before Apache Kafka 2.8 creates a missing internal topic for any Metadata request that names it, whatever allowAutoTopicCreation and auto.create.topics.enable say. The refusal is a QueryError such as: Topic "__consumer_offsets" is internal to Kafka, and the client this provider uses drops internal topics from its metadata, so it is not readable here. A topic the broker marks internal that this set does not hold, which the client's metadata answers with nothing too, is refused in the same words.
• A partition with no leader is answered by Kafka with LEADER_NOT_AVAILABLE or LISTENER_NOT_FOUND, and the client throws on any partition error while keeping the whole answer on its error, so the adapter reads the metadata and the topic listing from that answer: the tree lists the topic offline, and the counts, the overview, health and storage keep working. The client reads a topic's offsets as a whole, so a read of that topic is refused with a QueryError naming the leaderless partitions, and its source shows its partitions without offsets and says why.
• The Brokers folder lists the live brokers only, because the Metadata answer names no broker that is down, and marks no controller: on KRaft the answer's controller id is a random live broker, and with dedicated controllers the controller is never a listed broker.
• A group's identity is its id, unique per cluster across both protocols, and a broker's is its node id.

Topic columns

A topic's columns are fixed, because they are the shape of a read result and not a schema the broker holds: partition, offset, timestamp, key, key_encoding, value, value_encoding, headers. timestamp, key and value are declared nullable, where a read answers null: a record written with the protocol's no-timestamp value has no timestamp, a record may carry no key, and a tombstone carries no value. describeObject([topic], "topic") answers them after confirming the topic exists, and describeObjects([], "topic", limit) answers them for every listed topic from the listing read alone, one round trip.

Listing, counting and scale

One reader per kind feeds both countObjects and listObjects, so the count is the listed length. Topics come from one listing, capped at KAFKA_TOPIC_LIST_CAP: past it the Topics count is a floor, which the tree badges 2,000+ from "one topic listing capped at 2,000 names", and the listing holds the first 2,000 in name order. Past the cap plan mode grounds topics only, because the grounding walk stops at the first truncated batch, and with more consumer groups than the object budget leaves after the topics it reaches no broker (docs/BACKLOG.md B89). A topic row carries no row count: latest minus earliest summed over partitions is an offset span, not a message count, because compaction, transaction markers and retention leave gaps, and publishing it as a count is the defect #424 records for Citus and TimescaleDB; the span appears only in the topic's source, labelled as such. Listings are filtered by the principal's Describe ACL: a principal with no ACL gets an empty topic listing and an empty group listing with no error, so its tree is indistinguishable from an empty cluster, and the protocol gives the provider no way to tell the two apart.

Consumer groups and lag

Groups are listed with both types, classic and the Kafka 4 consumer protocol (KIP-848), because a listing with no type filter omits KIP-848 groups. The listing keeps consumer groups only, by Kafka's own rule, the one kafka-consumer-groups.sh --list applies: a consumer group, or a classic group whose protocol type is consumer or empty. Kafka Connect and Schema Registry coordinate through classic groups that are not consumer groups; they are not listed, because a consumer group is what this reading lists and they have no committed offsets to show lag for. A broker below ListGroups v5, such as Apache Kafka before 3.8, reports no group type, so its groups are listed as classic, and a Kafka 3.7 broker with the early-access KIP-848 protocol switched on can hold consumer-protocol groups that this reading cannot tell apart. A classic group is described by describeGroups, and a consumer group by ConsumerGroupDescribe (API 69), because describeGroups reports a KIP-848 group as Dead while the broker says Empty, and API 69 refuses a classic one. An error API 69 answers for the group is read by its protocol error name, as §10 reads any other and never by the broker's text: a refused group or topic is an AuthenticationError naming it, a coordinator that moved or is still loading a QueryError naming the error, and GROUP_ID_NOT_FOUND, for an id with no consumer-protocol group, a QueryError saying so. A group's existence is decided by the listing, because describeGroups answers Dead for a name that does not exist. The kind declares countIsListing, since its count is its listing's length, so on an agent-mode handle, where groups are not read, the refused count carries its reason into the grounding walk rather than an empty folder (§15.3). Lag per partition is the high watermark minus the committed offset, the log end kafka-consumer-groups.sh --describe measures against, for every partition of each topic the group has committed on or is assigned. A partition with no committed offset shows lag as null with the note "no committed offset", never 0 and never the whole log, and a committed or assigned topic whose latest offsets cannot be read, an internal topic or one with a leaderless partition, keeps its rows with no latest offset and the reason. The broker leaves every topic the principal may not describe out of a group's committed offsets, with no error, and the protocol gives the provider no way to see it: such a topic the group only committed on has no rows, and one a member of a classic group is assigned keeps its rows with no committed or latest offset and the broker's refusal as the reason. A consumer-protocol group with a member assigned such a topic is refused its whole description, which its source shows as that refusal (§6.2). A classic group with a member whose subscription or assignment the client cannot decode is refused its whole description, a QueryError naming the group, because the client then answers only the members before that one and the list would read as complete. The listing, every other group and every topic are unaffected.

6.2 Object source (#789)

Every source is JSON, serialised with JSON.stringify, language: "json", form: "complete", origin: "rendered", whose shipped caption, "A structured definition, rendered here as JSON", is true of it. Server text is data: a config value, a header value or a group member's client id is shown exactly as the broker sent it and cannot forge structure.

Kind	Reads	Parts
topic	the topic's metadata, its earliest offsets and high watermarks, and its configs	Partitions: per partition leader, leaderEpoch, replicas, isr, offlineReplicas, earliestOffset, latestOffset and offsetSpan, "latest minus earliest, not a message count"; a partition an offsets answer leaves out shows null with a note, never 0; on a topic with a leaderless partition the offsets are not read, and the part says why. Configs: the entries whose source is not the default, each with its value, its source in Kafka's words and readOnly
consumer_group	the listing entry, the description, the committed offsets and the high watermarks	Group: its type, state, protocol or assignor, and members with client id, host and assignment. Committed offsets and lag: per topic and partition the committed offset, the latest offset and the lag
broker	the broker's configs	Every config with its value, its source and readOnly; an entry the broker marks sensitive answers null from the broker itself and is shown as "redacted by the broker"

A name that does not exist answers a QueryError naming it, and an internal topic's name answers the refusal of §6.1.

A part whose own read the broker refuses is that refusal, and the parts that were read are kept: a refusal and an absence are different answers (docs/ADDING_A_PROVIDER.md), and DescribeConfigs is a right of its own, apart from Describe. So a principal without DescribeConfigs on a topic sees its partitions and offsets beside a configs part that carries the refusal and no text, and one without DescribeConfigs on the cluster, which kafka-auth's reader lacks, sees a broker's source as that one refused part. A topic whose offsets are refused shows its partitions without offsets and says why, as a topic with a leaderless partition does. A group's two parts rest on two reads, its description and its committed offsets, and the broker can refuse either for a group it lists: ConsumerGroupDescribe refuses a consumer-protocol group whole while a member holds a topic the principal may not describe, and a principal that lists groups by its Describe on the cluster alone is refused both reads of every group. A refused description is the group part's refusal, and the lag part then holds the rows of the committed offsets alone, since no assignment was read, and its label says so. Refused committed offsets are the lag part's refusal, and no high watermark is read for it, since rows with no committed offset read would call every assigned partition uncommitted. An assigned topic whose high watermark is refused keeps its rows with no committed or latest offset and the refusal as the reason (§6.1). A refusal of the read that decides the object exists, the topic's metadata, the broker listing or the group listing, still fails the source, and so does any failure that is not a refusal.

6.3 Object edit (#789)

Nothing to write, and the absence is the product's, not the engine's. Kafka has writes: a producer appends records, and the admin API creates and deletes topics, changes configs and resets a group's offsets. This product declines them in v1 by decision (#1088, section 2), so no kind declares acceptsRowWrites or acceptsSourceEdits, and the provider never produces, commits an offset, joins a group or creates a topic. The absence is kafka's entry in EXPECTED_EDIT_ABSTAINERS (tests/helpers/object-edit-expectation.ts), and tests/isolated/object-edit-declarations.test.ts is what holds that entry and this section together.

---

7. Monitoring & health

Each surface reads the Kafka protocol only, and a surface that cannot give an honest number is left empty rather than filled.

Method	Source	Notes
getHealth()	a metadata read of the brokers, forced past the client's cache, then the log directories	The forced read must succeed, or the mapped error is thrown; databaseSize is the overview's figure, the cache hit ratio "N/A", and no active connection count, because Kafka does not report one and absence differs from zero
getOverview()	the forced broker read, the topic listing, the configs of the lowest listed node id, and the log directories	tableCount is the exact topic count, internal topics excluded; maxConnections is that broker's max.connections, 0 ("no limit published") when the value is withheld or the configs are refused, and a value that is not a whole number is refused as a protocol error; databaseSize is the log-directory sum over every broker, labelled "on disk, all replicas, internal topics excluded", or "N/A" when that read is refused; version and uptime are "N/A", because the protocol reports neither a product version nor a start time
getStorageStats()	the log directories	One row per broker and log directory, broker <id>: <path>, with the summed partition sizes of the listed topics and a usage percentage where the broker reports its totals; [] when the read is refused
getTableStats()	none	[]: TableStats.rowCount is a required number, and a topic has no honest message count
getSlowQueries(), getActiveSessions()	none	[], with the empty states "Kafka exposes no query log" and "Kafka does not report client sessions over its protocol"
getIndexStats()	none	[]: a topic has no index
getPerformanceMetrics()	none	Every metric absent; the panel renders unavailable
getPoolStats()	not implemented	The route detects it by presence and shows its fallback

A principal without Describe on the cluster is refused the log directories, and one without DescribeConfigs on the cluster the broker configs, and each refusal degrades only its own figure; any other failure, and a refusal of any other read, still fails the panel. The operations agent workflow reads these surfaces on every engine in both modes, so agent runs reach the broker through them too.

On a handle opened for an agent-mode profile, which the operations workflow, plan-mode grounding and MCP open, every surface reads through the agent's client, which holds the Metadata and ListOffsets reads alone (§15), so no surface reads a config or a log directory there:

Method	On an agent-mode handle
getHealth()	the forced broker read alone; databaseSize is the sentence that the size on disk needs the log directories, which agent mode does not read, never a figure or "N/A"
getOverview(), getStorageStats()	refused with that reason: the overview needs the broker configs and the log directories, storage the log directories
getTableStats(), getIndexStats(), getSlowQueries(), getActiveSessions()	refused with the reason Kafka has no such figure, rather than answered [], so the model reads that the reading was not taken, never an empty reading as the broker's answer
getPerformanceMetrics()	unchanged, every metric absent; the agent's operations readings do not read it

---

8. Maintenance

None. supportsMaintenance is false and maintenanceOperations is empty, so maintenanceControl() places no control, and the required maintenance labels, worded true for Kafka, are never rendered. A direct call of runMaintenance is refused with a QueryError.

---

9. Capabilities & labels

getCapabilities() (index.ts)

• queryLanguage: "json", queryDialect: "kafka".
• writeReach, which declares the editor absent with the API key set KAFKA_READ_APIS in platformatic-client.ts as its proof, and object edit, maintenance and transactions absent (§11.5).
• createTable: absent; the provider declares no createTable.
• supportsExplain, supportsTransactions, supportsMaintenance, supportsInlineRowEdit, supportsTestDataGeneration, supportsResultPagination, supportsExternalQueryLimiting, supportsConnectionString: all false.
• tablesAreDerivedGroupings: false: a topic is a relation the broker holds, unlike a Redis key prefix.
• declaresForeignKeys: false, statementTerminator: "none", defaultPort: 9092, containerLevels: [], maintenanceOperations: [], and the three object kinds of §6.1.
• metadataOnly: the four agent reads' columns and kinds (§15.3); a record's key, value, headers and timestamp are stored values, so no agent answer holds one.
• schemaRefreshPattern: "(?!)", which matches nothing: no read request changes what the tree shows, and the base class's SQL pattern would reload the whole tree after a read of a topic named like orders-drop.
• No keyScan: a topic listing is a catalog read, so there is no key browser and no Browse Keys.

getLabels() (index.ts)

entityName "Topic" and "Topics", rowName "Message" and "Messages", selectAction "Read Latest 50", generateAction "Generate Read Request", and the two empty-state sentences of §7. statementLanguage carries the read request's schema, because plan mode states it verbatim and it is the only fact about how a statement is written that plan mode's prompt carries per engine: the four keys, the from forms, the defaults, two examples, and that no other key is taken, that offset and timestamp go inside from, that the inventory's other columns are fields each message comes back with and not keys of the request, and that a read request reads one topic's messages, never a group's lag. The integration test parses each example in it with the provider's own parser.

SQL writing

kafka/sql-writing.ts declares how a generated statement spells a literal, a name and a bind for this engine, and it is this engine's row of SQL_WRITING (src/lib/db/sql-writing/index.ts), which the readers in src/lib/sql/values.ts and src/lib/sql/identifier.ts look up. A literal is written by literalStandard and read back by unquoteStandard: the quote is doubled and a backslash is data. A name is quoted by identifierDoubleQuoted, the SQL standard form: a double-quoted name with an inner double quote doubled. There is no placeholder (placeholder: null), so a generator quotes the value with quoteLiteral instead of emitting a bind. This engine declares queryLanguage: "json", so no statement is ever built for it to read; what a generator emits for such a connection is portable SQL meant to run elsewhere, and the standard form is the only thing it can claim. Inline row editing reads a key column's kind through the shared INSTANT_TYPE_NAMES, FLOAT64_TYPE_NAMES and EXACT_DECIMAL_TYPE_NAMES of the module's inlineEdit, and binds every edit as typed.

The module's export field writes a result export's CREATE TABLE and INSERT from the shared building blocks alone: STANDARD_DDL_TYPES for the column types, NOTHING_STANDS_ALONE and NO_BARE_SPELLING, so every bare declared name is re-spelled from its family, NULL_NON_FINITE for NaN and the infinities, and binaryStandardHex for a binary cell (X'0102'). The engine declares queryLanguage: "json", so no SQL statement is ever built for it to read, and a file exported from it is meant to run somewhere else: every bare name in it is re-spelled portably rather than kept as one engine's private word.

---

10. Error handling

Mapped from the protocol error name, the client's error code and the Node error code, never from text a broker wrote; where the client gives no code that tells two failures apart, the adapter reads the client's own fixed text, and live tests against local listeners catch a client upgrade that rewords it.

Condition	Class
host, port, credential, SASL mechanism or TLS setting, SASL over plaintext, a credential with no mechanism, an SSH tunnel, bound params	DatabaseConfigError, never echoing a value
a package the client library requires that the installation does not resolve, at connect (§2.5)	DatabaseConfigError naming the package, never the runtime's text, which carries the server's paths
empty editor text	QueryError
a read request that is invalid JSON or breaks the schema of §5.1	QueryError
a partition the topic does not have	QueryError naming the partitions the topic has
an unknown topic ("Unknown topic .")	QueryError: the topic does not exist
an internal topic, refused by name before any request names it	QueryError: internal to Kafka and not readable here
a classic group whose member's subscription or assignment the client cannot decode	QueryError naming the group: the member list would be incomplete (§6.1)
a partition with no leader	QueryError naming the leaderless partitions
OFFSET_OUT_OF_RANGE on a fetch	QueryError naming the partition, the offset and the range read again once after the refusal, or the offsets answer read again that left the partition out (§5.1)
a broker whose Fetch range does not hold version 13	QueryError naming the broker's range and the version a read sends
NOT_LEADER_OR_FOLLOWER on a fetch already retried once against the leader read again	QueryError: leadership moved during the read, run it again
an answer the client's parser cannot read, such as a batch that will not decompress	QueryError naming the parser's code, such as Z_DATA_ERROR
TOPIC_AUTHORIZATION_FAILED, GROUP_AUTHORIZATION_FAILED, CLUSTER_AUTHORIZATION_FAILED	AuthenticationError naming the resource type, never a credential
SASL authentication failure	AuthenticationError, never echoing the credential
TLS handshake or verification failure	ConnectionError carrying the Node error code
a bootstrap address that refuses, does not answer the connect in time, or closes the connection	ConnectionError carrying the Node error code, or connect-timeout or connection-lost where there is none
a connect() whose broker read, retries included, has not settled when the query timeout runs out (§2.4)	ConnectionError with connect-timeout
an advertised broker address Studio cannot reach after the bootstrap succeeded	ConnectionError naming the advertised host and port (§4.4)
a request the broker accepted and did not answer in time, and a read past the query timeout	TimeoutError
any other protocol error	QueryError naming the protocol error name, or the client's code where there is none, never the client's message, which can carry a host and port

Which Node code a failure carries is the runtime's: TLS to a listener that answers in plaintext gave ERR_SSL_WRONG_VERSION_NUMBER under Bun and ERR_SSL_PACKET_LENGTH_TOO_LONG under Node, and an address with no route ECONNREFUSED under Bun and ENETUNREACH under Node, so each row reads a code by its form. An error that did not come from the client, such as a defect in the provider's own mapping, is rethrown untouched, never dressed up as a broker's answer.

---

11. Testing

11.1 How the tests work

File	Owns
tests/integration/db/kafka-provider.test.ts	The provider end to end over the real adapter, with the recorded library of tests/helpers/kafka-fixtures.ts injected through the constructor's createClient parameter, so only the broker is fake; the answers are the captures of tests/fixtures/kafka/
tests/unit/db/kafka/	One file per module: client, connection-options, request, decode, results, read (the fetch loop against a reference over generated logs), groups, objects, monitoring, errors, platformatic-client (the adapter over the recorded library, and sessionless fetches against a local broker), provider (the composition over a fake KafkaReadClient), provider-library-load (a client library the installation cannot load, over the runtimes' own resolution errors), tls-handshake (real handshakes, and live transport failures against local listeners), seam-guard (K4), write-reach (the client's API key set against the real library, §11.5), dependency-resolution and provider-doc (this document against the code)

No mock.module() anywhere in this suite: it is process-wide in Bun. The captured answers live in tests/fixtures/kafka/, and its README says how each was taken, with the image digests and cluster ids. TLS certificates are generated with openssl at test time into a temporary directory and never committed, and the TLS tests need node on the path: the server-name rule runs the library under Node in a child process.

11.2 Run it

# Just this provider, one process per file
bun tests/run-tests.ts tests/unit/db/kafka/request.test.ts tests/unit/db/kafka/read.test.ts tests/integration/db/kafka-provider.test.ts

# Everything, CI-equivalent
bun run test

11.3 The live fixtures

docker compose -f database-compose.yml up -d kafka kafka-init
bun docker/kafka/seed-binary.ts localhost:9092
docker compose -f database-compose.yml --profile kafka-cluster up -d
docker compose -f database-compose.yml --profile kafka-auth up -d kafka-auth

kafka (port 9092) is a single combined KRaft node, seeded by docker/kafka/seed.sh from its kafka-init sidecar and by docker/kafka/seed-binary.ts from the host: a 3-partition orders topic with keys, headers and JSON values, one topic per codec (codec-gzip, codec-snappy, codec-lz4, codec-zstd), a 900 KB record in big, a non-UTF-8 value and a Confluent-framed value in bytes, a LogAppendTime topic ts-append, a committed and an aborted transaction in txn, and the groups lag-classic, lag-kip848 and lag-partial, the last with a partition that has no committed offset. kafka-cluster is three nodes on ports 9192 to 9194 with every topic at replication factor 3, where several brokers and under-replicated are seen. kafka-auth (port 19094) is TLS with SCRAM-SHA-512 and an authorizer; its principal reader is created after start with a password of your choice and no ACL, for the permission and plaintext refusals; the compose file's comment gives the commands.

11.4 The live read-only check

tests/live/kafka-read-only.ts runs a real KafkaProvider over every surface against the fixtures above and fails on any change to the broker's state; the runner skips tests/live/, so it runs by hand.

bun tests/live/kafka-read-only.ts --tripwire        # recreates and re-seeds kafka first
bun tests/live/kafka-read-only.ts --redpanda
bun tests/live/kafka-read-only.ts --bootstrap localhost:9192
bun tests/live/kafka-read-only.ts --bootstrap localhost:9192 --failover

It snapshots the broker before and after the run: the topic and group lists, every group's committed offsets, every partition's earliest and latest offsets, the topic and broker configs, and on kafka-auth the ACLs, through the broker's own tools (rpk on Redpanda). Each line of the snapshot carries the tool that printed it and the block it sits under, such as All configs for topic orders are:, before the lines are sorted, and the two snapshots must hold the same lines the same number of times: twelve topics print the same default config line, so a line compared without its topic could change hands unseen. Between the two it runs the counts, the three listings and one source of each kind, a read in every from form and of each seeded topic, the refusals of a missing topic, an out-of-range offset and __consumer_offsets, the three panels, the group listing and each group's lag against the broker's own CLI, two metadata([]) calls counted as two Metadata frames at a forwarder, a bootstrap at ::1, a read through a bootstrap forwarder that sends no Fetch frame through it, three reads at once, and the process's sockets after disconnect(). With --tripwire it first reads every seeded topic on a broker never asked for a group coordinator and checks that __consumer_offsets is still absent. Its log searches for an automatic topic creation and for the never-joined group id are each paired with a control that finds that kind of line in the broker's whole log.

Measured on 2026-09-25: every check passed on kafka (57, with the tripwire), on redpanda (39), on kafka-cluster (47), and on kafka-auth as reader (7), and every snapshot after a run equalled the one before it. Measured again on 2026-09-28 with @platformatic/kafka 2.12.0: the same four counts, eight of eight with --failover, and every snapshot after a run equalled the one before it. The same day, under Node through next start, a connect, the topic listing and two reads on a broker never asked for a group coordinator left __consumer_offsets absent, and the group seed that followed created it. Measured again on 2026-10-05 with @platformatic/kafka 2.13.0: the same four counts, eight of eight with --failover, and every snapshot after a run equalled the one before it. Measured again on 2026-10-26 with @platformatic/kafka 2.14.0: the same four counts, eight of eight with --failover, and every snapshot after a run equalled the one before it. Against the provider with one rule broken at a time, the check failed each time: a disconnect() that closes nothing, autocreateTopics: true, a metadata([]) answered from the cache, lag that ignores the committed offset, a transaction filter that keeps aborted records, a group listing without the consumer type, and a lag listing that shows one partition twice. A config write made during the run from outside the provider, setting orders to the compression.type=gzip line codec-gzip already holds, failed the check too; the check as first committed, which compared bare lines as a set, passed it.

Redpanda

Redpanda v26.2.2 is a full relative of this provider (README): every surface answered, with data wherever Kafka held data, and the broker's state was unchanged by the run. Measured again on 2026-09-27: every check passed on redpanda, and the broker's state was unchanged. As on Kafka, Studio never produces, commits an offset, joins a group or creates a topic there (§3.2).

A Redpanda cluster is a connection of type Apache Kafka: once that type is chosen, the dialog names Redpanda as a verified relative. The host and port are those of a Kafka listener Redpanda advertises to where Studio runs, such as the fixture's external listener on 29092, because the client connects next to the advertised address, not to the one typed (§4.4). SASL and TLS take the fields of §4.1 to §4.3; the runs above used neither, and SCRAM over TLS was measured on Apache Kafka's kafka-auth fixture, not on Redpanda. Nothing was measured on Redpanda Cloud (Serverless, BYOC or Dedicated), so no claim here covers it.

Measured in a browser against the published @libredb/studio@0.17.0 on 2026-09-27: the tree listed the topics, consumer groups and brokers, a topic click read its latest records, and a group's committed offsets and lag equalled the broker's own rpk.

Four things read differently from Kafka. Max connections reads 0, no limit published, because Redpanda's DescribeConfigs answer for a broker holds no max.connections. That answer holds nine entries where Kafka 4.3.1's holds 340, so a broker's source is short. The Storage panel shows no usage percentage, because its DescribeLogDirs answer carries no total or usable bytes. Every group reads as classic, which is right there: Redpanda answers ListGroups up to v4, which carries no group type, and has no KIP-848 consumer protocol.

Result size (KM2)

Measured in a browser against bun dev on 2026-09-25: {"topic":"big","from":"earliest","limit":1} answered one row whose value was cut at KAFKA_CELL_LIMIT, 65,984 bytes on the wire; {"topic":"orders","from":"earliest","limit":500}, over records of about 2 KB each, answered 500 rows in 966,938 bytes, rendered 67 ms after the response arrived, and the grid scrolled at a mean of 47 ms a step. The grid stayed responsive for both, so KAFKA_RESULT_BYTE_BUDGET and KAFKA_CELL_LIMIT keep the values of §3.6.

Several brokers, under-replication and offline partitions (KM8)

On kafka-cluster the Brokers folder lists nodes 1, 2 and 3, each partition of orders is read from its own leader, and lag on the three groups equals kafka-consumer-groups.sh --describe. With node 3 stopped, orders reads as under-replicated, the Brokers folder lists nodes 1 and 2, every partition of orders still reads, including one whose leader moved off node 3, and the three panels answer; after the restart every topic is ok again, and the broker's state before the stop equalled its state after the restart. offline was reached on a throwaway single node with two log directories whose second directory was made unreadable: the partitions on it went offline within about 10 seconds. There the Topics listing shows the topic offline, its source shows its partitions with no offsets, a read of it is refused naming the leaderless partitions, and a read of a healthy topic still answers. The three panels do not answer there: the broker answers DescribeLogDirs for the failed directory with KAFKA_STORAGE_ERROR, and the overview, health and storage fail with that error (docs/BACKLOG.md D125).

11.5 Write reach, measured live

Kafka's write reach is none: every request the client sends is one of the read APIs of the frozen table KAFKA_READ_APIS in platformatic-client.ts, and none of them produces, creates or deletes a topic or records, alters a config, commits or deletes an offset, joins, syncs, heartbeats or leaves a group, or starts a producer or a transaction. write-reach.test.ts holds that four ways. The table itself: frozen, every entry a read API under its own protocol key, and none of the protocol's write, group-membership or admin APIs. The real @platformatic/kafka, loaded through loadPlatformatic(), driven through every method of the client against a local listener that records the API key of every request frame: the keys recorded are exactly the table's, every Metadata request carries allow_auto_topic_creation false, and each method that names a topic refuses __consumer_offsets, __transaction_state and __share_group_state before any frame is sent. A parse of platformatic-client.ts that finds no call on a library object outside a map of the read calls to the table's APIs, no raw protocol call but the Fetch and ConsumerGroupDescribe ones, and no library client but the four it builds, so a direct call on one of the adapter's named library objects in a branch no run drives fails it too. An alias or a cast of a library object is outside that parse, and the seam guard, seam-guard.test.ts, refuses it. And the replay of the live capture below. The integration test opens a readOnly: true connection, which the factory accepts, drives every surface over the captured broker, and holds each library call to those read calls.

The live run is tests/fixtures/kafka/4.3.1/write-reach/run.sh: a fresh apache/kafka:4.3.1 by digest, one KRaft node with no authentication, no authorizer and auto.create.topics.enable=true, so any write Studio sent would land, seeded with a three-partition topic and a group that committed part of it. tests/live/kafka-write-reach.ts takes the broker snapshot of §11.4 (the topic and group lists, every group's committed offsets, every partition's earliest and latest offsets, and the topic and broker configs) before and after every surface of a read-only connection, of the agent read-only profile and of the agent operations profile, each of which also reads a topic no one created. The snapshot does not see a group joined and left with nothing committed, a heartbeat, a producer id, or a transaction begun and aborted, which leave it unchanged. Those are shown absent by the recorded request list instead, sentByStudio, which holds the API key and version of every request the library sent in the run, read at its one request writer, Connection.prototype.send. Measured 2026-11-03: the snapshot was unchanged across every handle and the missing topic was never created; sentByStudio held Fetch, ListOffsets, Metadata, OffsetFetch, FindCoordinator, DescribeGroups, ListGroups, ApiVersions, DescribeConfigs and DescribeLogDirs, all in the table, the fixture holding no consumer-protocol group or SASL listener for the other three; the script's own produce, group commit and topic creation afterwards moved the topic list, the end offsets and the committed offsets, which is the control that a write would have been seen. The test replays that capture.

---

12. Usage examples

12.1 Programmatic (via the factory)

import { createDatabaseProvider } from "@/lib/db/factory";

const provider = await createDatabaseProvider({
  id: "events",
  name: "Kafka",
  type: "kafka",
  host: "kafka.internal",
  port: 9093,
  saslMechanism: "SCRAM-SHA-512",
  user: "reader",
  password: process.env.KAFKA_PASSWORD,
  ssl: { mode: "verify-full" },
});

await provider.connect();

const result = await provider.query('{"topic": "orders", "from": "latest", "limit": 50}');
console.log(result.fields, result.rows.length);

const groups = await provider.listObjects([], "consumer_group");
await provider.disconnect();

12.2 Over the API

curl -X POST http://localhost:3000/api/db/query \
  -H 'Content-Type: application/json' \
  -H 'Cookie: auth-token=<jwt>' \
  -d '{"connectionId":"seed:events-kafka","sql":"{\"topic\": \"orders\", \"from\": \"earliest\", \"limit\": 10}"}'

seed:events-kafka is the Kafka seed example of docs/SEED_CONNECTIONS.md; an inline connection object works as well. See docs/API_DOCS.md for the full request and response contract.

---

13. Known limitations & future work

• The broker chooses further hosts, with the connection's credentials (§4.4), and a Kafka connection takes no SSH tunnel (§4.5).
• The client decompresses a whole answer before any bound applies (§5.4).
• The broker chooses how much work a SCRAM exchange takes (§4.2).
• The broker chooses how often a connection authenticates again (§4.2).
• A malformed answer can end a host process with no uncaughtException handler (§3.5).
• A read the budget stops answers older rows than the newest that would fit (§5.4).
• Past KAFKA_TOPIC_LIST_CAP topics, plan mode grounds topics only (docs/BACKLOG.md B89).
• A principal with no Describe ACL sees an empty cluster, with no error to tell it apart (§6.1).
• No OAUTHBEARER, GSSAPI or AWS MSK IAM (§4.2).
• No Schema Registry decoding, so Avro, Protobuf and JSON Schema values show their schema id (§5.3).
• No live tail, no streaming results and no push subscriptions.
• No ksqlDB and no SQL over Kafka: ksqlDB's licence forbids offering it as a competing service, most clusters do not run it, and its queries create server-side groups and topics; SQL over Kafka, if ever wanted, is a separate type-id.
• Agent mode and MCP's run_kafka_read read metadata only, and send no fetch (§15, §16, docs/BACKLOG.md B157): no record, group, config or log directory reaches an agent or MCP answer, and a platform's own topics are withheld.
• A client certificate the broker refuses reads as a lost connection (§4.3).
• A broker with a failed log directory fails the overview, health and storage panels with KAFKA_STORAGE_ERROR, where the rest of the object surface keeps working (§11.4, docs/BACKLOG.md D125).

---

Migration DDL

KAFKA_MIGRATION_DDL is what the schema-diff migration generator writes for kafka, read through migrationDdlFor in src/lib/db/migration-ddl/. It declines the whole diff: refusal is the one line the migration prints after its header, in place of any DDL, and transactionWrapper is null. kafka text is a JSON read request, not SQL, so there is no table DDL to wrap (#284, #1088).

The refusal reason it prints, verbatim:

• "A topic holds messages, not rows with declared columns, so there is no column definition to change."

---

14. References

• Source: src/lib/db/providers/stream/kafka/
• Design: #1088
• Client: @platformatic/kafka, version 2.14.0
• Kafka protocol: https://kafka.apache.org/protocol
• KIP-848, the consumer group protocol: https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol
• KIP-516, topic identifiers: https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers

---

15. Agent mode

Agent mode reads Kafka metadata through queryReadOnly in index.ts. The request reader is agent-statement.ts, the reads and their metadata-only projection are agent-metadata-read.ts, and every other surface of an agent-mode handle is agent-surface.ts. Every claim below about the broker cites a capture under tests/fixtures/kafka/4.3.1/agent-boundary/ or tests/fixtures/redpanda/26.2.2/agent-boundary/, each recorded by its run.sh against a fresh digest-pinned broker with automatic topic creation on, with every refused call sent raw by the broker's own Java tools, past every Studio check, and the broker snapshot of §3.2 taken before and after. On Apache Kafka the raw calls are sent twice: as agent-reader, a SCRAM-SHA-512 user over TLS under StandardAuthorizer that holds Describe on every topic, every group and the cluster and nothing else, and as the plaintext listener's super user, the writable positive control; Redpanda runs with no authentication, so each call is sent once, as that control.

15.1 The boundary

The boundary is a structural closed table: every agent-mode surface reads through KafkaAgentReadClient (client.ts), which picks the client's metadata, listTopics and offsets alone, and the provider hands those surfaces a new object holding those three reads and nothing more. On the wire that is the Metadata request, sent with allowAutoTopicCreation the literal false, and the ListOffsets request at the earliest (-2) and latest (-1) sentinels; agent-boundary.test.ts pins each read's full requests at the library, and the seam guard pins the client type, the agent modules' imports and calls, and that object (§3.2). So no fetch, produce, topic or config change, record deletion, offset commit, group, config, log-directory or ACL read, and no timestamp lookup can be sent, and no request names an internal topic (§3.3).

A Describe ACL is not that boundary, so no credential is required of the agent: measured on 4.3.1 as agent-reader (raw, as: "reader"), the broker refuses a produce, a fetch, a topic creation and deletion, a record deletion, an offset reset and a config change with an authorization error and the snapshot does not move, but it answers a ListOffsets at the max-timestamp sentinel (-3) and by timestamp, which only the closed table keeps off the agent's path. As the super user (as: "super") every one of those calls acts: the produce and the auto-creating producer's Metadata add records and a topic, the record deletion moves a log start, the reset moves a group's committed offset, the config change and the topic creation and deletion all show in the snapshot, and the fetch returns a record; Redpanda 26.2.2 does the same, except that it refuses the max-timestamp sentinel as an unsupported version. The agent's own reads, as agent-reader and as the super user, leave the snapshot unchanged, and a read of a topic that does not exist answers that it does not exist and creates nothing on either broker (agent, missingTopicCreated: false), where the producer's Metadata for another missing name, auto-creation allowed, created it.

15.2 What an agent request may be

One JSON object of at most four keys, read, topic, after and limit, each written once:

read	Takes	Sends
"topics"	after (the last topic name of the answer before), limit	Metadata of every topic, then Metadata of the page's topics
"brokers"	after (the last node id), limit	one brokers-only Metadata
"partitions"	topic, after (the last partition id), limit	one Metadata naming the topic
"offsets"	topic, after, limit	one Metadata naming the topic, then ListOffsets at -2 and at -1

limit is 1 to 500, default 50; one past the run's row budget is refused before anything is sent, and a page the request's own limit leaves unfinished says that more follow and how to read on, without naming what follows. after is compared in Studio against what the broker answered and is never sent. A topic must be a legal Kafka topic name, other than . and .., never one of Kafka's internal topics, and never a platform's own topic (§15.3). Any other key, a timestamp, an offset sentinel, an isolation level, a partition filter, a group or an auto-creation flag above all, any other read, a key written twice, and text that is not one JSON object are refused before anything is sent, each in agent mode's own sentence followed by the list of reads, and no refusal quotes what the request wrote. The admission refuses a request longer than 4,096 characters. Each read runs under the request's time budget, the wait and not the work, and nothing of one call is kept for the next.

15.3 Metadata only

Topic names, partition ids, broker node ids and racks and the cluster id are object names; a record's key, value, headers and timestamp are stored values, and a record's timestamp can be the producer's own, so no read fetches a record or looks anything up by time, and no request can count or filter by a value. A broker's advertised host and port, topic and broker configs, log-directory paths and consumer groups (their ids, members, client ids and hosts, committed offsets and lag) are not read on the agent path either. A platform's own topics are withheld on every agent-mode handle: a name with a leading underscore, as Schema Registry's _schemas, Confluent's _confluent-* and ksqlDB's command topics have, or starting connect-, as Kafka Connect's configs, offsets and status topics have by default (isWithheldKafkaTopic in agent-statement.ts). The broker marks none of them internal, so its listing answers them to any principal that may describe them, measured on 4.3.1 (the control check of tests/fixtures/kafka/4.3.1/mcp-boundary/); they hold connector configs, schemas and offsets rather than the cluster's data. The agent's client drops them from every listing it answers and refuses a request that names one before it is sent, and the reader refuses such a request in agent mode's own sentence; a platform that names its topics otherwise is not recognised, and the principal's own Describe grants are what keep those out. Kafka declares ProviderCapabilities.metadataOnly, and every answer is projected through src/lib/db/redaction/metadata-only.ts:

Read	Columns
topics	topic (name), partitions, replication_factor, offline_partitions, under_replicated_partitions (counts)
brokers	node_id (count), rack and cluster_id (names, null when the broker gives none)
partitions	partition, leader (counts, leader null when there is none), replicas, in_sync_replicas, offline_replicas (node ids as one name, 1,2,3, null when empty), offline (flag)
offsets	partition, earliest_offset, end_offset (the log end), offset_span (counts; a side the broker did not answer is null, never 0)

An offset span is how far a partition's offsets reach, not how many records it holds: compaction, transaction markers and retention make the two differ, and the posture, the tool description and the workflow rules all say so. A cell that does not fit its column, an offset past 2^53 - 1 among them, withholds the whole answer, naming the column and never the value, and a result past the byte budget is refused, never cut. A failure is worded by its category alone, never by the client's message, which can carry an advertised host and port: a missing topic says nothing was created, an authorization refusal, an unreachable broker and an unreadable answer each have their sentence, and a timeout is the time budget's refusal; the captures hold every refusal word for word.

Each agent-mode path reads this much:

Path	Handle	What it reads
Investigation reads	agent read-only profile	one read of §15.2's table, projected as above
Investigation grounding, plan-mode grounding, the operations workflow and MCP inspect_schema	either agent profile	topic names and statuses, broker node ids (MCP inspect_schema lists topics only), and a topic's columns partition, earliest_offset, end_offset and offset_span; consumer groups are counted as withheld with their reason, which the grounding walk carries since the kind declares countIsListing
Operations readings	operations profile	health from one forced Metadata read, its size on disk the reason it is not read; the overview, storage, session, slow-query, table and index readings answer their reason rather than an empty reading (§7)
MCP run_kafka_read	agent read-only profile	one read of §15.2's table, projected as above (§16)

On a handle of either agent profile the editor's read request, which fetches records, and an object's source, which holds configs, group members and committed offsets, are refused; an editor handle keeps every read.

15.4 Agent language pack

Statement-language family: kafkaread. Overlay: src/lib/agent/languages/kafkaread/engines/kafka.ts, with its client row in src/lib/agent/languages/kafkaread/engines/client.ts. The agent reads it through runServerPack and agentEnginePack. Both overlays are empty, so the engine resolves to the kafkaread family pack itself. The kafkaread statement-language family has no dialect-bound member. This overlay overrides no Shared member, so every Shared member is the kafkaread statement-language family's own. The family offers the investigation workflow, through run_read_query with one read request, and the operations workflow; it offers no data analysis, since its answers carry no stored value, and no query optimization or database assessment, since the broker composes no plan and no profile, and no workflow hands an answer to the editor. The workflow classifier offers the model only these two. A pack does not open agent mode: execution is gated by AGENT_EXECUTION_ENGINES and the execution-profile check. MCP clients send the same read requests with run_kafka_read, the tool the family declares in its MCP facet (src/lib/agent/languages/kafkaread/mcp.ts), and MCP_STATEMENT_READ.kafka is true (§16). A fetch joins the agent path only once a decompressed-size cap or an isolated process bounds one (§5.4).

15.5 Tests

• tests/unit/db/kafka/agent-statement.test.ts: the plan every admitted request is read to, and every refusal before the network in agent mode's own words.
• tests/unit/db/kafka/agent-metadata-read.test.ts: each read's calls on the agent's client, its rows, the budgets, the deadline and the wording of every failure category.
• tests/unit/db/kafka/agent-provider.test.ts: the profile, every request refused before anything is sent, and every surface of either agent profile over a client whose every other member throws, with no broker address in any answer.
• tests/unit/db/kafka/agent-boundary.test.ts: each read's full requests at the library, and every agent-mode surface held to Metadata and ListOffsets.
• tests/live/kafka-agent-boundary.ts, through each run.sh: the captures above, whose leaks list of seeded record markers, broker addresses, log-directory roots and group ids found in an agent answer is empty.

16. MCP

MCP clients read a Kafka connection with run_kafka_read, the tool the kafkaread family declares in its MCP facet (src/lib/agent/languages/kafkaread/mcp.ts); MCP_STATEMENT_READ.kafka is true, and list_connections names it as a Kafka connection's read_tool. It takes the agent's own read request as statement, admits it with the agent's own reader, and runs it under the agent read-only profile through queryReadOnly, so every rule of §15 holds for it unchanged and nothing is added or relaxed for MCP. Every claim below cites a check of tests/fixtures/kafka/4.3.1/mcp-boundary/boundary.json, recorded by its run.sh against a fresh digest-pinned Apache Kafka 4.3.1 broker with automatic topic creation on, through the tool's own handler and the factory's own acquisition.

16.1 The boundary

The boundary is §15.1's structural closed table: the agent's client holds the Metadata request, sent with topic auto-creation off, and the ListOffsets request at the earliest and latest sentinels alone. The admission in front of it is defence in depth: with the admission removed, a fetch, a produce, the editor's record read, a timestamp lookup, an auto-creation flag, a group read, an internal topic, a platform topic, a key written twice and a limit past 500 are each refused by the provider before anything is sent, and the broker snapshot (topics, every partition's offsets, the topic configs and the ACLs) does not move across any call. With the admission on, each is refused before the connection is acquired, in the sentence an agent run reads.

16.2 Data scope

Two layers decide what an MCP client may see. The first is the broker's: a topic listing, a partition read and an offset read reach only the topics the seed's login may Describe. Measured as mcp-narrow, a SCRAM user holding Describe on the topics prefixed app- alone, the listing answered app-audit and app-orders, and a partition or offset read of a topic it may not describe answered the same sentence as one of a topic that does not exist (the broker refused the principal the read), so a refusal does not tell a hidden topic from a missing one; the super user and mcp-wide, Describe on every topic and the cluster, listed every data topic. The second is Studio's, independent of the request: the metadata-only projection of §15.3, which never fetches a record and never answers a broker's host and port, and the withholding of Kafka's internal topics and a platform's own topics, which the agent's client applies to every answer whatever the login may describe. Measured as the super user, which may describe every topic, neither run_kafka_read's listing nor inspect_schema's listed _schemas, _confluent-metrics, connect-configs, connect-offsets, connect-status or __consumer_offsets, while an editor's handle listed every platform topic, and no answer held a seeded record key or value, a broker address, the consumer group's id or a platform topic's name.

A Kafka seed needs no least-privilege principal, and none is proven, since the agent's client can send no write whatever the login may do; what MCP lists is every topic the login may Describe, so opt in a login that holds Describe on the topics meant to be listed and nothing on the rest.

16.3 Limits

A read answers at most 500 rows (its limit), below the 1000-row provider budget, so that budget never refuses one; a result past max_rows or the 32 KiB result limit is cut, and the hint says to read on with after, since the tool takes no offset. The provider's note that more rows follow a page its own limit left unfinished is not part of an MCP answer, so the tool's description says that an answer holding as many rows as its limit may have more after it. A deadline ends the wait, not the work: the read runs under the time left of timeout_ms, and a request already sent settles under the client's own timers while its answer is dropped. Measured with the broker paused, a read at the minimum timeout_ms of 500 was answered as the timeout after 502 ms, and a client cancel 300 ms into a call ended the handler's wait at once; through the HTTP route a cancel ends the wait when the client closes its request, as docs/MCP.md says for every engine. The broker answered the next call once it resumed. inspect_schema takes the agent-operations profile and reads through the same client, so its topic listing withholds the same topics.

16.4 Tests

• tests/integration/mcp/run-kafka-read.test.ts: the tool through the real route and handler over the real provider with a fake client whose every member off the agent's path throws: each read, a platform's own topics left out of the listing, no broker address in any answer, every request outside the table refused before anything is acquired, a provider not opened under the profile refused, the paging words and the timeout.
• tests/unit/mcp/statement-arms.test.ts: the arm's input, its admission, its row, byte and cut-result words and its description.
• tests/live/kafka-mcp-boundary.ts, through tests/fixtures/kafka/4.3.1/mcp-boundary/run.sh: the checks above.