opspile is a unified inbox for an ops team. It ingests every channel the team works in on Slack, Microsoft Teams and Discord, keeps the threads in Cloudflare R2, and tracks read/unread state per person and per team, so that you can see which threads somebody else has already seen and you have not.
It follows the spirit of klickhaus: a small static UI in plain JavaScript, fed by separate ingestion services, with the data store doing the heavy lifting.
Everything runs on Cloudflare (Workers, R2, D1, Durable Objects, Workers AI, Vectorize, Artifacts, AI Gateway) in a single account; the thread agents' model is Claude on Amazon Bedrock, reached through AI Gateway.
flowchart LR
S[Slack Events API] --> R
T[Microsoft Graph<br/>change notifications] --> R
D[Discord REST API<br/>polled every minute] --> R
B[Backfill from history APIs] --> R
R[(1. raw events<br/>R2, append-only)] --> C
C[(2. channel folders<br/>R2, one file per message)] --> I
C <--> K[(3. knowledge bases<br/>Artifacts, one repo per org / team)]
I[(4. fast index<br/>D1 + Durable Objects)] --> UI[opspile UI]
UI -- read events --> I
Each pile is derived from the one before it. Pile 2 never talks to the platforms directly, so it, and everything after it, can be rebuilt by replaying pile 1.
One R2 bucket, append-only, holding events exactly as the platform API delivered them. When
Slack or Microsoft add fields, we keep them; normalization happens downstream. The one
exception is credentials: Slack's deprecated verification token and Graph's clientState
are removed before an event is stored.
<source>/<tenant>/<channel>/<yyyy>/<mm>/<dd>/<eventTs>-<eventId>.json
- The key is derived from the platform's event id, so redelivered events (Slack retries with
the same
event_id, Graph may notify twice) overwrite themselves. Ingestion is idempotent. - Slack delivers edits as
message_changed(with the full new message), deletions asmessage_deleted, and reactions as separate events. All of them are stored. A file uploaded to a Slack Connect channel arrives as a stub (file_access: check_file_info: an id and a file type, no name or link); the normalizer looks it up once withfiles.infoand keeps the answer here, next to the channel's events, asslack/<tenant>/<channel>/files/<file id>.json({ type: 'file_info', file }, withoutshares, which quotes other channels), so a replay asks Slack nothing. It is not an event and is skipped. A file Slack does not show the app is not kept, and the message says a file of that type is there. - Teams change notifications usually carry only the resource path. We store the
notification and the message we then fetch as two objects. Graph notifications have no
event id, so a notification is keyed by its own hash (
notification-<sha256>.json), and a fetched message by its version (<lastModified ms>-<messageId>.json). The notification'sresourcenames the message; the message's metadata names the notification. The files of a Teams message are kept here too, as bytes: the images pasted into it (hosted contents, fetched with the app's Graph token) and the files attached to it (SharePoint or OneDrive links, fetched through Graph's/shareswithFiles.Read.All), underteams/<tenant>/<team>/<channel>/files/<sha256 of the URL>with their content type and name (src/teamsfiles.js). The normalizer copies them when it first sees the message; the message keeps the platform's link, and an image attached as a file is shown where Teams shows it. The inbox's media proxy serves the copies, only to those who may see the message's thread (SharePoint files only from there, since the app could read any of them), and agents'read_filereads them. They are not events and are skipped; one Graph does not hand over is not kept, and the file stays a link. - Discord is polled (see Ingestion): one Durable Object (
DiscordPoller) stores each state of a message it reads (new, edited, other reactions) and each deletion it finds as an envelope{ t, via, guild, d, thread?, root?, reactions?, at? }.dis what Discord sent; around it is what only the poller knew at the time: the thread a message was posted in (Discord names just the thread's channel id), the conversation an inline reply belongs to, who reacted (messages carry counts only) and since when the poller has seen each reaction, and when it found a message gone. Keys are<message id>-<hash of the envelope>and<message id>-delete, so the same state read twice shares a key. - Backfill pages from the history APIs land here too, marked
via: backfill(Discord's:via: history).
channels/<org>/<team>/<source>-<channelId>/<rootId>/<messageId>.md
Every channel belongs to one org (a customer, a partner or a vendor) and one team (the
ops team that owns it). The mapping is a small rules engine over rows in D1: each rule matches the
channel id, the channel name, the Teams team name or a Slack Connect team id with a regular
expression, and sets the team, the org or the kind (channel, alert, ignore); the lowest
priority wins. Naming conventions (aem-<customer> exists on both platforms) are rules, and
so are the overrides for vendor channels, alert channels and irregular names. Channels that
no rule matches are filed under _unassigned. A manual assignment of one channel is a
rule too: its channel_id, priority 10.
Shared channels. A channel can be in several teams' inboxes, but it is filed under one team,
its first: the rules pick it, and its folders, thread rows (threads.team), knowledge base and
agents are that team's. The other teams follow it through channel_teams (source, channel id,
team): the channel's threads are in their inboxes, and their members read, reply, react, resolve
and draft there; read state in a thread is that of the team it is looked at from. Adding a team
to a channel moves the channel only for its first team, when the admin team (or nobody) holds
it; added to a channel another team holds, the new team shares it and nothing moves. Handing a
channel back to the admin team is always a move. The shares are configuration, like the rules,
so a replay of pile 1 keeps them.
An org is a slug. What it is called, its icon (a domain, whose favicon is shown, or an image
URL) and whether it is a customer, a partner or a vendor are a row in orgs; an org without
one is a customer shown by its slug and the favicon of its usual mail domain. The inbox and
the agents show the name and the type.
The org list in the admin pages shows each org with the icon the inbox resolves for it (its
own, else the favicon of its usual mail domain: the most common one among the people outside
Adobe who post in its threads, free mail and tenant domains left out), and an empty square
where there is none, so the orgs without an icon stand out (and can be listed alone). Favicons
come through /api/admin/favicon, which answers 404 where Google's favicon service has only
its globe. An org's page suggests icons (src/favicons.js): the favicons of the websites its
mail domains, slug and name suggest (<slug>.com, www.<slug>.com, <name>.com, …) and the
icons those sites name (<link rel="icon">, apple-touch-icon), each image with what Clef
thinks of it: whether the site is the org's own (not a namesake's, an agency's or a parked
domain's; Clef is told each site's title where it can be read) and whether the image is a logo
rather than a stock icon. Picking one makes it the org's icon. Sites are asked with a browser's
user agent (many answer a bot's with an error page); an .ico of bitmaps, which Clef cannot
read, is decoded into a PNG for it (src/ico.js), and an SVG icon is offered without Clef's
verdict.
Administration. The admin team manages all of this at /inbox/admin: ops teams and
their members (found by name or email among the people and platform accounts opspile knows;
names come from the platforms and are not edited there), the rules that file channels under a team (a name prefix like aem-, a Teams
team, a Slack Connect team), manual assignments of channels to a team or an org, the channels
shared with more teams, and each org's name, icon and type. A team's page suggests channels for it among the ones the admin team
holds (the rule of last resort files every channel no other rule claims there): the ones where
the team's members post, react or are members (listed from Slack, or from the Teams team, into
channel_members), ranked tf-idf style so that someone active in most of the admin team's
channels (an admin, say) counts for little and a member active in a few counts the most, and the
ones Clef gives the team. The cron asks Clef about a batch of the admin
team's channels every run, one question set per channel (channel_labels): which ops team should
follow it (each described by its rules, some of its channels and its members, or none of them),
and whether it is a conversation, an alert feed or not worth following. Clef is told the channel's
name, its Teams team, its topic, its customer, how busy it is, the Adobe people who post in it
and are in it with their ops teams, the technologies seen, and its recent threads' titles; it
looks again when the channel has new activity, the ops teams change, or the label is a week old.
Every rule change applies at once (the page names the channels it
moved; the API still answers dryRun with what a change would move), and the page sends its
changes one after the other, so many can be made in a row. Applied, each moved channel is re-filed by a refile job on the
NORMALIZE queue: its thread folders move to channels/<new org>/<new team>/… (an object
already at the new place, from a message that arrived meanwhile, is newer and kept), its
thread rows take the new org, team and kind, and the new org and team get their knowledge
bases. A job starts from where each thread is filed in D1, and also moves what is still where it was
when the job was queued (a message filed meanwhile points the row at the new place), so several
changes to a channel in a row end up where the last one says. A large channel moves a few hundred files per job.
Team managers. A member of a team can be its manager: they get the admin pages and
the knowledge-base editor for that team only, where they add and remove members, promote
them to manager or demote them, and edit the team's knowledge base (its systemprompt.md
too). Teams, rules, assignments, orgs and the customers' knowledge bases stay with the admin
team. A team can be self-managed: every member is a manager, and so is everyone who joins.
Turning that off keeps the roles as they are; from then on new members join as members, and
managers can be demoted. The role is team_members.role, the flag teams.self_managed;
only the admin team sets the flag.
Channel requests. A team manager does not assign channels, but asks for them: their team's
page has the same channel search the admin team adds channels with (every channel, by name,
Teams team, customer or id), and picking one sends a request, "add this channel to this team",
with an optional note (channel_requests, migration 0028; one open request per channel and
team). The admin team sees the open ones under Requests in the admin pages, their number on
Requests in every page's header. Opening one runs the manual assignment as a dry run
and says what confirming would do: re-file the channel under the team (and which threads move),
or share it with the team when another team holds it. Confirm then makes that assignment,
exactly as adding the channel on the team's page does, and records who confirmed it and what it
did; Decline takes an optional reason. The manager sees each request on their team's page:
pending, confirmed (moved or shared), declined with the reason, or withdrawn (they may take an
open one back). An admin who adds a requested channel to the team directly answers the request
with it, and so does a rule that moves the channel there. Confirming closes the request first, so
one declined or withdrawn meanwhile moves nothing, and a failed assignment opens it again. Nothing
is sent by mail or chat.
Channel names and authors are not in the events, so the normalizer looks them up
(conversations.info and users.info on Slack, Graph on Teams) and caches them in D1 for a
day. These are its only platform calls. Files only move forward to a newer version, and a
tombstone is final, so a replay of pile 1 in any order rebuilds the same pile 2. A rule
changed in D1 by hand needs pile 2 emptied and replayed; one changed in the admin UI re-files
the channels it moves (below).
A renamed channel is re-filed the same way, by the same refile jobs: the rules read a
channel's name, its Teams team's name and its Slack Connect teams, so a channel renamed to a
name a rule maps to an org (or another team, or a kind) takes its threads along. The normalizer
notices it when a Slack rename event arrives, when a Teams notice says the channel or its team
was renamed (it looks the channel up again then), and when a lookup (a day after the last, from a
message or the cron) finds them changed; Discord renames are found by the lookups. It queues the
moves only when the rules classify the channel differently than before, before it files the
message that came with the change. A manual assignment is a rule, so a rename does not undo it
(a channel assigned to a team keeps it, and takes the org its new name gives); the teams the
channel is shared with stay. A rename event older than what D1 knows of the channel (a replay)
is left out.
One markdown file per message, in a folder per platform thread (the root message's id; the
root message is <rootId>/<rootId>.md). Message bodies are markdown themselves (Slack mrkdwn
and Teams HTML are converted; a Teams system notice, whose body is an empty
<systemEventMessage/>, is written from its eventDetail: "Meeting ended after 50 s. Only Ada
joined.", with the names looked up where Graph leaves them out), so keeping each message in its own file avoids nesting one
document inside another, and makes every file a natural retrieval chunk. There is no
per-thread rollup file; reading a thread means listing its folder.
Folder snapshots are a read cache beside the folders, not part of pile 2's record:
snapshots/<folder>.json holds every file of one folder, parsed, with its R2 version
(src/filecache.js). A Worker may wait on only six R2 calls at a time, so an isolate that had
not read a thread before read a thread of 300 messages in some fifty rounds (three seconds);
with the snapshot it lists the folder, reads the snapshot, and reads only the files whose
version the snapshot does not have, then writes the snapshot again behind the answer. A file
is never taken from a snapshot for a version it does not have, so a stale or missing snapshot
costs reads, never correctness; re-filing a folder deletes its snapshot, and emptying pile 2
may drop them all.
---
source: slack
thread: slack:<enterpriseId>:<channelId>:<thread_ts>
logicalThread: slack:<enterpriseId>:<channelId>:<other_thread_ts> # only if re-threaded
id: <ts>
at: 2026-10-14T07:37:26Z
author: { kind: person, name: …, org: …, staff: true }
editedAt: …
deletedAt: …
raw: [<raw object key>, …]
permalink: https://…
---
The message body, as markdown.thread key slack:<enterpriseId>:<channelId>:<thread_ts>
teams:<tenantId>:<teamId>:<channelId>:<rootMessageId>
discord:<guildId>:<channelId>:<rootMessageId>
- A message that quotes another (Slack: one forwarded with it,
is_share, or the preview Slack adds under a link to a message,is_msg_unfurl) has it below its text and above its files, as a markdown quote whose first line is its author, linked to it with the title"Quoted message, <ISO time>"(quoteMarkdownin src/markdown.js). The inbox shows it as a card. Its words are not its author's: they count for no mentions, nor for the vectors of what people talk about, and editing the message leaves the quote alone. - New messages are new objects, so concurrent replies never contend for the same file.
- An edit rewrites exactly one file. Earlier versions remain in pile 1.
- A deletion keeps the file as a tombstone (
deletedAtset, body removed), so links and read watermarks stay valid. - A Slack
permalinknames the channel's workspace (https://<workspace>.slack.com/…, fromteam.info, kept inchannels.domain): a bareslack.comlink opens in whichever workspace the browser used last, and people are often in several. A channel of the whole Enterprise org, or one shared between its workspaces, links to the workspace most channels live in. Links filed before a channel's workspace was known are rewritten in D1 when it becomes known.
Enrichment on write. When a message is copied into this pile:
- Clef (a decision model)
classifies the thread: who has to act next (
us,customer,vendor,nobody), what kind of thread it is (incident, security, question, request, alert), and urgency. Thread-level results live in the index (pile 4). - Deterministic extraction handles structured tokens: CVE ids, URLs, ticket links,
org/siteidentifiers. - Clairic (an extraction model) handles loose facts, and only values above a confidence threshold are kept.
Re-threading. People often post a follow-up as a new root message instead of replying,
or split one request over several messages, and alert channels post the same alert again
and again. A root by the same person within three minutes of their previous root in the
channel, with nobody else's root between them, continues it (a burst), unless that root is
deleted by then (people delete a message and post it again, and the new one is the thread), or
someone has answered it in its thread already (then the new one is a message of its own, as the
next of several questions each in its thread). A root that only greets ("Hello Ada", "Hi team
:wave:": a salutation, the people it is addressed to and emoji, greets) says nothing of its own,
and its author's next root within 15 minutes continues it the same way: people greet, then take
minutes to write what it is about, and Clef reads a greeting as a conversation of its own. A root
that says nothing but @-mentions ("@", "@Ada Lovelace", at most emoji and punctuation
besides, or not even a mention: "?", ":eyes:"; mentionsOnly) continues its author's previous
root within those 15 minutes the same way, with the same conditions (nobody else's root between
them, that root not deleted and not answered in its thread yet): its author is calling the bot or a
colleague to the question they just asked, and it has no words for Clef or vector search to place
it by. A mention is @ and up to four capitalized words, what the platforms write for one, so
"@Ada can you check" says something and is Clef's to decide. Somebody else's bare mention is not
theirs to continue: one of a conversation's people, @-mentioned by name, is an addressed reply
(below), and the same mention by its own asker, after somebody else's root, or later than 15
minutes, is Clef's. In the case that showed it, a customer asked whether a tool had an issue,
then pinged the bot three and a half minutes later, 30 seconds past BURST_MS, and the ping's own
entry got an agent that asked the customer what he needed. When a
root message
arrives, its skeleton (the text without timestamps, long numbers, hex and UUID ids, query
strings and user agents; short numbers like status codes stay) is embedded and the most similar
earlier roots in the same channel, within 14 days, are found by vector search.
- Alert channels: an exact repeat of the skeleton or a near-identical root (cosine ≥ 0.97) is a match outright; otherwise Clef chooses between up to four candidates above 0.82 and "a new topic", told how long before each was posted, and a match needs a probability of at least 0.7. Candidates of the same group share their probability.
- Conversation channels: in a chat, answers, thanks, nudges and escalations posted as new
messages ("done, you have access now", "this is a blocker, please help") share few words with
what they follow up, people answer hours later, and the same words ("thanks!") are not a repeat.
So Clef reads the channel's latest three conversations active in the last three days and the
groups of up to four similar roots (cosine ≥ 0.6) as a transcript: each conversation's first
message and its latest two before the new root, marked with the conversation's letter and how
long before they were posted, then the new message, and says which conversation it continues or
that it is a new topic. A match needs a probability of at least 0.8, and 0.99 for a conversation
with replies in its platform threads (
MIN_CLEF_THREADED), and for any conversation in a channel that replies in threads (of its roots in the last 30 days, at least five, half or more got a person's reply in their own: a question that is still waiting for its first reply there is held in threads all the same): people continue a chat with more top-level messages where the chat is top-level messages, and where they reply in threads, a new top-level message is nearly always a topic of its own, however close it reads. Folded into conversations held in threads at 0.8, new roots got replies of their own as often as the channel's new topics; folded into ones without, half as often. In channels that reply in threads, those folded at Clef's usual certainty got replies of their own twice as often as the other folds. Comparing the new root with each candidate on its own spread Clef's answer over several candidates and left most of a chat in pieces. Platform notices ("… has joined the channel") are left alone. - Replies posted top-level: somebody who answers what they just read in the channel's box,
rather than in the thread's, posts a reply as a root: it @-mentions the person it answers, and
it comes at once. A root that @-mentions, by name, somebody who spoke in one of the
conversations Clef reads, less than five minutes after that conversation's latest message, is
its addressed reply: Clef's usual certainty in a conversation (0.8) is enough to fold it,
whatever the channel's habit and the conversation's thread replies, and even when it has
replies of its own already. The fold is
logical_via = 'reply', and, like a burst member, it stays in its group when people answer it in its platform thread (below): that thread is where the conversation went on, so whoever answers the reply answers the group. The lower bar is only for the conversation the root addresses: where Clef picks another one, it needsMIN_CLEF_THREADEDas before. Of production's roots over two weeks, 13 were addressed replies, and Clef gave 0.8 or more to three of them: a chat's answer and its follow-up, and a staff answer to a customer. The ten it kept apart say so themselves ("in other news", "another ticket", a question of their own), which is what the five minutes are for: an @-mention of whoever was active last, hours later, is as often a topic of its own. - Quotes: a root that quotes a message (below) stays on its own, whatever Clef or a burst
would say. Forwarding a message into the channel, or linking to one, starts a topic on purpose:
whoever meant to continue that conversation would have replied in it, and the text that goes
with a quote ("first step in that direction") reads like a follow-up of whatever was said last.
The root after it can still continue it. In the transcripts Clef reads, a quote is
> <author> wrote:and its text. - Thread marks: a root that announces a thread (🧵,
:thread:) stays on its own too, and out of a burst: nearly all of them are answered in their thread. - Replies of its own (de-threading): a conversation's root that people answer in its own
platform thread holds a conversation of its own, however much it reads like a follow-up (a
different customer raising a related problem by @-mentioning whoever was active last). One
that has a person's reply when it is decided is not folded. One that gets its first after it
was folded is split back out (
dethread, on that reply): its row loseslogical_threadand getslogical_via = 'split', which re-threading (forced, or a burst before it) leaves alone for good, its root file loseslogicalThread, the roots that followed it in a burst go with it, and the open inboxes reload the group (splitin the room'sthreadevent). So the group's entry stops interleaving the two platform threads, and its replies go to its own platform thread again (src/post.jstarget, the entry's latest). A burst member, an addressed reply (logical_via = 'reply', above) and alerts stay in their group. - Replies the inbox routed: a group's replies go to the platform thread it was last active
in, which may be a folded root's. Once one does (a person's from the inbox, the app's, or a
thread agent's), the group's conversation goes on there, where the channel sees it: that root
gets
routed_at(routeReply, written before the post) and stays in its group for good. Its replies do not split it out, neither the inbox's own nor those of the people who answer them there, and a forced re-threading leaves it alone. Splitting it would leave the answer the channel read in an entry without the question it answered.
A match sets logicalThread on that one message file and on its thread row, always pointing at
the group's first thread. The file stays where it is and the platform thread is never modified, so undoing
the match is a one-field change. Replays arrive in any order, so an alert that is matched also
pulls in near-identical later alerts that had found nothing earlier, and a root pulls in the
next root by the same person within three minutes (15 after a greeting, or for a root that only
mentions). A conversation's root is decided from the
conversations before it, so re-threading a channel again goes oldest root first
(scripts/replay.mjs --rethread --channel <id>). The inbox shows a group as
one entry: the first thread's title, the latest activity and the latest member's label, the most
pressing attention of any member, and all members' messages in order. A first thread whose messages
were all deleted after a member folded into it has no activity, title or link: the next member's
stand in. A root is re-threaded 30 seconds after it arrives,
so it can be in the inbox on its own until then: a match tells the open inboxes which threads
it folded, which take those threads' own entries off the list once the group's is there (as does
a reload of the first page that has them in a group, should the announcement be missed), one
that has a folded thread open shows its group instead, and opening a member's key (a link, a
tab) opens its group too.
Vector search uses Workers AI embeddings in Vectorize, written at ingestion time so that a message can be matched seconds after it arrives.
A second index, CONTENT (src/vectors.js), holds what was said, for
search over topics and who talks about them: one vector per message a person posted in a
channel thread (type: message, with the author's person, staff and author key) and
one per such thread (type: thread), the mean of the embeddings of its title and messages
in pieces of about 512 tokens (a long thread keeps its first piece and its latest). Both
carry channel, thread, org, team and at; the text stays in pile 2. Two minutes
after a person posts, or a message is deleted, the thread is vectorized: each message keeps
a hash of the text its vector was made from (messages.embedded), so only new and edited
messages are embedded again, and a deleted one loses its vector. The cron enqueues the
threads whose vectors are older than their latest message (threads.vectorized_through),
which also fills the index the first time. Alert threads (a bot's posts) get no vectors here.
A re-filed thread's vectors are made again (its hashes cleared, a vectorize job queued), so
that they carry its new team and org; re-filed as an alert or ignored, it loses them.
A third index, SEARCH (src/fulltext.js), is for full-text search: an
AI Search instance with built-in storage, one
item per message of a channel thread (people's and bots', not system notices), its markdown
body as it is. It carries team, channel (a hash of the channel key: AI Search filters read
the first 64 bytes of a value, and a Teams channel key is longer), message (<thread>|<id>)
and at. The vectors find what a message is about; BM25 here finds the words it has, an error
string, a hostname or an id, which an embedding blurs. Its items are kept in step by the same
job as the vectors: each message keeps a hash of its team and text (messages.indexed) and
its item's id (messages.search_item), so an edit uploads one item again (a re-upload under
the same name replaces it), a deletion takes it away, and a re-filed thread's items are
uploaded again under the new team (re-filed as an alert or ignored, it loses them). A failure
leaves a message's row as it was, so the next run tries again, and does not keep the vectors
from being made; an item whose deletion failed keeps its id until it is gone. Search only
counts and shows an item whose message D1 has and is not deleted, so a deleted message's words
are never found, whatever is left in the index. A job uploads at most 100; threads.indexed_through tells
the cron's sweep which threads are behind. AI Search's own vectors (the instance is hybrid)
are not used yet: the CONTENT vectors already do that part.
One Artifacts repository per org and one per team: a pile of markdown files with git history.
kb-org-<slug> what we know about this customer or vendor, across all teams
kb-team-<slug> product and process knowledge for this team, across all customers
The repositories follow the Open Knowledge Format
(OKF v0.2), the formalised LLM wiki
pattern: one markdown file per concept (a site, a known issue, a playbook, a decision), YAML
frontmatter with a required type, links between concepts, an index.md per directory and a
log.md at the root. Provenance is in each concept's frontmatter: sources lists the threads
(permalinks) a concept was learned from, with who said it and when, and footnotes tie claims
to them; generated says which agent wrote it and when, and verified will record staff
review. Every change is a commit, so a bad memory is a git revert.
Thread agents are the writers. A thread's agent writes its team's and its
customer's knowledge base, and those of the teams its channel is shared with (channel_teams);
every other team's it reads but cannot write (src/agent/repos.js, accessibleRepos). Code, not the agent, writes index.md and log.md
and stamps generated, and a write that is not a well-formed concept is refused. For now
every write is trusted and lands on main; staff review comes later, against the git
history and the trust tiers OKF derives from verified.
Agent instructions. Each knowledge base may have a systemprompt.md at its root (a
concept of type System Prompt). Every thread agent appends the bodies of those that apply
to it, its team's first, then those of the teams its channel is shared with, then the customer's,
to the end of its system prompt, read again
before each model request; HTML comments in them are left out. Only people write them,
whoever may edit the knowledge base, in the knowledge-base editor: agents' writes to systemprompt.md are refused, so a
prompt injection cannot change what later agents are told.
Editor. Members of the admin team, and team managers for the knowledge bases of their
teams and of the customers their teams own, have an editor at /inbox/kb: every knowledge base, its files, its history, and an editor for systemprompt.md
and the concepts. The members of every ops team see every team's knowledge base there read-only (its
files, history, thread agents, tools and MCP servers: the API answers them GETs only, and
says per knowledge base whether they may change it, may), as every thread agent reads them,
and the customers' their teams own the same way. A team owns the customers with a channel filed under it or shared with it
(threads.team, channel_teams; ignored channels aside), so ownership follows the channels
when they are re-filed or shared; src/kbaccess.js says who looks and who changes. What the
members change is only their
own settings, whether the agent's thinking shows in their inbox (see In the inbox, below). It saves as you type: a pause of a second (or leaving the file, or ⌘S)
commits the file, authored by the person and stamped generated: { by: human:<email> },
through the same write path as the agents' (OKF checks, indexes and log written by code).
A save names the blob it was written from, so a colleague's or an agent's newer version is
never overwritten unseen: the editor offers to load theirs or save over it. index.md and
log.md are read-only there. It asks only in the page (no native dialogs, which a debugger
attached over CDP dismisses), and only when edits could not be saved or before a delete; the
one exception is leaving the page (Back, reload, closing the tab) with edits that could not be
saved, where only the browser's own prompt can ask.
The list of knowledge bases has the ops teams first, then the customers and vendors, by
activity (the default: most entries first, then most tools and MCP servers turned on for their
threads) or by name, as the person last chose in that browser. An entry is a file other than
index.md, log.md and systemprompt.md; knowledge_bases.entries keeps the count, which
every write updates (and opening a knowledge base in the editor, when its listing is complete),
and the cron counts the knowledge bases not counted yet, a batch per run, over their whole tree.
Thread agents per channel. The editor shows, for each knowledge base, the channels it
learns from (a team's: the team's channels; a customer's: theirs, in every team) and whether
each one's thread agent is on; the list of knowledge bases marks those with
an agent in any of their channels. The admin team, and the managers of the team that owns a
channel (the team of its latest thread), turn its agent on or off there, right away (no
question: it is one channel, and switches back as easily). The switch is a row in
agent_channels with who set it and when. A team's knowledge base also has the team's
default, On by default (agent_teams, after a question in the page, since it switches many
channels at once): on, every channel of the team that nobody switched has an agent, the
channels it gets later too. A new team starts with it on (the admin team's name for it in
changed_by), and with the agent tools computer and extraction_rules and
the global servers browser and github on for the team (those marked
newTeams); the teams from before that have them as they had them, and deleting a team
deletes them. A channel nobody switched has an agent when its team's default or
AGENT_CHANNELS, the deployment's default, turns it on. Alert channels have no
agents. Turned off, a channel's agents stop at their next message and skip their wrap-up;
their drafts and what they filed stay.
MCP servers. Next to the thread agents, the editor lists a knowledge base's MCP
servers: those of its team (kb-team-*) or its customer (kb-org-*), which the agents of
that team's or customer's threads may call from code mode (see Thread agents).
Whoever may edit the knowledge base adds, checks, edits, switches and deletes them. A server is a
name, an https:// URL, what it is for (the agents are told), request headers for its
credentials (sent once, sealed with TOKEN_KEY, then shown by name only) and optionally the
only tools agents may call. Adding or changing one connects to it and lists its tools. Above
them are the global servers, with a switch for the team (in a team's
knowledge base) and one per channel the knowledge base learns from. In a team's knowledge base
the per-channel switches fold, the channels switched on their own first and then by name, with a
filter when there are many; in a customer's they are all there is, so they are shown unfolded.
Every page has the same header (ui/header.js, ui/header.css): a switch between Inbox
and Agents (this editor: the agents' knowledge bases, channels, MCP servers and
instructions) for every member of a team, Teams too for team managers, and for the admin
team Orgs and Requests too, the sections of /inbox/admin, Topics (/inbox/topics,
below) and COGS (/inbox/cogs, below). Each opens as it was left in that
browser tab: the header keeps every page's address (sessionStorage) whenever it changes, so
Inbox comes back with its team, filters, sort, unread, the open thread and "view as",
Agents with its knowledge base and file, and each admin section with what was picked in it.
From the inbox, what is on screen picks what the others open on: Agents and Orgs the customer
on screen (the open thread's, or the one customer filtered on; Agents its knowledge base if it
has one, else the team's), Teams the team picked (in the team views, for whoever may manage it).
With nothing on screen for them, they open as they were left, and Agents, not opened yet in the
tab, on the team's. The inbox, not
opened yet, opens from a team's knowledge base on that team. (back=, the inbox's
address on links made before, is still taken.) The editor's URL names the knowledge base and
the file (or the history), and each one opened is a history entry, so the browser's Back and
Forward step through them; going between the admin sections is one too. The editor itself is
always the admin's own, also when they came from a "view as" inbox. On a phone the switch is
the page you are on, with a menu of the others.
Your menu, on every page, has Appearance: the color scheme, the system's (the default),
light or dark, kept per browser.
The repositories are created through the Worker's Artifacts binding (one namespace per
environment, helix-opspile and helix-opspile-stage) as soon as their org or team is
known: when the normalizer discovers a channel, it makes sure the repositories for the
channel's org and team exist, and the cron creates one for every team in D1, so a team that
was added by hand gets one before it has any channel. knowledge_bases in D1 records the
repositories that exist and their git remotes. Internal slugs (_unassigned) get none.
- D1 holds the queryable index: containers, threads (last activity, last speaker,
classification), people and their platform identities, ops teams, and read watermarks. It
answers the inbox queries, for example "everything waiting on us for my team" or
"everything from this customer in the last 7 days".
It keeps one piece of text: each message's first 200 characters as one line of plain text
(
messages.preview, written by the normalizer), and the latest one of every thread (threads.last_text, derived with the otherlast_*columns), so the inbox's rows show the latest reply without reading the thread. Threads indexed before it was kept (migration 0035) get it from the cron, a few hundred per run, the most recent first (src/previews.js, from the latest message's file in pile 2). - One Durable Object per ops team holds live read state. UI clients connect over WebSocket, read events fan out to everyone on the team immediately, and the object writes them back to D1 in batches. New messages, reactions and drafts are announced in the room of every team whose inbox has the thread (a shared channel's are in several). A message posted in the last hour that mentions people it did not mention before (a new message, or an edit that adds a mention) names them in its announcement, with who wrote it, for their notifications; a backfill, a replay or a late catch-up names nobody. An inbox on screen reloads its first page (and the open thread, if it changed) half a second after an announcement. A tab in the background does not: it notes which threads changed and catches up when it is shown again, unless a bell watches the inbox it shows (bells are checked on its reloads). The thread agent's status is not polled from the background either. A socket that was down has missed what its room announced, so when it comes back the inbox and the open thread reload, in the background tab's case once it is shown.
- The index is derived: a
messagesrow per pile-2 file, and athreadsrow recomputed from its messages on every write (last activity by a person, last speaker and whether they are staff, title, message count), so it is the same in any replay order. Alert threads use the last message of any kind, because they consist of bot posts. - The UI is one static page, served by the Worker at
/inboxbehind Cloudflare Access: facet cards (status, type, next actor, urgency, kind, customer, platform) on the left, threads in fixed-height rows in the middle, and the thread as a chat on the right (customers on the left, Adobe on the right, with reactions and images; only your own messages are blue, colleagues' are grey like the customer's). Avatars are the Slack profile picture or the Gravatar for the email; the customer's icon is the favicon of their mail domain. Reactions are kept in D1 (reactions) from Slack's reaction events and from the reactions Teams reports on each message. - Search is the box over the thread list (
/jumps to it): one search over the view on screen, a team's inbox or the personal view (view=personal), on every platform at once, Slack, Teams and Discord alike (GET /api/search, src/search.js). It looks three ways and adds them up. By meaning: the query is embedded (bge, with its instruction for short queries) and theCONTENTindex answers with the team's message and thread vectors and those of the channels shared with it; a match needs a similarity of 0.6 and to be within 0.12 of the best one. By the words of a message:SEARCHwith BM25, every word in one message (Porter stemming: "purged" finds "purge"), the recent boosted; the best scores 0.7, the others by their BM25 score relative to it. By the words of the thread: every word of the query in the title, the channel or the customer (the more of them in the title, the better), or in a fact (a URL, ticket, CVE, site, the problem). Members of a re-threaded group count for its entry, and D1 decides what the viewer sees, as for the inbox. The results take the list's place, best match first, each with the message that matched (its words marked); opening one scrolls the thread to that message. The filter cards count and filter the results, j and k step through them, and Esc or Clear brings the inbox back. - A click on a facet value picks it; Shift-click, or the ≠ beside it, excludes it (negative
filters, as in klickhaus): picked values of a card are OR'd, excluded ones AND'd, and the
cards AND'd with each other. The URL keeps both,
f.<facet>=a|bandnot.<facet>=c; a picked value is tinted and outlined in the accent colour, an excluded one is struck through, and a plain click stops excluding it. - The thread list has the personal view and a view per team the viewer is in, switched above
the list (for a member of two teams: Personal · Admin · Helix). Personal
(
view=personal, and the default: any address withoutteam=) has the conversations in the channels you posted or were mentioned in, whichever team they are filed under (all of their threads, not only the ones you are in; no alerts), puts the latest activity first (a person's last post, or for a thread only bots posted in its last message) and keeps only the customer and platform cards (one customer is picked at a time); filters on the cards it does not show do not apply there, and are kept for the team views. Unless the expert view is on (the user menu, theexpertsetting), it is the simple one: a column of customers by logo and name instead of the cards, and the list by customer until one is picked. A customer's ≠ in that column, or a Shift-click on it, excludes it as on the cards (not.customer=in the URL, and in a bell on the view): its threads leave the list, and it stays at the end of the column, struck through and out of the red badge of All customers, with or without threads, until a click brings it back. All customers drops the pick and what is excluded. Besides its cards it has one switch, Mentions (the @ at the top right of the customers;mentions=1in the URL), keeps the threads that mention you, read or not (mentioned, all the thread's mentions, has you); with unread only, the ones the red badge counts. The customers without such a thread leave the column while it is on. Team views do not have it: their Involves card has Me (you posted or were mentioned). Its live updates come from the rooms of your teams and of the teams those channels are filed under; in the rooms of other teams, only about those channels (scoped sockets,src/room.js). Someone on no team (no-team mode,SIGNIN_DOMAINS) has only this view. A team's view (team=<slug>) puts what needs you most first and has every facet card. First comes the status (unseen and critical, unseen and hot, you, waiting too long, customer silent, team on it, the rest); within a status, the score ofui/rank.js, boosted trees over what the inbox knows of a thread (how long ago, who is waited on, urgency, resolved, kind, …) learned from Clef's pairwise judgements of production threads, from what people open in the list and from which customers get answered first (npm run rank, see the README), plus how close the thread is to you (mentioned you, you posted, 🛟 or 👀 by a colleague). Opening a thread by clicking it in the list records the pick, with the threads on screen and their features, inrank_events(POST /api/rank/pick, migration 0040), for that training. A facet value with threads you have not read has its label in bold, and a red badge counts the messages that mention you and that you have not read yet (Slack's and Teams' mention count); a thread's "N replies" is bold and blue while it has replies you have not read. - Channels a message names (
#name, or#C…where Slack sent the id without the name, as it does for bots) are resolved when the thread is read (src/channelrefs.js, thechannelsof/api/thread), so channels opspile learned about later are named too: the thread shows#name, linked to the inbox that has the channel (the view on screen if it does, else your team's, else Personal) with the channel's customer picked, or to the channel on Slack or Discord when it is in none of yours. Inbox titles and last texts get the names of the ids. Links to a channel (not to a message in it) are resolved the same way, on every platform (src/channelurl.js, thechannel_linksof/api/thread): a Teams channel's link in a Slack message, or a Slack channel's in a Teams post, opens the channel's threads in the inbox, and a bare link shows#name. A link to a channel that is in none of your inboxes stays a link to its platform. - The list stays put while it is the same list (the same view, filters, unread only and Mentions): reading a thread neither moves nor hides its row. Every thread it has shown stays in it, read or not (unread only is "unread when the list was opened"); the rows in view, and those scrolled past, keep their order, and the rows below them are sorted as the view sorts. Threads that come in since (new ones, or with unread only, read ones with a new message) are added at the end of a team's list, except critical and hot ones, which come in right below the rows you have seen; in the personal view, threads with activity since that you have not seen in it come to the top. Another filter or view, unread only, or a click on the view already shown sorts it afresh; the thread open then stays, if it passes the filters.
- Notifications are shown by the browser while an inbox is open (a tab, or the installed
app, also in the background), through the service worker, so that a click opens the thread
in the inbox already open, or in a new one. There are three kinds, settings of the person in
D1 (
person_settings,GET/PUT /api/settings), so every device has them: a message that mentions you (on unless turned off in the user menu; the room's announcement says whom it mentions), a new message from someone else in a thread of yours (you posted in it, were mentioned in it or put 👀 on it, in any thread of its group; on unless turned off, and the announcement says whose thread it is; bots' messages and edits do not count, and a message that mentions you notifies as a mention), and a bell on a view. The bell above the thread list turns notifications on for the view on screen with its filters (picked and excluded); a thread of that view that gets a message you have not read then notifies you. The inbox listens to the rooms of the teams its notifications need (all of yours, for mentions, your threads and Personal), checks a bell on the inbox on screen when it reloads, and fetches the first page of a belled view that is not on screen when its room announces activity. The menu lists the bells, each opening its view or turning it off. Inboxes in several tabs show one notification per thread, and none for the thread you are reading, nor while viewing as someone else. The first visit asks once (a toast) whether the browser may show them; browsers only ask on a click. - The toolbar is the same on the inbox and the admin pages: the pages and label review on the
left, and on the right whoever is signed in, with the view on screen under their name. Their
name opens a menu with "View as" (admin team; while it is on, a chip next to the name says
whose inbox this is and leaves it in one click), the notifications, the keyboard shortcuts, and Log out,
which is Access's
/cdn-cgi/access/logout. Live updates are not shown while they work; the toolbar says "Reconnecting…" when a WebSocket has been down for a few seconds.
Not a pile of its own but a copy of pile 4 for statistics: one row per message in ClickHouse,
for klickhaus' opspile view (opspile.html), which has
the facets, the chart and the long time ranges of klickhaus' other views
(src/clickhouse.js). Every thread in D1 is exported, of every team and
kind (channels the rules ignore never get there).
- No text. A row is the message's metadata (when, which platform, channel, team and the
teams it is shared with, org; its author's kind, relation, name, mail domain and ops teams;
whether it was edited, its reactions, mentions and files) and its thread's classification:
the title, whether it is resolved, who it waits on, Clef's label (who acts next, kind,
urgency and its level,
resolved_p) and the facts (tickets, sites, CVEs, URLs, the problem). Itsurlopens the thread in the inbox's personal view. - As the inbox shows it. The label, title, resolution and facts are the inbox entry's: a
re-threaded group's latest member's label (with reviews), its first thread's title, the facts
of every member, and resolved by a staff ✅ on a member's root (until a customer writes again)
or else by Clef (
resolved_p≥ 0.5). Who it waits on is the inbox'swaiting_on: Clef's next actor, except that a thread where Adobe posted last never waits on us, an unlabeled one waits on us when the customer posted last, and a 🙏 after the last message leaves nothing pending. The urgency level is the inbox's bucket (none is low). So exporting a thread exports its group; a group of more than 100 threads exports only the threads that changed and its first. - One table, rows replaced.
opspile_messages_rawis a ReplacingMergeTree keyed by(timestamp, thread, message_id);_versionis the export's time, so the latest export of a message wins, and_deletedmakes it a tombstone: a deleted message, and every message of a thread re-filed as ignored. klickhaus reads it through a view withFINAL. Rows are kept 18 months. - Written by the Worker. An export reads D1 only, and POSTs
INSERT … FORMAT JSONEachRowwithasync_insert=1&wait_for_async_insert=1toCLICKHOUSE_URLas a user that may only insert there (credentials in headers). What D1 did not keep before (the author's kind, edited, the number of files:messages.author_kind,edited,attachments, migration 0041) is read from pile 2 once and kept; the normalizer writes it for every new message. - When. A message written queues an export of its thread three minutes later, after the
label job; the label job queues one when it writes a label, and so do re-threading, a reaction
and a re-file. A job queued before the thread's latest export is dropped, so a burst exports
once. Each thread remembers what was exported (
threads.exported_through, the row'supdated_at, andexported_at); the cron enqueues up to 1,500 threads a run, newest first, whose rows are behind: a message written since, a label, a review or re-threading since the export, or a re-file (which clears the watermark). That is the backfill too, and the retry: a failed insert leaves the watermark as it was. Without theCLICKHOUSE_*bindings nothing is exported. - Not re-exported by itself: a renamed org or a changed type, a partner domain, team membership and shared teams. A thread's rows take them on with its next export.
In the inbox, the header has Stats for members of the teams in KLICKHAUS_TEAMS: it opens
KLICKHAUS_URL/opspile.html in a new tab, filtered to the customer and team on screen.
Read state is a watermark per person per thread: "has read up to message timestamp X".
- Unread for me: the thread's last message is newer than my watermark.
- Seen by somebody else, not by me: another member of my team has a watermark at or past the last message, and I do not.
- Unseen by the whole team: nobody on the team has a watermark at or past the last message.
Neither platform exposes other users' read state for channels, and each exposes the signed-in user's own state only coarsely, so opspile owns read state. Most of it is inferred rather than clicked:
| Signal | Effect |
|---|---|
| Person posts a message in the thread | Their watermark moves to that message |
| Person reacts to a message | Their watermark moves to that message (Slack reaction_added; Teams reports reactions on the message) |
| Person opens the thread in opspile | Their watermark moves to the last message |
| A message is edited, or a reaction is added by someone else | No change; edits never make a thread unread |
| Bot or system message | Stored and shown, but does not count as new activity; a system notice is no reply either, and titles a thread only when nobody said anything in it |
The inbox sorts every thread into one pile, most pressing first: critical (critical
urgency, nobody at Adobe has seen it), hot (high urgency, unseen by Adobe), you (you
posted or were mentioned, and the customer spoke last or you were mentioned), waiting
(waiting on us longer than 15 min / 1 h / 3 h for critical / high / normal), silent
(waiting on the customer or vendor longer than 30 min / 1 h / 1 day: follow up or close; for
normal and low urgency, Saturdays and Sundays in the viewer's time zone do not count),
team (your team is in it and all is well) and quiet. Who a thread waits on is Clef's
next_actor, except that a thread where Adobe posted last never waits on us (the label can
be stale or wrong), and an unlabeled thread waits on us when the customer posted last; the
API sends it as waiting_on. Admins can look at the inbox as another person, read-only.
Read state answers "who has seen this"; attention ranks what that means for the person looking, most pressing first:
- Nobody at Adobe has seen it: no staff member at all (in any team or none) has posted, reacted or opened the thread since its last message. Staff become people in D1 the first time they post, so their reads count even outside the ops teams.
- You were mentioned after you last read the thread (a platform mention, or your full name in the text).
- A teammate was mentioned after they last read it.
- Nobody on your team has seen it.
- You have not seen it.
Alert threads skip the first level: they are bot posts and nobody outside is waiting.
A reply whose root has never been seen (it was posted before ingestion started, or before
the bot joined) triggers one request for the thread's history: conversations.replies on
Slack, the message and its /replies on Graph. Every message lands in pile 1 marked
via: backfill and is normalized like a live event, so pile 2 cannot tell them apart. The
backfills table makes the request idempotent. An error that stays (the bot is not in the
channel) ends a request. Any other leaves it open: the queue retries it, and the cron sends out
what is still open after an hour, an hour apart, until the thread has been quiet for a week.
Two minutes after a person posts, the thread is labeled (later posts reuse the same label job, and an unchanged thread is never labeled twice):
- Clef answers four questions: who acts next (
us,customer,vendor,nobody), what kind of thread it is (incident,security,question,request,access,people,meeting,other; alerts arealertwithout asking), urgency (0–3) and whether it is resolved. The state it reads is the thread as text, with every author marked as Adobe staff or customer.accessis about permissions and getting into tools; people joining, leaving, being introduced or away arepeople; scheduling and sharing a call's notes aremeeting. - In the same call, Clef answers a yes/no question per topic (src/label.js
TOPICS): authoring and content, sign-in and permissions, project plan and go-live, caching and delivery, domains and certificates, performance, preview and publishing, SEO, URLs and redirects, code and deployment, blocks and front-end code, errors and traffic, limits and quotas, security and compliance, roadmap and feedback. The kind says what the thread asks for, the topics what concern it is about, and the tech stack (below) which products: an incident and a question can both be about caching, on Akamai or Fastly. Topics name no products. A thread keeps the topics at 0.55 or above (labels.topics): the best single cut on 225 threads labeled by hand (macro F1 0.82; the ten topics before, kept at 0.8, had 0.56 and left 59% of the threads with no topic, these 31%, the hand labels 24%). The list came from the topic map. Clef's answers move a little with the other questions of a call, so the four questions above and the topics are asked in a call of their own, as they were measured, and the tech stack in calls of its own: an entry added or changed does not move the topics' answers. Measure again when the questions or the topics change. A label from before showsgoliveasproject, andcommerceandintegrations, products now, not at all, until the thread is labeled again. - Extraction rules (
src/extract.js) find facts and detect technologies: regular expressions, or scripts, which are configuration of the ops teams (extraction_rules), not code. The admin team's apply to every channel, any other team's to the channels it owns, on top. The admin team's start as migration 0043 wrote them: CVE ids, ticket keys, Edge Deliveryorg/sitepairs (in hostnames, includingaem.networkandaem.reviews, da.live paths and admin API paths), the admin API operation, AEM Cloud ServicepNNN-eNNNenvironments, GitHub repos, aem.live documentation pages and tools, configuration files (cdn.yaml,redirects.xlsx, …), HTTP status codes of failures, response headers that name a request or an error, support cases, and URLs (not chat links and attachments); and detections: an Edge Delivery or Document Authoring link, an AEM Cloud Service, Assets or App Builder host, an Akamai error reference, a Cloudflare ray id, a Fastly or CloudFront header, another git host. A rule is a pattern with a value template ($2/$1), a case and anexcludepattern, that writes a fact of itsfieldor detects technologyfield. A regular expression cannot be stopped once it runs (and a Worker's clock stands still while it does), so a pattern that repeats a group that itself repeats without bound, as(a+)+, is refused when it is saved and when it runs: that shape backtracks without end on text that almost matches. A script is an ES module whose default export gets the thread and returns[{ field, value }, { tech }]; it runs in a Dynamic Worker of its own (LOADER), without network access, within two seconds. Rules are edited on the Agents page (Extraction rules of a team's knowledge base), with the MCP server's tools (list_extraction_rules,save_extraction_rule, …), and by thread agents where the team turned on theirextraction_rulestool (see "Agent tools"). - A language model (
llama-3.3-70b-instruct) writes a title of at most eight words. For an alert group it names the specific failure (the error or failing request), once per group. The title leaves out what the inbox shows next to it: the customer, the channel, the platform, the kind of thread and Edge Delivery itself. When the model still names the customer or the product, it is asked once more, told which names to leave out, and its second answer is kept whole (the names are never cut out of a title, which leaves broken phrases such as "Headless options in without App Builder"). When the model finds nothing to name ("None", "No title available"), the thread has no title, and the inbox shows its first words. - Clef also answers yes/no questions about the tech stack the thread shows (Edge Delivery,
Document Authoring, Universal Editor, the CDN, the content source, …): the technologies of
the teams' configuration (
tech_stack), each asked as "Does this thread show that the customer uses ?". The admin team's are asked in every channel; a team's own are asked in its channels too, and one with the id of an admin team's replaces it there (turned off, it is not asked there). Clef gets at most 24 questions per call; the rest are asked in calls of their own, at the same time. The channel keeps the highest probability any thread showed (a rule's detection counts as 1), so every thread of a customer shows their stack; the label keeps what the thread itself showed (labels.tech, migration 0059), so its header can name that first. The inbox shows a team's own name for a technology in its inbox and threads, the admin team's elsewhere. An entry also has aliases (tech_stack.aliases, migration 0057): the names it goes by in threads ("AEMaaCS", "IMS Org", "EdgeWorkers"), which match product names to it and are not asked of Clef. A feature people ask about by its own name is an entry, not an alias: the AEM Sidekick was an alias of Edge Delivery until migration 0061, so Clef never asked about it, the Topics page drew it as Edge Delivery, and a thread about it showed nothing of its own. Every entry is part of a product (tech_stack.product, migration 0056): the Adobe product it belongs to, by its name on Adobe's product list (Edge Delivery, Document Authoring, the AEM Sidekick, Universal Editor, Experience Workspace, AEM as a Cloud Service, MSM and Edge Functions are Experience Manager Sites; Dynamic Media is Experience Manager Assets; CX Enterprise Coworker, Brand Concierge, Brand Visibility, Journey Optimizer, … are their own), or a category for other companies' (CDN, Code hosting, Consent, …). Clef is asked about each entry; the product groups them, on the Agents page and the Topics page, where an entry is a feature of its product. Migration 0056 added what the product census (Clairic over all 822 channel threads with a person's message) found the threads naming that no entry covered, and the Adobe products they named that had none; it also put the names people use into the descriptions of others (Google Tag Manager and Tealium are their own entry now, Adobe's analytics another), where nobody had changed them since migration 0043. They are what Clef's topics about commerce, assets, Sites Optimizer, integrations and the inbox would have been. - The inbox filters by kind, topic and tech stack (the channel's, at 0.8 or above), and so do
the MCP server's
list_threadsand the inbox's WebMCPfilter_threads. - Clairic extracts the loose facts: the problem in the customer's words, the affected URL,
the error, a deadline and a go-live date. Only values at or above a confidence threshold are
kept (0.8, and 0.6 for the free-text problem). In the same call it lists the products the
thread names (
productfacts), for the topic map to match against the tech stack. Clairic lists nearly every name in the state, soproductFacts(src/label.js) keeps few of them: only its spans count (counted in code points, as Clairic counts; a string here counts UTF-16 units, one more per emoji), not those in the header or an author line, not a mention (@Name) or a piece of a word, not the names of the thread's people and customer, nor generic words (src/products.jsGENERIC). Of the rest it keeps what the tech stack covers, one name per entry, and at most six it does not, the most named first, when they look like the name of a product (productLike: a proper name of at most four words, not an id, a variable, a header, a host name, a short acronym, a possessive, a phrase that ends in a lower-case noun, or a word the thread also writes in lower case). The list's one confidence says nothing (it shrinks as the list grows) and is not used. On the census's 822 threads: two names a thread instead of six, two in three of them names of known products (two in five before, and a sixth generic words), the names nobody knows yet a third, and 83% of the products a hand-made alias table finds. A product name says the thread mentions it; Clef's tech questions still say what the customer uses.
Label review. Members of the admin team can review labels in the inbox: a queue of current labels nobody has reviewed, least confident first, and per question the right answer. A review is calibration data (Clef's agreement rate and most common corrections per question) and overrides the model's answer for that label version; when the thread changes and is labeled again, the model answers again and the old review only counts as history.
Clef's topics are asked: each is a question someone wrote, and a thread about something nobody
asked about gets none. A topic map (src/topicmap.js, the Topics page) finds what the
threads are about from what was said in them instead, for the admin team, to see the common
themes, who works on which and for whom, and where Clef's topics miss: what to ask Clef next.
- The threads: channel threads with a
CONTENTthread vector, at least three messages, and not ameetingorpeoplethread (whose vectors say "let's meet" more than what about). Their vectors are centred (the mean taken away, then of unit length again): bge's vectors share a large common part, and without it k-means makes a few big clusters and many of one thread. - k-means (spherical, cosine; k-means++ seeds, the best of four runs, seeded so the same threads make the same map) into about one topic per twelve threads, at most 30. A few hundred threads take a quarter of a second.
- One call names every topic (two to five words) and says in a sentence what its customers need: it reads all of them at once, each as the titles of its fifteen most typical threads (nearest its centre), the Clef topics and the products they have, with how many threads have each, so the names tell the topics apart. The model is the agents' (Claude on Bedrock, through AI Gateway, counted for COGS) where agents are configured, else the language model of the titles. A name that is vague (made of words such as "issues", "management", "content delivery"), too long, or another topic's, and a topic it did not name, are named on their own from the titles. No name names a customer; one given twice gets a number.
- With them, per thread: the Adobe people who posted (staff, by
people.id, with their messages), the customer (orgs), Clef's topics (labels.topics), the tags on the page, and the products Clairic named. Each product is matched against the tech stack (every team's enabled entries, src/products.jscoveredBy): an entry covers it when its id, name or one of its aliases is the product's; or is in it as words, where the product is written as a name ("Akamai Property Manager" is Akamai's, "en-to launch" is not Adobe Launch's); or its description names it where it writes a name, cut at commas, "or", "such as" and brackets ("a consent manager such as OneTrust or Cookiebot" names OneTrust and Cookiebot, not "consent manager"). A word alone is no name. A covered product is named as its entry, and drawn as the product the entry is part of (its features listed when it is picked); one no entry covers is a gap, named as most threads write it. "AEM" and "Adobe" in front do not count, except where a name is generic without them ("AEM Author"), and generic names (API, CDN, JSON, Slack, …) are left out. - The cron builds one when the newest is a week old, and the admin team with Rebuild; a row
of
topic_mapseach (the map as JSON), the newest five kept. - The expert finder reads the newest (
src/topicindex.js): which topic each thread is in and who worked each topic's threads, so it can tell a thread agent which topics a person's similar threads are in, and list who works a topic (list_topics).
The page draws the map as one force-directed graph (its own, after d3-force's model): a node per person, topic, tag and customer, as big as its threads, a line between two that share threads, thicker the more (with the topics shown, a line joins a topic to the others, and a person to a customer; the topics are the hubs). One kind shown alone is joined to itself: two people, tags or products by the threads they share, and two topics or two customers, which never share a thread, by the people they share (as many people as the threads a line needs, each with that many threads in both). The filters are a team, the kinds shown and the threads a line needs. A node picked shows what it is joined to and its threads, each a link into the inbox; without a pick, the panel lists the gaps first (Not in the tech stack: the products no entry names, by their threads; those two customers or more name, then, folded, those of one customer, which are as likely a brand or a system of theirs, and a count of the names of one thread; adding one on the Agents page, or its name to an entry's description, makes Clef ask about it), then the topics with the share of their threads Clef tagged at all, the fewest first on a click: the topics Clef rarely tags are what its topic questions miss. A gap is drawn as a hollow ring.
Replies are written in the inbox and sent as the person, never as a bot: each person
connects their own Slack account (user scopes chat:write and reactions:write) and Microsoft
account (delegated ChannelMessage.Send, which also allows reactions) once. The account must be theirs (the same person in D1, or the same
email); tokens are encrypted with TOKEN_KEY, Microsoft tokens are refreshed when they expire.
A reply goes to the latest platform thread of an inbox entry and comes back through the
normal ingestion path, like any other message. Replies are written in Markdown; on Slack they
are sent in mrkdwn (**bold** as *bold*, links as <url|label>, bullets as •).
A reply shows in the thread at once, as it will look once sent (the person's name, avatar and
blue bubble from their latest message on the platform), with a thin bar under it, as in
iMessage. The platform takes a reply in about a second (POST /api/post); there is no real
progress to report, so the bar eases towards 90% and completes and fades once the post
returns. The bubble then stays until the message comes back through ingestion (some ten
seconds later: the Slack event, the normalize queue's batch, the live update), and the message
takes its place unchanged. Replies to a thread go out one after the other, in the order they
were written. One that failed says why, with Retry and Discard; it stays until then.
A page that lost the answer cannot know whether the post went out, and a platform would take the
same post twice, so a reply carries a key, the same on every retry (post_keys, migration 0049,
src/postkeys.js; the cron deletes keys after a day). A retry that finds the first answer gets
it, without posting. One that finds the first try unfinished says the reply may have been sent,
and offers Send again, under a new key, so that sending it twice is the person's decision. A
reply the platform refused is not kept, so its Retry posts. The MCP tools send no key.
Leaving the page while a reply is still on its way (a reload, closing the tab) asks first,
with the browser's "Leave page?": the bubble lives in the page only, and the post may be cut
off.
Emoji names. A :name: in a reply goes out as the platform knows it (src/shortcodes.js):
Slack gets Slack's name, Teams and Discord the character. Another platform's name or spelling is
Slack's (:facepalm: is :face_palm:, :thumbs-up: is :thumbsup:, from the GitHub and
Discord names in src/emoji.js and by leaving out separators, src/emojinames.js), and a
workspace's own emoji (one somebody reacted with, and on Slack every one of emoji.list, bot
scope emoji:read, remembered per isolate for ten minutes) stays. For any other name, Clef
picks the closest emoji from at most 16 candidates (the most alike by name and CLDR keywords,
then common ones), at most three names per reply, within 2.5 seconds; it needs a probability of
0.5, else the name stays as typed. On Slack, Clef is asked only once emoji.list answered:
without it (no emoji:read, Slack failing; tried again after a minute) a name that is not a
standard one may be a custom emoji and stays as typed. Answers are remembered per isolate. Code
(fenced blocks of three or more backticks or tildes, code spans of any number of backticks) and
addresses (bare, autolinks in <…>, link destinations) are left alone. Edits (POST /api/edit)
go the same way. The inbox shows every platform's names as emoji the same way.
Custom emoji in the inbox. GET /api/emoji/custom gives the page what emoji.list says,
aliases followed: a custom emoji shows as its image, in messages, titles, previews, reactions
and the thread list's moods, and an alias of a standard emoji as that emoji. The images load
straight from Slack's CDN, which wants no sign-in; one that fails shows as :name:. Reactions of
other platforms never use them: Teams stores like, which a workspace may have an emoji of its
own for. emoji.list covers the workspace of the bot token only, so the other side's emoji in a
Slack Connect channel have no image. For such a name, and any other nobody has, the page asks
GET /api/emoji/closest: the emoji Clef picks as in a reply, three names a request and 40 a page
load, shown with a dotted underline and a tooltip that says what it stands for. Only once
emoji.list answered, since the name may be on it, and only for a :name: on its own, so
urn:aem:sites stays as it is. Previews in plain text, like notifications, keep the name.
Emoji suggestions. Typing : and two more characters of a name in the reply box (or in a
message being edited) suggests up to eight emoji, as typing @ suggests people. First, at
once, those whose Slack name, or another platform's name for them, has what was typed: the
whole name, then names it starts, then names with a part it starts, then any, one name per
emoji (matchEmoji, src/emojinames.js), in a Slack thread the workspace's custom emoji too.
When that leaves room, Clef fills the list
(GET /api/emoji): it ranks the candidates above by how likely each is what is being
typed, with no threshold, and the list takes them in that order, skipping the emoji it
already has, until it is full. Picking one writes :name: . A ranking is the same for everyone:
the Worker keeps it in the data center's cache for a day, and the browser keeps it too.
Files. A reply can carry files (src/attach.js): the 📎 button beside the reply box,
files dropped anywhere on the open thread (which shows where they go, or why they cannot), or
an image pasted into the reply box (a screenshot, named for when it was taken). They wait as
thumbnails and chips above the box, each with an ×, and go in the same message as the text,
up to 10 at a time. What a platform takes is a rule the Worker sends with GET /api/me, which
the page checks before a file is added and the Worker again before it uploads:
| as the person | as the opspile app | |
|---|---|---|
| Slack | any file up to 20 MB, 25 MB in all, with the user scope files:write (files.getUploadURLExternal, then files.completeUploadExternal in the thread, the text as initial_comment) |
the same, with the bot token (bot scope files:write) |
| Teams | images (PNG, JPEG, GIF) inline as hosted contents of the reply, up to 3 MB in all (Graph takes 4 MB of base64); other files and larger images uploaded to the channel's SharePoint folder and attached by reference, with the delegated Files.ReadWrite.All |
images inline only (Bot Framework data URIs, up to 1 MB): Teams lets bots send other files in personal chats only |
| Discord | none: opspile does not post as the person | any file, 10 MB in all, in the message (files[n]) |
The scopes for uploads are asked for when they are first needed: a person whose token lacks
them is told why when they attach a file, with a link that connects the account again with
file access (/api/auth/<platform>/start?files=1); a Microsoft token is refreshed with the
scopes it was granted. Where nothing can be attached the button is dimmed and says why. The
reply shows its files while it is being sent, and once it comes back through ingestion, as any
message's files are shown.
The one exception is posting as the opspile app (POST /api/post/app, src/postapp.js,
used by the MCP tool post_as_bot and the WebMCP tool of the same name): the same post as a
thread agent's post_message (see "Agent tools"), and allowed exactly where that is, in
channels whose Post messages switch is on (the channel's, else its team's) on a platform
whose bot is set up, for staff of the thread's teams. The switch says whether the app speaks
in a channel at all, whoever writes the text. The post names no person (who wrote it is
kept in app_posts only). The people picked with @ (or that the shared draft it sends
mentions) are mentioned as in a reply: <@U…> on Slack, in text and in the markdown
block, which keeps the id form as a mention; <@id> on Discord, whose allowed_mentions ping
those people and nobody else (no @everyone, no roles); on Teams an <at>Name</at> in the
bot's text with a mention entity by the person's Entra object id, which the Bot Framework
takes. A picked Teams bot stays plain text there: Teams gives no bot what another bot writes.
It comes back through ingestion by the app, and a thread agent there is told about it like
any message.
Editing. A message sent from opspile, or from the platform, can be edited in the inbox the
way it was sent (POST /api/edit, src/edit.js): hovering one of your own messages shows ✎,
which turns the bubble into an editor (Enter saves, Escape cancels, @ suggests people as in
the reply box). Your own messages are edited as you, with your token: Slack chat.update
(chat:write covers your own messages) and Teams PATCH on the message, which needs the
delegated ChannelMessage.ReadWrite; that takes the tenant's admin consent, so it is asked for
only once TEAMS_EDITS says it was given. A message you posted as the opspile app is recorded
in app_posts (user state, like drafts) and is yours to edit as the app, where the app may
still post: Slack chat.update with the bot token, the Teams bot's update activity, Discord
PATCH with the bot, with mentions as when it was posted. Nobody else's messages, and nothing on Discord as a person, are offered.
The accounts the message mentioned (@Name in its markdown, one person by that name, by their
account for the thread, picked as for a thread agent's post: src/post.js mentionsByName) stay mentioned. Nothing is written to the piles: the edit comes back through ingestion like every
edit made on the platform, and the inbox shows the new text and "(edited)" meanwhile. A Slack
change without edited (a bot's chat.update) that changes the text counts as an edit too.
Discord threads are read-only in the inbox: Discord lets only bots post and react through its API, and opspile acts as the person or not at all. They have a link to reply in Discord, no reply box, reactions or ✅; Discord's own ✅ by staff still resolves them.
Mentions. @ in a reply suggests the people opspile knows on the thread's platform, and
on Teams also the bots of the thread's Teams team (bot_teams in D1): the bots installed there,
which the cron lists from Graph (installedApps) for every Teams team with a channel in D1, and
any bot that posted, or was mentioned, in one of its channels. Bots people used come first. A
bot nobody has mentioned in a team yet can be mentioned from the inbox all the same; an @name
typed without picking it goes out as plain text, which the bot never receives. A picked bot is sent as an application mention,
which the bot receives exactly like a mention typed in Teams, from the person who wrote the
reply. That is how a team member is added from the inbox: @<admin bot> add someone@example.com
asks the bot that manages the Teams team, which answers in the thread (and in turn comes back
through ingestion). A bot is mentioned by the name people use for it (handle, the mention text),
which can differ from the name it posts under; for an installed bot that is the app's name,
which is what Teams writes when someone mentions it.
What follows the @ matches, in any case and without accents, the start of any word of a
person's name, of their other names (identities.aliases: the Slack login and display name,
the Discord username) or of their mail address before the @ (src/mention-match.js). People
outside Adobe who posted in the thread come first. Then come the Adobe people who know what the
thread is about (src/expertise.js): the thread's CONTENT vector (a re-threaded group's
mean; its title embedded while it has none yet) is matched against the staff messages of every
channel, the 50 closest that are no less than 0.65 similar and within 0.15 of the closest, and
each author gets what their five closest have over the weakest that counts (their messages on
every platform together, when D1 knows them as one person). The viewer's own messages do not
count. The affinities are kept for ten minutes per version of the thread's vectors, as every
typed letter asks again. Then anyone else who posted in the thread, the members of its channel,
and everyone else; among those, the higher affinity first, then a match at the start of the
name before one further in, then staff, then by name. The suggestion says "Adobe · related" for
such an expert. A person with several accounts is suggested once, by their account for the
thread (above, "One account a person"), and a mention of another of theirs goes to that one. The
members are channel_members: Slack's conversations.members (on Teams, the team's members
from Graph, where the app may list them) and whoever posted in the channel (on Teams, in any
channel of the team). Opening the suggestions lists a channel's members when that is a week
old, and looks up the members opspile has never seen (users.info), so a colleague who is in
the channel but never posted anywhere opspile reads can be mentioned too.
Mentioning someone who is not in the channel. Slack notifies nobody of a mention in a
channel they are not in, so a Slack message the inbox posts first adds the colleagues it
mentions to the channel (src/invite.js): as a person (POST /api/post, a reply or a new
thread), as the opspile app (POST /api/post/app) and from an agent's post_message. Only
Adobe people (staff on the account or its person), not bots, not the one who posts, and
up to ten a message: anyone else may be a customer, and is never put in another customer's
channel. The opspile Slack app adds them, one conversations.invite each (bot scopes
channels:write.invites, and groups:write.invites for private channels), so nobody has to
connect their account again; Slack answers already_in_channel for whoever is in already, so no
listing of members is read first. The post goes out either way, after the invites. The answer
says whom the bot added (invited) and whom Slack would not add (uninvited, with Slack's
error: in another workspace of the grid, a guest); the page says so in a toast, and the agent
and the MCP client in their result. A call that fails as such (the scope missing, the app not
in the channel) refuses nobody and is only logged (route: invite). Teams adds people to a team
through an approval flow of its own, and a message written in Slack itself is left to Slack,
which asks there; a thread agent still points out a mention of someone not in the channel
(src/agent/members.js).
The suggestions that open with the @ bring along everything they are picked from (pool of
GET /api/mentionable: the accounts of the workspace, one a person, each marked as in the thread, in the
channel or by its affinity, and the bots), and the page ranks what is typed after it itself,
with the Worker's own suggestMentions (in the markdown bundle), so completing a name asks the
Worker nothing. Until the pool is there, each typed letter asks the Worker, as before. The page
keeps a thread's pool for the page's life, and asks for it again in the background once it is
two minutes old, or when a message came in in its channel since (the room's thread event: it
may be an admin bot's confirmation that someone was added, or the platform's notice of it); the
suggestions follow the new pool when it comes. A name the kept pool has nobody for asks the
Worker at once, when the pool is more than five seconds old. The Worker, for a name that matches
nobody, lists the channel's members again when that is more than 30 seconds old, and waits for
it: someone just added who never posted is not in D1 until then (a Teams notice that they were
added names them, and so records them, but not every channel gets one), and is in the answer
when the listing found them. The Worker reads the thread's place first, then whether the viewer may see
it and all the suggestions need at once: one round trip to D1 after the place, not up to four.
The thread's header. Line 1 is the customer's icon with the platform's on it, the title, and the channel's name at the end where the line has room for it. The icon opens the thread on the platform; a re-threaded group lists its posts instead (the first and the latest eleven of a large one), each with its root's link. Hovering it names the customer and its type. Line 2 has two dropdowns, each named by what matters most for the thread:
- Tech stack: what the thread shows (
labels.techof its members, the likeliest first), then the rest of the channel's (channel_tech), then Clef's topics of the thread. A thread labeled before labels kept their technologies counts those it set the channel's highest probability for (channel_tech.thread), until it is labeled again. The dropdown is named by what the thread shows, else by its first topic, and only then by the channel's likeliest: that is the customer's, and a da.live link in one thread would otherwise name every other thread of the channel Document Authoring. - Links: the pages mentioned in the thread (Clairic's
affected_urlfirst, then the rules'urlfacts), then each Edge Delivery site's production URL (main--<site>--<org>.aem.live), then links to documentation and code (aem.live,github.com,adobe.comhosts without--), then tickets, CVEs (linked to cve.org) and support cases. The first one is the dropdown's name and opens with a click; ▾ lists them all.
A dropdown is left out when it has nothing to list. Whether the thread is resolved is not in the header: the ✅ on its root says so, and so do the list and the Send button's menu. After the dropdowns comes whom a thread agent suggests pulling in ("Agent suggests Lena Berg", below), when one did; the expert finder's own ranking is not shown.
Expert finder. Adobe colleagues worth pulling into a thread who are not in it yet, each
with why: "knows about this topic", "typically responds in about 4 hours" (GET /api/experts,
src/experts.js). The inbox does not show its ranking: line 2 of a thread's header named the
top one ("Lena Berg knows about this topic"), with a ▾ list of up to eight, each with ✕ for "not
the right person", and that was taken out, because its picks were mostly wrong and made the
inbox look less capable than it is. The header shows only what a thread agent picked, for a
reason (suggest_expert, below): it asks GET /api/experts?suggested=1, which reads those
suggestions and ranks and records nothing. The ranking is left for the thread agents
(list_experts) and the API; @ mentions rank people by what they know (src/expertise.js,
above) on their own.
It speaks only when someone is needed. Once Adobe has the thread in hand (three messages of
staff in it, IN_HAND: a first answer may only say who will look), its own people are the ones
to ask, and the most active in the channel
are mostly who is in it already: GET /api/experts then lists only an agent's suggestion, if
there is one. The exception is the person who was mentioned and has not posted since, who may be looking
for whom to hand it to. The list is still computed and recorded (held, below), for the training
to see whom the team pulled in anyway. A thread agent's list_experts always gets the full list.
Five signals add up, each scaled to its best person: the threads someone worked in this channel
and for this customer in the last 180 days (D1; the customer's counts half, as it mostly repeats
the channel's), the staff messages closest to the thread by meaning (CONTENT, as for the
mention suggestions) and by words (SEARCH, the title with BM25, for messages that have two
neighbouring words of it, plural or not, neither of them as common as "is": a search for any
word lets one word like "media" or "dev" say someone knows the topic). On 60 production
threads where a customer asked and staff answered, looking only at what was there before the
question, the first to answer was in the top 3 of 36 of them and in the top 5 of 38; the vectors
alone had 22 and 27, channel history alone 33 and 33. Labels' topics changed nothing and are not
used. Asking for neighbouring words, on 70 such threads, moved the first to answer from first
place on 13 of them to 18, from the top 3 of 25 to 28 and from the top 5 of 28 to 31; any two
words of the title got 12, 28 and 32, and no words at all 18, 27 and 30.
The fifth is similar threads: the channel threads of any channel closest to this one as a
whole (CONTENT thread vectors, similarThreads in src/expertise.js, those no less than 0.6
similar and within 0.08 of the closest), and each Adobe person who posted in them gets what their
three closest have over the weakest that counts. The meaning signal only sees what someone wrote:
whoever answered a customer's question with "is this still happening?" said nothing about the
topic, and only the thread as a whole says what it was about. On 65 production threads (August to
October, a customer asked without mentioning anyone from Adobe, Adobe answered; only what was there
before the question, the root message's vector as the thread's, and without the words, which
cannot be replayed), the first to answer was first, in the top 3 and in the top 5 of 17, 23 and 28
of them before, and 15, 27 and 30 with it; on the 48 of those where they had not worked the channel
before, 4, 8 and 11 before and 2, 11 and 13 with it. Someone whose similar threads count says so
("worked on 2 similar threads"), with how many, when they last posted in one and the topics of the
topic map those are in (similar); the list says which topics the thread's closest
threads are in (topics, for the agents: GET /api/experts answers without them).
A thread agent can say what the thread is about (list_experts with about), when its title and
first message say it vaguely (a customer's name for a feature is rarely the product's): the
meaning and similar threads go by that sentence instead of the thread's vector,
embedded as a query (with bge's Represent this sentence for searching relevant passages: in
front, QUERY_PREFIX: without it, a short sentence is closest to what every thread has in
common), and the words by it instead of the title. On the 65 threads with their titles as about
it ranked about as well (15, 23 and 30); on a vague one checked by hand, the colleague who had
answered the same question in another channel came first with it, and was not listed without it. list_topics gives an agent the topic map itself: the topics,
with what their customers need and their size, and for one of them the Adobe people who worked
its threads and could be pulled in (topicExperts, with the same exclusions as the list). Topic
names never name a customer, so they may say why someone fits to an agent of any team
(src/topicindex.js reads the newest map, at most every ten minutes per isolate).
The list leaves out the viewer, whoever posted or is mentioned in the thread (what is said in the thread itself does not count either), anyone who has not posted for 90 days, and anyone without an account on the thread's platform, who could not be mentioned. Someone with several accounts there is listed once, by their account for the thread ("One account a person"), with the signals of all of them.
A response time is the median between a mention and the person's next post in that thread (a
re-threaded group as one), over the last 180 days, in calendar time, nights and weekends
included. It is only said with at least three answered mentions, most of those made. "Not in this
channel" is only said with a complete list of the channel's members, as for find_member: most
channels have none, and then nothing is claimed. Such a person ranks lower, as their mention may not reach them; GET /api/experts lists the members in the background when that is a week old, as the mention
suggestions do. The answer has names, numbers and the topic map's names only, never message text
or a thread's title: the signals read
channels of every team, which neither the viewer nor a thread agent may read. It is kept for ten
minutes per version of the thread, per isolate.
A thread agent can pull someone in too: list_experts reads the same list, and suggest_expert
puts a colleague at the top of it, with its reason (expert_suggestions, migration 0045; user
state, like drafts). Nobody is told or mentioned; a person decides. It takes only staff with an
account on the thread's platform, three per thread at most, and a suggestion goes away once the
person is in the thread. It tells the open inboxes (a room event experts), and the header shows
it as "Agent suggests …": clicking it mentions them in the reply box, as picking them after @
does, and ▾ lists each suggestion with its reason when there are several. Nothing is sent: the
reply stays the person's to write.
What the finder learns from. History says who answered threads, but not what the finder
showed when, nor what people made of it, and the signals cannot be worked out again for a past
moment. So each list GET /api/experts computes for someone is kept (expert_events, migration
0047, src/expertevents.js): every candidate it ranked, up to 20 and not only the eight it
answers with, by account and person, with the numbers that placed them (each signal scaled and as
found, the score, topic, channel membership, response times, an agent's suggestion) and the
finder's version (FINDER in src/experts.js), and held when it answered with none of it,
never what anyone wrote. The answer has its id (list), and the page said what happened to it
(POST /api/experts/event): shown once each time the thread was opened with it in the header,
open for its ▾ list, pick of someone (their place comes from the list, not the page),
reject for ✕, and dismiss when the ▾ list was closed, or left for another thread or page,
with nobody picked: the clearest "none of these". Since the header lost the finder the inbox
sends none of these and asks for no lists, so new ones come only from other callers of the API;
the rejects recorded before still leave those people out of that person's lists of the thread.
Whom the person mentioned afterwards, and
who answered, are in mentions and messages already, so they are joined in training, not
recorded twice. Nothing is recorded while viewing as someone else. GET /api/experts/events
(admin team) gives them out a page at a time; npm run rank -- experts, and the weekly run,
fetch them into .rank/expert-events.jsonl, and the cron deletes them after 90 days
(EXPERT_EVENTS_DAYS), all but the rejects, which the finder goes by (migration 0048 indexes
them).
Reactions are made the same way, from a small picker or by clicking a reaction a message already has (which adds yours, or takes it off). The Worker records a reaction in D1 as soon as the platform accepts it, exactly as the platform's event will, so it shows at once; the event then changes nothing. A removal only takes away a reaction made before it, and a later add moves the reaction's time forward, so a remove-and-add comes out right in either order.
✅ or ✔️ on a thread's root message (white_check_mark / heavy_check_mark on Slack, ✅ /
✔️ on Teams) by a staff member means resolved. e, or the Resolve button that appears
when the pointer rests on Send for a second, adds ✅ as the person; a ✅ put on a reply in the
inbox goes on the root as well (and comes off both). It lands on the root of the
entry's latest platform thread; undo takes the person's own ✅ off again (someone else's ✅ is
theirs to remove). It overrides Clef's resolved_p, and a thread resolved this way is not in
the "you", "waiting" or "silent" status piles.
A customer message after the ✅ reopens the thread: it is open until Clef has labeled the thread with that message in it, and from then on Clef decides again (so "thanks, works now" does not keep it open), until staff resolve it again, which moves the ✅ (off and on again) so that it is newer than the customer's message. Staff and bot messages after the ✅ change nothing. A ✅ without a time (from a backfilled Slack message) counts from the root message's time.
The models see reactions too: each message is followed by who reacted with what (customer or Adobe staff), and a reaction relabels the thread, so a customer's ✅ or 👍 on the answer can make Clef call it resolved.
The ✎ button next to the list's bell opens a composer in the pane, where a thread would show.
What it posts goes through POST /api/post as a reply does, as the person, with the same
text, mentions and files, but names a channel instead of a thread, so no thread exists yet.
The inbox has no channel filter: the channel is known when the customer and platform filters
leave one channel of the view on screen (channelsInView, over GET /api/channels). The other
filters pick threads, which can be in any channel. Otherwise the composer's header searches
every channel the person may post in, the view's first, and Post stays off until one is
picked; POST /api/post itself refuses a post that names neither a thread nor a channel.
Alert and ignored channels are not offered, nor is Discord, where the inbox cannot post as the
person. Nothing is written to D1: the thread comes back through ingestion like any message,
some ten seconds later (one in ten takes 20 seconds or more: the normalize queue's delivery), and
the page opens it once /api/thread has it, as soon as the team's room announces it. Until then
the pane shows the thread as it will look, from the moment Post is pressed: its title, channel
and message, with the bar a reply being sent has, and a reply box: what is sent from it shows at
once too, and goes out as soon as the platform has named the first message, into that thread on
the platform (POST /api/post with channel and root), in the order it was written. Nothing
waits for the inbox to have the thread. A post that fails takes the replies written meanwhile
back into the composer, after its text. The channels to pick from are asked for when the pointer
reaches ✎.
A reply can wait as a draft: Save draft appears above Send, next to Resolve, when the pointer rests on Send for a second (or Send is pressed for a second on touch), and ⌘S / Ctrl+S in the reply box does the same. An inbox entry carries at most one draft, and it is shared: everyone in the teams that follow the channel (the one it is filed under, and those it is shared with) sees it, and can change, send or discard it. Admins, who can open any thread, see its draft but change only their own teams' drafts.
- Drafts live in D1 (
drafts), keyed by the inbox entry (a re-threaded group's first thread). They are user state, like watermarks: rebuilding the index from pile 2 keeps them. A draft on a thread that is later folded into a group shows on the group, and moves to the group's key when it is saved again. - A draft's
updated_atis its version. A save names the version its text was written from, and a draft that changed meanwhile is not overwritten: the colleague's draft is shown, with an offer to overwrite it. A discard does the same. A save from a draft that is gone (discarded or sent) saves nothing either, and offers to save it as a new one. - The draft waits on its own line under the reply box, never in it: who saved it (a colleague, you, or the thread agent), when, and how it starts, on one line, rendered as the message will be (its Markdown, mentions and emoji, as in the thread). View draft opens the line to all of it, and Hide draft closes it again, with a click or a tap alike; a mouse resting on the line for 300 ms opens it too, until it leaves. The line grows upwards, so it stays under the pointer, instead of moving away from it and closing again, over and over. The reply box is yours, and a draft never replaces what you typed. Edit takes the draft into the box to edit; with text there already, Add to reply puts it after that text. View draft, Edit and Discard are icons (their names are their labels and tooltips), on the line's first row with who saved the draft; the sends are labelled, past a hairline, and get a row of their own at the bottom of a narrow line.
- The line sends the draft as it is: Send as me, or Send as bot (as the opspile app,
POST /api/post/app) where the thread says the viewer may (can_post_as_bot: the channel's Post messages switch is on and its platform's bot is set up). On Discord, and before the Slack or Teams account is connected, Send as bot is the only way to send it from the inbox. - Sending a reply written from the draft uses it up (unless it changed since), from the line or from the box. Saves and discards are fanned out over the rooms of every team that follows the channel, so every open inbox follows at once; a reply box that took the draft in and still holds it unchanged follows the draft, one with your own changes keeps them (and the line says the draft changed since). What you type and do not save stays with its thread in the tab.
- On a phone (a screen up to 699 px wide) the draft's line and the command cards (see
"Agent tools") would leave the thread a few lines of the screen, so they are in a drawer
(
renderLines,ui/inbox.js). Under the reply box only its tab shows: one line that says what waits there ("1 command to decide", "Draft by …"), outlined as a command's card while one waits for a decision. A tap opens the lines as a sheet from the screen's bottom edge, over the thread, the rest dimmed. ✕, a tap beside the sheet and Escape close it; so do Edit and the sends, which happen in the thread, and opening another thread. Run, Deny, View draft and Discard leave it open, and Undo of a discard shows over it. It is the same markup on every screen: on a wider one the stylesheet shows no tab and makes the drawer no box (display: contents), so the lines are the reply box's own. - In the inbox, an entry with a draft shows a pencil in place of its status dot (as a resolved one shows ✓), in the status's color and outlined while the team has not read the thread; the personal inbox shows none. The Draft card filters by drafts saved last by you, by a colleague, or none. A draft does not change the status pile. The outline is the pencil's own shape, not the ring a dot has for "our move": a ringed pencil would mean two things in one column. A draft that waits on a thread nobody has read yet is what the outline points out; "our move" stays in the pencil's tooltip, after the draft.
- A draft can carry sources (
drafts.sources, src/sources.js): what an agent says it rests on, as{ url, title?, quote? }, at most 10, http(s) only. They are for the person reviewing the draft: they stay in the inbox and the tools, and no reply ever carries them. Whoever may see a draft sees its sources; sign-in (Cloudflare Access) keeps that to staff. The tools (update_draft,post_draft) replace a draft's sources with each save, so they describe its text; a person's edit in the inbox keeps them, and Undo of a discard brings them back. - The sources sit on a row of the draft's line. Resting the pointer on one (or moving the
keyboard focus to it) shows it beside the draft's line, with the agent's quote above it; the
reply box keeps what it held (the line's View draft shows all of the draft); clicking it
opens it in a new tab.
GET /api/draft/sourcetells the inbox how: a browser cannot tell why a frame stayed empty, so the Worker fetches the page, without the viewer's credentials, and reads itsX-Frame-Optionsandframe-ancestors. A page that may be framed shows in a sandboxed, credentialless frame (no top navigation, no forms, no referrer). Markdown and text are rendered, and so is what a page that refuses frames says in its HTML (pageMarkdown, which sees what the server sent, not what scripts build). Anything else, such as a page behind a sign-in, is a reason and a link. The route fetches only the sources of the thread's current draft, follows at most 4 redirects, each checked (public names, never an address or the inbox's own host), and reads at most 512 KB. - A source that links to a Slack or Teams message is never fetched (the platform's page needs a
sign-in a frame does not have). It is the message as opspile has it (
view: 'message', src/messagelinks.js), for a viewer who may see its channel, and the inbox shows it as the card a link to that message in a thread shows (src/ui-markdown.js), in a popover only as tall as the card. Its time shows the message in the inbox: where it is in the open thread, else its thread, opened at it. A message opspile does not have, or the viewer may not see, is a reason.
Resting the pointer on a message for 300 ms shows the quick reactions ✅ 👀 🙏 👍 😀 🚀 🛟. Three of them steer the inbox, on any message of the thread:
- 👀 seen: the person who put it is on it. For them it counts like having posted (higher); for the rest of the team, a colleague's 👀 since the customer last wrote puts the thread in "team on it" and halves its score in the team view, unless it mentions them.
- 🙏 thanks: after the last message, nothing is pending (
waiting_onis nobody). - 🛟 help: a colleague asks the team to look: the thread is in "you" for everyone else and its score in the team view doubles, until it is resolved.
One agent per inbox entry (a thread, or a re-threaded group under its first thread's key),
in the channels where agents are on: the admin team and team managers switch them per channel
in the knowledge-base editor, and their team's default and AGENT_CHANNELS turn on the rest. Each is a Durable Object
(ThreadAgent) running Pi through
the Agents SDK's PiHarness: the transcript, tool calls and retries live in the object's
SQLite, so a run survives evictions and deploys, and a model request that fails is retried.
Prompt caching asks for the one-hour TTL (cacheRetention: 'long'). Turns are often minutes
apart, past pi's five-minute default, and Bedrock bills a later hit inside the hour as a read.
But every write costs 2× the input price for an hour against 1.25× for five minutes, so whether
that pays depends on how far apart turns are. Experiments (src/experiments.js) settle it:
for a fixed window, each thread is in one arm by a hash of its key (salted with the experiment,
the same arm all along; knowledge-base agents take no part), and the provider asks for the arm's
retention at every request (retainingApi, src/agent/model.js), so an agent that is running
at the window's edges switches on its next request. The cache experiment puts half the threads
on short from 2026-11-07 to 2026-11-14 (UTC); outside it every thread is on long. What an
agent in an arm uses is counted for COGS with the arm in the detail (below).
When it runs. Every new message of the entry pokes the agent through the NORMALIZE queue (after 15 seconds, so a burst arrives together; a new root's after it is re-threaded, so that a root folded into a group pokes the group's agent, not one of its own); it reads the messages after its watermark from pile 2 and gets them as one input, the whole thread on first contact. A new message during a run joins that run. A day after the latest message, its own posts included, a schedule wakes it once more to wrap up: file what was learned, or put a follow-up in the draft. It is also told, as an input of their own, that a person discarded its draft, and how a command it asked a person to run ended (see "Agent tools").
In the inbox. The open thread shows the agent's latest turn under its last message,
on the Adobe side, as "Opspile Agent" with opspile's icon: while it works, a spinner and what
it is doing (reading the thread, checking a web page, updating the draft…); once done, its
closing note for the team, which the prompt asks for at the end of every turn (1 to 3
sentences: what it found and what it changed). It is outlined, not filled, since nobody posted
it. The turn is read from the agent's transcript (src/agent/turn.js). A finished one is also
kept in D1 (agent_turns, src/agent/kept.js) and comes with the thread (agent_turn in
GET /api/thread, and so in the threads the inbox fetches ahead), so it shows with the messages
instead of from the Durable Object a moment after them. The agent writes it a moment after its
answer (or a failed request), and whenever status() finds a finished turn it has not kept (one
that stopped half way and counts as failed now). The page asks GET /api/agent when the thread
opens or changes, and every few seconds while the agent works or a new message waits for it;
what the agent says wins over the kept turn when it is about a later one. A turn in progress is
only ever the agent's to say, so it comes in after the messages. Its avatar opens the sidebar
(below), and its name the agent settings of the thread's team (Agents, its thread agents, at the
thread's channel).
Whether it shows is each person's own choice, in their settings (person_settings): traces,
on unless they turn it off, for every channel, and channelTraces, by channel id, where they
chose otherwise, which wins. The inbox's user menu changes both (the open thread's channel);
Agents does too, a select on each channel with an agent. Hidden, the page does not ask
GET /api/agent at all.
The sidebar. Anyone who may work a thread can talk to its agent in the inbox, about the
thread, without the channel seeing any of it (POST /api/agent/sidebar, src/sidebar.js). Each
message goes into the agent's own conversation as an input of its own (ThreadAgent.sidebar,
src/agent/sidebar.js), after the run it is in, never steered into one; so the agent remembers
it in later turns. New messages of the thread do not steer into it either: while a sidebar
message is pending, a delivery follows it as a run of its own. It goes into the thread's history
too: sidebar_messages in D1, the person's message when it is sent, the agent's answer when the
turn ends (the turn's closing text, up to 8000 characters, kept by keepTurn instead of the
thread's note, which stays the turn before). Every finished sidebar turn in the transcript is
kept, not only the latest: an input queued behind one starts at the boundary its answer is
committed at.
Nothing of it is posted: while the agent's latest input is a sidebar message, post_message,
add_reaction and request_membership refuse (the admin bots answer in the channel), and so do
they when the transcript cannot be read. The input starts with a line that names the message,
and only an input that starts with it is one: a customer quoting it is told inside a delivery.
The inboxes of the thread's teams hear of each message and answer through the team's room
({ type: 'sidebar' }).
In the inbox, the agent's icon enters sidebar mode: its avatar on its note, the button with
it beside the reply box, Continue in the sidebar on an earlier conversation, or a. Each run
of sidebar messages is one line between the thread's messages where it happened ("Lena Berg had
a sidebar conversation with Thread Agent · 4 messages · not posted"), for everyone who sees the
thread in the inbox. Where the conversation itself shows depends on the room (ui/inbox.js,
"the sidebar"):
- Beside the thread, where the list, the thread and a panel fit side by side: 1180px from
the list's left edge to the content area's right (the list's least 380, 440 for the thread,
the panel's least 360), measured, because the filters fold to a rail on some screens. While
the panel shows, the content area is shared 3 : 5 : 3 between the list, the thread and the
panel, so the thread stays the widest by far and the list gives up room too, down to its
least width; the panel is 360px to 560px. The panel (
#side) has every sidebar conversation of the thread, each under when it started, the agent working on its answer, and a box of its own, so the thread keeps its reply box and can be read and answered meanwhile. A line in the thread opens the panel at its conversation, also for someone who may not talk to the agent (viewing as someone else): they read there, without a box. - In the thread, on a narrower screen: a line unfolds its conversation where it is, the one you are in stays open at the bottom, with the agent working on its answer, and the reply box turns into the sidebar's.
Both are purple, say that only the team sees it and that nothing goes to the platform, and are left by ✕ (and Esc, and opening another thread). A window that grows or shrinks moves the sidebar you are in from one to the other, with what is typed in its box.
Traces. The admin team sees everything an agent did with the MCP tool get_thread_trace:
the thread's messages and the agent's steps in one timeline (src/agent/trace.js, read from the
same transcript through the agent's trace()): each input, its thinking and text, every tool
call with its arguments and result, the MCP server calls its code mode scripts made (pi keeps
no record of a script's calls, so the agent keeps its latest 200 in its storage, mcp-calls:
when, which tool, the arguments clipped, the outcome, never the result), prompt and tool
changes, and the tokens and cost per step. An agent that does not answer leaves the thread's
messages, with agent.error.
It covers the transcript since the agent's last reset; long text is clipped.
Prompt injection. Before an agent sees a message that Adobe staff did not write (customers, partners, bots), Clef answers one yes/no question: does it try to instruct or reconfigure an AI that reads it? A message at or above p = 0.5, or one the check could not answer, is withheld: the agent only learns that it exists. Verdicts are kept per message in the agent's storage, so each message is checked once. A check that could not answer is not a verdict: the queue tries the delivery again a few times before the agent is told the message is withheld, and reading the thread asks again. Text the agent does see is fenced and labelled as written by people outside its control.
What it can do.
| Tool | |
|---|---|
read_thread |
the whole entry, flagged messages withheld; with thread (a key, or a link to one of its messages as people paste them), another conversation of the same channel, its first and latest 100 messages |
read_file |
a file attached to a message of the entry (src/agent/files.js), named by its link as the message shows it or by a Slack file id, never any other: fetched with the app's credentials (the Slack bot's files.info and token, the Discord bot's refreshed links, the copies of Teams files in pile 1), up to 10 MB. Images (PNG, JPEG, GIF, WebP up to 3.75 MB and 8000 pixels a side, read from their headers) go to the model as images, larger ones through the Markdown conversion; an image the model would refuse never reaches it (src/agent/images.js), since the API refuses the whole request and the image would stay in the history: the provider also leaves such images out of every request (over 2000 pixels a side when it has more than 20 images); plain text as it is; PDF, Word, Excel, OpenDocument, Numbers, HTML, XML and CSV through Workers AI's Markdown conversion (AI.toMarkdown). Text comes in parts of 30,000 characters (offset), and every part goes through Clef's injection check first, as a message would; a flagged file is withheld. Teams files are read from their copies in pile 1 (one without a copy cannot be read). Files of withheld messages cannot be read. |
list_channel_threads |
the other conversations of the agent's channel, latest activity first (a re-threaded group as one, no join messages), paged by before; with query, those whose title has it, then those whose root is similar to it (Vectorize, the roots' embeddings from re-threading). Only its own channel: an agent reads another channel's threads only with read_any_thread |
read_any_thread |
only where it is turned on (see "Agent tools"): any thread of the inbox, by its key or a link to one of its messages, its first and latest 99 messages, flagged messages withheld |
list_experts, list_topics, suggest_expert |
colleagues not in the thread yet who could help, with the similar threads they worked on and the topics those are in (about: what the thread is about, in the agent's words), the topic map's topics and who works each, and a suggestion to the team to pull one in (see "Expert finder"). It tells nobody: a person decides |
find_member |
whether someone is in the agent's channel (on Teams, its Teams team), by name or mail address (src/mention-match.js), from the platform's list of members in channel_members, listed again when over an hour old; two accounts of one mail address are one match. With no complete list to go by (a failed listing, a Teams shared channel, over 2000 members) it says it cannot tell |
get_draft, update_draft, discard_draft |
the entry's shared draft and its sources, saved as the person opspile agent with the same version check as a person's save. discard_draft removes a draft the agent saved last when it is out of date (staff answered themselves); a colleague's draft it leaves alone. Unless post_message is on, this and suggest_expert are its only writes outside the knowledge bases: it cannot post, react or resolve. A person reads the draft and sends it. |
post_message |
only where it is turned on (see "Agent tools"): posts a reply in the thread as the opspile app |
request_membership |
only where it is turned on (see "Agent tools"): asks the platform's admin bot to add people to the channel, or remove them, for the author of a message of the thread that asked; the bot's admins approve |
add_reaction |
with post_message: reacts to a message of the thread with an emoji, as the opspile app, on Slack and Discord |
bash |
only where the computer is turned on (see "Agent tools"): a shell with files of its own |
request_bash |
with the computer: asks a person on the team to run a command in that shell, with values the person fills in; it answers at once, and the agent is told later how it ended |
git_ls, git_read, git_grep, git_log |
the team's and the customer's knowledge base (kb-team, kb-org), every other team's (kb-team-<slug>), and the GitHub repositories listed for the team, the customer or everyone in agent_repos (to start: this repository) |
kb_write |
create, replace or delete concepts in one commit, in the team's or the customer's knowledge base or that of a team the channel is shared with; the other teams' are read only |
codemode |
a JavaScript script that calls the tools above, and the team's and the customer's MCP servers, as await tools.<name>(args); only what the script outputs reaches the model |
Mentions of people who are not in. When a message the agent is told about mentions someone
on the platform (a via = 'id' row in mentions; a name in the text does not count) who is not
in the channel, or on Teams the Teams team, the agent is told so below the messages and says so in
its note, so a person can add them. src/agent/members.js goes by channel_members, and lists
them again first when that is over an hour old, so someone added since is not called out. It says
nothing when it cannot be sure: a failed listing (on Teams, without TeamMember.Read.All), nobody
listed, a Teams shared channel (the list is the team's, and others are in the channel), or over
2000 members. The member of the mention suggestions is no use for this: it is also true of
anyone who posted.
GitHub repositories are read from Artifacts mirrors: the cron imports each listed repository into a fresh Artifacts repo, promotes it once it is readable and deletes the mirror it replaces, at most every six hours. Reads need no GitHub token and hit no rate limit; private repositories cannot be imported yet.
Code mode and MCP servers. The codemode tool is pi-durable's code mode: the script runs
in a QuickJS sandbox on the agent's own thread (createInlineCodemodeSandbox; one object per
thread, so a script that computes holds up only its own agent, for at most its interrupt
budget), every call it makes is a nested tool call, and a script interrupted by an eviction
resumes where it was without making its finished calls again. Each ops team and each customer
can have MCP servers (mcp_servers, maintained in the knowledge-base editor next to the
per-channel switch; credentials are request headers sealed with TOKEN_KEY, or an OAuth
grant, below). Each server is on or off for every channel of its team or customer, and a
channel can have a switch of its own that wins (mcp_server_switches), as the global servers'
do. An agent gets the servers of its thread's team and customer that are on in its channel, and
connects only to those: each tool becomes mcp__<server>__<tool>, callable only from scripts, never
offered to the model as a tool of its own, and each server is a code mode namespace that
describeNamespace() and searchTools() find. So a server with a hundred tools costs the
prompt a few lines, and a script can call several and filter what they return. A server's
allow list limits the tools agents get; a tool that declares itself read-only may run again
after an eviction, any other is reported to the model as interrupted instead. The list of
servers and tools (no credentials) is kept in the agent's storage, so a restarted agent has the
same tools at once; before the agent is told anything it is checked against D1, and the servers
are asked again when it changed or after an hour. MCP results come from systems outside
opspile; the prompt says they are data, not instructions, like customer messages.
MCP servers with OAuth. A server can sign in with OAuth instead of headers (auth
'oauth', src/agent/mcpoauth.js); opspile is then the OAuth client. Someone who manages the
server connects it in the editor: opspile asks the server without credentials and follows
its 401 to the protected resource metadata (RFC 9728) and its authorization server's metadata
(RFC 8414 or OpenID), and signs in as the client staff gave, else with opspile's client ID
metadata document (/oauth/client.json, public; its URL is the client id) where the
authorization server advertises client_id_metadata_document_supported, else as a client it
registers there (RFC 7591); it sends the browser to the authorization endpoint with PKCE
(S256), a resource indicator and a state whose hash finds the server again. The callback,
/api/kb/mcp/oauth/callback, is behind Access, and only the person who started may finish, within ten minutes; the grant (client,
tokens, the authorization server's metadata) is kept sealed in mcp_servers.oauth, with who
connected it and when in plain columns. The agents act as that person at the server.
Agents and the editor's check get the token from the transport's auth provider: the stored one
until two minutes before it expires, else a refresh, and a refresh too when the server answers
401. Many agents share one grant and refresh tokens may be single-use (opspile's own are), so
only the holder of a 30-second lease in D1 (oauth_lock) refreshes, and the others wait and
read its result. Disconnecting and connecting again do not wait for the lease, so a refresh only
writes over the grant it started from: one that comes back to another grant leaves it alone, and
revokes the tokens it got. A grant the server rejects (invalid_grant) is dropped with the
reason, which the editor and the agents' prompt show until someone connects again; a new URL,
client or scope, a switch back to headers, Disconnect and deleting the server revoke it where the
server has a revocation endpoint. The cron refreshes grants nobody refreshed for a week, so
refresh tokens that expire unused (opspile's own after 30 days) stay alive. opspile's own
/mcp is such a server, so agents can be given what a person sees in the inbox.
Some servers are not a team's or a customer's but opspile's own, built in
(src/agent/globals.js) and set up once: browser and github. A new team starts
with both on; otherwise they are off until the admin team or a team's managers turn one on, for an ops team or for a single channel
(mcp_global_switches; a channel's switch wins over its team's, and a channel without one
follows its team). An agent gets those on for its thread's team or channel, next to the team's
and the customer's, as mcp__browser, mcp__github; its id there is global:<key>. Their
tools are those the last check found (mcp_global_tools), so an agent does not ask a
server or mint a token just to list them; the editor's Check and the cron, once a day,
ask again. The prompt says that these are opspile's own, and that customer data does not go
into websites or URLs that are not the customer's or Adobe's.
browser: Cloudflare Browser Run. Cloudflare's own hosted
Browser Run MCP server
(https://browser.mcp.cloudflare.com/mcp), with an account API token that may only use Browser
Run (BROWSER_RUN_TOKEN, Browser Run Write; account-scoped, so the server finds the account
itself). Its tools load a page in Browser Run's Chrome and return it as markdown or HTML, a
screenshot, a PDF, its links, elements by CSS selector or JSON extracted from it, or crawl a
site. Each call loads the page anew, signed in nowhere: there is no session to click through.
The tools that list and kill the account's browser sessions are left out of what agents get.
github: GitHub's MCP server, read-only, as the opspile GitHub App. The server is GitHub's
hosted one, at https://api.githubcopilot.com/mcp/readonly with X-MCP-Readonly and
X-MCP-Lockdown (public issue content from people without push access stays hidden: text
from strangers that nobody screened). It authenticates with installation tokens of the
opspile GitHub App (src/github.js), so it reads exactly the repositories the app is
installed on, all of an account's or a selection, and nothing else; the app asks for read
permissions only. The app is public, so that customers' organizations can install it too;
since anyone can, an installation is used only after the admin team allowed it in the editor
(github_installations), and a revoked one gets no more tokens. The admin team creates the app from the editor with GitHub's manifest flow:
the page posts the manifest to GitHub, GitHub asks to confirm and sends them back with a code
(and a state opspile signed for them), and opspile trades the code for the app and its private
key, which stays in D1 (github_apps), sealed with TOKEN_KEY; then GitHub's installation
page. Nobody copies a key. A token lives an hour: the Worker signs a JWT with the key (RS256,
Web Crypto; GitHub's PKCS#1 key is wrapped as PKCS#8), trades it for a token per installation,
keeps it per isolate until shortly before it expires, and an agent's client is replaced
before its token runs out. An app installed on several accounts is a server per account,
github_<account>.
Writes clone the knowledge base into memory (isomorphic-git), apply the change, rewrite the
indexes and the log, commit and push with a short-lived write token. Many agents write the
same team knowledge base, so a rejected push starts over from a fresh clone; a change that
names the blob it was written from (base) fails instead of overwriting a newer version.
Seeding. Staff can teach an agent: POST /api/agent/seed (or the MCP tool seed_agent)
with a thread hands the note to that thread's agent, and with a team and a customer slug to a
knowledge-base agent (kb:<team>:<org>) that has the same tools minus the thread and the
draft. Staff notes are trusted and not checked by Clef.
Left at that, the agent picks the knowledge base. kb (team or org, on either form) chooses
it, and code holds the agent to it: the note goes into a session of the same Durable Object
pinned to that knowledge base (seed-pin:<session> in its storage, src/agent/seed.js), and
kb_write checks the calling session, so a code-mode script is refused like the model. A session
and not a flag, because the agent's conversation goes on meanwhile: a turn already running may
write anywhere, and a new customer message steers into whatever is running. The session starts
without the transcript, reads the thread with read_thread when it needs to, and its turns show
in neither the thread's latest turn nor the trace; the thread's agent meets the note in the
knowledge base. A thread's kb is only ever that thread's own team or customer. kb leaves
agent names alone: { team, kb: "team" } needs no customer and reaches kb:<team>: (the org
left empty), which has no kb-org at all, and only the chosen knowledge base is created when it
is missing.
Discarded drafts. When a person discards a draft the agent saved (DELETE /api/draft, the
inbox's Discard), the agent is told who did and what the draft said, as feedback: it looks
for why it was not wanted, does not save it again, and files a lesson in kb-team when there is
one. It hears it 30 seconds later, after the inbox's Undo has run out, and not at all when the
draft came back (the same text saved again by a person; src/agent/discard.js). Drafts a person saved last, drafts a reply used up, and drafts the agent discarded itself are not told.
The model. Claude Opus 5.5 on Amazon Bedrock (the inference profile
us.anthropic.claude-opus-5-5 in us-east-2), through Cloudflare AI Gateway: Bedrock's
bedrock-runtime endpoint speaks the Anthropic Messages API at /anthropic, AI Gateway
passes it through, and the Bedrock API key goes in Authorization: Bearer. (bedrock-mantle
speaks it too, but serves only older Claude models in that region.) The gateway meters every
request but keeps no logs: prompts carry customer messages, so request logging is off
(turn it off on any new gateway too).
Replay. Agents are not part of the replay path. Their output is user state (drafts) or pile 3 (git history); replaying pile 1 pokes them, but they find nothing past their watermark.
Four tools are off until they are turned on (a new team starts with computer and
extraction_rules on for the team). Three by the admin team or a team's managers, as
the global MCP servers are: for an ops team or a single channel (agent_tool_switches; a channel's switch wins
over its team's), in the knowledge-base editor (Tools and MCP servers, src/agent/agenttools.js).
Before an agent is told anything it reads the switches of its thread's team and channel; the
tools it has are kept in its storage and installed as their own extension (opspile-tools,
right after opspile's own), which a change replaces in place. Each comes with a section of the
system prompt on how to use it.
post_message. The agent posts a reply in the latest platform thread of its entry right
away (src/agent/posting.js), as the opspile app, never as a person: on Slack with the bot
token (chat.postMessage in the thread, the Markdown in a markdown block, which Slack renders
itself, and text in mrkdwn for notifications and for the message file), on Discord with the bot, in the Discord thread when
the conversation is one and as an inline reply to its first message otherwise, with mentions
switched off (but for the people a person posting as the app picked, see "Replying"). On Teams with the opspile bot (src/teamsbot.js): Microsoft Graph lets no app
post to a channel, but a Bot Framework bot can, through the Bot Connector, in a team it is
installed in. The bot is the opspile Entra app itself, single tenant, registered as an Azure
Bot with the Teams channel, and packaged as a notification-only Teams app in the org catalog
(TEAMS_APP_ID). A post replies in the channel thread (<channel id>;messageid=<root id>)
with a Bot Framework token of the app; when the bot is not in the team yet, opspile installs
the app there through Graph (TeamsAppInstallation.ReadWriteSelfForTeam.All, which lets the
app install only itself) and posts again. Its messaging endpoint (/teams/bot) answers every
activity and keeps nothing: opspile reads Teams through Graph. A platform whose bot is not set
up does not get the tool. The prompt asks the agent
to post only what the team would send as it is, and to keep anything uncertain or sensitive in
the draft. A post may name the draft version it sends, which uses the draft up. The reply comes
back through the platform's events like any message; the agent keeps the ids of its latest
posts and is not told about them, so it never answers itself.
With post_message the agent also reacts (add_reaction, reactAsApp in
src/agent/posting.js) to a message of its entry, as the app: on Slack reactions.add with the
bot token (bot scope reactions:write), on Discord PUT …/reactions/<emoji>/@me as the bot
(Add Reactions), in the channel or Discord thread the message is in. It names an emoji by its
Slack name in any platform's spelling or as the character; Slack also takes a workspace's
own, Discord gets the character. Not on Teams: Graph's setReaction is delegated only, and the
Bot Framework has no call for a bot to react, so the prompt tells an agent on Teams it cannot.
👀, 🙏, 🛟 and ✅ (✔️) are refused: they steer the inbox or resolve a thread, and a bot's 🙏
would clear "waiting on us". The reaction comes back through the platform's events like any
other; the agent is not told about reactions.
computer. pi's own bash tool, run on Cloudflare Computer
(@cloudflare/computer, src/agent/computer.js). The ThreadAgent object holds a Workspace: a
filesystem in its own SQLite, so each thread has a working directory (/workspace) whose files
last from one turn to the next, and nobody else's. Commands run in a WorkerShellBackend:
just-bash in a Dynamic Worker (the LOADER
binding), which reaches the files back over Workers RPC through WorkspaceServiceProxy (the
Worker exports it) and the object's __getWorkspaceStub(). The shell has the text tools (cat,
grep, sed, awk, sort, find, jq…) and curl, with network access through the Worker's
SecretGateway (egress http-gateway, no filter; see "Computer secrets" below), and no npm,
node or python. What curl fetches is untrusted, like
customer messages; nothing stops an agent from sending what it read, so the prompt says not to
send anything to sites that are not the customer's or Adobe's. The
bridge is an ExecutionEnv for pi-durable whose exec is workspace.runtime.exec; pi's file
tools are not installed, so the rest of the environment answers not_supported.
/workspace/uploads is read-only and holds the files attached to the thread's messages
(src/threadfiles.js). Every file is copied to the FILES bucket, under
<thread>/<sha256 of the file's id>, by a queue job when its message arrives, or at the latest
before the agent is told about it. It is kept for 30 days: the bucket's lifecycle rule deletes
it, and a file whose message is older is neither copied nor shown. The mount is filled by
uploadsMount into the object's SQLite, so each change of the list takes a new Workspace, and a
schedule (expireUploads) empties it when the oldest file expires, even if the agent sleeps.
Filling the mount makes /workspace, and the Workspace refuses any change to a directory a
read-only mount is in, even a mkdir -p of one that exists: the computer makes /workspace
only when it is missing (ensureCwd).
Files of withheld messages stay out of it, as they do of read_file, but a file itself is not
checked for prompt injection here, and a raw HAR holds its recorder's credentials: the prompt
says not to quote them, and code does not stop the shell from reading or sending them.
Computer secrets. API keys and tokens of a team or a customer (agent_secrets, migration
0060, src/agent/secrets.js), kept in the Computer tool of its knowledge base by whoever edits
it (/api/kb/tools/secrets, src/secretadmin.js), sealed with TOKEN_KEY and never shown
again. A team's wins over the customer's of the same name. The agent never holds a value: its
prompt lists the names, hosts and descriptions, and pi's bash gets each as a variable whose
value is a placeholder ($ACME_API_KEY is opspile-secret-ACME_API_KEY), through the tool's
prepare. The Workspace's egress is http-gateway with SecretGateway (src/agent/gateway.js,
exported by the Worker), a WorkerEntrypoint that becomes the Dynamic Worker's
globalOutbound: every fetch of the shell, curl's too, arrives there, with the agent's name in
its props, which the shell cannot touch. A request that names no placeholder goes out as it is.
One that does is sent with the values in place of the placeholders (in the URL, the headers, a
Basic Authorization, a textual body up to 1 MiB; percent-encoded in the URL and a form body,
escaped in JSON and XML, a "form" that parses as JSON as JSON, which is what curl -d sends
without a content type; so a + does not arrive as a space), only over HTTPS to a host the secret lists,
and with redirect: manual, so a redirect comes back to the shell and its next request is
checked again; what comes back has the values replaced by their placeholders, in its headers and
its body, as it is or escaped (scrubber, which also finds a value split between chunks). The
prompt and the shell's variables come from the agent's team and customer as they are now too,
not from its cached target. A placeholder for another
host, over HTTP, or for a secret the agent's team and customer do not have is refused with a 403,
and nothing is sent. The gateway reads the secrets from D1 on each such request, by the agent's
team and customer as they are now, so a deleted secret or a moved thread counts at once. What it
does not stop: a host on the list that publishes what it receives (the agent could post the key
into a ticket of the API it is for), and a value the host sends back encoded otherwise.
Commands a person runs. With the computer comes request_bash (src/agent/bashrequest.js;
the record is bash_requests, migration 0062, src/bashrequests.js): the agent asks a person on
the team to run a command in its shell, for what needs a credential it does not have, or a
person's go-ahead. It gives the command, its reason, a timeout (how long the command may run:
60 seconds unless it says, 180 at most, the shell's own limit), a deadline (until when the
request stays open: a duration from now such as 4h, or a time; a day unless it says, a week at
most) and env, the variables a person is to fill in, by name, each with what it is for. The
tool answers at once, and the agent goes on: nothing waits on the request, and at most 5 of a
thread wait or run at a time. A thread's agent only: a knowledge-base agent has no thread to show
the request under.
The inbox shows each request as a card between the reply box and the draft's line
(bash_requests in GET /api/thread; bashCard in ui/inbox.js): its id, which is how the
agent names it in its notes, the command (highlighted as shell, to be read before it is run),
the reason, a password field per variable, Run and Deny, for whoever may work the
thread; Deny and Run stay in sight at the cards' bottom edge while a tall card scrolls. A person can
have the page remember what they fill in, by variable name, on that device: encrypted
(AES-GCM) in localStorage, with a key made in the browser that cannot be exported and is kept in
IndexedDB; the next request with a variable of that name has it filled in, and Forget removes
it. POST /api/bash/run and /api/bash/deny go to the agent's own object (runBash,
denyBash), which decides: the request has to be pending still (the first click wins, the
second is told who was first), the computer still on, every variable filled in. The run is the
agent's computer's, as its own bash: the same Workspace, working directory and team secrets
(as placeholders), with the person's values as variables of that one run. run answers 202 once
the command has started; the page hears how it ended through the team's room, as it hears of
every change of a request ({ type: 'bash' }), and keeps the request for an hour, on one line,
saying how it ended (a nonzero exit, a timeout and a failure marked) and who decided. An agent
often asks a few times until a command works, so of the requests that ended only the latest
shows (bashCards), with a button for the earlier ones (2 earlier), all in one box. The
card itself says that a command runs or was denied; a toast only says what went wrong. On a
phone the cards are in the drawer the draft's line is in (see "Drafts").
How a request ended comes back to the agent as an input of its own, as a discarded draft
does (harness.submit, with the request's id as the operation, so once): what the command
printed and its exit code; that it was stopped at its timeout, with what it printed until then;
that the computer failed; that a person denied it, and who; or that nobody decided before the
deadline, which a schedule of the agent's object closes. A watchdog schedule, armed before each
run, settles a run whose object restarted under it as failed (docs/STATE-MACHINES.md).
The values are kept nowhere: not in D1, not in the object's storage, not in the transcript,
not in a log. They go from the page to the Worker, to the agent's object and into the shell's
environment, for one run. What the agent is told of the output has each value of 8 characters or
more replaced by its variable's name ([API_TOKEN]), as it is and as it would be escaped in a
URL, JSON or XML (hider); the shell's own record of the run, which Cloudflare Computer keeps
for an hour, is deleted, and output it saved to a file is rewritten the same way (removed, over
4 MB). Unlike the team's secrets, these are real values in the shell, not placeholders the
gateway fills in, so that a person's own token works with any host and any tool. What that does
not stop: the command itself can write a value to a file, which the agent can read afterwards,
send it to any host, or print it encoded. The person who runs the command is the check, as with
a command a colleague asks them to run: the card shows it whole, and says so.
The shell reads a file whole, in one isolate: above about 30 MB grep no longer finds it, and
jq runs out of memory on a large one. So files over 30 MB stay out (200 MB a thread), and
INDEX.txt says which. A HAR also gets <name>.requests.ndjson (src/har.js): one line per
request, no bodies or cookies, credentials hidden, read as a stream, whatever the HAR's size.
extraction_rules. The agent reads, tests and changes its team's extraction rules and tech
stack (list_extraction_rules, test_extraction_rule, save_extraction_rule,
delete_extraction_rule, save_tech, delete_tech; see "Labels and facts"), as
opspile-agent/<model>. Testing a script rule (MCP test_extraction_rule, the page's Test)
runs code in a Dynamic Worker, so it is for those who may save the rule, as saving is. What it saves applies to every channel of its team from the next
labeling of a thread; it cannot change the admin team's, which apply to every channel. Its
section of the prompt asks it to add a rule only for what recurs and the existing rules miss,
to test it first, and never to put a customer's name or a secret into one.
any_thread. Agents read only their own channel's threads, but staff in an internal channel
paste links to customers' threads elsewhere. With this tool the agent has read_any_thread:
any thread of the inbox, whatever its channel, team or customer, by its key or a link to one of
its messages (resolveAnyThread, src/agent/history.js), except in channels the rules ignore.
It has no search: a person points it to a thread. The messages go through Clef's injection check
like any others, and each thread an agent reads is kept in agent_thread_reads, since a
transcript can be reset.
Only the admin team switches it, one channel at a time (no team switch), and only in an
internal channel: filed under an org of type internal, and not shared (channels.shared,
channels.connected). A channel that stops being one loses the tool, whoever turned it on: it is
checked with the switch before each read, and a channel opspile has no details of does not count.
Neither says that nobody outside is in the channel (guests are not recorded), so the org's type
alone does not turn it on. The prompt asks the agent to keep other customers' details out of the
knowledge bases and out of what it posts; that and the channel's other tools are not restricted
by code: what it read can reach them, bash and the browser server included.
membership. Customers and colleagues ask in a thread for someone to be added to the
channel, and on each platform the admins have a bot for that: admin-bot (teams-admin-agent) on
Teams, AEM Ops Automation (the slack-automation relay) on Slack. A request that does not reach
the bot properly (a plain "@admin, can you add …") is easy to miss, and the thread does not wait
on us when Adobe posted last. With this tool the agent files it (request_membership,
src/agent/membership.js), naming the author of the message that asked. Neither bot listens to
other bots (Teams delivers no bot's message to another bot; the relay ignores bot messages), so
opspile calls each through a service binding to its MembershipRequests entrypoint, on the same
Cloudflare account; a named entrypoint answers no HTTP, so no secret is shared:
- Teams (
TEAMS_ADMIN):submit({ action, emails, teamId, conversationId, requester, via })files it as an@admin-bot addwould: an approval card in the admins' Slack channel per person not yet as asked, and admin-bot's answer in the Teams thread, then and once reviewed. - Slack (
SLACK_AUTOMATION): the agent posts the request inMEMBERSHIP_SLACK_CHANNEL(#general) as the opspile bot, mentioning onlyMEMBERSHIP_SLACK_BOT(the relay reads every other<@…>as a person to add), with the asker's name and a link to the thread, then hands that message tosubmit({ channel, ts, text, requester, via }). The relay threads its confirmations under it; the asker confirms if they are a member of the workspace, an admin otherwise.
Nobody is added until it is approved, and the prompt says not to tell anyone otherwise.
What the inbox costs, at list prices, for the admin team (/inbox/cogs, GET /api/admin/cogs):
per service, per message and thread, per customer and team, per day, and per month for what is
kept. Three sources are rolled up into one table, cogs_usage: a row per day, source, service,
metric, detail (a model, a bucket, an index), customer and team, with a quantity. Prices are not
stored: the dashboard prices the quantities with src/cogs/prices.js, so a corrected price applies
to the past too, and a quantity without a price (a model added since) is listed and costs nothing.
The meter (src/cogs/meter.js). Every invocation has a Meter: a request, a queue batch
(a scope per job), a cron run, and each Durable Object (an agent, a team's room, the Discord
poller). A scope is an environment with the same bindings, counted as they answer: D1 rows read and
written (meta; a first() counts one row read, since D1 does not say), R2 Class A and B
operations per bucket, queue operations (three per 64 KB of a message: write, read, delete), Workers
AI tokens per model (the model's usage, else a quarter of the characters it was given),
Vectorize dimensions queried and written, AI Search queries (full-text or semantic) and the tokens
it ingests; and one Workers request per invocation (a queue batch's shared by its jobs). Code adds
what no binding shows: an agent's Bedrock tokens and the time its global browser server's calls
take. Whom the work was for is the scope's subject: a request's thread parameter, a queue job's
thread, a raw event's channel once it is filed (processKey names the customer and team its rules
gave), an agent's thread (or kb:<team>:<org>), a room's team; else nobody, shared. At the end
of the invocation (a request's after its waitUntil work too; a Durable Object's 15 seconds after
it counted, or a room's at the end of each broadcast and flush, so it can hibernate) the subjects
are resolved (a thread's customer and team from D1, once per invocation) and written to Analytics
Engine (USAGE), one data point per customer, team, service, metric and detail, never more than
the 250 an invocation may write (details fold into their metric first). Counting never fails the
work: a binding's answer is returned as it came. Without USAGE (tests, the preview) the
environment is the plain one.
An agent's tokens. pi keeps a ledger of what each session's model requests used (usage(),
per model and tool, cumulative). The agent reads it after each model round (a hook on pi's
generation task) and before it is told something, and counts the difference since the last time
(kept in its storage, cogs:usage) as Bedrock input, output, cache read and cache write tokens
(five-minute and one-hour writes apart, as Bedrock prices them), and what pi's catalog says they
cost, which prices a model the list does not know. While an experiment runs, the arm the agent
is in follows the model in the detail (us.anthropic.claude-opus-5-5#cache=short), which the
prices read past. An agent that was never metered starts from
its ledger as it is: what it used before is not counted on the day COGS started. Once a day it
writes the size of its SQLite (cogs_objects), for the agents' storage.
Cloudflare's analytics (src/cogs/rollup.js, the Analytics SQL binding ANALYTICS, no
token) for what the Worker cannot see, about opspile's own resources only, by name
(COGS_RESOURCES, written by the deploy config; the account has other projects):
- Durable Objects (
events.durableObjectsInvocations): requests and wall time per object of the Worker's script, as GB-seconds of 128 MB. An agent's object is named by its thread, so its duration is its customer's. A team room's WebSocket connections hibernate, so itshttpinvocations (the connections) count as requests and not as duration. Invocations that overlap are billed once; the wall times are added up, so this is an upper bound. - The Worker's CPU time and log events (
logs.workersLogs): CPU from the invocations' own events (fetch, queue, scheduled), shared. - Artifacts operations per repository (
events.artifactsEventsByNamespaceAccount, without token minting): a knowledge base's are its customer's or team's (knowledge_bases), the mirrors shared. - What the buckets (
states.r2CombinedStorage) and indexes (states.vectorizeStorage, times 768 dimensions) hold, per day.
A query that fails leaves the day's rows of its services as they were, and the others are written.
D1: the day's messages and the threads started, per customer and team (volume), and, once a
day, what only D1 knows is kept: its own size (meta.size_after), the agents' storage
(cogs_objects), the full-text items (their number times the average upload of the last 30 days)
and the Workers Paid plan.
The Access API (seatRows, with ACCESS_API_TOKEN and ACCESS_ACCOUNT_ID), once a day too,
and written as d1 rows: the account's Zero Trust seats (users with access_seat or
gateway_seat), one seats each, every one at $7 (the Free plan's 50 are not taken off: past
them, Pay-as-you-go bills every seat). A holder whose email is a person on ops teams is those teams', split evenly; anyone else's seat is
shared. The emails stay in the run. A failed read writes nothing, and the next run tries again.
The rollup. The cron rolls up today and yesterday every run, and up to three of the last 30 days that have no rows of a source yet (backfilling what Cloudflare's analytics and D1 still have; the meter's counts start with it). A day of a source is replaced whole, writing only the rows that changed and deleting the ones that are gone, so a run that changed nothing writes nothing.
By the hour. For the chart of a short period, the flow of the meter and of Cloudflare's
analytics is rolled up by the UTC hour too (toHour(timestamp)), into cogs_hours: the same
queries grouped by the hour as well, without what the buckets and indexes hold (stock), and
without D1's messages and storage. Separate queries, so a day with many objects active in many
hours cannot truncate the daily rows the totals come from. The same days, replaced the same way,
and up to three of the last seven before yesterday without hours yet; hours older than eight
days are deleted.
The agents' drafts (src/draftevents.js, draft_events, migration 0058). Every save, send
and discard of a shared draft (src/drafts.js) is recorded with who did it, the entry's customer
and team, and the draft it was about (its created_at, which stays through its versions). A draft
remembers who saved its first version (origin) and the latest text that person saved
(origin_body); a send says how it went out (as the person, as the opspile app, by the agent's
post_message) and how much the text sent differs from that, by word (changed, 0 to 1). So an
agent's draft a person sent is accepted: as it was (changed 0) or edited (in the reply box,
or saved over by a colleague first). One a person discarded is discarded, one the agent
discarded itself (discard_draft) withdrawn. An Undo saves the same text again: a save with no
draft, by the person who discarded one with that text (its hash, never the text) in the minute
before, deletes the discard's event and goes on as that draft, with its origin. The agent's
post_message that used no draft up is recorded as posted. The report reads the events as they
are (they are few), per customer and team; nothing is rolled up.
The report (src/cogs/report.js). Flow costs what the period used; per message and per thread
it is divided by the period's messages and new threads of what is selected. These estimates, and
a month at this rate, count from the first whole day of the source that started last (each
source's first day is partial, and before it messages were counted without all of their cost),
and a month at this rate divides by the time so far (estimateWindow: today counts its hours). A
period that ends before then has no estimates. Stock costs a month:
each series at its average over the days it was measured. Shared rows are shared out by the
period's messages: a team's (a room's) over that team's customers, nobody's over everyone; then the
filters apply, so a customer's cost includes its share. The chart has about as many columns
whatever the period: by the hour for a week or less (from cogs_hours, while it has them, up to
the current hour), by the day up to 45 days, by the week (from Monday, cut to the period) for
longer; the totals and tables always come from the days. Its columns are by service, and asked
(by), by customer or team too: a customer's own rows and its share of the shared ones, the ten
biggest customers of what is selected apiece (or those picked) and the rest as (other), so the
answer stays small. Where Cloudflare's analytics and the meter
count the same metric on the same day, Cloudflare's count is used. Per accepted draft (the flow
over the agents' drafts a person sent) and per agent reply (over those and the agents' own posts)
count from the first whole day of the drafts' events as well (drafts.estimate), so that a period
before they were recorded does not divide by none; a service's per accepted draft is its own flow,
so Bedrock's is what the agents' model cost per accepted draft. Costs are at the margin: the
allowances of a plan are listed with the prices, not subtracted. An experiment's arms are set side
by side (experiments, of what is selected): per arm, the Bedrock tokens and cost, and the prompt
(input, cache reads and writes), which a cache policy does not change, so what the prompt cost per
million tokens compares the arms however the threads fell between them.
Authors are normalized to { kind: person | bot | system, org, staff }.
- On Slack, bot posts carry
bot_idand nouser, and Slack Connect marks people from another workspace withuser_team. - On Discord, bots carry
author.bot, webhookswebhook_id. Discord shares no email with bots, so a member is staff when they have one of theDISCORD_STAFF_ROLES(identities.staff), or when they connected the account in the inbox (OAuthidentify), which links it to their person: their posts move their watermark and count as theirs. - On Teams, the support tenant invites everyone as a guest, staff included, so
userTypeandtenantIddo not distinguish staff from customers. The organization is derived from the person's mail domain instead. Bot posts carryfrom.application; a bot (applicationIdentityType: bot) keeps its id as its identity, webhooks and connectors stay anonymous.
One account a person (src/accounts.js). Someone can have several accounts in one
workspace or tenant: on Slack, besides their account of the Enterprise Grid, one of another
workspace of the grid, or their own workspace's account seen through Slack Connect (a shared
channel's user_team, kept as identities.home), or a guest account. D1 already knows them as
one: identities are linked to their person by mail address (identities.person), and accounts
of people outside Adobe, who have no people row, are one person's when they have one mail
address. Wherever people are listed or ranked for a thread, each person is there once, by their
account for the thread, and their other accounts count as that one: the mention suggestions
and their pool (GET /api/mentionable), the mentions a reply, an edit or an agent's @Name
makes (knownMentions, mentionsByName, and so the colleagues a post adds to the channel), the
expert finder (findExperts, GET /api/experts, list_experts), the agents' suggestions
(suggest_expert) and the ✕ on one, and find_member. The account for the thread is, of the
person's, the first by:
- of the thread's own workspace or tenant (no other
home, or the channel's workspace) and in its channel (listed inchannel_members, or posted there; on Teams, in its Teams team); - in its channel at all, then of its own workspace at all: a mention reaches someone in the channel, and the inbox can add a colleague of its own workspace to it (an account of another workspace it cannot);
- a full member before a guest (
identities.guest, migration 0061: Slack'sis_restrictedoris_ultra_restricted, Graph'suserType; NULL until the account is looked up again); - the one that posted in the last 90 days, then the one with the most messages in the last 180, in the most channels, that posted last, and by id.
What the signals say of the person adds up over all of their accounts and goes to that one: the
finder's history, response times and affinities are by person already (messages.person,
mentions.person); a suggestion is in the thread or channel when any of their accounts is, finds
them by the names and logins of all of them, and has the affinity of the person (or, for someone
D1 has no person for, of their accounts together). An agent's suggestion is kept as the person's
account for the thread, and one kept by another of their accounts (made before) shows as it; a ✕
on any of them leaves the person out.
Profile cards. Hovering over (focusing, or tapping) a name or avatar in the inbox shows
what opspile knows about the person: name, picture, platform account, job title and company
(Slack profile title, Graph jobTitle and companyName, cached with the identity), mail
domain, ops teams, the channel's org, and above all who they are to us:
- Adobe: staff (a person marked staff, or a mail domain in
STAFF_DOMAINS); - Partner: their mail domain, or a parent domain, is in the
partner_domainstable (agencies and system integrators, maintained by hand, like the rules); - Customer: anyone else; the card notes when their domain is not the usual customer domain of the channel, which is how an unlisted partner shows up;
- Bot.
A list of partner domains is explicit and never wrong, where guessing from "not staff and not
the customer's domain" would call a customer's second domain a partner. The cards travel with
the inbox and thread payloads (profiles, keyed by messages.author_key), so a hover costs no
request, and emails stay on the server.
These were measured against the live APIs during exploration.
- Both platforms are two-level: a root message and a flat list of replies, no deeper nesting. A root without replies is a thread of length one.
- Slack message ids are
tsstrings, unique per channel. A thread is keyed by the root'sts; replies carrythread_tsandparent_user_id. The root'slatest_replychanges whenever a reply arrives, which makes change detection cheap. - Teams message ids are epoch-millisecond strings, unique per channel. Replies carry
replyToId. The root'slastModifiedDateTimedoes not change when replies arrive, so new activity cannot be detected by polling roots. - Bots edit their own messages later, for example to add a ticket link, which is why edits never make a thread unread.
- Discord is two-level in another way. A channel's messages are flat, and a thread is
a channel of its own (
channel_idis the thread's id, itsparent_idthe channel). A thread started from a message has that message's id, and a forum post's first message has the post's id, so the thread id is the root message id: a thread's messages are filed under its parent channel and the thread id, next to the message it started from. People also answer with an inline reply (type 19,message_reference) instead of a thread; that reply joins the conversation of the message it answers, and a thread started from such a reply joins it too. Message ids are snowflakes (ms since 2015 shifted by 22 bits), unique across Discord. Joins, pins, boosts and "started a thread" notices are not stored in pile 2. - Alert channels consist entirely of bot posts without replies. They are filed as
kind: alertand counted, rather than shown as individual unread threads.
Two new apps:
-
A Slack app, installed into the enterprise grid, subscribed to message, edit, deletion, and reaction events. Slack only delivers message events for channels the app is a member of, so the app has to be added to every ingested channel, including Slack Connect channels. With
SLACK_JOIN_TEAMSit adds itself to every public channel of those workspaces (src/autojoin.js, bot scopechannels:join): a channel as it is created or unarchived, and on the cron, at most 40 a run, every other public channel it is not in. Channels Slack refuses (app_cannot_join_channel, say) are tried again after a week (slack_joins). Private channels still need an invite; channels the rules markignoreare not joined. -
A Microsoft Entra app registration in the support tenant with application permissions to read teams, channels and channel messages. One tenant-wide change-notification subscription on
/teams/getAllMessagesdelivers root messages and replies. It carries no resource data (so no encryption certificate) and lives at most 60 minutes, so a cron renews it every 20 minutes. Microsoft stopped metering Teams APIs on 2025-08-25, and creating the subscription needed no protected-API approval (measured 2026-10-20). Polling is not an alternative:/messages/deltaand/messagesreturn root messages only, never replies. -
A Discord application with a bot in each server opspile reads (
DISCORD_GUILDS), permissions View Channels and Read Message History and the privileged Message Content intent, which REST reads need as well. Discord delivers messages only over its gateway WebSocket, which would keep a Durable Object in memory, and billed, around the clock; a support server gets a few messages a day, so opspile polls instead. The poller object's alarm asks every minute for each server's channels and active threads, which name their latest message, and reads the messages after the last one stored where there are newer ones: two requests when nothing happened. An archived thread that gets a message is active again, so archived threads are read only on the first poll,DISCORD_HISTORY_DAYSback. Every fifth poll it reads the conversations active in the last day again and compares them with what it saw: an edit or other reactions are stored as a new state of the message (a reaction timed when it was first seen), a message no longer there as a deletion. The object stores each thread's parent, each reply's root, the last message id per channel and thread, and its view of the last day's messages; the cron pokes it, so a stopped alarm restarts.
Failed jobs. The NORMALIZE queue retries a job five times. Its last failure is recorded in
dead_jobs (migration 0050, src/deadjobs.js; ids and keys only, kept for 30 days), which
/diag/dead lists and sends out again. Most jobs need nobody to do that: the cron's sweeps find
vectors, full-text items and exports that are behind, roots whose re-threading died
(skeleton_hash set, rethreaded_at not), recent threads with a person's message after their
label, and open backfills. Re-threading and labels are swept once an hour, a batch spread out over
minutes, and a job that died within the day waits. A dead refile does not heal by itself, since sweeping would apply
today's rules to every channel, and waits to be sent out again.
- Slack Connect: does adding the app to a shared channel require approval from the other workspace?
- Re-threading: what confidence threshold keeps false merges rare, measured on real follow-up posts?