Skip to content

feat: Track personal recents on the server - #5135

Draft
taratatach wants to merge 11 commits into
masterfrom
feat/personal-recents
Draft

feat: Track personal recents on the server#5135
taratatach wants to merge 11 commits into
masterfrom
feat/personal-recents

Conversation

@taratatach

@taratatach taratatach commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Fixes #4951

Drive's recents view lists the files most recently modified by anyone,
computed in the browser from the DataProxy. It should show my own
activity: what I edited or opened, across my files and the ones shared
with me. This branch moves the recents into the stack, as described in
ADR 007, behind the drive.recents.enabled feature flag (off by
default).

A protected io.cozy.files.recents doctype stores one
FileDirActivity row per user and item, in the user's own database:
blocked from /data, excluded from the instance export, refused to
realtime subscriptions. A row is keyed by the source instance ID and
the file ID, and keeps separate edit and open dates, stamped by the
server. Activities of the same kind less than a minute apart rewrite
nothing, which keeps note typing and sync uploads cheap.

Activity is captured where the stack sees it: file and folder
creation, overwrite, restore and version revert; OnlyOffice openings
and saves, where every participant of a session is credited through
atomic updates of the session store; note creation, steps and loading;
and Drive's editors, the excalidraw editor and the PDF viewer, through
GET /editor/:file-id/open. A download is never an open. Sharing
members are credited on their own instance, read-only openings
included; a member on another stack loses the activity, with a
warning.

GET /files/recents returns a JSON:API page of these items. Each
entry is re-checked at read time against the owner's instance, so
trashed, deleted or no longer accessible items are hidden. Entries and
cursors carry a hash of the row key, never the internal ID of the
source instance. On the first read, a seed fills up to 20 rows from
the latest files and folders, so that switching to the server-side
recents never shows an empty page; it needs two new io.cozy.files
indexes (IndexViewsVersion 42).

@taratatach taratatach self-assigned this Oct 30, 2026
@taratatach
taratatach force-pushed the feat/personal-recents branch 5 times, most recently from 1f72d14 to 7022c71 Compare November 2, 2026 12:13
@taratatach
feat: Add protected io.cozy.files.recents doctype
7c27d09 
  Personal recents need a per-user store no client may touch directly.
  Declare the doctype, block it from `/data`, keep it out of the
  `model/move` export and import, and reject realtime subscriptions and
  `@event` triggers on it, as for the public link events, so only the
  dedicated endpoint will serve it later. The instance lifecycle now
  creates the database and its sort index on `(last_activity_at, _id)`.

  New package `model/recents` declares the row, `FileDirActivity`, keyed
  by `ResourceKey` (source instance ID + file ID), and `PublicID`, the
  hash of that key given to the clients so that the internal ID of the
  source instance never leaves the stack. It also holds the
  `drive.recents.enabled` feature flag (off by default, set per instance
  or context) that gates the whole feature.

  `Enabled` reads the flags of the instance first, without any I/O, and
  otherwise asks `feature.GetFlags`, like every other caller of the
  stack.

Co-Authored-By: AI
@taratatach
feat: Carry person and instance on request actor
2186636 
  The request actor resolved by the permission middleware only knows a
  kind, a display name and a domain. Recording personal activity needs
  to act for the person behind a request: without more, it would have to
  redo the token inspection and the instance lookup, and would lose the
  original token on the shared drive proxy, which swaps it for the drive
  token.

  `Actor` gains `Instance`, the actor's instance on this stack, and
  `Person`, computed once from the original token: a session-bound app
  token, the flagship app or `cozy-desktop` OAuth client, an OAuth token
  with the `*` scope (only the flagship app asks for it, and the stack
  grants it without certification when the context skips it), or a
  share-interact code.

  `Person` survives the drive proxy swap because the share-interact
  branch keeps an actor that is already set. `Instance` is the request
  instance for the owner's tokens, and the member's instance for a
  share-interact code, but only once the member has accepted the
  sharing: before that, the member's Cozy URL comes from the sharer's
  contacts and cannot be trusted. `Domain` keeps its current value for
  `trashed_by`.

  `MemberInstance` loads the instance of a member by its domain. A
  member on another stack (not found or illegal domain) is silent, any
  other error is logged as a warning.

Co-Authored-By: AI
@taratatach
feat: Record activity on files and folders
712bcec 
  Personal recents must reflect my own edits and opens, not the file's
  latest change. `RecordActivity` upserts the row of the actor on an
  item with a monotone max per component (`last_edited_at`,
  `last_viewed_at`, `last_activity_at`), stamped server-side, idempotent
  under replay. Two activities of the same kind less than a minute apart
  rewrite nothing, and a new activity refreshes the `source_locator`
  after a domain change.

  Each process also remembers for a minute the activities it stored, so
  that the hot write paths skip the read of the row. A row deleted by
  other means cannot be recreated by that process during that minute,
  which no legitimate activity needs. An activity on a moved source
  instance records nothing, as its rows would point to an instance that
  is going away.

  The files handlers record an edit right after a successful create,
  overwrite, restore, version revert or folder creation; renames, trash,
  copies, downloads and permission changes record nothing. The automatic
  writes of the flagship app record nothing either: the photo backup
  (files tagged with `metadata.backupDeviceIds`), any folder it creates
  (backup tree, konnector folders, logs) and its files under
  `/Settings/Logs` and `/Settings/AALogs`. Uploads from its share sheet
  still count.

  Identity comes from the actor middleware: the actor must be a person,
  and its instance on this stack is the one that gets the row. On the
  shared drive proxy, the person is checked on the original token of the
  recipient; a proxied route without authentication pins an actor that
  is not a person before the swap to the drive token.

  The capture is gated by the `drive.recents.enabled` feature flag of
  the actor's instance. A failed row write is logged, counted in
  `recents_activity_failures_total` and swallowed; a member who has not
  accepted the sharing yet, or whose instance is not on this stack,
  silently loses the activity.

  `lifecycle.Reset` tolerates a recents database recreated by an
  activity of another instance while the databases are deleted: it
  resets it, or keeps it if it is recreated again. The ADR documents the
  identities, the exceptions to the edits and the one-minute memory.

Co-Authored-By: AI
@taratatach
fix: Copy office sessions in the memory store
29a0038 
  The memory store of the OnlyOffice sessions kept the session it was
  given, and `GetDoc` returned a pointer to the stored one: a caller
  changing a session it had read changed it in the store, outside the
  lock, and the `MD5Sum` slice stayed shared with the caller that added
  the session. The redis store has no such sharing, as it reads and
  writes JSON.

  `ConflictDetector.Clone` makes a deep copy of a session. The memory
  store clones a session when it stores it (`AddDoc`, `UpdateDoc`) and
  when it returns it (`GetDoc`), so that it behaves like the redis
  store. No caller changed a session it had read, so the behavior is
  unchanged today.

Co-Authored-By: AI
@taratatach
feat: Record activity on OnlyOffice documents
b9cab0c 
  Opening an OnlyOffice document records a view for the person who opens
  it. The document server does not report who saved, so each save
  callback credits an edit to every person who opened the session, kept
  in a `Participants` list on the session key. A participant who only
  viewed is credited with the edit too, the closest exact attribution
  available.

  Adding a participant, and moving the session to the revision written
  by a save, are atomic updates of the session (the lock of the memory
  store, a `WATCH`/`MULTI` transaction in redis): concurrent openings
  and saves never drop a participant. When a conflict copies the file,
  `MoveToRevision` also moves the file index to the copy, which
  `UpdateSecret` did in a separate call that callers had to order. An
  update never revives an expired session of the memory store.

  Known limit, left as is: in redis, the session and the index of its
  file live under two keys that a cluster may place on different slots,
  where a transaction cannot span both. The index is moved after the
  session, and until then the old file still points to the session.
  Making it atomic would need both keys on the same slot, through a hash
  tag in their names: a change of the stored key format that would lose
  the sessions open at deploy time, with their pending saves. The window
  is short, and the separate `UpdateSecret` call already had it.

  As for files, the capture is gated by the `drive.recents.enabled`
  feature flag of each participant's instance, and a participant whose
  instance is not on this stack silently loses the activity: any other
  failure to load it is logged with a warning. A session store failure
  that loses an opening, when reading the session or adding the
  participant, is logged and counted in
  `recents_activity_failures_total`.

Co-Authored-By: AI
@taratatach
feat: Record activity in notes and Drive editors
af5da57 
  Notes are written through `model/note` instead of the hooked files
  handlers: creating a note and applying its steps (`PATCH /notes/:id`)
  now record an edit, and loading it in the editor (`GET /notes/:id`)
  records an open, as the notes editor never downloads a note.

  Drive shows a drawing in its excalidraw editor and a PDF file in its
  viewer after opening it with `GET /editor/:file-id/open`, which
  records the open. A content read is never an open: it is the same
  request as a download, a preview or a sync. The ADR is updated
  accordingly.

  Sharing members are credited on their own instance, read-only openings
  included. `ParseJWT` keeps the member to whom a share code was given
  in the request context (`GetShareMember`), and `ActorInstance`
  resolves a share-preview code to that member once they have accepted
  the sharing: until then, their Cozy URL comes from the sharer's
  contacts. The capture is gated by the `drive.recents.enabled` feature
  flag, like the others.

  The ADR also covers the opens of the mobile app, made in webviews with
  session-bound app tokens, and records a known limit: whoever holds the
  code of a member counts as that member.

Co-Authored-By: AI
@taratatach
feat: Reference the Photos folder on creation
d788071 
  `cozy-client` and the flagship app look for the Photos magic folder by
  its `io.cozy.apps/photos` reference, but the stack creates /Photos at
  instance creation without it. The reference is only added later, when
  a konnector or the photo backup asks for the magic folder. Until then,
  the folder looks like one the person created, and the seed of the
  personal recents would list it among their folders.

  Add the reference when the folder is created, as done for the
  Administrative folder. Existing instances keep their folder as is.

Co-Authored-By: AI
@taratatach
feat: Seed personal recents
c9b57a2 
  Switching from the browser DataProxy to server-side recents must not
  show an empty page. `Seed` fills up to 20 rows from the local
  `io.cozy.files`: files carry their modification date, folders their
  creation date, never the seed time. A `_local` marker in the recents
  database makes the seed run once per instance; the upsert by resource
  key absorbs concurrent seeds, and a concurrent marker write is not a
  failure. `move.Import` removes the marker with `ForgetSeed` once the
  parts are imported: a seed that ran while the import was starting saw
  the replaced files, and the first read seeds the imported ones.

  The items the user did not create or edit are left out by the
  selectors, so that they never take one of the 20 places: the system
  folders (fixed `io.cozy.files.` ids), the trashed folders (by path, as
  folders have no trashed field), the magic folders referenced by an app
  or a konnector (Administrative, Photos, Notes, photo backup...), the
  folders created by the flagship app, and the files of the photo backup
  (`metadata.backupDeviceIds`). A Mango negation does not match a
  missing field, hence the explicit `$exists` branches.

  The dates come from the clients. A date up to 14 hours in the future,
  at most a local time written as UTC (UTC+14), is kept with a warning;
  a later one is skipped, as its row would stay on top until then. The
  selectors bound the dates at now plus 14 hours, as a date read as a
  string is at most that far ahead, so that the items dated far in the
  future do not take the places of the others; the parsed dates are
  checked again, with a warning for an item skipped there.

  Files and folders are queried through two indexed requests (new
  indexes, hence `IndexViewsVersion` 43) and merged by seeded date,
  capped at 20. A failed seed is logged and counted in
  `recents_seed_failures_total`: it leaves an empty page, never an
  error. The list endpoint will trigger it on the first read.

Co-Authored-By: AI
@taratatach
feat: Resolve access of several docs at once
1fea186 
  The list of personal recents will check, for each item owned by
  another instance of the stack, whether the reader can still read it.
  `ResolveForMember` works on one target at a time: it loads the doc
  again, then makes one request for its parent, one for the shared roots
  on its path and one for the sharings, so a page of items would cost a
  few requests each.

  `ResolveDocsForInstance` takes docs already loaded by the caller and
  makes at most three requests for all of them: one `_all_docs` for the
  parents of the files, one `dir-by-path` find on the deduplicated union
  of their ancestor paths, and one load of the sharings. A missing
  parent gives an error for its target only. The reader is matched on
  every sharing as the ready member whose Cozy URL has one of the
  instance domains, aliases included. The email is never compared, as
  the settings email of an instance is not a proof of identity.

  The find and the collection of shared roots are extracted from
  `applicableSharingsForDoc` with no change of behavior for its callers.
  `vfs.DirOrFileDocsByIDs` loads files and folders in one `_all_docs`,
  and `vfs.NewFilePatherWithDirs` computes paths from parents loaded
  beforehand, without fetching again a parent known to be missing.

Co-Authored-By: AI
@taratatach
taratatach force-pushed the feat/personal-recents branch 3 times, most recently from 18193a1 to 202a76f Compare November 3, 2026 17:04
@taratatach
feat: Serve personal recents from the stack
ebd5240 
  Add `GET /files/recents` so Drive can read its recents from the
  backend instead of the browser DataProxy. JSON:API page of up to 50
  items (`page[limit]` capped at 100), cursor over the
  `(last_activity_at, _id)` index, and `meta.incomplete` when a source
  cannot be checked or the work limit is hit. An instance whose recents
  database does not exist yet gets an empty page.

  Entries and cursors carry the public ID of a row, a hash of its key:
  the internal ID of the source instance, possibly the owner of a shared
  drive, never leaves the stack. A cursor is resolved among the rows of
  its date; when its row changed meanwhile, the traversal resumes before
  that date and sets `meta.incomplete`.

  Every candidate is re-checked at read time: fresh metadata is read
  from the database of the instance that holds the item (same stack),
  trashed, deleted or unauthorized items are skipped and the page
  refilled within a bounded work limit, and losing access only hides a
  row — regaining it shows the row again with the user's own date. The
  access of the user on the items of another instance is checked by
  `ResolveDocsForInstance`, which matches the user's domains only.

  The source of a row is resolved from the instance ID of its key, as
  the domain of an instance never changes: a change of domain is a move,
  which creates a new instance and keeps the file IDs. The rows of a
  moved source are migrated under the key of its successor on the stack,
  found from `moved_to` or from the old domain of the new instance, with
  the dates of both rows merged; the user's own rows are never migrated.
  The rows of a source gone with no successor, of a destroyed item or
  with a malformed key are deleted, a stale `source_locator` is
  refreshed, and a source that cannot be checked hides its rows.

  Rows are read by batches, the size of the page plus one first, then
  twice the previous one. Each batch costs one find for the rows, one
  request for the unknown source instances, one `_all_docs` per source
  and at most three for the access; a moved or deleted source adds the
  lookups of its successor, and the rows it migrates one `_all_docs`.
  The repairs are written in bulk at the end with
  `couchdb.BulkWriteDocs`, which reports each document. A cursor always
  points to a row that is still there: the next cursor is the last
  examined row that is not deleted, at the end of the page or where the
  work limit is hit. The item of a migrated row ending a page may then
  come back on the next page. The first page of a traversal triggers the
  lazy seed.

  The endpoint is gated by the `drive.recents.enabled` feature flag of
  the instance, and answers 404 while it is off.

Co-Authored-By: AI
@taratatach
feat: Follow moved or deleted sources in recents
d69f239 
  A moved or destroyed instance leaves rows keyed by its ID in the
  recents of the members of its sharings. The reads migrate or delete
  them, but only when they happen: once a moved instance is deleted, a
  month after the move, its successor can no longer be found.

  `move.Finalize` pushes a `recents-source` job to each member of the
  sharings of the moved instance that lives on this stack and has the
  recents enabled. The job migrates the rows on its items to the
  successor at the end of the chain of moves, with the dates merged at
  their maximum, or deletes them when the target is on another stack. It
  is given the successor by its ID, as its domain may be given to
  another instance before the job runs.

  `lifecycle.Destroy` calls `recents.PrepareDestroy` before deleting the
  databases, as it reads the members and the successor of a moved
  instance: the rows on a moved instance, destroyed a month after its
  move, are migrated to its successor, which catches the ones the jobs
  of the move missed, and the other rows are deleted. When the successor
  cannot be checked, no job is pushed, and the reads handle the rows.
  Both look for the successor only when the instance has such members,
  loaded in one request.

  `MigrateSource` and `DropSource` read the rows of a source by batches
  from the range of its key prefix in `_all_docs`, and write them in
  bulk. They never touch the user's own rows, and running them again is
  a no-op. A former member removed from the sharings is not found, and a
  member whose recents are disabled is skipped: the reads remain the
  safety net.

Co-Authored-By: AI
@taratatach
taratatach force-pushed the feat/personal-recents branch from 202a76f to d69f239 Compare November 4, 2026 12:47
This was referenced Nov 4, 2026
Open
Open
Open

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shepilov shepilov Awaiting requested review from shepilov shepilov will be requested when the pull request is marked ready for review shepilov is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@taratatach taratatach

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Implement personal recents backend

1 participant

@taratatach