MONGOID-5943: Add changeset persistence layer#6148
Draft
jamis wants to merge 48 commits into
Draft
Conversation
Implement an immutable Struct to represent a single staged database operation, capturing type (insert/update/delete/update_many/delete_many), collection, selector, payload, document, and session at the moment an operation is staged. Includes comprehensive specs covering all attributes and nil document handling.
…card Introduces Mongoid::Changeset with entry accumulation, depth tracking for nested scopes, discard, and a flush stub (full implementation in Task 6). Also adds Mongoid::Errors::InvalidOperation for guarding against post-termination use.
Add Mongoid.changeset{} and Mongoid.current_changeset, backed by
Threaded.current_changeset. Nested calls accumulate without flushing;
the outermost block flushes on exit or discards on error.
The raising stub broke lifecycle units and forced rescue scaffolding throughout tests. A no-op with a TODO comment is the right placeholder until Task 6 lands.
Use == instead of equal? in _build_batches so that two distinct Mongo::Collection objects for the same logical collection are correctly grouped into a bulk_write batch. Replace doc.send(:in_transaction?) with entry.session&.in_transaction? so the session captured at staging time is used, consistent with how persistable.rb uses _session directly. Extract _dispatch_commit helper to keep _flush_entries within the cyclomatic-complexity limit.
Replace the old prepare_insert (which called insert_as_root/insert_as_embedded directly and wrapped everything in run_callbacks(:commit)) with a new version that stages a Changeset::Entry and delegates execution to Changeset#flush. Root documents stage an :insert entry; embedded documents stage an :update entry against the root collection (matching the existing driver call shape). Remove insert_as_root, insert_as_embedded, and post_process_insert — document state updates (new_record = false, flag_descendants_persisted) now happen inside Changeset#_update_document_state at flush time. Extend _update_document_state to handle embedded-insert :update entries: when an :update entry carries a document that is still a new record, flip new_record and persist descendants, matching what :insert does. Guard Threaded.add_modified_document in post_process_persist so it is only called when outside a changeset; inside a changeset, _dispatch_commit handles transaction tracking after flush.
RootInsertable overrode insert_as_root (removed); update to override _stage_insert_as_root instead. Also update a stale comment in touchable.rb that referenced the old method name.
…entries Using doc.new_record? as a discriminator on :update entries to detect embedded inserts is fragile. Introduce a dedicated :embedded_insert type so the intent is explicit and future :update entries cannot accidentally trigger insert-state transitions. Also document why dirty state is cleared inside the changeset block before the driver write.
Removing the explicit move_changes call from update_document's block prevents post_process_persist from clearing previous_changes a second time and corrupting after_save callback state.
…geset Also fix Changeset#run to return the block's value so that destroy can capture whether the callback chain aborted. Removes the manual run_callbacks(:commit) wrapper from destroy — commit dispatch is now handled by Changeset#_dispatch_commit at flush time.
Introduce :embedded_delete as a distinct entry type for embedded document deletes (parallel to :embedded_insert), so the changeset can mark the document destroyed at flush time. Update _execute_single, _bulk_op_for, and _update_document_state accordingly, fix the stale type comment in Entry, and add an integration spec covering the deferred delete path.
Also fix Savable#save to return true when an insert is staged in an active changeset (the document stays new_record? until flush, so the old check produced a false negative when called inside a changeset block).
Replace the depth-counter/atomic-context-stack mechanism in
atomically with delegation to Mongoid.changeset{}. Nesting now
works through the changeset's own depth tracking. join_context: false
is deprecated and implemented via _atomically_independent, which
temporarily hides the enclosing changeset.
Remove executing_atomically?, persist_or_delay_atomic_operation,
and the _mongoid_push/pop/remove/reset atomic context helpers. Adapt
prepare_atomic_operation and persist_atomic_operations to stage
Changeset::Entry objects directly. process_atomic_operations now
always calls remove_change at stage time rather than deferring.
Update logical, multipliable, renamable, and unsettable to remove
executing_atomically? guards, always updating attributes via
process_attribute (or direct hash manipulation for rename). Update
all "marks a dirty change when executing atomically" specs to reflect
that dirty tracking is cleared at stage time.
Each of the 11 atomic-op modules (inc, mul, set, unset, bit, set_max, set_min, pop, push, add_to_set, pull, pull_all, rename) now builds its ops hash directly and stages a Changeset::Entry, removing the prepare_atomic_operation / process_atomic_operations indirection. Those two helpers are removed from Persistable; persist_atomic_operations is kept because touchable.rb still calls it.
Rewires Contextual::Mongo#delete / #delete_all and #update_all to accumulate :delete_many and :update_many entries in the changeset instead of hitting the driver directly. Adds an :opts field to Changeset::Entry for driver-level options (e.g. array_filters, collation), and threads it through _execute_single. A private _view_opts helper captures view-level options (collation) so they survive the indirection through the changeset.
has_one and has_many proxy setters now call save! instead of save when persisting a newly assigned document. Validation failures raise Mongoid::Errors::Validations immediately rather than being silently ignored.
Wrap the add_to_set and save! calls in Mongoid.changeset{} so both the
base document's foreign key update and the child document's save share a
single changeset and flush together. Change doc.save to doc.save! so
validation failures raise rather than silently succeed.
Fix Incrementable#inc to update in-memory attributes even on unpersisted documents. The early return on `persisted?` was moved after the attribute loop so in-memory state is always updated, matching the original behavior. The DB-staging path is still gated on persisted?. Also add a "Planning and Design" section to AGENTS.md.
This is a better pattern.
…ded in a changeset with other operations)
run_before_callbacks(:flush) and run_after_callbacks(:flush) are the correct way to invoke the before and after sides of the :flush callback chain separately; run_callbacks(:before_flush) does not exist.
Contributor
There was a problem hiding this comment.
Pull request overview
Introduces a brand-new changeset persistence layer that defers all writes (saves, destroys, atomic ops, criteria-level updates/deletes, embedded batch ops) to a per-thread/fiber Mongoid::Changeset, flushed (via single-collection bulk_write where possible) when the outermost Mongoid.changeset { ... } scope exits. atomically is reimplemented on top of this layer (with join_context: false deprecated), new before_flush/after_flush callbacks and staged?/staged predicates are added, transactions inside changesets are forbidden, and several persistence return-value/timing semantics shift.
Changes:
- New
Mongoid::Changeset+Entrycore,Mongoid.changesetentry point,before_flush/after_flushcallbacks, andstaged?/stagedpredicates. - All persistence call sites (insert/update/delete/upsert, atomic operators, embedded batchable, contextual delete/update_all, HABTM setter) rewritten to stage entries instead of issuing direct driver calls.
- Behavior changes:
atomically(join_context: false)deprecated; transactions inside a changeset raiseErrors::TransactionInChangeset; deferred timing of save/commit callbacks;deletereturn-value semantics adjusted; HABTM/referenced-association setters usesave!.
Reviewed changes
Copilot reviewed 66 out of 66 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| lib/mongoid.rb | Adds Mongoid.changeset entry point. |
| lib/mongoid/changeset.rb, changeset/entry.rb | New core: depth-tracked entry buffer with batched flush. |
| lib/mongoid/threaded.rb | Adds thread/fiber-local current changeset accessor. |
| lib/mongoid/stateful.rb | Adds staged? / staged predicates. |
| lib/mongoid/interceptable.rb | Defines before_flush/after_flush callback hooks. |
| lib/mongoid/persistable.rb | Replaces atomic-context machinery with changeset-based staging helpers. |
| lib/mongoid/persistable/{creatable,updatable,upsertable,savable,deletable,destroyable}.rb | Rewritten to stage insert/update/delete/upsert entries. |
| lib/mongoid/persistable/{incrementable,logical,maxable,minable,multipliable,poppable,pullable,pushable,renamable,settable,unsettable}.rb | Atomic operators now stage update entries with skip_callbacks/dirty-field bookkeeping. |
| lib/mongoid/contextual/{mongo,memory}.rb | Criteria-level delete / update_all route through the changeset. |
| lib/mongoid/association/embedded/batchable.rb | Embedded batch insert/clear/remove/replace stage updates. |
| lib/mongoid/association/referenced/{has_one,has_many,has_and_belongs_to_many}/proxy.rb | Setters use save!; HABTM << uses an explicit changeset. |
| lib/mongoid/clients/sessions.rb | Raises TransactionInChangeset if transaction opens inside a changeset. |
| lib/mongoid/errors/{invalid_changeset_operation,transaction_in_changeset}.rb + en.yml | New error classes/messages. |
| lib/mongoid/touchable.rb | Comment update referencing renamed staging method. |
| lib/mongoid/warnings.rb | New deprecation warning for join_context: false. |
| lib/config/locales/en.yml | New error message entries. |
| spec/** (numerous) | Adds changeset/entry/atomic-callbacks/staged specs; updates atomic-op specs to reflect cleared dirty tracking; updates deletable/destroyable/upsertable/embeds_many specs to cover deferred behavior; updates persistable_spec/atomic semantics; updates interceptable spec for flush callbacks; updates threaded/stateful/transactions specs. |
| spec/mongoid/interceptable_spec_models.rb | Renames insert_as_root hook to _stage_insert_as_root. |
| AGENTS.md | Adds note that design docs should remain unstaged. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ecessary state changes on flush
Mirror the four state-transition calls (new_record = false, remember_storage_options!, flag_descendants_persisted, _reset_memoized_descendants!) that _stage_insert_as_root already performs, so that after_save callbacks on an embedded document see correct persisted state and do not cause atomic_updates to emit a redundant $push.
Jibola
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a changeset persistence layer for Mongoid. Rather than writing each save, destroy, or atomic operation immediately to MongoDB, all operations are now staged in a
Mongoid::Changesetand executed together as an optimized batch (using bulk writes where possible) when the outermost scope exits.New public API
Mongoid.changeset { ... }— opens a changeset scope. All persistence calls inside the block (save, destroy, atomic ops) are staged and flushed together when the outermost block returns. Changesets nest: inner calls accumulate into the outermost one.before_flush/after_flushcallbacks — fire around the batch write, at the document level.staged?/stagedpredicates onMongoid::Document— true while the document has a pending entry in the current changeset.Mongoid::Changeset::Entry— immutable value object representing one staged database operation (:insert,:update,:delete,:upsert, etc.).Mongoid::Errors::InvalidChangesetOperation— raised when an operation is attempted on a terminated changeset.How
atomicallychangesatomicallyis now implemented in terms ofMongoid.changeset. Nesting and join semantics are unchanged for the default (join_context: nil) case.join_context: falsestill works but is deprecated (see below).Breaking changes
1.
atomically(join_context: false)is deprecatedThis option caused the inner block to persist immediately, independent of any enclosing context. It now emits a deprecation warning. The replacement is to call
saveoutside any enclosing changeset scope. The option still functions for now but will be removed in a future version.2.
after_save/after_update/after_createcallbacks run before changes are persistedThese callbacks used to fire after the document was persisted. Now, they run inside of a changeset and always fire after the document is staged (but before it is persisted). If you need logic to only run after the record is persisted, you should use
after_flushorafter_commit.3.
after_commitcallbacks are deferred to flush timeafter_commitcallbacks now fire at flush time — after all staged entries have been written to the database — rather than immediately after each individual save. Code that depended on commit callbacks firing right after a nested save will observe different timing.4. Referenced association setters call
save!Setters for referenced associations (
has_one,has_many,belongs_to) now callsave!rather thansave, so validation errors raise rather than being silently swallowed. They also go through a changeset, rather than writing directly.5. Nested attributes do not persist immediately
Assigning to an association via a nested attributes accessor no longer immediately persists the association. Instead, it is saved when the parent is saved, which gives validations a chance to run and influence whether the parent (and the child associations) are saved.