Conversation
This implements state DAGs, without support for federation. A general overview: - It adds a new room version and new event type. - It adds a new field `calculated_auth_event_ids` to internal metadata. - It stores the state DAG via new state DAG edges / forward extremities tables. - It adds new auth rules as per the MSC. - It uses the new `prev_state_events` field instead of `prev_event_ids()` when doing state resolution.
Builds off #19424 Adds federation compatibility for state DAG rooms. Overview: - Adds extra HTTP API fields as per the MSC. - Adds methods for walking and extracting the state DAG for a room (for `/get_missing_events` and `/send_join` respectively). - Adds impl for processing the federation requests, as well as `/send`.
| logger.debug("get_room_state_ids dest=%s, room=%s", destination, room_id) | ||
|
|
||
| path = _create_v1_path("/state_ids/%s", room_id) | ||
| qps = {"event_id": event_id} |
Contributor
There was a problem hiding this comment.
qps is too ambiguous. Also mentioned this on matrix-org/complement#806 (comment)
| create_event = e | ||
| break | ||
|
|
||
| if room_version.msc4242_state_dags and response.state_dag: |
Contributor
There was a problem hiding this comment.
Feels like this should just be if room_version.msc4242_state_dags and error if response.state_dag doesn't exist.
Comment on lines
+1215
to
1220
| auth_events = response.auth_events | ||
| create_event = None | ||
| for e in state: | ||
| if (e.type, e.state_key) == (EventTypes.Create, ""): | ||
| create_event = e | ||
| break |
Contributor
There was a problem hiding this comment.
Should we put this as an else below?
Comment on lines
+780
to
+782
| if event.room_version.msc4242_state_dags and caller_supports_partial_state: | ||
| # TODO(kegan): for now, MSC4242 won't support partial state for ease of prototyping. | ||
| caller_supports_partial_state = False |
Contributor
There was a problem hiding this comment.
Comment about the nuance in the docstring
Comment on lines
+807
to
+814
| # NOTE: we don't return the state dag for forward extremities that aren't part of this | ||
| # join event to make it easier for the receiving server to set their own forward | ||
| # extremities (they are equal to the join event's prev_state_events). This means we may | ||
| # fail to sync concurrent forks not on the path to the join event, but this is an | ||
| # outstanding problem in general. | ||
| state_dag = await self.store.get_state_dag( | ||
| room_id, set(event.prev_state_events) | ||
| ) |
Contributor
There was a problem hiding this comment.
Is it really that bad if we just pass all forward extremities?
| e_id | ||
| for e_id in ( | ||
| ev.auth_event_ids() | ||
| if not isinstance(ev, FrozenEventVMSC4242) |
Contributor
There was a problem hiding this comment.
assert not isinstance(ev, FrozenEventVMSC4242)
| ] | ||
| for ev in event_map.values() | ||
| } | ||
| # XXX: confusing name: this isn't _only_ auth events! |
Comment on lines
+1931
to
+1934
| # TODO(kegan): check before merging if we can be stricter here. | ||
| # Otherwise, we are somewhat lenient and just persist the event | ||
| # as rejected, for moderate compatibility with older Synapse | ||
| # versions. |
| else: | ||
| calculated_auth_events = await self._store.get_events( | ||
| calculated_auth_event_ids, | ||
| allow_rejected=True, |
Contributor
There was a problem hiding this comment.
Why allow_rejected=True,?
Comment on lines
+1956
to
+1960
| # The above function is typically not async, and so won't yield to | ||
| # the reactor. For large rooms let's yield to the reactor | ||
| # occasionally to ensure we don't block other work. | ||
| if (i + 1) % 1000 == 0: | ||
| await self._clock.sleep(Duration(seconds=0)) |
Contributor
There was a problem hiding this comment.
Is this true?
Both process and prep look like they have yieldable points that are always hit. Perhaps good to have this explicit anyway as the other code can evolve.
Co-authored-by: Eric Eastwood <erice@element.io>
Co-authored-by: Eric Eastwood <erice@element.io>
Co-authored-by: Eric Eastwood <erice@element.io>
Co-authored-by: Eric Eastwood <erice@element.io>
3 tasks
Co-authored-by: Eric Eastwood <erice@element.io>
Co-authored-by: Eric Eastwood <erice@element.io>
Co-authored-by: Eric Eastwood <erice@element.io>
Co-authored-by: Eric Eastwood <erice@element.io>
Co-authored-by: Eric Eastwood <erice@element.io>
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds off #19424
Adds federation compatibility for MSC4242 state DAG rooms. Overview:
/get_missing_eventsand/send_joinrespectively)./send.Complement tests: matrix-org/complement#841
Pull Request Checklist
EventStoretoEventWorkerStore.".code blocks.