81 KiB
Element Call oxidation plan
Move Element Call's MatrixRTC participation logic off matrix-js-sdk's
MatrixRTCSession and onto the Rust matrix-rtc crate
(~/Projects/matrix-rust-rtc/MatrixSdkArchitectureDraft), consumed through its
uniffi wasm bindings. The Element Call component then depends on one
host-supplied object, a MatrixDriver, and never on a MatrixClient. The
standalone app and the widget become hosts like any other: they build a
MatrixDriver from their matrix-js-sdk client and hand it to the component.
Revision 2 — incorporates the independent review (§10 lists what changed and the assumptions taken where only the user can decide).
Status legend: ☐ todo · ◐ in progress · ☑ done.
Where things stand (2026-09-14): S0a, S0b, S1a, S1b and S2 are
implemented and green (pnpm lint, pnpm format:check, pnpm test:unit:
101 files / 785 tests). Nothing is committed yet, in either repository: the
crate changes (C2–C8; C1 was reverted in favour of slot opening; C11 and C12 done;
C9 and C10 are pending) sit uncommitted in
~/Projects/matrix-rust-rtc/MatrixSdkArchitectureDraft, and Element Call's
branch toger5/oxidation holds the vendored bindings, the driver layer
(src/driver/**), the participation layer (src/state/rtc/**) and the
config/lint changes. S3a is the next slice; its design is in §6 and the
files it touches are listed there.
1. Goals and non-goals
Goals
ElementCall(component) takesdriver: MatrixDriverinstead ofclient: MatrixClient. Nothing rendered underCallViewimportsMatrixClient,Room,RoomMember,MatrixEventormatrix-js-sdk/lib/matrixrtc.- All MatrixRTC participation logic (session projection, own membership
join/leave/keep-alive, transport tokens, media key exchange) comes from the
crate's
FfiParticipationManager. Element Call keeps only what the crate deliberately leaves to the host: the LiveKit media plane, tiles, room metadata, reactions, notifications, UI. - The standalone SPA, widget mode and
sdk/main.tsconstruct aJsSdkMatrixDriver(a port of the draft'sweb-test-app/src/jsSdkDriver.tsthat also works on js-sdk'sRoomWidgetClient, extended with what Element Call needs beyond RTC) and stop usingclient.matrixRTC. - Every existing gate stays green:
pnpm lint(tsc, oxlint, knip, component externals),pnpm format:check,pnpm test(unit + storybook),pnpm i18n:check, all four builds, Playwright (standalone, widget, component).
Non-goals
- Replacing
matrix-js-sdkin the standalone shell (login, registration, room creation, home page, crypto bootstrap). The shell keeps its client and wraps it.src/home/useGroupCallRooms.tsstays onclient.matrixRTCfor now. - Removing
matrix-js-sdk/lib/logger. It is isolated behindsrc/utils/logger.ts(S6) so a later swap is one line. - Writing a matrix-rust-sdk-backed driver (Element X). The interface is shaped so one can be written; none is written here.
- Publishing the crate as an npm package. Until it exists, the generated bindings are vendored (§5.1).
- Turning on MSC4153 (cross-signed sender) enforcement. Parity first (§5.8).
2. Where the code is today (inventory)
Entry: component/index.tsx → src/room/CallView.tsx → LobbyView |
ActiveCall (src/room/InCallView.tsx) | CallEndedView. ActiveCall
builds the view model with
createCallViewModel$(scope, rtcSession, matrixRoom, mediaDevices, muteStates, options, raisedHands$, reactions$, trackProcessorState$).
| Concern | Files | js-sdk surface used |
|---|---|---|
| Session memberships | src/state/SessionBehaviors.ts, src/useMatrixRTCSessionMemberships.ts |
rtcSession.memberships, MembershipsChanged, membership.getTransport, isKeyRotationSuppressed |
| Own membership | localMember/LocalMember.ts (enterRTCSession), localMember/HomeserverConnected.ts |
joinRTCSession, leaveRoomSession(1000), updateCallIntent, MembershipManagerEvent.*, ClientEvent.Sync; delegation probe at LocalMember.ts:298-313, delegation through the JWT service at :731-756 |
| SFU / JWT | src/livekit/openIDSFU.ts, localMember/LocalTransport.ts, localMember/RtcTransportAutoDiscovery.ts, remoteMembers/Connection.ts, ConnectionFactory.ts, ConnectionManager.ts |
getOpenIdToken, _unstable_getRTCTransports, POST /get_token (+ delay_id, delay_timeout, delay_cs_api_url), legacy /sfu/get |
| Remote member ↔ LiveKit identity | remoteMembers/MatrixLivekitMembers.ts |
rtcBackendIdentity, userId, deviceId, memberId |
| E2EE media keys | src/e2ee/matrixKeyProvider.ts, src/e2ee/sharedKeyManagement.ts |
EncryptionKeyChanged, reemitEncryptionKeys, room.hasEncryptionStateEvent |
| Room metadata | remoteMembers/MatrixMemberMetadata.ts, src/utils/displayname.ts, src/room/useRoomName.ts, useRoomState.ts, useRoomAvatar.ts, useJoinRule.ts, InviteModal.tsx, CallView.tsx |
getMembersWithMembership, RoomStateEvent.Members, room.name, getMxcAvatarUrl, getJoinRule, getCanonicalAlias |
| Own profile / avatars | src/profile/useProfile.ts, src/Avatar.tsx |
getUser, UserEvent.*, setDisplayName, setAvatarUrl, uploadContent, mxcUrlToHttp, getAccessToken |
| Reactions / hand raise | src/reactions/ReactionsReader.ts, useReactionsSender.tsx, src/reactions/index.ts |
RoomEvent.Timeline/Redaction/LocalEchoUpdated, MatrixEventEvent.Decrypted, relations.getChildEventsForEvent, sendEvent, redactEvent, membership eventId, RelationType |
| Call notifications | CallViewModel/CallNotificationLifecycle.ts |
DidSendCallNotification, RoomEvent.Timeline + EventType.RTCDecline |
| Rageshake / dev settings | src/settings/submit-rageshake.ts, rageshake.ts, FeedbackSettingsTab.tsx, DeveloperSettingsTab.tsx |
getCrypto, sendEvent(org.matrix.rageshake_request), ClientEvent.Event, secureRandomString, doesServerSupportUnstableFeature, getSFUConfigWithOpenID |
| Analytics | src/analytics/PosthogEvents.ts, PosthogAnalytics.ts |
rtcSession.statistics, account data |
| Types only | src/UrlParams.ts, src/state/MediaDevices.ts, AndroidControlledAudioOutput.ts, IOSControlledAudioOutput.ts, initialMuteState.ts, state/media/RingingMediaViewModel.ts (RTCCallIntent), src/useEvents.ts (TypedEventEmitter types) |
replaced by a local CallIntent type / kept as generic emitter typing |
| Runtime misc | src/useLocalStorage.ts (TypedEventEmitter), src/room/GroupCallErrorBoundary.tsx (MatrixError), src/room/KnockLobbyView.tsx (shell) |
see S6 |
| Context | src/ClientContext.tsx |
useClient/useClientState used by Avatar, sharedKeyManagement, useReactionsSender, submit-rageshake, DisconnectedBanner |
Hosts: component/index.tsx:291, src/room/useLoadGroupCall.ts:335,
sdk/main.ts:128 (own MatrixRTCSessionManager; waits on
MatrixRTCSessionEvent.JoinStateChanged at :292). Test kit:
src/utils/test.ts (MockRTCSession, mockRtcMembership, mockMatrixRoom),
src/utils/test-viewmodel.ts, CallViewModelTestUtils.ts. Baseline on
main (fe911628): tsc green, 97 unit files / 757 tests green.
Component build: vite-component.config.ts (single string fileName,
externals list with 18 matrix-js-sdk/lib/* subpaths), pnpm lint:externals,
component/package.json (matrix-js-sdk: "*" peer, exports without a
wildcard).
3. What the crate gives us, what it does not, and what must change in it
Verified against src/uniffi_api/mod.rs, src/participation/mod.rs,
src/session/state.rs, src/own_membership/machine.rs,
src/encryption/matrix_encryption_event.rs and the generated
web-test-app/src/generated/matrix_rtc.ts (acceptance suites: 32 pass).
Provided (FfiParticipationManager, one per (room, slot), any number
share one FfiMatrixDriver):
join(FfiTransportIntent, FfiJoinParams)/leave(code?, reason?); aPublishintent with a bare LiveKit transport triggers discovery throughdriver.getRtcTransports()(connections/mod.rs:470-499); a driver error there isNoTransport, not a fallback.memberships()+ listener:FfiMembership { member { memberId, userId, deviceId, displayName?, avatarUrl?, intent?, applicationType?, publishedTransports, canSubscribe }, state: Joined | LeftWithKeys, connections: serviceUrl[] (the FFI doc comment saying ws urls is wrong), transportIdentity?, mediaKey? }.transportIdentityis today'srtcBackendIdentity.LeftWithKeysentries have emptyconnections.connections()+ listener:{ connection { serviceUrl, wsUrl, jwtToken, expiresAtTs }, members }[]; tokens re-minted a minute beforeexp.keyMap()+setKeyMapListener(map, change):FfiMediaKey { memberId, key: ArrayBuffer, index, creationTsMs: bigint }, inbound keys and our own (encryption/inbound.rs:251-257).status()+ listener:Disconnected{cause} | Joining | Connected{keepAlive, membership, roster, encryption, impairments} | Leaving.session(),ownMemberId()(available as soon asjoin()starts),ownMembership(),connectionProblems(),debugSnapshot().- Member display names and avatars, from the room's
m.room.memberstate (C8): the session keeps them current, so a rename is a memberships change. - Slots are required: once the seed has read slot state, a slot with no event
is closed, so a room without an
m.rtc.slothas no call, in every dialect.openSlot(application, encrypted)/closeSlot()send the state event; the slot idm.call#ROOMmatches js-sdk's default. FfiElementCallCompat.{Off, StickyEvents, StateEvents}today;StickyEventsis removed by C9, leavingOff(spec MSC4143) andStateEvents(MSC3401).- All
u64fields arebigintin TypeScript;Vec<u8>isArrayBuffer. - Listener callbacks arrive one timer tick after the emitting call (pumps
sleep through
setTimeout,executor.rs:86); getters are fresh.
Not provided — Element Call keeps it, through the driver (§4.1 slices):
| Need | Why not in the crate | Where it goes |
|---|---|---|
| Room name, alias, avatar, join rule, encryption flag | room metadata | RoomDriver.getRoomInfo |
| Profiles of room members who are not in the call (ringing, name-tag threshold) | not RTC | RoomDriver.subscribeRoomMembers |
Reactions, hand raise, org.matrix.rageshake_request |
application events | TimelineDriver |
| MSC4075 notification + decline | out of scope (only wire_event_type knows the type) |
TimelineDriver.sendRoomEvent, decided in CallNotificationLifecycle (§4.3) |
| Own profile read/write | not RTC | ProfileDriver |
mxc:// thumbnails with auth |
not RTC | MediaDriver.thumbnailUrl |
| Homeserver sync connectivity | needed by the crate too | RtcMatrixDriver (isHomeserverConnected, subscribeConnectivity, C12); reaches Element Call as HomeserverUnreachable in the status |
| Sticky-events support probe | capability probe | MatrixDriver.getCapabilities() |
Must change in the crate (S0a) — each blocks a later slice:
| # | Problem | Change |
|---|---|---|
| C1 | A successful read_state("m.rtc.slot") returning [] marks slot state supplied and every slot other than the legacy "" resolves Closed (session/state.rs); join() then fails with SlotClosed and every MSC4143 peer is excluded. Element Call never sent m.rtc.slot. |
Kept as the crate has it: no slot means no call. Element Call opens the slot when nobody has (CallParticipation.join with a SlotPolicy: openSlot("m.call", encrypted), then wait for the echo), which needs the power level to send org.matrix.msc4143.rtc.slot; without it the join fails with NoOpenSlotError. Existing rooms keep working because the first call in a room opens its slot. A compat-mode relaxation was tried and reverted. |
| C2 | manage_media_keys, require_cross_signed_sender, use_key_delay_ms are not settable over the FFI; defaults are true, true, 1000 ms. |
New record FfiParticipationConfig { compat, manage_media_keys, require_cross_signed_sender, use_key_delay_ms } as the constructor argument (replaces the bare compat). |
| C3 | StickyEvents compat sent keys as org.matrix.msc4143.rtc.encryption_key while deployed clients read only io.element.call.encryption_keys. |
Made moot by C9 and reverted with it: the sticky dialect goes away entirely, so Off sends the spec key message and StateEvents the legacy one, with no middle case. |
| C4 | FfiMember has no membership event_id; reactions relate to it (§4.3). |
Member.event_id: Option<String> threaded through session/dispatch.rs → convert/* → state.rs (currently dropped at state.rs:402) → FfiMember.event_id. |
| C5 | Delegating the delayed leave (MSC4195) is split between the crate and the host: the draft's driver method makes the adapter perform the whole delegation, its demo adapter calls a homeserver endpoint that 401s on a widget client, and Element Call today does it differently (probe, then the authorisation service's get_token with delay_id/delay_timeout/delay_cs_api_url). Nothing of this exists in the real crates/ yet. |
The crate owns the policy; the driver keeps two primitives. (a) delegate_delayed_leave_via_homeserver(room, slot, member, delay_id): one authenticated POST to the CS API endpoint (/_matrix/client/unstable/io.element.msc4195/rtc/livekit/delegate_delayed_leave, the spelling Element Call probes today; a widget client answers Unsupported). (b) LivekitTokenRequest.delegation: Option<{ delay_id, delay_timeout_ms }>: the adapter appends delay_id, delay_timeout and delay_cs_api_url (its own homeserver URL) to the get_token / sfu/get body it already sends. The own-membership machine tries (a) first and, on Unsupported or any failure, (b) against the transport we publish on, i.e. Element Call's OpenID → JWT → scheduled-event path; only if both fail does it keep restarting the switch itself. Arm-after-confirm: the short delayed leave stays armed through the join; delegation arms a second, ≥ 1 h delayed leave, delegates that, and cancels the short one on success (or the long one on failure), so no moment is left without an armed leave and a failed delegation costs nothing. KeepAlive::Delegated says which route succeeded. The interim C5 (service URL and delay on the request) and Element Call's driver-side probe and JWT delegation are replaced by this. |
| C6 | The transport identity is a pure function (connections/mod.rs:116-135) but not exported; the own identity is needed before our membership echo to set our own media key. |
Export FfiParticipationManager.own_transport_identity(): Option<String>. |
| C7 | Doc comment on FfiMembership.connections says ws_urls. |
Fix the comment. |
| C8 | Both converters set display_name / avatar_url to None, so every host would re-derive them from room members. |
The session records each m.room.member profile (whether or not the roster condition is enforced) and stamps it on members at projection time. |
| C9 | ElementCallCompat::StickyEvents models the 2025 Element Call sticky dialect (member: {user_id, device_id, id}, flat rtc_transports, versions, legacy key message). No deployment uses it: sticky-event calls have not shipped. |
Remove the mode. Delete own_membership/compat_2025.rs, the 2025 block in session/convert/msc4143.rs and its dispatch arm, the StickyEvents variants of ElementCallCompat / FfiElementCallCompat, and their tests and acceptance tests; outbound_event_type / build_content keep two arms (Off, StateEvents). Element Call then maps Matrix_2_0 → Off: spec MSC4143 sticky events with slots (which Element Call opens, C1) and the spec key message. |
| C10 | MediaKeyState has holds_our_key, have_their_key and rejection, so a tile learns about an unsigned sender only when require_cross_signed_sender is on (the key is discarded, rejection: NotCrossSigned). With the check off the verdict is dropped on accept and the tile cannot show an unverified sender. |
Keep the MSC4153 verdict of the accepted key per member and expose it as FfiMediaKeyState.sender_cross_signed: Option<bool> (None when the host could not tell). Element Call runs with the check off (§5.8) and wants to show the state on the tile until it is turned on. |
| C11 | No way to change application["m.call.intent"] while joined; Element Call flips it between audio and video when the camera is toggled (updateCallIntent). |
update_application(intent) on the own-membership manager, facade and FFI: while connected the membership is re-published at once on the refresh path (a failure retries like a refresh); during a join the join event carries it; refused with NotJoined otherwise. |
| C12 | Homeserver connectivity lived only in Element Call's driver; the crate could not tell a dead homeserver from a quiet one, and a participation's status said nothing about it. | Done. ConnectivityDriver (is_homeserver_connected, subscribe_connectivity) joins the MatrixDriver sum; the FFI adds ConnectivitySink, the two callback methods and FfiParticipationManager.is_homeserver_connected(); the facade pump consumes the stream and reports Impairment::HomeserverUnreachable { since_ts } (Critical, sorted first) in every non-disconnected status until the driver reports the homeserver back. The web-test-app mock and js-sdk driver implement it. A matrix-rust-sdk adapter implements the same two methods later. |
No crate work is deferred: update_application is C11.
4. Target architecture
host (SPA · widget · sdk · a third-party page)
├─ RtcMatrixDriver = the crate's MatrixDriverCallback, verbatim
│ (events, to-device, tokens, sinks, connectivity)
└─ ElementCallMatrixClientDriver = RoomDriver + TimelineDriver + ProfileDriver
+ MediaDriver + capabilities
│
▼ component/index.tsx <ElementCall rtcDriver clientDriver …/>
┌─ MatrixDriverProvider (src/driver/MatrixDriverContext.tsx) ─────────────┐
│ CallView owns one CallParticipation for lobby → call → ended │
│ CallParticipation (src/state/rtc/CallParticipation.ts) │
│ FfiMatrixDriver(driver) → FfiParticipationManager(room, slot, me, cfg)│
│ memberships$ · connections$ · keyChanges$ · status$ · session$ │
│ ownMemberId$ · ownMembership$ · ownTransportIdentity$ │
│ join(intent, params) · leave(reason) │
│ │ │
│ ▼ │
│ createCallViewModel$(scope, callParticipation, roomInfo, mediaDevices, …) │
│ ConnectionManager ← connections$ (wsUrl + jwt; no OpenID in EC) │
│ RemoteMembers ← memberships$ (transportIdentity ↔ LK participant)│
│ MatrixKeyProvider ← keyChanges$ (memberId → transportIdentity) │
│ LocalMember ← status$, ownMembership; join/leave → participation│
│ MemberMetadata ← driver.room members │
│ Notifications ← driver.timeline + memberships$ │
│ React: Lobby / InCall / Settings / Avatar / Reactions │
│ read the driver via useMatrixDriver(), never a client │
└──────────────────────────────────────────────────────────────────────────┘
4.1 The two drivers (host-facing, framework-neutral)
Two files, two objects. src/driver/RtcMatrixDriver.ts is one line: the
crate's MatrixDriverCallback, re-exported. Everything MatrixRTC, including
homeserver connectivity (C12), goes through it and is consumed by the crate.
src/driver/ElementCallMatrixClientDriver.ts is what a call needs beyond
MatrixRTC. Plain TypeScript: async methods and
subscribeX(listener) → unsubscribe pairs, mirroring the crate's sink style.
No RxJS crosses this boundary; Element Call wraps subscriptions into
Behaviors internally (src/driver/observe.ts).
export interface ElementCallMatrixClientDriver
extends RoomDriver, TimelineDriver, ProfileDriver, MediaDriver {
readonly userId: string;
readonly deviceId: string;
/** The room this driver is bound to (one driver per room, as in the crate). */
readonly roomId: string;
getCapabilities(): Promise<DriverCapabilities>;
/** Free-form diagnostics for rageshakes (crypto version, sync state, …). */
getDiagnostics?(): Promise<Record<string, string>>;
}
export interface DriverCapabilities {
stickyEvents: boolean;
/** The host's events carry decryption metadata (false on a widget client). */
verifiedEventOrigins: boolean;
/** The host can evaluate MSC4153 cross-signing of senders. */
crossSigningVerdicts: boolean;
}
// src/driver/RtcMatrixDriver.ts
export type RtcMatrixDriver = MatrixDriverCallback;
export interface RoomInfo {
name: string;
canonicalAlias: string | null;
avatarUrl: string | null;
joinRule: string | null;
encrypted: boolean;
}
export interface RoomMemberProfile {
userId: string;
displayName: string | null;
avatarUrl: string | null;
membership: "join" | "invite";
}
export interface RoomDriver {
getRoomInfo(): RoomInfo;
subscribeRoomInfo(listener: (info: RoomInfo) => void): () => void;
getRoomMembers(): RoomMemberProfile[];
subscribeRoomMembers(
listener: (members: RoomMemberProfile[]) => void,
): () => void;
}
export interface TimelineEvent {
eventId: string;
type: string;
sender: string;
content: Record<string, unknown>;
originServerTs: number;
redacts?: string;
}
export interface TimelineDriver {
sendRoomEvent(
eventType: string,
content: unknown,
): Promise<{ eventId: string }>;
redactEvent(eventId: string): Promise<void>;
/** Decrypted live room events (not sticky), incl. redactions; no local echoes. */
subscribeTimeline(listener: (event: TimelineEvent) => void): () => void;
/** Events already known that relate to `eventId` (hand-raise catch-up). */
getRelatedEvents(
eventId: string,
relType: string,
eventType: string,
): TimelineEvent[];
}
export interface OwnProfile {
displayName: string | null;
avatarUrl: string | null;
}
export interface ProfileDriver {
getOwnProfile(): OwnProfile;
subscribeOwnProfile(listener: (profile: OwnProfile) => void): () => void;
/** Absent when the host does not allow profile changes. */
setDisplayName?(name: string): Promise<void>;
setAvatar?(file: Blob): Promise<void>;
}
export interface MediaDriver {
/** An `<img>`-usable URL for an mxc thumbnail (may be a blob: URL), or null. */
thumbnailUrl(
mxcUrl: string,
width: number,
height: number,
resizeMethod: "crop" | "scale",
): Promise<string | null>;
}
The client driver is a union of capability slices, the way the crate splits its own driver; the RTC driver is the crate's contract untouched, so a host with a crate-side adapter (matrix-rust-sdk) implements nothing extra for MatrixRTC.
4.2 CallParticipation (Element Call's RxJS view of the manager)
Naming: participation is the crate's FFI concept (FfiParticipationManager,
FfiParticipationConfig); CallParticipation is Element Call's RxJS wrapper
over it.
src/state/rtc/CallParticipation.ts, a class taking the scope in its constructor:
new CallParticipation(scope, driver, {
slotId: "m.call#ROOM", compat, manageMediaKeys, requireCrossSignedSender,
useKeyDelayMs, transportFallbackUrl?, logger })
memberships$: Behavior<Epoch<FfiMembership[]>> // Joined only; LeftWithKeys filtered (v1)
connections$: Behavior<FfiConnectionWithMembers[]>
keyChanges$: Observable<FfiMediaKey> // one changed key per emission
keyMap$: Behavior<FfiMediaKey[]>
status$: Behavior<FfiStatus>
session$: Behavior<FfiSessionSnapshot>
ownMemberId$: Behavior<string | null>
ownTransportIdentity$: Behavior<string | null>
ownMembership$: Behavior<FfiMembership | null>
join(intent: FfiTransportIntent, params: FfiJoinParams): Promise<void>
leave(code?: string, reason?: string): Promise<void>
- Wraps
new FfiMatrixDriver(driver)andnew FfiParticipationManager(...);uniffiDestroy()on scope end (leave first unlessDisconnected). transportFallbackUrldecoratesgetRtcTransports: when the host's call throws or returns no LiveKit transport, answer withConfig.get().livekit.livekit_service_url(today's precedence,RtcTransportAutoDiscovery.ts:72-94).- Lives at
CallViewlevel (the lobby reads memberships for the participant count, auto-mute threshold and notification decision,CallView.tsx:164-413;useReactionsSenderneeds the own membership).join()afterleave()on one manager is supported and mints a fresh member id. - Behaviors are seeded from the getters and updated by the listeners.
4.3 createCallViewModel$ after the change
createCallViewModel$(
scope,
participation,
roomInfo,
mediaDevices,
muteStates,
options,
handsRaised$,
reactions$,
trackProcessorState$,
);
// roomInfo: { roomId, userId, deviceId, members$: Behavior<RoomMemberProfile[]>,
// homeserverConnected$: Behavior<boolean>, timeline: TimelineDriver }
| Today | After |
|---|---|
createMemberships$(scope, rtcSession) |
callParticipation.memberships$ |
membershipsAndTransports$ (getTransport(oldest)) |
membership.connections[] (service urls) |
createLocalTransport$ + RtcTransportAutoDiscovery + openIDSFU |
deleted; join(Publish(custom url or bare)); own transport = ownMembership.member.publishedTransports[0] |
Connection.start() fetching a JWT |
Connection takes { wsUrl, jwt, expiresAtTs } from connections$, keyed by serviceUrl, holds token$; a refreshed token is used on the next full (re)connect; expiresAtTs logged on connect; livekitAlias decoded from the JWT stays |
createRemoteMatrixLivekitMembers$ on rtcBackendIdentity |
matches membership.transportIdentity; key = member.memberId |
MatrixKeyProvider.setRTCSession |
MatrixKeyProvider.attach(participation): keyChanges$ × memberships$ × ownTransportIdentity$ → onSetEncryptionKey(material, identity, index); keys whose member has no identity yet are held per member id and replayed |
enterRTCSession (joinRTCSession(...)) |
callParticipation.join(intent, joinParamsFromConfig(...), { encrypted: roomInfo.encrypted, canOpen: roomInfo.canOpenSlot }) in the same scope.reconcile: opens the room's slot first when none is open (power level permitting, otherwise NoOpenSlotError), then joins; cleanup calls callParticipation.leave() |
createHomeserverConnected$ |
status$ alone: Impairment::HomeserverUnreachable (C12) = disconnected; Connected with keepAlive Armed/Delegated/Unavailable = connected; RestartFailing/Expired = reconnecting (behaviour change: local media pauses in that window, today it does not). Outside a participation, CallParticipation.homeserverConnected$ from the manager's getter |
delayId$ + JWT-service delegation |
gone from Element Call. FfiJoinParams.delegateDelayedLeave is always true; the crate tries the CS API, then the authorisation service's token endpoint (with delay_id, delay_timeout, delay_cs_api_url), then falls back to its own restarts (C5). config.matrix_rtc_session.delegated_delayed_leave.delay_ms becomes FfiJoinParams.delegatedDelayMs (default 1 h) |
createMatrixMemberMetadata$(scope, matrixRoom) |
tiles read member.displayName / avatarUrl from the crate; disambiguation runs over the call's members; roomInfo.members$ remains only for the ringing name and the name-tag threshold |
createSentCallNotification$ / createReceivedDecline$ |
CallNotificationLifecycle: after our own membership echo (ownMembership$ non-null) and when no other member was in the session before our join, send m.rtc.notification (wire org.matrix.msc4075.rtc.notification) via driver.sendRoomEvent with the fields js-sdk sends today (m.mentions, notification_type, sender_ts, lifetime 90 s, m.call.intent, m.relates_to: m.reference → own membership event id, MatrixRTCSession.ts:725-756); decline from driver.subscribeTimeline on both org.matrix.msc4310.rtc.decline and m.rtc.decline |
updateCallIntent on camera toggle |
callParticipation.updateApplication(videoEnabled ? "video" : "audio") (C11); Element X's room-header intent keeps following the camera |
createKeyRotationSuppressed$ + key_rotation_participant_limit |
dropped (the crate has no participant limit; the indicator has no equivalent) |
rtcSession.statistics (PostHog) |
counts of keyChanges$ (sent = own member id, received = others) |
MembershipManagerError → StickyEventsRequiredError |
status$ Disconnected{cause: JoinFailed{error: Driver/Unsupported}} → StickyEventsRequiredError; NoTransport → MatrixRTCTransportMissingError; ManagerStopped/SlotClosed → ConnectionLostError |
impairments |
fed into the inert src/state/ServiceInterruptionsViewModel.ts (S6) |
Join parameters (joinParamsFromConfig, from Config.get().matrix_rtc_session):
stickyDurationMs = BigInt(Math.min(membership_event_expiry_ms ?? 4 h, 1 h))
(js-sdk caps sticky at 1 h, the crate clamps to 1 h; the config default is
undefined today); keepAliveTimeoutMs = BigInt(delayed_leave.delay_ms);
degradedLifetimeMs = undefined; applicationType = "m.call";
intent = options.callIntent. Config keys that stop having an effect and are
documented as such in docs/: delayed_leave.restart_ms,
restart_timeout_ms, network_error_retry_ms,
key_rotation_participant_limit, delegated_delayed_leave.* (the crate arms
1 h when delegating). wait_for_key_rotation_ms maps to
FfiParticipationConfig.useKeyDelayMs.
Compat: MatrixRTCMode.Compatibility → StateEvents, Matrix_2_0 → Off (spec
MSC4143: sticky member events, slots, the spec key message). Until C9 lands the
code still maps Matrix_2_0 to the crate's StickyEvents; compatForMode is
the one place that changes.
4.4 React tree
src/driver/MatrixDriverContext.tsx:MatrixDriverProviderholding both drivers,useRtcMatrixDriver()/useClientDriver(); replaces everyuseClient()/useClientState()underCallView.ClientContextstays for the shell.CallViewprops:{ driver, isPasswordlessUser, confineToRoom, preload, skipLobby }. It creates theCallParticipation(scope tied to its mount) and hands it toLobbyView,ActiveCall,useReactionsSender.MatrixInfocomes fromdriver.getRoomInfo()/driver.getOwnProfile().InCallView's own id becomes${driver.userId}:${driver.deviceId}.Avatar→driver.thumbnailUrl(host bridgedownloadMediafirst).useProfile(client)→useOwnProfile();ProfileSettingsTabhides editing whensetDisplayNameis absent.ReactionsReader(scope, participation, driver),useReactionsSenderoverTimelineDriver. Relation target stays the current own membership event id (member.eventId, C4) — protocol status quo; the reader keys raised hands bymemberIdand re-resolves the event id on re-send instead of dropping the hand.useRoomEncryptionSystemreadsgetRoomInfo().encrypted.submit-rageshake:useMatrixDriver()for ids andgetDiagnostics?(); rageshake requests viasubscribeTimeline.DeveloperSettingsTab: sticky probe →getCapabilities(); custom LiveKit URL validation →driver.getLivekitToken(...).DisconnectedBanner→HomeserverUnreachableinstatus$(C12).window.rtcSessiondebug handle →window.matrixRtc = { participation }.
4.5 Hosts
- Component:
ElementCallProps.rtcDriver: RtcMatrixDriverandclientDriver: ElementCallMatrixClientDriver;roomIdstays as an optional prop for one release and must equalclientDriver.roomIdwhen given (assertion), then goes.initializeElementCall(config, { matrixRtcWasm? })awaitsinitMatrixRtcSdk(). Externals shrink toreact*,livekit-client,matrix-js-sdk/lib/logger.JsSdkRtcMatrixDriverandJsSdkElementCallMatrixClientDriverare exported from a second entry@element-hq/element-call-component/matrix-js-sdk(needslib.fileNameas a function and a newexportskey); only that entry hasmatrix-js-sdkas a peer. - SPA / widget:
useLoadGroupCallreturns theRoom;RoomPagememoises the two js-sdk drivers and renders<CallView rtcDriver clientDriver>. The widget capability list insrc/widget.tsgrows by: send and receive statem.rtc.slotandorg.matrix.msc4143.rtc.slot(opening the slot),m.room.avatar,m.room.canonical_alias,m.room.join_rules; send/receive to-deviceorg.matrix.msc4143.rtc.encryption_keyandm.rtc.encryption_key(alongsideio.element.call.encryption_keys); eventsm.rtc.decline. sdk/main.ts: builds the driver from the widget client, aCallParticipation, and waits onstatus$instead ofJoinStateChanged.component/devharness: the two js-sdk drivers per pane.
4.6 The js-sdk drivers — two classes, two clients each
src/driver/jsSdk/JsSdkRtcMatrixDriver.ts (the crate seam, a port of the
draft's jsSdkDriver.ts, connectivity from ClientEvent.Sync) and
JsSdkElementCallMatrixClientDriver.ts (the §4.1 slices), both written so
that they work on a full MatrixClient
and on js-sdk's RoomWidgetClient (node_modules/matrix-js-sdk/src/embedded.ts).
Differences from the draft, all required by the widget client:
- to-device inbound: listen on
ClientEvent.ToDeviceEvent(the widget client never emitsReceivedToDeviceMessage,embedded.ts:788-802); on a full client useReceivedToDeviceMessagefor theencryptionInfo. - to-device outbound: always
encryptAndSendToDevice(works without a crypto backend on the widget client,embedded.ts:598-612; plainsendToDevicethere is unencrypted,:614-619). - transports:
client._unstable_getRTCTransports()(the widget override,embedded.ts:641-646), never a rawhttp.authedRequest;.well-knownfallback only on a full client. - event origins: on a full client from decryption metadata (as the draft); on
a widget client events arrive decrypted without metadata, so the driver
reports
Encrypted{ senderDeviceId: content.member.device_id }for member events andEncrypted{ senderDeviceId: content.device_id }for key events, i.e. the claimed trust level js-sdk applies today (ToDeviceKeyTransport.ts:133-140).getCapabilities().verifiedEventOriginssays which. - cross-signing verdict:
undefinedon a widget client (crossSigningVerdicts: false); Element Call then forcesrequireCrossSignedSender = false. - delegation: two primitives and no policy.
delegateDelayedLeaveViaHomeserveris oneauthedRequeston a full client andUnsupportedon a widget client;getLivekitTokenappendsdelay_id,delay_timeoutanddelay_cs_api_url(client.baseUrl) when the request carries a delegation. The crate decides when to call which (C5). Until C5 lands the driver still carries Element Call's probe and JWT delegation; both go then. - sticky listener attached after
startClient()resolves (the widget room only exists then,embedded.ts:326). getLivekitTokenreuses today's request shapes (slot_id: "m.call#ROOM", legacy/sfu/get), errors mapped toRtcErrorincl.M_LIMIT_EXCEEDED → RateLimited, 403 →Rejected, 404/M_UNRECOGNIZED→Unsupported.
5. Decisions and assumptions
- Vendored bindings.
src/matrix-rtc-sdk/generated/holdsmatrix_rtc.ts,matrix_rtc-ffi.ts,wasm-bindgen/index.js,index_bg.wasmand a hand-writtenwasm-bindgen/index.d.ts(noallowJsintsconfig.json), synced byscripts/sync-matrix-rtc-sdk.sh(runsubrn build webin the draft without theruntime-probefeature, copies). Committed so CI works.@ubjs/corebecomes a dependency. The wasm is ~6.5 MB unoptimised. Assumption: committing the binary is acceptable for the draft phase. - Wasm loading. Verified: Vite 8 library mode inlines
?urlandnew URL(…, import.meta.url)assets as base64 regardless ofassetsInlineLimit;?url&no-inlineemits a file. App builds use?url; the component build uses?url&no-inlineplus anexportsentry for./dist/assets/*, andinitializeElementCall(config, { matrixRtcWasm })lets a host point elsewhere. vitest reads the file from disk; wasm boot is lazy (only suites that need it callinitMatrixRtcSdk()), never insrc/vitest.setup.ts; Storybook boots it in.storybook/preview.tsxbeforeAll. Suites usingvi.useFakeTimersnever share a file with real-wasm tests (pumps sleep onsetTimeout). - Own identity via the driver.
userId/deviceIdare properties of the driver, not props. - One driver per room, as in the crate.
matrix-js-sdk/lib/loggerstays behindsrc/utils/logger.ts.updateCallIntentstays, through the crate'supdate_application(C11).- Reactions relate to the current own membership event id (status quo);
reader keys by
memberId. Alternatives (stable join event id, ormemberIdas relation target) are protocol changes left to the user. - MSC4153 default off (
requireCrossSignedSender = false) for every host, confirmed: js-sdk performs no such check today, and a passwordless SPA peer would otherwise be inaudible to everyone. The intent is to turn it on; the default carries a TODO (src/state/rtc/joinParams.ts), and C10 puts the sender's verdict on the tile so the UI can show it meanwhile. - Delegation is the crate's alone. Element Call always asks for it; the crate tries the CS API, then the authorisation service's token endpoint, then its own restarts, and arms the long delegated leave only once delegation is confirmed (C5). Element Call keeps no probe and no delegation code.
- Widget trust model: origins synthesised from claimed device ids equal
today's js-sdk trust level; the crate records them as
DeviceAttribution::Verifiedbecause it cannot tell. Documented in the driver; aClaimedattribution flag on the sink is a follow-up crate ask. - Scratch files go to
agent-workspace/oxidation/; this plan lives at the repo root because it was asked for by name. - Branch
toger5/oxidation, one commit per slice.
6. Work breakdown
Each slice is independently green (pnpm lint && pnpm test:unit at least).
S0a — crate changes ☑ (in MatrixSdkArchitectureDraft)
C2–C12 from §3 (C1 reverted), each with a Rust unit test; acceptance tests in
web-test-app/test/: a slot-less room refuses joins until openSlot, in every dialect;
FfiParticipationConfig with manageMediaKeys: false exchanges no keys;
no StickyEvents variant left anywhere (C9); member.eventId present;
delegation tries the homeserver first, then the token endpoint with the delay fields, and keeps the short leave armed until one succeeds;
ownTransportIdentity() equals the joined membership's transportIdentity.
Gate: cargo test --features uniffi, cargo clippy --all-targets --features uniffi -- -D warnings,
npm run ubrn:web && npm test.
S0b — SDK intake ☑
scripts/sync-matrix-rtc-sdk.sh,src/matrix-rtc-sdk/generated/**,src/matrix-rtc-sdk/index.ts(loader + curated re-exports),src/matrix-rtc-sdk/index.test.ts(boot, one join round-trip against the TS mock driver).package.json(@ubjs/core),knip.ts(ignoreforgenerated/**),.oxlintrc.jsonignorePatterns,.oxfmtrc.jsonignore,vite.config.ts(nothing needed for?url; verifiedvite-plugin-wasmignores it),tsconfig.jsonuntouched thanks to the.d.ts.- Gate:
pnpm lint && pnpm format:check && pnpm test:unit.
S1a — driver interface, mock driver ☑
src/driver/RtcMatrixDriver.ts,src/driver/ElementCallMatrixClientDriver.ts,src/driver/observe.ts,MockRtcMatrixDriver.ts(port ofweb-test-app/src/mockDriver.ts) andMockElementCallMatrixClientDriver.ts(in-memory room info, members, timeline, profile), with a test each.knip.tsignoreforsrc/driver/**until consumed, with the reason.
S1b — JsSdkMatrixDriver ☑
src/driver/jsSdk/JsSdkRtcMatrixDriver.tsandJsSdkElementCallMatrixClientDriver.ts(§4.6) + tests against two fakes: aMatrixClient-shaped one (mockMatrixRoom) and aRoomWidgetClient-shaped one (ToDeviceEvent, no crypto,_unstable_getRTCTransports, sticky updates afterstartClient). Asserts request shapes of_unstable_sendStickyEvent,_unstable_sendStickyDelayedEvent,_unstable_updateDelayedEvent,encryptAndSendToDevice,/get_tokenbody with and without the delegation fields,delegateDelayedLeaveViaHomeserveron both clients, sink emission and origin synthesis, room-info/member updates.
S2 — CallParticipation ☑
src/state/rtc/CallParticipation.ts,joinParams.ts,transportIntent.ts,errors.ts(cause →ElementCallError).CallParticipation.test.tsthrough the real wasm +MockMatrixDriver: memberships follow a remote join/leave;LeftWithKeysfiltered; join →Connected;connections$carries the token;keyChanges$fires for a peer key;ownTransportIdentity$set before the echo; leave →Disconnected{LeftByHost}; join → leave → join; scope end destroys the manager; fallback transport when the host throws or advertises none.
S3 — view model, four slices ☐
- S3a
Connection/ConnectionManager/ConnectionFactorykeyed byserviceUrl, fed byBehavior<{ serviceUrl, wsUrl, jwt, expiresAtTs }[]>, with a temporary adapter from today'sSFUConfigsoopenIDSFUstays until S3c. IntroducesmockCallParticipation()andmockFfiMembership()insrc/utils/test.ts. - S3b
MatrixLivekitMembers+MatrixKeyProvideronFfiMembership/ key changes, with an adapter fromCallMembershipfor the still-js-sdkmemberships$. - S3c
LocalMemberoncallParticipation.join/leave+ status-derived connectivity; deleteLocalTransport.ts,RtcTransportAutoDiscovery.ts,HomeserverConnected.ts,openIDSFU.ts,enterRTCSession. - S3d
createCallViewModel$signature,CallNotificationLifecycle,MatrixMemberMetadata,SessionBehaviors.tsanduseMatrixRTCSessionMemberships.tsdeleted,ReactionsReaderonparticipation+TimelineDriver; test kit swapped (MockRTCSession/mockRtcMembershipdeleted). To stay green before S5,CallViewbuilds aJsSdkMatrixDriverfrom its existingclient/rtcSession.roomprops as a temporary shim.
S4 — React tree, two slices ☐
- S4a views/hooks/settings on the driver:
CallView.tsx(ownsCallParticipation),InCallView.tsx,LobbyView.tsx,CallEndedView.tsx,VideoPreview.tsx,useRoomInfo()(replacesuseRoomName/Avatar/JoinRule/State),InviteModal.tsx,Avatar.tsx,useOwnProfile.ts,ProfileSettingsTab.tsx,SettingsModal.tsx,DeveloperSettingsTab.tsx,submit-rageshake.ts,DisconnectedBanner.tsx,analytics/PosthogEvents.ts,controls.ts, and a firstCallView.stories.tsx(lobby, in call, ended) driven byMockMatrixDriver. - S4b reactions and notifications:
useReactionsSender.tsx,ReactionsReaderkeyed bymemberId,CallNotificationLifecyclesending through the driver; tests incl. a membership re-send mid-call.
S5 — hosts ☐
component/index.tsx,component/matrix-js-sdk.ts,component/package.json(exports, peers),vite-component.config.ts(two entries,fileNamefunction, externals shrink,?url&no-inline),component/tsconfig.build.json,component/dev/Harness.tsx,README.md.src/room/useLoadGroupCall.ts,RoomPage.tsx,src/widget.ts(capabilities, §4.5),sdk/main.ts,playwright/spa-helpers.ts(delegation route helper),docs/config notes.- Gate: all four builds; Playwright standalone + widget + component against
pnpm backend; a widget media-key round trip andreconnect.spec.tsre-checked under the new delegation path.
S6 — fence and cleanup ☐
src/utils/logger.ts; localCallIntenttype for the sixRTCCallIntentusers;useLocalStorage.tson a local emitter; oxlintno-restricted-importsscoped to the call tree (allow-list:src/home,src/auth,src/utils/spa.ts,src/utils/matrix.ts,src/driver/jsSdk/**,src/ClientContext.tsx,src/widget.ts,src/initializer.tsx,src/IndexedDBWorker.ts,src/room/KnockLobbyView.tsx,src/settings/rageshake.ts) banningmatrix-js-sdkexceptmatrix-js-sdk/lib/logger.ServiceInterruptionsViewModelfed fromstatus$.impairments.docs/agents/architecture.md,docs/matrix_rtc_modes.mdupdated;src/@types/matrix-js-sdk.d.tsremoved if nothing merges into js-sdk types.
7. Risks and mitigations
| Risk | Mitigation |
|---|---|
| A user without the power level to open a slot, in a room that never had a call | NoOpenSlotError names the remedy; a room admin opens the slot by starting the first call |
| A client on the removed sticky dialect (none is deployed) | Not supported: Off and StateEvents are the only dialects (C9) |
| Component bundle size / wasm delivery | ?url&no-inline + exports wildcard + matrixRtcWasm override; verified in S0b and S5 |
| Token refresh vs LiveKit reconnect | Connection.token$; full reconnect uses the latest token; expiresAtTs logged |
| Own identity before the echo | C6 export; JWT sub only as a logged cross-check |
| Keys before identity | key provider buffers per member id and replays |
| Widget mode regressions (origins, to-device event, capabilities) | S1b tests against a RoomWidgetClient fake; S5 widget e2e adds a media-key round trip |
| Delegation fails after the long leave was armed | Arm-after-confirm (C5): the short leave stays armed until delegation is confirmed |
| Behaviour drift in keep-alive (6 s vs 4 s restarts) and dropped config keys | documented in docs/ |
| Test time: wasm per file | lazy boot in the suites that need it only |
8. Verification matrix
| Check | S0a | S0b | S1 | S2 | S3 | S4 | S5 | S6 |
|---|---|---|---|---|---|---|---|---|
cargo test/clippy, web-test-app suites |
● | |||||||
pnpm lint + format:check |
● | ● | ● | ● | ● | ● | ● | |
pnpm test:unit |
● | ● | ● | ● | ● | ● | ● | |
pnpm test:storybook |
● | ● | ● | |||||
| four builds | ● | ● | ● | |||||
| Playwright standalone + widget + component | ● | ● | ||||||
| Manual: two harness panes hear each other, E2EE, hand raise, reaction, leave | ● |
9. Open questions for the user (answered by the assumptions in §5 until told otherwise)
- Slot semantics, answered: no slot means no call; Element Call opens the slot on the first call in a room, power level permitting.
- Delegation, answered: CS API endpoint first, then Element Call's OpenID → JWT → scheduled-event path, all inside the crate and invisible to Element Call (C5).
- Media-key type under the sticky compat, answered: there are no deployed sticky-event clients, so the compat mode is removed (C9) and the question with it.
- MSC4153, answered: off everywhere for now, with a TODO to turn it on and C10 so tiles show unverified senders in the meantime.
- Reactions relation target: current membership event id (taken), stable join
event id, or
memberId? - Committing the 6.5 MB wasm (taken) vs a build-time fetch.
roomIdprop: optional for one release (taken) vs removed outright.updateCallIntent, answered: added to the crate asupdate_application(C11); Element Call keeps the behaviour.
10. Review log
Independent review findings incorporated in this revision: slot enforcement
blocker (C1); delegation protocol and 1 h arm (C5, §5.9; later redefined as crate-only with arm-after-confirm); StickyEvents key
type interop (C3, then made moot by removing the mode, C9); MSC4153 default (§5.8); widget-client differences for the
js-sdk drivers (§4.6) and missing widget capabilities (§4.5); own identity
export (C6); CallParticipation lifetime at CallView (§4.2); notification
timing and content (§4.3); reactions/event-id semantics (§5.7); config mapping
defaults and dropped keys (§4.3); bigint/ArrayBuffer types; inventory
gaps (§2); knip ignore vs ignoreFiles, oxlint/oxfmt ignores, .d.ts for
the glue, lazy wasm boot, Storybook beforeAll; slice re-cut (S0a/b, S1a/b,
S3a–d, S4a/b, temporary CallView shim); sdk/main.ts status wiring and the
Playwright delegation helper.