Matrix/element-call-Github
1
0
Fork 0
mirror of https://github.com/vector-im/element-call.git synced 2026-10-16 23:45:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

[SDK] Update SdkArchitecture.md

Browse Source
This commit is contained in:
Timo K.
2026-10-08 13:53:57 +02:00
parent 3cdb452d5d
commit 66f326fcf0
1 changed files with 34 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sdk/SdkArchitecture.md
+34 -16
View File
@@ -24,7 +24,7 @@ therefore `Element Call → sdk`, never the reverse. The migration from today's
│ MatrixRTCClient │ │ MatrixRTCClient │
│ │ │ │
│ public API, all Behaviors: │ │ public API, all Behaviors: │
│ join() leave() status$ connected$ reconnecting$ fatalError$ │ │ join() leave() state$ │
│ localMember$ remoteMembers$ memberCount$ sendData() data$ │ │ localMember$ remoteMembers$ memberCount$ sendData() data$ │
│ keyRotationSuppressed$ connectedTransports$ │ │ keyRotationSuppressed$ connectedTransports$ │
│ │ │ │
@@ -44,8 +44,8 @@ therefore `Element Call → sdk`, never the reverse. The migration from today's
│ │ leave, retries │ mediaKeys$ ────▶│ one LiveKit room per │ │ │ │ leave, retries │ mediaKeys$ ────▶│ one LiveKit room per │ │
│ │ discovery of the local │ mediaFor$ ─────▶│ transport url, local │ │ │ │ discovery of the local │ mediaFor$ ─────▶│ transport url, local │ │
│ │ transport (homeserver) │ ◀─ tracks$, │ published, remote │ │ │ │ transport (homeserver) │ ◀─ tracks$, │ published, remote │ │
│ │ join state machine │ encryptionError$│ subscribed │ │ │ │ membership state machine │ encryptionError$│ subscribed │ │
│ │ (LocalMember) │ local.publish, │ Publisher: local tracks, │ │ │ │ (LocalMembership) │ local.publish, │ Publisher: local tracks, │ │
│ │ transport registry │ setPublishing▶│ upstream paused until │ │ │ │ transport registry │ setPublishing▶│ upstream paused until │ │
│ │ (TransportMetadata) │ ◀─ connections$ │ joined │ │ │ │ (TransportMetadata) │ ◀─ connections$ │ joined │ │
│ │ data packet → member │ ◀─ data$ │ LivekitMemberMedia: │ │ │ │ data packet → member │ ◀─ data$ │ LivekitMemberMedia: │ │
@@ -61,11 +61,12 @@ therefore `Element Call → sdk`, never the reverse. The migration from today's
``` ```
The left box is MatrixRTC: who is in the session, with which transport, the The left box is MatrixRTC: who is in the session, with which transport, the
keys, the delayed leave, and the join state machine. The right box is a media keys, the delayed leave, and the membership state machine. The right box is a
backend: whatever carries the media, behind the `MediaBackend` interface in media backend: whatever carries the media, behind the `MediaBackend` interface in
`sdk/src/media-backend/api.ts`. The client is the join of the two: a member exists `sdk/src/media-backend/api.ts`. The client is the join of the two: a member exists
once its membership does, and gets its media once the backend has something once its membership does, and gets its media once the backend has something
for that membership on its transport. LiveKit is the one backend today; a for that membership on its transport; `state$` is the membership state combined
with the local media connection. LiveKit is the one backend today; a
cascading SFU or full mesh is another folder under `sdk/src/media-backend/`, handed cascading SFU or full mesh is another folder under `sdk/src/media-backend/`, handed
in through `options.backend`, and nothing in the left box changes. in through `options.backend`, and nothing in the left box changes.
@@ -79,7 +80,7 @@ in through `options.backend`, and nothing in the left box changes.
│ CallViewModel │ │ CallViewModel │
│ layout, ringing and notifications, auto-leave, sounds, │ │ layout, ringing and notifications, auto-leave, sounds, │
│ reactions, hand raise, settings, audio routing, host bridge │ │ reactions, hand raise, settings, audio routing, host bridge │
│ ▲ status$, localMember$, remoteMembers$, tracks$ │ │ ▲ state$, localMember$, remoteMembers$, tracks$ │
│ MatrixRTCClient @element-hq/matrixrtc-sdk │ │ MatrixRTCClient @element-hq/matrixrtc-sdk │
└────────────────────────────────────────────────────────────────┘ └────────────────────────────────────────────────────────────────┘
│ MatrixClient + Room │ livekit-client │ MatrixClient + Room │ livekit-client
@@ -111,7 +112,7 @@ LiveKit participants.
│ │ create tracks, resume upstream ──────────────────────────────▶│ │ │ create tracks, resume upstream ──────────────────────────────▶│
│ │ media key to every other member (to-device) ──▶│ │ │ │ media key to every other member (to-device) ──▶│ │
│ │◀── other memberships (sync) ──────────│◀── participants ────────│ │ │◀── other memberships (sync) ──────────│◀── participants ────────│
│◀─ status$ "connected", localMember$, remoteMembers$ with tracks$ ─────────────│ │◀─ state$ connected, localMember$, remoteMembers$ with tracks$ ───────────────│
``` ```
The session discovers the transport from the homeserver and the backend prepares The session discovers the transport from the homeserver and the backend prepares
@@ -121,8 +122,8 @@ business: for LiveKit the token exchange, and the probe that decides whether the
SFU can take over the delayed leave, which picks the delayed leave timings the SFU can take over the delayed leave, which picks the delayed leave timings the
join uses. Until `join()` the tracks exist but their upstream is paused: a host join uses. Until `join()` the tracks exist but their upstream is paused: a host
can show a preview without anyone hearing it. The upstream flows exactly while the can show a preview without anyone hearing it. The upstream flows exactly while the
member is joined and the homeserver is reachable; `leave()` or a sync outage pauses membership is `joined`: a sync outage pauses it, and `leave()` pauses it for good
it, and `leave()` also sends the leave. and sends the leave.
## Vocabulary ## Vocabulary
@@ -134,7 +135,7 @@ the sections below still describe `MatrixRTCClient` until that lands.
| Word | Meaning here | Elsewhere | | Word | Meaning here | Elsewhere |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- | | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- |
| slot | an `RTCSlot`: the namespace in a room that memberships join into, `application` plus `id`. It exists while the room does, with nobody in it too. Holds `members$`, its open or closed `status$`, and `join()` | the js-sdk `MatrixRTCSession` is one per slot; the MSC4143 slot event | | slot | an `RTCSlot`: the namespace in a room that memberships join into, `application` plus `id`. It exists while the room does, with nobody in it too. Holds `members$`, its open or closed `status$`, and `join()` | the js-sdk `MatrixRTCSession` is one per slot; the MSC4143 slot event |
| participation | an `RTCParticipation`: the span between our `join()` of a slot and our `leave()`. Holds the connection and its `status$`, `localMember$`, `remoteMembers$`, the connected transports and the data channel. Made by `slot.join()`, never reused | "session" is not used here, because in the js-sdk it means the slot | | participation | an `RTCParticipation`: the span between our `join()` of a slot and our `leave()`. Holds the connection and its `state$`, `localMember$`, `remoteMembers$`, the connected transports and the data channel. Made by `slot.join()`, never reused | "session" is not used here, because in the js-sdk it means the slot |
| member | an `RTCMember`: one membership in a slot with what the SDK derives from it, `rtcBackendIdentity`, `userId`, `deviceId`, `memberId`, `displayName$`, `avatarUrl$`, `transport$` and `applicationData$`. No media. What the slot hands out in `members$` | the js-sdk `CallMembership` behind it, which never leaves the SDK; Element Call's old "matrixLivekitMember" | | member | an `RTCMember`: one membership in a slot with what the SDK derives from it, `rtcBackendIdentity`, `userId`, `deviceId`, `memberId`, `displayName$`, `avatarUrl$`, `transport$` and `applicationData$`. No media. What the slot hands out in `members$` | the js-sdk `CallMembership` behind it, which never leaves the SDK; Element Call's old "matrixLivekitMember" |
| local member, remote member | `LocalRTCMember` and `RemoteRTCMember`: an `RTCMember` plus its `MemberMedia`, which only a participation can carry. The local one's tracks carry the controls. What the participation hands out in `localMember$` and `remoteMembers$` | LiveKit `LocalParticipant` and `RemoteParticipant`, which never leave the SDK | | local member, remote member | `LocalRTCMember` and `RemoteRTCMember`: an `RTCMember` plus its `MemberMedia`, which only a participation can carry. The local one's tracks carry the controls. What the participation hands out in `localMember$` and `remoteMembers$` | LiveKit `LocalParticipant` and `RemoteParticipant`, which never leave the SDK |
| media | a `MemberMedia`: what a member sends once it has arrived on its transport, `tracks$`, null while waiting, and `encryptionError$` beside it. The media backend supplies exactly these two fields per member | Element Call's `MediaViewModel` is a view of it | | media | a `MemberMedia`: what a member sends once it has arrived on its transport, `tracks$`, null while waiting, and `encryptionError$` beside it. The media backend supplies exactly these two fields per member | Element Call's `MediaViewModel` is a view of it |
@@ -353,6 +354,21 @@ export interface DataMessage {
} }
``` ```
`state$` is two state machines, one per box in the diagram, combined by one pure
function. `LocalMembership` knows only the Matrix side: `waitingForTransport`,
`joining`, `joined`, `reconnecting` with a `MatrixDisconnectReason`, `failed`,
`left`. The client folds the local media connection in: `joined` with the media
up is `connected`, a media drop after that is `reconnecting: "media"`, a
connection error is `failed`. Each step is computed in priority order from the
previous state, so `left` and `failed` absorb everything after them, and
`reconnecting` is only ever reached from `connected`; a membership that drops
before the media ever came up stays `connecting`. One reason, not a set, because
the links are a chain: without sync the membership is unknowable, and the media
is worth little while the Matrix side is down, so the first broken link in that
order is what a host shows and what Element Call's analytics count. A failed
publication is not in here at all: the member can still receive, so it stays on
the local member's media.
### Members ### Members
```ts ```ts
@@ -532,9 +548,10 @@ export interface MediaBackend {
``` ```
What stays on the client's side: discovery of the local transport from the What stays on the client's side: discovery of the local transport from the
homeserver's list (a homeserver endpoint, so not a backend's), the join state homeserver's list (a homeserver endpoint, so not a backend's), the membership
machine in `LocalMember.ts`, the choice of delayed leave timings, mapping a data state machine in `LocalMembership.ts` and its combination with
packet's sender to a member, and the transport registry, which keys on the raw `local.connectionState$` into `state$`, the choice of delayed leave timings,
mapping a data packet's sender to a member, and the transport registry, which keys on the raw
transport serialised with sorted keys and derives `connectedTransports$` and each transport serialised with sorted keys and derives `connectedTransports$` and each
`TransportMetadata.resolved$` from `connections$`. `TransportMetadata.resolved$` from `connections$`.
@@ -700,9 +717,10 @@ sdk/
src/config.ts MatrixRTCMode, session timings, media quality src/config.ts MatrixRTCMode, session timings, media quality
src/encryption.ts E2eeType, EncryptionSystem src/encryption.ts E2eeType, EncryptionSystem
src/MatrixRTCClient.ts createMatrixRTCClient: wires the two folders below src/MatrixRTCClient.ts createMatrixRTCClient: wires the two folders below
src/participationState.ts the membership state combined with the media connection
src/reactive/ Behavior, ObservableScope, the observable operators src/reactive/ Behavior, ObservableScope, the observable operators
src/matrixrtc/ the MatrixRTC side: the local member, memberships, member src/matrixrtc/ the MatrixRTC side: the local membership, memberships, member
metadata, transports, discovery, status, the js-sdk join metadata, transports, discovery, the js-sdk join
src/media-backend/ the MediaBackend interface in api.ts; livekit/ is the one src/media-backend/ the MediaBackend interface in api.ts; livekit/ is the one
backend: connections, publisher, JWT, keys, the tracks backend: connections, publisher, JWT, keys, the tracks
src/utils/ LazyBehavior, mapScoped, network retry, display names, test helpers src/utils/ LazyBehavior, mapScoped, network retry, display names, test helpers