Compare commits

...
Author SHA1 Message Date
Johannes Marbach 8be229a438 Merge pull request #4354 from element-hq/toger5/johennes/rtc-remote-publish
Fall back to publishing on a remote SFU if possible
2026-10-15 16:27:36 +02:00
Timo K. ef7dce7b07 Update url-transport.test.ts 2026-10-15 16:10:02 +02:00
Timo K. 2de5ee2df5 Update test to consider EWs support for cs-api 2026-10-15 12:53:37 +02:00
Timo K. a427206792 Only return null if there is no endpoint available. 2026-10-15 12:42:36 +02:00
Timo K. 1913a099ab bump EW to inlcude the new widget capabilities 2026-10-15 11:28:45 +02:00
TimoandJohannes Marbach 93b893527e Update sdk/src/media-backend/livekit/RemotePublishTransport.ts
Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
2026-10-15 09:28:40 +02:00
Timo K. 2b26fd472f fmt 2026-10-14 19:03:22 +02:00
Timo K. 281fff0e9f array -> prop|null in discoverLocalTransport 2026-10-14 19:01:14 +02:00
Johannes Marbach bc557dfb5d Revert "Temporarily enable full output to debug the Claude action"
This reverts commit 4827ab5fdc.
2026-10-14 17:55:35 +02:00
Johannes Marbach 4827ab5fdc Temporarily enable full output to debug the Claude action 2026-10-14 17:50:20 +02:00
Timo K. 7af6525f01 fix lints 2026-10-14 16:44:31 +02:00
Timo K. 3b763d87f2 make the transport discovery lk edgecase a EC exclusive workaround
Dont add it to the sdk.
2026-10-14 16:40:27 +02:00
Timo K.andClaude Fable 5.1 8638d4f159 Fall back to publishing on a remote SFU if possible
Port of edf1feae from johannes/rtc-remote-publish onto the SDK split,
where
the MatrixRTC logic lives in sdk/ rather than src/state/CallViewModel/.

Where the original lands:

- getLocalTransport's remote fallback: the LiveKit backend's new
  findRemotePublishTransport (sdk/src/media-backend/livekit/
  RemotePublishTransport.ts), which tries the candidates from the
  js-sdk's
  getRemoteLivekitPublishCandidates oldest first and keeps the first
  whose
  homeserver issues a token with the publish grant.
  discoverLocalTransport
  asks the backend for it after the host's fallback, with a snapshot of
  the
  session's memberships from RTCParticipation, and throws
  MatrixRTCTransportMissingError where that finds nothing too. Transport
  discovery stays backend neutral: the MSC4195 token check is the
  backend's.
- LocalMember's "no delegation on a remote SFU" and the custom
  transport's
  server_name: LivekitBackend.prepareLocalTransport locates the
  transport on
  the homeserver it names (serverNameOfTransport) and skips the
  delegation
  probe where that is not our own.
- SessionBehaviors' membership.getTransportServerName: the same helper
  in
  LivekitBackend.mediaFor.
- decodeSFUJWTPayload, SFUJWTPayload.video.canPublish and the exported
  isCsApiUnavailable: sdk/src/media-backend/livekit/auth/.
- DeveloperSettingsTab, the widget capabilities, the locale string and
  the
  js-sdk pin (9405287c, 44.0.0): applied as is. The error boundary
  snapshots follow the string.

Tests: the LocalTransport cases go to RemotePublishTransport.test.ts and
LocalTransport.test.ts, the LocalMember remote case and the
SessionBehaviors
server-name case to LivekitBackend.test.ts, plus the memberships
snapshot in
RTCParticipation.test.ts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-14 16:09:11 +02:00
Timo 4b73ce68e3 Merge pull request #4319 from element-hq/toger5/use-sdk
[MatrixRTCSdk] use the matrixRtcSdk
2026-10-14 10:59:05 +02:00
Timo K. d37599a794 Tag the video element with its track source for e2e specs
The SDK MediaView renders a plain video element, so LiveKit's
data-lk-source attribute is gone. Expose data-source from the SDK track
and point the screen-share spec at it.
2026-10-14 10:23:29 +02:00
Timo K. 3f98e36c56 test MatrixRTCError conversion 2026-10-14 10:23:29 +02:00
Timo K. 042a8cfbee Note the SDK coverage added on the package branch and the reconcile fix 2026-10-14 10:23:29 +02:00
Timo K. e726c36f67 Drop the auth modules main's PR 4331 added under src
The SDK replaced `src/livekit` and the remote-member modules before that PR
landed on `main`, so its `src/livekit/auth/*` and `remoteMembers/ConnectionMap.ts`
have no consumer here: the same code lives under `sdk/src/media-backend/livekit`.
The imports of them that `main` added to the view model, the developer
settings tab and the test utils go with them.
2026-10-14 10:23:29 +02:00
Timo K. dd25238427 Document how PR 4331 lands on the SDK split
Per-commit mapping of element-hq/element-call#4331 onto sdk/ and src/, the changes to api.ts, where the PR's tests went and what was checked.
2026-10-14 10:23:29 +02:00
Timo K. 432ebadd44 Key media tiles by the membership id
The bumped js-sdk reports the legacy identity as rtcBackendIdentity for every membership, so a device that rejoins after an improper leave, which has two memberships until the stale one expires, produced two tiles with the same key and crashed the call view. The membership id tells them apart; the audio renderer keys by it too.
2026-10-14 10:23:29 +02:00
RobinandJohannes Marbach 289260531b Let developers override the whole transport
The developer setting that overrode the LiveKit service URL becomes a
JSON transport, so that a `url`-only or combined transport can be tested
against a homeserver that does not advertise one yet. The settings modal
shows each LiveKit room's role instead of a never-populated isLocal flag.

Folds "Avoid null in transport dev setting" from the same PR. The setting feeds the SDK's transport option; the config fallback becomes a transport the host builds.

Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
2026-10-14 10:23:29 +02:00
Robin f97dd2adab Show the whole transport in debug UI and logs
A transport may soon carry a LiveKit SFU `url` instead of, or as well as,
a `livekit_service_url`, so a service URL alone no longer identifies it.
Pass the whole transport to the tiles' debug stats and print it as JSON in
log lines instead of picking out the service URL.

Element Call's media view models carry the member's raw transport from the SDK's TransportMetadata to the tiles.
2026-10-14 10:23:29 +02:00
Robin 1700e9fe20 Match LiveKit participants against all backend identities
A member authenticated through the legacy JWT endpoint publishes under a
`user:device` identity, while one authenticated through the default
endpoint publishes under a hashed identity. Both are listed in the
membership's backend identities, so match against the whole list rather
than only the primary one.

Element Call's test membership mock, matching the SDK's RTCMember.
2026-10-14 10:23:29 +02:00
Timo K. f638604fe7 codecov as seperate coverage figures 2026-10-14 10:23:29 +02:00
Timo K. b0659279ee update to sdk api change: audioLevel stats$() 2026-10-14 10:23:29 +02:00
Timo K. 79c1f82448 use abstracted livekti processor 2026-10-14 10:23:29 +02:00
Timo K. eaf747c107 adjust to encrypted$ -> encrypted and VideoProcessor independent of LK 2026-10-14 10:23:29 +02:00
Timo K. aac60b74dc remove waitForKeyRotationMs as its not used anymore 2026-10-14 10:23:29 +02:00
Timo K. 598ec42390 use updated state logic in the sdk 2026-10-14 10:23:29 +02:00
Timo K. cec5031f62 sdk-target-based-on-call-view-model client -> slot/participation 2026-10-14 10:23:29 +02:00
Timo K. a5a0efe92b use sdk client -> participation/slot seperation 2026-10-14 10:23:29 +02:00
Timo K. 7c5d1324e7 Update to changes in the sdk.
- plain tracks instead of media$
2026-10-14 10:23:29 +02:00
Timo K. 52aaf84c3e update EC based on sdk media refactor
No more call specifics in the sdk layer
2026-10-14 10:23:29 +02:00
Timo K. 5f3db07b76 use the matrixRtcSdk 2026-10-14 10:23:29 +02:00
Timo 10b34b0218 Merge pull request #4316 from element-hq/toger5/matrixrtc-sdk-package
[MatrixRTCSdk] add the sdk
2026-10-14 10:23:24 +02:00
Timo K. d6b8207218 Merge branch 'main' into toger5/matrixrtc-sdk-package 2026-10-14 10:00:56 +02:00
renovate[bot] 956671eef5 Update ghcr.io/element-hq/element-web:develop Docker digest to e464b2a (#4351)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-14 02:33:22 +00:00
Timo K.andClaude Fable 5.1 28e1a9e22f Keep the CI sdk package name so main's preview deploy finds it
ploy previews run pr-deploy.yaml from main, which downloads
build-output-sdk,
so a renamed artifact can never be found from a pull request. Keep the
package
name and point build:sdk:production and build:sdk:development at the
CallViewModel bundle the preview is meant to carry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-13 19:04:02 +02:00
Timo K. 27dcfb8bb7 [SDK] Drop the landed plan documents and cut the migration document down
MediaBackendPlan.md and MemberMediaPlan.md described changes that have
landed. SdkMigration.md keeps only what moved from src/ into sdk/, with
the
paths as they are today. README.md gains the onboarding notes on the
three
API files, and its example uses createRTCSlot, which replaced
createMatrixRTCClient.
2026-10-13 17:10:04 +02:00
Timo K. 038991fa97 [SDK] Cover transport discovery, the join options and the room options
Unit tests for the pieces that had none: which transport the local
member advertises (override, homeserver, fallback, none), what the js-sdk
join is called with, how a publish request becomes LiveKit capture and
publish options, and the room options the connection factory builds from
the initial requests and the media quality settings, including what a
resolved connection reports while it is up. The backend's `mediaFor` gains
the cases from Element Call's member matching tests: a member on another
transport, a member changing transport, and a member without one.
2026-10-13 16:17:13 +02:00
Timo K. 2994bb4ed0 [SDK] Cover the publisher, the local media and the participation
The publisher is driven through a real LocalParticipant whose device and
signalling calls are faked, so that LiveKit's own pending-publication
bookkeeping is part of what is tested: the cases ported from Element
Call's Publisher tests (one permission prompt for both devices, nothing
created while muted, the upstream held back until the member publishes,
a mute that lands while the track is still being created) plus the
publish, unpublish and destroy paths.

The local media runs on a fake connection map: publishing before and
after the connection exists, republishing on a new connection, mutes
remembered across connections, the first publish error kept, and the
output device applied to the current room and the ones to come.

The participation runs on a fake media backend: the join options, the
state machine through connecting, connected, reconnecting, failed and
left, publishing only while joined, the delayed leave delegation, the
member ids per mode, the key replay and the data channel's sender check.
2026-10-13 16:13:47 +02:00
Timo K. d84dcc75de [SDK] Keep a value that reaches reconcile as its loop winds down
The loop that drives `scope.reconcile` was marked as running until the
promise of its last run had settled, which is a few microtasks after the
loop itself had checked for new values. A value arriving in that window
found the loop "running", was left to it, and was never processed: a
connection appearing at that moment would never get its publisher.

The flag is now cleared the moment the loop ends, so such a value starts
a new loop. The regression test drives the window microtask by microtask.
2026-10-13 16:13:47 +02:00
Timo K. 7c3f3d3ac7 Merge remote-tracking branch 'origin/main' into toger5/matrixrtc-sdk-package
# Conflicts:
#	sdk/README.md
2026-10-13 13:15:23 +02:00
Timo K. 7e4e260273 [SDK] Describe the transport auth flows in the architecture document
How a LiveKit transport is authenticated with by shape and role, the server name on connections and the options that now take a Transport.

(cherry picked from commit 26d9e1b0ae1e1f1ba8c3689bd50ba2e34aa0e820)
2026-10-13 13:00:58 +02:00
Robin 4aa249daf5 [SDK] Let developers override the whole transport
The developer setting that overrode the LiveKit service URL becomes a
JSON transport, so that a `url`-only or combined transport can be tested
against a homeserver that does not advertise one yet. The settings modal
shows each LiveKit room's role instead of a never-populated isLocal flag.

On the SDK the participation options take a whole Transport for the override and the fallback instead of two LiveKit service URLs, which also takes the last LiveKit knowledge out of transport discovery.

(cherry picked from commit d3eb40db0bcf7370418c4948049b3eac176bbdc1)
2026-10-13 13:00:58 +02:00
Robin b4bd1f81a0 [SDK] Move SFU authentication into livekit/auth and choose the endpoint by role
The OpenID flow moves to src/livekit/auth/openID.ts with the SFU config
types in a sibling file, takes an options object, and replaces the
MatrixRTC mode with the role we take on the transport plus the session's
slot ID instead of a hard-coded m.call#ROOM.

This changes behaviour: publishers on a livekit_service_url transport
always use the legacy /sfu/get endpoint, so that subscribers can predict
their user:device participant identity, and subscribers always use the
default /get_token endpoint. The MatrixRTC mode no longer influences which
endpoint is used.

On the SDK this lands as sdk/src/media-backend/livekit/auth/ with the connection map, the discriminated connection state and the client-server API flow of the following commits of element-hq/element-call#4331, which rewrite the same files:
- Model ConnectionState as a discriminated union
- Show the whole transport in debug UI and logs (SDK part)
- Let the publishing Connection authenticate itself
- Reduce ConnectionManager to a ConnectionMap
- Use the unstable LiveKit transport type
- Identify transports by server name as well as URLs
- Get SFU tokens and delegate delayed leaves through the MSC4195 CS API
- Fall back to get_token as subscriber on widget API errors
- Share a connection between homeservers using the same JWT service
- Make the auth choice mode independent again

Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
(cherry picked from commit adb380cd9ed108856e42d992fc03ced8c44f1e56)
2026-10-13 13:00:58 +02:00
Robin de8b5d98ec [SDK] Match LiveKit participants against all backend identities
A member authenticated through the legacy JWT endpoint publishes under a
`user:device` identity, while one authenticated through the default
endpoint publishes under a hashed identity. Both are listed in the
membership's backend identities, so match against the whole list rather
than only the primary one.

On the SDK this is RTCMember.backendIdentities, which the LiveKit backend matches media and data packets against; the harness keys its tiles by the membership id, which the backend identity no longer tells apart.

(cherry picked from commit c9aad2a2b3279782d101c9bc6c80af0417ccbb92)
2026-10-13 13:00:58 +02:00
Johannes Marbach 7685947a69 Switch JS SDK back to develop
The js-sdk at this commit also renamed MatrixRTCSession.leaveRoomSession to leave, which the SDK follows.

(cherry picked from commit 405348945febc1df10edf9c00748f1edd6b39e71)
2026-10-13 13:00:58 +02:00
Johannes Marbach b158c4fcd7 Merge pull request #4331 from element-hq/robin/livekit-updates
Support migration to new LiveKit transport types and endpoints
2026-10-13 07:38:02 +02:00
renovate[bot] dc8f8e585a Update ghcr.io/element-hq/element-web:develop Docker digest to 06ac258 (#4349)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-13 01:36:04 +00:00
Johannes Marbach ee7dcdee7e Cover the subscriber token path, widget fallback and server names 2026-10-12 21:28:04 +02:00
Johannes Marbach 8d1ed31b0b Cover identity matching and delegation across transport shapes 2026-10-12 21:09:15 +02:00
Johannes Marbach 54e948fdc0 Make the auth choice mode independent again 2026-10-12 20:37:14 +02:00
Timo 441fe5accd Merge pull request #4348 from element-hq/toger5/room-v-12-create-room-fix
Room v12 compatibility: Dont send invalid `users` power level section.
2026-10-12 17:39:52 +02:00
Johannes Marbach 37d25d4293 Merge branch 'main' into robin/livekit-updates 2026-10-12 16:29:13 +02:00
Timo 079185b232 Merge pull request #4347 from element-hq/toger5/CI-playwright-improvements
[CI Playwirght] Make reporter more verbose + fail faster
2026-10-12 16:18:13 +02:00
Timo K. 558fbe2188 Dont send invalid users power level section. 2026-10-12 16:13:48 +02:00
Johannes Marbach 022415e98e Avoid null in transport dev setting 2026-10-12 16:11:03 +02:00
Johannes Marbach a1575cb554 Share a connection between homeservers using the same JWT service 2026-10-12 16:08:56 +02:00
Johannes Marbach 97b5e61ad8 Fall back to get_token as subscriber on widget API errors 2026-10-12 16:03:00 +02:00
Johannes Marbach f8258f6a1d Switch JS SDK back to develop 2026-10-12 15:43:44 +02:00
Johannes Marbach 68107d119e Retain compatiblity in compatiblity mode 2026-10-12 15:40:27 +02:00
Timo e96f578b63 Merge pull request #4346 from element-hq/renovate/compound
Update dependency @vector-im/compound-design-tokens to v11.2.0
2026-10-12 14:12:06 +02:00
renovate[bot] cfaae8f7e2 Update dependency @vector-im/compound-design-tokens to v11.2.0 2026-10-12 11:01:00 +00:00
Timo 1a90e813e8 Merge pull request #4338 from element-hq/toger5/compound-portal-root
[ElementCallComponent] Use compound-web's PortalRoot in EC.
2026-10-12 12:59:28 +02:00
Timo K.andClaude Fable 5.1 9d0a12c38a Fix PortalRoot hiding the component's menus and tooltips
compound-web 10.4.0 treats a PortalRoot whose root is null as one whose root
is not available yet, so floating-ui never mounts the portal and every tooltip
label, and with it every icon button's accessible name, is missing. The
component defaulted portalRoot to null and wrapped the call unconditionally.

Wrap the call only when the host supplied a root, keeping null as the
not-yet-available case Compound defines, and say so on the prop. Cover it in
CallPortalRoot.test.tsx. Drop the git allowBuilds entry for compound-web now
that it is a release again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 12:43:29 +02:00
Timo K. a1c98ac4e6 make reporter more verbose + let CI playwright fail faster 2026-10-12 12:38:10 +02:00
Timo K. b72493f47f bump compound-web to release 10.4.0 2026-10-12 11:15:03 +02:00
Johannes Marbach 49223e720b Cover url transports end to end
The specs replace the transports the e2e homeserver advertises with each
shape a migrating deployment may produce and record which token endpoints
each side used: a url-only or combined transport on both sides goes
through the CS API alone, a legacy guest joining a url host uses the
legacy endpoint to publish and the CS API to subscribe, a subscriber falls
back to the JWT service when the homeserver lacks MSC4195, a publisher
gets an error instead, a delayed leave delegated through the CS API fires
on disconnect, and subscribers get tokens for a remote homeserver's SFU
over federation.

The JWT service's app service registration has to cover users of the e2e
Synapse's real server name for the get_token flow, and lk-jwt-service
0.8.0 implements the endpoints.
2026-10-12 10:33:58 +02:00
Robin c5044119ce Deprecate the livekit.livekit_service_url config option
Homeservers should advertise their transports over the MSC4143
rtc/transports endpoint instead of relying on a fallback in the app
config.
2026-10-12 10:33:38 +02:00
Robin f366e7cbe2 Let developers override the whole transport
The developer setting that overrode the LiveKit service URL becomes a
JSON transport, so that a `url`-only or combined transport can be tested
against a homeserver that does not advertise one yet. The settings modal
shows each LiveKit room's role instead of a never-populated isLocal flag.
2026-10-12 10:33:19 +02:00
Robin e336ee5cfa Get SFU tokens and delegate delayed leaves through the MSC4195 CS API
src/livekit/auth now fronts two flows. For a transport with a LiveKit SFU
`url`, tokens come from our own homeserver's MSC4195 get_token endpoint
and delayed leaves are delegated through delegate_delayed_leave, both via
the js-sdk's unstable client methods. Transports with only a
livekit_service_url keep using the OpenID flow against the JWT service.

A transport advertising both shapes lets a subscriber fall back to the
OpenID flow when the homeserver answers get_token with a 404 (Synapse
reports M_UNRECOGNIZED for unknown routes rather than M_NOT_FOUND), so a
client can still join calls hosted on servers that have not deployed
MSC4195 yet. Publishers never fall back: a homeserver that advertises a
`url` but does not implement the endpoints is misconfigured and the
admin should hear about it, so they get NoMatrix2AuthorizationService.
Delegation uses the same flow as the token so that the participant
identity the service computes matches.

The local member only probes a transport's delegate_delayed_leave
endpoint when it has a service URL. The developer settings tab validates
a custom service URL through the same facade.
2026-10-12 10:33:00 +02:00
Robin 17c3a7cb5d Pin matrix-js-sdk to the member_id branch
Temporary, until matrix-org/matrix-js-sdk#5588 merges and this can go
back to a develop bump. The branch only changes the get_token request
body from a member object to a member_id, so this is kept separate from
the code that uses it and can be dropped without touching that code.
2026-10-12 10:32:13 +02:00
Robin 53a0e273d6 Identify transports by server name as well as URLs
A transport with only a LiveKit SFU `url` has to be authenticated through
our own homeserver, which needs to know the name of the server owning the
SFU. Carry that name alongside the transport as a TransportLocator,
derived from the member's user ID for remote transports and from the
client's domain for our own, and key the connection map on the SFU URL,
the service URL and the server name together so that `url`, service URL
and combined transport shapes are deduplicated consistently.
2026-10-12 10:31:36 +02:00
Robin 83e1396bdd Use the unstable LiveKit transport type
UnstableLivekitTransport covers every shape a homeserver may advertise
while MSC4195 evolves: a LiveKit SFU `url`, a `livekit_service_url`, or
both. Switching to it means a service URL is no longer guaranteed, so the
remaining code that relies on one checks for it explicitly: delegation
probing skips transports without one, the connection map keys transports
by whichever URL they carry, and the OpenID flow rejects transports
without a service URL until the client-server API flow lands.
2026-10-12 10:30:52 +02:00
Robin d5bbb43e3a Reduce ConnectionManager to a ConnectionMap
createConnectionMap$ now returns the behavior of connections and their
participants directly and takes an already resolved local transport
instead of an observable of it. The CallViewModel wraps it with mapScoped
so that a new map is built, in a fresh scope, whenever the local transport
changes, and swallows local transport errors there since the local member
already surfaces them in the UI.
2026-10-12 10:26:54 +02:00
Robin 24b6a632dd Let the publishing Connection authenticate itself
getLocalTransport no longer authenticates with the SFU up front and hands
the resulting config to the connection. Instead the ConnectionManager tags
the connection whose transport equals the local one as the publisher, and
every Connection fetches its own SFU config for its role.

The local member now only joins the MatrixRTC session once its connection
is authenticated, so that the LiveKit room exists before remote members
try to get a subscriber token for it. Subscribers authenticating with the
OpenID flow lack permission to create the room and would otherwise fail
without retrying, causing a cascade of leaves and rejoins at the start of
a call.
2026-10-12 10:22:33 +02:00
Robin 17c3223883 Move SFU authentication into livekit/auth and choose the endpoint by role
The OpenID flow moves to src/livekit/auth/openID.ts with the SFU config
types in a sibling file, takes an options object, and replaces the
MatrixRTC mode with the role we take on the transport plus the session's
slot ID instead of a hard-coded m.call#ROOM.

This changes behaviour: publishers on a livekit_service_url transport
always use the legacy /sfu/get endpoint, so that subscribers can predict
their user:device participant identity, and subscribers always use the
default /get_token endpoint. The MatrixRTC mode no longer influences which
endpoint is used.
2026-10-12 10:17:39 +02:00
Timo K. c79675e6a9 bump to compound-web main branch 2026-10-12 10:14:49 +02:00
Robin f260755f46 Group livekitRoomItems$ by connection
The old reducer keyed items by service URL and mutated the accumulator's
entries in place, so a transport without a service URL could not be
represented and items were not safe to compare. Group members by their
Connection object instead and carry the whole transport on the item, which
the audio renderer now receives in place of a URL.
2026-10-12 10:11:54 +02:00
Robin 3927c2b7dc Show the whole transport in debug UI and logs
A transport may soon carry a LiveKit SFU `url` instead of, or as well as,
a `livekit_service_url`, so a service URL alone no longer identifies it.
Pass the whole transport to the tiles' debug stats and print it as JSON in
log lines instead of picking out the service URL.
2026-10-12 10:09:50 +02:00
Robin 4e4ab23cda Model ConnectionState as a discriminated union
The enum mixed our own lifecycle states with a copy of livekit-client's
connection states, and errors were smuggled through the same observable as
a separate union member. A discriminated union keeps the LiveKit state
nested under "authenticated", carries the error as data, and lets the
local member represent the time before a connection exists as an
"initializing" state instead of null.
2026-10-12 10:08:22 +02:00
Robin 3f030e64dd Match LiveKit participants against all backend identities
A member authenticated through the legacy JWT endpoint publishes under a
`user:device` identity, while one authenticated through the default
endpoint publishes under a hashed identity. Both are listed in the
membership's backend identities, so match against the whole list rather
than only the primary one.
2026-10-12 10:04:47 +02:00
Johannes Marbach d8becaf868 Advertise the loopback address from the dev LiveKit SFUs
The container IP is not routable from the host when Docker runs in a VM
(colima, Docker Desktop), so calls against the dev stack drop once ICE
fails. Advertising 127.0.0.1 instead lets the browser reach the SFU through
the published ports. Firefox ignores loopback candidates, so this only
works in Chromium-based browsers.
2026-10-12 10:03:03 +02:00
Robin 994cd36c44 Fix typo in the name of the createJWTToken Playwright fixture 2026-10-12 10:03:03 +02:00
Johannes Marbach 7ac5373af9 Remove stray pause and improve logging for hung e2e tests
The list reporter is the only progress a hung CI run leaves behind, and a
global timeout under the job's 60 minute limit means a hung run still ends
with a report instead of being cancelled by GitHub.
2026-10-12 10:03:03 +02:00
renovate[bot] 32fcede65e Update ghcr.io/element-hq/element-web:develop Docker digest to d43e8d2 (#4345)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-12 02:40:40 +00:00
renovate[bot] 6438c18b73 Update ghcr.io/element-hq/element-web:develop Docker digest to 88eeb34 (#4344)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-11 00:49:17 +00:00
renovate[bot] f07e9f5c70 Update ghcr.io/element-hq/element-web:develop Docker digest to 1488790 (#4343)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-10 02:09:04 +00:00
Johannes Marbach c97e3bf7a6 Merge pull request #4337 from element-hq/johannes/remove-per-participant-e2ee-param
Remove perParticipantE2EE
2026-10-09 12:01:45 +02:00
Timo K. 674692c926 Fix breaking Ci tests due to compound-web 10.1.0 -> 10.3.1 bump 2026-10-09 11:09:41 +02:00
Timo K. c3d07b89bd review: unnecassary matchMedia overwrite on test start. 2026-10-09 11:08:03 +02:00
renovate[bot] f89cbc140e Update ghcr.io/element-hq/element-web:develop Docker digest to 181a526 (#4339)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-09 01:18:17 +00:00
Timo K. 3bd892ebd3 bump compound-web 2026-10-08 21:26:34 +02:00
Timo K. f322af2742 [ElementCallComponent] Use compound-web's PortalRoot in EC. 2026-10-08 19:14:01 +02:00
Johannes Marbach a9fa48f68c Remove perParticipantE2EE 2026-10-08 17:15:19 +02:00
Robin 471c76c0e7 Merge pull request #4336 from element-hq/robin/vite-warning
Fix Vite warning
2026-10-08 16:40:45 +02:00
Timo K. 799619557b [SDK] make stats$ a polled method and add audioLevel$ to audio tracks 2026-10-08 16:36:25 +02:00
Timo K. 5f914b8a01 [SDK] refactor track processor api to be lk independent 2026-10-08 16:36:17 +02:00
Timo K. aa897ae9bc [SDK] simplify public api (no lk processor, no encryption$ as obs) 2026-10-08 16:36:11 +02:00
Johannes Marbach 4881fd5fc6 Merge pull request #4280 from element-hq/johannes/claude
Add workflow for automated Claude review of pull requests
2026-10-08 15:27:13 +02:00
Johannes Marbach 49c29f160a Add workflow for automated Claude review of pull requests 2026-10-08 14:57:19 +02:00
Robin aa8f73e632 Fix Vite warning 2026-10-08 14:23:59 +02:00
Timo K. 2bb870f6e7 [SDK] fix slots missing initial status value 2026-10-08 13:53:57 +02:00
Timo K. e44dfe1898 [SDK] add codecov 2026-10-08 13:53:57 +02:00
Timo K. b5fac4a2ff Adapt to matrix-js-sdk develop and cover RTCSlot's js-sdk imports 2026-10-08 13:53:57 +02:00
Timo K. 66f326fcf0 [SDK] Update SdkArchitecture.md 2026-10-08 13:53:57 +02:00
Timo K. 3cdb452d5d [SDK] update state structure (less duplication, one state enum)
- remove media from LocalMembership file
2026-10-08 13:53:57 +02:00
Timo K. f49c1cd3d4 [SDK] rename file LocalMember -> LocalMembership 2026-10-08 13:53:57 +02:00
Timo K. dc983e5493 [SDK] split the clint object into -> slot and participation 2026-10-08 13:53:57 +02:00
Timo K. 9cc9eb0889 Create Publisher.test.ts 2026-10-08 13:53:57 +02:00
Timo K. ca4f0cdeae fixup docs 2026-10-08 13:53:57 +02:00
Timo K. 4790af08a4 flatten media api (no MemberMedia wrapper object 2026-10-08 13:53:57 +02:00
Timo K. 6ffab92f0f split api.ts into api.ts and media-api.ts 2026-10-08 13:53:57 +02:00
Timo K. f1ea4cc5a2 Merge LocalMediaInputs into MatrixRTCClientOptions
The SDK now takes `publish` requests and audio output control directly
in the main options object, removing the separate `LocalMediaInputs`
type. Audio output device selection moves from a reactive behavior to an
imperative method on `MatrixRTCClient`, giving hosts direct control over
when devices switch.
2026-10-08 13:53:57 +02:00
Timo K. 5b13e1da81 renames and restructure
- session -> matrixrtc (folder)
 - move MatrixRtcClient to top level of sdk
 - RtcTransport -> Transport
 - backend -> media-backend
2026-10-08 13:53:57 +02:00
Timo K. d0a2ff12f1 Put the media behind a backend seam and move LiveKit into it
The session speaks MatrixRTC and sees media through MediaBackend in
sdk/src/backend/api.ts. Everything LiveKit lives in
sdk/src/backend/livekit/,
the only place in the SDK that imports livekit-client; the import
boundary rule
now enforces that outside sdk/src/backend/ the LiveKit packages appear
in type
imports only.

The seam sits above the connection manager: one connection per transport
url
is the LiveKit multi-SFU strategy, not a MatrixRTC concept, so a
cascading SFU
or full mesh replaces the folder, not the session. A backend prepares
the local
transport the session discovered (the token exchange and the delegation
probe,
reported as canDelegateDelayedLeave), hands the delayed leave over per
delay
id, gives the local member one publish switch plus publish/unpublish,
builds a
remote member's media from its membership and registers that
membership's
transport for as long as the member lives, and reports its connections
for the
transport registry, which keys on the raw transport serialised with
sorted
keys. The backend's context carries no Matrix client (a backend picks
what it
calls in its own options), no membership list, and no js-sdk session:
media
keys arrive as a stream replayed on subscribe, and MatrixKeyProvider
subscribes
to it. The MatrixRTC mode no longer reaches LiveKit; it becomes a
tokenEndpoint
choice made where the default backend is built.

Two behaviour changes ride along. Publishing is one boolean, joined and
homeserver reachable, where before a sync hiccup after leave() resumed
the
upstream of tracks that were meant to stay silent. And
ResolvedConnection keeps
one ResolvedTransport object per fetch, so a state change no longer
reads as a
change of what was fetched.

MatrixRTCClientOptions gains backend?: MediaBackendFactory, defaulting
to
createLivekitBackend built from the client options.
authenticateWithTransport
takes tokenEndpoint instead of matrixRTCMode.
2026-10-08 13:53:57 +02:00
Timo K.andClaude Fable 5.1 64d7eb99cc Add the media words to the architecture vocabulary
media, track, source, publish request, active and attach/detach, between
participant and transport, and the participant row names the adapter as
the only reader.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 13:53:57 +02:00
Timo K.andClaude Fable 5.1 cad00aef08 Mark the speaker detection check as fixme
Chromium's fake microphone never registers as a speaker at the SFU, and Firefox's steady tone does so in about two runs out of three, with or without the browser's audio processing. The test stays, out of the gate, until the dev SFU gets a lower speaker threshold or the harness a louder fake device.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 13:53:57 +02:00
Timo K.andClaude Fable 5.1 6bd6e3f953 Take the word call out of the member API
membership$ is typed as RTCMembership, the subset of the js-sdk CallMembership that hosts read. UnknownCallError is UnknownRTCError throughout, not only in the export. applicationData reaches the membership whole, since the js-sdk takes it next to the intent. The plan records the three slices as implemented.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 13:53:57 +02:00
Timo K.andClaude Fable 5.1 b9f79be106 Publish and unpublish on the local member
One way to put media on the session: publish(request) resolves with the local track once LiveKit has it, and unpublish(id) removes it. A request names a source with its device, processor and capture settings, so CaptureSettings on the client options goes, as do setMicrophoneEnabled and setCameraEnabled on the client and the four screen share fields on the local member; sharing is tracks$ having a screenShare track, and a capture failure rejects the promise. LocalMediaInputs becomes the list of initial requests, which also become the room defaults so that one permission prompt covers microphone and camera. The publisher keeps the requests across a reconnection and republishes what is not muted. The harness gains a Share screen button.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 13:53:57 +02:00
Timo K.andClaude Fable 5.1 5a43eaf5a6 Give the local tracks their controls
Our own tracks are LocalAudioMediaTrack and LocalVideoMediaTrack, with setEnabled and setDevice, and on video facingMode$, switchFacingMode and setProcessor. switchCamera on the local media goes, as do the device and processor behaviors in LocalMediaInputs: a host calls the track when its setting changes instead of the SDK watching behaviors it does not own. The input device ids and the processor stay as initial values for the tracks created at the join. setMicrophoneEnabled and setCameraEnabled stay for the case of no track yet.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 13:53:57 +02:00
Timo K.andClaude Fable 5.1 63ccfa50be Test and document the MemberMedia track list
Unit tests for tracks$ and trackBySource$, an end-to-end check of isActive$ through a mute cycle (skipped on Chromium, whose fake microphone the SFU never hears), the architecture document brought in line with the track list, and the changed behaviour recorded in the plan.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 13:53:57 +02:00
Timo K. 18e0065086 specific call tracks -> generic tracks 2026-10-08 13:53:57 +02:00
Timo K. d7ab2b27b6 Create MemberMediaPlan.md 2026-10-08 13:53:57 +02:00
Timo K. 3afe52bd4a Create MediaBackendPlan.md 2026-10-08 13:53:57 +02:00
Timo K. d89c058275 Update SdkMigration.md 2026-10-08 13:53:57 +02:00
Timo K. 2af87a760d add support for sending data to the rtcSdk 2026-10-08 13:53:56 +02:00
Timo K. 5c6d2f0891 Update SDK to be consumed by Element Call's CallViewModel
Update the public API to export the primitives and operators the call
model needs, and change `LocalMediaInputs` from observable flags to
immediate booleans with imperative setters. Add session timings, media
quality and capture settings as configuration options, and allow the
host to override the transport discovery with explicit URLs.
2026-10-08 13:53:56 +02:00
Timo K. 31eafb394b 7000 loc COPIED from element call
They are copied for now but will be removed from element call once it
adopts the sdk. Those are no real line additions.
2026-10-08 13:53:56 +02:00
Timo K. 61e353d919 renames + rewriting docs 2026-10-08 13:53:56 +02:00
Timo K. 1a999eaa54 sdk implementation 2026-10-08 13:53:56 +02:00
Timo K.andClaude Fable 5.1 9910496b90 Add the MatrixRTC SDK package, its harness and a failing smoke test
Set up `sdk/` as a package shaped exactly like `component/`: its own
package.json with peer dependencies, the pnpm workspace marker so a host can
install it as a git dependency, a build tsconfig for type emission, and a vite
lib config with an externals list that `pnpm lint:externals` now checks for
both library builds.

The package holds only the interface for now. `createRtcSession` returns an
empty object; `RtcSession`, `RtcMember`, `MemberMedia` and `TransportMetadata`
are the contract Element Call's `CallViewModel` will be rebuilt on (see
sdk-plan.md). The shared primitives are re-exported from `src` until the first
implementation slice moves them.

`sdk/dev` is the smallest consumer of the SDK: a plain page that logs in, joins
a room and renders one tile per member, served by `pnpm dev:sdk` on port 3002.
Playwright starts it as a third web server, and `playwright/sdk/smoke.spec.ts`
has two browsers join one session. It fails on the status line with the SDK's
own error until there is an implementation behind the interface.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 13:53:56 +02:00
Timo K.andClaude Fable 5.1 2b9fdfe4d5 Move the SDK demo to sdk-target-based-on-call-view-model
The `sdk/` directory is about to become the MatrixRTC SDK package, so the
demo that lives there — one bundle of Element Call's CallViewModel for a plain
page or widget — moves aside, unchanged. It keeps building under a script and
vite config of its own, `pnpm build:sdk-target-based-on-call-view-model`, into
its own `dist` rather than the repository root's, and no longer copies the
developer's config.json out of `public`. Its README says which script to run
and where the output lands.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 13:53:56 +02:00
Johannes Marbach 14c561af8d Merge pull request #4300 from element-hq/johannes/pure-layout-functions
Make the layout decisions pure / testable functions
2026-10-08 13:20:44 +02:00
Johannes Marbach 4fd2f66949 Explain why we use merge 2026-10-08 11:14:26 +02:00
Johannes Marbach 53e038b09b layoutShallowEquals -> shallowEquals 2026-10-08 10:57:03 +02:00
renovate[bot] 3b7ccb63e2 Update ghcr.io/element-hq/element-web:develop Docker digest to 53a4fe6 (#4334)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-08 02:17:28 +00:00
Johannes Marbach 1f07f949b5 Merge pull request #4330 from element-hq/renovate/npm-postcss-selector-parser-vulnerability
Update dependency postcss-selector-parser to v7.1.6 [SECURITY]
2026-10-07 14:59:28 +02:00
renovate[bot] a60036b3d7 Update dependency postcss-selector-parser to v7.1.6 [SECURITY] 2026-10-07 12:37:14 +00:00
Timo 2eb3a2fd2d Merge pull request #4329 from element-hq/toger5/remove-lk-client-from-external-deps
Remove livekit-client from the list of external dependencies
2026-10-07 14:34:47 +02:00
Timo K. 8b6adf067e remove unnecassary (confused) comment 2026-10-07 14:20:42 +02:00
Timo K. b408610277 Remove livekit-client from the list of external dependencies 2026-10-07 12:01:21 +02:00
renovate[bot] 2faa6c3dac Update ghcr.io/element-hq/element-web:develop Docker digest to 589c490 (#4328)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-07 01:42:16 +00:00
renovate[bot] 9b3c3569ec Update ghcr.io/element-hq/element-web:develop Docker digest to 39c94f0 (#4327)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-06 02:57:18 +00:00
Johannes Marbach a4654d9d00 Merge pull request #4315 from element-hq/johannes/workflow-call
Allow workflow_call on lint and test workflows
2026-10-05 14:07:07 +02:00
Johannes Marbach 6e5f71c796 Revert back to negative flags 2026-10-05 13:53:52 +02:00
Johannes Marbach 067f3d3038 Fix repo selection and flags 2026-10-05 13:44:48 +02:00
Johannes Marbach 6d70221fe6 Flip boolean flags 2026-10-05 13:04:34 +02:00
renovate[bot] 427376f78b Update ghcr.io/element-hq/element-web:develop Docker digest to 573daa2 (#4323)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-05 09:47:57 +00:00
Johannes Marbach ccfd36923d Merge pull request #4324 from element-hq/renovate/compound
Update dependency @vector-im/compound-design-tokens to v11.1.0
2026-10-05 08:41:57 +02:00
Johannes Marbach 89567726f8 Fix tests 2026-10-05 08:13:58 +02:00
Robin 9cd111b5f2 Merge pull request #4318 from element-hq/redact-posthog-user-id
Redact analytics ID from rageshakes
2026-10-05 07:44:16 +02:00
renovate[bot] 096aaac9f3 Update dependency @vector-im/compound-design-tokens to v11.1.0 2026-10-05 01:27:09 +00:00
renovate[bot] 8835a6d75d Update ghcr.io/element-hq/element-web:develop Docker digest to 3cf23ff (#4322)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-04 09:34:33 +00:00
renovate[bot] bfb868aec3 Update ghcr.io/element-hq/element-web:develop Docker digest to 5371503 (#4321)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-03 01:48:09 +00:00
Robin 4e8fb6e4cb Redact analytics ID from rageshakes
To improve our minimization of external data we collect in rageshakes.
2026-10-02 16:51:00 +02:00
Robin 7c53a6be5a Merge pull request #4317 from element-hq/remove-wait-for-rotation
Remove wait_for_key_rotation_ms config option
2026-10-02 15:14:14 +02:00
Robin 64b932f128 Remove unused vite-plugin-generate-file dependency 2026-10-02 14:57:34 +02:00
Robin bfad85a3af Remove wait_for_key_rotation_ms config option
I learned that makeKeyDelay has been deprecated in matrix-js-sdk ever since the switch to the new encryption manager, so this config option has no effect.
2026-10-02 14:49:44 +02:00
Johannes Marbach be822adec8 Allow workflow_call on lint and test workflows 2026-10-02 13:33:29 +02:00
Johannes Marbach 4c0f02ef47 Merge pull request #4309 from element-hq/johannes/layered
Set up a layered build for CI similar to what EW does
2026-10-02 13:22:19 +02:00
Johannes Marbach ca6319f06d Remove layered boolean 2026-10-02 13:07:09 +02:00
Timo 1d57a8eee1 Merge pull request #4308 from element-hq/toger5/fix-component-sounds
[ElementCall Component] Decode inlined sounds directly instead of fetching them
2026-10-02 12:45:31 +02:00
Timo K. 0bb9632b60 review play sounds does not throw 2026-10-02 11:46:54 +02:00
Timo K. cbe195d449 review audio data loading 2026-10-02 11:46:40 +02:00
Robin e800fec74d Merge pull request #4314 from element-hq/renovate/element-web
Update ghcr.io/element-hq/element-web:develop Docker digest to 588f0b4
2026-10-02 11:29:05 +02:00
Robin 9d0725ad31 Fix Element Web integration tests 2026-10-02 10:51:31 +02:00
renovate[bot] 241412f29b Update ghcr.io/element-hq/element-web:develop Docker digest to 588f0b4 2026-10-02 02:54:17 +00:00
Timo K. d96d00b64a Merge branch 'main' into toger5/fix-component-sounds 2026-10-01 19:07:39 +02:00
Robin cde88a7c93 Merge pull request #4312 from element-hq/js-sdk-develop
Reset matrix-js-sdk to develop branch, adopt new join/leave signatures
2026-10-01 18:55:16 +02:00
Robin d5733e8274 Reset matrix-js-sdk to develop branch, adopt new join/leave signatures 2026-10-01 17:11:24 +02:00
Timo K. 3befe3d34b Merge branch 'main' into toger5/fix-component-sounds 2026-10-01 13:58:23 +02:00
Timo 3ad778e7e8 Merge pull request #4306 from element-hq/toger5/fix-component-tooltips
[ElementCall Component] Make compound a peer dependency with EW. (fix tooltips in embedded mode)
2026-10-01 13:57:58 +02:00
Timo K. 9a286032f2 shorten readme 2026-10-01 13:41:07 +02:00
Timo K. 363a8feab2 Stop bundling Compound, let the host supply it 2026-09-30 17:10:17 +02:00
Johannes Marbach bd8c3142a6 Set up a layered build for CI similar to what EW does 2026-09-30 11:35:06 +02:00
Timo K. 16c60c8b6b Warn when the audio context fails to set up. 2026-09-30 11:22:19 +02:00
Timo K. 63ac1ba03e Decode inlined sounds directly instead of fetching them, with the test. 2026-09-30 11:22:15 +02:00
Robin 9e190e9e57 Merge pull request #4307 from element-hq/update-js-sdk
Adapt to matrix-js-sdk removal of livekit_alias and oldest_membership
2026-09-30 10:41:37 +02:00
Robin f287fb3800 Adapt to matrix-js-sdk removal of oldest_membership support 2026-09-30 00:45:19 +02:00
Robin f9b601504b Stop sending livekit_alias in memberships
It's been many versions since anything used the value.
2026-09-30 00:45:19 +02:00
Timo K. 9140365251 Merge branch 'main' into toger5/fix-component-tooltips 2026-09-29 16:25:15 +02:00
Timo 81233e5cfb Merge pull request #4269 from element-hq/toger5/component-api-entry
Export EC component related types
2026-09-29 16:24:26 +02:00
Timo K.andClaude Fable 5.1 e296d88969 Keep the component's tooltips inside its root
Compound's Tooltip floats its bubble through floating-ui's FloatingPortal,
into the body: outside the element the component build confines its
stylesheet to, so a label tooltip shows as bare text at all times. An
enclosing FloatingPortal rooted at the container makes the nested portals
land inside it instead.

The call goes in an isolating wrapper there, standing in for the standalone
page's #root, so that its footer layer no longer paints above the modals.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 12:08:44 +02:00
Robin fbce73ae55 Merge pull request #4302 from element-hq/always-multi-sfu
Remove multi-SFU check which is always true
2026-09-29 09:14:42 +02:00
renovate[bot] ce79274ccc Update ghcr.io/element-hq/element-web:develop Docker digest to cbb7be9 (#4303)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-29 01:07:59 +00:00
Robin 8db2e8ac41 Remove multi-SFU check which is always true
Compatibility and Matrix 2.0 are the only MatrixRTC modes now.
2026-09-28 22:03:16 +02:00
Johannes Marbach 12352b94f5 Merge pull request #4299 from element-hq/johannes/drop-observable-hooks
Drop observable-hooks in favor of useBehavior
2026-09-28 18:12:43 +02:00
Johannes MarbachandRobin e393b6a984 Swap conditional ordering
Co-authored-by: Robin <robin@robin.town>
2026-09-28 17:56:08 +02:00
Johannes Marbach 3f63a6fc3c Drop observable-hooks in favor of useBehavior
With this change, views now read every reactive value through `useBehavior`, as
`architecture.md` already prescribes. Where a view reads a plain stream, the view
model now exposes a `Behavior`.
2026-09-28 17:54:50 +02:00
Johannes Marbach 6600c891f3 Extract the layout decision into a pure function 2026-09-28 17:36:24 +02:00
Johannes Marbach 33cfd2d8ad Build the layout media from one snapshot of its inputs 2026-09-28 17:34:22 +02:00
Johannes Marbach 5a8714805c Extract the spotlight and PiP choice into a pure function 2026-09-28 17:29:41 +02:00
Johannes Marbach 232e3f7ad4 Extract the spotlight speaker choice into a pure function 2026-09-28 17:28:14 +02:00
Johannes Marbach 10adf6658b Let layoutShallowEquals compare any layout-shaped object 2026-09-28 17:27:04 +02:00
Johannes Marbach 29742006df Move WindowMode next to the layout types to prevent cyclic import 2026-09-28 17:26:11 +02:00
Johannes Marbach addc1b10a0 Merge pull request #4297 from element-hq/renovate/compound
Update dependency @vector-im/compound-web to v10.1.0
2026-09-28 08:23:32 +02:00
renovate[bot] ee0d7dc0cb Update ghcr.io/element-hq/element-web:develop Docker digest to cfbe7ce (#4296)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-28 01:14:27 +00:00
renovate[bot] 7b1f3f28ed Update dependency @vector-im/compound-web to v10.1.0 2026-09-28 01:03:27 +00:00
renovate[bot] 8c44ecd61b Update ghcr.io/element-hq/element-web:develop Docker digest to 8301993 (#4295)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-27 01:25:15 +00:00
renovate[bot] 7d92ddf0c9 Update ghcr.io/element-hq/element-web:develop Docker digest to d97061c (#4294)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-26 02:21:38 +00:00
Johannes Marbach 5c497cc6a8 Fix build on Windows 2026-09-25 16:15:39 +02:00
fkwpandClaude Opus 5 b7bd7d18b2 [Feature] Quick audio menu (#4275)
* Give the microphone level its own observable
* Add a microphone level meter
* Show speakers and microphones in the quick audio menu
* Wire speaker selection through the call footer
* Add stories for the meter and the device menu
* Add end-to-end specs for the quick audio menu
* Keep the footer while a menu opened from it is open
* Use the shared audio capture stub in the lobby test
---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 09:52:02 +00:00
fkwpandClaude Opus 5.5 b4b72432b9 Tighten the comment guidance in AGENTS.md (#4279)
* Say what earns a comment its place

- Add a comments section to the code style: no comment by default, the reason
  only, and the kinds that go stale.
- Point to it from the PR checklist in AGENTS.md.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 08:44:04 +00:00
Robin ac1f5e2c8c Merge pull request #4277 from element-hq/vitest-warning
Fix Vitest import warning
2026-09-23 16:04:47 +02:00
renovate[bot] 98d77cc816 Update ghcr.io/element-hq/element-web:develop Docker digest to 0636bbe (#4276)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-23 01:55:54 +00:00
Robin d5812ddd34 Fix Vitest import warning
Fixes the following warning:

[warn] (!) Your Vite config uses features that are unsupported by `configLoader: 'native'`, which is planned to become the default in a future major version of Vite:
  - import "./vite.config" without a file extension (vitest.config.ts:3:35). Add the file extension
Set `VITE_CONFIG_NATIVE_IGNORE_WARNING=true` to suppress this warning.
2026-09-22 23:34:19 +02:00
Johannes Marbach cf7c275fb7 Merge pull request #4274 from element-hq/hs/bell
Bell reaction sound.
2026-09-22 12:32:38 +02:00
Half-Shot 93b2661513 Switch to 🛎️ 2026-09-22 11:21:11 +01:00
Robin 1f84cb9c5c Merge pull request #4273 from element-hq/simplify-local-transport
Simplify local transport code
2026-09-22 12:15:34 +02:00
Half-Shot 4269aaa163 update snap 2026-09-22 10:04:12 +01:00
Half-Shot bc62bb9565 lint 2026-09-22 10:01:22 +01:00
Half-Shot be56bbd05b lint 2026-09-22 10:01:10 +01:00
Half-Shot eb44fd545d Bell reaction sound. 2026-09-22 09:57:43 +01:00
Robin c9f911b7c9 Swallow local transport errors in connection manager 2026-09-22 10:01:14 +02:00
Robin 3a189d155b Remove references to outdated variables or types 2026-09-22 09:56:02 +02:00
Robin 0211d5c367 Remove unused shareReplay operator 2026-09-22 09:45:48 +02:00
Robin c88f884fc9 Simplify local transport code
As of the removal of 'Legacy' mode, there can no longer be a difference between the transport you advertise in your membership and the transport you publish media on, so the local transport code can be simplified considerably. For instance, the function which gets the local transport can simply return a promise rather than being reactive.

I kept the local transport as an Observable in other modules so that they could easily be tested with existing marble tests.
2026-09-21 12:51:28 +02:00
Johannes Marbach 9f7c35cd8f Merge pull request #4270 from element-hq/renovate/major-compound
Update dependency @vector-im/compound-design-tokens to v11
2026-09-21 07:51:35 +02:00
renovate[bot] ac81a7c0ab Update dependency @vector-im/compound-design-tokens to v11 2026-09-21 02:56:00 +00:00
Timo K. 8ebb0b5a90 Export component related types. 2026-09-19 08:46:30 +02:00
Timo 03f81bcd64 Merge pull request #4266 from toger5/toger5/memberId-based-on-matrixRTCMode
Make the memberId dependent on the matrixRTC mode.
2026-09-17 14:02:32 +02:00
Timo K. 9f6c8bb434 Make the memberId dependent on the matrixRTC mode.
This outherwise creates to-device messages with the uuid for members
that use the userId:deviceId memberId in the state event.
2026-09-17 13:44:25 +02:00
renovate[bot] f56c74adf9 Update ghcr.io/element-hq/element-web:develop Docker digest to 3993d53 (#4265)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-17 00:51:42 +00:00
Johannes Marbach 3c88d217f2 Merge pull request #4244 from element-hq/matthew/default-output-first
Prefer the browser default audio output when no output has been chosen
2026-09-16 15:39:12 +02:00
renovate[bot] 967a06ba39 Update ghcr.io/element-hq/element-web:develop Docker digest to 506caaa (#4262)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-16 00:29:13 +00:00
renovate[bot] 71593f1a24 Update ghcr.io/element-hq/element-web:develop Docker digest to c76d299 (#4260)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-15 02:40:31 +00:00
renovate[bot]andJohannes Marbach 5d38a13c46 Update ghcr.io/element-hq/element-web:develop Docker digest to 9d8d8fc (#4243)
* Update ghcr.io/element-hq/element-web:develop Docker digest to 1850bf6

* Adapt to latest EW

* Reformat

* Next try

* Fix regexp

* Fix DM path

* Next try

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
2026-09-14 08:51:33 +00:00
Johannes Marbach fe91162869 Merge pull request #4257 from element-hq/matthew/log-livekit-ice
Always log LiveKit ICE candidates and states to the rageshake
2026-09-14 08:31:42 +02:00
Johannes Marbach 74328d7e42 Merge pull request #4258 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.22.3
2026-09-14 08:03:12 +02:00
renovate[bot] 6046337877 Update dependency livekit-client to v2.22.3 2026-09-14 02:37:28 +00:00
Matthew Hodgson 16b2c31ebb Always log LiveKit ICE candidates and states to the rageshake
LiveKit's livekit-ice logger records each local ICE candidate, ICE
candidate errors and ICE connection state changes, but only at debug,
which we cap to info unless extended LiveKit logs are switched on. That
means a 'could not establish pc connection' rageshake carries no way to
tell 'no relay candidates at all' from 'relay candidates that never
paired'. Keep that one logger at debug regardless of the toggle.
2026-09-11 18:30:33 +01:00
fkwpandClaude Opus 5 a58a5f615f Add AGENTS.md documenting the repo's conventions (#4256)
* Add AGENTS.md documenting the repo's conventions

Codifies what is currently tribal knowledge or only discoverable by reading CI
config: the view model / view contract, the rule that nothing reads the page,
Clean Code ordering, the three test layers, and reuse-before-you-build.

The root AGENTS.md is the always-loaded minimum and routes to docs/agents/, so
an agent loads only the detail its task needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Allow either hand at the commit, and measure continuous redraws

Committing from the agent rules out running it where there is no git
identity, a sandbox included. Either is now fine: commit on the user's
word where the setup allows it, or write the message and hand it over.

Redrawing continuously is the other addition. A level meter reported a
fresh value every animation frame, so its whole menu reconciled sixty
times a second, silence included, and nobody noticed until review. A PR
that adds anything of the kind now says what it costs, or better, counts
the redraws — commits rather than render calls, since React runs a
component it then discards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 15:12:15 +02:00
Robin 2cac2d7e8b Merge pull request #4231 from element-hq/matthew/blur-processor-ended-track
Don't attach the background blur processor to an ended camera track
2026-09-10 21:27:11 +02:00
Robin 1695d3f518 Use flushPromises for readability 2026-09-10 21:15:48 +02:00
Robin 96a2e9b481 Merge pull request #4230 from element-hq/matthew/connection-stop-during-connect
Don't report an aborted connect as a connection error
2026-09-10 20:25:12 +02:00
Robin 23ba8f3204 Merge pull request #4249 from element-hq/matthew/audio-renderer-warn-spam
Stop MatrixAudioRenderer warning on every render
2026-09-10 19:56:56 +02:00
Matthew HodgsonandClaude Fable 5.1 b284bb00c0 Cover the stopProcessor failure path and trackProcessorSync
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GSB8Qb89KDoTNRV83dWJHa
2026-09-10 18:28:57 +01:00
Johannes Marbach cb0ae89143 Merge pull request #4253 from element-hq/johannes/jwt-0.7.0
Bump lk-jwt-service to 0.7.0 in the docker setup and integrate it as an app service
2026-09-10 17:46:02 +02:00
Johannes Marbach 344aac21f9 Force undelegated mode in tests 2026-09-10 14:15:38 +02:00
Johannes Marbach 21d7f24f61 Bump lk-jwt-service to 0.7.0 in the docker setup and integrate it as an app service 2026-09-10 12:26:49 +02:00
Johannes Marbach 029622f07b Merge pull request #4251 from element-hq/renovate/npm-postcss-selector-parser-vulnerability
Update dependency postcss-selector-parser to v7.1.3 [SECURITY]
2026-09-10 07:42:07 +02:00
renovate[bot] 251535d371 Update dependency postcss-selector-parser to v7.1.3 [SECURITY] 2026-09-09 17:42:33 +00:00
Timo 5c6fa7c864 Merge pull request #4233 from element-hq/valere/component_ec_M1
ElementCall as a react component: refactor + a local dev harness
2026-09-09 19:39:30 +02:00
Timo K. be4023dd66 fix layout in spotlight (top inset) 2026-09-09 19:20:31 +02:00
Timo K. be5f3390f7 more tests 2026-09-09 17:01:16 +02:00
Timo K. 59ef5f5400 ci 2026-09-09 16:05:18 +02:00
Matthew Hodgson dbd2423e7b Stop MatrixAudioRenderer warning on every render
The "has no matching matrix call member" warning fired on every re-render
(i.e. every active speaker update) and also for the local participant's own
track, which is never rendered here anyway. Skip local tracks and warn once
per unexpected identity.
2026-09-09 11:20:49 +01:00
Timo K. 2ac95df75f update readmes 2026-09-09 12:15:19 +02:00
Timo K. 7821b465a5 Merge branch 'main' into valere/component_ec_M1 2026-09-09 10:23:47 +02:00
Robin b90ed5bdf7 Merge pull request #4247 from element-hq/fix-cd
Fix call.element.dev not updating after branch rename
2026-09-09 09:07:51 +02:00
Robin 1d7e2288f7 Fix call.element.dev not updating after branch rename 2026-09-09 08:49:08 +02:00
Robin 61eeb8171a Merge pull request #4242 from element-hq/delayed-leave-timings
Adapt delayed leave timings when delegation is available
2026-09-09 08:19:58 +02:00
Matthew Hodgson 426acd222e Prefer the browser default audio output when no output has been chosen
On browsers without a "default" pseudo-device (Firefox, Safari), the
virtual default output entry was appended after the physical devices, so
with no saved preference EC selected the first physical device and pinned
every remote audio element to it with setSinkId. Pinned sinks are not
re-routed by the browser: on Firefox/Linux a Bluetooth headset switching
from A2DP to HFP when its microphone is opened (i.e. on unmute) destroys
the pinned sink and all remote audio goes silent, with no error and no
fallback (rageshake 17320).

List the virtual default first so it is the fallback both when nothing was
chosen and when the chosen output disappears, and stop labelling it with
the first device's name since the browser default is not necessarily that
device.
2026-09-08 23:22:44 +01:00
Robin cff8766aa6 Delegate delayed leaves in LocalMember rather than LocalTransport
IMO this is where the delegation calls should have lived all along, since the leave event is part of the membership lifecycle, and we otherwise end up with an awkward hack to ignore transport updates.

Doing this now ensures that the client won't send any delegation requests if delegation is unsupported, and prepares the code for a future change in which we use the dedicated delegation endpoint from the CS API.
2026-09-08 21:22:52 +02:00
Robin 5cfe7ea4df Base JWT service endpoint choice directly on the MatrixRTC mode
Since we removed the extra 'legacy' mode, meaning there's nothing simplified anymore by translating things over to a JWT service version.

(Also, simplify some things by making the mode static for the duration of a call.)
2026-09-08 21:22:52 +02:00
Robin d38daef567 Reflect session config always being defined 2026-09-08 21:22:52 +02:00
Robin 09782a4d84 Give JWT token fetch procedures more appropriate names 2026-09-08 21:22:52 +02:00
Robin 839e36abd4 Make delegation support check fail immediately in case of CORS errors 2026-09-08 21:22:52 +02:00
Timo K. 3485ce47bc Merge branch 'main' into valere/component_ec_M1 2026-09-08 20:05:27 +02:00
Timo K. 273ee3b632 add allowJoinUnmutedViaIntent to the bridge
This allows us to control to never start unmuted in spa but be able to
start unmuted in widget and component mode.
2026-09-08 18:12:40 +02:00
Timo K.andClaude Fable 5.1 486ecbfda0 Stop offering media downloads on the component's host bridge
`downloadMedia` exists because a widget's client cannot fetch media: it
has no access token, and its only route to a file is MSC4039 over the
widget API. A component host hands over a client that does hold
credentials, so Element Call fetches media with it and the callback was
never needed there. It comes off `ElementCallHostBridge`, leaving a host
with only what it must understand, and stays on the internal HostBridge
for the widget.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 17:14:55 +02:00
Timo K.andClaude Fable 5.1 b722cc277e Make the theme a prop, next to the language
The theme is state — what Element Call should look like right now — and
so belongs beside `language` as a prop, not on the imperative handle
(where it was a request, `setTheme`, because the internal host bridge
speaks the widget API and a widget's host sends theme changes as
requests) and not in the configuration (where `config.theme` only ever
set the starting theme).

The `theme` prop feeds the same channel the rest of Element Call listens
to for a host's theme, replayed so that whatever subscribes after the
host has set it still hears the current one. Changing it re-themes the
container and nothing else; unlike the language, it is per component.
`setTheme` and `config.theme` are gone, and the harness gets a theme
picker in place of its per-pane buttons.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 17:02:24 +02:00
Timo K.andClaude Fable 5.1 f994586eeb Speak every language in the component, not just English
The component bundled English alone: the standalone app fetches its
locale files at runtime from URLs its own build emits, which a host
serving the library from elsewhere could not resolve, so bundling one
language was the self-contained option. Now every locale is a chunk of
its own that the host's bundler loads the first time it is needed, with
English still bundled in so that the fallback never waits.

Element Call starts in the browser's language and follows the host's own
setting through a `language` prop; `supportedLanguages` says what it
accepts. Translations are shared by every Element Call on the page, so
the most recently set language wins for all of them. The harness gets a
language picker, and the app and the component share the parsing of
locale paths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 16:03:48 +02:00
Timo K.andClaude Fable 5.1 b7285064fd Let the shell say where home is, and drop the router from the call
The call reached for react-router in five places to send the user
"home": on leaving without a lobby, from the lobby's recents link, from
the post-call screen, from the error page's return button and from the
header logo. Home is the standalone app's home page; the call has no
idea where that is, and a component has no such place at all — its host
decides what follows a call. Yet the component had to mount a
MemoryRouter just so those hooks would not throw.

`useLeaveToHome` is the way home as the shell supplies it: the app
provides `navigate("/")` from inside its router, the component provides
nothing, and everything that used to link to "/" now either calls it or,
when there is none, offers no way out. The logo becomes a plain logo,
the recents and "not now" links disappear, the error page's button does
too. `ClientProvider`'s logout goes the same way. The component no
longer renders a router.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:39:00 +02:00
Timo K.andClaude Fable 5.1 f5a94ce702 Offer a component host only the configuration it can use
`ElementCallConfiguration` was every URL parameter, which put the widget
plumbing (widgetId, parentUrl, isWidget), the user's identity, the
homeserver, the analytics and Sentry settings, and the standalone app's
shared room secret in front of a host that has no business setting any
of them — the component reads none of those. It is now the behavioural
configuration a widget's URL can carry plus the two properties a host
has a say in: the theme and the background. The background is now
actually applied to the component's root, which it was not before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:39:00 +02:00
Timo K.andClaude Fable 5.1 3dc7cfe10b Say "component" for the component
"Embedded" already means something here: the embedded package is the
widget build. Where a comment meant Element Call running as a React
component inside a host application, it now says so, and the params it
starts from are `componentProperties`. "Host" stays the word for the
application on the other side, whether that is a widget container or an
application rendering the component.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:39:00 +02:00
Timo K.andClaude Fable 5.1 f70fba1dd6 Rename GroupCallView to CallView, and fold ElementCallView into it
Three layers stood between a shell and the call: `ElementCall` (the
component), `ElementCallView` and `GroupCallView`. The middle one only
held the `joined` flag and the mute state, both of which nothing outside
the call reads any more, so it is folded into the view it wrapped. That
view is now `CallView`, since "group call" is a name from before
Element Call handled anything else.

Its docstring says what it is: the whole lifecycle of a call — lobby,
the call itself, and what comes after — with the lobby and the post-call
screen each present or skipped depending on the parameters and the host.
Both the standalone RoomPage and the component render it directly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:38:59 +02:00
Timo K.andClaude Fable 5.1 8e8bc5ddde Talk to a component host in callbacks, not observables
The component exposed the internal HostBridge to hosts as-is, which
carried the host's requests as rxjs observables. That made rxjs part of
the public API of a package that bundles its own copy of it, so a host
would build bridges with a different rxjs than the one Element Call
consumed them with — and asked every host to learn rxjs to change the
theme.

A component host now implements plain async callbacks for what Element
Call tells it (`ElementCallHostBridge`, all optional), and makes its own
requests through an imperative handle on the component's `ref`
(`ElementCallHandle`: setTheme, join, hangUp, setDeviceMute), each
resolving once Element Call has acted and rejecting when nothing in
Element Call can. `component/host.ts` adapts that to the HostBridge the
rest of Element Call still speaks, with a bridge whose identity never
changes, so a host re-creating its callbacks on render restarts nothing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:38:59 +02:00
Timo K.andClaude Fable 5.1 d18e82c546 Catch the component's errors inside the component
Only the call body sat inside an error boundary; anything thrown in the
lobby, the mute state or the providers propagated into the host's React
tree, where React unmounts everything up to the nearest boundary — the
host's, or the host's whole application. The standalone app has a
boundary at its top; the component now has the same, showing the error
page inside its container and asking the host to stop keeping it on
screen.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:38:59 +02:00
Timo K.andClaude Fable 5.1 779ee7a008 Finish measuring the root rather than the viewport
The move from media queries to `@container element-call` queries left a
few places still asking the viewport: the room header's compact size
and the lobby's footer placement went through `useMediaQuery`, the
lobby's video preview was `50vh` tall, the reaction picker was capped at
`100vw`, and the content insets and the gradient background were sized
from `100vw`/`100vh`. Embedded in a corner of a host's page, each of
those answered for the page rather than the corner.

`useRootSizeMatches` is `useMediaQuery` for the root element, built on
the same `observeElementSize$` the layout uses; the lengths become
container units. Container units resolve against the nearest query
container, and there are others in the tree (the spotlight layouts, the
media tiles), so base.css says when they may be used.

jsdom gives the body no size at all, which would have every such query
read as a tiny window; the test setup now gives it a desktop's, matching
what the media query mock already answered.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:38:59 +02:00
Timo K.andClaude Fable 5.1 7cd21476f7 Say outright whether the profile is ours to change
Whether to offer the profile settings was inferred from whether the host
could close Element Call. For a component with no host bridge — the
default — nothing could, so an embedded Element Call let the user edit
the profile of an account that belongs to the host application.

`HostBridge.supportsProfileChanges` states it directly: true standalone,
where Element Call signed the user in itself; false for a widget's host
and for anything embedding the component (which sets it itself, since
the client it hands over is its own). The profile tab and the profile
shortcut follow that. What a host's ability to close us still decides —
what to show after the call ends — is a question about who owns our
lifetime, and stays keyed on `close`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:38:59 +02:00
Timo K.andClaude Fable 5.1 f0359259f9 Format base.css
The formatter wants the layered import of the Compound design tokens on
one line; "de-globalise styles" left it wrapped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 15:38:59 +02:00
Timo K.andClaude Fable 5.1 7137e6ec14 Update the developer settings snapshot for the scoped pre style
"de-globalise styles" gave the tab's <pre> elements a class instead of
styling the bare element, but did not update the snapshot.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 13:56:10 +02:00
Timo K.andClaude Fable 5.1 6ae4f0539a Leave the page title to the page
The call and the lobby each set `document.title`, so a component
embedded in a host renamed the host's tab to "Element Call | <room>".
The title belongs to whoever owns the page: the standalone app's
RoomPage now sets it, for whichever room it has got as far as knowing
about, and the call itself no longer touches it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 13:53:50 +02:00
Timo K.andClaude Fable 5.1 bc58aed0d7 Keep keyboard shortcuts within Element Call's root
The call's shortcuts were listened for on the window and allowed
whenever focus was inside the standalone app's `#root` — which, for a
component embedded in a host, is the host's own root, or nothing. So m,
v and space fired while the user typed in the host's composer, and two
Element Calls on one page both answered every key.

Listen on the element Element Call treats as its root instead (the body
standalone, so nothing changes there), and judge whether a key press is
spoken for by what has focus — a dialog or a text field — rather than by
where it sits in the DOM, since the modals are now portalled to that
same root.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 13:52:34 +02:00
Timo K.andClaude Fable 5.1 608f107f8d End the component's media device scope on unmount
The component built its `MediaDevices` once, in a scope nothing ever
ended, so every mount left device observers running for the rest of the
page's life. Building it in an effect ties the scope to the component's
lifetime — and to the options it was built with, which were previously
frozen at first render.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 13:50:17 +02:00
Timo K.andClaude Fable 5.1 a010cc983f Compare the component's config by value
Everything downstream of the component's params — the mute state, the
call view model and with it the media connection — is keyed on the
identity of the params object, which was memoised on the identity of
the `config` prop. A host writing `config={{ ... }}` inline, which is
the natural way to write it, therefore tore the whole call down on
every render. The harness happened to pass a constant, so nothing
noticed.

`useStableValue` hands out the same object for as long as a deep
comparison says nothing changed, so an inline config costs nothing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 13:49:19 +02:00
Johannes Marbach 00d0adc3c4 Merge pull request #4222 from element-hq/renovate/all-minor-patch
Update all non-major dependencies
2026-09-08 11:40:20 +02:00
Johannes Marbach ddc5a50529 Disable new react compiler rules 2026-09-08 11:29:36 +02:00
renovate[bot] 27734a1a02 Update all non-major dependencies 2026-09-08 06:13:17 +00:00
Timo K. 489c1af426 test resizing via @container 2026-09-07 17:35:25 +02:00
Timo K. 365204d04c fix media queries in component based element call. 2026-09-07 14:50:36 +02:00
Johannes Marbach 14c61db43d Merge pull request #4241 from element-hq/matthew/encryption-mismatch-logging
Log track encryption flag and cryptor events for remote tracks
2026-09-07 13:30:50 +02:00
Robin 6a9583a2ce Adapt delayed leave timings when delegation is available
Splits the config options for the timings of a delayed leave event into two sets: one for when delegation is available (as you can relax the timings and get more stable calls this way), and another for when it's unavailable (as we must continue to gracefully downgrade even after Matrix 2.0 is fully rolled out).

This works by bluntly hitting the delegation endpoints without auth before joining to check for a 404.
2026-09-07 13:10:54 +02:00
Timo K.andClaude Fable 5.1 218ee46b7d Make the component installable as a git dependency
`component/package.json` describes the built component as a package
(`@element-hq/element-call-component`: entry, stylesheet subpath, types,
peer dependencies) so that a host can depend on
`github:element-hq/element-call#<ref>&path:/component`. Its `prepare`
script builds on install, since nothing is published yet.

For that the component build now lands in `component/dist` instead of the
repository's `dist`, and `pnpm build:component` also emits the type
declarations (`component/tsconfig.build.json`, `build:component:types`),
which previously had to be produced by hand.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 15:29:55 +02:00
Timo K. 38c28a1c8f scaling based on component size instead webview size 2026-09-04 14:16:40 +02:00
Timo K. f5458bb03e de-globalise styles 2026-09-04 13:51:47 +02:00
Valere ace78de749 Cover Element Call as a component with end-to-end tests
The widget tests cannot reach what makes a component different, because the
iframe used to guarantee it: that Element Call stays inside the space it was
given, and that two of them can exist in one page. So the development harness
gets driven by Playwright.

Three tests. Two components in one page holding a real call between two devices
of one account. The settings dialog and the reaction picker staying inside the
container the host gave, asserted by bounding box. And the host bridge
reporting in both directions, including a host-initiated mute coming back as a
report of the new state.

The containment test is the one worth having: both of the escapes found by
running the harness by hand — the settings dialog centred on the window, and
the reaction picker at 82vh landing below the container — would have failed it,
and neither was visible to typechecking, linting or the unit tests.

Users and rooms are created through the Synapse admin and client-server APIs
rather than by driving an interface, and the harness now takes its credentials
from its own query string so that a test can say which account and room to use.
A host reading its own URL is proper; it was Element Call doing so that was the
mistake.

Playwright gains a second web server for the harness on port 3001, a Vite dev
server whether or not the app itself is served from Docker, since the harness is
a development page with nothing to build.
2026-09-03 18:09:12 +02:00
Valere 3361ce2b60 Keep what the component draws inside the container it was given
Opening settings in an embedded call put the dialog in the middle of the host's
window, spilling outside the container, and with two calls on a page the second
drew over the first's dialog.

The container established a stacking context but not a containing block, which
are two different things and only the first had been done. `position: fixed`
resolves against the viewport unless an ancestor makes itself the containing
block, so the modal scrim and dialog in Overlay.module.css — `fixed`, `inset:
0`, centred, because in the standalone app they are meant to cover the page —
were positioned and sized against the window. Layout and paint containment
makes the container the containing block for those descendants and clips what
we paint to our own box. The second-call-on-top symptom goes with it, since the
dialog now stays inside the first call's box and there is nothing to overlap.
Two sibling components still cannot draw over one another by construction,
neither being able to leave its own stacking context, but that only shows if a
host overlaps them.

Containment clips a box measured in viewport units but cannot resize it, so
anything positioned that way is simply put somewhere outside the container and
disappears. That applied to the reactions overlay, a `100vw` by `100vh` box, and
to the reaction picker, which sits at `82vh` so as to appear near the footer it
belongs to. Both are positioned against whatever Element Call treats as its
root — `[data-overlay-container]` in the app, which is the size of the page, and
the host's container when embedded — so percentages mean the same thing there
and the right thing here. The earpiece overlay is `inset: 0` with no viewport
units, so containment is enough for it.

Three viewport-relative sizes remain, all of which need more than a change of
unit: the lobby's video preview is `50vh` on a flex item with an aspect ratio,
`--content-inset-*` ramps up to a desktop inset from the window width, and the
picker's `max-width` cap is the window's.

The clipping cuts both ways: a menu near the edge of a small container is
trimmed rather than overflowing into the host. That is the trade an embedded
component makes.
2026-09-03 17:30:16 +02:00
Valere c0a54dcaf9 Derive the component's defaults from an intent, not the host's URL
The component seeded its parameters with `computeUrlParams()`, which reads
`window.location`. Element Call is not the page any more, so what it found
there was the host's URL: no `widgetId`, therefore not a widget, therefore no
intent, therefore the standalone app's preset.

The visible symptom was the Element Call logo in the footer of a call embedded
in someone else's application, since `showLogo` follows `header ===
HeaderStyle.Standard`. The rest of that preset mattered more. A hosted call
defaulted to `perParticipantE2EE: false` and `confineToRoom: false` — so
unencrypted, and willing to take the user out of the room a host had put them
in — and a host could not correct either without naming every parameter
itself, nor even name `header`, since the enums were not exported.

So the intent presets move out of `computeUrlParams` into
`configurationForIntent`, and the component builds its parameters from an
intent plus the properties a host has no URL to supply. `intent` becomes a
prop, defaulting to joining an existing group call: the lobby first, confined
to the room, encrypted per participant, and no Element Call branding in
someone else's interface. A host that knows which button the user pressed
should say which.

One deliberate difference from the widget presets: the background defaults to
solid rather than the gradient, which is drawn by a `position: fixed`
pseudo-element and would escape the container to cover the host.

`UserIntent.Unknown` still means the standalone preset, so the app and widget
are unchanged — including a widget that names no intent, which has always been
given those defaults.
2026-09-03 16:39:14 +02:00
Valere 9f5fa6049f Add a harness for Element Call embedded as a component
`pnpm dev:component` serves a page that stands in for a host application: it
signs in twice against the development backend and shows two calls side by
side, in resizable boxes, with furniture of its own around them. Two devices of
one account, so a real call happens between the two components and anything
Element Call keeps once per process rather than once per call shows itself.

The host bridge is driven by hand and reports both directions in a log along
the bottom, which is the first exercise the theme, hang-up and device-mute
requests have had outside widget mode. Each pane can be unmounted and remounted
to see what Element Call leaves behind, and there is a `position: fixed` dialog
belonging to the host to see whether it covers the calls. The page uses none of
Element Call's design tokens, so anything that looks styled outside a pane came
from Element Call reaching out of its container. It reaches Element Call only
through the component's public interface, which is how the exports missing from
that interface came to light.

Three things about the component build the harness turned up on the way, all
too small to be worth their own commits:

- It copied `public/` into `dist/`, including the developer's own gitignored
  config.json, into output we would publish. `publicDir: false`, as the
  embedded build already does. The sdk build has the same leak; untouched.
- `pnpm lint:externals` now exists, which the build config already claimed it
  did. It reads the external list out of that config and fails if the source
  imports React, the Matrix SDK or LiveKit by a path the list does not name.
  Since the bundler silently ignores the pattern form of that option, an
  unnamed subpath is bundled with no warning at all — which is how a host would
  end up with a second React.
- `lint:oxlint` ran over `src playwright`, so nothing in `component/` had ever
  been linted.

Serving a page also meant the shared plugin list could no longer inject the
app's HTML entry point unconditionally, so that is now optional — and off for
the library build too, which never had an HTML page to inject it into.
2026-09-03 16:11:39 +02:00
Valere db0f6837ce Give the component what the app shell was providing
The component built and typechecked in the previous commit, but only because
nothing had rendered it. Everything Element Call needs that `src/main.tsx`
side-loads was missing from it.

Its stylesheet: only main.tsx imported index.css, so the library build emitted
CSS-module styles with every `--cpd-*` and `--font-size-*` unresolved. Split
into base.css, which both the app and the component import, and the rules that
are about owning a page, which only the app does. The split is a straight move
— comment-stripped and sorted, the old file and the two new ones differ by
exactly one line — and that line is the deliberate part: `.no-scroll-body`
becomes `body.no-scroll-body`. Element Call adds that class to whatever it
treats as its root, and since the root can now be a container, `position:
fixed` would have taken that container out of the host's layout. Pinning the
page is what it always meant.

Its translations: `initializeElementCall` called `i18n.init` with neither
resources nor a backend, so every key would have rendered as itself. English is
bundled in. The app fetches locale files from URLs its own build emits, which a
host serving the library from somewhere else could not resolve, so how a host
picks a language is left open.

And the types a host needs: `HostBridge` alone is not enough to implement
`HostBridge` — `HostRequest`, `DeviceMuteState`, `DeviceMuteRequest` and
`JoinCallData` all appear in its signatures, and `ConfigOptions` in
`initializeElementCall`'s.
2026-09-03 16:10:26 +02:00
Matthew Hodgson 2e5b5cb186 Don't attach the background blur processor to an ended camera track
If the camera track has already ended by the time the blur processor is
applied to it, MediaStreamTrackProcessor cannot be constructed and
setProcessor rejects. Both call sites used `void`, so this surfaced as

    Unhandled promise rejection: TypeError: Failed to construct
    'MediaStreamTrackProcessor': Input track cannot be ended

(element-call-rageshakes#17225). Skip ended tracks, and catch and log
processor attach/detach failures instead of leaking them.
2026-09-03 14:50:28 +01:00
Matthew Hodgson 8060d10fcc Don't report an aborted connect as a connection error
When the ConnectionManager stops a Connection while livekitRoom.connect()
is still pending (which happens on every join, because the local
membership emits twice in quick succession and the connection set is
recomputed), livekit-client rejects the pending connect with "Client
initiated disconnect". Connection.start() then treated that as a failure:
it pushed an UnknownCallError into the state of a connection that was
already stopped and rethrew, and since start() is fire-and-forget the
throw surfaced as

    Unhandled promise rejection: Error: Failed to connect to Livekit server

in every rageshake, right after a "livekitRoom.connect FAILED ... Client
initiated disconnect" line. Nothing was actually wrong: the replacement
connection connects fine a moment later.

Mark the connection as stopped before disconnecting, and have start()
return quietly when the rejection is the abort we asked for.
2026-09-03 14:48:23 +01:00
Valere 979b521563 Build Element Call as a component a host can import
Adds component/index.tsx as a fourth build target: <ElementCall client
roomId /> and an initializeElementCall to await once beforehand. It gives
Element Call everything it would otherwise take from the page it is on —
the parameters, the host bridge, media devices, translations, a container
to confine itself to — and hands it the host's client rather than finding
one of its own.

React, the Matrix SDK and LiveKit stay external, since the host has them
and a second copy of any would not merely be wasteful: React would hold
two sets of hooks and the client would run two sync loops. Every subpath
has to be listed by name, because the pattern and callback forms of
rollupOptions.external are silently ignored here — a lesson worth the
comment that records it.

Element Call's own navigation runs in a MemoryRouter, so being embedded
cannot disturb the host's URL. ClientContext and GroupCallView both
navigate, so some router has to be present.

The bundle is not yet a reasonable size: library mode base64-inlines
assets referenced through import.meta.url, so MediaPipe's vision runtime
lands in it whole. Left for its own change, since the fix — loading the
background blur transformer lazily — is worth doing for the standalone app
too.
2026-09-03 15:18:03 +02:00
Valere a166fbbd08 Let the component own its mute state
ElementCallView took muteStates as a prop, which would have meant an
embedder building one before it could show a call. It could not simply make
its own: RoomPage created one on mount whichever branch it went on to
render, and two would both report the user's mute state to the host,
talking over each other.

Move the construction into a useMuteStates hook, and give the lobby shown
while waiting to be let into a room its own component. Each lobby now holds
mute state only while it is on screen, so there is never a second one, and
the component can own the call's.

KnockLobbyView also takes the room summary and label handling that RoomPage
was assembling on its behalf, leaving the page with arriving at a call
rather than being in one.
2026-09-03 15:15:51 +02:00
Valere 856e055d30 Find Element Call's root by attribute, not by being the body
Six selectors named `body` directly — the gradient backdrop and the
platform font overrides in index.css, and the iOS adjustments in
AppBar.module.css and Modal.module.css — so they only applied when Element
Call owned the page. A host mounting it into a container would have got an
interface decorated correctly and styled incorrectly, with nothing to show
that anything was wrong.

Mark the root element with data-element-call-root and match on that
instead. Scoping this way keeps the selectors more specific than they were,
rather than less: widening them to a bare [data-platform=…] would have
dropped specificity from (0,1,1) to (0,1,0) and changed which rules win.

The platform attribute moves with them, from the initializer's write onto
document.body to a layout effect on the root, alongside the theme — so it
still lands before anything is painted.

No visual change while Element Call owns the page: the root is the body,
which now carries the attribute, so every rewritten selector matches the
element it always did.
2026-09-03 12:55:52 +02:00
Valere 1bfb018d90 Introduce ElementCallView
Being in a call and deciding to show one are different jobs, but RoomPage
did both: routing, authentication, resolving room aliases and knocking, and
then the call itself. Only the first set belongs to whatever is hosting
Element Call.

Add ElementCallView as the seam between them. It takes a client and a call
to join, and owns whether the user has joined — which is the call's own
business rather than its host's. RoomPage keeps everything about arriving
at a call and renders this for the call itself.

Nothing else moves yet. muteStates is still passed in, because the
standalone shell shares one with the lobby it shows while waiting to be let
into a room, and two instances would both report the user's mute state to
the host.
2026-09-03 12:10:04 +02:00
Valere ebb7fe07b7 add more test 2026-09-03 11:47:35 +02:00
Valere 873dfdbfbd Drop the unused root element provider
RootElementProvider was exported but never used: nothing supplies a root
element, so every consumer falls back to the document body. Knip reports
it, which is what is failing CI.

M1 adds a provider back along with the component that mounts Element Call
into a container. Until then there is nothing to provide.
2026-09-02 19:46:44 +02:00
Valere e68d1d1b5f Say what the root element seam cannot do yet
RootElementContext's documentation promised that Element Call confines its
decoration to the given element, but several selectors still name body
directly and the initializer writes data-platform onto it. A non-body root
is decorated correctly and styled incorrectly, with no error to show for
it, so the documentation should say so until the stylesheets are scoped.
2026-09-02 19:28:14 +02:00
Valere b7efa02adc review: Follow a swapped client 2026-09-02 19:27:44 +02:00
Valere c807b8ef7d Always stop the widget transport when closing
close() sent io.element.close and then stopped the transport, so a
rejected send skipped the stop. Both call sites this replaced stopped it
unconditionally — ErrorView in a finally, GroupCallView outside its
try/catch — because a host that never acknowledges the request would
otherwise leave the messaging live and the close button doing nothing.

Restore that with a finally.
2026-09-02 19:26:59 +02:00
Valere 6b5f396e1e Share the mapping from parameters to call view model options
The SDK stopped sending join notifications: it threaded callIntent into
createCallViewModel$ but not its pair sendNotificationType, which
enterRTCSession used to read for itself, so an explicit
?sendNotificationType=ring — or an intent that implies one — no longer
reached joinRTCSession.

The mechanism is worth fixing rather than the instance. The defaults on
CallViewModelOptions describe a standalone Element Call, so a widget
caller that misses a field gets standalone behaviour rather than an error,
and the SDK is only ever a widget. Give both callers one shared mapping so
they cannot drift, and cover the whole chain from URL to options in tests.

autoLeaveWhenOthersLeft and waitForCallPickup stay out of it: the view
model never read those from the parameters, so enabling them for the SDK
would be a change in its behaviour rather than a fix.
2026-09-02 19:26:33 +02:00
Valere 182be8676d Return the widget from initializeWidget instead of assigning a global
Element Call reached the widget API through a mutable module-level
binding, which every consumer imported directly. Nothing outside the app
shell needs it any more, so hand it back from initializeWidget and thread
it through: the initializer returns it, main passes it to App, and App
uses it to build the host bridge and to await the client the host is
lending us.

ClientContext's loadClient is now only about restoring or creating a
session of Element Call's own, since a widget's client arrives as a prop
like any other host's would.

Also fixes an early return added in the previous commit, which skipped
starting the analytics settings listener when a client was supplied. That
was harmless until now, but would have stopped analytics following the
user's choices in widget mode.

sdk/main.ts asked the host to close by hand; it now uses the bridge, which
also stops the transport as the app does.
2026-09-02 16:56:09 +02:00
Valere f6f47ede62 Let a host supply Element Call's client
ClientProvider found its own client: from the widget API, or by restoring
or creating a session. A host that embeds Element Call already has one,
and owns the user's session, so accept it as a prop and skip all of that.

A supplied client seeds the state synchronously, since there is no session
of ours to restore and so nothing to wait for.

Also guard the broadcast that shuts down other instances of the app. It
protects Element Call's own session and crypto stores, which is why it was
already skipped in widget mode — a host's client has the same property, so
without this an embedded Element Call would close down the user's other
tabs on mount.

Adds the first tests for ClientContext, covering both
2026-09-02 15:14:10 +02:00
Valere eb117249d1 Configure analytics explicitly instead of discovering it
PosthogAnalytics read its own configuration out of the environment on
first use: the URL parameters, config.json, and the widget global. An
embedded Element Call has none of those to offer, and analytics that
configure themselves cannot be switched off by a host that does its own
reporting.

Take an AnalyticsConfig through PosthogAnalytics.configure() instead,
called from the initializer once the config has loaded. Unconfigured
analytics stay off.

Note the two halves of that config are decided differently, and have to
be: where the credentials come from depends on the package, but who owns
the user's analytics identity depends on how Element Call is running,
since the full package can be used as a widget too.

Drop the widget check around cryptoVersion, which never did anything —
widget mode never initialises crypto, so getCrypto() is already undefined
there.

Move the tests covering which package reads which credential source onto
analyticsConfigFromEnvironment, where that decision now lives.
2026-09-02 15:05:12 +02:00
Valere f28d9e9b06 Stop asking whether Element Call is a widget
The remaining reads of the widget global were all asking one of two
different questions, so they get two different answers.

The app shell — the auth hooks, automatic guest registration, the group
call loader's diagnostic and the initial mute state — wants to know
whether Element Call was launched as a widget. That is a property of the
URL it was launched with, so expose the isWidget that computeUrlParams
already computed internally, documented as being for shell use only.

The call interface — whether to offer the profile settings tab — wants to
know something about its host, so it asks the bridge. A host that can
dismiss Element Call owns the user's account, so their profile is not ours
to edit; this reuses the close capability as a proxy, with a TODO
alongside the others.

ClientContext also takes supportsReactions from the bridge rather than
checking four widget capabilities itself, which removes widgetApi from
InitResult — a field that was always null outside widget mode.

Note this changes behaviour for a malformed widget URL: one carrying a
widget ID and parent URL but missing the room, user, device or base URL
would previously have fallen back to registering a guest user, and will
now not.
2026-09-02 15:04:24 +02:00
Valere bc6c232ef0 Move the state layer onto the host bridge
MuteStates, CallViewModel and LocalMember reached the host through the
widget global. None of them are React components, so they take the bridge
as an explicit parameter: a constructor argument for MuteStates, a field
on CallViewModelOptions, and one on createLocalMembership$'s props.

src/state no longer refers to the widget API.

The conditionals around it mostly disappear: nullHostBridge's observables
are NEVER, so there is nothing to guard, and a request carries its own
reply rather than needing the transport and the original event.

CallViewModelWidget.test.ts drove hangup by emitting on the mocked
widget's action emitter, so it now injects a bridge instead, and checks
that the request is acknowledged.
2026-09-02 13:23:01 +02:00
Valere 8e78ac76bd Drop ErrorView's widget prop in favour of the host bridge
ErrorView needed a widget only to decide whether to offer a close button
or a link home. That prop was threaded down through ErrorPage, RichError
and GroupCallErrorBoundary from seven call sites, to answer the single
question of whether the host can dismiss Element Call — which the host
bridge now answers directly through the presence of close().

Read the bridge from context in ErrorView and remove the prop, along with
the plumbing that carried it.

No functional change: the rendered output is unchanged, as the existing
snapshot confirms.

Move GroupCallView onto the host bridge

GroupCallView asked the widget API to keep it on screen, to close it, and
to tell it when a preloaded call should join. Route all three through the
host bridge and drop the widget prop.
2026-09-02 13:03:23 +02:00
Valere d8d70812b8 Read the shared room key from the parameters context
useRoomEncryptionSystem is used from GroupCallView, inside the part of
Element Call that will become the embeddable component, but it reached for
getUrlParams() via getKeyForRoom(). Extract the lookup into a helper taking
the room ID and password explicitly: the hook supplies them from the
parameters context, while getKeyForRoom keeps reading the URL for its one
remaining caller in the app shell.

No functional change.
2026-09-02 11:00:42 +02:00
Valere 6e44332fd1 Stop reading URL parameters from inside the call path
The view models reached for getUrlParams() — and so window.location —
from deep inside the call path: CallViewModel, MediaDevices, Publisher,
LocalMember and the footer view model. An embedded Element Call has no URL
of its own, so these values have to arrive as arguments instead.

Add the relevant options to CallViewModelOptions, to the MediaDevices and
Publisher constructors, to createLocalMembership$ and enterRTCSession, and
to createCallFooterViewModel. The remaining React consumers read the
context added in the previous commit. AppViewModel now takes its audio
output options too, moving that URL read out to main.tsx, where the app
shell can act as the adapter.

The new CallViewModelOptions fields are optional, defaulting to what the
URL parameters resolve to outside widget mode; the MediaDevices and
Publisher arguments are required, so that every construction site has to
be explicit.

useTheme.test.ts mocked the UrlParams module with a factory, so it needed
updating to mock the hook rather than getUrlParams.

No functional change.
2026-09-02 10:31:48 +02:00
Valere bc6015eff2 Supply Element Call's parameters through a context
Element Call's parameters come from its URL, which works while it owns
the page but leaves an embedder with nowhere to put them. Add a context
so they can be provided directly, falling back to parsing
window.location when no provider is present.

This also decouples the thirteen consumers from react-router: useUrlParams
called useLocation, so each of them required a router ancestor, which the
embedded build will not have. The standalone and widget builds keep their
URL-derived behaviour via useUrlParamsFromLocation, provided in App.

No functional change.
2026-09-02 09:58:34 +02:00
Valere 0dca5835f6 Route Element Call's DOM decoration through an injectable root element
Element Call writes its theme classes, background and layout attributes
straight onto document.body, and portals its modals there too. That is
only correct while it owns the page; embedded in a host application it
has to confine itself to the container it was mounted into.

Add a RootElementContext, defaulting to document.body so that the
standalone and widget builds are unaffected, and point the theme classes,
data-background, no-scroll-body and the fullscreen target at it.

Give the Modal and Toast portals an explicit container as well. Radix and
vaul both default to document.body, so without this every modal, drawer
and toast would render outside the container and lose the theme and
platform attributes set on it.

No functional change: the root element is document.body until an embedder
provides otherwise.
2026-09-01 18:54:27 +02:00
Valere 0a8c24bca8 Allow the config to be supplied by an embedder
Config.init() derives the location of config.json from window.location,
which only makes sense while Element Call owns the page. When it is
embedded in a host application the host owns the configuration, so add
Config.initWith() to accept it directly.

Share the defaulting and validation between both paths via resolveConfig(),
so that an injected config behaves identically to a fetched one, and mark
initialization as complete so that the init() calls already on the startup
path resolve immediately instead of fetching over the top of it.

initDefault() becomes initWith({}), which also stops it handing out a
shallow copy of DEFAULT_CONFIG whose nested objects were shared with the
module-level default.
2026-09-01 18:40:53 +02:00
Valere 7485b3d71f Use a dedicated i18next instance instead of the global singleton
Element Call configured the global i18next singleton. When Element Call
runs embedded in a host application rather than as its own page, that
singleton belongs to the host, so configuring it would clobber the
host's translations.

Create Element Call's own instance in utils/i18n.ts, configure it in the
initializer, and pass it to components via <I18nextProvider>. Drop
.use(initReactI18next) from the initializer: it registers the instance
as react-i18next's global default, which is the global we are trying to
avoid. Tests and stories keep using it, so that they do not need to wrap
every render in a provider.

Two modules imported `t` directly from "i18next" and so were bound to
the global instance: utils/errors.ts now calls i18n.t() on the instance
(reached at call time, since i18next only assigns `t` during init), and
QrCode uses useTranslation() like every other component.

No functional change.
2026-09-01 18:30:05 +02:00
398 changed files with 31203 additions and 14158 deletions
+2 -2
View File
@@ -5,7 +5,7 @@
> Every PR must have a linked issue
> that a maintainer has reviewed and approved **before you started writing code**.
> PRs that don't meet this requirement will not be reviewed.
> See [CONTRIBUTING.md](https://github.com/element-hq/element-call/blob/livekit/CONTRIBUTING.md) for ElementCall decided for this approach.
> See [CONTRIBUTING.md](https://github.com/element-hq/element-call/blob/main/CONTRIBUTING.md) for ElementCall decided for this approach.
## Content
@@ -39,7 +39,7 @@ Uncomment the markdown table below and fill in the last line:
## Checklist
- [ ] A linked, pre-approved issue exists for this feature or UI change.
- [ ] I have read [CONTRIBUTING.md](https://github.com/element-hq/element-call/blob/livekit/CONTRIBUTING.md) in full.
- [ ] I have read [CONTRIBUTING.md](https://github.com/element-hq/element-call/blob/main/CONTRIBUTING.md) in full.
- [ ] Pull request includes screenshots or videos for any UI changes.
- [ ] Tests written for new code (and existing touched code where feasible).
- [ ] Linter and other CI checks pass.
+3
View File
@@ -102,6 +102,9 @@
- name: "backport-candidate"
description: "Something that is a candidate for backport to a particular release branch"
color: "0B8D85"
- name: "Claude-Review"
description: "Request code review from Claude"
color: "e57157"
- name: "customer-retainer"
color: "F44A5F"
- name: "dependencies"
+4 -1
View File
@@ -44,7 +44,9 @@ jobs:
node-version-file: ".node-version"
- name: Install dependencies
# ignore-pnpmfile should never be commited. Make CI crash if it happened (`pnpmfileChecksum` is present)
run: "pnpm install --frozen-lockfile --ignore-pnpmfile"
run: "scripts/layered.sh --ignore-pnpmfile"
env:
PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
- name: Build Element Call
run: pnpm run build:"$PACKAGE":"$BUILD_MODE"
env:
@@ -62,5 +64,6 @@ jobs:
with:
name: build-output-${{ inputs.package }}
path: dist
if-no-files-found: error
# We'll only use this in a triggered job, then we're done with it
retention-days: 1
+9 -6
View File
@@ -6,7 +6,7 @@ on:
- opened
- labeled
push:
branches: [livekit, full-mesh]
branches: [main]
jobs:
build_full_element_call:
# Use the full package vite build
@@ -22,8 +22,8 @@ jobs:
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
deploy_develop:
# Deploy livekit branch to call.element.dev after build completes
if: github.ref == 'refs/heads/livekit'
# Deploy main branch to call.element.dev after build completes
if: github.ref == 'refs/heads/main'
needs: build_full_element_call
runs-on: ubuntu-latest
steps:
@@ -43,8 +43,8 @@ jobs:
}
})
docker_for_develop:
# Build docker and publish docker for livekit branch after build completes
if: github.ref == 'refs/heads/livekit'
# Build docker and publish docker for main branch after build completes
if: github.ref == 'refs/heads/main'
needs: build_full_element_call
permissions:
contents: write
@@ -73,7 +73,10 @@ jobs:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
build_sdk_element_call:
# Use the embedded package vite build
# The 'sdk' package is the CallViewModel bundle that predates the sdk/
# package (build:sdk:production, sdk-target-based-on-call-view-model/), not
# the package itself. The name is shared with pr-deploy.yaml, which always
# runs from main, so it stays until the bundle's consumers have moved over.
uses: ./.github/workflows/build-element-call.yaml
with:
package: sdk
+90
View File
@@ -0,0 +1,90 @@
name: Claude
on:
pull_request:
types: [labeled]
permissions:
# The action exchanges the GitHub OIDC token for a short-lived Anthropic
# access token. GitHub API access goes through CLAUDE_GH_TOKEN instead.
id-token: write
jobs:
review:
name: Review
# Only run on labelling and skip forks.
if: github.event.label.name == 'Claude-Review' && github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
# The review looks at the individual commits, not just the combined diff
fetch-depth: 0
persist-credentials: false
- name: Review
uses: anthropics/claude-code-action@6fed3ca145920b639991cb756090506e1bcaf515 # v1.0.245
with:
anthropic_federation_rule_id: fdrl_01XQmWo4nka1ZTqZdsdeScyr
anthropic_organization_id: dfbc9a23-d636-4653-8970-035204ef60fa
anthropic_service_account_id: svac_018Ay5qaA7ZW8X7XsbdekqUM
anthropic_workspace_id: wrkspc_01Aw8LHjBS6fZysu9yqzYifW
# The token used here must grant read access to code, issues, and metadata as well as
# read and write access to pull requests.
github_token: ${{ secrets.CLAUDE_GH_TOKEN }}
use_sticky_comment: true
track_progress: true
prompt: |
Review this pull request under the following (and no further) angles:
1. Scope:
- Does the PR do one thing?
- Is the PR free of unrelated fixes and drive-by refactorings?
- If the PR is large, could it reasonably be split into smaller PRs
that still deliver value?
2. Commits:
- Is the work split into self-contained commits of reasonable size?
- Are the commits arranged in a sensible order?
- Have required refactorings been split out into separate commits?
- Is the history free of "wip", checkpoint and fixup-style commits
which only correct an earlier commit in the same PR?
3. Code:
- Are the code changes functionally correct?
- Do the changes introduce new security vulnerabilities or metadata
leaks?
- Is the new code performant enough?
- Don't check anything that is already validated via other GitHub
workflows like linting, formatting or tests passing.
Read the commits themselves with `git log` and `git diff` against
the base branch of the PR.
Drop any findings that are outside the above review angles or that don't
require resolution. Grade the severity of each finding as either "🟡 low",
"🟠 medium" or "🔴 high". Devise at least one possible resolution for each
finding.
Report the graded findings and resolutions from the "Code" angle as inline
comments. Add a footer "_-- Finding from automated review_" to each inline
comment.
Finally, report your overall result as PR comment. Start your message with
"# Automated review". Then add one section per review angle with:
- A verdict of "✅ PASS" or "❌ NEEDS WORK"
- A bullet list summarizing the remaining non-inlined graded findings and
resolutions (as nested bullets).
Be brief and concise in all your communication. Do not mention what you have
not covered and don't bring up issues that have already been fixed.
Use Markdown links for anything that can be navigated to on GitHub such
as files, commits, issues, pull requests, workflow runs, etc.
claude_args: |
--model claude-fable-5-1
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(git log:*),Bash(git diff:*),Bash(git show:*)"
--max-turns 50
+12 -1
View File
@@ -1,6 +1,12 @@
name: Lint, format & type check
on:
pull_request: {}
workflow_call:
inputs:
matrix-js-sdk-sha:
type: string
required: false
description: The matrix-js-sdk commit to build against instead of the locked one
jobs:
lint:
name: Lint, format & type check
@@ -9,6 +15,8 @@ jobs:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
# Inside a called workflow, github.repository is the caller so we need to make sure to still check out the EC repo here.
repository: ${{ inputs.matrix-js-sdk-sha && 'element-hq/element-call' || github.repository }}
persist-credentials: false
- name: Enable Corepack
run: corepack enable
@@ -19,7 +27,10 @@ jobs:
node-version-file: ".node-version"
- name: Install dependencies
# ignore-pnpmfile should never be commited. Make CI crash if it happened (`pnpmfileChecksum` is present)
run: "pnpm install --frozen-lockfile --ignore-pnpmfile"
run: "scripts/layered.sh --ignore-pnpmfile"
env:
PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
JS_SDK_GITHUB_BASE_REF: ${{ inputs.matrix-js-sdk-sha }}
- name: Formatting
run: "pnpm run format:check"
- name: i18n
+36 -3
View File
@@ -3,6 +3,25 @@ on:
pull_request: {}
push:
branches: [main]
workflow_call:
inputs:
matrix-js-sdk-sha:
type: string
required: false
description: The matrix-js-sdk commit to build against instead of the locked one
disable_coverage:
type: boolean
required: false
default: false
description: Skip generating and uploading coverage reports
disable_e2e_tests:
type: boolean
required: false
default: false
description: Skip the end-to-end tests
secrets:
CODECOV_TOKEN:
required: false
jobs:
vitest:
name: Run unit tests
@@ -11,6 +30,8 @@ jobs:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
# Inside a called workflow, github.repository is the caller so we need to make sure to still check out the EC repo here.
repository: ${{ inputs.matrix-js-sdk-sha && 'element-hq/element-call' || github.repository }}
persist-credentials: false
- name: Enable Corepack
run: corepack enable
@@ -21,7 +42,10 @@ jobs:
node-version-file: ".node-version"
- name: Install dependencies
# ignore-pnpmfile should never be commited. Make CI crash if it happened (`pnpmfileChecksum` is present)
run: "pnpm install --frozen-lockfile --ignore-pnpmfile"
run: "scripts/layered.sh --ignore-pnpmfile"
env:
PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
JS_SDK_GITHUB_BASE_REF: ${{ inputs.matrix-js-sdk-sha }}
- name: Get Playwright version
run: echo "PLAYWRIGHT_VERSION=$(pnpm list @playwright/test --depth=0 --json | jq -r '.[0].devDependencies["@playwright/test"].version')" >> $GITHUB_ENV
- name: Cache Playwright binaries
@@ -35,8 +59,11 @@ jobs:
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: pnpm playwright install --with-deps
- name: Vitest
run: "pnpm run test:coverage"
run: pnpm vitest --coverage.enabled="$ENABLE_COVERAGE"
env:
ENABLE_COVERAGE: ${{ !inputs.disable_coverage }}
- name: Upload to codecov
if: ${{ !inputs.disable_coverage }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
@@ -45,11 +72,14 @@ jobs:
fail_ci_if_error: true
playwright:
name: Run end-to-end tests
if: ${{ !inputs.disable_e2e_tests }}
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
# Inside a called workflow, github.repository is the caller so we need to make sure to still check out the EC repo here.
repository: ${{ inputs.matrix-js-sdk-sha && 'element-hq/element-call' || github.repository }}
persist-credentials: false
- name: Enable Corepack
run: corepack enable
@@ -59,7 +89,10 @@ jobs:
node-version-file: ".node-version"
- name: Install dependencies
# ignore-pnpmfile should never be commited. Make CI crash if it happened (`pnpmfileChecksum` is present)
run: pnpm install --frozen-lockfile --ignore-pnpmfile
run: scripts/layered.sh --ignore-pnpmfile
env:
PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
JS_SDK_GITHUB_BASE_REF: ${{ inputs.matrix-js-sdk-sha }}
- name: Get Playwright version
run: echo "PLAYWRIGHT_VERSION=$(pnpm list @playwright/test --depth=0 --json | jq -r '.[0].devDependencies["@playwright/test"].version')" >> $GITHUB_ENV
- name: Cache Playwright binaries
+6
View File
@@ -10,6 +10,9 @@ public/config.json
backend/synapse_tmp/*
backend/synapse_tmp_othersite/*
/coverage
# Transient agent scratch files
/agent-workspace/
config.json
# Yarn
@@ -38,3 +41,6 @@ yarn-error.log
*storybook.log
storybook-static
# Nested checkout made by scripts/layered.sh
/matrix-js-sdk/
+12
View File
@@ -35,10 +35,22 @@
],
"element-call/no-observablescope-leak": "error",
"element-call/no-top-level-logger-get-child": "error",
"element-call/sdk-import-boundary": "error",
"jsdoc/empty-tags": "error",
"jsdoc/check-property-names": "error",
"jsdoc/require-param-description": "warn",
"react/display-name": "error",
// TODO: These React Compiler-based rules joined oxlint's correctness
// category in v1.81 and flag many pre-existing patterns, including hooks
// like useLatest/useInitial/useReactiveState that intentionally access
// refs during render. Re-evaluate them when adopting the React Compiler.
"react/immutability": "off",
"react/no-deriving-state-in-effects": "off",
"react/preserve-manual-memoization": "off",
"react/refs": "off",
"react/set-state-in-effect": "off",
"react/static-components": "off",
"react/use-memo": "off",
// TODO: Re-enable once oxlint supports lint rules that rely on TypeScript type-awareness.
// "rxjs/no-exposed-subjects": "error",
// "rxjs/finnish": [
+4 -2
View File
@@ -7,14 +7,16 @@ Please see LICENSE in the repository root for full details.
import type { Preview } from "@storybook/react-vite";
import { TooltipProvider } from "@vector-im/compound-web";
import i18n from "i18next";
import { logger } from "matrix-js-sdk/lib/logger";
import EN from "../locales/en/app.json";
import { initReactI18next } from "react-i18next";
import { i18n } from "../src/utils/i18n";
import "../src/index.css";
// Bare-minimum i18n config
// Bare-minimum i18n config.
// Unlike the app, stories register the instance as react-i18next's default
// rather than wrapping every story in an <I18nextProvider>.
i18n
.use(initReactI18next)
.init({
+56
View File
@@ -0,0 +1,56 @@
# AGENTS.md — Element Call
MatrixRTC (MSC4143) + LiveKit video calling, shipped standalone, as a widget in
Element Web and Element X, as embedded packages, and as a React component in a
host's page. It is the MatrixRTC reference implementation.
## Non-negotiables
- Features and UI changes need a pre-approved issue. No issue, no review.
- Exactly one `PR-*` label per PR. CI enforces it; it drives the changelog.
- New behaviour ships with unit tests, a story if it renders, and an e2e spec if it
is user-facing. All three.
- Reuse an existing component. If you genuinely cannot, say so in the PR body and
name what you rejected — never add a new one silently.
- Nothing reads the page. No `window.location`, `document.body`, `window.inner*`,
`@media`, global `i18next` or the `widget` global — take it from a provider.
- The change works standalone, as a widget and as a component. Say what you checked.
- Every gate below is green before you push.
- Hand off after the first implementation, before the quality pass. Then commit on
the user's word where the setup allows it, or hand them the message.
## Read before you
| … | … |
| -------------------------------- | ---------------------------------------------------------- |
| start a task of any size | [docs/agents/workflow.md](docs/agents/workflow.md) |
| touch call logic or a view model | [docs/agents/architecture.md](docs/agents/architecture.md) |
| write any code | [docs/agents/code-style.md](docs/agents/code-style.md) |
| write a test, story or e2e spec | [docs/agents/testing.md](docs/agents/testing.md) |
| open a PR | [CONTRIBUTING.md](CONTRIBUTING.md) |
| implement from a feature spec | [FEATURES_SPEC/AGENTS.md](FEATURES_SPEC/AGENTS.md) |
## Gates
```sh
pnpm lint # tsc, oxlint, knip, component externals
pnpm format # oxfmt
pnpm test # vitest: unit and storybook projects
pnpm i18n:check
```
`knip` fails on dead code; a file deliberately inert ahead of its consumer goes in
`knip.ts` `ignoreFiles` with a reason.
## A good PR
- One slice, under ~400 changed lines, green on its own, linked to its issue.
- View model + marble tests, thin view, a story per state, an e2e spec.
- Any new shared component called out explicitly, with why nothing existing fit.
- Template filled for real: what, why, before/after screenshots, repro steps.
- Comments only where the code can't give the reason — see
[code-style.md](docs/agents/code-style.md#comments).
- Anything that redraws continuously — an animation frame, a stream — says what it
costs, or better, counts the redraws in a test.
- Branch `<handle>/<topic>`. Plain imperative commit subjects, no prefixes.
- Once review starts, fix forward. Never force-push a regeneration over a review.
+232
View File
@@ -0,0 +1,232 @@
# Merging PR #4331 onto the SDK / Element Call split
How [element-hq/element-call#4331](https://github.com/element-hq/element-call/pull/4331)
("LiveKit updates": MSC4195 client-server API tokens, `url` transports, delegation
through the homeserver, a transport dev override) lands on `toger5/use-sdk`, where
the MatrixRTC logic lives in `sdk/` and Element Call consumes it through
`sdk/src/api.ts`. The PR was written against `main`, where all of this still sat
in `src/state/CallViewModel/`. This document goes through the PR commit by commit,
says whether each one lands in `sdk/` or `src/`, and how; the section
[Changes to `api.ts`](#changes-to-apits) covers the public interface in full. The
end state: this branch behaves exactly as the PR does, with the split and the
`api.ts` seam intact.
The PR's branch is checked out at `agent-workspace/pr4331-merge/worktree` for
comparison. The `matrix-js-sdk` dependency is pinned to the same `develop`
commit the PR uses (`4df2f2bf`, 43.0.0), which brings `UnstableLivekitTransport`,
`CallMembership.backendIdentities`, `_unstable_getLivekitToken` and
`_unstable_delegateDelayedLeave`. That bump also renamed
`MatrixRTCSession.leaveRoomSession` to `leave`, which the SDK now calls.
## Where each commit lands
| # | Commit | Lands in | How |
| --- | ------------------------------------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | Remove stray pause and improve logging for hung e2e tests | e2e | Applied as is: `playwright.config.ts`, `playwright/widget/federated-call.test.ts`. |
| 2 | Fix typo in the name of the createJWTToken Playwright fixture | e2e | Applied as is. |
| 3 | Advertise the loopback address from the dev LiveKit SFUs | backend | Applied as is: `backend/dev_livekit*.yaml`. |
| 4 | Match LiveKit participants against all backend identities | sdk | `RTCMember.backendIdentities` (new, see api.ts). `LivekitBackend.mediaFor` matches `member.backendIdentities.includes(p.identity)`; `RTCParticipation.data$` maps a packet's sender the same way. `mockRtcMembership` in both `sdk/src/utils/test.ts` and `src/utils/test.ts` takes the new 4-argument constructor and a `backendIdentities` override. |
| 5 | Model ConnectionState as a discriminated union | sdk | `Connection.ts` state is `{state: "initialized" \| "authenticating" \| "authenticated" (+ `livekitState`) \| "error" (+ `error`) \| "stopped"}`. Internal to the LiveKit backend: `connectionState.ts` maps it onto the backend-neutral `MediaConnectionState`, so nothing above the backend changes. `DataChannel.ts` and `ConnectionFactory.ts` follow. |
| 6 | Show the whole transport in debug UI and logs | sdk + src | SDK log lines use `JSON.stringify(transport)`. Element Call: `focusUrl$: Behavior<string>` on the member media view models becomes `transport$: Behavior<Transport>` (`TransportMetadata.raw`), carried through `GridTile`, `SpotlightTile`, `MediaView` to `RTCConnectionStats`, which prints the JSON. The old code showed `extractDomain(transport.id)`, which on this branch was already a JSON string, so this also fixes a latent bug. |
| 7 | Group livekitRoomItems$ by connection | n/a | `livekitRoomItems$` and `LivekitRoomAudioRenderer` do not exist on this branch: audio renders from `tracks$` (`MemberAudioRenderer`). Nothing to port. |
| 8 | Move SFU authentication into livekit/auth and choose the endpoint by role | sdk | `sdk/src/media-backend/livekit/auth/{index,openID,csApi,types}.ts` replace `openIDSFU.ts`. Publisher → legacy `/sfu/get`; subscriber → `/get_token`. `slotId` replaces the hard-coded `m.call#ROOM`: `MediaBackendContext.slotId`, taken from the js-sdk session in `RTCParticipation`. The `tokenEndpoint` option of `createLivekitBackend` and the derivation from `MatrixRTCMode` are gone. |
| 9 | Let the publishing Connection authenticate itself | sdk | Every `Connection` has a `role` and fetches its own token; `existingSFUConfig` is gone. `prepareLocalTransport` no longer fetches a token itself: it starts the publisher connection (through the connection map) and resolves once that connection reports `authenticated`, rejecting with its error otherwise. Since `LocalMembership` joins only after `preparedTransport$` resolves, the join waits for the publisher's token exactly as the PR's `authenticated$` gate does. |
| 10 | Reduce ConnectionManager to a ConnectionMap | sdk | `ConnectionMap.ts` replaces `ConnectionManager.ts`; `createConnectionMap$` takes a resolved local transport, and `LivekitBackend` wraps it in `mapScoped` over the local transport, as the PR's `CallViewModel` did. `LocalMedia.ts`, `DataChannel.ts` take `connectionMap$`. |
| 11 | Use the unstable LiveKit transport type | sdk + src | `UnstableLivekitTransport` / `isUnstableLivekitTransport` throughout the backend; `src/settings/settings.ts`, `DeveloperSettingsTab`, test helpers. Transport discovery in `sdk/src/matrixrtc/` is now backend neutral (filters by `type`, see commit 15). |
| 12 | Identify transports by server name as well as URLs | sdk | `TransportLocator {transport, serverName}` in `auth/types.ts`; `ConnectionMap.keyFor` keys on `(url, livekit_service_url, serverName)`. The server name is derived from the user id (`serverNameOf`) for remote members and from our own membership identity for the local transport, so neither `api.ts` nor the backend options need it. |
| 13 | Pin matrix-js-sdk to the member_id branch | superseded | By commit 19; the lockfile pins the PR's final `develop` commit. |
| 14 | Get SFU tokens and delegate delayed leaves through the MSC4195 CS API | sdk | `auth/csApi.ts` (`_unstable_getLivekitToken`, `_unstable_delegateDelayedLeave`), `auth/index.ts` facade with the fallback rules. `LivekitBackend.delegateDelayedLeave` goes through the facade; the delegation probe asks only the service the publisher flow uses. `LivekitBackendOptions.client` picks the two new client methods. |
| 15 | Let developers override the whole transport | sdk + src | `RTCParticipationOptions.transport` / `fallbackTransport` take a `Transport` instead of `transportUrl` / `fallbackTransportUrl` (see api.ts). Element Call: setting `custom-transport` (JSON) replaces `custom-livekit-url`; `DeveloperSettingsTab` edits JSON and validates through `authenticateWithTransport`; `InCallView` turns `config.livekit.livekit_service_url` into the fallback transport. Locale keys `developer_mode.custom_transport.*`. |
| 16 | Deprecate the livekit.livekit_service_url config option | src | Applied as is in `ConfigOptions.ts`. |
| 17 | Cover url transports end to end | e2e + backend | `playwright/url-transport.spec.ts`, `backend/app-service*.yaml`, `docker-compose-dev.yml` (lk-jwt-service 0.8.0) copied as is. |
| 18 | Retain compatibility in compatibility mode | superseded | Reverted by commit 23 inside the PR; only its e2e spec edits survive, in the final spec file. |
| 19 | Switch JS SDK back to develop | deps | `pnpm-lock.yaml` pins `4df2f2bf` and `matrix-widget-api` 1.20.0; `package.json` keeps `#develop`. |
| 20 | Fall back to get_token as subscriber on widget API errors | sdk | `isCsApiUnavailable` in `auth/index.ts`; `doNetworkOperationWithRetry(op, shouldRetry)` in `sdk/src/utils/network.ts` with `isNetworkError` in `csApi.ts`. |
| 21 | Share a connection between homeservers using the same JWT service | sdk | `keyFor` ignores the server name for transports without a `url`; `removeDuplicateTransports` keeps the first locator so the local one wins. |
| 22 | Avoid null in transport dev setting | src | `transportText()` in `DeveloperSettingsTab.tsx`. |
| 23 | Make the auth choice mode independent again | sdk | `chooseAuthFlow(transport, role)` has no mode parameter; `config.ts` documents that `MatrixRTCMode` decides how the membership is written, not how the transport is authenticated. |
| 24 | Cover identity matching and delegation across transport shapes | sdk + e2e | Unit tests in `LivekitBackend.test.ts` (see below); `playwright/url-transport.spec.ts` final version. |
| 25 | Cover the subscriber token path, widget fallback and server names | sdk + e2e | `auth/openID.test.ts` subscriber cases; `playwright/widget/url-transport.test.ts`; the server-name test lives in `LivekitBackend.test.ts` ("connects to a remote member's transport on the member's homeserver"). |
## Changes to `api.ts`
Three changes to the public SDK interface, all forced by the PR's behaviour.
### `RTCMember.backendIdentities` (new)
```ts
/** The membership's `rtcBackendIdentity`, `${userId}:${deviceId}`: a key and a debug label. */
rtcBackendIdentity: string;
/**
* Every identity the member may publish under on its transport: the
* participant identity the SFU sees, and the sender of its data packets.
* One per authentication flow the member's transport allows, so a transport
* with both a `url` and a `livekit_service_url` lists the hashed and the
* legacy identity, and the backend matches whichever shows up.
*/
backendIdentities: readonly string[];
```
Commit 4 of the PR. A member on a transport that carries both a `url` and a
`livekit_service_url` may have authenticated through either flow and so has
two candidate identities; the js-sdk now lists them on the membership. The
backend matches media and data packets against the list.
`rtcBackendIdentity` stays as the stable per-membership key Element Call uses
for tile ids and the debug overlay, and its doc says that it is no longer
what the backend matches on. Read off the membership, like the single identity
before it; the backend will derive both once the identity scheme is its own.
### `RTCParticipationOptions.transport` / `fallbackTransport` replace the url options
```ts
/**
* Use this transport instead of asking the homeserver. Any shape the
* backend serves: for LiveKit a `url`, a `livekit_service_url` or both.
*/
transport?: Transport;
/** Use this transport when the homeserver advertises none of the backend's type. */
fallbackTransport?: Transport;
```
Commit 15 of the PR. The developer override has to carry a whole transport so
that a `url`-only or combined shape can be tested against a homeserver that
does not advertise one yet; a URL cannot say which shape it is. This also
removes the one piece of LiveKit knowledge that sat outside the backend:
`LocalTransport.ts` and `TransportAutoDiscovery.ts` no longer build a
`{type: "livekit", livekit_service_url}` object and filter by `transportType`
only, which `LocalTransport.ts` had marked as the moment to do this. The
host turns its config URL into a transport (`InCallView.rtcParticipationOptions`).
### `TransportMetadata` is unchanged, with one documented limit
`TransportMetadata.id` is still the raw transport serialised with sorted keys.
The LiveKit backend now keys its connections on the transport plus the server
name where the transport has a `url` (commits 12 and 21), so two homeservers
advertising the same `url` get two connections but one `TransportMetadata`.
`connectedTransports$` deduplicates, and `resolved$` reports the first
connection's resolution. This is diagnostics only; the media and data paths
go through the backend's own keys. Extending `TransportMetadata` with the
server name was left out to keep the type transport neutral; a developer panel
that needs it can be given a `serverName` on `BackendConnection` later.
### Related, outside `api.ts`
- `media-backend/api.ts`: `MediaBackendContext.slotId` (commit 8). The
client-server API issues a token for a slot, so the backend needs it;
`RTCParticipation` reads it from the js-sdk session.
- `sdk/index.ts`: `authenticateWithTransport` is now the auth facade's
`getSFUConfig({client, membership, transport, serverName, roomId, slotId, role, logger})`
(object parameter, whole transport, server name); `AuthenticateWithTransportParams`
is exported for it; `TokenEndpoint` is gone.
- `createLivekitBackend` options: `tokenEndpoint` is gone; `client` now picks
`_unstable_getLivekitToken` and `_unstable_delegateDelayedLeave` as well.
- `config.ts`: the `MatrixRTCMode` doc no longer claims the mode picks the
JWT endpoint (commit 23).
## Behaviour, side by side
| Behaviour | PR on `main` | This branch |
| -------------------------------------- | ------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| Endpoint choice | `chooseAuthFlow` in `src/livekit/auth` | Same function, `sdk/src/media-backend/livekit/auth/index.ts` |
| Join waits for the publisher's token | `authenticated$` gate in `LocalMember.ts` | `prepareLocalTransport` resolves on the publisher connection's `authenticated` state; `LocalMembership` joins after it |
| Delegation probe | The flow's service only, in `LocalMember.joinParams$` | The flow's service only, in `prepareLocalTransport` |
| Delegation handover | `delegateDelayedLeave` facade from `LocalMember` | Same facade from `LivekitBackend.delegateDelayedLeave`, with the delay timeout from the session timings instead of `Config` |
| Transport keys and connection sharing | `ConnectionMap.keyFor` on `TransportLocator` | Same, moved |
| Server names | `membershipsAndTransports$` from the user id; `getLocalTransport` from `client.getDomain()` | `serverNameOf(userId)` for members; our own membership identity's user id for the local transport |
| Identity matching | `membership.backendIdentities.includes(p.identity)` | `member.backendIdentities.includes(p.identity)` in `mediaFor` and for data packets |
| Dev transport override | `customTransport` setting → `getLocalTransport` | `customTransport` setting → `RTCParticipationOptions.transport` |
| Config fallback | `RtcTransportAutoDiscovery.tryConfigTransport` | Host builds the transport, `RTCParticipationOptions.fallbackTransport` |
| Error for a `url` host without MSC4195 | `NoMatrix2AuthorizationService` from the publisher connection | Same error, surfaced through `prepareLocalTransport` → `state$ failed` → `fromMatrixRTCError` |
One deliberate difference: on `main` the publisher connection fetches its token
on every (re)creation and nothing else does; here it is the same, since the
up-front token fetch in `prepareLocalTransport` is gone. There is no second
token request per join.
## Where the PR's new tests went
| PR test | Here |
| ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `src/livekit/auth/index.test.ts` | `sdk/src/media-backend/livekit/auth/index.test.ts`. Rewritten on the network layer (fetch-mock for the JWT service, mocked client methods) because a `vi.mock` of the sibling `openID` module does not reach the facade's own import under `sdk/`. Same cases, plus a table for `chooseAuthFlow`. |
| `src/livekit/auth/openID.test.ts` | `sdk/src/media-backend/livekit/auth/openID.test.ts`, replacing `openIDSFU.test.ts` |
| `ConnectionMap.test.ts` (incl. commit 21's sharing tests) | `sdk/src/media-backend/livekit/ConnectionMap.test.ts`, replacing `ConnectionManager.test.ts`; plus a role-tagging test |
| `Connection.test.ts` changes | `sdk/src/media-backend/livekit/Connection.test.ts`; plus a publisher-through-the-JWT-service case |
| `LocalMember.test.ts` "joins session with %s delegation support" (by transport shape) | `LivekitBackend.test.ts` "prepares the local transport and asks %s about delegation", "waits for the publisher connection to hold its token before the join", "fails the preparation with the publisher connection's error", "delegates the leave through the flow that authenticated the publisher" |
| `MatrixLivekitMembers.test.ts` two-candidate-identity tests | `LivekitBackend.test.ts` › `mediaFor` |
| `SessionBehaviors.test.ts` server-name test | `LivekitBackend.test.ts` "connects to a remote member's transport on the member's homeserver" |
| `LocalTransport.test.ts` / `RtcTransportAutoDiscovery.test.ts` changes | `sdk/src/matrixrtc/TransportAutoDiscovery.test.ts` (fallback is a transport) |
| `DeveloperSettingsTab.test.tsx` custom transport cases | `src/settings/DeveloperSettingsTab.test.tsx`, snapshot updated for the label |
| `DataChannel`, `LivekitBackend` registry tests | Adapted to the state union and `TransportLocator` |
| `playwright/url-transport.spec.ts`, `playwright/widget/url-transport.test.ts` | Copied as is: they drive the app, which behaves the same |
The PR's `CallViewModel.test.ts`, `CallViewModelTestUtils.ts`, `LocalMember.test.ts`
(the rest) and `MatrixAudioRenderer.test.tsx` edits have no counterpart: those
modules were replaced by the SDK before this merge.
## One fix the e2e run forced
The bumped js-sdk sets `rtcBackendIdentity` to the legacy `${userId}:${deviceId}`
identity for every membership (it is deprecated there; the hashed one now only
appears in `backendIdentities`). On this branch the call view model keyed its
tiles by `[dup, mediaId, rtcBackendIdentity]`, so a device that rejoined after an
improper leave, which has two memberships until the stale one expires, produced
duplicate keys and crashed the host's page with `UNKNOWN_ERROR`
(`spa-call-sticky.spec.ts`, "rejoin after improper leave"). The key is now the
membership's own `memberId`, and `MemberAudioRenderer` and the SDK harness key
by it too. The PR on `main` keyed by the participant object as well and did not
hit this.
## Coverage added on the package branch, and one bug it found
The PR's tests mostly covered the auth layer; the SDK modules under them had
little or none. Added on the package branch, which this branch is rebased onto:
| Module | Test |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `media-backend/livekit/Publisher.ts` | `Publisher.test.ts`: the cases from `main`'s `localMember/Publisher.test.ts` on a real `LocalParticipant` with faked device and signalling calls, plus publish, unpublish and destroy |
| `media-backend/livekit/LocalMedia.ts` | `LocalMedia.test.ts`: on a fake connection map; publish before and after the connection, republish on a new connection, mutes remembered, first publish error, output device |
| `RTCParticipation.ts` | `RTCParticipation.test.ts`: on a fake media backend; join options, state machine, publishing only while joined, delegation, member ids per mode, key replay, data sender check |
| `matrixrtc/LocalTransport.ts` | `LocalTransport.test.ts`: override, homeserver, fallback, none |
| `matrixrtc/joinJsSdkSession.ts` | `joinJsSdkSession.test.ts`: the js-sdk join options, the retry budget, the intent and sticky events |
| `media-backend/livekit/publishOptions.ts` | `publishOptions.test.ts` |
| `media-backend/livekit/ConnectionFactory.ts` | `ConnectionFactory.test.ts`: the room options from the requests and media quality (the cases from `main`'s `ECConnectionFactory.test.ts`), `resolved$` |
| `LivekitBackend.ts` › `mediaFor` | A member on another transport, a member changing transport, a member without one (from `main`'s `MatrixLivekitMembers.test.ts`) |
`ConnectionFactory.test.ts` reads the options off the real LiveKit room the
factory builds instead of mocking `livekit-client`: a `vi.mock` of the package
reached the factory's import on the package branch but not on this one.
The `LocalMedia` tests found a race in `ObservableScope.reconcile`: the loop was
marked as running until the promise of its last run had settled, a few
microtasks after it had last checked for new values, so a value arriving in
that window (a connection appearing at just that moment) was never processed.
Fixed in `[SDK] Keep a value that reaches reconcile as its loop winds down`,
with a test that drives the window microtask by microtask. Element Call's
`src/state/ObservableScope.ts` re-exports the SDK's class, so the fix applies
to the app as well.
## Checked
- `pnpm lint` (tsc, oxlint, knip, externals), `pnpm format:check`, `pnpm i18n:check`.
- `pnpm test`: 119 unit files, 950 tests; 4 storybook files, 45 tests (after the coverage above).
- Playwright on Chromium against the local Playwright backend, one worker:
- `playwright/url-transport.spec.ts`: 10 of 10 pass, including delegation
through the homeserver and the JWT service, the subscriber fallback, the
publisher error and the federated token.
- `playwright/widget/url-transport.test.ts`: passes.
- The rest of `playwright/*.spec.ts`, `playwright/component`, `playwright/sdk`:
48 pass, 1 skipped, 1 fails (`reconnect.spec.ts`, which fails on `main`
locally too because the local JWT service is slower than the spec's wait).
- `playwright/widget`: 7 of 10 pass, including `federated-call`. `huddle-call`
fails before any call code runs (the Element Web develop image's room list no
longer matches the helper's "Open room" selector), and `screen-share` looks
for `video[data-lk-source="screen_share"]`, an attribute the LiveKit React
components set and the SDK's plain `<video>` elements do not: a failure of this
branch that predates the PR.
- Local environment work needed for that: the othersite JWT service and nginx
containers had to be recreated from the compose file, two stale containers held
port 16080, and the widget helper's hard-coded `https://localhost:3000` was
pointed at 3100 for the run only, since port 3000 is taken on this machine.
- Standalone, widget and component all go through `InCallView.rtcParticipationOptions`
and the SDK, so the change is the same in all three modes; the widget-specific
fallback (a host without MSC4533) is in `isCsApiUnavailable`.
+129 -10
View File
@@ -34,15 +34,16 @@ You can find the latest development version continuously deployed to
## ✨ Key Features
✅ **Decentralized & Federated** – No central authority; works across Matrix
homeservers.
✅ **End-to-End Encrypted** – Secure and private calls.
✅ **Standalone & Widget Mode** – Use as an independent app or embed in Matrix
clients.
✅ **WebRTC-based** – No additional software required.
homeservers.
✅ **End-to-End Encrypted** – Secure and private calls.
✅ **Standalone, Widget & Component Mode** – Use as an independent app, embed
in Matrix clients as a widget, or (experimentally) mount it as a React component
inside your own application.
✅ **WebRTC-based** – No additional software required.
✅ **Scalable with LiveKit** – Supports large meetings via SFU
([MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)).
([MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)).
✅ **Raise Hand** – Participants can signal when they want to speak, helping to
organize the flow of the meeting.
organize the flow of the meeting.
✅ **Emoji Reactions** – Users can react with emojis 👍️ 🎉 👏 🤘, adding
engagement and interactivity to the conversation.
@@ -90,7 +91,9 @@ and voice calls within Matrix rooms.
Element Call offers two packaging options: one for standalone or widget
deployment, and another for seamless widget-based integration into messenger
apps. Below is an overview of each option.
apps. A third, experimental option builds it as a React component library for
applications that want to render a call inside their own page rather than in an
iframe. Below is an overview of each option.
**Full Package** – Supports both **Standalone** and **Widget** mode. It is
hosted as a static web page and can be accessed via a URL when used as a widget.
@@ -107,6 +110,11 @@ recommended method for embedding Element Call.
<img src="./docs/embedded_package.drawio.png" alt="Element Call Embedded Package">
</p>
**Component Package (experimental)** – A library build of Element Call as a
React component, consumed as a dependency by a host application that already
has a Matrix client. See
[Element Call as a component](#element-call-as-a-component-experimental) below.
For more details on the packages, see the
[Embedded vs. Standalone Guide](./docs/embedded_standalone.md).
@@ -213,6 +221,114 @@ See also:
- [Developing with linked packages](./docs/linking.md)
#### Element Call as a component (experimental)
Element Call can also be embedded directly into another React application
rather than being loaded in an iframe as a widget. `pnpm build:component`
builds it as a library into `component/dist` (the bundle, its stylesheet and
type declarations), and
```sh
pnpm dev:component
```
serves a harness on port 3001 that stands in for such an application: it signs
in twice against the development backend and shows two calls side by side, in
resizable boxes, with page furniture of its own around them. Use it to see how
Element Call behaves when it does not own the page — the size it is given,
whether it stays inside its container, and what it says to its host, which is
logged along the bottom. The harness is served with the same development
certificate as the app, so unless the development CA is trusted, the browser
needs a certificate exception for `https://localhost:3001` as well (see the
note under [Backend](#backend)). It reads the same `public/config.json` as
`pnpm dev` if one exists, and runs with Element Call's defaults otherwise.
The call lays itself out for the size of the element it is mounted in, not the
window: a host that shrinks the container to a corner of its page gets the
picture-in-picture layout, just as a host that shrank the whole iframe used to.
The breakpoints in the stylesheets the component uses are
`@container element-call` queries against its root element for the same reason;
for the standalone app the root is the page, so they mean what the media queries
they replaced did. (The standalone-only views, such as the home and login pages,
still use plain media queries, since the component never shows them.)
The component's stylesheet is confined to the element it is mounted in: the
build rewrites every selector so that it matches only Element Call's root or
what is inside it, with `html`, `body` and `:root` standing for that root (see
`component/build/scopeStylesToRoot.ts`). A host's own page keeps its styles,
and Element Call brings its own fonts and design tokens along.
The component speaks every language the app does. English is bundled in; the
other locales are split into chunks the host's bundler loads the first time
they are needed. It starts in the browser's language, and follows the host's
own language setting through the `language` prop (`supportedLanguages` lists
the tags it accepts, and anything else falls back to its base language or to
English). The `theme` prop works the same way and takes the same values as the
widget's `theme` URL parameter: `light`, `dark`, `light-high-contrast` or
`dark-high-contrast`. Both can change while a call is running without
disturbing it.
A host must call and await `initializeElementCall(config)` once before
rendering the component: it loads the `Intl` polyfills, applies the
deployment-wide `config.json`-style configuration and sets up translations.
The component itself takes the host's `client` and the `roomId` to call in, an
`intent` saying what the user asked for (which decides whether to show the
lobby, ring, and so on), an optional `config` overriding what the intent
implies, and an optional `hostBridge` through which Element Call tells the host
that the user has joined or hung up, that it wants to stay on screen, and so
on. The host makes its own requests (`join`, `hangUp`, `setDeviceMute`) through
the handle exposed on `ref`. The full API is documented in the type declarations
(`component/index.tsx` and `component/host.ts`).
A few things differ from the widget on purpose: the component draws a solid
background rather than a gradient unless told otherwise, never offers to edit
the user's profile (the account is the host's), scopes its keyboard shortcuts to
its own root element so that several instances can share a page, and only shows
its own post-call and error screens when the host has not supplied a `close()`
callback; with one, it asks the host to unmount it instead. The
[global JS controls](./docs/controls.md) on `window` are unchanged and remain
page-wide, so with several instances on one page they apply to all of them.
The package is not published yet. A host installs it as a git dependency on the
`component` directory of this repository,
```json
"@element-hq/element-call-component": "github:element-hq/element-call#main&path:/component"
```
whose `prepare` script runs the build on install. That build needs pnpm (via
Corepack) on the host's machine, runs a full `pnpm install` of this repository,
since it inherits the `--max-old-space-size` setting of
the app build; the host's pnpm also has to allow it to run at all
(`allowBuilds` in its `pnpm-workspace.yaml`). Note that `component/` is a pnpm
project of its own for this reason, so pnpm commands run from inside that
directory target it rather than the repository; run them from the repository
root. The host imports the component from
`@element-hq/element-call-component` and the stylesheet from
`@element-hq/element-call-component/style.css`, and has to provide `react`,
`react-dom`, `matrix-js-sdk`, `@vector-im/compound-web` and
`@vector-im/compound-design-tokens` itself, since the bundle leaves them
external.
Compound's stylesheet are the host's to load too: the component shares
the host's copy of Compound, (required for example for tooltips)
The component is large, and a host will usually load it lazily, only once a
call is shown. Everything a host needs in order to talk about a call before
then is also exported
from `@element-hq/element-call-component/api` (a small entry point that does not
load the component).
- props and the types they are made of
- the handle
- the host bridge
- `UserIntent`
- `HeaderStyle`
- `BackgroundStyle` enums
- `configurationForIntent`, the defaults each intent implies
Import from it wherever a value such as
`BackgroundStyle.Solid` is needed on a path that must stay light; the main
entry point re-exports all of it too.
### Backend
A docker compose file `docker-compose-dev.yml` is provided to start the
@@ -274,7 +390,10 @@ running Playwright by following
However the Playwright tests are run, an element-call instance must be running
on https://localhost:3000 (this is configured in `playwright.config.ts`) - this
is what will be tested.
is what will be tested. The tests under `playwright/component` instead drive
the component harness (`pnpm dev:component`) on https://localhost:3001, which
Playwright starts as a second web server; it is always a Vite dev server, even
when the app itself is served from Docker with `USE_DOCKER`.
The local backend environment should be running for the test to work:
`pnpm backend`
@@ -373,7 +492,7 @@ We do this so that we can reuse the labels between repositories.
## 📝 Copyright & License
Copyright 2021-2025 New Vector Ltd
Copyright 2021-2026 New Vector Ltd
This software is dual-licensed by New Vector Ltd (Element). It can be used
either:
+17
View File
@@ -0,0 +1,17 @@
id: "lk-jwt-service-othersite"
as_token: "ec_dev_as_token_othersite_0123456789ab"
hs_token: "ec_dev_hs_token_othersite_0123456789ab"
sender_localpart: "_lk_jwt_service"
namespaces:
users:
- exclusive: false
regex: "@.*:synapse\\.othersite\\.m\\.localhost" # Cover all local users
# Required for Synapse to call POST /_matrix/app/v1/ping on this service.
url: "http://auth-server-1:16080"
# Enable membership look-ups via /is_joined.
io.element.msc4502.scopes:
["urn:matrix:client:io.element.msc4502:rooms:is_joined"]
# Route /_matrix/client/(v*|unstable/*)/rtc/livekit/* C-S and S-S requests to this
# service.
io.element.msc4512.proxy_prefix: "rtc/livekit"
io.element.msc4512.proxy_url: "http://auth-server-1:16080"
+17
View File
@@ -0,0 +1,17 @@
id: "lk-jwt-service"
as_token: "ec_dev_as_token_main_0123456789ab"
hs_token: "ec_dev_hs_token_main_0123456789ab"
sender_localpart: "_lk_jwt_service"
namespaces:
users:
- exclusive: false
regex: "@.*:synapse\\.m\\.localhost" # Cover all local users
# Required for Synapse to call POST /_matrix/app/v1/ping on this service.
url: "http://auth-server:6080"
# Enable membership look-ups via /is_joined.
io.element.msc4502.scopes:
["urn:matrix:client:io.element.msc4502:rooms:is_joined"]
# Route /_matrix/client/(v*|unstable/*)/rtc/livekit/* C-S and S-S requests to this
# service.
io.element.msc4512.proxy_prefix: "rtc/livekit"
io.element.msc4512.proxy_url: "http://auth-server:6080"
+7
View File
@@ -42,6 +42,13 @@ experimental_features:
msc4354_enabled: true
# MatrixRTC
msc4143_enabled: true
# Enable membership look-up via /is_joined.
msc4502_enabled: true
# Enable C-S & S-S request proxying for application services.
msc4512_enabled: true
app_service_config_files:
- /data/cfg/app-service.yaml
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no
+7
View File
@@ -42,6 +42,13 @@ experimental_features:
msc4354_enabled: true
# MatrixRTC
msc4143_enabled: true
# Enable membership look-up via /is_joined.
msc4502_enabled: true
# Enable C-S & S-S request proxying for application services.
msc4512_enabled: true
app_service_config_files:
- /data/cfg/app-service.yaml
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no
+5
View File
@@ -6,6 +6,11 @@ rtc:
port_range_start: 50300
port_range_end: 50400
use_external_ip: false
# Advertise the loopback address instead of the container IP, which is not
# routable from the host when Docker runs in a VM (colima, Docker Desktop).
# The browser then reaches the SFU through the published ports. Firefox
# ignores loopback candidates, so this only works in Chromium-based browsers.
node_ip: 127.0.0.1
turn:
enabled: false
domain: localhost
+5
View File
@@ -6,6 +6,11 @@ rtc:
port_range_start: 50100
port_range_end: 50200
use_external_ip: false
# Advertise the loopback address instead of the container IP, which is not
# routable from the host when Docker runs in a VM (colima, Docker Desktop).
# The browser then reaches the SFU through the published ports. Firefox
# ignores loopback candidates, so this only works in Chromium-based browsers.
node_ip: 127.0.0.1
turn:
enabled: false
domain: localhost
@@ -42,6 +42,13 @@ experimental_features:
msc4354_enabled: true
# MatrixRTC
msc4143_enabled: true
# Enable membership look-up via /is_joined.
msc4502_enabled: true
# Enable C-S & S-S request proxying for application services.
msc4512_enabled: true
app_service_config_files:
- /data/cfg/app-service.yaml
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no
+7
View File
@@ -42,6 +42,13 @@ experimental_features:
msc4354_enabled: true
# MatrixRTC
msc4143_enabled: true
# Enable membership look-up via /is_joined.
msc4502_enabled: true
# Enable C-S & S-S request proxying for application services.
msc4512_enabled: true
app_service_config_files:
- /data/cfg/app-service.yaml
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no
+20
View File
@@ -0,0 +1,20 @@
# Coverage comes from a single vitest run (see .github/workflows/test.yaml), so
# the app and the SDK are split by path into codecov components rather than
# uploaded under separate flags. Each component gets its own coverage figure in
# the PR comment and on codecov.io.
#
# Validate with: curl --data-binary @codecov.yml https://codecov.io/validate
comment:
layout: "reach, diff, flags, components, files"
component_management:
individual_components:
- component_id: app
name: app
paths:
- src/**
- component_id: sdk
name: sdk
paths:
- sdk/**
+57
View File
@@ -0,0 +1,57 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "vitest";
import { render, screen } from "@testing-library/react";
import { type ReactNode } from "react";
import { Tooltip, TooltipProvider } from "@vector-im/compound-web";
import { CallPortalRoot } from "./CallPortalRoot";
// A button whose accessible name comes from its tooltip, as every icon button
// in the call does: the name is lost whenever the tooltip has nowhere to go.
const LabelledButton = (): ReactNode => (
<TooltipProvider>
<Tooltip label="Microphone">
<button>🎤</button>
</Tooltip>
</TooltipProvider>
);
test("floats into the host document with no root", () => {
render(
<CallPortalRoot root={undefined}>
<LabelledButton />
</CallPortalRoot>,
);
expect(screen.getByRole("button", { name: "Microphone" })).toBeTruthy();
});
test("floats into the root it is given", () => {
const root = document.createElement("div");
document.body.appendChild(root);
render(
<CallPortalRoot root={root}>
<LabelledButton />
</CallPortalRoot>,
);
const button = screen.getByRole("button", { name: "Microphone" });
const label = document.getElementById(
button.getAttribute("aria-labelledby")!,
);
expect(root.contains(label)).toBe(true);
root.remove();
});
test("waits for a root that is not available yet", () => {
render(
<CallPortalRoot root={null}>
<LabelledButton />
</CallPortalRoot>,
);
expect(screen.queryByRole("button", { name: "Microphone" })).toBeNull();
});
+26
View File
@@ -0,0 +1,26 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC, type ReactNode } from "react";
import { PortalRoot } from "@vector-im/compound-web";
interface Props {
root: HTMLElement | null | undefined;
children: ReactNode;
}
/**
* Where Compound floats the call's menus and tooltips. With no `root` the call
* is not wrapped at all and they go into the host document's body; `null` is
* Compound's "not available yet", and the floating parts wait for it.
*/
export const CallPortalRoot: FC<Props> = ({ root, children }): ReactNode =>
root === undefined ? (
children
) : (
<PortalRoot root={root}>{children}</PortalRoot>
);
+51
View File
@@ -0,0 +1,51 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/* The container a host mounts us into. It fills whatever space the host gives
it, and nothing we draw may leave it.
That takes two separate things, which are easy to mistake for one. `isolation`
gives us a stacking context, so nothing inside can be layered above the host's
own interface. Containment makes us the containing block for `position: fixed`
descendants, and clips what we paint to our own box: without it, the modal
scrim and dialog — which are positioned `fixed` and centred, since in the app
they are meant to cover the page — resolve against the viewport and appear in
the middle of the host's window rather than in the middle of the call.
The clipping cuts both ways: a menu near the edge of a small container is
trimmed rather than overflowing into the host. That is the trade being a
component rather than a page makes. */
.root {
display: flex;
flex-direction: column;
inline-size: 100%;
block-size: 100%;
isolation: isolate;
contain: layout paint;
position: relative;
background-color: var(--cpd-color-bg-canvas-default);
color: var(--cpd-color-text-primary);
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
-webkit-tap-highlight-color: transparent;
}
/* The call itself. It is a stacking context of its own, as the standalone
page's `#root` is, so that the layers the call view uses for its header, footer
and overlays stay below the modals and toasts, which are portalled into the
root and carry no layer of their own.
What Compound floats — tooltips, menus — is not in here at all: it goes into
the host's body, as it would for any other component on the host's page, and is
styled and layered by the host's copy of Compound. */
.content {
display: flex;
flex-direction: column;
block-size: 100%;
position: relative;
isolation: isolate;
}
+138
View File
@@ -0,0 +1,138 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* EXPERIMENTAL
*
* What a host needs in order to talk about Element Call, without Element Call:
* the component's props and the types they are made of, the handle and the
* host bridge, the intents a call can be started with and the configuration
* each implies.
*
* This is an entry point of its own, `@element-hq/element-call-component/api`,
* so that a host can import it on a path that must stay light — a call model
* that needs `BackgroundStyle.Solid` as a value, say — and load the component
* itself, and everything it brings, only when a call is shown. The main entry
* point re-exports all of it, so a host that does not care may import
* everything from there.
*
* Nothing in here may reach anything that is not a type or a constant: the
* point of this module is what it does not import.
*/
import type { Ref } from "react";
import type { MatrixClient } from "matrix-js-sdk";
import type { UrlProperties } from "../src/UrlParams";
import type { ElementCallHandle, ElementCallHostBridge } from "./host";
import {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
} from "../src/UrlConfiguration";
// How the host and Element Call talk to each other, and what they say
export type { ElementCallHandle, ElementCallHostBridge } from "./host";
export type { DeviceMuteRequest, DeviceMuteState } from "../src/HostBridge";
export type { JoinCallData } from "../src/widget";
// The deployment-wide configuration, as distinct from ElementCallConfiguration
// below, which is per call
export type { ConfigOptions } from "../src/config/ConfigOptions";
// The values that appear in ElementCallConfiguration and in the intent, and
// what each intent means by default
export {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
};
/**
* How Element Call should behave. Everything is optional; anything left out
* takes the default that {@link ElementCallProps.intent} implies.
*
* This is the behaviour a widget can be configured with through its URL, plus
* the one fact about the call a host has a say in here, the background. The
* rest of what a widget's URL carries — who the user is, how to reach the
* homeserver, where to report analytics, the shared secret of a room that is
* encrypted with one — a component host supplies by other routes, or not at
* all; and what can change while the call is running, the theme and the
* language, is a prop of its own.
*/
export type ElementCallConfiguration = Partial<UrlConfiguration> &
Partial<Pick<UrlProperties, "background">>;
export interface ElementCallProps {
/**
* The client to place the call with. Element Call does not authenticate
* anyone or manage a session of its own; this one is the host's.
*/
client: MatrixClient;
/** The room to call in. The host's client must already know about it. */
roomId: string;
/**
* What the user asked for — whether they started the call or joined one that
* was already running, and whether it is a call in a group or a DM. Element
* Call decides what each of those means: whether to show the lobby first,
* whether to ring, and so on.
*
* Defaults to joining an existing group call, which is the most conservative
* reading, but a host that knows which button the user pressed should say so.
*/
intent?: UserIntent;
/**
* How Element Call should behave, overriding whatever {@link intent} implies.
* A host that finds itself setting a lot of these probably wants a different
* intent instead.
*
* Compared by value, so it is fine to write this inline; only a change to
* what it says restarts anything.
*/
config?: ElementCallConfiguration;
/**
* What Element Call tells the host while the call is running: that the user
* has joined or hung up, that it would like to be kept on screen, and so on.
* Without one, Element Call assumes nobody is listening.
*/
hostBridge?: ElementCallHostBridge;
/**
* What the host tells Element Call: to hang up, to mute, to join. Available
* once the component has rendered.
*/
ref?: Ref<ElementCallHandle>;
/**
* The theme to show Element Call in, `light` or `dark`. Left out, Element
* Call picks. Changes take effect at once, and cost nothing else.
*/
theme?: string;
/**
* The language to show Element Call in, as a BCP 47 tag: one of the
* `supportedLanguages` the main entry point exports, or something that falls
* back to one (`de-AT` to `de`). Left out, the browser's language is used.
*
* Translations are one thing shared by every Element Call on the page, so
* the most recently set language wins for all of them.
*/
language?: string;
/**
* Where the parts of Element Call that float above it — its menus and
* tooltips, which Compound portals out of the call — are rendered. Left out,
* they go into the host document's body, as they do for any other component
* on the host's page.
*
* A host that moves Element Call's DOM into another document, such as a
* Document Picture-in-Picture window, passes that document's body here so
* that a menu opens in the window its button is in. Changes take effect at
* once. `null` means the root is not available yet, as from a callback ref
* that has not fired: menus and tooltips wait for it rather than falling
* back to the body.
*/
portalRoot?: HTMLElement | null;
}
+112
View File
@@ -0,0 +1,112 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it } from "vitest";
import postcss from "postcss";
import { ROOT_SELECTOR, scopeStylesToRoot } from "./scopeStylesToRoot";
const inRoot = `:where(${ROOT_SELECTOR}, ${ROOT_SELECTOR} *)`;
const isRoot = `:where(${ROOT_SELECTOR})`;
async function scope(css: string, file = "base.css"): Promise<string> {
const result = await postcss([scopeStylesToRoot()]).process(css, {
from: file,
});
return result.css;
}
describe("scopeStylesToRoot", () => {
it("makes the root stand in for the document", async () => {
expect(await scope("html { line-height: 1.15 }")).toBe(
`${isRoot} { line-height: 1.15 }`,
);
expect(await scope("body { margin: 0 }")).toBe(`${isRoot} { margin: 0 }`);
expect(await scope(":root { --a: 1 }")).toBe(`${isRoot} { --a: 1 }`);
expect(await scope("body.no-scroll-body { position: fixed }")).toBe(
`${isRoot}.no-scroll-body { position: fixed }`,
);
expect(await scope("body .x { color: red }")).toBe(
`${isRoot} .x { color: red }`,
);
});
it("collapses selectors that all became the root", async () => {
expect(await scope("html, body, input { font: inherit }")).toBe(
`${isRoot},input${inRoot} { font: inherit }`,
);
});
it("confines everything else to the root and what is inside it", async () => {
expect(await scope("h1 { margin: 0 }")).toBe(`h1${inRoot} { margin: 0 }`);
expect(await scope(".cpd-theme-dark { --a: 1 }")).toBe(
`.cpd-theme-dark${inRoot} { --a: 1 }`,
);
expect(await scope("* { box-sizing: border-box }")).toBe(
`*${inRoot} { box-sizing: border-box }`,
);
expect(await scope(".a > .b + .c { color: red }")).toBe(
`.a>.b+.c${inRoot} { color: red }`,
);
});
it("keeps pseudo-elements last", async () => {
expect(await scope("button::-moz-focus-inner { border: 0 }")).toBe(
`button${inRoot}::-moz-focus-inner { border: 0 }`,
);
expect(await scope(".a .b:hover::after { content: '' }")).toBe(
`.a .b:hover${inRoot}::after { content: '' }`,
);
expect(await scope("p:first-letter { color: red }")).toBe(
`p${inRoot}:first-letter { color: red }`,
);
});
it("leaves alone what already names the root", async () => {
const css = `${ROOT_SELECTOR}[data-platform="ios"] { --a: 1 }`;
expect(await scope(css)).toBe(css);
});
it("reaches into layers and media queries", async () => {
expect(
await scope(
"@layer normalize { h1 { margin: 0 } } @media (min-width: 1px) { p { margin: 0 } }",
),
).toBe(
`@layer normalize { h1${inRoot} { margin: 0 } } @media (min-width: 1px) { p${inRoot} { margin: 0 } }`,
);
});
it("does not touch keyframes or nested rules", async () => {
expect(
await scope("@keyframes spin { from { opacity: 0 } to { opacity: 1 } }"),
).toBe("@keyframes spin { from { opacity: 0 } to { opacity: 1 } }");
expect(
await scope(
".a { color: red; &:hover { color: blue } .b { color: green } }",
),
).toBe(
`.a${inRoot} { color: red; &:hover { color: blue } .b { color: green } }`,
);
});
it("only touches the bare selectors of a CSS module", async () => {
const file = "Settings.module.css";
expect(await scope("pre { font-size: 1px }", file)).toBe(
`pre${inRoot} { font-size: 1px }`,
);
expect(await scope(".modal pre { font-size: 1px }", file)).toBe(
`.modal pre${inRoot} { font-size: 1px }`,
);
expect(await scope(".box_abc12 { border: 0 }", file)).toBe(
".box_abc12 { border: 0 }",
);
expect(await scope(".a .b_abc12:hover { border: 0 }", file)).toBe(
".a .b_abc12:hover { border: 0 }",
);
});
});
+162
View File
@@ -0,0 +1,162 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type AtRule,
type Container,
type Document,
type Plugin,
type Rule,
} from "postcss";
import selectorParser, {
type Node,
type Pseudo,
type Selector,
} from "postcss-selector-parser";
/**
* How the stylesheets find Element Call's root element. The attribute is put
* there by `useTheme`, on the container the host gives the component.
*/
export const ROOT_SELECTOR = "[data-element-call-root]";
// Both are `:where()`, which has no specificity of its own, so the rules keep
// exactly the weight they had before being scoped and nothing in Element Call's
// cascade changes — only where it applies.
//
// The root, or anything inside it. Appended to the element a rule is about,
// rather than prepended to the whole selector, so that a rule about the root
// itself (its theme class, say) still matches.
const IN_ROOT = `:where(${ROOT_SELECTOR}, ${ROOT_SELECTOR} *)`;
// The root itself, standing in for the document.
const IS_ROOT = `:where(${ROOT_SELECTOR})`;
/**
* Confines a stylesheet to Element Call's root element, for the build of
* Element Call as a component.
*
* As a page of its own, Element Call can style the document: normalize.css and
* its own base styles speak of `html`, `body` and bare elements, and its custom
* properties are declared on `:root`. As a component, all of that would land on
* the host's document too. This rewrites every selector so that it matches only
* the root or its descendants:
*
* - `html`, `body` and `:root` become the root element, which is what stands in
* for the document inside a host.
* - Everything else keeps its selector and gains `:where([data-element-call-root],
* [data-element-call-root] *)` on the element it styles.
* - Selectors that already name the root are left alone, as are keyframe
* selectors and rules nested inside another rule, which are relative to it.
*
* CSS modules are scoped by their class names already, so only their selectors
* that would match by element alone — `pre` rather than `.pre` — are touched.
*
* The root's custom properties are still inherited by everything inside it,
* the way they were from `:root`. Compound's own stylesheet — the design
* tokens, the component styles, the fonts — is not in this build at all: the
* host supplies it, unscoped, along with its copy of Compound.
*/
export function scopeStylesToRoot(): Plugin {
return {
postcssPlugin: "element-call-scope-styles-to-root",
Once(root) {
const isModule =
root.source?.input.file?.endsWith(".module.css") ?? false;
root.walkRules((rule) => {
if (isRelative(rule)) return;
rule.selector = (isModule ? scopeBare : scopeAll).processSync(
rule.selector,
{ lossless: false },
);
});
},
};
}
/** Whether a rule's selectors are relative to something other than the document. */
function isRelative(rule: Rule): boolean {
let parent: Container | Document | undefined = rule.parent;
while (parent !== undefined) {
if (parent.type === "rule") return true;
if (parent.type === "atrule") {
const { name } = parent as AtRule;
if (name.endsWith("keyframes") || name === "page") return true;
}
parent = parent.parent;
}
return false;
}
const processor = (isModule: boolean): ReturnType<typeof selectorParser> =>
selectorParser((selectors) => {
selectors.each((selector) => {
scopeSelector(selector, isModule);
});
// Mapping `html, body` onto the root leaves the same selector twice
const seen = new Set<string>();
selectors.each((selector) => {
const text = String(selector).trim();
if (seen.has(text)) selector.remove();
else seen.add(text);
});
});
// Everything, for stylesheets that speak of the document; only what a class
// does not already confine, for CSS modules
const scopeAll = processor(false);
const scopeBare = processor(true);
function scopeSelector(selector: Selector, isModule: boolean): void {
if (String(selector).includes(ROOT_SELECTOR)) return;
const compounds = splitCompounds(selector);
if (compounds.length === 0) return;
// Something said of the document is said of the root instead
const document = compounds[0].find(isDocumentSelector);
if (document !== undefined) {
document.replaceWith(pseudo(IS_ROOT));
return;
}
const subject = compounds.at(-1)!;
if (isModule && subject.some((node) => node.type === "class")) return;
// Pseudo-elements have to come last in a compound selector
const pseudoElement = subject.find(isPseudoElement);
if (pseudoElement === undefined) selector.append(pseudo(IN_ROOT));
else selector.insertBefore(pseudoElement, pseudo(IN_ROOT));
}
/** The compound selectors making up a complex selector, in order. */
function splitCompounds(selector: Selector): Node[][] {
const compounds: Node[][] = [[]];
for (const node of selector.nodes) {
if (node.type === "combinator") compounds.push([]);
else if (node.type !== "comment") compounds.at(-1)!.push(node);
}
return compounds.filter((compound) => compound.length > 0);
}
function isDocumentSelector(node: Node): boolean {
return (
(node.type === "tag" && (node.value === "html" || node.value === "body")) ||
(node.type === "pseudo" && node.value === ":root")
);
}
function isPseudoElement(node: Node): node is Pseudo {
if (node.type !== "pseudo") return false;
return (
node.value.startsWith("::") ||
[":before", ":after", ":first-line", ":first-letter"].includes(node.value)
);
}
function pseudo(text: string): Pseudo {
return selectorParser().astSync(text).nodes[0].nodes[0].clone() as Pseudo;
}
+46
View File
@@ -0,0 +1,46 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type ElementCallHostBridge } from "../index";
/**
* A host bridge that reports everything it is told, so that the harness can
* watch what Element Call says to its host. (What the host says to Element
* Call goes through the component's handle, and is logged by the pane.)
*/
export function createDevHostBridge(
log: (message: string) => void,
/** What the host does when Element Call asks to be closed. */
onClose: () => void,
): ElementCallHostBridge {
/**
* Records something Element Call told the host. Nothing is sent anywhere, so
* this is only asynchronous because a real host's answer would have to be.
*/
const told = async (message: string): Promise<void> => {
log(`→ ${message}`);
await Promise.resolve();
};
return {
setAlwaysOnScreen: async (alwaysOnScreen): Promise<void> =>
await told(`setAlwaysOnScreen(${alwaysOnScreen})`),
contentLoaded: async (): Promise<void> => await told("contentLoaded"),
notifyJoined: async (): Promise<void> => await told("notifyJoined"),
notifyHungUp: async (): Promise<void> => await told("notifyHungUp"),
notifyDeviceMute: async (state): Promise<void> =>
await told(
`notifyDeviceMute(audio: ${state.audio_enabled}, video: ${state.video_enabled})`,
),
// Present because this host really can dismiss Element Call, which is what
// makes it offer a close affordance at all
close: async (): Promise<void> => {
await told("close");
onClose();
},
};
}
+140
View File
@@ -0,0 +1,140 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
.credentials {
display: flex;
flex-direction: column;
gap: 8px;
max-inline-size: 420px;
margin: 48px auto;
padding: 24px;
background-color: #ffffff;
border: 1px solid #d4d4d8;
border-radius: 8px;
}
.field {
display: flex;
flex-direction: column;
gap: 4px;
}
.error {
color: #b91c1c;
}
.harness {
display: grid;
grid-template-rows: auto 1fr auto;
block-size: 100%;
}
.header,
.paneBar {
display: flex;
align-items: center;
gap: 12px;
padding: 8px 12px;
border-block-end: 1px solid #d4d4d8;
background-color: #ffffff;
}
.header h1 {
font-size: 16px;
margin: 0;
}
.middle {
display: flex;
min-block-size: 0;
}
.sidebar {
flex: 0 0 220px;
padding: 12px;
border-inline-end: 1px solid #d4d4d8;
background-color: #ffffff;
overflow-y: auto;
}
.panes {
display: flex;
flex-wrap: wrap;
/* So that a pane is the size it was dragged to, rather than being stretched
to fill the row */
align-items: flex-start;
align-content: flex-start;
gap: 16px;
padding: 16px;
flex: 1;
min-inline-size: 0;
overflow: auto;
}
.pane {
display: flex;
flex-direction: column;
border: 1px solid #d4d4d8;
border-radius: 8px;
overflow: hidden;
background-color: #ffffff;
}
.paneBar {
flex-wrap: wrap;
gap: 6px;
border-block-end: none;
}
/* The space the host gives Element Call. Resizable so that the sizes it has to
cope with can be found by dragging rather than by rebuilding, and `overflow:
hidden` both to enable the resize handle and to show up anything inside Element
Call that does not fit the box it was given. */
.paneCall {
inline-size: 560px;
block-size: 420px;
min-inline-size: 180px;
min-block-size: 180px;
resize: both;
overflow: hidden;
}
.log {
max-block-size: 180px;
overflow-y: auto;
padding: 8px 12px;
border-block-start: 1px solid #d4d4d8;
background-color: #ffffff;
font-size: 12px;
}
.log h2 {
font-size: 13px;
margin: 0 0 4px;
}
.log ol {
margin: 0;
padding: 0;
list-style: none;
}
/* A host overlay, which Element Call must not be able to draw over */
.dialogScrim {
position: fixed;
inset: 0;
display: grid;
place-items: center;
background-color: rgb(0 0 0 / 50%);
z-index: 10;
}
.dialog {
padding: 24px;
border-radius: 8px;
background-color: #ffffff;
}
+380
View File
@@ -0,0 +1,380 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type FC,
type FormEvent,
type ReactNode,
useCallback,
useMemo,
useRef,
useState,
} from "react";
import { type MatrixClient } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import {
ElementCall,
type ElementCallHandle,
supportedLanguages,
} from "../index";
import { createDevHostBridge } from "./DevHostBridge";
import { createSession, joinRoom } from "./session";
import styles from "./Harness.module.css";
interface Credentials {
homeserver: string;
username: string;
password: string;
room: string;
}
const CREDENTIALS_KEY = "element-call-component-harness";
const DEFAULT_CREDENTIALS: Credentials = {
homeserver: "https://synapse.m.localhost",
username: "",
password: "",
room: "",
};
/**
* The credentials to start with: the last ones used, so that a reload does not
* mean typing them again, overridden by anything in the query string.
*
* A host reading its own URL is entirely proper — it was Element Call doing so
* that was the mistake. It lets the end-to-end tests, or a shared link, say
* which account and room to use.
*/
function loadCredentials(): Credentials {
let stored: Partial<Credentials> = {};
try {
const json = localStorage.getItem(CREDENTIALS_KEY);
if (json !== null) stored = JSON.parse(json) as Credentials;
} catch (e) {
logger.warn("Could not read the stored harness credentials", e);
}
const query = new URLSearchParams(location.search);
const fromUrl = Object.fromEntries(
(["homeserver", "username", "password", "room"] as const)
.map((name) => [name, query.get(name)])
.filter(([, value]) => value !== null),
) as Partial<Credentials>;
return { ...DEFAULT_CREDENTIALS, ...stored, ...fromUrl };
}
interface Session {
label: string;
client: MatrixClient;
}
type State =
| { phase: "credentials" }
| { phase: "starting"; progress: string }
| { phase: "started"; roomId: string; sessions: Session[] }
| { phase: "failed"; error: string };
interface LogEntry {
pane: string;
message: string;
at: string;
}
/**
* One Element Call component, with the controls a host would have over it: the
* requests it can make of Element Call, and the ability to take it off screen
* altogether.
*/
const Pane: FC<{
session: Session;
roomId: string;
theme: string | undefined;
language: string | undefined;
log: (pane: string, message: string) => void;
}> = ({ session, roomId, theme, language, log }): ReactNode => {
const [mounted, setMounted] = useState(true);
const bridge = useMemo(
() =>
createDevHostBridge(
(message) => log(session.label, message),
() => setMounted(false),
),
[log, session.label],
);
// What the host asks of Element Call goes through the component's handle.
// Worth saying out loud when a request is refused — asking to hang up when
// there is no call, say — since that is the sort of thing the harness is for.
const handle = useRef<ElementCallHandle>(null);
const ask = (
name: string,
make: (handle: ElementCallHandle) => Promise<unknown>,
): void => {
if (handle.current === null) {
log(session.label, `← ${name}: not mounted`);
return;
}
log(session.label, `← ${name}`);
make(handle.current).then(
(reply) =>
log(
session.label,
`→ ${name} acknowledged${reply === undefined ? "" : `: ${JSON.stringify(reply)}`}`,
),
(e: unknown) => log(session.label, `→ ${name} refused: ${e}`),
);
};
return (
<section className={styles.pane} data-testid="call-pane">
<div className={styles.paneBar}>
<strong>{session.label}</strong>
<code>{session.client.getDeviceId()}</code>
<button onClick={(): void => setMounted((m) => !m)}>
{mounted ? "Unmount" : "Mount"}
</button>
<button
onClick={(): void =>
ask(
"setDeviceMute(audio: false)",
async (h) => await h.setDeviceMute({ audio_enabled: false }),
)
}
>
Mute
</button>
<button
onClick={(): void => ask("hangUp", async (h) => await h.hangUp())}
>
Hang up
</button>
</div>
{/* Resizable, because how Element Call copes with the size it is given is
one of the things we cannot find out from the standalone app */}
<div className={styles.paneCall} data-testid="call-container">
{mounted && (
<ElementCall
ref={handle}
client={session.client}
roomId={roomId}
hostBridge={bridge}
theme={theme}
language={language}
/>
)}
</div>
</section>
);
};
/** Host furniture, to make it visible if Element Call styles anything but itself. */
const HostChrome: FC = (): ReactNode => (
<nav className={styles.sidebar}>
<h2>Host chrome</h2>
<p>
This column belongs to the host. If Element Call&apos;s stylesheet reaches
outside its own container, it shows up here.
</p>
<hr />
<ul>
<li>Some room</li>
<li>Another room</li>
</ul>
<button>A host button</button>
</nav>
);
/**
* A dialog of the host's own, over the top of the calls. Element Call as a
* component has to sit underneath this — being unable to is one of the reasons
* for a component rather than an iframe.
*/
const HostDialog: FC<{ onClose: () => void }> = ({ onClose }): ReactNode => (
<div className={styles.dialogScrim}>
<div className={styles.dialog}>
<h2>A dialog belonging to the host</h2>
<p>This should cover the calls completely.</p>
<button onClick={onClose}>Close</button>
</div>
</div>
);
/**
* Stands in for a host application using the Element Call component: it owns the Matrix
* clients, the page and the space each call is given, and reaches Element Call
* only through the component's public interface.
*
* Two calls at once, from two devices of the same account, so that a real call
* happens between them and anything Element Call keeps once per process rather
* than once per call shows itself.
*/
export const Harness: FC = (): ReactNode => {
const [credentials, setCredentials] = useState(loadCredentials);
const [state, setState] = useState<State>({ phase: "credentials" });
const [entries, setEntries] = useState<LogEntry[]>([]);
const [dialogOpen, setDialogOpen] = useState(false);
// The host's language setting, which Element Call follows. Undefined means
// the host has none and Element Call uses the browser's.
const [language, setLanguage] = useState<string | undefined>(undefined);
const [theme, setTheme] = useState<string | undefined>(undefined);
const log = useCallback((pane: string, message: string): void => {
setEntries((entries) =>
[
...entries,
{ pane, message, at: new Date().toLocaleTimeString() },
].slice(-100),
);
}, []);
const start = useCallback(
(event: FormEvent): void => {
event.preventDefault();
localStorage.setItem(CREDENTIALS_KEY, JSON.stringify(credentials));
const { homeserver, username, password, room } = credentials;
const progress = (message: string): void =>
setState({ phase: "starting", progress: message });
progress("Starting");
void (async (): Promise<void> => {
try {
// One at a time: two logins at once from the same account is the
// shape of request homeservers rate limit
const sessions: Session[] = [];
for (const label of ["Call A", "Call B"])
sessions.push({
label,
client: await createSession(
homeserver,
username,
password,
(message) => progress(`${label}: ${message}`),
),
});
progress("Joining the room");
let roomId = room;
for (const { client } of sessions)
roomId = await joinRoom(client, roomId);
setState({ phase: "started", roomId, sessions });
} catch (e) {
logger.error("The harness could not start", e);
setState({ phase: "failed", error: `${e}` });
}
})();
},
[credentials],
);
const field = (
name: keyof Credentials,
label: string,
type = "text",
): ReactNode => (
<label className={styles.field}>
{label}
<input
type={type}
value={credentials[name]}
onChange={(e): void =>
setCredentials((c) => ({ ...c, [name]: e.target.value }))
}
/>
</label>
);
if (state.phase !== "started")
return (
<form className={styles.credentials} onSubmit={start}>
<h1>Element Call component harness</h1>
<p>
Signs in twice and shows the Element Call component twice, in a page
that is not Element Call&apos;s own.
</p>
{field("homeserver", "Homeserver")}
{field("username", "Username")}
{field("password", "Password", "password")}
{field("room", "Room ID or alias")}
<button type="submit" disabled={state.phase === "starting"}>
Start
</button>
{state.phase === "starting" && <p>{state.progress}</p>}
{state.phase === "failed" && (
<p className={styles.error}>{state.error}</p>
)}
</form>
);
return (
<div className={styles.harness}>
<header className={styles.header}>
<h1>Element Call component harness</h1>
<code>{state.roomId}</code>
<button onClick={(): void => setDialogOpen(true)}>
Open a host dialog
</button>
<label>
Theme{" "}
<select
value={theme ?? ""}
onChange={(e): void => setTheme(e.target.value || undefined)}
>
<option value="">Element Call&apos;s choice</option>
<option value="light">light</option>
<option value="dark">dark</option>
</select>
</label>
<label>
Language{" "}
<select
value={language ?? ""}
onChange={(e): void => setLanguage(e.target.value || undefined)}
>
<option value="">Browser default</option>
{supportedLanguages.map((tag) => (
<option key={tag} value={tag}>
{tag}
</option>
))}
</select>
</label>
</header>
<div className={styles.middle}>
<HostChrome />
<main className={styles.panes}>
{state.sessions.map((session) => (
<Pane
key={session.label}
session={session}
roomId={state.roomId}
theme={theme}
language={language}
log={log}
/>
))}
</main>
</div>
<section className={styles.log} data-testid="bridge-log">
<h2>Host bridge</h2>
<ol>
{entries.map((entry, i) => (
<li key={i}>
<code>{entry.at}</code> <strong>{entry.pane}</strong>{" "}
{entry.message}
</li>
))}
</ol>
</section>
{dialogOpen && <HostDialog onClose={(): void => setDialogOpen(false)} />}
</div>
);
};
+23
View File
@@ -0,0 +1,23 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/* The host page's own styles. Deliberately plain, and deliberately not using
Element Call's design tokens: the harness should look like it does because of
this file, not because Element Call styled it. */
html,
body,
#root {
height: 100%;
}
body {
margin: 0;
font-family: system-ui, sans-serif;
background-color: #f4f4f5;
color: #18181b;
}
+21
View File
@@ -0,0 +1,21 @@
<!doctype html>
<!--
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
-->
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Element Call component harness</title>
</head>
<body>
<!-- Deliberately plain: this page stands in for a host application, so
anything it looks like must have come from the host's own styles or from
Element Call reaching outside its container. -->
<div id="root"></div>
<script type="module" src="./main.tsx"></script>
</body>
</html>
+52
View File
@@ -0,0 +1,52 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import { logger } from "matrix-js-sdk/lib/logger";
// What a host provides along with its copy of Compound: its stylesheet, the
// design tokens and the fonts they name. The component does not bundle them.
import "@fontsource/inter/400.css";
import "@fontsource/inter/500.css";
import "@fontsource/inter/600.css";
import "@fontsource/inter/700.css";
import "@fontsource/inconsolata/400.css";
import "@fontsource/inconsolata/700.css";
import "@vector-im/compound-design-tokens/assets/web/css/compound-design-tokens.css";
import "@vector-im/compound-web/dist/style.css";
import { type ConfigOptions, initializeElementCall } from "../index";
import { Harness } from "./Harness";
// After Element Call's, so that the host has the last word on its own page
import "./host.css";
/**
* The development app's own `config.json`, so that the harness runs Element
* Call the way `pnpm dev` does. It is not in the repository — developers copy
* it from `config/config.devenv.json` — so its absence is expected rather than
* an error.
*/
async function loadConfig(): Promise<ConfigOptions> {
try {
const response = await fetch("/config.json");
if (response.ok) return (await response.json()) as ConfigOptions;
logger.warn(
`No config.json (${response.status}); running with Element Call's defaults`,
);
} catch (e) {
logger.warn("Could not read config.json", e);
}
return {};
}
await initializeElementCall(await loadConfig());
createRoot(document.getElementById("root")!).render(
<StrictMode>
<Harness />
</StrictMode>,
);
+79
View File
@@ -0,0 +1,79 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
ClientEvent,
createClient,
type MatrixClient,
MemoryStore,
SyncState,
} from "matrix-js-sdk";
/**
* Logs in and brings up a client the way a host application would, so that the
* component is handed a real one rather than something Element Call built for
* itself.
*
* Everything is kept in memory and a fresh login happens on every reload. That
* costs a device on the development homeserver each time, which is harmless,
* and buys the harness two clients that cannot tread on each other's storage.
* Persisting the login to make reloads quicker would mean persisting the
* crypto store too: reusing a device ID with a fresh crypto store generates new
* device keys, and uploading them conflicts with the ones the server already
* holds.
*/
export async function createSession(
homeserver: string,
username: string,
password: string,
onProgress: (message: string) => void,
): Promise<MatrixClient> {
onProgress("Logging in");
const login = await createClient({ baseUrl: homeserver }).login(
"m.login.password",
{ identifier: { type: "m.id.user", user: username }, password },
);
const client = createClient({
baseUrl: homeserver,
accessToken: login.access_token,
userId: login.user_id,
deviceId: login.device_id,
store: new MemoryStore(),
useAuthorizationHeader: true,
fallbackICEServerAllowed: true,
});
onProgress(`Setting up crypto for ${login.device_id}`);
await client.initRustCrypto({ useIndexedDB: false });
onProgress(`Syncing ${login.device_id}`);
await client.startClient();
await new Promise<void>((resolve) => {
const onSync = (state: SyncState): void => {
if (state !== SyncState.Prepared && state !== SyncState.Syncing) return;
client.off(ClientEvent.Sync, onSync);
resolve();
};
client.on(ClientEvent.Sync, onSync);
});
return client;
}
/**
* The room to call in, joining it if this session is not in it yet — a host
* hands Element Call a room it already knows about, so the harness has to get
* itself into that position first.
*/
export async function joinRoom(
client: MatrixClient,
roomIdOrAlias: string,
): Promise<string> {
const room = await client.joinRoom(roomIdOrAlias);
return room.roomId;
}
+161
View File
@@ -0,0 +1,161 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { renderHook } from "@testing-library/react";
import { createRef } from "react";
import { describe, expect, test, vi } from "vitest";
import {
type ElementCallHandle,
type ElementCallHostBridge,
useComponentHostBridge,
} from "./host";
describe("useComponentHostBridge", () => {
test("keeps one identity while the host supplies new objects", () => {
const { result, rerender } = renderHook(
({ supplied }: { supplied: ElementCallHostBridge }) =>
useComponentHostBridge(supplied, undefined, undefined),
{ initialProps: { supplied: {} } },
);
const first = result.current;
rerender({ supplied: { notifyJoined: async () => {} } });
expect(result.current).toBe(first);
});
test("forwards to whatever the host most recently supplied", async () => {
const before = vi.fn().mockResolvedValue(undefined);
const after = vi.fn().mockResolvedValue(undefined);
const { result, rerender } = renderHook(
({ supplied }: { supplied: ElementCallHostBridge }) =>
useComponentHostBridge(supplied, undefined, undefined),
{ initialProps: { supplied: { notifyJoined: before } } },
);
rerender({ supplied: { notifyJoined: after } });
await result.current.notifyJoined();
expect(before).not.toHaveBeenCalled();
expect(after).toHaveBeenCalledOnce();
});
test("is quiet about what the host did not implement", async () => {
const { result } = renderHook(() =>
useComponentHostBridge(undefined, undefined, undefined),
);
await expect(result.current.contentLoaded()).resolves.toBeUndefined();
await expect(
result.current.notifyDeviceMute({
audio_enabled: true,
video_enabled: false,
}),
).resolves.toBeUndefined();
expect(result.current.supportsReactions).toBe(true);
// Starting the user unmuted unasked is something a host has to opt into
expect(result.current.allowJoinUnmutedViaIntent).toBe(false);
});
test("lets the host allow joining unmuted on the intent", () => {
const { result, rerender } = renderHook(
({ supplied }: { supplied: ElementCallHostBridge }) =>
useComponentHostBridge(supplied, undefined, undefined),
{ initialProps: { supplied: {} } },
);
expect(result.current.allowJoinUnmutedViaIntent).toBe(false);
// Read through to whatever the host most recently said
rerender({ supplied: { allowJoinUnmutedViaIntent: true } });
expect(result.current.allowJoinUnmutedViaIntent).toBe(true);
});
test("only has a close when the host has one, since that is a signal", () => {
const { result, rerender } = renderHook(
({ supplied }: { supplied: ElementCallHostBridge }) =>
useComponentHostBridge(supplied, undefined, undefined),
{ initialProps: { supplied: {} } },
);
expect(result.current.close).toBeUndefined();
const close = vi.fn().mockResolvedValue(undefined);
rerender({ supplied: { close } });
expect(result.current.close).toBeDefined();
});
test("never offers profile changes, since the account is the host's", () => {
const { result } = renderHook(() =>
useComponentHostBridge(undefined, undefined, undefined),
);
expect(result.current.supportsProfileChanges).toBe(false);
});
describe("the handle", () => {
test("delivers a request to what is listening and resolves on its reply", async () => {
const ref = createRef<ElementCallHandle>();
const { result } = renderHook(() =>
useComponentHostBridge(undefined, ref, undefined),
);
const received = vi.fn();
result.current.deviceMute$.subscribe(({ data, reply }) => {
received(data);
reply({ audio_enabled: data.audio_enabled!, video_enabled: true });
});
await expect(
ref.current!.setDeviceMute({ audio_enabled: false }),
).resolves.toEqual({ audio_enabled: false, video_enabled: true });
expect(received).toHaveBeenCalledWith({ audio_enabled: false });
});
test("refuses a request nothing in Element Call is listening for", async () => {
const ref = createRef<ElementCallHandle>();
renderHook(() => useComponentHostBridge(undefined, ref, undefined));
await expect(ref.current!.hangUp()).rejects.toThrow(
"Nothing in Element Call can hang up right now",
);
});
});
describe("the theme", () => {
test("reaches a subscriber that arrives after it was set", () => {
const { result } = renderHook(() =>
useComponentHostBridge(undefined, undefined, "light"),
);
const names: (string | undefined)[] = [];
result.current.themeChange$.subscribe(({ data }) =>
names.push(data.name),
);
expect(names).toEqual(["light"]);
});
test("follows the prop", () => {
const { result, rerender } = renderHook(
({ theme }: { theme: string | undefined }) =>
useComponentHostBridge(undefined, undefined, theme),
{ initialProps: { theme: "light" } },
);
const names: (string | undefined)[] = [];
result.current.themeChange$.subscribe(({ data }) =>
names.push(data.name),
);
rerender({ theme: "dark" });
expect(names).toEqual(["light", "dark"]);
});
test("says nothing when the host leaves the theme to Element Call", () => {
const { result } = renderHook(() =>
useComponentHostBridge(undefined, undefined, undefined),
);
const names: (string | undefined)[] = [];
result.current.themeChange$.subscribe(({ data }) =>
names.push(data.name),
);
expect(names).toEqual([]);
});
});
});
+201
View File
@@ -0,0 +1,201 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* How a host application and the Element Call component talk to each other.
*
* Inside Element Call the host is a {@link HostBridge}, which carries the
* host's requests as rxjs observables because that is what the widget API and
* the view models work in. A host should not have to know about rxjs, or agree
* with us on a version of it, so a component host sees neither: it implements
* plain async callbacks for what Element Call tells it, and makes its own
* requests through an imperative handle on the component, the way it would
* call `play()` on a video element. This module adapts the one to the other.
*/
import { type Ref, useEffect, useImperativeHandle } from "react";
import { ReplaySubject, Subject } from "rxjs";
import {
type DeviceMuteRequest,
type DeviceMuteState,
type HostBridge,
type HostRequest,
} from "../src/HostBridge";
import { type JoinCallData } from "../src/widget";
import { useInitial } from "../src/useInitial";
import { useLatest } from "../src/useLatest";
/**
* What Element Call tells the application hosting it as a component.
* Everything is optional: a host implements what it wants to hear about.
*
* Compared by nothing — Element Call always calls whichever one it was most
* recently given, so this may be written inline.
*/
export interface ElementCallHostBridge {
/**
* Asks the host to keep Element Call on screen (or stop doing so), so that a
* call in progress is not torn down when the user navigates elsewhere.
*/
setAlwaysOnScreen?(alwaysOnScreen: boolean): Promise<void>;
/** Tells the host that Element Call has finished loading. */
contentLoaded?(): Promise<void>;
/** Tells the host that the user has joined the call. */
notifyJoined?(): Promise<void>;
/** Tells the host that the user has hung up. */
notifyHungUp?(): Promise<void>;
/** Tells the host the user's current audio and video mute state. */
notifyDeviceMute?(state: DeviceMuteState): Promise<void>;
/**
* Asks the host to close Element Call: to unmount the component. Its
* presence is what makes Element Call offer a close button on its error
* screens, and leave the host to decide what is shown once a call has ended.
* Without it, Element Call shows its own post-call screen, if it has one for
* the situation, or nothing.
*/
close?(): Promise<void>;
/**
* Whether Element Call may send and receive reactions in this room.
* Defaults to true.
*/
readonly supportsReactions?: boolean;
/**
* Whether the user may start unmuted when the intent skips the lobby, so
* that they never see their devices before joining. Defaults to false: the
* user starts muted and unmutes themselves. A host that chose the intent on
* the user's behalf, and is sure they expect to be heard and seen at once,
* says so here — as a Matrix client hosting Element Call as a widget does.
*/
readonly allowJoinUnmutedViaIntent?: boolean;
}
/**
* What a host can ask of a mounted Element Call, reached through the
* component's `ref`. Each request resolves once Element Call has acted on it,
* and rejects if nothing in Element Call is in a position to act: hanging up
* when there is no call, say.
*/
export interface ElementCallHandle {
/**
* Joins the call, when Element Call was configured to `preload` and is
* waiting to be told to. Says which devices to join with.
*/
join(devices: JoinCallData): Promise<void>;
/** Leaves the call. */
hangUp(): Promise<void>;
/**
* Changes the mute state, for whichever of audio and video is given, and
* reports the state that results.
*/
setDeviceMute(request: DeviceMuteRequest): Promise<DeviceMuteState>;
}
/** Hands a request to Element Call and waits for it to be acknowledged. */
async function request<Data, Reply>(
listeners: Subject<HostRequest<Data, Reply>>,
what: string,
data: Data,
): Promise<Reply> {
if (!listeners.observed)
throw new Error(`Nothing in Element Call can ${what} right now`);
return await new Promise((resolve) =>
listeners.next({ data, reply: resolve }),
);
}
/**
* The {@link HostBridge} the rest of Element Call sees, built from what a
* component host supplies and wired to the handle it is given.
*
* The bridge is created once and never changes identity — everything that
* depends on it would otherwise restart when the host re-rendered with a new
* object — and forwards each call to whatever the host most recently passed.
*/
export function useComponentHostBridge(
supplied: ElementCallHostBridge | undefined,
ref: Ref<ElementCallHandle> | undefined,
/** The theme the host wants, or undefined to leave it to Element Call. */
theme: string | undefined,
): HostBridge {
const latest = useLatest(supplied ?? {});
const requests = useInitial(() => ({
// The theme is state, not an event: a `theme` prop rather than a request
// on the handle. It travels this channel because that is how the rest of
// Element Call hears about a host's theme, and replays so that whatever
// subscribes after the host has set it — everything, on first render —
// still hears the current one.
themeChange$: new ReplaySubject<HostRequest<{ name?: string }>>(1),
join$: new Subject<HostRequest<JoinCallData>>(),
hangUp$: new Subject<HostRequest<Record<string, never>>>(),
deviceMute$: new Subject<HostRequest<DeviceMuteRequest, DeviceMuteState>>(),
}));
useEffect(() => {
if (theme !== undefined)
requests.themeChange$.next({ data: { name: theme }, reply: () => {} });
}, [requests, theme]);
const bridge = useInitial((): HostBridge => ({
setAlwaysOnScreen: async (alwaysOnScreen) => {
await latest.current.setAlwaysOnScreen?.(alwaysOnScreen);
},
contentLoaded: async () => {
await latest.current.contentLoaded?.();
},
notifyJoined: async () => {
await latest.current.notifyJoined?.();
},
notifyHungUp: async () => {
await latest.current.notifyHungUp?.();
},
notifyDeviceMute: async (state) => {
await latest.current.notifyDeviceMute?.(state);
},
// Whether these exist is itself information, so they are read through
// rather than wrapped unconditionally
get close() {
const close = latest.current.close;
return close === undefined
? undefined
: async (): Promise<void> => await close();
},
// Not offered to a component host: the client it hands over holds the
// credentials to fetch media itself. A widget's client does not, which
// is what the internal bridge's `downloadMedia` is for.
get supportsReactions(): boolean {
return latest.current.supportsReactions ?? true;
},
get allowJoinUnmutedViaIntent(): boolean {
return latest.current.allowJoinUnmutedViaIntent ?? false;
},
// Whatever the host says or does not say, the account is its own: it
// signed the user in and handed us the client. So Element Call never
// offers to edit the profile from inside a component.
supportsProfileChanges: false,
...requests,
}));
useImperativeHandle(
ref,
(): ElementCallHandle => ({
join: async (devices) =>
await request(requests.join$, "join a call", devices),
hangUp: async () => await request(requests.hangUp$, "hang up", {}),
setDeviceMute: async (muteRequest) =>
await request(
requests.deviceMute$,
"change the mute state",
muteRequest,
),
}),
[requests],
);
return bridge;
}
+282
View File
@@ -0,0 +1,282 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* EXPERIMENTAL
*
* Element Call as a React component, for an application that wants to show a
* call inside itself rather than in an iframe.
*
* The host supplies the client and says which room to call in; Element Call
* supplies the call. Everything it would otherwise take from the page it is on
* — the URL, the document body, a Matrix session of its own — comes from the
* host instead, or is confined to the container it is mounted in.
*/
// The element defaults and custom properties every Element Call stylesheet
// builds on. Written for a page, they speak of `html`, `body` and bare
// elements; the component build confines them, and every other stylesheet in
// this bundle, to the root element below (see build/scopeStylesToRoot.ts), so
// that the host's document is left as it was.
//
// Compound is not in here. The component uses the host's copy of it (a peer
// dependency, left external by the build), and the host loads Compound's
// stylesheet, design tokens and fonts along with it: what Compound floats into
// the host's body — tooltips, menus — is then styled by the same stylesheet as
// everything else there, and Element Call's root takes the tokens and fonts of
// the theme class it carries, as any element on the host's page would.
//
// Where these land relative to the component stylesheets is the bundler's
// choice — the standalone app puts them first, this build puts them in the
// middle — so nothing in base.css may depend on winning or losing against a
// component's own rules at equal specificity. It currently does not: what it
// declares unlayered is custom properties on Element Call's root, which
// components inherit rather than compete with.
import "../src/base.css";
import {
type FC,
type JSX,
type ReactNode,
useEffect,
useLayoutEffect,
useMemo,
useRef,
useState,
} from "react";
import { logger } from "matrix-js-sdk/lib/logger";
import { I18nextProvider } from "react-i18next";
import { TooltipProvider } from "@vector-im/compound-web";
import { ErrorBoundary } from "@sentry/react";
import { shouldPolyfill as shouldPolyfillSegmenter } from "@formatjs/intl-segmenter/should-polyfill";
import { shouldPolyfill as shouldPolyfillDurationFormat } from "@formatjs/intl-durationformat/should-polyfill.js";
import LanguageDetector from "i18next-browser-languagedetector";
import EN from "../locales/en/app.json";
import { CallView } from "../src/room/CallView";
import { ErrorPage } from "../src/FullScreenView";
import { ClientProvider } from "../src/ClientContext";
import { HostBridgeProvider } from "../src/HostBridge";
import { RootElementProvider, useRootElement } from "../src/RootElementContext";
import {
configurationForIntent,
componentProperties,
type UrlParams,
UrlParamsProvider,
UserIntent,
useUrlParams,
} from "../src/UrlParams";
import { MediaDevicesContext } from "../src/MediaDevicesContext";
import { MediaDevices } from "../src/state/MediaDevices";
import { ObservableScope } from "../src/state/ObservableScope";
import { ProcessorProvider } from "../src/tracks/TrackProcessorContext";
import { Config } from "../src/config/Config";
import { type ConfigOptions } from "../src/config/ConfigOptions";
import { i18n } from "../src/utils/i18n";
import { useTheme } from "../src/useTheme";
import { useStableValue } from "../src/useStableValue";
import styles from "./ElementCall.module.css";
import { useComponentHostBridge } from "./host";
import { CallPortalRoot } from "./CallPortalRoot";
import { type ElementCallProps } from "./api";
import { supportedLanguages, translationsBackend } from "./localization";
// The languages Element Call can be shown in
export { supportedLanguages } from "./localization";
// Everything a host needs to talk about Element Call — the props, the handle,
// the host bridge, the intents and what each means — also available without
// the component itself from `@element-hq/element-call-component/api`
export * from "./api";
/**
* Prepares the things Element Call needs before it can be shown: translations,
* `Intl` polyfills for older browsers, and its configuration.
*
* Await this once, before rendering {@link ElementCall}.
*/
export async function initializeElementCall(
config: ConfigOptions = {},
): Promise<void> {
const polyfills: Promise<unknown>[] = [];
if (shouldPolyfillSegmenter())
polyfills.push(import("@formatjs/intl-segmenter/polyfill-force"));
if (shouldPolyfillDurationFormat())
polyfills.push(import("@formatjs/intl-durationformat/polyfill-force.js"));
await Promise.all(polyfills);
Config.initWith(config);
await i18n
.use(translationsBackend)
.use(new LanguageDetector())
.init({
fallbackLng: "en",
defaultNS: "app",
keySeparator: ".",
nsSeparator: false,
pluralSeparator: "_",
contextSeparator: "|",
supportedLngs: [...supportedLanguages],
interpolation: { escapeValue: false },
// English is bundled in, so the fallback never has to be loaded; every
// other language arrives from the backend when first asked for.
partialBundledLanguages: true,
resources: { en: { app: EN } },
detection: {
// The browser's language, until the host says otherwise through the
// `language` prop. Nothing is remembered: the choice is the host's.
order: ["navigator"],
caches: [],
},
});
}
/** Applies the theme and background to the container, before it is painted. */
const Decoration: FC<{ children: JSX.Element }> = ({ children }) => {
useTheme();
const { background } = useUrlParams();
const rootElement = useRootElement();
useLayoutEffect(() => {
rootElement.setAttribute("data-background", background);
}, [rootElement, background]);
return children;
};
export const ElementCall: FC<ElementCallProps> = ({
client,
roomId,
intent = UserIntent.JoinExistingCall,
config,
hostBridge: suppliedHostBridge,
ref,
theme,
language,
portalRoot,
}): ReactNode => {
const hostBridge = useComponentHostBridge(suppliedHostBridge, ref, theme);
useEffect(() => {
if (language !== undefined)
i18n
.changeLanguage(language)
.catch((e) => logger.error(`Could not switch to ${language}`, e));
}, [language]);
// The container is what Element Call decorates and portals into, so nothing
// inside can render until we have it.
const [container, setContainer] = useState<HTMLDivElement | null>(null);
// Element Call has no URL of its own to read any of this from, and the
// host's URL is not Element Call's business, so the defaults come from the
// intent with the host's wishes over the top.
//
// Everything downstream — the mute state, the call view model and with it
// the media connection — is keyed on the identity of this object, so it has
// to be stable for as long as its contents are. A host writing `config`
// inline would otherwise tear the call down on every render.
const stableConfig = useStableValue(config);
const params = useMemo(
(): UrlParams => ({
...componentProperties,
roomId,
...configurationForIntent(intent),
...stableConfig,
}),
[roomId, intent, stableConfig],
);
// Created in an effect so that the scope it lives in ends when the component
// is unmounted (or these options change), rather than keeping its device
// observers running for the rest of the page's life. Null until then, which
// is one render.
const { controlledAudioDevices, callIntent } = params;
const [mediaDevices, setMediaDevices] = useState<MediaDevices | null>(null);
useEffect(() => {
const scope = new ObservableScope();
setMediaDevices(
new MediaDevices(scope, { controlledAudioDevices, callIntent }),
);
return (): void => {
setMediaDevices(null);
scope.end();
};
}, [controlledAudioDevices, callIntent]);
const room = client.getRoom(roomId);
const rtcSession = useMemo(
() => (room === null ? null : client.matrixRTC.getRoomSession(room)),
[client, room],
);
if (rtcSession === null)
logger.error(
`Element Call was asked to call in ${roomId}, which its host's client does not know about`,
);
// Everything the call needs is in hand once these exist, and the first
// render with them is where the call itself appears: the moment the host
// is told that Element Call has loaded, as the widget tells its client once
// its own initialisation is over. Once per mount, however often the pieces
// are later swapped out.
const ready =
container !== null && rtcSession !== null && mediaDevices !== null;
const announcedLoaded = useRef(false);
useEffect(() => {
if (!ready || announcedLoaded.current) return;
announcedLoaded.current = true;
hostBridge
.contentLoaded()
.catch((e) => logger.error("Could not tell the host we had loaded", e));
}, [ready, hostBridge]);
return (
<I18nextProvider i18n={i18n}>
<HostBridgeProvider value={hostBridge}>
<UrlParamsProvider value={params}>
<div ref={setContainer} className={styles.root}>
{ready && (
// Stands in for the standalone page's `#root`: see `.content`
// in ElementCall.module.css
<div className={styles.content}>
<CallPortalRoot root={portalRoot}>
<RootElementProvider value={container}>
{/* Whatever goes wrong in here is shown in here. Left to
propagate, an error would unmount the host's own tree. */}
<ErrorBoundary
fallback={(error) => <ErrorPage error={error} />}
// A broken call should not hold the host on screen
onError={() => void hostBridge.setAlwaysOnScreen(false)}
>
<Decoration>
<TooltipProvider>
<ClientProvider client={client}>
<MediaDevicesContext value={mediaDevices}>
<ProcessorProvider>
<CallView
client={client}
rtcSession={rtcSession}
isPasswordlessUser={false}
confineToRoom={params.confineToRoom}
preload={params.preload}
skipLobby={params.skipLobby}
/>
</ProcessorProvider>
</MediaDevicesContext>
</ClientProvider>
</TooltipProvider>
</Decoration>
</ErrorBoundary>
</RootElementProvider>
</CallPortalRoot>
</div>
)}
</div>
</UrlParamsProvider>
</HostBridgeProvider>
</I18nextProvider>
);
};
+46
View File
@@ -0,0 +1,46 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, test } from "vitest";
import { supportedLanguages, translationsBackend } from "./localization";
const read = async (
language: string,
namespace = "app",
): Promise<Record<string, unknown>> =>
await new Promise((resolve, reject) =>
translationsBackend.read(language, namespace, (error, data) => {
if (error) reject(error);
else resolve(data as Record<string, unknown>);
}),
);
describe("component translations", () => {
test("offer every language in locales/, tagged as its directory is", () => {
expect(supportedLanguages).toContain("en");
expect(supportedLanguages).toContain("de");
expect(supportedLanguages).toContain("zh-Hans");
expect(new Set(supportedLanguages).size).toBe(supportedLanguages.length);
});
test("load a language's translations on demand", async () => {
const de = await read("de");
expect(de).toHaveProperty("action");
expect(de).not.toEqual(await read("en"));
});
test("refuse a language there are no translations for", async () => {
await expect(read("xx")).rejects.toThrow("No app translations for xx");
});
test("refuse a namespace there are no translations for", async () => {
await expect(read("en", "other")).rejects.toThrow(
"No other translations for en",
);
});
});
+55
View File
@@ -0,0 +1,55 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* Translations for Element Call as a component.
*
* The standalone app fetches its locale files at runtime from URLs its own
* build emits, which a host serving the library from somewhere else could not
* resolve. The component instead has the bundler split every locale into a
* chunk of its own, loaded the first time its language is asked for; English,
* the fallback, is bundled in so that the first paint never waits for it.
*/
import { type BackendModule, type ResourceKey } from "i18next";
import { languageOfLocalePath } from "../src/utils/i18n";
/** Every locale, as a lazily imported module. */
const translations = import.meta.glob<{ default: ResourceKey }>(
"../locales/*/app.json",
);
/**
* The languages Element Call can be shown in, as BCP 47 tags — `en`, `de`,
* `zh-Hans` and so on. A language that is not one of these falls back to its
* base language where there is one (`de-AT` to `de`), and to English otherwise.
*/
export const supportedLanguages: readonly string[] = [
...new Set(Object.keys(translations).map(languageOfLocalePath)),
];
/** Loads translations on demand. */
export const translationsBackend: BackendModule = {
type: "backend",
init(): void {},
read(language: string, namespace: string, callback): void {
const load = translations[`../locales/${language}/${namespace}.json`];
if (load === undefined) {
callback(new Error(`No ${namespace} translations for ${language}`), null);
return;
}
load().then(
(module) => callback(null, module.default),
(error: unknown) =>
callback(
error instanceof Error ? error : new Error(String(error)),
null,
),
);
},
};
+47
View File
@@ -0,0 +1,47 @@
{
"name": "@element-hq/element-call-component",
"version": "0.0.0",
"description": "Element Call as a React component. Consumed straight from the repository as a git dependency (github:element-hq/element-call#<ref>&path:/component): the host's package manager runs `prepare`, which builds `dist/`.",
"license": "SEE LICENSE IN ../README.md",
"repository": {
"type": "git",
"url": "https://github.com/element-hq/element-call",
"directory": "component"
},
"type": "module",
"devEngines": {
"packageManager": {
"name": "pnpm"
}
},
"files": [
"dist"
],
"main": "./dist/element-call.js",
"module": "./dist/element-call.js",
"types": "./dist/types/component/index.d.ts",
"exports": {
".": {
"types": "./dist/types/component/index.d.ts",
"default": "./dist/element-call.js"
},
"./api": {
"types": "./dist/types/component/api.d.ts",
"default": "./dist/api.js"
},
"./style.css": "./dist/element-call.css"
},
"sideEffects": [
"*.css"
],
"scripts": {
"prepare": "cd .. && pnpm install --frozen-lockfile && pnpm build:component"
},
"peerDependencies": {
"@vector-im/compound-design-tokens": "^11.0.0",
"@vector-im/compound-web": "^10.3.1",
"matrix-js-sdk": "*",
"react": "^19",
"react-dom": "^19"
}
}
+9
View File
@@ -0,0 +1,9 @@
lockfileVersion: '9.0'
settings:
autoInstallPeers: false
excludeLinksFromLockfile: false
importers:
.: {}
+14
View File
@@ -0,0 +1,14 @@
# Makes `component/` a pnpm project of its own rather than a directory inside
# the repository's workspace. That matters when a host installs the component
# straight from this repository (`github:element-hq/element-call#…&path:/component`):
# pnpm prepares such a dependency by running `pnpm install` in this directory,
# and only a project root gets its `prepare` script (see package.json) run, which
# is what builds `dist/`. Inside the repository's own workspace the install
# would silently target the repository instead and build nothing.
#
# Consequence for development: pnpm commands run from within this directory see
# this project, not the repository; run them from the repository root.
# Nothing to install here: the peers in package.json are the host's, and the
# build runs against the repository's own node_modules (see `prepare`).
autoInstallPeers: false
+20
View File
@@ -0,0 +1,20 @@
{
// Declaration output for the component package (`pnpm build:component:types`).
// The root tsconfig only type-checks; this one emits `.d.ts` files, and nothing
// else, for everything the component's entry point reaches. The layout under
// `dist/types` mirrors the repository (`component/index.d.ts`, `src/…`), which
// is what `package.json` points its `types` at.
"extends": "../tsconfig.json",
"compilerOptions": {
"noEmit": false,
"emitDeclarationOnly": true,
"declaration": true,
"declarationMap": false,
"rootDir": "..",
"outDir": "./dist/types"
},
// The entry points, plus the ambient declarations (CSS modules, `?react` SVGs,
// `import.meta.env`, …) that the sources they reach rely on.
"include": ["./index.tsx", "./api.ts", "../src/@types/*.d.ts"],
"exclude": []
}
-3
View File
@@ -7,10 +7,7 @@
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"delayed_leave_event_delay_ms": 18000,
"delayed_leave_event_restart_ms": 4000,
"network_error_retry_ms": 100
}
}
-3
View File
@@ -11,10 +11,7 @@
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_mode": "compatibility",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"delayed_leave_event_delay_ms": 18000,
"delayed_leave_event_restart_ms": 4000,
"network_error_retry_ms": 100
}
}
-3
View File
@@ -7,10 +7,7 @@
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"delayed_leave_event_delay_ms": 18000,
"delayed_leave_event_restart_ms": 4000,
"network_error_retry_ms": 100
},
"posthog": {
-3
View File
@@ -7,10 +7,7 @@
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"delayed_leave_event_delay_ms": 18000,
"delayed_leave_event_restart_ms": 4000,
"network_error_retry_ms": 100
}
}
+28 -4
View File
@@ -3,7 +3,7 @@ networks:
services:
auth-service:
image: ghcr.io/element-hq/lk-jwt-service:0.4.4
image: ghcr.io/element-hq/lk-jwt-service:0.8.0
pull_policy: always
hostname: auth-server
environment:
@@ -15,6 +15,14 @@ services:
# a self-signed certificate
- LIVEKIT_INSECURE_SKIP_VERIFY_TLS=YES_I_KNOW_WHAT_I_AM_DOING
- LIVEKIT_FULL_ACCESS_HOMESERVERS=*
# Registers this instance as an application service on the main homeserver
- LIVEKIT_AS_REGISTRATION_FILE=/etc/lk-jwt-service/app-service.yaml
- LIVEKIT_HS_SERVER_NAME=synapse.m.localhost
# Neither homeserver serves /.well-known/matrix/client, so the C-S API
# location has to be given explicitly.
- LIVEKIT_CS_API_URL_OVERRIDES=synapse.m.localhost=http://homeserver:8008,synapse.othersite.m.localhost=http://homeserver-1:18008
volumes:
- ./backend/app-service.yaml:/etc/lk-jwt-service/app-service.yaml:Z
deploy:
restart_policy:
condition: on-failure
@@ -25,7 +33,7 @@ services:
- ecbackend
auth-service-1:
image: ghcr.io/element-hq/lk-jwt-service:0.4.4
image: ghcr.io/element-hq/lk-jwt-service:0.8.0
pull_policy: always
hostname: auth-server-1
environment:
@@ -37,6 +45,14 @@ services:
# a self-signed certificate
- LIVEKIT_INSECURE_SKIP_VERIFY_TLS=YES_I_KNOW_WHAT_I_AM_DOING
- LIVEKIT_FULL_ACCESS_HOMESERVERS=*
# Registers this instance as an application service on the other homeserver.
- LIVEKIT_AS_REGISTRATION_FILE=/etc/lk-jwt-service/app-service.yaml
- LIVEKIT_HS_SERVER_NAME=synapse.othersite.m.localhost
# Neither homeserver serves /.well-known/matrix/client, so the C-S API
# location has to be given explicitly.
- LIVEKIT_CS_API_URL_OVERRIDES=synapse.m.localhost=http://homeserver:8008,synapse.othersite.m.localhost=http://homeserver-1:18008
volumes:
- ./backend/app-service-othersite.yaml:/etc/lk-jwt-service/app-service.yaml:Z
deploy:
restart_policy:
condition: on-failure
@@ -94,7 +110,10 @@ services:
synapse:
hostname: homeserver
image: ghcr.io/element-hq/synapse:latest
# develop (not latest) is required for application service C-S/S-S proxying
# (MSC4512) and membership look-ups (MSC4502), which are not in a stable
# release yet.
image: ghcr.io/element-hq/synapse:develop
pull_policy: always
environment:
- SYNAPSE_CONFIG_PATH=/data/cfg/homeserver.yaml
@@ -107,12 +126,16 @@ services:
volumes:
- ./backend/synapse_tmp:/data:Z
- ./backend/dev_homeserver.yaml:/data/cfg/homeserver.yaml:Z
- ./backend/app-service.yaml:/data/cfg/app-service.yaml:Z
networks:
- ecbackend
synapse-1:
hostname: homeserver-1
image: ghcr.io/element-hq/synapse:latest
# develop (not latest) is required for application service C-S/S-S proxying
# (MSC4512) and membership look-ups (MSC4502), which are not in a stable
# release yet.
image: ghcr.io/element-hq/synapse:develop
pull_policy: always
environment:
- SYNAPSE_CONFIG_PATH=/data/cfg/homeserver.yaml
@@ -125,6 +148,7 @@ services:
volumes:
- ./backend/synapse_tmp_othersite:/data:Z
- ./backend/dev_homeserver-othersite.yaml:/data/cfg/homeserver.yaml:Z
- ./backend/app-service-othersite.yaml:/data/cfg/app-service.yaml:Z
networks:
- ecbackend
+10 -4
View File
@@ -3,17 +3,23 @@
services:
synapse:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/synapse:latest@sha256:da325af40104051923899e5f5a2f1d537e6e3ccf2f0f38285689ae9bbfdb190a
# develop (not latest) is required for application service C-S/S-S proxying
# (MSC4512) and membership look-ups (MSC4502), which are not in a stable
# release yet.
image: ghcr.io/element-hq/synapse:develop@sha256:337921fd22be310d453344b2265f6c9942315446595df3cf81866f71ca2055a7
volumes:
- ./backend/playwright_homeserver.yaml:/data/cfg/homeserver.yaml:Z
synapse-1:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/synapse:latest@sha256:da325af40104051923899e5f5a2f1d537e6e3ccf2f0f38285689ae9bbfdb190a
# develop (not latest) is required for application service C-S/S-S proxying
# (MSC4512) and membership look-ups (MSC4502), which are not in a stable
# release yet.
image: ghcr.io/element-hq/synapse:develop@sha256:337921fd22be310d453344b2265f6c9942315446595df3cf81866f71ca2055a7
volumes:
- ./backend/playwright_homeserver-othersite.yaml:/data/cfg/homeserver.yaml:Z
element-web:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:6ed62b2963ae098a1f79521eb516834dcf29452168e281cc359d9cfb4380adf6
image: ghcr.io/element-hq/element-web:develop@sha256:b2a5a30afb4f583a7560448b005d8332bee29672f15826e4a995b393058f0291
element-web-1:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:6ed62b2963ae098a1f79521eb516834dcf29452168e281cc359d9cfb4380adf6
image: ghcr.io/element-hq/element-web:develop@sha256:b2a5a30afb4f583a7560448b005d8332bee29672f15826e4a995b393058f0291
+10
View File
@@ -3,8 +3,18 @@
This folder contains documentation for setup, usage, and development of Element Call.
- [Embedded vs standalone mode](./embedded_standalone.md)
- [Element Call as a React component (experimental)](../README.md#element-call-as-a-component-experimental)
- [Url format and parameters](./url_params.md)
- [Global JS controls](./controls.md)
- [MatrixRTC modes](./matrix_rtc_modes.md)
- [Self-Hosting](./self_hosting.md)
- [Developing with linked packages](./linking.md)
### Writing code for Element Call (agent-readable conventions)
Entry point: [AGENTS.md](../AGENTS.md).
- [Agent workflow](./agents/workflow.md)
- [Architecture](./agents/architecture.md)
- [Code style](./agents/code-style.md)
- [Testing](./agents/testing.md)
+95
View File
@@ -0,0 +1,95 @@
# Architecture
## Logic lives in view models; components render
- Call logic lives in `src/state/`. A component that derives call state is
misplaced logic.
- Composing behaviors: a factory taking `(scope, ...deps$)`, returning named
behaviors plus callbacks. Copy `src/state/LayoutSwitchViewModel.ts`. Prefer this
for anything new.
- Owning a resource: a class taking the scope in its constructor — `MediaDevices`,
`MuteStates`, `TileStore`, `Publisher`, `Connection`. Both styles are current;
don't convert one to the other in passing.
- `foo$` is an observable. `Behavior<T>`, an observable with a current value, is
the default for anything a view reads.
## The view model / view contract
- Components take `vm: ViewModel<Snapshot>` and read state through `useBehavior`.
See `CallFooter`, `InCallView`, `LobbyView`, `SettingsModal`.
- A snapshot is `Actions & State`; every field becomes a `foo$` behavior, and none
is optional.
- An unavailable action is `undefined`, not a separate `canDoThing` flag — the
presence of the callback drives the rendering.
- Subscribe in an effect only to drive a side effect off an event stream, as
`ReactionAudioRenderer` does; never to read state a behavior already holds.
## Scopes own lifetimes
- `ObservableScope` bounds every subscription a view model creates.
- Reference only the scope defined in the same function; one captured from an
enclosing scope outlives its owner, and `no-observablescope-leak` rejects it.
## Nothing reads the page
Element Call can be mounted several times inside a host's React tree, so it owns
neither window, URL, document nor router. Each seam defaults to the old standalone
and widget behaviour.
| Never | Use |
| --------------------------------------------- | ------------------------------------------------------------- |
| the `widget` global | `useHostBridge()` — `src/HostBridge.ts` |
| `getUrlParams()`, `window.location` | `useUrlParams()` |
| `useNavigate("/")`, `<Link to="/">` | `useLeaveToHome()`, `LeaveToHomeLink` |
| `document.body` | `useRootElement()` |
| `window.innerWidth/Height`, `useMediaQuery` | `useRootSizeMatches()` in views, `windowSize$` in view models |
| global `i18next` | the instance in `src/utils/i18n.ts` |
| config or analytics reading their environment | `Config.initWith()`, `PosthogAnalytics.configure()` |
- View models take values as options, never `getUrlParams()`
(`callViewModelOptionsFromParams`, `CallViewModelOptions.hostBridge`).
- The host bridge is the only channel to the host: `createWidgetHostBridge(widget)`,
`nullHostBridge` standalone, `useComponentHostBridge`. State a capability
(`supportsReactions`, `supportsProfileChanges`); never infer it from being a
widget.
- Shortcuts and portals attach to the root element, so two instances don't fight.
- Known debt, not precedent: `Grid` measures `window.innerHeight`; `ErrorView`'s
reload and `getAbsoluteRoomUrl` use `window.location`; recaptcha appends to
`document.body` on the standalone login path.
## Context differences are options, not checks
- Named options with per-intent defaults (`configurationForIntent`), never a
runtime check for who is hosting.
- `controlledAudioDevices`, not the platform, is what makes `MediaDevices` pick
`AndroidControlledAudioOutput` / `IOSControlledAudioOutput` over the web
`AudioOutput`. Intent presets set it on non-desktop; standalone leaves it off.
- URL params are a published contract: change one, update `docs/url_params.md`.
## Build targets
Code that builds in only one is a bug.
- `build:full` — standalone app, also widget mode.
- `build:embedded` — `@element-hq/element-call-embedded`.
- `build:sdk` — `@element-hq/matrixrtc-sdk`, sources in `sdk/`, entry `sdk/index.ts`.
Packaged exactly like the component (own pnpm project, run pnpm from the repo
root, types from `sdk/tsconfig.build.json`, externals checked by
`pnpm lint:externals`). Interface only so far; the design is `sdk-plan.md`.
- `build:sdk-target-based-on-call-view-model` — the SDK demo that predates the
package, one bundle of the `CallViewModel`; entry
`sdk-target-based-on-call-view-model/main.ts`. CI's `sdk` package
(`build:sdk:production` / `build:sdk:development`, the Netlify `-sdk` preview)
is still this bundle, not `build:sdk`.
- `build:component` — `@element-hq/element-call-component`, sources in `component/`
(its own pnpm project; run pnpm from the repo root). Host API is in the README;
`pnpm lint:externals` rejects an import of a `react` / `react-dom` /
`matrix-js-sdk` / `@vector-im/compound-web` /
`@vector-im/compound-design-tokens` subpath the externals list omits.
`livekit-client` is bundled on purpose: no host imports it, and the pinned
version travels with the component.
Compound's stylesheets are loaded by `src/index.css` for the page and by the
host for the component, never by `src/base.css`. Two entry
points: `index.tsx` (the component) and `api.ts` (types, enums and
`configurationForIntent`, for a host that must not load the component). `api.ts`
may only reach types and constants — `src/UrlConfiguration.ts`, not `UrlParams`.
+95
View File
@@ -0,0 +1,95 @@
# Code style
_Clean Code_ (Robert C. Martin): small functions, intention-revealing names, low
noise.
## Reuse before you build
Look before you write, in this order:
1. **Compound Web** — `@vector-im/compound-web` for buttons, tooltips, alerts,
dialogs, menus, form controls, avatars; icons in
`@vector-im/compound-design-tokens/assets/web/icons`.
2. **This repo** — `src/button/`, `src/components/`, `src/input/`, `src/form/`,
`src/tabs/`, and `Modal`, `Avatar`, `Toast`, `Slider`, `ErrorView`,
`FullScreenView` at the root of `src/`.
3. **The nearest existing feature** — grep for the same shape first. A hook is
often what you want to extract, not a component.
If nothing fits, name the new shared component in the PR body and say what you
rejected and why — never add one silently. Extend or parameterise before forking,
and extract shared logic into a hook both callers use before copying it; if you
copy, the PR says so.
## Ordering
- **Newspaper order.** Headline first, detail down. A file opens with what it
exists to provide.
- **Stepdown rule.** Caller above callee. A helper used by one function sits
beneath it, which needs a `function` declaration, not a `const` arrow. Existing
arrow components above their callers are not a pattern to extend — nor to churn.
- **Suites.** Test cases first, helpers below. Older suites invert this; follow the
rule in new ones, don't reorder old ones.
- **Names.** What a thing means, not what it is made of: `naturalLayout$`, not
`computedLayout$`.
## Comments
No comment is the default. One earns its place by giving a reason the code
cannot: a platform quirk, an outside constraint, a trap that looks like a
simplification.
- Say the reason, then stop. Test each sentence: does the reason still stand
without it? If so, cut it. Most reasons fit on a line, so a long comment is
worth a second look — it is usually carrying something that isn't one.
- Write about the line the comment sits on, in terms of what the code does
there — leave layout to the stylesheet and measurements to the design.
- Name the condition ("where the platform can't route audio"), not the browser
or the spec entry.
- Put history — what was tried, removed or rejected, the dev journal — in the
commit or the PR description.
- In tests, comment only structure that looks wrong but isn't: batching, timers,
why this scroll position.
Before keeping one: would it still be true after a CSS change or a refactor of
its caller? If not, cut it.
## TypeScript, React and RxJS idiom win on a clash
- Hooks stay unconditional at the top of a component.
- `useCallback` / `useMemo` dependency arrays sometimes force inlining.
- An RxJS pipeline stays one expression. Name the behavior, not each operator.
- A long factory is fine when it reads as a list of named behaviors.
`CallViewModel.ts` is long because the domain is.
- Marble test tables are dense on purpose.
## Enforced by lint
- Copyright header on every file: `Copyright <current year> Element Creations Ltd.`
plus the AGPL / commercial SPDX line.
- `logger` from `matrix-js-sdk/lib/logger`, never `console`, and no top-level
`logger.getChild()`. The `console` ban is only enforced under `src/*/**`.
- No floating or misused promises; async functions typed `Promise<T>`.
- Inline type imports (`import { type Foo }`), so matrix-js-sdk stays lazily
loadable.
- Deep-import `matrix-js-sdk/lib/<module>` as the codebase does. Banned is the bare
`matrix-js-sdk/lib`, `lib/index` and anything under `src/`.
## CSS
- Compound components and `--cpd-*` tokens in CSS modules. A hardcoded colour or px
spacing is a design question, not a licence to inline a hex.
- Size against the root, not the window: `@container element-call (…)` and
`cqw` / `cqh`, never `@media (width)` or `vw` / `vh`. Media queries stay correct
only in standalone-only views — home, login.
- Style `[data-element-call-root]`, never `body` or `:root`; the component build
makes those stand for the root (`component/build/scopeStylesToRoot.ts`).
## Strings and a11y
- Strings through `t()`: add the key, run `pnpm i18n`, fill `locales/en/app.json`.
Other locales come from Localazy; never hand-edit them.
- `t` from `useTranslation()`, or `src/utils/i18n.ts` outside React. Never the
`i18next` global — several instances share a page.
- Accessible names on controls, `aria-pressed` on toggles, keyboard reachability.
`jsx-a11y` rules are errors; `vitest-axe` is available.
+57
View File
@@ -0,0 +1,57 @@
# Testing
Three layers; a user-facing feature needs all three. Unit proves the logic,
Storybook proves the rendering, e2e proves the feature. Codecov gates 80% on the
lines a PR touches.
## Unit
- View model tests use `withTestScheduler` + `testScope()` from `src/utils/test.ts`
and read as ASCII timelines. Copy `LayoutSwitchViewModel.test.ts`.
- Component tests use Testing Library, queried by role and accessible name.
- Reuse the factories in `src/utils/test.ts` — `mockRemoteParticipant`,
`mockMatrixRoom`, `mockLivekitRoom`, `MockRTCSession`. Hand-rolled mocks drift.
- Snapshots live in `__snapshots__/`; update with `pnpm test <Name> -u`.
- How often something redraws is testable: drive the frames and count commits, not
render calls. An effect with no dependency array runs once per commit.
- `component/**/*.test.ts` runs in the same jsdom project as `src`.
## Storybook
A deliverable, not documentation: `pnpm test:storybook` runs every story as a real
browser test in the same CI job as the unit suite. **A UI change without a story is
incomplete.** Only three stories exist so far, so copy `CallFooter.stories.tsx`.
- Drive the component from a snapshot via `useStaticViewModel`, never internals.
- One named story per state that matters — loading, error, empty, denied, mobile.
- Assert in a `play` function with `userEvent` / `expect` from `storybook/test`.
- Mobile is `globals: { viewport: { value: "mobile2" } }`, not a second component.
- Expose interesting props through `argTypes`.
`.storybook/preview.tsx` supplies `TooltipProvider`, `src/index.css` and English
translations. The browser is Playwright's, so a fresh clone needs
`pnpm playwright install` once.
## End-to-end
- `playwright/*.spec.ts` — standalone, on Chromium and Firefox.
- `playwright/widget/` — widget mode, where cross-context bugs surface.
- `playwright/component/` — the component in a host page, via the harness on port
3001 that Playwright starts as a second web server. Catches container-relative
layout, styles escaping the root, two instances on a page, host-bridge reports.
- `playwright/sdk/` — the MatrixRTC SDK through its harness on port 3002, a page
with none of Element Call on it. The smoke test is what the implementation is
built against.
- `playwright/mobile/` — Pixel 7, `mobile` project only.
Test what a user observes: the peer sees the change, it survives a reconnect, it is
right after a reload. Reuse `playwright/spa-helpers.ts`,
`playwright/widget/test-helpers.ts`, `playwright/fixtures/`.
```sh
pnpm test # unit + storybook
pnpm backend # Synapse + LiveKit, required for e2e
pnpm test:playwright # or :open
pnpm dev:component # component harness, port 3001
pnpm dev:sdk # SDK harness, port 3002
```
+50
View File
@@ -0,0 +1,50 @@
# Agent workflow
## Scratch files live in `agent-workspace/`
Git-ignored. One kebab-case subfolder per task, matching the branch topic:
`agent-workspace/<slug>/`.
| File | Purpose |
| --------------------------- | -------------------------------------------------------------------------------- |
| `plan.md` | The concrete approach — files, symbols, edit order — once direction is confirmed |
| `commit-msg.txt` | `git commit -F agent-workspace/<slug>/commit-msg.txt` |
| `pr-body.md` | `gh pr create --body-file agent-workspace/<slug>/pr-body.md` |
| `implementation-summary.md` | What was built, decisions, trade-offs |
| `NN-prompt.md` | Raw prompt text, numbered, when worth keeping |
- Nothing here is durable. The folder is ignored, so anything worth keeping is
promoted into the PR body, a doc, or a feature spec before the task ends.
- Never write scratch files to the repo root. Stray `load_test_summary.md` and
`config.json_` files are what this folder prevents.
- Start `pr-body.md` from `.github/PULL_REQUEST_TEMPLATE.md` and fill every
section.
- Commit subjects are plain imperative English. No conventional-commits prefixes.
- A `plan.md` and a feature spec sit at different altitudes; neither replaces the
other. A spec in `FEATURES_SPEC/` is durable and deliberately abstract —
behaviour, decisions, acceptance criteria — so the feature can be rebuilt against
a `main` nobody has seen yet. A plan is one slice of it landing on today's
`main`: the paths, symbols and edit order the spec's decisions and criteria must
not name. `FEATURES_SPEC/AGENTS.md` wins wherever the two genuinely overlap.
## Hand off before the quality pass
- Implement the change, run the narrowest check that rules out an obviously broken
handoff, then stop and ask whether the direction is right.
- Full `pnpm lint`, the whole suite, coverage and benchmarks come after the
direction is confirmed.
- Wider refactors, extra tests and documentation polish are follow-up work, not
part of the first handoff.
- If a check is needed before feedback, keep it to the touched code and say why.
## Commit and PR readiness
- Commit once the user confirms direction, or asks for one. Not before. Where the
setup has no git identity, a sandbox included, write the message to
`commit-msg.txt` and hand it over instead.
- Ready means every gate in [AGENTS.md](../../AGENTS.md#gates) is green and the
change is covered at the layers [testing.md](./testing.md) asks for. Read the diff
against [code-style.md](./code-style.md) first.
- Re-run the whole checklist after any fix. Only on green, either way.
- Once a human has started reviewing, fix forward — never force-push a
regeneration over a review in progress.
+3
View File
@@ -3,6 +3,9 @@
Element Call is available as two different packages: Full Package and Embedded Package.
The Full Package is designed for standalone use, while the Embedded Package is designed for widget mode only.
There is also an experimental third option, a build of Element Call as a React component for applications
that want to render a call inside their own page rather than in an iframe; see
[Element Call as a component](../README.md#element-call-as-a-component-experimental) in the README.
The table below provides a comparison of the two packages:
+15
View File
@@ -60,6 +60,21 @@ git config --local core.hooksPath .githooks
This will add the hook path for this repository only to .gihooks. which is a tracked (by git) folder containing the pre-commit hook.
## Testing against a matrix-js-sdk branch in CI
Linking covers local development. In CI, `scripts/layered.sh` plays the same
role for `matrix-js-sdk`: after the usual frozen install, `scripts/fetchdep.sh`
looks for a matrix-js-sdk branch with the same name as the pull request's head
branch (in the fork first, when the pull request comes from one). If there is
one, it is cloned, packed and installed over the locked version. If there is
none, the install stays as `pnpm-lock.yaml` pins it.
This means a change that needs a not yet merged matrix-js-sdk change can be tested
by giving both branches the same name. This mirrors how element-web tests against
matrix-js-sdk. Once the matrix-js-sdk branch is merged and deleted, the pull
request falls back to the lockfile and needs a lockfile bump to stay green,
which is also what keeps `main` consistent after the merge.
## Background
Information, why this approach is used can be found in the [linking concept reasoning](./linking_concept_reasoning.md) document.
+1 -1
View File
@@ -24,7 +24,7 @@ homeserver does not support sticky events. This is the default.
MSC4354 and all clients on v0.17.0 or later. The local membership requests its
token from the Matrix 2.0 JWT endpoint of the
[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)
and identifies the room by a hashed identity instead of a `livekit_alias`.
and identifies the room by a hashed identity instead of the Matrix `room_id`.
(Remote memberships always try the new endpoint first and fall back to the
legacy one, so remote participants can be on either.)
-1
View File
@@ -61,7 +61,6 @@ These parameters are relevant to both [widget](./embedded_standalone.md) and [st
| `homeserver` | | Not applicable | No | Homeserver for registering a new (guest) user, configures non-default guest user server when creating a spa link. |
| `lang` | [BCP 47](https://www.rfc-editor.org/info/bcp47) code | No | No | The language the app should use. |
| `password` | | No | No | E2EE password when using a shared secret. (For individual sender keys in embedded mode this is not required.) |
| `perParticipantE2EE` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Enables per participant encryption with Keys exchanged over encrypted matrix room messages. |
| `controlledAudioDevices` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the [global JS controls for audio devices](./controls.md#audio-devices) should be enabled, allowing the list of audio devices to be controlled by the app hosting Element Call. |
| `roomId` | [Matrix Room ID](https://spec.matrix.org/v1.12/appendices/#room-ids) | Yes | No | Anything about what room we're pointed to should be from useRoomIdentifier which parses the path and resolves alias with respect to the default server name, however roomId is an exception as we need the room ID in embedded widget mode, and not the room alias (or even the via params because we are not trying to join it). This is also not validated, where it is in `useRoomIdentifier()`. |
| `showControls` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Displays controls like mute, screen-share, invite, and hangup buttons during a call. |
+128
View File
@@ -0,0 +1,128 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { dirname, resolve, sep } from "node:path";
import { ESLintUtils } from "@typescript-eslint/utils";
/**
* Packages a host supplies and the SDK must not know about: UI, i18n and
* Element Call's design system.
*/
const BANNED_PACKAGES = [
"react",
"react-dom",
"react-i18next",
"i18next",
"@vector-im/",
];
/**
* Packages only a media backend may import. The session speaks MatrixRTC and
* sees media through `sdk/src/media-backend/api.ts`; a backend under
* `sdk/src/media-backend/<name>/` is the one place its protocol's library appears.
*/
const BACKEND_PACKAGES = ["livekit-client", "@livekit/"];
/**
* Keeps the SDK free of Element Call: nothing under `sdk/` may import a file
* outside `sdk/` by relative path (that is `src/`, `component/`,
* `playwright/`) or one of the packages above. The development harness under
* `sdk/dev/` is a host, not part of the library, and is exempt. Keeps the
* session free of any one media backend, too: outside `sdk/src/media-backend/`,
* a backend's packages may only be imported for their types, and only in
* the public API, the errors and the test helpers that have to name them.
*/
const rule = ESLintUtils.RuleCreator(
() => "https://github.com/element-hq/element-call",
)({
name: "sdk-import-boundary",
meta: {
type: "problem",
docs: {
description:
"Disallow the SDK importing Element Call (anything outside sdk/) or host-supplied packages.",
},
messages: {
outsideSdk:
"The SDK must not import '{{specifier}}': it resolves outside sdk/. Copy or move what it needs into sdk/src.",
bannedPackage:
"The SDK must not import '{{specifier}}': a host supplies that, the SDK has no UI.",
backendPackage:
"Only a media backend under sdk/src/media-backend/ may import '{{specifier}}'; the session sees media through media-backend/api.ts.",
},
schema: [],
},
defaultOptions: [],
create(context) {
const filename = context.filename;
const sdkRoot = resolve(context.cwd, "sdk") + sep;
if (!filename.startsWith(sdkRoot)) return {};
if (filename.startsWith(resolve(sdkRoot, "dev") + sep)) return {};
const inBackend = filename.startsWith(
resolve(sdkRoot, "src", "media-backend") + sep,
);
const matchesPackage = (specifier, pkg) =>
specifier === pkg ||
specifier.startsWith(pkg.endsWith("/") ? pkg : `${pkg}/`);
const check = (node, specifier, typeOnly = false) => {
if (typeof specifier !== "string") return;
if (
!inBackend &&
!typeOnly &&
BACKEND_PACKAGES.some((pkg) => matchesPackage(specifier, pkg))
) {
context.report({
node,
messageId: "backendPackage",
data: { specifier },
});
return;
}
if (specifier.startsWith(".")) {
const target = resolve(dirname(filename), specifier);
if (!target.startsWith(sdkRoot))
context.report({
node,
messageId: "outsideSdk",
data: { specifier },
});
} else if (
BANNED_PACKAGES.some((pkg) => matchesPackage(specifier, pkg))
) {
context.report({
node,
messageId: "bannedPackage",
data: { specifier },
});
}
};
return {
ImportDeclaration(node) {
check(
node,
node.source.value,
node.importKind === "type" ||
node.specifiers.every((s) => s.importKind === "type"),
);
},
ExportNamedDeclaration(node) {
if (node.source) check(node, node.source.value);
},
ExportAllDeclaration(node) {
check(node, node.source.value);
},
ImportExpression(node) {
if (node.source.type === "Literal") check(node, node.source.value);
},
};
},
});
export default rule;
+1
View File
@@ -4,5 +4,6 @@ module.exports = {
"no-observablescope-leak": require("./NoObservableScopeLeak").default,
"no-top-level-logger-get-child": require("./NoTopLevelLoggerGetChild")
.default,
"sdk-import-boundary": require("./SdkImportBoundary").default,
},
};
+9 -1
View File
@@ -9,7 +9,15 @@ import { type KnipConfig } from "knip";
export default {
vite: {
config: ["vite.config.ts", "vite-embedded.config.ts", "vite-sdk.config.ts"],
config: [
"vite.config.ts",
"vite-embedded.config.ts",
"vite-sdk.config.ts",
"vite-sdk-dev.config.ts",
"vite-sdk-target-based-on-call-view-model.config.ts",
"vite-component.config.ts",
"vite-component-dev.config.ts",
],
},
entry: ["src/main.tsx", "eslint/index.js", "i18next.config.ts"],
ignoreBinaries: [
+13 -4
View File
@@ -59,10 +59,10 @@
"developer_mode": {
"always_show_iphone_earpiece": "Show iPhone earpiece option on all platforms",
"crypto_version": "Crypto version: {{version}}",
"custom_livekit_url": {
"current_url": "Currently set to: ",
"custom_transport": {
"current": "Currently set to: ",
"from_config": "Currently, no overwrite is set. Url from config is used.",
"label": "Custom Livekit-url",
"label": "Custom transport JSON",
"reset": "Reset overwrite",
"save": "Save",
"saving": "Saving..."
@@ -106,7 +106,7 @@
"insufficient_capacity_description": "The server has reached its maximum capacity and you cannot join the call at this time. Try again later, or contact your server admin if the problem persists.",
"livekit_connection_error": "Failed to connect to Livekit server",
"livekit_connection_error_description": "An error occurred while connecting to the Livekit server (<1>Reason:</1> <2>{{ reason }}</2>).",
"matrix_rtc_transport_missing": "The server is not configured to work with {{brand}}. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",
"matrix_rtc_transport_missing": "The server is not configured to work with {{brand}} and no other call participant provides a usable transport. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",
"membership_manager": "Membership Manager Error",
"membership_manager_description": "The Membership Manager had to shut down. This is caused by many consecutive failed network requests.",
"no_matrix_2_authorization_service": "The authorization service for your media server (SFU) is out of date.",
@@ -163,6 +163,12 @@
"login_auth_links_prompt": "Not registered yet?",
"login_subheading": "To continue to Element",
"login_title": "Login",
"microphone_level": {
"label": "Microphone level",
"no_device": "No microphone found. Connect one, then try again.",
"permission_denied": "Microphone access is blocked. Allow it, then try again.",
"value": "{{level}} of {{max}}"
},
"microphone_off": "Microphone off",
"microphone_on": "Microphone on",
"mute_microphone_button_label": "Mute microphone",
@@ -215,11 +221,14 @@
"activating": "Activating…",
"camera": "Camera",
"camera_numbered": "Camera {{n}}",
"camera_source": "Camera Source",
"change_device_button": "Change audio device",
"default": "Default",
"default_named": "Default <2>({{name}})</2>",
"default_named_plain": "Default ({{name}})",
"handset": "Handset",
"loudspeaker": "Loudspeaker",
"mic_source": "Mic Source",
"microphone": "Microphone",
"microphone_numbered": "Microphone {{n}}",
"speaker": "Speaker",
+24 -14
View File
@@ -6,23 +6,34 @@
"dev": "pnpm dev:full",
"dev:full": "vite",
"dev:embedded": "vite --config vite-embedded.config.js",
"dev:component": "vite --config vite-component-dev.config.ts",
"dev:sdk": "vite --config vite-sdk-dev.config.ts",
"build": "pnpm build:full",
"build:full": "NODE_OPTIONS=--max-old-space-size=16384 vite build",
"build:full": "node --max-old-space-size=16384 node_modules/vite/bin/vite.js build",
"build:full:production": "pnpm build:full",
"build:full:development": "pnpm build:full --mode development",
"build:embedded": "pnpm build:full --config vite-embedded.config.js",
"build:embedded:production": "pnpm build:embedded",
"build:embedded:development": "pnpm build:embedded --mode development",
"build:sdk:development": "pnpm build:sdk --mode development",
"build:sdk": "pnpm build:full --config vite-sdk.config.js",
"build:sdk:production": "pnpm build:sdk",
"build:sdk": "pnpm build:sdk:js && pnpm build:sdk:types",
"build:sdk:js": "pnpm build:full --config vite-sdk.config.js",
"build:sdk:types": "tsc -p sdk/tsconfig.build.json",
"build:sdk-target-based-on-call-view-model": "pnpm build:full --config vite-sdk-target-based-on-call-view-model.config.js",
"build:sdk:production": "pnpm build:sdk-target-based-on-call-view-model",
"build:sdk:development": "pnpm build:sdk-target-based-on-call-view-model --mode development",
"build:component": "pnpm build:component:js && pnpm build:component:types",
"build:component:js": "pnpm build:full --config vite-component.config.js",
"build:component:types": "tsc -p component/tsconfig.build.json",
"build:component:production": "pnpm build:component",
"build:component:development": "pnpm build:component:js --mode development && pnpm build:component:types",
"serve": "vite preview",
"format": "oxfmt",
"format:check": "oxfmt --check; rc=$?; [[ $rc -ne 0 ]] && printf '\\033[46;30m INFO \\033[0m To fix, run: pnpm format\\n' >&2; exit $rc",
"lint": "pnpm lint:types && pnpm lint:oxlint && pnpm lint:knip",
"lint:oxlint": "oxlint src playwright",
"lint:oxlint-fix": "oxlint --fix src playwright",
"lint": "pnpm lint:types && pnpm lint:oxlint && pnpm lint:knip && pnpm lint:externals",
"lint:oxlint": "oxlint src component sdk playwright",
"lint:oxlint-fix": "oxlint --fix src component sdk playwright",
"lint:knip": "knip",
"lint:externals": "node scripts/check-component-externals.mjs",
"lint:types": "tsc",
"i18n": "npx i18next-cli extract",
"i18n:check": "npx i18next-cli extract --ci",
@@ -76,12 +87,12 @@
"@types/sdp-transform": "^2.4.5",
"@typescript-eslint/utils": "^8.61.0",
"@use-gesture/react": "^10.2.11",
"@vector-im/compound-design-tokens": "^10.0.0",
"@vector-im/compound-web": "^10.0.0",
"@vector-im/compound-design-tokens": "^11.0.0",
"@vector-im/compound-web": "^10.4.0",
"@vitejs/plugin-react": "^6.0.2",
"@vitest/browser-playwright": "^4.1.5",
"@vitest/coverage-v8": "^4.0.18",
"@vitest/ui": "4.1.10",
"@vitest/ui": "4.1.11",
"babel-plugin-react-compiler": "^1.0.0",
"classnames": "^2.3.1",
"copy-to-clipboard": "^3.3.3",
@@ -96,17 +107,17 @@
"livekit-client": "^2.18.1",
"lodash-es": "^4.17.21",
"loglevel": "^1.9.1",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#develop",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#3f05dc4d104a3dfc753c5e3549a18c74fab4b076",
"matrix-widget-api": "^1.18.0",
"node-stdlib-browser": "^1.3.1",
"normalize.css": "^8.0.1",
"observable-hooks": "^4.2.3",
"oxfmt": "^0.61.0",
"oxfmt": "^0.66.0",
"oxlint": "^1.70.0",
"oxlint-tsgolint": "^0.25.0",
"pako": "^2.0.4",
"postcss": "^8.4.41",
"postcss-preset-env": "^10.0.0",
"postcss-selector-parser": "^7.1.1",
"posthog-js": "1.408.2",
"qrcode": "^1.5.4",
"react": "19",
@@ -122,7 +133,6 @@
"uuid": "^14.0.0",
"vaul": "^1.0.0",
"vite": "^8.0.0",
"vite-plugin-generate-file": "^0.3.0",
"vite-plugin-html": "^3.2.2",
"vite-plugin-node-polyfills": "^0.28.0",
"vite-plugin-node-stdlib-browser": "^0.2.1",
+58 -11
View File
@@ -11,6 +11,9 @@ import { join } from "path";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { COMPONENT_HARNESS_URL } from "./playwright/component/harness.ts";
import { SDK_HARNESS_URL } from "./playwright/sdk/harness.ts";
const baseURL = process.env.USE_DOCKER
? "http://localhost:8080"
: "https://localhost:3000";
@@ -36,8 +39,20 @@ export default defineConfig({
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: "html",
/**
* Reporter to use. See https://playwright.dev/docs/test-reporters
*
* On CI, `list` prints a line per test as it finishes, which keeps the job
* log moving; `github` annotates failures on the PR; `html` is uploaded as
* an artifact and must never try to open a browser there.
*/
reporter: process.env.CI
? [["list"], ["github"], ["html", { open: "never" }]]
: [["list"], ["html"]],
/* Stop a broken run early on CI instead of waiting out every retry. */
maxFailures: process.env.CI ? 5 : undefined,
/* Under the job's 60 minute limit, so a hung run still ends with a report. */
globalTimeout: process.env.CI ? 50 * 60 * 1000 : undefined,
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
@@ -84,6 +99,11 @@ export default defineConfig({
// enumerateDevices work on CI runners without real hardware.
"media.navigator.streams.fake": true,
"media.navigator.permission.disabled": true,
// Vite serves HTTPS over HTTP/2, and Firefox intermittently stalls
// on Node's HTTP/2 server with a page that never finishes loading
// (one run in five or so, locally). Every server in the suite
// still speaks HTTP/1.1, so nothing is lost by insisting on it.
"network.http.http2.enabled": false,
},
},
},
@@ -115,14 +135,41 @@ export default defineConfig({
],
/* Run your local dev server before starting the tests */
webServer: {
command: "./scripts/playwright-webserver-command.sh",
url: baseURL,
reuseExistingServer: !process.env.CI,
ignoreHTTPSErrors: true,
gracefulShutdown: {
signal: "SIGTERM",
timeout: 500,
webServer: [
{
command: "./scripts/playwright-webserver-command.sh",
url: baseURL,
reuseExistingServer: !process.env.CI,
ignoreHTTPSErrors: true,
gracefulShutdown: {
signal: "SIGTERM",
timeout: 500,
},
},
},
{
// The harness that embeds Element Call as a component. Always a Vite dev
// server, whether or not the app itself is being served from Docker,
// since there is nothing to build: it is a development page only.
command: "pnpm dev:component",
url: COMPONENT_HARNESS_URL,
reuseExistingServer: !process.env.CI,
ignoreHTTPSErrors: true,
gracefulShutdown: {
signal: "SIGTERM",
timeout: 500,
},
},
{
// The harness that uses the MatrixRTC SDK without Element Call. A Vite
// dev server for the same reason as the component's.
command: "pnpm dev:sdk",
url: SDK_HARNESS_URL,
reuseExistingServer: !process.env.CI,
ignoreHTTPSErrors: true,
gracefulShutdown: {
signal: "SIGTERM",
timeout: 500,
},
},
],
});
+229
View File
@@ -0,0 +1,229 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test, type Locator, type Page } from "@playwright/test";
import { SpaHelpers } from "./spa-helpers.ts";
import { installFakeDevices } from "./utils/fake-devices.ts";
test.describe("the quick audio menu", () => {
test("lists speakers and microphones with a live level meter", async ({
page,
}) => {
await installFakeDevices(page, { microphones: 3, speakers: 3 });
await joinACall(page, "Menu user", "Audio menu");
await openAudioMenu(page);
await expect(page.getByRole("group", { name: "Speaker" })).toBeVisible();
await expect(page.getByRole("group", { name: "Microphone" })).toBeVisible();
// By name, not count: the browser adds its own fake output and a Default.
for (const n of [1, 2, 3])
await expect(
page
.getByRole("group", { name: "Speaker" })
.getByRole("menuitemradio", { name: `Fake Speaker ${n}` }),
).toBeVisible();
await expect(
page.getByRole("menuitemradio", { checked: true }),
).toHaveCount(2);
const meter = page.getByRole("meter", { name: "Microphone level" });
await expect(meter).toBeVisible();
await expect(meter).toHaveAttribute("aria-valuenow", /\d+/);
await expect(meter).toHaveAttribute("aria-valuetext", /\d+ of \d+/);
// Both browsers here can route audio; the platform that can't is a unit check.
await expect(
page
.getByRole("group", { name: "Speaker" })
.getByRole("menuitemradio")
.first(),
).toHaveAttribute("aria-disabled", "false");
});
test("moves the microphone and the speaker without disturbing the call", async ({
browser,
}) => {
// Two browsers and a real call.
test.slow();
const hostContext = await browser.newContext({ reducedMotion: "reduce" });
const host = await hostContext.newPage();
await installFakeDevices(host, { microphones: 3, speakers: 3 });
await joinACall(host, "Host", "Device switch");
const inviteLink = await SpaHelpers.getCallInviteLink(host);
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guest = await guestContext.newPage();
await SpaHelpers.joinCallFromInviteLink(guest, inviteLink, "Guest");
await SpaHelpers.expectVideoTilesCount(guest, 2);
await openAudioMenu(host);
await selectDevice(host, "Microphone", "Fake Microphone 2");
await openAudioMenu(host);
await selectDevice(host, "Speaker", "Fake Speaker 2");
// Not a rejoin: neither side drops, and the guest still has both tiles.
await expect(
host.getByRole("dialog", { name: "Reconnecting…" }),
).not.toBeVisible();
await expect(
guest.getByRole("dialog", { name: "Reconnecting…" }),
).not.toBeVisible();
await SpaHelpers.expectVideoTilesCount(guest, 2);
await expect(guest.getByText("Waiting for media...")).not.toBeVisible();
await hostContext.close();
await guestContext.close();
});
test("keeps the meter moving while muted, and sends nothing", async ({
browser,
}) => {
// Two browsers and a real call.
test.slow();
const hostContext = await browser.newContext({ reducedMotion: "reduce" });
const host = await hostContext.newPage();
await installFakeDevices(host);
await joinACall(host, "Muted host", "Muted meter");
const inviteLink = await SpaHelpers.getCallInviteLink(host);
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guest = await guestContext.newPage();
await SpaHelpers.joinCallFromInviteLink(guest, inviteLink, "Listener");
await SpaHelpers.expectVideoTilesCount(guest, 2);
const mute = host.getByTestId("incall_mute");
await mute.click();
await expect(mute).toHaveAttribute("aria-checked", "false");
await openAudioMenu(host);
const meter = host.getByRole("meter", { name: "Microphone level" });
await expect(meter).toBeVisible();
// By test id: the modal menu hides the rest of the call from the a11y tree.
await expect(mute).toHaveAttribute("aria-checked", "false");
await expect(mute).toBeVisible();
// And the guest is shown the mute.
await expect(
guest.getByTestId("videoTile").filter({ hasText: "Muted host" }),
).toBeVisible();
await hostContext.close();
await guestContext.close();
});
test("keeps the meter at the foot of the list while it scrolls", async ({
page,
}) => {
await installFakeDevices(page, { microphones: 20, speakers: 4 });
await joinACall(page, "Scroller", "Long device list");
await openAudioMenu(page);
const meter = page.getByRole("meter", { name: "Microphone level" });
await expect(meter).toBeVisible();
// Scrolled so the microphones start at the top: only there does a pinned
// meter differ from one that is simply last.
const list = page.locator("[role='menu'] div[role='none']").first();
await list.evaluate((element) => {
const group = element.querySelector("[role='group'][aria-label*='icro']");
element.scrollTop +=
group!.getBoundingClientRect().top -
element.getBoundingClientRect().top;
});
await expect(meter).toBeInViewport();
await expectPinnedInside(meter, list);
await expect(
page.getByRole("menuitemradio", { name: "Fake Microphone 20" }),
).toBeVisible();
});
test("shows the focus ring only when the keyboard moved the focus", async ({
page,
browserName,
}) => {
test.skip(
browserName === "firefox",
"Headless Firefox does not deliver synthetic key presses reliably; see reconnect.spec.ts",
);
await installFakeDevices(page);
await joinACall(page, "Keyboard user", "Focus ring");
await openAudioMenu(page);
const first = page.getByRole("menuitemradio").first();
// Opened by pointer: no ring, though Radix has moved focus into the menu.
await expect.poll(async () => outlineWidth(first)).toBe(0);
await page.keyboard.press("ArrowDown");
const focused = page.locator("[role='menuitemradio']:focus");
await expect.poll(async () => outlineWidth(focused)).toBeGreaterThan(0);
// And the pointer takes it away again.
await first.hover();
await expect.poll(async () => outlineWidth(focused)).toBe(0);
});
});
/** Creates a call and joins it, leaving the page in the call. */
async function joinACall(
page: Page,
userName: string,
callName: string,
): Promise<void> {
await page.goto("/");
await SpaHelpers.createCall(page, userName, callName, true);
await expect(page.getByTestId("name_tag")).toContainText(userName);
// The media controls stay disabled until devices have enumerated.
await expect(page.getByTestId("incall_mute")).toBeEnabled({
timeout: 10_000,
});
}
async function openAudioMenu(page: Page): Promise<void> {
await page.getByRole("button", { name: "Microphone" }).click();
await expect(page.getByRole("menu")).toBeVisible();
}
async function selectDevice(
page: Page,
section: "Speaker" | "Microphone",
name: string,
): Promise<void> {
const item = page
.getByRole("group", { name: section })
.getByRole("menuitemradio", { name });
await item.click();
// Selecting doesn't close the menu, so dismiss it.
await page.keyboard.press("Escape");
await expect(page.getByRole("menu")).not.toBeVisible();
}
/** Asserts the meter sits within the scrollport and inside the menu's frame. */
async function expectPinnedInside(
meter: Locator,
list: Locator,
): Promise<void> {
const meterBox = (await meter.boundingBox())!;
const listBox = (await list.boundingBox())!;
const frame = (await meter.page().getByRole("menu").boundingBox())!;
expect(meterBox.y + meterBox.height).toBeLessThanOrEqual(
listBox.y + listBox.height + 1,
);
expect(meterBox.y).toBeGreaterThanOrEqual(listBox.y - 1);
expect(meterBox.x).toBeGreaterThan(frame.x);
expect(meterBox.x + meterBox.width).toBeLessThan(frame.x + frame.width);
}
/** Painted outline width, in px. */
async function outlineWidth(item: Locator): Promise<number> {
if ((await item.count()) === 0) return 0;
return item.first().evaluate((element) => {
const { outlineStyle, outlineWidth } = getComputedStyle(element);
return outlineStyle === "none" ? 0 : Number.parseFloat(outlineWidth) || 0;
});
}
+135
View File
@@ -0,0 +1,135 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Locator, type Page, test } from "@playwright/test";
import { createUserAndRoom, resizeContainer, startHarness } from "./harness.ts";
import { installFakeDevices } from "../utils/fake-devices.ts";
/**
* The device menu with Element Call as a component in a host page, where the
* portalled menu must be sized against the call, not the window. Driven from
* the lobby: it builds the same menu, without two connections' worth of flake.
*/
// Sign-in, crypto setup and sync happen twice before anything shows.
test.describe.configure({ timeout: 180_000 });
test("sizes the device list against the call, not the window", async ({
page,
}) => {
await installFakeDevices(page, { microphones: 20, speakers: 4 });
const { username, roomId } = await createUserAndRoom("menusize");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
// A short call in a much taller page.
const container = pane.getByTestId("call-container");
await resizeContainer(container, { width: 900, height: 400 });
await expect(pane.getByTestId("lobby_joinCall")).toBeVisible({
timeout: 60_000,
});
const list = await openDeviceList(page, pane);
const callHeight = (await container.boundingBox())!.height;
const windowHeight = page.viewportSize()!.height;
const listHeight = (await list.boundingBox())!.height;
expect(callHeight).toBeLessThan(windowHeight * 0.75);
expect(listHeight).toBeLessThanOrEqual(callHeight);
expect(listHeight).toBeLessThan(windowHeight * 0.6);
});
test("follows the call area when the host resizes it", async ({ page }) => {
await installFakeDevices(page, { microphones: 20, speakers: 4 });
const { username, roomId } = await createUserAndRoom("menuresize");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
const container = pane.getByTestId("call-container");
await resizeContainer(container, { width: 900, height: 360 });
await expect(pane.getByTestId("lobby_joinCall")).toBeVisible({
timeout: 60_000,
});
const list = await openDeviceList(page, pane);
const whenShort = (await list.boundingBox())!.height;
// The host grows the call while the menu is open.
await resizeContainer(container, { width: 900, height: 700 });
await expect
.poll(async () => (await list.boundingBox())!.height)
.toBeGreaterThan(whenShort);
});
test("keeps every device reachable in a small container", async ({ page }) => {
await installFakeDevices(page, { microphones: 20, speakers: 4 });
const { username, roomId } = await createUserAndRoom("menureach");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
const container = pane.getByTestId("call-container");
await resizeContainer(container, { width: 400, height: 360 });
await expect(pane.getByTestId("lobby_joinCall")).toBeVisible({
timeout: 60_000,
});
const list = await openDeviceList(page, pane);
// More devices than fit, so the list must scroll.
expect(await list.evaluate((el) => el.scrollHeight > el.clientHeight)).toBe(
true,
);
const last = page.getByRole("menuitemradio", { name: "Fake Microphone 20" });
await last.scrollIntoViewIfNeeded();
await expect(last).toBeInViewport();
// Reachable means clickable: the menu may be drawn outside the call area.
await last.click();
await expect(last).toHaveAttribute("aria-checked", "true");
});
test("tracks the focus source of its own call, not the page", async ({
page,
}) => {
await installFakeDevices(page);
const { username, roomId } = await createUserAndRoom("menufocus");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
const other = panes.nth(1);
await expect(pane.getByTestId("lobby_joinCall")).toBeVisible({
timeout: 60_000,
});
await openDeviceList(page, pane);
const menu = page.getByRole("menu");
// Asserted on the attribute: the component build's scoped stylesheet doesn't
// reach the portalled menu, so its ring can't be read here.
await expect(menu).toHaveAttribute("data-focus-source", "pointer");
// A key pressed in the other call says nothing about this menu.
await other.evaluate((element) =>
element.dispatchEvent(
new KeyboardEvent("keydown", { key: "ArrowDown", bubbles: true }),
),
);
await expect(menu).toHaveAttribute("data-focus-source", "pointer");
await page.keyboard.press("ArrowDown");
await expect(menu).toHaveAttribute("data-focus-source", "keyboard");
});
/** Opens a component's microphone menu and returns its device list. */
async function openDeviceList(page: Page, pane: Locator): Promise<Locator> {
await pane
.getByRole("button", { name: "Microphone" })
.click({ timeout: 60_000 });
await expect(page.getByRole("menu")).toBeVisible();
const list = page.getByRole("menu").getByTestId("device-list");
await expect(list).toBeVisible();
return list;
}
+285
View File
@@ -0,0 +1,285 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Locator, type Page, test } from "@playwright/test";
import {
createUserAndRoom,
expectWithin,
resizeContainer,
startHarness,
} from "./harness.ts";
import { SpaHelpers } from "../spa-helpers.ts";
/**
* Element Call embedded as a React component, driven through the development
* harness in `component/dev`.
*
* What these cover that the widget tests cannot is everything that follows from
* sharing a page with a host: whether Element Call stays inside the space it
* was given, and whether two of it can exist at once. As a widget, the iframe
* guaranteed both.
*/
// Each test signs in twice, sets up crypto twice and syncs twice before
// anything is on screen, and then waits for media to connect; the waits below
// are sized for that, so the tests have to be too
test.describe.configure({ timeout: 180_000 });
/** The settings button, whichever of the two the footer is currently showing. */
function settingsButton(pane: Locator): Locator {
return pane
.getByTestId("settings-bottom-left")
.or(pane.getByTestId("settings-bottom-center"))
.filter({ visible: true })
.first();
}
test("holds a call between two components on one page", async ({ page }) => {
const { username, roomId } = await createUserAndRoom("twocomponents");
const panes = await startHarness(page, username, roomId);
// Each component shows a lobby of its own, and neither has joined anything
// just by being rendered
for (const index of [0, 1])
await expect(panes.nth(index).getByTestId("lobby_joinCall")).toBeVisible({
timeout: 60_000,
});
for (const index of [0, 1])
await panes.nth(index).getByTestId("lobby_joinCall").click();
// Two devices of one account, so each component should see itself and the
// other. This is the part that proves two Element Calls in one page are two
// calls, and not one shared thing wearing two hats.
for (const index of [0, 1])
await expect(panes.nth(index).getByTestId("videoTile")).toHaveCount(2, {
timeout: 60_000,
});
});
test("keeps its modals inside the container it was given", async ({ page }) => {
const { username, roomId } = await createUserAndRoom("containment");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
const container = pane.getByTestId("call-container");
// In the flat container the harness gives it by default, Element Call hides
// its controls a few seconds after the call starts, as it would in a flat
// window. A full-size container keeps them on screen to be clicked.
await resizeContainer(container, { width: 900, height: 640 });
await pane.getByTestId("lobby_joinCall").click({ timeout: 60_000 });
await expect(pane.getByTestId("footer-container")).toBeVisible({
timeout: 60_000,
});
// Both of these are positioned `fixed`, and were centred on the window
// rather than the container until it was made a containing block. The
// settings dialog spilled over the host's interface; the reaction picker sat
// at 82vh, which put it below the container entirely and so out of sight.
await settingsButton(pane).click();
await expectWithin(pane.getByRole("dialog"), container);
await pane.getByTestId("modal_close").click();
await pane.getByRole("button", { name: "Reactions" }).click();
await expectWithin(
pane.getByRole("dialog", { name: "Pick reaction" }),
container,
);
});
test("keeps the call inside the container, wherever the host put it", async ({
page,
}) => {
const { username, roomId } = await createUserAndRoom("tileswithin");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
const container = pane.getByTestId("call-container");
// Large enough for the layout switch to be offered
await resizeContainer(container, { width: 900, height: 640 });
await pane.getByTestId("lobby_joinCall").click({ timeout: 60_000 });
await expect(pane.getByTestId("footer-container")).toBeVisible({
timeout: 60_000,
});
// The spotlight layout draws its tile in the fixed grid, which is positioned
// against Element Call's root rather than laid out in flow. The harness puts
// the container below a header of its own, so a grid offset measured from
// the top of the page instead of from the root would land the tile on top of
// the footer and out of the bottom of the container.
await pane.getByRole("radio", { name: "Spotlight" }).check();
const tile = pane.getByTestId("videoTile").first();
await expect(tile).toBeVisible({ timeout: 60_000 });
await expectWithin(tile, container);
});
test("leaves the host's own page unstyled", async ({ page }) => {
const { username, roomId } = await createUserAndRoom("hoststyles");
const panes = await startHarness(page, username, roomId);
await expect(panes.first().getByTestId("lobby_joinCall")).toBeVisible({
timeout: 60_000,
});
// Element Call's stylesheet is written for a page of its own: normalize.css
// gives `html` a line height, Compound gives `body` its font and feature
// settings, and the design tokens live on `:root`. None of that may reach the
// host's document — the harness sets none of these itself, so anything other
// than the browser's defaults here came from us.
const host = await page.evaluate(() => {
const html = getComputedStyle(document.documentElement);
const body = getComputedStyle(document.body);
return {
lineHeight: html.lineHeight,
fontFeatureSettings: body.fontFeatureSettings,
token: html.getPropertyValue("--cpd-color-text-primary"),
};
});
expect(host).toEqual({
lineHeight: "normal",
fontFeatureSettings: "normal",
token: "",
});
// While inside the container, the same rules do apply
const root = panes.first().locator("[data-element-call-root]");
await expect(root).toHaveCSS("font-feature-settings", /"kern"/);
});
test("tells its host what it is doing", async ({ page }) => {
const { username, roomId } = await createUserAndRoom("hostbridge");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
const log = page.getByTestId("bridge-log");
// Every component reports to its host through the bridge, whether that host
// is a widget container or an application embedding it directly
await expect(log).toContainText("contentLoaded", { timeout: 60_000 });
await pane.getByTestId("lobby_joinCall").click({ timeout: 60_000 });
await expect(log).toContainText("notifyJoined", { timeout: 60_000 });
await expect(log).toContainText("setAlwaysOnScreen(true)", {
timeout: 60_000,
});
// And takes instructions back: the host asking for a mute should come back
// as the component reporting the new state
await pane.getByRole("button", { name: "Mute" }).click();
await expect(log).toContainText("notifyDeviceMute(audio: false", {
timeout: 30_000,
});
});
test("lays itself out for the space it is given, not the page", async ({
page,
}) => {
const { username, roomId } = await createUserAndRoom("containersize");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
const container = pane.getByTestId("call-container");
const call = pane.locator("[data-layout]");
await pane.getByTestId("lobby_joinCall").click({ timeout: 60_000 });
await expect(call).toBeVisible({ timeout: 60_000 });
await expect(call).not.toHaveAttribute("data-layout", "pip");
// As a widget, Element Call's container and its window were one and the same:
// a host wanting a picture-in-picture made the iframe small, and Element Call
// saw the window shrink. A component gets no such signal from the window,
// which stays as large as it ever was; only the container changes.
await resizeContainer(container, { width: 300, height: 300 });
await expect(call).toHaveAttribute("data-layout", "pip");
await resizeContainer(container, { width: 900, height: 700 });
await expect(call).not.toHaveAttribute("data-layout", "pip");
});
/**
* The shape of a call at whatever size it has been given: the layout it chose,
* how much of the height the tile and the footer take, and which controls the
* footer shows. Two calls with the same shape look the same, participants aside.
*/
async function callShape(scope: Page | Locator): Promise<{
layout: string | null;
tileHeight: number;
footerHeight: number;
buttons: (string | null)[];
}> {
const call = scope.locator("[data-layout]");
const footer = scope.getByTestId("footer-container");
await expect(footer).toBeVisible();
// The tile arrives with the media connection, which can take a while
const tile = scope.getByTestId("videoTile").first();
await expect(tile).toBeVisible({ timeout: 60_000 });
const tileBox = (await tile.boundingBox())!;
const footerBox = (await footer.boundingBox())!;
// Buttons and switches alike: the mute controls are switches
const buttons = await footer
.locator("button")
.filter({ visible: true })
.evaluateAll((elements) =>
elements.map((element) => element.getAttribute("aria-label")),
);
return {
layout: await call.getAttribute("data-layout"),
tileHeight: Math.round(tileBox.height),
footerHeight: Math.round(footerBox.height),
buttons,
};
}
test("looks the same in a small container as in a small window", async ({
page,
browser,
}) => {
// Two calls to set up, one of them through the harness's two logins
test.setTimeout(300_000);
const size = { width: 300, height: 300 };
// The reference is Element Call owning a window of that size, which is what
// a mobile app's webview or a browser's picture-in-picture gives it, and
// what its small-window styling was written for.
// No permissions to grant: each browser is launched with fake media that is
// handed out without asking (see playwright.config.ts), and Firefox rejects
// a request for `camera` or `microphone` outright
const referenceContext = await browser.newContext({
viewport: size,
ignoreHTTPSErrors: true,
});
const referencePage = await referenceContext.newPage();
await referencePage.goto("/");
await SpaHelpers.createCall(referencePage, "Reference", "smallwindow", true);
const reference = await callShape(referencePage);
await referencePage.screenshot({
path: test.info().outputPath("small-window.png"),
});
// The component gets a container of that size, in a window that is far larger
const { username, roomId } = await createUserAndRoom("smallcontainer");
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
const container = pane.getByTestId("call-container");
await resizeContainer(container, size);
await pane.getByTestId("lobby_joinCall").click({ timeout: 60_000 });
await expect(pane.locator("[data-layout]")).toBeVisible({ timeout: 60_000 });
const component = await callShape(pane);
await container.screenshot({
path: test.info().outputPath("small-container.png"),
});
await referenceContext.close();
// The breakpoints in Element Call's stylesheets are container queries, so a
// small container gets the compact footer a small window does, rather than
// the full-width one the window's own size would call for
expect(component).toEqual(reference);
// And that footer is the compact one: a single row of controls, not the
// full-height bar with its logo and layout switch that a large window gets
expect(component.footerHeight).toBeLessThan(size.height / 3);
expect(component.tileHeight + component.footerHeight).toBeLessThanOrEqual(
size.height,
);
});
+132
View File
@@ -0,0 +1,132 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Locator, type Page } from "@playwright/test";
import { SynapseAdmin } from "../utils/synapse-admin.ts";
/**
* Where the component harness is served — `component/dev`, which embeds Element
* Call the way a host application would. Not the `baseURL` the rest of the
* suite uses: these tests drive a page that contains Element Call rather than
* Element Call itself.
*/
export const COMPONENT_HARNESS_URL = "https://localhost:3001";
const HOMESERVER_URL = "https://synapse.m.localhost";
const PASSWORD = "foobarbaz1!";
/**
* Registers a user through the Synapse admin API and creates a room for it to
* call in, without touching a browser. The harness signs into this account
* twice, giving two devices in one page and so a real call between the two
* components.
*/
export async function createUserAndRoom(
name: string,
): Promise<{ username: string; roomId: string }> {
const username = `${name}_${Date.now()}`;
const { access_token: accessToken } = await SynapseAdmin.forHomeserver(
HOMESERVER_URL,
).registerUser(username, PASSWORD, name);
const response = await fetch(
`${HOMESERVER_URL}/_matrix/client/v3/createRoom`,
{
method: "POST",
headers: {
Authorization: `Bearer ${accessToken}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ name: `${name}'s call`, preset: "private_chat" }),
},
);
if (!response.ok)
throw new Error(
`Could not create a room: ${response.status} ${await response.text()}`,
);
const { room_id: roomId } = (await response.json()) as { room_id: string };
return { username, roomId };
}
/**
* Opens the harness signed in as the given user, and waits for both embedded
* calls to appear.
*
* @returns The two containers the host gave Element Call, in order.
*/
export async function startHarness(
page: Page,
username: string,
roomId: string,
): Promise<Locator> {
const query = new URLSearchParams({
homeserver: HOMESERVER_URL,
username,
password: PASSWORD,
room: roomId,
});
await page.goto(`${COMPONENT_HARNESS_URL}/?${query.toString()}`);
await page.getByRole("button", { name: "Start" }).click();
const panes = page.getByTestId("call-pane");
// Two logins, two crypto setups and two initial syncs happen first
await expect(panes).toHaveCount(2, { timeout: 120_000 });
return panes;
}
/**
* Asserts that one element is drawn entirely inside another.
*
* This is the check that being a component rather than an iframe costs us: an
* iframe could not paint outside itself whatever its stylesheets said, whereas
* a component shares the page and has to be made to stay put.
*/
export async function expectWithin(
inner: Locator,
outer: Locator,
): Promise<void> {
await expect(inner).toBeVisible();
// Wait for animations to settle.
await inner.evaluate(async (element) => {
for (const animation of element.getAnimations({ subtree: true }))
await animation.finished;
});
const innerBox = await inner.boundingBox();
const outerBox = await outer.boundingBox();
if (innerBox === null || outerBox === null)
throw new Error("Expected both elements to be laid out");
// A pixel of slack, for subpixel layout
const slack = 1;
expect(innerBox.x).toBeGreaterThanOrEqual(outerBox.x - slack);
expect(innerBox.y).toBeGreaterThanOrEqual(outerBox.y - slack);
expect(innerBox.x + innerBox.width).toBeLessThanOrEqual(
outerBox.x + outerBox.width + slack,
);
expect(innerBox.y + innerBox.height).toBeLessThanOrEqual(
outerBox.y + outerBox.height + slack,
);
}
/**
* Gives one of the harness's containers a new size. Element Call lays itself
* out for the size of its container, so this is how a test puts it into a
* particular mode: a flat or narrow one, a picture-in-picture, or a full-size
* window, without the window itself changing at all.
*/
export async function resizeContainer(
container: Locator,
size: { width: number; height: number },
): Promise<void> {
await container.evaluate((element, { width, height }) => {
element.style.width = `${width}px`;
element.style.height = `${height}px`;
}, size);
}
+2 -2
View File
@@ -7,7 +7,7 @@ Please see LICENSE in the repository root for full details.
import { expect, test } from "@playwright/test";
import { createJTWToken } from "./fixtures/jwt-token";
import { createJWTToken } from "./fixtures/jwt-token";
test("Should show error screen if fails to get JWT token", async ({ page }) => {
await page.goto("/");
@@ -95,7 +95,7 @@ test("Should show error screen if call creation is restricted", async ({
contentType: "application/json",
body: JSON.stringify({
url: "wss://badurltotricktest/livekit/sfu",
jwt: createJTWToken("@fake:user", "!fake:room"),
jwt: createJWTToken("@fake:user", "!fake:room"),
}),
}),
);
+1 -1
View File
@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
export function createJTWToken(sub: string, room: string): string {
export function createJWTToken(sub: string, room: string): string {
return [
{}, // header
{
+14 -5
View File
@@ -109,10 +109,15 @@ export const widgetTest = test.extend<MyFixtures>({
await TestHelpers.dismissInviteUnknownUserModal(ewPage1);
// Accept the invite
await TestHelpers.closeReleaseAnnouncement(
ewPage2,
"Introducing Sections",
);
await TestHelpers.expandAllSections(ewPage2);
await expect(
ewPage2.getByRole("option", { name: "Welcome Room" }),
TestHelpers.roomListItem(ewPage2, "Welcome Room"),
).toBeVisible();
await ewPage2.getByRole("option", { name: "Welcome Room" }).click();
await TestHelpers.roomListItem(ewPage2, "Welcome Room").click();
await ewPage2.getByRole("button", { name: "Accept" }).click();
await expect(
ewPage2
@@ -127,7 +132,7 @@ export const widgetTest = test.extend<MyFixtures>({
await ewPage1
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.getByRole("button", { name: "New", exact: true })
.click();
await ewPage1.getByRole("menuitem", { name: "Start chat" }).click();
@@ -152,8 +157,12 @@ export const widgetTest = test.extend<MyFixtures>({
).toBeVisible();
// Accept the DM invite from brooks
// This how playwright record selects the DM invite in the room list
await ewPage2.getByRole("button", { name: "Open room" }).click();
await TestHelpers.closeReleaseAnnouncement(
ewPage2,
"Introducing Sections",
);
await TestHelpers.expandAllSections(ewPage2);
await TestHelpers.roomListItem(ewPage2, brooksDisplayName).click();
await ewPage2.getByRole("button", { name: "Start chatting" }).click();
}
+8
View File
@@ -7,6 +7,8 @@ Please see LICENSE in the repository root for full details.
import { expect, test } from "@playwright/test";
import { SpaHelpers } from "./spa-helpers";
// Skip test for Firefox, due to page.keyboard.press("Tab") not reliable on headless mode
test.skip(
({ browserName }) => browserName === "firefox",
@@ -17,6 +19,12 @@ test("can only interact with header and footer while reconnecting", async ({
page,
}) => {
await page.goto("/");
// The reconnecting state is entered via the probablyLeft timer, which
// mirrors the delayed leave event's timeout. With delegation that timeout
// is one hour, putting it out of reach of the clock fast-forward below.
// Keep the leave client-managed so its short timeout applies.
await SpaHelpers.disableLeaveDelegation(page);
await page.getByTestId("home_callName").click();
await page.getByTestId("home_callName").fill("Test call");
await page.getByTestId("home_displayName").click();
+41
View File
@@ -0,0 +1,41 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { startPair, tileOf } from "./harness";
test.describe.configure({ timeout: 300_000 });
test("a message sent beside the media reaches the peer, and only the peer", async ({
browser,
}) => {
const { pages, users } = await startPair(browser, "sdkdata");
const [sender, receiver] = pages;
// Both are in the call before anything is sent, so the receiver knows the
// sender as a member and does not drop the message
for (const page of pages)
await expect(page.getByTestId("member")).toHaveCount(2, {
timeout: 60_000,
});
await expect(tileOf(receiver, users[0])).toBeVisible();
await sender.getByPlaceholder("message").fill("hello from the SDK");
await sender.getByRole("button", { name: "Send" }).click();
const message = receiver.getByTestId("message");
await expect(message).toHaveText(
`${users[0].displayName}: hello from the SDK`,
{
timeout: 30_000,
},
);
await expect(message).toHaveAttribute("data-topic", "chat");
await expect(message).toHaveAttribute("data-user-id", users[0].userId);
// Our own messages are not echoed back
await expect(sender.getByTestId("message")).toHaveCount(0);
});
+167
View File
@@ -0,0 +1,167 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type Browser,
expect,
type Locator,
type Page,
} from "@playwright/test";
import { SynapseAdmin } from "../utils/synapse-admin.ts";
/**
* Where the SDK harness is served — `sdk/dev`, a page that uses the SDK the way
* a host would, with none of Element Call in it.
*/
export const SDK_HARNESS_URL = "https://localhost:3002";
const HOMESERVER_URL = "https://synapse.m.localhost";
const PASSWORD = "foobarbaz1!";
/** A registered user, with the session the registration opened. */
export interface User {
username: string;
displayName: string;
userId: string;
deviceId: string;
accessToken: string;
}
/**
* Registers two users through the Synapse admin API and has the first create a
* public room, so that the second can join it by id without an invite. The
* registration logs each user in, which spares the harness a `/login` call
* that the homeserver rate-limits.
*/
export async function createUsersAndRoom(
name: string,
): Promise<{ users: [User, User]; roomId: string }> {
const admin = SynapseAdmin.forHomeserver(HOMESERVER_URL);
const users = (await Promise.all(
["a", "b"].map(async (letter, index): Promise<User> => {
const username = `${name}_${letter}_${Date.now()}`;
const displayName = `${name} ${"AB"[index]}`;
const registration = await admin.registerUser(
username,
PASSWORD,
displayName,
);
return {
username,
displayName,
userId: registration.user_id,
deviceId: registration.device_id,
accessToken: registration.access_token,
};
}),
)) as [User, User];
const response = await fetch(
`${HOMESERVER_URL}/_matrix/client/v3/createRoom`,
{
method: "POST",
headers: {
Authorization: `Bearer ${users[0].accessToken}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
name: `${name}'s session`,
preset: "public_chat",
// Per-participant media keys travel in encrypted to-device messages,
// which only reach devices the sender's crypto tracks, and it tracks
// the members of encrypted rooms
initial_state: [
{
type: "m.room.encryption",
state_key: "",
content: { algorithm: "m.megolm.v1.aes-sha2" },
},
],
// Every member has to be allowed to write its own membership, which
// a public room does not grant by default; the same levels Element
// Call gives a room it creates for a call
power_level_content_override: {
state_default: 0,
events_default: 0,
users_default: 0,
events: { "org.matrix.msc3401.call.member": 0 },
},
}),
},
);
if (!response.ok)
throw new Error(
`Could not create a room: ${response.status} ${await response.text()}`,
);
const { room_id: roomId } = (await response.json()) as { room_id: string };
return { users, roomId };
}
/**
* Opens the harness signed in as the given user and waits until the SDK
* reports the session as connected. The status line carries the SDK's error
* if it does not get there, so the failure says why.
*/
export async function startHarness(
page: Page,
user: User,
roomId: string,
): Promise<void> {
const query = new URLSearchParams({
homeserver: HOMESERVER_URL,
accessToken: user.accessToken,
userId: user.userId,
deviceId: user.deviceId,
room: roomId,
});
await page.goto(`${SDK_HARNESS_URL}/?${query.toString()}`);
await page.getByRole("button", { name: "Start" }).click();
await waitForConnected(page);
}
/**
* A login, a crypto setup and an initial sync happen before the session can
* connect. An error is final, so it is not worth waiting out the timeout for
* "connected" after one.
*/
export async function waitForConnected(page: Page): Promise<void> {
const status = page.getByTestId("status");
await expect(status).toHaveText(/^(connected|Error)/, { timeout: 120_000 });
const text = await status.textContent();
if (text !== "connected")
throw new Error(`The harness did not connect to the session: ${text}`);
}
/**
* A context of its own per user, since a browser profile holds one login. No
* permissions to grant: each browser is launched with fake media that is
* handed out without asking (see playwright.config.ts).
*/
export async function newPage(browser: Browser): Promise<Page> {
const context = await browser.newContext({ ignoreHTTPSErrors: true });
return context.newPage();
}
/** Two users in one session, each on a page of their own. */
export async function startPair(
browser: Browser,
name: string,
): Promise<{ pages: [Page, Page]; users: [User, User]; roomId: string }> {
const { users, roomId } = await createUsersAndRoom(name);
const pages = await Promise.all([newPage(browser), newPage(browser)]);
await Promise.all(
pages.map(async (page, i) => startHarness(page, users[i], roomId)),
);
return { pages: pages as [Page, Page], users, roomId };
}
/** The tile a page shows for a user. */
export function tileOf(page: Page, user: User): Locator {
return page.locator(`[data-testid="member"][data-user-id="${user.userId}"]`);
}
+67
View File
@@ -0,0 +1,67 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { startPair, tileOf, waitForConnected } from "./harness.ts";
/**
* Members coming and going: a member that leaves disappears for the peer, one
* that comes back is shown again, and the harness is left in the state its
* status line claims.
*/
test.describe.configure({ timeout: 300_000 });
test("a member that leaves disappears for the peer", async ({ browser }) => {
const {
pages: [leaver, stayer],
users: [leaving, staying],
} = await startPair(browser, "sdkleave");
await expect(stayer.getByTestId("member")).toHaveCount(2, {
timeout: 60_000,
});
await leaver.getByRole("button", { name: "Leave" }).click();
await expect(leaver.getByTestId("status")).toHaveText("Left");
await expect(leaver.getByTestId("member")).toHaveCount(0);
await expect(tileOf(stayer, leaving)).toHaveCount(0, { timeout: 60_000 });
await expect(tileOf(stayer, staying)).toHaveCount(1);
await expect(stayer.getByTestId("status")).toHaveText("connected");
});
test("a member that reloads is shown again with its media", async ({
browser,
}) => {
const {
pages: [reloader, watcher],
users: [reloading],
} = await startPair(browser, "sdkreload");
await expect(watcher.getByTestId("member")).toHaveCount(2, {
timeout: 60_000,
});
// The page comes back on the same device, so the new membership replaces
// the old one rather than sitting next to it
await reloader.reload();
await reloader.getByRole("button", { name: "Start" }).click();
await waitForConnected(reloader);
await expect(tileOf(watcher, reloading)).toHaveCount(1, {
timeout: 120_000,
});
const video = tileOf(watcher, reloading).locator("video");
await expect
.poll(async () => video.evaluate((v: HTMLVideoElement) => v.videoWidth), {
timeout: 60_000,
})
.toBeGreaterThan(0);
await expect(reloader.getByTestId("member")).toHaveCount(2, {
timeout: 60_000,
});
});
+186
View File
@@ -0,0 +1,186 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Locator, type Page, test } from "@playwright/test";
import { startPair, tileOf, type User } from "./harness.ts";
/**
* What a member's media looks like from the other side: it plays, it is
* encrypted, it follows the mute switches, and the SFU sends only as much as
* the tile on screen can show. Each test reads the labels the harness puts on
* its media elements, which come straight from the SDK's track behaviors.
*
* One session serves every test here: logging two browsers in and connecting
* them takes most of a minute, and none of the tests leaves the session in a
* state the next cannot start from.
*/
test.describe.configure({ mode: "serial", timeout: 300_000 });
let pages: [Page, Page];
let users: [User, User];
test.beforeAll(async ({ browser }) => {
({ pages, users } = await startPair(browser, "sdkmedia"));
for (const page of pages)
await expect(page.getByTestId("member")).toHaveCount(2, {
timeout: 60_000,
});
});
test.afterAll(async () => {
for (const page of pages ?? []) await page.context().close();
});
test("the peer's camera and microphone play", async () => {
const peer = remoteTile(1, 0);
const video = peer.locator("video");
await expect
.poll(async () => video.evaluate((v: HTMLVideoElement) => v.videoWidth), {
timeout: 60_000,
})
.toBeGreaterThan(0);
await expect(video).toHaveJSProperty("paused", false);
const audio = peer.locator("audio");
await expect
.poll(async () =>
audio.evaluate((a: HTMLAudioElement) => a.srcObject !== null),
)
.toBe(true);
await expect(audio).toHaveJSProperty("paused", false);
});
test("the peer's tracks are encrypted", async () => {
const peer = remoteTile(1, 0);
await expect(peer.locator("video")).toHaveAttribute("data-encrypted", "true");
await expect(peer.locator("audio")).toHaveAttribute("data-encrypted", "true");
});
test("muting the microphone and the camera is seen by the peer", async () => {
const [page] = pages;
const peer = remoteTile(1, 0);
const video = peer.locator("video");
const audio = peer.locator("audio");
await expect(video).toHaveAttribute("data-muted", "false");
await expect(audio).toHaveAttribute("data-muted", "false");
await page.getByRole("button", { name: "Microphone" }).click();
await expect(audio).toHaveAttribute("data-muted", "true", {
timeout: 20_000,
});
await expect(video).toHaveAttribute("data-muted", "false");
await page.getByRole("button", { name: "Camera" }).click();
await expect(video).toHaveAttribute("data-muted", "true", {
timeout: 20_000,
});
await page.getByRole("button", { name: "Microphone" }).click();
await page.getByRole("button", { name: "Camera" }).click();
await expect(audio).toHaveAttribute("data-muted", "false", {
timeout: 20_000,
});
await expect(video).toHaveAttribute("data-muted", "false", {
timeout: 20_000,
});
});
test("the peer's microphone is active while it sends sound, and not while muted", async () => {
// Chromium's fake microphone never registers as a speaker at the SFU, and
// Firefox's steady tone does so in about two runs out of three, with or
// without the browser's audio processing. The pipeline is proven when it
// passes; it needs a louder fake device or a lower SFU threshold to be
// relied on.
test.fixme(true, "the fake microphones do not reliably register at the SFU");
const [page] = pages;
const audio = remoteTile(1, 0).locator("audio");
// The SFU only announces a change of speakers, so a tone that has been
// playing since before the peer subscribed is not announced to it. Muting
// and unmuting is the change that is.
await page.getByRole("button", { name: "Microphone" }).click();
await expect(audio).toHaveAttribute("data-muted", "true", {
timeout: 20_000,
});
await expect(audio).toHaveAttribute("data-active", "false");
await page.getByRole("button", { name: "Microphone" }).click();
await expect(audio).toHaveAttribute("data-active", "true", {
timeout: 30_000,
});
await page.getByRole("button", { name: "Microphone" }).click();
await expect(audio).toHaveAttribute("data-active", "false", {
timeout: 20_000,
});
await page.getByRole("button", { name: "Microphone" }).click();
});
test("a larger tile receives a higher resolution", async () => {
const peer = remoteTile(1, 0);
const video = peer.locator("video");
// The tile starts small enough for the lowest simulcast layer
const small = await receivedWidth(video, (width) => width > 0);
await resize(peer, 1280);
const large = await receivedWidth(video, (width) => width > small);
expect(large).toBeGreaterThan(small);
await resize(peer, 240);
await receivedWidth(video, (width) => width < large);
});
test("a hidden tile stops receiving video, and a shown one resumes", async () => {
const peer = remoteTile(1, 0);
const video = peer.locator("video");
await expect(frames(video)).resolves.toBeGreaterThan(0);
await peer.evaluate((tile) => (tile.style.display = "none"));
await expect
.poll(async () => framesStill(video), { timeout: 60_000 })
.toBe(true);
await peer.evaluate((tile) => (tile.style.display = ""));
await expect
.poll(async () => !(await framesStill(video)), { timeout: 60_000 })
.toBe(true);
});
/** The tile on one page for the user of the other page. */
function remoteTile(viewer: 0 | 1, shown: 0 | 1): Locator {
return tileOf(pages[viewer], users[shown]);
}
async function resize(tile: Locator, width: number): Promise<void> {
await tile.evaluate((t, w) => (t.style.width = `${w}px`), width);
}
/** The width of the video as decoded, once it satisfies the condition. */
async function receivedWidth(
video: Locator,
until: (width: number) => boolean,
): Promise<number> {
// Switching simulcast layers takes the SFU a few seconds
let width = 0;
await expect
.poll(
async () => {
width = await video.evaluate((v: HTMLVideoElement) => v.videoWidth);
return until(width);
},
{ timeout: 60_000 },
)
.toBe(true);
return width;
}
async function frames(video: Locator): Promise<number> {
return Number(await video.getAttribute("data-frames"));
}
/** Whether no frame arrived over a couple of seconds. */
async function framesStill(video: Locator): Promise<boolean> {
const before = await frames(video);
await new Promise((resolve) => setTimeout(resolve, 2500));
return (await frames(video)) === before;
}
+32
View File
@@ -0,0 +1,32 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { startPair, tileOf } from "./harness.ts";
/**
* The MatrixRTC SDK driven through its harness in `sdk/dev`: no Element Call
* on the page, only the SDK's public API.
*/
// Two browsers each log in, set up crypto and sync before anything is on
// screen, then wait for media to connect
test.describe.configure({ timeout: 300_000 });
test("two browsers see each other in one session", async ({ browser }) => {
const { pages, users } = await startPair(browser, "sdksmoke");
// Each page shows itself and the other, each tile named by its user
for (const page of pages) {
await expect(page.getByTestId("member")).toHaveCount(2, {
timeout: 60_000,
});
for (const user of users)
await expect(tileOf(page, user)).toContainText(user.displayName);
}
});
+11 -1
View File
@@ -19,6 +19,7 @@ async function setupTwoUserSpaCall(
browser: Browser,
page: Page,
browserName: string,
opts: { disableGuestLeaveDelegation?: boolean } = {},
): Promise<{ guestPage: Page }> {
test.skip(
browserName === "firefox",
@@ -52,6 +53,9 @@ async function setupTwoUserSpaCall(
await guestPage.goto("/");
if (opts.disableGuestLeaveDelegation)
await SpaHelpers.disableLeaveDelegation(guestPage);
let pevaraHasSentStickyEvent = false;
const pevaraResolver = Promise.withResolvers<void>();
@@ -102,7 +106,13 @@ test("One to One rejoin after improper leave does not crash EC", async ({
page,
browserName,
}) => {
const { guestPage } = await setupTwoUserSpaCall(browser, page, browserName);
// With delegation, the backend sends the guest's delayed leave event within
// moments of the improper leave, so the stale membership this test needs
// would be cleaned up before the rejoin. Keep the guest's leave
// client-managed so the stale membership lingers.
const { guestPage } = await setupTwoUserSpaCall(browser, page, browserName, {
disableGuestLeaveDelegation: true,
});
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
+19
View File
@@ -111,6 +111,24 @@ async function setRtcModeFromSettings(
await page.getByTestId("modal_close").click();
}
/**
* Makes the delayed-leave delegation support probes fail so that the client
* manages its delayed leave event itself instead of delegating it to the
* backend.
*
* Must be installed before the page joins a call.
*/
async function disableLeaveDelegation(page: Page): Promise<void> {
// Covers both the transport probe (<livekit_service_url>/delegate_delayed_leave)
// and the homeserver probe (MSC4195, .../rtc/livekit/delegate_delayed_leave).
await page.route("**/delegate_delayed_leave", async (route) =>
route.fulfill({
status: 404,
headers: { "Access-Control-Allow-Origin": "*" },
}),
);
}
/**
* Expect a certain number of video tiles to be present and visible.
*/
@@ -133,5 +151,6 @@ export const SpaHelpers = {
createCall,
getCallInviteLink,
joinCallFromInviteLink,
disableLeaveDelegation,
expectVideoTilesCount,
};
+493
View File
@@ -0,0 +1,493 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Page, test } from "@playwright/test";
import { SpaHelpers } from "./spa-helpers";
import { type RtcMode } from "./widget/test-helpers.ts";
const sfuUrl = "wss://matrix-rtc.m.localhost/livekit/sfu";
const serviceUrl = "https://matrix-rtc.m.localhost/livekit/jwt";
const otherHomeserver = {
base_url: "https://synapse.othersite.m.localhost",
server_name: "synapse.othersite.m.localhost",
};
const otherSfuUrl = "wss://matrix-rtc.othersite.m.localhost/livekit/sfu";
interface Transport {
type: "livekit";
url?: string;
livekit_service_url?: string;
}
/**
* The transport shapes a homeserver may advertise while migrating to the
* latest MSC4195. `legacy` is what the e2e Synapse advertises on its own.
*/
const shapes = {
url: { type: "livekit", url: sfuUrl },
both: { type: "livekit", url: sfuUrl, livekit_service_url: serviceUrl },
legacy: null,
} satisfies Record<string, Transport | null>;
type Shape = keyof typeof shapes;
/** Which endpoint a page asks for its SFU tokens. */
type Flow = "cs-api" | "service-legacy" | "service-default";
const cases: {
hostShape: Shape;
guestShape: Shape;
/** The MatrixRTC mode both sides use; the default is compatibility mode. */
mode?: RtcMode;
hostFlows: Flow[];
guestFlows: Flow[];
}[] = [
{
hostShape: "url",
guestShape: "url",
hostFlows: ["cs-api"],
guestFlows: ["cs-api"],
},
{
// Publishers keep the legacy participant identity that clients predating
// MSC4195 expect, which only the JWT service assigns.
hostShape: "both",
guestShape: "both",
hostFlows: ["service-legacy"],
guestFlows: ["service-legacy"],
},
{
// The MatrixRTC mode has no bearing on the choice.
hostShape: "both",
guestShape: "both",
mode: "2_0",
hostFlows: ["service-legacy"],
guestFlows: ["service-legacy"],
},
{
hostShape: "url",
guestShape: "legacy",
hostFlows: ["cs-api", "service-default"],
guestFlows: ["service-legacy", "cs-api"],
},
];
for (const { hostShape, guestShape, mode, hostFlows, guestFlows } of cases) {
test(`One to one call with ${hostShape} host and ${guestShape} guest transport${mode === "2_0" ? " in Matrix 2.0 mode" : ""}`, async ({
browser,
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host starts the call.
await advertiseTransport(page, shapes[hostShape]);
const hostRequests = recordAuthRequests(page);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true, mode);
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// The guest joins.
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guestPage = await guestContext.newPage();
await advertiseTransport(guestPage, shapes[guestShape]);
const guestRequests = recordAuthRequests(guestPage);
await SpaHelpers.joinCallFromInviteLink(
guestPage,
inviteLink,
"Pevara",
mode,
);
// Wait for the call to connect and render.
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
// The host and guest ran through the expected flows.
expect(flowsOf(hostRequests)).toEqual(new Set(hostFlows));
expect(flowsOf(guestRequests)).toEqual(new Set(guestFlows));
// All requests succeeded and the SFU URL was set on C-S requests.
for (const request of [...hostRequests, ...guestRequests]) {
expect(request.status).toBe(200);
if (request.flow === "cs-api") expect(request.sfuUrl).toBe(sfuUrl);
// Clients predating MSC4195 match a publisher by the legacy identity,
// which only the JWT service's legacy endpoint assigns. The other flows
// hash it.
expect(request.identity).toMatch(
request.flow === "service-legacy" ? legacyIdentity : hashedIdentity,
);
}
});
}
test("Subscriber falls back to the JWT service when the homeserver lacks MSC4195", async ({
browser,
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host starts the call with both transport shapes.
await advertiseTransport(page, shapes.both);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true);
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// The guest joins but their C-S requests fail.
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guestPage = await guestContext.newPage();
await rejectCsApiTokens(guestPage);
const guestRequests = recordAuthRequests(guestPage);
await SpaHelpers.joinCallFromInviteLink(guestPage, inviteLink, "Pevara");
// Wait for the call to connect and render.
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
// The guest ran through all flows.
expect(flowsOf(guestRequests)).toEqual(
new Set<Flow>(["cs-api", "service-legacy", "service-default"]),
);
// The C-S token requests failed.
for (const request of guestRequests) {
expect(request.status).toBe(request.flow === "cs-api" ? 404 : 200);
}
// The C-S token requests preceeded the fallback request.
const flows = guestRequests.map((r) => r.flow);
expect(flows.indexOf("service-default")).toBeGreaterThan(
flows.indexOf("cs-api"),
);
});
test("Publisher shows an error when the homeserver lacks MSC4195", async ({
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host tries to start a call with only the C-S transport shape available but
// their C-S requests fail.
await advertiseTransport(page, shapes.url);
await rejectCsApiTokens(page);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true);
// We land on an error page.
await expect(
page.getByRole("heading", { name: "Something went wrong" }),
).toBeVisible();
await expect(
page.getByText(
"The authorization service for your media server (SFU) is out of date.",
),
).toBeVisible();
});
/**
* Delegation follows the flow that authenticated the publisher: the homeserver
* for a `url`-only transport, the JWT service wherever there is one.
*/
const delegationCases: { shape: Shape; via: Delegation["via"] }[] = [
{ shape: "url", via: "homeserver" },
{ shape: "legacy", via: "service" },
{ shape: "both", via: "service" },
];
for (const { shape, via } of delegationCases) {
test(`Delayed leave event delegated through the ${via} for a ${shape} transport gets sent on disconnect`, async ({
browser,
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host starts the call.
await advertiseTransport(page, shapes[shape]);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true);
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// The guest joins.
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guestPage = await guestContext.newPage();
await advertiseTransport(guestPage, shapes[shape]);
const delayedLeave = recordDelayedLeave(guestPage);
const delegations = recordDelegations(guestPage);
await SpaHelpers.joinCallFromInviteLink(guestPage, inviteLink, "Pevara");
// Wait for the call to connect and render.
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
// The guest handed its delayed leave over, after probing for delegation
// support with bodiless requests, which count as supported unless they
// 404. The homeserver is always probed; the JWT service only when there
// is one.
await expect
.poll(() => delegations.at(-1))
.toEqual({
via,
status: 200,
delayId: delayedLeave.delayId,
...(via === "homeserver" && { sfuUrl }),
});
const probes = delegations.slice(0, -1);
expect(probes.some((probe) => probe.via === via)).toBe(true);
for (const probe of probes) {
expect(probe.status).toBeGreaterThanOrEqual(400);
expect(probe.status).not.toBe(404);
}
// The delay is long enough that a leave within the window below cannot
// come from the delayed event expiring on its own.
expect(delayedLeave.delayMs).toBeGreaterThan(60_000);
// The guest drops without leaving.
await guestPage.reload();
// The host eventually sees the guest go once the delayed event fires.
await expect(page.getByTestId("videoTile")).toHaveCount(1, {
timeout: 30_000,
});
});
}
test("Subscribers get tokens for a remote homeserver's SFU over federation", async ({
browser,
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host lives on the other homeserver and publishes on its SFU.
await useHomeserver(page, otherHomeserver);
await advertiseTransport(page, { type: "livekit", url: otherSfuUrl });
const hostRequests = recordAuthRequests(page);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true);
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// The guest lives on the default homeserver and joins over federation.
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guestPage = await guestContext.newPage();
await advertiseTransport(guestPage, shapes.url);
const guestRequests = recordAuthRequests(guestPage);
await SpaHelpers.joinCallFromInviteLink(
guestPage,
`${inviteLink}&viaServers=${otherHomeserver.server_name}`,
"Pevara",
);
// Wait for the call to connect and render.
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
// Each side got a token for its own SFU and for the other side's SFU, all
// through its own homeserver.
for (const requests of [hostRequests, guestRequests]) {
expect(flowsOf(requests)).toEqual(new Set<Flow>(["cs-api"]));
expect(new Set(requests.map((r) => r.sfuUrl))).toEqual(
new Set([sfuUrl, otherSfuUrl]),
);
for (const request of requests) expect(request.status).toBe(200);
}
});
/** Skips the calling test if we're running on Firefox. */
function skipOnFirefox(browserName: string): void {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
}
/** Points this page's Element Call at another homeserver. */
async function useHomeserver(
page: Page,
homeserver: typeof otherHomeserver,
): Promise<void> {
await page.route("**/config.json", async (route) => {
const response = await route.fetch();
const config = (await response.json()) as Record<string, unknown>;
await route.fulfill({
response,
json: {
...config,
default_server_config: { "m.homeserver": homeserver },
},
});
});
}
/**
* Maps recorded requests to a set of flows. Publisher and subscriber connections
* start concurrently so using arrays for comparison isn't convenient.
*/
function flowsOf(requests: AuthRequest[]): Set<Flow> {
return new Set(requests.map((r) => r.flow));
}
/**
* Replaces the transports the homeserver advertises to this page. The real
* response is fetched first so its CORS headers carry over.
*/
async function advertiseTransport(
page: Page,
transport: Transport | null,
): Promise<void> {
if (transport === null) return;
await page.route("**/org.matrix.msc4143/rtc/transports", async (route) => {
if (route.request().method() !== "GET") return route.continue();
const response = await route.fetch();
await route.fulfill({ response, json: { rtc_transports: [transport] } });
});
}
/**
* Makes the homeserver look like one without MSC4195: its `get_token`
* endpoint answers as Synapse does for an unknown route.
*/
async function rejectCsApiTokens(page: Page): Promise<void> {
await page.route("**/rtc/livekit/get_token", async (route) => {
if (route.request().method() !== "POST") return route.continue();
await route.fulfill({
status: 404,
json: { errcode: "M_UNRECOGNIZED", error: "Unrecognized request" },
headers: { "Access-Control-Allow-Origin": "*" },
});
});
}
/** A recorded delayed leave event. */
interface DelayedLeave {
delayMs?: number;
delayId?: string;
}
/** Records the delayed leave event the page schedules on joining. */
function recordDelayedLeave(page: Page): DelayedLeave {
const delayedLeave: DelayedLeave = {};
page.on("response", (response) => {
const delay = new URL(response.request().url()).searchParams.get(
"org.matrix.msc4140.delay",
);
if (delay === null || response.status() !== 200) return;
delayedLeave.delayMs = Number(delay);
response
.json()
.then((body: { delay_id: string }) => {
delayedLeave.delayId = body.delay_id;
})
.catch(() => {});
});
return delayedLeave;
}
/** A recorded delegation request for a delayed leave event. */
interface Delegation {
/** Who was asked to send the leave event on the guest's behalf. */
via: "homeserver" | "service";
status: number;
sfuUrl?: string;
delayId?: string;
}
/**
* Records requests that delegate a delayed leave event, or probe for support:
* the MSC4195 `delegate_delayed_leave` endpoint of the homeserver, and the JWT
* service's `delegate_delayed_leave` probe and `/sfu/get` with delay fields.
*/
function recordDelegations(page: Page): Delegation[] {
const delegations: Delegation[] = [];
page.on("response", (response) => {
const request = response.request();
if (request.method() !== "POST") return;
const url = request.url();
const body = request.postDataJSON() as {
url?: string;
delay_id?: string;
} | null;
if (url.endsWith("/rtc/livekit/delegate_delayed_leave")) {
delegations.push({
via: "homeserver",
status: response.status(),
...(body && { sfuUrl: body.url, delayId: body.delay_id }),
});
} else if (url.endsWith("/livekit/jwt/delegate_delayed_leave")) {
delegations.push({ via: "service", status: response.status() });
} else if (url.endsWith("/livekit/jwt/sfu/get") && body?.delay_id) {
delegations.push({
via: "service",
status: response.status(),
delayId: body.delay_id,
});
}
});
return delegations;
}
/** A recorded token request. */
interface AuthRequest {
flow: Flow;
status: number;
/** The SFU a request to the MSC4195 `get_token` endpoint asked for. */
sfuUrl?: string;
/** The participant identity the issued token carries. */
identity?: string;
}
/** The legacy `<user_id>:<device_id>` participant identity. */
const legacyIdentity = /^@[^:]+:[^:]+:[A-Z]+$/;
/** A hashed participant identity: unpadded base64 of a SHA-256. */
const hashedIdentity = /^[A-Za-z0-9+/]{43}$/;
/** Reads the subject of a JWT without verifying it. */
function jwtSubject(jwt: string): string {
const [, payload] = jwt.split(".");
return (
JSON.parse(Buffer.from(payload, "base64url").toString()) as {
sub: string;
}
).sub;
}
/** Records token requests, including the identity each token carries. */
function recordAuthRequests(page: Page): AuthRequest[] {
const requests: AuthRequest[] = [];
page.on("response", (response) => {
const request = response.request();
if (request.method() !== "POST") return;
const url = request.url();
let entry: AuthRequest;
if (url.endsWith("/rtc/livekit/get_token")) {
entry = {
flow: "cs-api",
status: response.status(),
sfuUrl: (request.postDataJSON() as { url: string }).url,
};
} else if (url.endsWith("/livekit/jwt/sfu/get")) {
entry = { flow: "service-legacy", status: response.status() };
} else if (url.endsWith("/livekit/jwt/get_token")) {
entry = { flow: "service-default", status: response.status() };
} else {
return;
}
requests.push(entry);
if (entry.status !== 200) return;
response
.json()
.then((body: { jwt: string }) => {
entry.identity = jwtSubject(body.jwt);
})
.catch(() => {});
});
return requests;
}
+85
View File
@@ -0,0 +1,85 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type Page } from "@playwright/test";
/**
* Adds synthetic devices beside the browser's one fake microphone and speaker,
* so the menu has a choice to show. They share one capture and `setSinkId` is
* accepted but not honoured: routing still needs hardware and a manual check.
* Call before the page navigates.
*/
export async function installFakeDevices(
page: Page,
{ microphones = 2, speakers = 2 } = {},
): Promise<void> {
await page.addInitScript(
({ microphones, speakers }) => {
const synthetic = (
kind: MediaDeviceKind,
count: number,
name: string,
): MediaDeviceInfo[] =>
Array.from({ length: count }, (_, i) => {
const info = {
deviceId: `${kind}-${i + 1}`,
groupId: `${kind}-group-${i + 1}`,
kind,
label: `${name} ${i + 1}`,
};
return { ...info, toJSON: () => info } as MediaDeviceInfo;
});
const ids = new Set(
[
...synthetic("audioinput", microphones, ""),
...synthetic("audiooutput", speakers, ""),
].map((d) => d.deviceId),
);
const devices = navigator.mediaDevices;
const enumerate = devices.enumerateDevices.bind(devices);
devices.enumerateDevices = async (): Promise<MediaDeviceInfo[]> => [
...(await enumerate()),
...synthetic("audioinput", microphones, "Fake Microphone"),
...synthetic("audiooutput", speakers, "Fake Speaker"),
];
// Our ids name no hardware, so drop the exact-device constraint.
const getUserMedia = devices.getUserMedia.bind(devices);
devices.getUserMedia = async (
constraints?: MediaStreamConstraints,
): Promise<MediaStream> => {
const audio = constraints?.audio;
if (typeof audio === "object") {
const requested = audio.deviceId;
const id =
typeof requested === "object" && requested !== null
? ((requested as ConstrainDOMStringParameters).exact as string)
: (requested as string | undefined);
if (id !== undefined && ids.has(id))
return getUserMedia({ ...constraints, audio: true });
}
return getUserMedia(constraints);
};
// Accept routing to our ids on both media elements and the AudioContext,
// which the app also routes.
for (const proto of [
HTMLMediaElement.prototype,
AudioContext.prototype,
]) {
const sink = proto as { setSinkId?: (id: string) => Promise<void> };
const setSinkId = sink.setSinkId;
if (setSinkId === undefined) continue;
sink.setSinkId = async function (id: string): Promise<void> {
if (!ids.has(id)) await setSinkId.call(this, id);
};
}
},
{ microphones, speakers },
);
}
-4
View File
@@ -37,8 +37,6 @@ modePairs.forEach(([rtcMode1, rtcMode2]) => {
await TestHelpers.acceptRoomInvite(roomName, timo.page);
await florian.page.pause();
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
florian.page,
rtcMode1,
@@ -74,8 +72,6 @@ modePairs.forEach(([rtcMode1, rtcMode2]) => {
await TestHelpers.expectVisibleVideoCount(frame, 2);
}
// await florian.page.pause();
},
);
});
+6 -6
View File
@@ -76,9 +76,9 @@ widgetTest("Sharing screen in group call", async ({ addUser, browserName }) => {
timeout: 5000,
});
await expect(
frame.locator('video[data-lk-source="screen_share"]'),
).toHaveCount(1);
await expect(frame.locator('video[data-source="screenShare"]')).toHaveCount(
1,
);
}
// Alice should be in grid mode as she is local sharing
@@ -117,9 +117,9 @@ widgetTest("Sharing screen in group call", async ({ addUser, browserName }) => {
timeout: 5000,
});
await expect(
frame.locator('video[data-lk-source="screen_share"]'),
).toHaveCount(2);
await expect(frame.locator('video[data-source="screenShare"]')).toHaveCount(
2,
);
// Expect 2 indicators at the bottom
await expect(frame.getByTestId("screenshare-indicator")).toHaveCount(2);
+28 -5
View File
@@ -11,6 +11,7 @@ import {
type JSHandle,
type Page,
type FrameLocator,
type Locator,
} from "@playwright/test";
import { type MatrixClient } from "matrix-js-sdk";
@@ -203,6 +204,16 @@ export class TestHelpers {
}
}
public static async expandAllSections(page: Page): Promise<void> {
try {
await page
.getByRole("button", { name: "Expand all sections" })
.click({ timeout: 2000 });
} catch {
// Already expanded or button not present
}
}
public static async createRoom(
name: string,
page: Page,
@@ -212,7 +223,7 @@ export class TestHelpers {
await page
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.getByRole("button", { name: "New", exact: true })
.click();
await page.getByRole("menuitem", { name: "New Room" }).click({
@@ -253,9 +264,9 @@ export class TestHelpers {
roomName: string,
page: Page,
): Promise<void> {
await page.getByRole("option", { name: roomName }).click({
timeout: 10000,
});
await TestHelpers.closeReleaseAnnouncement(page, "Introducing Sections");
await TestHelpers.expandAllSections(page);
await TestHelpers.roomListItem(page, roomName).click({ timeout: 10000 });
await page.getByRole("button", { name: "Accept" }).click({
timeout: 5000,
});
@@ -350,6 +361,17 @@ export class TestHelpers {
.click();
}
/**
* Locates a room in the room list by its name.
*
* Matches on the aria-label prefix because the item's role differs between
* the flat room list (`option`) and the sectioned room list (`button`), and
* the label may carry a suffix such as " invitation.".
*/
public static roomListItem(page: Page, roomName: string): Locator {
return page.locator(`[aria-label^="Open room ${roomName}"]`);
}
/**
* Switches to a room in the room list by its name.
* @param page - The EW page
@@ -359,7 +381,8 @@ export class TestHelpers {
page: Page,
roomName: string,
): Promise<void> {
await page.getByRole("option", { name: `Open room ${roomName}` }).click();
await TestHelpers.expandAllSections(page);
await TestHelpers.roomListItem(page, roomName).click();
}
public static async dismissInviteUnknownUserModal(page: Page): Promise<void> {
+104
View File
@@ -0,0 +1,104 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Page, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user.ts";
import { HOST1, TestHelpers } from "./test-helpers.ts";
const sfuUrl = "wss://matrix-rtc.m.localhost/livekit/sfu";
const serviceUrl = "https://matrix-rtc.m.localhost/livekit/jwt";
/** Which endpoint a page asks for its SFU tokens. */
type Flow = "cs-api" | "service-legacy" | "service-default";
widgetTest(
"Call between a url + livekit_service_url transport and a legacy one",
async ({ addUser, browserName }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
test.slow();
const [host, guest] = await Promise.all([
addUser("Androl", HOST1),
addUser("Pevara", HOST1),
]);
// The host's homeserver has started advertising a `url` next to its JWT
// service; the guest's still advertises the JWT service alone.
await advertiseTransport(host.page, {
type: "livekit",
url: sfuUrl,
livekit_service_url: serviceUrl,
});
// Element Web answers the widget from a cache it filled at login, so make
// it fetch the transports again now that they are rerouted
await host.clientHandle.evaluate(async (client) => {
await client.cachedRtcTransports.refresh();
});
const hostFlows = recordFlows(host.page);
const guestFlows = recordFlows(guest.page);
const roomName = "Transport Room";
await TestHelpers.createRoom(roomName, host.page, [guest.mxId]);
await TestHelpers.acceptRoomInvite(roomName, guest.page);
await TestHelpers.startCallInCurrentRoom(host.page, false);
await TestHelpers.joinCallFromLobby(host.page);
await TestHelpers.joinCallInCurrentRoom(guest.page);
for (const user of [host, guest]) {
const frame = user.page
.locator('iframe[title="Element Call"]')
.contentFrame();
await expect(frame.getByTestId("videoTile")).toHaveCount(2, {
timeout: 10000,
});
await expect(frame.getByText("Waiting for media...")).not.toBeVisible({
timeout: 10000,
});
await TestHelpers.expectVisibleVideoCount(frame, 2);
}
// Element Web used for Playwright does support MSC4195 for widgets.
expect(hostFlows).toEqual(new Set(["service-legacy", "service-default"]));
expect(guestFlows).toEqual(new Set(["service-legacy", "cs-api"]));
},
);
/**
* Replaces the transports the homeserver advertises to this page, including
* to the widget host, which fetches them on the widget's behalf. The real
* response is fetched first so its CORS headers carry over.
*/
async function advertiseTransport(
page: Page,
transport: { type: "livekit"; url?: string; livekit_service_url?: string },
): Promise<void> {
await page.route("**/org.matrix.msc4143/rtc/transports", async (route) => {
if (route.request().method() !== "GET") return route.continue();
const response = await route.fetch();
await route.fulfill({ response, json: { rtc_transports: [transport] } });
});
}
/** Records which token endpoints the page's widget asks. */
function recordFlows(page: Page): Set<Flow> {
const flows = new Set<Flow>();
page.on("response", (response) => {
const request = response.request();
if (request.method() !== "POST") return;
const url = request.url();
if (url.endsWith("/rtc/livekit/get_token")) flows.add("cs-api");
else if (url.endsWith("/livekit/jwt/sfu/get")) flows.add("service-legacy");
else if (url.endsWith("/livekit/jwt/get_token"))
flows.add("service-default");
});
return flows;
}
+1515 -1585
View File
File diff suppressed because it is too large Load Diff
+4
View File
@@ -1,3 +1,7 @@
supportedArchitectures:
os: [current, linux]
cpu: [current, arm64]
libc: [current, glibc]
minimumReleaseAgeExclude:
- "@vector-im/compound-design-tokens"
- "@vector-im/compound-web"
+162
View File
@@ -0,0 +1,162 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* Checks that the library builds leave the packages a host must supply to the
* host.
*
* A host application already has React, the Matrix SDK and Compound,
* and a second copy of any of them is worse than dead weight: React would hold
* two sets of hooks, the Matrix client would run two sync loops, and a second
* Compound would style the tooltips it floats into the host's body with class
* names the host's stylesheet does not know. So each library build lists them
* as external — but that list has to name every subpath, since the bundler
* silently ignores the pattern and callback forms of the option, and an import
* it does not cover is bundled with no warning at all. That is the failure
* this guards against.
*
* It reads each list from the build config itself, so there is one copy of it
* per build, and compares it against every import of those packages in the
* source.
*
* The comparison is deliberately over-approximate: it looks at all of `src`
* rather than only the modules a build actually pulls in, so it will sometimes
* ask for a subpath that only the standalone app imports. Listing one a build
* never imports costs nothing — the bundler ignores it — whereas missing one
* costs a duplicate package.
*/
import { readdir, readFile } from "node:fs/promises";
import { join } from "node:path";
import { loadConfigFromFile } from "vite";
/**
* The library builds, each with the directories it is built from and the
* packages whose duplication would break a host rather than merely enlarge it.
* A development harness is a host, not part of the library: it is the one that
* imports what a host supplies (Compound's stylesheets, say).
*/
const TARGETS = [
{
config: "vite-component.config.ts",
what: "the component build",
sources: ["src", "component"],
excluded: ["component/dev"],
mustBeExternal: [
"react",
"react-dom",
"matrix-js-sdk",
"@vector-im/compound-web",
"@vector-im/compound-design-tokens",
],
},
{
config: "vite-sdk.config.ts",
what: "the SDK build",
sources: ["src", "sdk"],
excluded: ["sdk/dev"],
mustBeExternal: ["matrix-js-sdk", "livekit-client", "rxjs"],
},
];
const isTestFile = (name) =>
name.includes(".test.") || name.includes(".stories.");
/** Every source file under the given directories, recursively. */
async function* sourceFiles(dir, excluded) {
for (const entry of await readdir(dir, { withFileTypes: true })) {
const path = join(dir, entry.name);
if (excluded.includes(path)) continue;
if (entry.isDirectory()) yield* sourceFiles(path, excluded);
else if (/\.(ts|tsx)$/.test(entry.name) && !isTestFile(entry.name))
yield path;
}
}
/**
* The module specifiers a source file imports. Covers `from "…"` (which is
* both static imports and re-exports), bare `import "…"` for side effects, and
* dynamic `import("…")`.
*/
function imports(source) {
const specifiers = [];
for (const pattern of [
/\bfrom\s*["']([^"']+)["']/g,
/\bimport\s*\(\s*["']([^"']+)["']\s*\)/g,
/^\s*import\s+["']([^"']+)["']/gm,
])
for (const [, specifier] of source.matchAll(pattern))
specifiers.push(specifier);
return specifiers;
}
/**
* Whether a specifier is an import of one of the packages we care about.
*
* Imports carrying a resource query — `?worker`, `?inline` and friends — are
* not, whatever package they name. Those ask the bundler for a script to run
* in a context of its own, which has to be self-contained and shares no state
* with the host's copy of anything. Worker sub-builds do not inherit this
* option anyway.
*/
const mustBeExternal = (specifier, packages) =>
!specifier.includes("?") &&
packages.some((pkg) => specifier === pkg || specifier.startsWith(`${pkg}/`));
/** @returns Whether the target's externals list covers its imports. */
async function check({
config,
what,
sources,
excluded,
mustBeExternal: packages,
}) {
const loaded = await loadConfigFromFile(
{ command: "build", mode: "production" },
config,
);
if (loaded === null) {
console.error(`Could not load ${config}`);
return false;
}
const declared = new Set(loaded.config.build?.rollupOptions?.external ?? []);
if (declared.size === 0) {
console.error(
`${config} declares nothing external. Either the option moved, or the ` +
`list is empty; either way this check is not looking at what it thinks.`,
);
return false;
}
// Where each missing specifier is imported, so the message can point at it
const missing = new Map();
for (const dir of sources)
for await (const file of sourceFiles(dir, excluded)) {
const source = await readFile(file, "utf8");
for (const specifier of imports(source)) {
if (!mustBeExternal(specifier, packages)) continue;
if (declared.has(specifier)) continue;
const files = missing.get(specifier) ?? [];
files.push(file);
missing.set(specifier, files);
}
}
if (missing.size === 0) return true;
console.error(
`${config} does not declare these imports external, so ${what} ` +
`would bundle its own copy of them:\n`,
);
for (const [specifier, files] of [...missing].sort())
console.error(` ${specifier}\n imported by ${files.join(", ")}`);
console.error(`\nAdd each one to the \`external\` list in ${config}.\n`);
return false;
}
let ok = true;
for (const target of TARGETS) ok = (await check(target)) && ok;
if (!ok) process.exit(1);
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env bash
# Clones the branch of a dependency's repository that has the same name as
# the pull request branch under test, so a change spanning both repositories
# can be tested before either side is merged. Inspired by scripts/fetchdep.sh
# element-web.
#
# Usage: scripts/fetchdep.sh <org> <repo> [ref]
#
# The clone lands in ./<repo>. When the pull request comes from a fork and
# PR_HEAD_REPO_OWNER was set, the fork's own copy of the repository is tried
# before <org>'s. Exits 0 when a branch was cloned and 1 when there is none,
# so the caller can keep the locked version in that case. Without a ref,
# nothing is closed outside a pull request.
set -eu
org=$1
repo=$2
ref=${3:-}
rm -rf "$repo"
# If a ref was passed, try to clone it or die.
if [ -n "$ref" ]; then
echo "fetchdep.sh: Using $org/$repo at $ref"
git clone --depth 1 "https://github.com/$org/$repo.git" "$repo"
git -C "$repo" fetch --depth 1 origin "$ref"
git -C "$repo" -c advice.detachedHead=false checkout FETCH_HEAD
git -C "$repo" log -1
exit 0
fi
# Otherwise ensure that we're on a pull request.
head_ref=${GITHUB_HEAD_REF:-}
if [ -z "$head_ref" ]; then
echo "fetchdep.sh: Not a pull request, nothing to match"
exit 1
fi
# Bots name their branches the same way in every repository, so a match on
# one of theirs would be a coincidence rather than a paired change.
case "$head_ref" in
renovate/* | dependabot/*)
echo "fetchdep.sh: Not matching bot branch $head_ref"
exit 1
;;
esac
# Clones a branch if it exists and ends the script on success.
clone() {
local try_org=$1
local branch=$2
echo "fetchdep.sh: Trying to use $try_org/$repo#$branch"
# Disable auth prompts: https://serverfault.com/a/665959
if GIT_TERMINAL_PROMPT=0 git clone "https://github.com/$try_org/$repo.git" "$repo" --branch "$branch" --depth 1; then
git -C "$repo" log -1
# Surface the match on the workflow run summary.
echo "::notice title=Matched $repo branch::Using $try_org/$repo#$branch at $(git -C "$repo" rev-parse HEAD)"
exit 0
fi
}
# Try cloning from the fork, then from the org.
head_owner=${PR_HEAD_REPO_OWNER:-}
if [ -n "$head_owner" ] && [ "$head_owner" != "${GITHUB_REPOSITORY_OWNER:-}" ]; then
clone "$head_owner" "$head_ref"
fi
clone "$org" "$head_ref"
# Nothing worked. Die.
echo "fetchdep.sh: No branch named $head_ref in $org/$repo"
exit 1
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Installs dependencies, then swaps matrix-js-sdk for a branch of the same
# name as the pull request branch under test if one exists. This is what
# lets CI test an Element Call branch together with a matrix-js-sdk branch
# before either is merged. Inspired by scripts/layered.sh in element-web.
#
# Usage: scripts/layered.sh [pnpm install flags]
#
# The layering is only performed when package.json uses the develop branch of
# matrix-js-sdk and the run is for a pull request whose branch has a matrix-js-sdk
# counterpart. Otherwise the dependency is left as pnpm-lock.yaml pins it.
#
# The matrix-js-sdk checkout is nested inside this directory because some CI
# systems do not allow moving above the primary checkout. It is git-ignored.
#
# Set JS_SDK_GITHUB_BASE_REF to a ref to use that exact ref of matrix-js-sdk
# instead of looking for a matching branch.
#
# This script is only meant to be used in CI. For local development use
# see docs/linking.md.
set -ex
# Paths below are relative to the repository root.
cd "$(dirname "$0")/.."
pnpm install --frozen-lockfile "$@"
js_sdk_dep=$(jq -r '.devDependencies["matrix-js-sdk"]' < package.json)
# Only layer when package.json follows the branch head rather than pinning a
# release, so a release build keeps its pinned version.
if [ "$js_sdk_dep" != "github:matrix-org/matrix-js-sdk#develop" ]; then
echo "layered.sh: Skipping matrix-js-sdk fetch and install as package.json pins $js_sdk_dep"
exit 0
fi
if ! scripts/fetchdep.sh matrix-org matrix-js-sdk "${JS_SDK_GITHUB_BASE_REF:-}"; then
echo "layered.sh: No matching matrix-js-sdk branch, keeping the locked version"
exit 0
fi
# matrix-js-sdk pins its pnpm version in `devEngines` only. Under corepack,
# pnpm does not switch versions on its own and corepack, finding no
# `packageManager` field there, would walk up to this project's and use
# ours. Pinning it in the clone makes every pnpm invocation inside that
# directory, including the `prepare` script run by pack, use the right one.
js_sdk_pnpm="pnpm@$(jq -r '.devEngines.packageManager.version' < matrix-js-sdk/package.json)"
jq --arg pm "$js_sdk_pnpm" '.packageManager = $pm' matrix-js-sdk/package.json > matrix-js-sdk/package.json.tmp
mv matrix-js-sdk/package.json.tmp matrix-js-sdk/package.json
export COREPACK_ENABLE_DOWNLOAD_PROMPT=0
# Install matrix-js-sdk's build dependencies. `prepare` runs on pack below,
# so it is skipped here.
(cd matrix-js-sdk && pnpm install --frozen-lockfile --ignore-scripts)
# Rather than `pnpm link`, build matrix-js-sdk into a tarball and install
# that. This better reflects a clean install: dependencies may be hoisted
# and shared, and there is no tsconfig.json to pick up sources from.
(cd matrix-js-sdk && pnpm pack --out matrix-js-sdk.tgz)
pnpm install "$(pwd)/matrix-js-sdk/matrix-js-sdk.tgz" "$@"
@@ -0,0 +1,68 @@
# SDK mode (LEGACY)
> Superseded by the `sdk/` package (`@element-hq/matrixrtc-sdk`) and its harness in
> `sdk/dev`; see `sdk/SdkArchitecture.md`. This bundle is in production use, so it
> keeps building as a thin layer over that package until its consumers have moved
> to the package directly. It will then be removed. Its members are the package's
> `RTCMember`s now, in place of its own wrapper around the raw membership.
EC can be build in sdk mode. This will result in a compiled js file that can be imported in very simple webapps.
It allows to use matrixRTC in combination with livekit without relying on element call.
This is done by instantiating the call view model and exposing some useful behaviors (observables) and methods.
This folder contains an example index.html file that showcases the sdk in use (hosted on localhost:8123 with a webserver allowing cors (for example `npx serve -l 81234 --cors`)) as a godot engine HTML export template.
## Getting started
To get started run
```
pnpm install
pnpm build:sdk-target-based-on-call-view-model
```
in the repository root. (`pnpm build:sdk` builds the `sdk/` package instead.)
CI's `sdk` package (`build:sdk:production`, `build:sdk:development`, the
`build-output-sdk` artifact and the `pr<N>-sdk` Netlify preview) is still this
bundle, not the `sdk/` package, until its consumers have moved over.
It will create `dist` containing the compiled js file, `matrixrtc-sdk.js`.
This file needs to be hosted. Locally (via `npx serve -l 81234 --cors`) or on a remote server.
Now you just need to add the widget to element web via:
```
/addwidget http://localhost:3000?widgetId=$matrix_widget_id&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
```
## Widgets
The sdk mode is particularly interesting to be used in widgets. In widgets you do not need to pay attention to matrix login/cs api ...
To create a widget see the example `index.html` file in this folder. And add it to EW via:
`/addwidget <widgetUrl>` (see **url parameters** for more details on `<widgetUrl>`)
### url parameters
The url parameters are needed to pass initial data to the widget. They will automatically be used
by the matrixRTCSdk to start the postmessage widget api (communication between the client (e.g. Element Web) and the widget)
```
widgetId = $matrix_widget_id
userId = $matrix_user_id
deviceId = $org.matrix.msc3819.matrix_device_id
baseUrl = $org.matrix.msc4039.matrix_base_url
```
`parentUrl = // will be inserted automatically`
Full template use as `<widgetUrl>`:
```
http://localhost:3000?widgetId=$matrix_widget_id&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
```
the `$` prefixed variables will be replaced by EW on widget instantiation. (e.g. `$matrix_user_id` -> `@user:example.com` (url encoding will also be applied automatically by EW) -> `%40user%3Aexample.com`)
@@ -30,12 +30,12 @@
div.appendChild(child);
});
sdk.members$.subscribe((memberObjects) => {
sdk.remoteMembers$.subscribe((memberObjects) => {
const div = document.getElementById("members");
div.innerHTML = "<h3>Members:</h3>";
// Create member list
const members = memberObjects.map((member) => member.membership.sender);
const members = memberObjects.map((member) => member.userId);
console.info("members changed", members);
for (const m of members) {
console.info("member", m);
+191
View File
@@ -0,0 +1,191 @@
/*
Copyright 2025-2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* LEGACY, in production use
*
* The entrypoint of `pnpm build:sdk-target-based-on-call-view-model`, a
* bundle for widgets that exposes `createMatrixRTCSdk`: join a slot and
* exchange realtime data, with the `@element-hq/matrixrtc-sdk` package and
* Element Call's call view model doing the work underneath. New consumers use
* the package directly; this bundle goes away once the existing ones have.
*/
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { EMPTY, filter, map, type Observable, of, switchMap, take } from "rxjs";
import {
type Behavior,
createRTCSlot,
E2eeType,
type LocalRTCMember,
ObservableScope,
type RemoteRTCMember,
type RTCParticipation,
} from "@element-hq/matrixrtc-sdk";
import {
callViewModelOptionsFromParams,
type CallViewModel,
createCallViewModel$,
initialPublishRequests,
} from "../src/state/CallViewModel/CallViewModel";
import {
captureSettings,
rtcParticipationOptions,
rtcSlotOptions,
} from "../src/room/InCallView";
import { createWidgetHostBridge } from "../src/HostBridge";
import { MediaDevices } from "../src/state/MediaDevices";
import { MuteStates } from "../src/state/MuteStates";
import { getUrlParams } from "../src/UrlParams";
import { observeElementSize$ } from "../src/utils/elementSize";
import { initializeWidget } from "../src/widget";
/** The data channel topic this bundle has always used, so that old and new builds talk. */
const TOPIC = "matrixRTC";
export interface MatrixRTCSdk {
/** Joins the slot; `connected$` tracks the progress. A second call does nothing. */
join: () => void;
leave: () => void;
/** Ends everything. The sdk cannot be restarted; create a new one. */
stop: () => void;
connected$: Behavior<boolean>;
/** Our own member, null until our membership has been seen in the room. */
localMember$: Behavior<LocalRTCMember | null>;
/** The other members of the slot, with the media the participation carries for them. */
remoteMembers$: Behavior<RemoteRTCMember[]>;
/** What other members sent with `sendData`. */
data$: Observable<{ rtcBackendIdentity: string; data: string }>;
/** Sends to every member on our transport. Rejects while not joined. */
sendData: (data: unknown) => Promise<void>;
sendRoomMessage: (message: string) => Promise<void>;
}
export async function createMatrixRTCSdk(
application = "m.call",
id = "",
sticky = false,
): Promise<MatrixRTCSdk> {
const logger = rootLogger.getChild("[MatrixRTCSdk]");
const scope = new ObservableScope();
const widget = initializeWidget(application, true);
if (!widget) throw new Error("No widget: this bundle only runs as a widget");
const client = await widget.client;
const hostBridge = createWidgetHostBridge(widget);
const urlParams = getUrlParams();
const { roomId, controlledAudioDevices, callIntent } = urlParams;
const room = roomId === null ? null : client.getRoom(roomId);
if (room === null) throw new Error("No room: the url needs a joined roomId");
const mediaDevices = new MediaDevices(scope, {
controlledAudioDevices,
callIntent,
});
const muteStates = new MuteStates(
scope,
mediaDevices,
{ audioEnabled: false, videoEnabled: false },
hostBridge,
);
const encryptionSystem = { kind: E2eeType.PER_PARTICIPANT } as const;
const capture = captureSettings();
const slot = createRTCSlot(scope, client, room, {
...rtcSlotOptions(encryptionSystem),
application,
id,
});
let vm: CallViewModel | undefined;
const failed =
(what: string) =>
(e: unknown): void =>
logger.error(what, e);
const over = <T>(
pick: (participation: RTCParticipation) => Observable<T>,
fallback: T,
): Behavior<T> =>
scope.behavior(
slot.participation$.pipe(
switchMap((p) => (p === null ? of(fallback) : pick(p))),
),
);
return {
join: (): void => {
if (slot.participation$.value !== null) return;
if (sticky)
widget.api
.setAlwaysOnScreen(true)
.catch(failed("Failed to make the widget sticky"));
const participation = slot.join({
...rtcParticipationOptions(urlParams),
publish: initialPublishRequests(muteStates, mediaDevices, capture),
});
vm = createCallViewModel$(
scope,
participation,
room,
mediaDevices,
muteStates,
{
...callViewModelOptionsFromParams(urlParams),
encryptionSystem,
capture,
hostBridge,
// The bundle owns its page, so the body is the space it has
windowSize$: scope.behavior(observeElementSize$(document.body)),
},
of({}),
of({}),
);
// Once the hang-up has gone through, unstick and ask the host to close us
vm.leave$
.pipe(
switchMap(() =>
participation.state$.pipe(
filter((state) => state.kind === "left"),
take(1),
),
),
scope.bind(),
)
.subscribe(() => {
hostBridge
.setAlwaysOnScreen(false)
.catch(failed("Failed to unstick the widget"))
.then(() => hostBridge.close?.())
.catch(failed("Failed to ask the host to close"));
});
},
leave: (): void => vm?.leave(),
stop: (): void => scope.end(),
connected$: over(
(p) => p.state$.pipe(map((state) => state.kind === "connected")),
false,
),
localMember$: over((p) => p.localMember$, null),
remoteMembers$: over((p) => p.remoteMembers$, []),
// Attested members only: the SDK has matched the sender
data$: slot.participation$.pipe(
switchMap((p) => p?.data$ ?? EMPTY),
filter(({ topic }) => topic === TOPIC),
map(({ member, text }) => ({
rtcBackendIdentity: member.rtcBackendIdentity,
data: text,
})),
),
sendData: async (data): Promise<void> => {
const participation = slot.participation$.value;
if (participation === null) throw new Error("Not joined");
await participation.sendData(TOPIC, JSON.stringify(data));
},
sendRoomMessage: async (message): Promise<void> => {
await client.sendTextMessage(room.roomId, message);
},
};
}
+94 -41
View File
@@ -1,59 +1,112 @@
# SDK mode (EXPERIMENTAL)
# MatrixRTC SDK (EXPERIMENTAL)
EC can be build in sdk mode. This will result in a compiled js file that can be imported in very simple webapps.
`@element-hq/matrixrtc-sdk` is the call model under Element Call, on its own:
it joins a MatrixRTC slot in a Matrix room, writes and watches the memberships,
connects to the media transport (LiveKit today), publishes the local media and
exposes every member and their tracks as observables. It has no UI. Element
Call's own `CallViewModel` is one consumer of it; any host can build a different
view on top. The design is in [`SdkArchitecture.md`](./SdkArchitecture.md), what
moved out of Element Call's `src/` in [`SdkMigration.md`](./SdkMigration.md).
It allows to use matrixRTC in combination with livekit without relying on element call.
**Status:** first implementation. `createRTCSlot` and `slot.join()` join the
session, connect to the transport, publish the local media and expose every
member's media; the development harness and its e2e tests in `playwright/sdk`
drive it, and Element Call's own `CallViewModel` is built on it. It depends on
nothing in Element Call's `src/`.
This is done by instantiating the call view model and exposing some useful behaviors (observables) and methods.
## Using it
This folder contains an example index.html file that showcases the sdk in use (hosted on localhost:8123 with a webserver allowing cors (for example `npx serve -l 81234 --cors`)) as a godot engine HTML export template.
```ts
import {
createRTCSlot,
E2eeType,
MatrixRTCMode,
ObservableScope,
trackBySource$,
} from "@element-hq/matrixrtc-sdk";
## Getting started
const scope = new ObservableScope();
const slot = createRTCSlot(
scope,
client, // a matrix-js-sdk MatrixClient, logged in and syncing
room, // the matrix-js-sdk Room, from client.getRoom() once the join has synced
{
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
matrixRTCMode: MatrixRTCMode.Compatibility,
},
);
const participation = slot.join({
publish: [{ source: "microphone" }, { source: "camera" }],
});
To get started run
// A <video> for each remote member's camera and an <audio> for its microphone.
// A track plays on an element once attached to it; detach before the element
// leaves the DOM. (`tileFor`, `setName` and `removeTilesExcept` are the host's.)
participation.remoteMembers$.subscribe((members) => {
for (const member of members) {
const { video, audio } = tileFor(member.memberId);
member.displayName$.subscribe((name) => setName(member.memberId, name));
trackBySource$(scope, member.tracks$, "camera").subscribe((track) =>
track?.attach(video),
);
trackBySource$(scope, member.tracks$, "microphone").subscribe((track) =>
track?.attach(audio),
);
}
removeTilesExcept(members.map((m) => m.memberId));
});
```
pnpm install
pnpm build:sdk
// later
participation.leave();
scope.end();
```
in the repository root.
`participation.localMember$` is one member of the same shape, null until our own
membership is in the room; its microphone would only echo.
It will create a `dist` folder containing the compiled js file.
[`dev/main.ts`](./dev/main.ts) is this example as a working page.
This file needs to be hosted. Locally (via `npx serve -l 81234 --cors`) or on a remote server.
## Reading the code
Now you just need to add the widget to element web via:
Three files define the surface. Read them in this order.
```
/addwidget http://localhost:3000?widgetId=$matrix_widget_id&perParticipantE2EE=true&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
- [`src/api.ts`](./src/api.ts): the MatrixRTC side. An `RTCSlot` is the
namespace in a room that memberships join into; `join()` returns an
`RTCParticipation`, which lasts until its `leave()` and exposes `state$`,
`localMember$` and `remoteMembers$`. Members, transports and the participation
state machine live here. `createRTCSlot` in `RTCSlot.ts` is the only way to get
an implementation.
- [`src/media-api.ts`](./src/media-api.ts): media as a host sees it, on
`remoteMembers$` and `localMember$`. `PublishRequest` says what to publish
(microphone, camera, screen share); `MediaTrack` is how it arrives. The
underlying tracks stay hidden: the host hands an element to `attach` and the
SDK plays into it. The local variants carry the controls (`setEnabled`,
`setDevice`, `setProcessor`).
- [`src/media-backend/api.ts`](./src/media-backend/api.ts): internal, the seam
between the session and whatever carries the media. The session speaks
MatrixRTC; a `MediaBackend` speaks one media protocol and hands back the
`MemberMedia` of each member through `mediaFor()`. A host rarely touches it;
it matters when working on the LiveKit backend under
`src/media-backend/livekit/`, the only place that imports `livekit-client`, or
when writing a new backend.
## Development
Everything runs from the repository root. This directory is a pnpm project of its
own only so that a host can install it as a git dependency (see
`pnpm-workspace.yaml`).
```sh
pnpm dev:sdk # the harness, https://localhost:3002
pnpm build:sdk # dist/matrixrtc-sdk.js and dist/types
pnpm test:playwright playwright/sdk # needs `pnpm backend`
```
## Widgets
## Installing
The sdk mode is particularly interesting to be used in widgets. In widgets you do not need to pay attention to matrix login/cs api ...
To create a widget see the example `index.html` file in this folder. And add it to EW via:
`/addwidget <widgetUrl>` (see **url parameters** for more details on `<widgetUrl>`)
### url parameters
The url parameters are needed to pass initial data to the widget. They will automatically be used
by the matrixRTCSdk to start the postmessage widget api (communication between the client (e.g. Element Web) and the widget)
```
widgetId = $matrix_widget_id
perParticipantE2EE = true
userId = $matrix_user_id
deviceId = $org.matrix.msc3819.matrix_device_id
baseUrl = $org.matrix.msc4039.matrix_base_url
```sh
pnpm add github:element-hq/element-call#<ref>&path:/sdk
```
`parentUrl = // will be inserted automatically`
Full template use as `<widgetUrl>`:
```
http://localhost:3000?widgetId=$matrix_widget_id&perParticipantE2EE=true&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
```
the `$` prefixed variables will be replaced by EW on widget instantiation. (e.g. `$matrix_user_id` -> `@user:example.com` (url encoding will also be applied automatically by EW) -> `%40user%3Aexample.com`)
The host supplies `matrix-js-sdk`, `livekit-client` and `rxjs`; the build leaves
them external.
+829
View File
@@ -0,0 +1,829 @@
# MatrixRTC SDK architecture
`@element-hq/matrixrtc-sdk` is the model under Element Call, on its own: one
`MatrixRTCClient` per room that joins and leaves the MatrixRTC session, discovers
and connects to the media transports, publishes the local media, distributes the
encryption keys, and hands out every member with its media as observables. It has
no UI, reads no configuration and no settings, and never names LiveKit in its
interface.
Element Call becomes one consumer of it: `CallViewModel` keeps what makes a session
a _call_ (ringing, auto-leave, layout, sounds, reactions, settings, the host
bridge) and takes everything else from the client. The dependency direction is
therefore `Element Call → sdk`, never the reverse. What moved out of Element
Call's `src/` to get there is listed in [`SdkMigration.md`](./SdkMigration.md).
## The architecture
```
host application (Element Call, a whiteboard, watch-together, …)
│
│ createMatrixRTCClient(scope, matrixClient, room, options)
▼
┌──────────────────────────────────────────────────────────────────────────────┐
│ MatrixRTCClient │
│ │
│ public API, all Behaviors: │
│ join() leave() state$ │
│ localMember$ remoteMembers$ memberCount$ sendData() data$ │
│ keyRotationSuppressed$ connectedTransports$ │
│ │
│ per member RTCMember { id userId deviceId membership$ displayName$ │
│ avatarUrl$ transport$ tracks$ encryptionError$ }│
│ per track MediaTrack { source kind muted$ encrypted stats$ attach() } │
│ │
│ ┌───────────────────────────┐ MediaBackend ┌───────────────────────────┐ │
│ │ session: MatrixRTC │ media-backend/ │ media backend │ │
│ │ │ api.ts │ media-backend/livekit/ │ │
│ │ js-sdk MatrixRTCSession, │ │ today, or options.backend │ │
│ │ later the rust-rtc │ prepareLocal │ │ │
│ │ crate behind a bridge │ Transport ───▶│ auth: MSC4195 CS API or │ │
│ │ memberships from room │ delegateDelayed │ JWT service, by shape │ │
│ │ state and sticky events │ Leave ───────▶│ and role; delegation │ │
│ │ own membership, delayed │ │ ConnectionMap: one │ │
│ │ leave, retries │ mediaKeys$ ────▶│ LiveKit room per │ │
│ │ discovery of the local │ mediaFor$ ─────▶│ transport locator, │ │
│ │ transport (homeserver) │ ◀─ tracks$, │ local published, │ │
│ │ membership state machine │ encryptionError$│ remote subscribed │ │
│ │ (LocalMembership) │ local.publish, │ Publisher: local tracks, │ │
│ │ transport registry │ setPublishing▶│ upstream paused until │ │
│ │ (TransportMetadata) │ ◀─ connections$ │ joined │ │
│ │ data packet → member │ ◀─ data$ │ LivekitMemberMedia: │ │
│ │ │ │ participant → tracks │ │
│ └───────────────────────────┘ └───────────────────────────┘ │
│ │
│ RTCMember = membership + mediaFor$(membership): the backend matches the │
│ member's backendIdentities to the identity it sees on the transport │
└──────────────────────────────────────────────────────────────────────────────┘
│ │
▼ ▼
MatrixClient (sync, room state, to-device, OpenID) LiveKit SFU
```
The left box is MatrixRTC: who is in the session, with which transport, the
keys, the delayed leave, and the membership state machine. The right box is a
media backend: whatever carries the media, behind the `MediaBackend` interface in
`sdk/src/media-backend/api.ts`. The client is the join of the two: a member exists
once its membership does, and gets its media once the backend has something
for that membership on its transport; `state$` is the membership state combined
with the local media connection. LiveKit is the one backend today; a
cascading SFU or full mesh is another folder under `sdk/src/media-backend/`, handed
in through `options.backend`, and nothing in the left box changes.
### How Element Call uses it
```
Element Call, as app, widget or component
┌────────────────────────────────────────────────────────────────┐
│ views: InCallView, tiles, header, footer, settings │
│ ▲ Behaviors │
│ CallViewModel │
│ layout, ringing and notifications, auto-leave, sounds, │
│ reactions, hand raise, settings, audio routing, host bridge │
│ ▲ state$, localMember$, remoteMembers$, tracks$ │
│ MatrixRTCClient @element-hq/matrixrtc-sdk │
└────────────────────────────────────────────────────────────────┘
│ MatrixClient + Room │ livekit-client
▼ ▼
homeserver SFU
```
All three modes already hold a `MatrixClient` and a `Room` by the time a call
starts: standalone after `useLoadGroupCall`, the component from the host's `client`
prop, the widget from `createRoomWidgetClient`. The view model passes them on and
builds its media view models from `RTCMember` and its `tracks$` instead of from
LiveKit participants.
### Joining, in order
```
host MatrixRTCClient / backend homeserver / JWT service LiveKit SFU
│ create ───▶│ │ │
│ │ GET rtc/transports (session) ────────▶│ │
│ │◀── first transport of the backend's type │
│ │ (none, LiveKit only: a member's SFU │ │
│ │ that lets us publish, get_token) ─▶│ │
│ │ prepareLocalTransport (backend): │ │
│ │ publisher token: CS API get_token, │ │
│ │ or OpenID token then /sfu/get ─────▶│ │
│ │◀── sfu url, jwt, alias, identity ─────│ = resolved$ of the │
│ │ probe delegate_delayed_leave ──────▶│ local transport │
│ │ connect(url, jwt) ───────────────────────────────────────────▶│
│ join() ───▶│ (once the publisher holds its token)│ │
│ │ membership state event + delayed leave ──▶│ │
│ │ delegateDelayedLeave(delayId) (backend) ─▶│ │
│ │ create tracks, resume upstream ──────────────────────────────▶│
│ │ media key to every other member (to-device) ──▶│ │
│ │◀── other memberships (sync) ──────────│◀── participants ────────│
│◀─ state$ connected, localMember$, remoteMembers$ with tracks$ ───────────────│
```
The session discovers the transport from the homeserver and the backend prepares
it, both before `join()`, so that the membership can name it and the first frames
go out as soon as the state event is sent. Preparing is the backend's own
business: for LiveKit the token exchange for the publisher connection, and the
probe that decides whether the service that issued the token can take over the
delayed leave, which picks the delayed leave timings the join uses. The join
waits for the publisher to hold its token, so that the LiveKit room exists
before a subscriber, who lacks the permission to create it, asks for a token
of its own. Until `join()` the tracks exist but their upstream is paused: a host
can show a preview without anyone hearing it. The upstream flows exactly while the
membership is `joined`: a sync outage pauses it, and `leave()` pauses it for good
and sends the leave.
## Vocabulary
The words the SDK is written in. They name the shape after the slot and
participation split in
[`client-into-slot-session-split-plan.md`](./client-into-slot-session-split-plan.md);
the sections below still describe `MatrixRTCClient` until that lands.
| Word | Meaning here | Elsewhere |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- |
| slot | an `RTCSlot`: the namespace in a room that memberships join into, `application` plus `id`. It exists while the room does, with nobody in it too. Holds `members$`, its open or closed `status$`, and `join()` | the js-sdk `MatrixRTCSession` is one per slot; the MSC4143 slot event |
| participation | an `RTCParticipation`: the span between our `join()` of a slot and our `leave()`. Holds the connection and its `state$`, `localMember$`, `remoteMembers$`, the connected transports and the data channel. Made by `slot.join()`, never reused | "session" is not used here, because in the js-sdk it means the slot |
| member | an `RTCMember`: one membership in a slot with what the SDK derives from it, `rtcBackendIdentity`, `userId`, `deviceId`, `memberId`, `displayName$`, `avatarUrl$`, `transport$` and `applicationData$`. No media. What the slot hands out in `members$` | the js-sdk `CallMembership` behind it, which never leaves the SDK; Element Call's old "matrixLivekitMember" |
| local member, remote member | `LocalRTCMember` and `RemoteRTCMember`: an `RTCMember` plus its `MemberMedia`, which only a participation can carry. The local one's tracks carry the controls. What the participation hands out in `localMember$` and `remoteMembers$` | LiveKit `LocalParticipant` and `RemoteParticipant`, which never leave the SDK |
| media | a `MemberMedia`: what a member sends once it has arrived on its transport, `tracks$`, null while waiting, and `encryptionError$` beside it. The media backend supplies exactly these two fields per member | Element Call's `MediaViewModel` is a view of it |
| track | a `MediaTrack`: one publication of a member, audio or video, with `source`, `kind`, `id` and the behaviors on it. `LocalAudioMediaTrack` and `LocalVideoMediaTrack` add the controls over our own | LiveKit `TrackPublication` and `Track`, which never leave the SDK |
| publish request | a `PublishRequest`: a source, where to capture it from and how to encode it. Given at the join in `publish`, or later to `publish` on the participation | |
| attach, detach | handing a `<video>` or `<audio>` element to a track and taking it back. The SDK owns the stream and the observers on the element | LiveKit `Track.attach` |
| transport | an `RTCTransport`: where media is exchanged, a LiveKit service url today. Carries the raw js-sdk `Transport` from the membership and, in `resolved$`, what the backend fetched to connect to it | MSC4143 "focus" |
| media backend | a `MediaBackend`: what carries the media, behind `sdk/src/media-backend/api.ts`. One per participation, chosen with `backend` in the join options; LiveKit under `sdk/src/media-backend/livekit/` is the one that exists | "SFU", a LiveKit room, a peer connection |
Names use `RTC` in capitals, as the js-sdk does: `RTCSlot`, `RTCParticipation`,
`RTCMember`, `matrixRTCMode`.
## Reactive primitives
The whole public API is made of `Behavior`s: an rxjs observable that always holds a
current value (`.value`) and replays it on subscription. A consumer subscribes to
what it renders and reads `.value` where it needs the present state. Everything is
bound to an `ObservableScope` the host creates; ending the scope ends the client,
its connections and its tracks. Both come from the SDK (`Behavior`, `constant`,
`ObservableScope`); the host needs rxjs and nothing else.
## What a host must provide
Found the hard way while building the harness; each one fails in a way that looks
like an SDK bug.
- **The room the sync loop maintains.** `client.getRoom(roomId)` once the join has
synced. `joinRoom` returns a detached copy for a room joined just now, and a
client built on it never sees a member.
- **Power to send the membership event.** Every member needs to write
`org.matrix.msc3401.call.member` (and the sticky event in Matrix 2.0 mode). A
plain room only lets its creator send state; Element Call's rooms set the level
to 0.
- **An encrypted room for per-participant keys.** Media keys travel in encrypted
to-device messages, which the crypto only delivers to devices it tracks, which
are the members of encrypted rooms. In an unencrypted room the key never arrives
and every remote track decodes nothing.
- **A logged-in, syncing client with crypto set up**, and the transport advertised
by the homeserver (`rtc/transports`). The SDK has no fallback url of its own.
## The client: `MatrixRTCClient`
Rule of thumb for what belongs here: if it would be the same for a different
application on top of the same MatrixRTC session, it is in the client. If it is
specific to calls, or depends on the window, a setting, a platform, or a React
context, it is in the view model.
### Creating one
```ts
import { type MatrixClient, type Room } from "matrix-js-sdk";
import { type RTCNotificationType } from "matrix-js-sdk/lib/matrixrtc";
export interface MatrixRTCClientOptions {
encryptionSystem: EncryptionSystem;
/** Resolved by the host; the SDK reads neither config.json nor settings. */
matrixRTCMode: MatrixRTCMode;
/** Published at the join; `publish` on the local member adds to it from then on. */
publish: PublishRequest[];
/**
* MSC4075 notification sent with the join. A parameter of the MatrixRTC
* join itself, so it is here even though it is named after calls; reacting
* to a notification (ringing, timeouts, declines) is the application's job.
*/
sendNotificationType?: RTCNotificationType;
/** The application the session belongs to, as named in the membership. Default `m.call`. */
application?: string;
/** The application's slot in the room. Default `ROOM`. */
slot?: string;
/**
* Whatever the application wants to say about itself in the membership.
* Opaque to the SDK. Today the js-sdk carries one key, `m.call.intent`
* (`"audio"` or `"video"`); anything else is dropped until it can.
*/
applicationData?: Record<string, unknown>;
/** Session timings the host has configured; the defaults otherwise. */
timings?: Partial<SessionTimings>;
/** Limits on what is published; LiveKit's defaults otherwise. */
mediaQuality?: MediaQuality;
/** Use this transport instead of asking the homeserver. */
transport?: Transport;
/** Use this transport when the homeserver advertises none of the backend's type. */
fallbackTransport?: Transport;
}
/**
* One thing to publish: a source, where to capture it from and how to encode
* it. Where the device and the settings are left out, the browser's and
* LiveKit's defaults apply.
*/
export type PublishRequest =
| { source: "microphone"; deviceId?: string; capture?: AudioCaptureSettings }
| {
source: "camera";
deviceId?: string;
/** Background blur and the like. */
processor?: VideoProcessor;
capture?: VideoCaptureSettings;
}
| {
source: "screenShare";
/** Whether to capture the screen's audio too. Default true. */
audio?: boolean;
capture?: VideoCaptureSettings;
};
```
Device enumeration, permission prompts and the lobby preview stay with the
host: it hands over publish requests naming the devices it chose.
```ts
/**
* Takes the whole MatrixClient, not a narrow pick, and finds the MatrixRTC
* session itself. The js-sdk is the MatrixRTC implementation today; when the
* rust-rtc crate replaces it, the SDK has to bridge the crate to the js-sdk
* client (events, state, to-device, delayed events, OpenID), and only the SDK
* knows what that bridge needs. Holding the client keeps that change inside
* the SDK. The js-sdk session is not part of the public API.
*
* The room has to be the one the client's sync loop maintains; see "What a
* host must provide".
*/
export function createMatrixRTCClient(
scope: ObservableScope,
client: MatrixClient,
room: Room,
options: MatrixRTCClientOptions,
): MatrixRTCClient;
```
### The client
```ts
/** The homeserver link that is down, in the order they depend on each other. */
export type MatrixDisconnectReason = "sync" | "membership" | "probablyLeft";
/** Why the participation is not connected: the first failing link, in dependency order. */
export type DisconnectReason = MatrixDisconnectReason | "media";
export type ParticipationState =
| { kind: "waitingForTransport" }
| { kind: "connecting" }
| { kind: "connected" }
| { kind: "reconnecting"; reason: DisconnectReason }
/** A failed publication is not fatal: the member can still receive, so it stays on the local member's media. */
| { kind: "failed"; error: MatrixRTCError }
/** `leave()` was called. Terminal. */
| { kind: "left" };
/**
* An error raised by the client. `code` and `category` say what went wrong in
* a form a host can translate; `cause` holds the backend's error.
*/
export class MatrixRTCError extends Error {
code: ErrorCode;
category: ErrorCategory;
}
export interface MatrixRTCClient {
join(): void;
leave(): void;
/**
* The Matrix membership combined with the local media connection. One
* value: a reason exists only while `reconnecting`, an error only once
* `failed`.
*/
state$: Behavior<ParticipationState>;
/** Null until our own membership has been seen in the room. */
localMember$: Behavior<LocalRTCMember | null>;
remoteMembers$: Behavior<RemoteRTCMember[]>;
/** `remoteMembers.length`, plus one for the local member once it exists. One per membership, so one user may count several times. */
memberCount$: Behavior<number>;
/**
* Whether the session has grown large enough that MatrixRTC has stopped
* rotating the media encryption key.
*/
keyRotationSuppressed$: Behavior<boolean>;
/**
* Transports the client currently holds a live connection to. The
* connections themselves stay internal; this is what a developer panel
* needs.
*/
connectedTransports$: Behavior<TransportMetadata[]>;
/**
* Plays the members' audio on this device from now on. Rejects where the
* browser cannot switch to it. A host that routes audio itself, or leaves
* the browser's choice, never calls it.
*/
setAudioOutputDeviceId(deviceId: string): Promise<void>;
/**
* Sends a short text to every member on the local transport, over a
* reliable data channel beside the media. A packet holds about 15 KiB;
* anything larger belongs in a room event. Rejects while the local
* transport is not connected. Encrypted on the wire like the media, but
* not end to end with the media key.
*/
sendData(topic: string, text: string): Promise<void>;
/**
* What remote members sent with `sendData`, on every transport the client
* is connected to. A message from an identity that is not a member is
* dropped, so a host only ever hears from attested members.
*/
data$: Observable<DataMessage>;
}
/** One `sendData` call as it arrives at a remote member. */
export interface DataMessage {
member: RemoteRTCMember;
topic: string;
text: string;
}
```
`state$` is two state machines, one per box in the diagram, combined by one pure
function. `LocalMembership` knows only the Matrix side: `waitingForTransport`,
`joining`, `joined`, `reconnecting` with a `MatrixDisconnectReason`, `failed`,
`left`. The client folds the local media connection in: `joined` with the media
up is `connected`, a media drop after that is `reconnecting: "media"`, a
connection error is `failed`. Each step is computed in priority order from the
previous state, so `left` and `failed` absorb everything after them, and
`reconnecting` is only ever reached from `connected`; a membership that drops
before the media ever came up stays `connecting`. One reason, not a set, because
the links are a chain: without sync the membership is unknowable, and the media
is worth little while the Matrix side is down, so the first broken link in that
order is what a host shows and what Element Call's analytics count. A failed
publication is not in here at all: the member can still receive, so it stays on
the local member's media.
### Members
```ts
/**
* What a membership says, as hosts read it. The js-sdk `CallMembership` is
* behind it; anything beyond this is the js-sdk's API, not the SDK's.
*/
export type RTCMembership = Pick<
CallMembership,
| "userId"
| "deviceId"
| "memberId"
| "rtcBackendIdentity"
| "application"
| "applicationData"
| "getTransport"
| "transports"
| "createdTs"
| "getAbsoluteExpiry"
>;
export interface RTCMember {
local: boolean;
/** The membership's `rtcBackendIdentity`, `${userId}:${deviceId}`: a key and a debug label. */
rtcBackendIdentity: string;
/**
* Every identity the member may publish under on its transport, one per
* authentication flow the transport allows: the hashed identity the
* MSC4195 client-server API assigns, the legacy one the JWT service's
* `/sfu/get` assigns, or both for a transport carrying both a `url` and a
* `livekit_service_url`. The backend matches whichever shows up.
*/
backendIdentities: readonly string[];
userId: string;
deviceId: string;
membership$: Behavior<RTCMembership>;
/**
* Matrix room state rather than MatrixRTC, but the SDK already holds the
* room and every consumer wants them next to the membership. Disambiguated
* among the members of the session.
*/
displayName$: Behavior<string>;
avatarUrl$: Behavior<string | undefined>;
/** Which transport this member is on; undefined when the membership names none. */
transport$: Behavior<TransportMetadata | undefined>;
/**
* The member's tracks, in publication order, once it has shown up on its
* transport; null until then ("waiting for media"). An entry stays the same
* object for as long as the same publication is behind it. Which track is
* which is in its `source`; `trackBySource$` picks one out. Hand raise and
* reactions are room events keyed by member and stay with the host for now;
* they are the obvious next fields here.
*/
tracks$: Behavior<(AudioMediaTrack | VideoMediaTrack)[] | null>;
/** Emits when the SFU reports a key problem for this member. */
encryptionError$: Observable<EncryptionError>;
}
export interface RemoteRTCMember extends RTCMember {
local: false;
}
export interface LocalRTCMember extends RTCMember {
local: true;
tracks$: Behavior<(LocalAudioMediaTrack | LocalVideoMediaTrack)[] | null>;
/**
* Publishes a source and resolves with its track once it is in `tracks$`.
* Before the transport is connected the request is remembered and applied
* once it is. Rejects where the device could not be used, including the
* user closing the picker. One publication per source: publishing a source
* again unmutes it.
*/
publish(
request: PublishRequest,
): Promise<LocalAudioMediaTrack | LocalVideoMediaTrack>;
/** Removes one of our tracks; a screen share takes its audio with it. */
unpublish(id: string): Promise<void>;
}
```
### Transports
```ts
import { type Transport } from "matrix-js-sdk/lib/matrixrtc";
/**
* One transport advertised in a membership. Transport independent: `type` and
* `id` are all the SDK needs; `raw` and `resolved$` are there for a
* backend-specific developer panel and for connection diagnostics. One object
* per transport, however many memberships name it.
*/
export interface TransportMetadata {
/** `"livekit"` today. */
type: string;
/** Stable key, unique per transport in the session: the raw transport, serialised with sorted keys. */
id: string;
/** The transport object as it appears in the membership, e.g. `{ type, livekit_service_url }`. */
raw: Transport;
/**
* What the backend had to fetch before it could connect. Undefined until
* the connection has resolved it, and again after the connection stops or
* fails. The local transport resolves before the join, since the membership
* needs it; remote ones when their connection starts.
*/
resolved$: Behavior<ResolvedTransport | undefined>;
}
export type ResolvedTransport =
| {
type: "livekit";
/** The SFU websocket url, as opposed to the JWT service url in `raw`. */
url: string;
/** A secret: fit for a developer panel, never for a log line. */
token: string;
roomAlias: string;
identity: string;
}
| { type: string; [key: string]: unknown };
```
### Call-specific things kept out of the client
Checked against `CallViewModel` and its options. These stay in Element Call because
they only make sense for a call:
| Today | Why it is call-specific | Where it lives instead |
| ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| `autoLeave$` / `AutoLeaveReason` (`"allOthersLeft" \| "timeout" \| "decline"`) | hanging up because the other side declined or everyone left is call etiquette, not session semantics | `CallViewModel.leave$`, built from `remoteMembers$` and the room timeline |
| `ringAttempts$`, `createCallNotificationLifecycle$`, `createSentCallNotification$`, `createReceivedDecline$` | ringing, pickup timeouts and decline events are MSC4075 call-notify UX | `src/state/CallViewModel/CallNotificationLifecycle.ts`, fed by `remoteMembers$` and the Matrix room |
| `waitForCallPickup`, `autoLeaveWhenOthersLeft` options | inputs to the above | `CallViewModelOptions` |
| `hideScreensharing` option | a UI policy; the session can always share a screen if the platform can | `CallViewModel` offers no screen share control |
| `ringingVm$`, `ringingStatusLocation` | view models | `CallViewModel` |
| `hostBridge.hangUp$`, `hangup()`, `userHangup$` | widget / host integration | `CallViewModel` |
| `ElementCallError` | carries Element Call translation keys | the view model wraps `MatrixRTCError`, reading `cause` |
| `participantCount$` sound limits, join/leave sounds | call UX | `CallViewModel` |
| `callIntent` | what a membership says about the call | `applicationData["m.call.intent"]`, set by the host |
Kept despite the name: `sendNotificationType`, because the js-sdk join sends the
notification as part of entering the session. Only _reacting_ to it is
call-specific.
## Media backend: `MediaBackend`
The client speaks MatrixRTC: memberships, the transports named in them, the
delayed leave, the keys. What carries the media is a backend behind one
interface, `MediaBackend` in `sdk/src/media-backend/api.ts`, created once per client.
LiveKit under `sdk/src/media-backend/livekit/` is the only backend today, and the only
place in the SDK that imports `livekit-client`; the import boundary lint rule
enforces that outside `sdk/src/media-backend/` the LiveKit packages appear in type
imports only.
The seam sits above the connection manager on purpose. One connection per
transport url, with a subscribe-only connection to every remote SFU, is the
LiveKit multi-SFU strategy, not a MatrixRTC concept: a cascading SFU keeps the
membership transport as it is and only changes which SFUs the client connects to,
and full mesh has one peer connection per member and no token service. So a
backend is handed the whole session's view and gives back media:
```ts
/** The fields of a member the backend supplies, as the member exposes them. */
type MemberMediaFields = Pick<RTCMember, "tracks$" | "encryptionError$">;
export interface MediaBackend {
readonly transportType: string; // "livekit"
/** Authenticate with the transport the client found; the probe for delegation runs here. */
prepareLocalTransport(
transport: Transport,
): Promise<{ canDelegateDelayedLeave: boolean }>;
/** Hand over the delayed leave, each time the membership manager has a new delay id. */
delegateDelayedLeave(delayId: string): Promise<void>;
readonly local: LocalMediaBackend; // tracks$ encryptionError$ connectionState$ setPublishing publishError$ publish unpublish
/** A remote member's media; also how the backend learns which transports to connect to. */
mediaFor$(
scope: ObservableScope,
membership$: Behavior<RTCMembership>,
): MemberMediaFields;
readonly connections$: Behavior<BackendConnection[]>; // transport, state, resolved; diagnostics
sendData(topic: string, text: string): Promise<void>;
readonly data$: Observable<DataPacket>;
}
```
What stays on the client's side: discovery of the local transport from the
homeserver's list (a homeserver endpoint, so not a backend's) with, for the
LiveKit backend only, a last resort of another member's SFU that lets us
publish (`findRemotePublishLivekitTransport`, wired in by `RTCParticipation` rather
than by the backend interface, since it is an MSC4195 matter), the membership
state machine in `LocalMembership.ts` and its combination with
`local.connectionState$` into `state$`, the choice of delayed leave timings,
mapping a data packet's sender to a member, and the transport registry, which keys on the raw
transport serialised with sorted keys and derives `connectedTransports$` and each
`TransportMetadata.resolved$` from `connections$`.
The backend's context carries no Matrix client and no membership list. A backend
that needs the client takes a `Pick` of exactly what it calls in its own options
(`createLivekitBackend(scope, context, { client, mediaQuality })`),
and learns of remote members only through `mediaFor$`, registering each
membership's transport for as long as the member's scope lives. Media keys reach
it as a stream, `mediaKeys$`, replayed on subscribe; the js-sdk session never
does. A host supplies its own backend through `MatrixRTCClientOptions.backend`;
without one, LiveKit is built from the LiveKit-flavoured client options.
### Authenticating with a LiveKit transport
A LiveKit transport comes in three shapes while MSC4195 settles: an SFU `url`,
a JWT service `livekit_service_url`, or both. `media-backend/livekit/auth/`
fronts the two flows and chooses between them per connection, from the shape
and from the role we take on the transport:
| Transport | Publisher | Subscriber |
| -------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- |
| `url` only | client-server API `get_token`, no fallback | client-server API `get_token`, no fallback |
| `livekit_service_url` only | JWT service `/sfu/get` (legacy identity) | JWT service `/get_token` (hashed identity) |
| both | JWT service `/sfu/get` (legacy identity) | client-server API, falling back to the JWT service `/get_token` where the homeserver, or a widget host, has no MSC4195 |
A publisher on a transport with a service URL keeps the legacy
`<user_id>:<device_id>` identity, because clients predating MSC4195 match a
membership against that and nothing else; a subscriber is matched by nobody and
takes the more private hashed identity. A publisher never falls back: a
homeserver that advertises a `url` without implementing MSC4195 is misconfigured
and fails with `NoMatrix2AuthorizationService`, so the admin hears about it.
The MatrixRTC mode plays no part in the choice. The delayed leave is delegated
through the same flow that issued the publisher's token, so that the identity
the service watches for on the SFU is the one we publish under, and the
delegation probe before the join asks only that service.
The client-server API needs the name of the homeserver the transport belongs to:
our own homeserver asks that server for the token, over federation where it is
not ours. The backend derives it from the member's user id, and from our own for
the local transport, and keys its connections on the transport plus the server
name where the transport has a `url`. A JWT service is reached directly, so
members of several homeservers advertising the same one share a connection.
## Media: `tracks$` and `MediaTrack`
Every member has media. Today, in Element Call, that media _is_ a `livekit-client`
participant, handed straight through to the view models. In the SDK the participant
becomes the member's `tracks$`, a list of `MediaTrack`s, so that nothing outside
the SDK imports `Participant`, `TrackPublication`, `Track` or `TrackReference`.
The adapter in `sdk/src/media-backend/livekit/` is the only place that reads a
participant. The backend supplies the same two fields through `mediaFor$`, so
there is one shape from the LiveKit participant to the host and nobody has to
subscribe through one Behavior to reach another.
```ts
export type MediaSource =
| "microphone"
| "camera"
| "screenShare"
| "screenShareAudio"
/** Published without a source; the application knows what it is. */
| "unknown";
export type MediaStreamStats =
RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined;
/** One published track of a member. */
export interface MediaTrack {
source: MediaSource;
kind: "audio" | "video";
/** Stable for the life of the track. */
id: string;
muted$: Behavior<boolean>;
/** False when the SFU reports the track as unencrypted. Fixed for the life of the track. */
encrypted: boolean;
/**
* Inbound statistics for a remote track, outbound for a local one, polled
* every `intervalMs` while the observable is subscribed and not at all
* otherwise, so a statistics panel costs nothing while closed.
*/
stats$(intervalMs: number): Observable<MediaStreamStats>;
/**
* Rendering. The view hands its <video> or <audio> element over; the SDK
* sets its stream and, for video, registers the size and on-screen
* observers that pick a simulcast layer and pause the subscription while
* the element is hidden. attach is idempotent per element; detach has to be
* called before the element leaves the DOM so those observers are released.
*/
attach(element: HTMLMediaElement): void;
detach(element: HTMLMediaElement): void;
}
export interface AudioMediaTrack extends MediaTrack {
kind: "audio";
/**
* Whether the track carries sound right now, as the backend measures it.
* False while muted. A call reads this on the microphone track and calls it
* "speaking". LiveKit measures it per member, so every audio track of a
* member reports the same value.
*/
isActive$: Behavior<boolean>;
/**
* How loud the track is right now, 0 to 1, as the backend measures it. 0
* while muted. Not what decides `isActive$`: the backend applies its own
* threshold and hysteresis to that.
*/
audioLevel$: Behavior<number>;
/** Route playback through Web Audio, for earpiece pan and gain. Undefined resets. */
setAudioContext(ctx: AudioContext | undefined, plugins?: AudioNode[]): void;
/** No-op for our own audio, which is never played back. */
setVolume(volume: number): void;
}
export interface VideoMediaTrack extends MediaTrack {
kind: "video";
}
/** The controls a member has over a track it publishes itself. */
export interface LocalMediaTrack {
/** Mutes or unmutes. Resolves with the state that resulted. */
setEnabled(enabled: boolean): Promise<boolean>;
/** Captures from another device. Rejects for a screen share, which has none. */
setDevice(deviceId: string): Promise<void>;
}
export interface LocalAudioMediaTrack
extends AudioMediaTrack, LocalMediaTrack {}
export interface LocalVideoMediaTrack extends VideoMediaTrack, LocalMediaTrack {
/** For mirroring; undefined where the camera does not say which way it faces. */
facingMode$: Behavior<"user" | "environment" | undefined>;
/** Restarts the camera facing the other way; resolves with the device now in use. */
switchFacingMode(): Promise<string | undefined>;
/** Background blur and the like; undefined removes the processor. */
setProcessor(processor: VideoProcessor | undefined): Promise<void>;
}
export type EncryptionError = "MissingKey" | "InvalidKey";
/** The member's first track of a source; undefined while there is none, or no media yet. */
export function trackBySource$<S extends MediaSource>(
scope: ObservableScope,
tracks$: Behavior<MediaTrack[] | null>,
source: S,
): Behavior<TrackOfSource<S> | undefined>;
```
### Rendering: `attach` and `detach`
The view passes its DOM element into the SDK and the SDK does the rest; there is no
`MediaStreamTrack` on the interface. The reason is adaptive stream: Element Call
runs with `adaptiveStream: true`, which means LiveKit decides the simulcast layer
from the size of the attached video element and pauses the subscription at the
SFU while no attached element is on screen. Those size and visibility observers are
set up inside LiveKit's `attach`. Handing the element over is the only way to keep
that without reimplementing the observers outside the SDK. The e2e tests check both
effects: a larger tile receives a higher resolution, a hidden tile stops receiving
frames.
What `attach` covers for free, so the host does not have to: `srcObject` and
`autoplay` setup with the Safari and Firefox black-video workarounds; `play()` and
the autoplay-blocked signal; following the track through mute, reconnect and
re-subscription into every attached element; the Web Audio graph for earpiece pan
and gain; the adaptive stream observers.
The interface is web-only. A non-DOM host would need a different media surface; if
one shows up, a `MediaStreamTrack` accessor is added next to `attach`, not instead
of it.
## Packaging
`sdk/` is packaged exactly as `component/` is: a `package.json` with peer
dependencies (`matrix-js-sdk`, `livekit-client`, `rxjs`), `exports` with types,
type emission through `sdk/tsconfig.build.json`, an externals list in
`vite-sdk.config.ts` that `pnpm lint:externals` enforces, and installation from the
repository as a git dependency with a `path:` suffix. `pnpm build:sdk` produces
`sdk/dist`. It is not a pnpm workspace package yet; the root is not a workspace, and
moving to one later carries the package name, entry point and externals over
unchanged.
Element Call consumes the SDK by its package name from day one, so the swap to a
published dependency is one line: `tsconfig.json` `paths` and the `resolve.alias`
in `vitePluginsConfig` both map `@element-hq/matrixrtc-sdk` to `sdk/index.ts`. The
alias has to live in `vitePluginsConfig`, which every config is built on. Put only
into the app's own config, the harness resolves the name through `sdk/package.json`
to whatever stale `sdk/dist` is lying around.
Layout:
```
sdk/
index.ts the entry point: re-exports the API and the primitives
src/api.ts the public types above: client, members, transports
src/media-api.ts the public media types: publish requests, tracks
src/errors.ts MatrixRTCError and its codes
src/config.ts MatrixRTCMode, session timings, media quality
src/encryption.ts E2eeType, EncryptionSystem
src/MatrixRTCClient.ts createMatrixRTCClient: wires the two folders below
src/participationState.ts the membership state combined with the media connection
src/reactive/ Behavior, ObservableScope, the observable operators
src/matrixrtc/ the MatrixRTC side: the local membership, memberships, member
metadata, transports, discovery, the js-sdk join
src/media-backend/ the MediaBackend interface in api.ts; livekit/ is the one
backend: connections, publisher, JWT, keys, the tracks
src/utils/ mapScoped, network retry, display names, test helpers
dev/ the harness (below)
SdkArchitecture.md this document
SdkMigration.md what moved from Element Call's src/ into the SDK
```
The lint rule `element-call/sdk-import-boundary` keeps it that way: nothing under
`sdk/` may import a file outside `sdk/` (that is `src/`, `component/`,
`playwright/`) or React, i18n or Compound. The harness under `sdk/dev/` is a host
and exempt.
## Dev harness: `sdk/dev/`
A web app that uses the SDK and nothing else: plain DOM, no React, no Compound, no
i18n, imports only `@element-hq/matrixrtc-sdk`, `matrix-js-sdk` and `rxjs`. It is
where the SDK is run during development and what the e2e tests in `playwright/sdk`
drive. If something cannot be done in the harness without reaching into `src/`, the
SDK API is missing something.
What it does: log in with a password, or with an access token the URL hands it (the
tests register users through the admin API and skip `/login`, which the dev
homeserver rate-limits); join the room and wait for the sync to hold it; create the
client with microphone and camera enabled; show one tile per member with the display
name, a `<video>` attached from the member's `"camera"` track and, for remote
members, an `<audio>` from the `"microphone"` track, both picked out of
`tracks$` with `trackBySource$`; toggle the microphone and camera; leave.
Every media element carries the track's state as `data-*` attributes (`muted`,
`encrypted`, `active`, `frameWidth`, `frames`), which is what the tests read to
check mute propagation, encryption, adaptive resolution and paused subscriptions
(the speaker detection check is a `fixme`: the fake microphones do not reliably
register at the SFU). That is more
than the smallest consumer needs, and the price of testing the SDK through its
public API only.
## Open decisions
- **Publishing before `join()`.** The initial requests are published at the
join; creating the tracks earlier, for a lobby preview, is still missing.
- **`applicationData` after the join.** The option is read once; there is no
way to update it on a live membership yet.
- **Raw membership state.** The Matrix side's own state machine
(`MembershipState` in `LocalMembership.ts`) stays internal; `state$` on the
participation is its public view combined with the media connection. A
developer panel may want it as `LocalRTCMember.membershipState$`.
- **Developer panel.** `resolved$` on each connected transport covers what the
panel shows today. Anything beyond that (LiveKit room state, connection
quality) needs an opaque `debug$` per transport.
- **Data beside the media.** `sendData`/`data$` ride on LiveKit's reliable data
packets: topic-agnostic, matched to members by identity, received on every
connection but sent only on the local transport, so a member on another
transport does not hear us. They are transport-encrypted, not end to end:
LiveKit can encrypt data packets with the media key, but that is not wired
through the key provider yet. A host that needs either reach or end-to-end
encryption uses room events.
- **Unencrypted rooms with per-participant keys.** Keys only reach devices the
crypto tracks, which are the members of encrypted rooms. If the SDK is to work
in an unencrypted room, tracking the members' devices becomes its job.
+53
View File
@@ -0,0 +1,53 @@
# What moved from `src/` into the SDK
Element Call's `CallViewModel` used to do everything; now it consumes the SDK and
keeps only what makes a session a call. This is the inventory of what moved out
of `src/` on the way, so that a reader of either side knows where to look. It
names source files, so it ages with the code; the shape they make is in
[`SdkArchitecture.md`](./SdkArchitecture.md).
Everything in the table is gone from `src/`. Each module came with its tests;
`sdk/src/utils/test.ts` and `test-fixtures.ts` are the subset of Element Call's
test helpers they need. `Behavior`, `ObservableScope`, the observable operators,
`E2eeType`, `EncryptionSystem` and `MatrixRTCMode` are re-exported from the SDK
by their old `src/` paths, so the rest of Element Call did not have to change.
| Was in `src/` | Is in `sdk/src/` | What changed on the way |
| ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `state/Behavior.ts`, `state/ObservableScope.ts`, `utils/observable.ts` | `reactive/` | as is |
| `state/SessionBehaviors.ts` | `matrixrtc/JsRtcSessionBehaviors.ts` | the memberships and key rotation behaviors only |
| `state/CallViewModel/localMember/HomeserverConnected.ts` | `matrixrtc/HomeserverConnected.ts` | the grace period defaults to `defaultSessionTimings` instead of `Config.get()` |
| `state/CallViewModel/localMember/LocalMember.ts` | `matrixrtc/LocalMembership.ts`, `media-backend/livekit/LocalMedia.ts`, `participationState.ts` | split into the Matrix side, the media side and the state machine that combines them |
| `state/CallViewModel/localMember/LocalTransport.ts`, `RtcTransportAutoDiscovery.ts` | `matrixrtc/LocalTransport.ts`, `TransportAutoDiscovery.ts` | the override and the fallback are transports from the options, not a setting and a resolved config |
| `state/CallViewModel/remoteMembers/MatrixLivekitMembers.ts` | `media-backend/livekit/LivekitBackend.ts` (`mediaFor`) | matches participants to members inside the backend; the SDK hands out `RTCMember`s |
| `state/CallViewModel/remoteMembers/MatrixMemberMetadata.ts`, `utils/displayname.ts` | `matrixrtc/MatrixMemberMetadata.ts`, `utils/displayname.ts` | as is |
| `state/CallViewModel/remoteMembers/Connection.ts` | `media-backend/livekit/Connection.ts` | throws `MatrixRTCError`s; `ConnectionState` is a discriminated union; fetches its own token for its role |
| `state/CallViewModel/remoteMembers/ConnectionFactory.ts` | `media-backend/livekit/ConnectionFactory.ts` | `LivekitConnectionFactory` builds the room options from the publish requests and `mediaQuality`, not from settings |
| `state/CallViewModel/remoteMembers/ConnectionManager.ts` | `media-backend/livekit/ConnectionMap.ts` | keyed by `TransportLocator`; the publisher role is the local transport's |
| `state/CallViewModel/localMember/Publisher.ts` | `media-backend/livekit/Publisher.ts` | rewritten on publish requests instead of `MuteStates` |
| `livekit/openIDSFU.ts`, later `livekit/auth/` | `media-backend/livekit/auth/` | the flow is chosen by the transport's shape and our role; the delegated leave timeout is a parameter |
| `livekit/options.ts` | `media-backend/livekit/livekitOptions.ts` | `buildLiveKitOptions(mediaQuality?)` only; the `Config` singleton is gone |
| `e2ee/matrixKeyProvider.ts`, the key provider selection | `media-backend/livekit/MatrixKeyProvider.ts`, `KeyProvider.ts` | as is; selection keyed off `encryptionSystem` |
| `e2ee/e2eeType.ts`, `EncryptionSystem` from `e2ee/sharedKeyManagement.ts` | `encryption.ts` | the React hooks stayed behind |
| `MatrixRTCMode`, the timings and media quality types from `config/ConfigOptions.ts` | `config.ts` | `defaultSessionTimings` and `defaultMediaQuality` replace `DEFAULT_CONFIG` |
| `utils/errors.ts` | `errors.ts` | `MatrixRTCError` with `code` and `category` replaces `ElementCallError`; messages are plain English, the host translates by code |
| `doNetworkOperationWithRetry` from `utils/matrix.ts` | `utils/network.ts` | takes a `shouldRetry` predicate |
| the join in `CallViewModel.ts` (`enterRTCSession`) | `matrixrtc/joinJsSdkSession.ts` | the timings and the delayed leave variant are parameters |
| `livekit/MatrixAudioRenderer.tsx`, the participant reads in `state/media/` | `media-backend/livekit/LivekitMediaTrack.ts`, `LivekitMemberMedia.ts` | the view models get `MediaTrack`s with `attach`; nothing in `src/` imports a LiveKit participant any more |
## What still binds Element Call to LiveKit
Each is a gap in the SDK's API rather than something to rename; the import goes
away when the SDK grows the piece. The names Element Call owes to the outside
stay as they are: the `livekit.livekit_service_url` key in `config.json`, the
persisted setting keys `extended-livekit-logs` and `custom-transport`, the
Posthog fields `callBackend` and `callReconnectingCountLivekit`, the rageshake
field `call_backend`, and the translated error strings.
| Gap | Where it shows |
| ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| Preview before `join()`: local tracks for the lobby, handed out as `MediaTrack`s | `LobbyView` (`usePreviewTracks`), `VideoPreview` (`LocalVideoTrack`, `facingModeFromLocalTrack`) |
| Device enumeration without LiveKit's helpers | `MediaDevices` (`createMediaDeviceObserver`), `CallView` (`Room.getLocalDevices`) |
| A `logging` option: log level and a sink for the transport's logs | `initializer.tsx`, the `extended-livekit-logs` setting |
| A `capabilities` field: output selection and background processors, asked of the SDK rather than of LiveKit | `CallFooterViewModel` (`supportsAudioOutputSelection`, `supportsBackgroundProcessors`) |
| Element Call's error and analytics types on the SDK's | `PosthogEvents` (`DisconnectReason`), `errors.ts` (`ConnectionError`, `LivekitConnectionError`) |
+78
View File
@@ -0,0 +1,78 @@
<!doctype html>
<!--
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
-->
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>MatrixRTC SDK harness</title>
<style>
body {
margin: 1rem;
font-family: system-ui, sans-serif;
}
form {
display: flex;
gap: 0.5rem;
flex-wrap: wrap;
}
#members {
display: flex;
gap: 0.5rem;
flex-wrap: wrap;
}
#members section {
width: 240px;
}
#members video {
width: 100%;
aspect-ratio: 4 / 3;
background: #222;
}
#members h2 {
font-size: 1rem;
margin: 0.25rem 0;
}
</style>
</head>
<body>
<form>
<input
name="homeserver"
placeholder="homeserver"
value="https://synapse.m.localhost"
/>
<input name="username" placeholder="username" />
<input name="password" type="password" placeholder="password" />
<input name="room" placeholder="room id or alias" />
<!-- A host that already holds a session passes it instead of a password -->
<input name="accessToken" type="hidden" />
<input name="userId" type="hidden" />
<input name="deviceId" type="hidden" />
<button type="submit">Start</button>
<button type="button" id="microphone" aria-pressed="true" hidden>
Microphone
</button>
<button type="button" id="camera" aria-pressed="true" hidden>
Camera
</button>
<button type="button" id="screenShare" aria-pressed="false" hidden>
Share screen
</button>
<button type="button" id="leave" hidden>Leave</button>
</form>
<p id="status" data-testid="status"></p>
<!-- A line of text to the other members, beside the media -->
<form id="data" hidden>
<input name="text" placeholder="message" />
<button type="submit">Send</button>
</form>
<ul id="messages"></ul>
<main id="members"></main>
<script type="module" src="./main.ts"></script>
</body>
</html>
+289
View File
@@ -0,0 +1,289 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* The smallest consumer of the SDK: log in, join a room, show every member's
* camera and play every remote member's microphone, mute, leave. What a host
* writes, and nothing a host would not.
*/
import { logger } from "matrix-js-sdk/lib/logger";
import { combineLatest, type Observable, of, switchMap } from "rxjs";
import {
type AudioMediaTrack,
createRTCSlot,
E2eeType,
type LocalRTCMember,
MatrixRTCMode,
ObservableScope,
type RemoteRTCMember,
type RTCParticipation,
trackBySource$,
type VideoMediaTrack,
} from "@element-hq/matrixrtc-sdk";
import { createSession, joinRoom, type Login } from "./session";
const form = document.querySelector("form")!;
const status = document.getElementById("status")!;
const members = document.getElementById("members")!;
const dataForm = document.getElementById("data") as HTMLFormElement;
const messages = document.getElementById("messages")!;
const buttons = {
microphone: document.getElementById("microphone") as HTMLButtonElement,
camera: document.getElementById("camera") as HTMLButtonElement,
screenShare: document.getElementById("screenShare") as HTMLButtonElement,
leave: document.getElementById("leave") as HTMLButtonElement,
};
// So that a test, or a bookmark, can fill the form from the URL
const params = new URLSearchParams(location.search);
for (const input of form.querySelectorAll("input"))
input.value = params.get(input.name) ?? input.value;
form.addEventListener("submit", (event) => {
event.preventDefault();
const fields = new FormData(form);
const field = (name: string): string => fields.get(name) as string;
const login: Login = field("accessToken")
? {
accessToken: field("accessToken"),
userId: field("userId"),
deviceId: field("deviceId"),
}
: { username: field("username"), password: field("password") };
void start(field("homeserver"), login, field("room"));
});
async function start(
homeserver: string,
login: Login,
roomIdOrAlias: string,
): Promise<void> {
try {
status.textContent = "Logging in";
const client = await createSession(homeserver, login);
const room = await joinRoom(client, roomIdOrAlias);
const scope = new ObservableScope();
const slot = createRTCSlot(scope, client, room, {
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
matrixRTCMode: MatrixRTCMode.Compatibility,
});
const rtcParticipation = slot.join({
publish: [{ source: "microphone" }, { source: "camera" }],
});
rtcParticipation.state$.pipe(scope.bind()).subscribe((state) => {
status.textContent =
state.kind === "failed" ? `Error: ${state.error.message}` : state.kind;
});
showMembers(scope, rtcParticipation);
showMessages(scope, rtcParticipation);
toggle(buttons.microphone, async (enabled) =>
setPublished(rtcParticipation, "microphone", enabled),
);
toggle(buttons.camera, async (enabled) =>
setPublished(rtcParticipation, "camera", enabled),
);
toggle(buttons.screenShare, async (enabled) =>
setPublished(rtcParticipation, "screenShare", enabled),
);
buttons.leave.hidden = false;
buttons.leave.onclick = (): void => {
rtcParticipation.leave();
scope.end();
members.replaceChildren();
messages.replaceChildren();
dataForm.hidden = true;
for (const button of Object.values(buttons)) button.hidden = true;
status.textContent = "Left";
};
} catch (e) {
status.textContent = `Error: ${e}`;
logger.error(e);
}
}
/**
* Turns one of our sources on or off: a microphone or camera is muted and
* unmuted while it exists and published when it does not; a screen share is
* published and unpublished.
*/
async function setPublished(
rtcParticipation: RTCParticipation,
source: "microphone" | "camera" | "screenShare",
enabled: boolean,
): Promise<boolean> {
const track = rtcParticipation.localMember$.value?.tracks$.value?.find(
(t) => t.source === source,
);
if (track && source === "screenShare") {
if (!enabled) await rtcParticipation.unpublish(source);
return enabled;
}
if (track) return track.setEnabled(enabled);
if (!enabled) return false;
await rtcParticipation.publish({ source });
return true;
}
/** A pressed button means the source is on; the SDK says what it could do. */
function toggle(
button: HTMLButtonElement,
set: (enabled: boolean) => Promise<boolean>,
): void {
button.hidden = false;
button.onclick = (): void => {
void set(button.ariaPressed !== "true").then((enabled) => {
button.ariaPressed = String(enabled);
});
};
}
/** Sends what the form holds on the "chat" topic, and lists what arrives. */
function showMessages(
scope: ObservableScope,
rtcParticipation: RTCParticipation,
): void {
dataForm.hidden = false;
dataForm.onsubmit = (event): void => {
event.preventDefault();
const text = new FormData(dataForm).get("text") as string;
rtcParticipation.sendData("chat", text).then(
() => dataForm.reset(),
(e: unknown) => {
status.textContent = `Error: ${e}`;
},
);
};
rtcParticipation.data$
.pipe(scope.bind())
.subscribe(({ member, topic, text }) => {
const line = messages.appendChild(document.createElement("li"));
line.dataset.testid = "message";
line.dataset.topic = topic;
line.dataset.userId = member.userId;
line.textContent = `${member.displayName$.value}: ${text}`;
});
}
function showMembers(
scope: ObservableScope,
rtcParticipation: RTCParticipation,
): void {
const tiles = new Map<string, HTMLElement>();
combineLatest([
rtcParticipation.localMember$,
rtcParticipation.remoteMembers$,
])
.pipe(scope.bind())
.subscribe(([local, remote]) => {
const current = local === null ? remote : [local, ...remote];
for (const [id, tile] of tiles)
if (!current.some((m) => m.memberId === id)) {
tile.remove();
tiles.delete(id);
}
for (const member of current)
if (!tiles.has(member.memberId)) {
const tile = memberTile(scope, member);
tiles.set(member.memberId, tile);
members.append(tile);
}
});
}
function memberTile(
scope: ObservableScope,
member: LocalRTCMember | RemoteRTCMember,
): HTMLElement {
const tile = document.createElement("section");
tile.dataset.testid = "member";
tile.dataset.userId = member.userId;
tile.dataset.local = String(member.local);
const name = tile.appendChild(document.createElement("h2"));
member.displayName$.pipe(scope.bind()).subscribe((n) => {
name.textContent = n;
});
render(
scope,
trackBySource$(scope, member.tracks$, "camera"),
tile.appendChild(document.createElement("video")),
);
// Our own microphone would only echo
if (!member.local)
render(
scope,
trackBySource$(scope, member.tracks$, "microphone"),
tile.appendChild(document.createElement("audio")),
);
return tile;
}
/**
* Plays a track on an element and labels the element with the track's state,
* so that the page shows what the member is sending and how well it arrives.
*/
function render(
scope: ObservableScope,
track$: Observable<AudioMediaTrack | VideoMediaTrack | undefined>,
element: HTMLMediaElement,
): void {
let attached: AudioMediaTrack | VideoMediaTrack | undefined;
track$.pipe(scope.bind()).subscribe((track) => {
attached?.detach(element);
attached = track;
track?.attach(element);
});
scope.onEnd(() => attached?.detach(element));
const label = (
key: string,
value$: Observable<string | number | boolean | undefined>,
): void => {
value$.pipe(scope.bind()).subscribe((value) => {
if (value === undefined) delete element.dataset[key];
else element.dataset[key] = String(value);
});
};
const of$ = <T>(
pick: (track: AudioMediaTrack | VideoMediaTrack) => Observable<T>,
): Observable<T | undefined> =>
track$.pipe(switchMap((track) => (track ? pick(track) : of(undefined))));
label(
"muted",
of$((t) => t.muted$),
);
label(
"encrypted",
of$((t) => of(t.encrypted)),
);
label(
"active",
of$((t) => (t.kind === "audio" ? t.isActive$ : of(undefined))),
);
const stats$ = of$((t) => t.stats$(1000));
label("frameWidth", stats$.pipe(switchMap((s) => of(frames(s)?.frameWidth))));
label("frames", stats$.pipe(switchMap((s) => of(frames(s)?.count))));
}
/** The frame counters an RTP stream reports, from either end of it. */
function frames(
stats: RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined,
): { frameWidth: number | undefined; count: number | undefined } | undefined {
if (stats === undefined) return undefined;
return {
frameWidth: stats.frameWidth,
count:
stats.type === "inbound-rtp"
? (stats as RTCInboundRtpStreamStats).framesDecoded
: (stats as RTCOutboundRtpStreamStats).framesEncoded,
};
}
+99
View File
@@ -0,0 +1,99 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
ClientEvent,
createClient,
type MatrixClient,
MemoryStore,
type Room,
SyncState,
} from "matrix-js-sdk";
import { KnownMembership } from "matrix-js-sdk/lib/types";
export type Login =
| { username: string; password: string }
| { accessToken: string; userId: string; deviceId: string };
/**
* Returns a client that has finished its first sync, logging in with a
* password unless the caller already holds a token.
*/
export async function createSession(
homeserver: string,
login: Login,
): Promise<MatrixClient> {
const credentials =
"accessToken" in login ? login : await logIn(homeserver, login);
const client = createClient({
baseUrl: homeserver,
...credentials,
store: new MemoryStore(),
useAuthorizationHeader: true,
fallbackICEServerAllowed: true,
});
await client.initRustCrypto({ useIndexedDB: false });
await client.startClient();
await new Promise<void>((resolve) => {
const onSync = (state: SyncState): void => {
if (state !== SyncState.Prepared && state !== SyncState.Syncing) return;
client.off(ClientEvent.Sync, onSync);
resolve();
};
client.on(ClientEvent.Sync, onSync);
});
return client;
}
/**
* Joins a room and returns it as the sync loop maintains it. The room object
* `joinRoom` itself returns for a room joined just now is a detached copy
* that never receives the state the sync delivers, and a session built on
* it would never see a member.
*/
export async function joinRoom(
client: MatrixClient,
roomIdOrAlias: string,
): Promise<Room> {
const { roomId } = await client.joinRoom(roomIdOrAlias);
const joinedRoom = (): Room | undefined => {
const room = client.getRoom(roomId);
return room?.hasMembershipState(client.getUserId()!, KnownMembership.Join)
? room
: undefined;
};
return (
joinedRoom() ??
new Promise<Room>((resolve) => {
const onSync = (): void => {
const room = joinedRoom();
if (room === undefined) return;
client.off(ClientEvent.Sync, onSync);
resolve(room);
};
client.on(ClientEvent.Sync, onSync);
})
);
}
async function logIn(
homeserver: string,
{ username, password }: { username: string; password: string },
): Promise<{ accessToken: string; userId: string; deviceId: string }> {
const login = await createClient({ baseUrl: homeserver }).login(
"m.login.password",
{ identifier: { type: "m.id.user", user: username }, password },
);
return {
accessToken: login.access_token,
userId: login.user_id,
deviceId: login.device_id,
};
}

Some files were not shown because too many files have changed in this diff Show More