Port of edf1feae from johannes/rtc-remote-publish onto the SDK split,
where
the MatrixRTC logic lives in sdk/ rather than src/state/CallViewModel/.
Where the original lands:
- getLocalTransport's remote fallback: the LiveKit backend's new
findRemotePublishTransport (sdk/src/media-backend/livekit/
RemotePublishTransport.ts), which tries the candidates from the
js-sdk's
getRemoteLivekitPublishCandidates oldest first and keeps the first
whose
homeserver issues a token with the publish grant.
discoverLocalTransport
asks the backend for it after the host's fallback, with a snapshot of
the
session's memberships from RTCParticipation, and throws
MatrixRTCTransportMissingError where that finds nothing too. Transport
discovery stays backend neutral: the MSC4195 token check is the
backend's.
- LocalMember's "no delegation on a remote SFU" and the custom
transport's
server_name: LivekitBackend.prepareLocalTransport locates the
transport on
the homeserver it names (serverNameOfTransport) and skips the
delegation
probe where that is not our own.
- SessionBehaviors' membership.getTransportServerName: the same helper
in
LivekitBackend.mediaFor.
- decodeSFUJWTPayload, SFUJWTPayload.video.canPublish and the exported
isCsApiUnavailable: sdk/src/media-backend/livekit/auth/.
- DeveloperSettingsTab, the widget capabilities, the locale string and
the
js-sdk pin (9405287c, 44.0.0): applied as is. The error boundary
snapshots follow the string.
Tests: the LocalTransport cases go to RemotePublishTransport.test.ts and
LocalTransport.test.ts, the LocalMember remote case and the
SessionBehaviors
server-name case to LivekitBackend.test.ts, plus the memberships
snapshot in
RTCParticipation.test.ts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The SDK MediaView renders a plain video element, so LiveKit's
data-lk-source attribute is gone. Expose data-source from the SDK track
and point the screen-share spec at it.
The SDK replaced `src/livekit` and the remote-member modules before that PR
landed on `main`, so its `src/livekit/auth/*` and `remoteMembers/ConnectionMap.ts`
have no consumer here: the same code lives under `sdk/src/media-backend/livekit`.
The imports of them that `main` added to the view model, the developer
settings tab and the test utils go with them.
The bumped js-sdk reports the legacy identity as rtcBackendIdentity for every membership, so a device that rejoins after an improper leave, which has two memberships until the stale one expires, produced two tiles with the same key and crashed the call view. The membership id tells them apart; the audio renderer keys by it too.
The developer setting that overrode the LiveKit service URL becomes a
JSON transport, so that a `url`-only or combined transport can be tested
against a homeserver that does not advertise one yet. The settings modal
shows each LiveKit room's role instead of a never-populated isLocal flag.
Folds "Avoid null in transport dev setting" from the same PR. The setting feeds the SDK's transport option; the config fallback becomes a transport the host builds.
Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
A transport may soon carry a LiveKit SFU `url` instead of, or as well as,
a `livekit_service_url`, so a service URL alone no longer identifies it.
Pass the whole transport to the tiles' debug stats and print it as JSON in
log lines instead of picking out the service URL.
Element Call's media view models carry the member's raw transport from the SDK's TransportMetadata to the tiles.
A member authenticated through the legacy JWT endpoint publishes under a
`user:device` identity, while one authenticated through the default
endpoint publishes under a hashed identity. Both are listed in the
membership's backend identities, so match against the whole list rather
than only the primary one.
Element Call's test membership mock, matching the SDK's RTCMember.
ploy previews run pr-deploy.yaml from main, which downloads
build-output-sdk,
so a renamed artifact can never be found from a pull request. Keep the
package
name and point build:sdk:production and build:sdk:development at the
CallViewModel bundle the preview is meant to carry.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MediaBackendPlan.md and MemberMediaPlan.md described changes that have
landed. SdkMigration.md keeps only what moved from src/ into sdk/, with
the
paths as they are today. README.md gains the onboarding notes on the
three
API files, and its example uses createRTCSlot, which replaced
createMatrixRTCClient.
Unit tests for the pieces that had none: which transport the local
member advertises (override, homeserver, fallback, none), what the js-sdk
join is called with, how a publish request becomes LiveKit capture and
publish options, and the room options the connection factory builds from
the initial requests and the media quality settings, including what a
resolved connection reports while it is up. The backend's `mediaFor` gains
the cases from Element Call's member matching tests: a member on another
transport, a member changing transport, and a member without one.
The publisher is driven through a real LocalParticipant whose device and
signalling calls are faked, so that LiveKit's own pending-publication
bookkeeping is part of what is tested: the cases ported from Element
Call's Publisher tests (one permission prompt for both devices, nothing
created while muted, the upstream held back until the member publishes,
a mute that lands while the track is still being created) plus the
publish, unpublish and destroy paths.
The local media runs on a fake connection map: publishing before and
after the connection exists, republishing on a new connection, mutes
remembered across connections, the first publish error kept, and the
output device applied to the current room and the ones to come.
The participation runs on a fake media backend: the join options, the
state machine through connecting, connected, reconnecting, failed and
left, publishing only while joined, the delayed leave delegation, the
member ids per mode, the key replay and the data channel's sender check.
The loop that drives `scope.reconcile` was marked as running until the
promise of its last run had settled, which is a few microtasks after the
loop itself had checked for new values. A value arriving in that window
found the loop "running", was left to it, and was never processed: a
connection appearing at that moment would never get its publisher.
The flag is now cleared the moment the loop ends, so such a value starts
a new loop. The regression test drives the window microtask by microtask.
How a LiveKit transport is authenticated with by shape and role, the server name on connections and the options that now take a Transport.
(cherry picked from commit 26d9e1b0ae1e1f1ba8c3689bd50ba2e34aa0e820)
The developer setting that overrode the LiveKit service URL becomes a
JSON transport, so that a `url`-only or combined transport can be tested
against a homeserver that does not advertise one yet. The settings modal
shows each LiveKit room's role instead of a never-populated isLocal flag.
On the SDK the participation options take a whole Transport for the override and the fallback instead of two LiveKit service URLs, which also takes the last LiveKit knowledge out of transport discovery.
(cherry picked from commit d3eb40db0bcf7370418c4948049b3eac176bbdc1)
The OpenID flow moves to src/livekit/auth/openID.ts with the SFU config
types in a sibling file, takes an options object, and replaces the
MatrixRTC mode with the role we take on the transport plus the session's
slot ID instead of a hard-coded m.call#ROOM.
This changes behaviour: publishers on a livekit_service_url transport
always use the legacy /sfu/get endpoint, so that subscribers can predict
their user:device participant identity, and subscribers always use the
default /get_token endpoint. The MatrixRTC mode no longer influences which
endpoint is used.
On the SDK this lands as sdk/src/media-backend/livekit/auth/ with the connection map, the discriminated connection state and the client-server API flow of the following commits of element-hq/element-call#4331, which rewrite the same files:
- Model ConnectionState as a discriminated union
- Show the whole transport in debug UI and logs (SDK part)
- Let the publishing Connection authenticate itself
- Reduce ConnectionManager to a ConnectionMap
- Use the unstable LiveKit transport type
- Identify transports by server name as well as URLs
- Get SFU tokens and delegate delayed leaves through the MSC4195 CS API
- Fall back to get_token as subscriber on widget API errors
- Share a connection between homeservers using the same JWT service
- Make the auth choice mode independent again
Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
(cherry picked from commit adb380cd9ed108856e42d992fc03ced8c44f1e56)
A member authenticated through the legacy JWT endpoint publishes under a
`user:device` identity, while one authenticated through the default
endpoint publishes under a hashed identity. Both are listed in the
membership's backend identities, so match against the whole list rather
than only the primary one.
On the SDK this is RTCMember.backendIdentities, which the LiveKit backend matches media and data packets against; the harness keys its tiles by the membership id, which the backend identity no longer tells apart.
(cherry picked from commit c9aad2a2b3279782d101c9bc6c80af0417ccbb92)
The js-sdk at this commit also renamed MatrixRTCSession.leaveRoomSession to leave, which the SDK follows.
(cherry picked from commit 405348945febc1df10edf9c00748f1edd6b39e71)
compound-web 10.4.0 treats a PortalRoot whose root is null as one whose root
is not available yet, so floating-ui never mounts the portal and every tooltip
label, and with it every icon button's accessible name, is missing. The
component defaulted portalRoot to null and wrapped the call unconditionally.
Wrap the call only when the host supplied a root, keeping null as the
not-yet-available case Compound defines, and say so on the prop. Cover it in
CallPortalRoot.test.tsx. Drop the git allowBuilds entry for compound-web now
that it is a release again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The specs replace the transports the e2e homeserver advertises with each
shape a migrating deployment may produce and record which token endpoints
each side used: a url-only or combined transport on both sides goes
through the CS API alone, a legacy guest joining a url host uses the
legacy endpoint to publish and the CS API to subscribe, a subscriber falls
back to the JWT service when the homeserver lacks MSC4195, a publisher
gets an error instead, a delayed leave delegated through the CS API fires
on disconnect, and subscribers get tokens for a remote homeserver's SFU
over federation.
The JWT service's app service registration has to cover users of the e2e
Synapse's real server name for the get_token flow, and lk-jwt-service
0.8.0 implements the endpoints.
The developer setting that overrode the LiveKit service URL becomes a
JSON transport, so that a `url`-only or combined transport can be tested
against a homeserver that does not advertise one yet. The settings modal
shows each LiveKit room's role instead of a never-populated isLocal flag.
src/livekit/auth now fronts two flows. For a transport with a LiveKit SFU
`url`, tokens come from our own homeserver's MSC4195 get_token endpoint
and delayed leaves are delegated through delegate_delayed_leave, both via
the js-sdk's unstable client methods. Transports with only a
livekit_service_url keep using the OpenID flow against the JWT service.
A transport advertising both shapes lets a subscriber fall back to the
OpenID flow when the homeserver answers get_token with a 404 (Synapse
reports M_UNRECOGNIZED for unknown routes rather than M_NOT_FOUND), so a
client can still join calls hosted on servers that have not deployed
MSC4195 yet. Publishers never fall back: a homeserver that advertises a
`url` but does not implement the endpoints is misconfigured and the
admin should hear about it, so they get NoMatrix2AuthorizationService.
Delegation uses the same flow as the token so that the participant
identity the service computes matches.
The local member only probes a transport's delegate_delayed_leave
endpoint when it has a service URL. The developer settings tab validates
a custom service URL through the same facade.
Temporary, until matrix-org/matrix-js-sdk#5588 merges and this can go
back to a develop bump. The branch only changes the get_token request
body from a member object to a member_id, so this is kept separate from
the code that uses it and can be dropped without touching that code.
A transport with only a LiveKit SFU `url` has to be authenticated through
our own homeserver, which needs to know the name of the server owning the
SFU. Carry that name alongside the transport as a TransportLocator,
derived from the member's user ID for remote transports and from the
client's domain for our own, and key the connection map on the SFU URL,
the service URL and the server name together so that `url`, service URL
and combined transport shapes are deduplicated consistently.
UnstableLivekitTransport covers every shape a homeserver may advertise
while MSC4195 evolves: a LiveKit SFU `url`, a `livekit_service_url`, or
both. Switching to it means a service URL is no longer guaranteed, so the
remaining code that relies on one checks for it explicitly: delegation
probing skips transports without one, the connection map keys transports
by whichever URL they carry, and the OpenID flow rejects transports
without a service URL until the client-server API flow lands.
createConnectionMap$ now returns the behavior of connections and their
participants directly and takes an already resolved local transport
instead of an observable of it. The CallViewModel wraps it with mapScoped
so that a new map is built, in a fresh scope, whenever the local transport
changes, and swallows local transport errors there since the local member
already surfaces them in the UI.
getLocalTransport no longer authenticates with the SFU up front and hands
the resulting config to the connection. Instead the ConnectionManager tags
the connection whose transport equals the local one as the publisher, and
every Connection fetches its own SFU config for its role.
The local member now only joins the MatrixRTC session once its connection
is authenticated, so that the LiveKit room exists before remote members
try to get a subscriber token for it. Subscribers authenticating with the
OpenID flow lack permission to create the room and would otherwise fail
without retrying, causing a cascade of leaves and rejoins at the start of
a call.
The OpenID flow moves to src/livekit/auth/openID.ts with the SFU config
types in a sibling file, takes an options object, and replaces the
MatrixRTC mode with the role we take on the transport plus the session's
slot ID instead of a hard-coded m.call#ROOM.
This changes behaviour: publishers on a livekit_service_url transport
always use the legacy /sfu/get endpoint, so that subscribers can predict
their user:device participant identity, and subscribers always use the
default /get_token endpoint. The MatrixRTC mode no longer influences which
endpoint is used.
The old reducer keyed items by service URL and mutated the accumulator's
entries in place, so a transport without a service URL could not be
represented and items were not safe to compare. Group members by their
Connection object instead and carry the whole transport on the item, which
the audio renderer now receives in place of a URL.
A transport may soon carry a LiveKit SFU `url` instead of, or as well as,
a `livekit_service_url`, so a service URL alone no longer identifies it.
Pass the whole transport to the tiles' debug stats and print it as JSON in
log lines instead of picking out the service URL.
The enum mixed our own lifecycle states with a copy of livekit-client's
connection states, and errors were smuggled through the same observable as
a separate union member. A discriminated union keeps the LiveKit state
nested under "authenticated", carries the error as data, and lets the
local member represent the time before a connection exists as an
"initializing" state instead of null.
A member authenticated through the legacy JWT endpoint publishes under a
`user:device` identity, while one authenticated through the default
endpoint publishes under a hashed identity. Both are listed in the
membership's backend identities, so match against the whole list rather
than only the primary one.
The container IP is not routable from the host when Docker runs in a VM
(colima, Docker Desktop), so calls against the dev stack drop once ICE
fails. Advertising 127.0.0.1 instead lets the browser reach the SFU through
the published ports. Firefox ignores loopback candidates, so this only
works in Chromium-based browsers.
The list reporter is the only progress a hung CI run leaves behind, and a
global timeout under the job's 60 minute limit means a hung run still ends
with a report instead of being cancelled by GitHub.
The SDK now takes `publish` requests and audio output control directly
in the main options object, removing the separate `LocalMediaInputs`
type. Audio output device selection moves from a reactive behavior to an
imperative method on `MatrixRTCClient`, giving hosts direct control over
when devices switch.
The session speaks MatrixRTC and sees media through MediaBackend in
sdk/src/backend/api.ts. Everything LiveKit lives in
sdk/src/backend/livekit/,
the only place in the SDK that imports livekit-client; the import
boundary rule
now enforces that outside sdk/src/backend/ the LiveKit packages appear
in type
imports only.
The seam sits above the connection manager: one connection per transport
url
is the LiveKit multi-SFU strategy, not a MatrixRTC concept, so a
cascading SFU
or full mesh replaces the folder, not the session. A backend prepares
the local
transport the session discovered (the token exchange and the delegation
probe,
reported as canDelegateDelayedLeave), hands the delayed leave over per
delay
id, gives the local member one publish switch plus publish/unpublish,
builds a
remote member's media from its membership and registers that
membership's
transport for as long as the member lives, and reports its connections
for the
transport registry, which keys on the raw transport serialised with
sorted
keys. The backend's context carries no Matrix client (a backend picks
what it
calls in its own options), no membership list, and no js-sdk session:
media
keys arrive as a stream replayed on subscribe, and MatrixKeyProvider
subscribes
to it. The MatrixRTC mode no longer reaches LiveKit; it becomes a
tokenEndpoint
choice made where the default backend is built.
Two behaviour changes ride along. Publishing is one boolean, joined and
homeserver reachable, where before a sync hiccup after leave() resumed
the
upstream of tracks that were meant to stay silent. And
ResolvedConnection keeps
one ResolvedTransport object per fetch, so a state change no longer
reads as a
change of what was fetched.
MatrixRTCClientOptions gains backend?: MediaBackendFactory, defaulting
to
createLivekitBackend built from the client options.
authenticateWithTransport
takes tokenEndpoint instead of matrixRTCMode.
media, track, source, publish request, active and attach/detach, between
participant and transport, and the participant row names the adapter as
the only reader.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Chromium's fake microphone never registers as a speaker at the SFU, and Firefox's steady tone does so in about two runs out of three, with or without the browser's audio processing. The test stays, out of the gate, until the dev SFU gets a lower speaker threshold or the harness a louder fake device.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
membership$ is typed as RTCMembership, the subset of the js-sdk CallMembership that hosts read. UnknownCallError is UnknownRTCError throughout, not only in the export. applicationData reaches the membership whole, since the js-sdk takes it next to the intent. The plan records the three slices as implemented.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One way to put media on the session: publish(request) resolves with the local track once LiveKit has it, and unpublish(id) removes it. A request names a source with its device, processor and capture settings, so CaptureSettings on the client options goes, as do setMicrophoneEnabled and setCameraEnabled on the client and the four screen share fields on the local member; sharing is tracks$ having a screenShare track, and a capture failure rejects the promise. LocalMediaInputs becomes the list of initial requests, which also become the room defaults so that one permission prompt covers microphone and camera. The publisher keeps the requests across a reconnection and republishes what is not muted. The harness gains a Share screen button.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Our own tracks are LocalAudioMediaTrack and LocalVideoMediaTrack, with setEnabled and setDevice, and on video facingMode$, switchFacingMode and setProcessor. switchCamera on the local media goes, as do the device and processor behaviors in LocalMediaInputs: a host calls the track when its setting changes instead of the SDK watching behaviors it does not own. The input device ids and the processor stay as initial values for the tracks created at the join. setMicrophoneEnabled and setCameraEnabled stay for the case of no track yet.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Unit tests for tracks$ and trackBySource$, an end-to-end check of isActive$ through a mute cycle (skipped on Chromium, whose fake microphone the SFU never hears), the architecture document brought in line with the track list, and the changed behaviour recorded in the plan.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Update the public API to export the primitives and operators the call
model needs, and change `LocalMediaInputs` from observable flags to
immediate booleans with imperative setters. Add session timings, media
quality and capture settings as configuration options, and allow the
host to override the transport discovery with explicit URLs.
Set up `sdk/` as a package shaped exactly like `component/`: its own
package.json with peer dependencies, the pnpm workspace marker so a host can
install it as a git dependency, a build tsconfig for type emission, and a vite
lib config with an externals list that `pnpm lint:externals` now checks for
both library builds.
The package holds only the interface for now. `createRtcSession` returns an
empty object; `RtcSession`, `RtcMember`, `MemberMedia` and `TransportMetadata`
are the contract Element Call's `CallViewModel` will be rebuilt on (see
sdk-plan.md). The shared primitives are re-exported from `src` until the first
implementation slice moves them.
`sdk/dev` is the smallest consumer of the SDK: a plain page that logs in, joins
a room and renders one tile per member, served by `pnpm dev:sdk` on port 3002.
Playwright starts it as a third web server, and `playwright/sdk/smoke.spec.ts`
has two browsers join one session. It fails on the status line with the SDK's
own error until there is an implementation behind the interface.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The `sdk/` directory is about to become the MatrixRTC SDK package, so the
demo that lives there — one bundle of Element Call's CallViewModel for a plain
page or widget — moves aside, unchanged. It keeps building under a script and
vite config of its own, `pnpm build:sdk-target-based-on-call-view-model`, into
its own `dist` rather than the repository root's, and no longer copies the
developer's config.json out of `public`. Its README says which script to run
and where the output lands.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
I learned that makeKeyDelay has been deprecated in matrix-js-sdk ever since the switch to the new encryption manager, so this config option has no effect.
Compound's Tooltip floats its bubble through floating-ui's FloatingPortal,
into the body: outside the element the component build confines its
stylesheet to, so a label tooltip shows as bare text at all times. An
enclosing FloatingPortal rooted at the container makes the nested portals
land inside it instead.
The call goes in an isolating wrapper there, standing in for the standalone
page's #root, so that its footer layer no longer paints above the modals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With this change, views now read every reactive value through `useBehavior`, as
`architecture.md` already prescribes. Where a view reads a plain stream, the view
model now exposes a `Behavior`.
* Give the microphone level its own observable
* Add a microphone level meter
* Show speakers and microphones in the quick audio menu
* Wire speaker selection through the call footer
* Add stories for the meter and the device menu
* Add end-to-end specs for the quick audio menu
* Keep the footer while a menu opened from it is open
* Use the shared audio capture stub in the lobby test
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Say what earns a comment its place
- Add a comments section to the code style: no comment by default, the reason
only, and the kinds that go stale.
- Point to it from the PR checklist in AGENTS.md.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Fixes the following warning:
[warn] (!) Your Vite config uses features that are unsupported by `configLoader: 'native'`, which is planned to become the default in a future major version of Vite:
- import "./vite.config" without a file extension (vitest.config.ts:3:35). Add the file extension
Set `VITE_CONFIG_NATIVE_IGNORE_WARNING=true` to suppress this warning.
As of the removal of 'Legacy' mode, there can no longer be a difference between the transport you advertise in your membership and the transport you publish media on, so the local transport code can be simplified considerably. For instance, the function which gets the local transport can simply return a promise rather than being reactive.
I kept the local transport as an Observable in other modules so that they could easily be tested with existing marble tests.
LiveKit's livekit-ice logger records each local ICE candidate, ICE
candidate errors and ICE connection state changes, but only at debug,
which we cap to info unless extended LiveKit logs are switched on. That
means a 'could not establish pc connection' rageshake carries no way to
tell 'no relay candidates at all' from 'relay candidates that never
paired'. Keep that one logger at debug regardless of the toggle.
* Add AGENTS.md documenting the repo's conventions
Codifies what is currently tribal knowledge or only discoverable by reading CI
config: the view model / view contract, the rule that nothing reads the page,
Clean Code ordering, the three test layers, and reuse-before-you-build.
The root AGENTS.md is the always-loaded minimum and routes to docs/agents/, so
an agent loads only the detail its task needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Allow either hand at the commit, and measure continuous redraws
Committing from the agent rules out running it where there is no git
identity, a sandbox included. Either is now fine: commit on the user's
word where the setup allows it, or write the message and hand it over.
Redrawing continuously is the other addition. A level meter reported a
fresh value every animation frame, so its whole menu reconciled sixty
times a second, silence included, and nobody noticed until review. A PR
that adds anything of the kind now says what it costs, or better, counts
the redraws — commits rather than render calls, since React runs a
component it then discards.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The "has no matching matrix call member" warning fired on every re-render
(i.e. every active speaker update) and also for the local participant's own
track, which is never rendered here anyway. Skip local tracks and warn once
per unexpected identity.
On browsers without a "default" pseudo-device (Firefox, Safari), the
virtual default output entry was appended after the physical devices, so
with no saved preference EC selected the first physical device and pinned
every remote audio element to it with setSinkId. Pinned sinks are not
re-routed by the browser: on Firefox/Linux a Bluetooth headset switching
from A2DP to HFP when its microphone is opened (i.e. on unmute) destroys
the pinned sink and all remote audio goes silent, with no error and no
fallback (rageshake 17320).
List the virtual default first so it is the fallback both when nothing was
chosen and when the chosen output disappears, and stop labelling it with
the first device's name since the browser default is not necessarily that
device.
IMO this is where the delegation calls should have lived all along, since the leave event is part of the membership lifecycle, and we otherwise end up with an awkward hack to ignore transport updates.
Doing this now ensures that the client won't send any delegation requests if delegation is unsupported, and prepares the code for a future change in which we use the dedicated delegation endpoint from the CS API.
Since we removed the extra 'legacy' mode, meaning there's nothing simplified anymore by translating things over to a JWT service version.
(Also, simplify some things by making the mode static for the duration of a call.)
`downloadMedia` exists because a widget's client cannot fetch media: it
has no access token, and its only route to a file is MSC4039 over the
widget API. A component host hands over a client that does hold
credentials, so Element Call fetches media with it and the callback was
never needed there. It comes off `ElementCallHostBridge`, leaving a host
with only what it must understand, and stays on the internal HostBridge
for the widget.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The theme is state — what Element Call should look like right now — and
so belongs beside `language` as a prop, not on the imperative handle
(where it was a request, `setTheme`, because the internal host bridge
speaks the widget API and a widget's host sends theme changes as
requests) and not in the configuration (where `config.theme` only ever
set the starting theme).
The `theme` prop feeds the same channel the rest of Element Call listens
to for a host's theme, replayed so that whatever subscribes after the
host has set it still hears the current one. Changing it re-themes the
container and nothing else; unlike the language, it is per component.
`setTheme` and `config.theme` are gone, and the harness gets a theme
picker in place of its per-pane buttons.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The component bundled English alone: the standalone app fetches its
locale files at runtime from URLs its own build emits, which a host
serving the library from elsewhere could not resolve, so bundling one
language was the self-contained option. Now every locale is a chunk of
its own that the host's bundler loads the first time it is needed, with
English still bundled in so that the fallback never waits.
Element Call starts in the browser's language and follows the host's own
setting through a `language` prop; `supportedLanguages` says what it
accepts. Translations are shared by every Element Call on the page, so
the most recently set language wins for all of them. The harness gets a
language picker, and the app and the component share the parsing of
locale paths.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The call reached for react-router in five places to send the user
"home": on leaving without a lobby, from the lobby's recents link, from
the post-call screen, from the error page's return button and from the
header logo. Home is the standalone app's home page; the call has no
idea where that is, and a component has no such place at all — its host
decides what follows a call. Yet the component had to mount a
MemoryRouter just so those hooks would not throw.
`useLeaveToHome` is the way home as the shell supplies it: the app
provides `navigate("/")` from inside its router, the component provides
nothing, and everything that used to link to "/" now either calls it or,
when there is none, offers no way out. The logo becomes a plain logo,
the recents and "not now" links disappear, the error page's button does
too. `ClientProvider`'s logout goes the same way. The component no
longer renders a router.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`ElementCallConfiguration` was every URL parameter, which put the widget
plumbing (widgetId, parentUrl, isWidget), the user's identity, the
homeserver, the analytics and Sentry settings, and the standalone app's
shared room secret in front of a host that has no business setting any
of them — the component reads none of those. It is now the behavioural
configuration a widget's URL can carry plus the two properties a host
has a say in: the theme and the background. The background is now
actually applied to the component's root, which it was not before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
"Embedded" already means something here: the embedded package is the
widget build. Where a comment meant Element Call running as a React
component inside a host application, it now says so, and the params it
starts from are `componentProperties`. "Host" stays the word for the
application on the other side, whether that is a widget container or an
application rendering the component.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Three layers stood between a shell and the call: `ElementCall` (the
component), `ElementCallView` and `GroupCallView`. The middle one only
held the `joined` flag and the mute state, both of which nothing outside
the call reads any more, so it is folded into the view it wrapped. That
view is now `CallView`, since "group call" is a name from before
Element Call handled anything else.
Its docstring says what it is: the whole lifecycle of a call — lobby,
the call itself, and what comes after — with the lobby and the post-call
screen each present or skipped depending on the parameters and the host.
Both the standalone RoomPage and the component render it directly.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The component exposed the internal HostBridge to hosts as-is, which
carried the host's requests as rxjs observables. That made rxjs part of
the public API of a package that bundles its own copy of it, so a host
would build bridges with a different rxjs than the one Element Call
consumed them with — and asked every host to learn rxjs to change the
theme.
A component host now implements plain async callbacks for what Element
Call tells it (`ElementCallHostBridge`, all optional), and makes its own
requests through an imperative handle on the component's `ref`
(`ElementCallHandle`: setTheme, join, hangUp, setDeviceMute), each
resolving once Element Call has acted and rejecting when nothing in
Element Call can. `component/host.ts` adapts that to the HostBridge the
rest of Element Call still speaks, with a bridge whose identity never
changes, so a host re-creating its callbacks on render restarts nothing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Only the call body sat inside an error boundary; anything thrown in the
lobby, the mute state or the providers propagated into the host's React
tree, where React unmounts everything up to the nearest boundary — the
host's, or the host's whole application. The standalone app has a
boundary at its top; the component now has the same, showing the error
page inside its container and asking the host to stop keeping it on
screen.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The move from media queries to `@container element-call` queries left a
few places still asking the viewport: the room header's compact size
and the lobby's footer placement went through `useMediaQuery`, the
lobby's video preview was `50vh` tall, the reaction picker was capped at
`100vw`, and the content insets and the gradient background were sized
from `100vw`/`100vh`. Embedded in a corner of a host's page, each of
those answered for the page rather than the corner.
`useRootSizeMatches` is `useMediaQuery` for the root element, built on
the same `observeElementSize$` the layout uses; the lengths become
container units. Container units resolve against the nearest query
container, and there are others in the tree (the spotlight layouts, the
media tiles), so base.css says when they may be used.
jsdom gives the body no size at all, which would have every such query
read as a tiny window; the test setup now gives it a desktop's, matching
what the media query mock already answered.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Whether to offer the profile settings was inferred from whether the host
could close Element Call. For a component with no host bridge — the
default — nothing could, so an embedded Element Call let the user edit
the profile of an account that belongs to the host application.
`HostBridge.supportsProfileChanges` states it directly: true standalone,
where Element Call signed the user in itself; false for a widget's host
and for anything embedding the component (which sets it itself, since
the client it hands over is its own). The profile tab and the profile
shortcut follow that. What a host's ability to close us still decides —
what to show after the call ends — is a question about who owns our
lifetime, and stays keyed on `close`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The formatter wants the layered import of the Compound design tokens on
one line; "de-globalise styles" left it wrapped.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
"de-globalise styles" gave the tab's <pre> elements a class instead of
styling the bare element, but did not update the snapshot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The call and the lobby each set `document.title`, so a component
embedded in a host renamed the host's tab to "Element Call | <room>".
The title belongs to whoever owns the page: the standalone app's
RoomPage now sets it, for whichever room it has got as far as knowing
about, and the call itself no longer touches it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The call's shortcuts were listened for on the window and allowed
whenever focus was inside the standalone app's `#root` — which, for a
component embedded in a host, is the host's own root, or nothing. So m,
v and space fired while the user typed in the host's composer, and two
Element Calls on one page both answered every key.
Listen on the element Element Call treats as its root instead (the body
standalone, so nothing changes there), and judge whether a key press is
spoken for by what has focus — a dialog or a text field — rather than by
where it sits in the DOM, since the modals are now portalled to that
same root.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The component built its `MediaDevices` once, in a scope nothing ever
ended, so every mount left device observers running for the rest of the
page's life. Building it in an effect ties the scope to the component's
lifetime — and to the options it was built with, which were previously
frozen at first render.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Everything downstream of the component's params — the mute state, the
call view model and with it the media connection — is keyed on the
identity of the params object, which was memoised on the identity of
the `config` prop. A host writing `config={{ ... }}` inline, which is
the natural way to write it, therefore tore the whole call down on
every render. The harness happened to pass a constant, so nothing
noticed.
`useStableValue` hands out the same object for as long as a deep
comparison says nothing changed, so an inline config costs nothing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Splits the config options for the timings of a delayed leave event into two sets: one for when delegation is available (as you can relax the timings and get more stable calls this way), and another for when it's unavailable (as we must continue to gracefully downgrade even after Matrix 2.0 is fully rolled out).
This works by bluntly hitting the delegation endpoints without auth before joining to check for a 404.
Extends the [RemoteTracks] logging from #4235 with the per-publication
encryption flag and the ParticipantEncryptionStatusChanged /
EncryptionError room events.
If a publisher encrypts frames while the subscriber believes the
publication is unencrypted, livekit-client bypasses the cryptor and hands
raw ciphertext to the decoder, which is audible as loud noise bursts.
The reverse mismatch (or a missing/invalid key) drops frames instead.
Neither case is visible in a rageshake today.
`component/package.json` describes the built component as a package
(`@element-hq/element-call-component`: entry, stylesheet subpath, types,
peer dependencies) so that a host can depend on
`github:element-hq/element-call#<ref>&path:/component`. Its `prepare`
script builds on install, since nothing is published yet.
For that the component build now lands in `component/dist` instead of the
repository's `dist`, and `pnpm build:component` also emits the type
declarations (`component/tsconfig.build.json`, `build:component:types`),
which previously had to be produced by hand.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Update ghcr.io/element-hq/element-web:develop Docker digest to a9bd290
* Fix Element Web integration tests
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Robin <robin@robin.town>
Warn when ConnectionManagerData merges participants from a second
connection to the same SFU URL, when more than one LiveKit participant
matches a member's backend identity, and when a scope is ended while one
of its behaviors is still delivering a value (which strands the later
bound subscribers on the old value). The per-member match log now also
names the connection the participant came from.
A single re-entrant emission can only strand a contiguous run of a
behavior's subscribers, so a tile that is both 'speaking' and 'waiting
for media' (a middle subscriber stranded) implies a nested re-entry.
Report the nesting depth and log each new depth once instead of only the
first re-entry, and tag splitBehavior-derived behaviors with their field
name so the warning identifies which behavior tore.
rxjs delivers a nested emission to every subscriber and then resumes
delivering the outer, older value to the remaining subscribers, which
leaves them permanently out of sync. Log the first occurrence per
behavior with a stack trace so the re-entrant path can be identified
from a rageshake.
The widget tests cannot reach what makes a component different, because the
iframe used to guarantee it: that Element Call stays inside the space it was
given, and that two of them can exist in one page. So the development harness
gets driven by Playwright.
Three tests. Two components in one page holding a real call between two devices
of one account. The settings dialog and the reaction picker staying inside the
container the host gave, asserted by bounding box. And the host bridge
reporting in both directions, including a host-initiated mute coming back as a
report of the new state.
The containment test is the one worth having: both of the escapes found by
running the harness by hand — the settings dialog centred on the window, and
the reaction picker at 82vh landing below the container — would have failed it,
and neither was visible to typechecking, linting or the unit tests.
Users and rooms are created through the Synapse admin and client-server APIs
rather than by driving an interface, and the harness now takes its credentials
from its own query string so that a test can say which account and room to use.
A host reading its own URL is proper; it was Element Call doing so that was the
mistake.
Playwright gains a second web server for the harness on port 3001, a Vite dev
server whether or not the app itself is served from Docker, since the harness is
a development page with nothing to build.
A tile shows "Waiting for media" for as long as its MatrixRTC member
cannot be matched to a LiveKit participant. Log each transition of that
match and of the tile's waitingForMedia state so rageshakes can tie a
stuck tile to the LiveKit participant and track events.
Rageshakes contained nothing about the state of remote tracks, so a tile
showing the wrong mute or video state for a member could not be
diagnosed. Log connect/disconnect, publish/unpublish, subscribe/
unsubscribe, subscription failures, remote mute/unmute and stream state
changes on each Connection's logger, and remove the listeners when the
connection scope ends.
Opening settings in an embedded call put the dialog in the middle of the host's
window, spilling outside the container, and with two calls on a page the second
drew over the first's dialog.
The container established a stacking context but not a containing block, which
are two different things and only the first had been done. `position: fixed`
resolves against the viewport unless an ancestor makes itself the containing
block, so the modal scrim and dialog in Overlay.module.css — `fixed`, `inset:
0`, centred, because in the standalone app they are meant to cover the page —
were positioned and sized against the window. Layout and paint containment
makes the container the containing block for those descendants and clips what
we paint to our own box. The second-call-on-top symptom goes with it, since the
dialog now stays inside the first call's box and there is nothing to overlap.
Two sibling components still cannot draw over one another by construction,
neither being able to leave its own stacking context, but that only shows if a
host overlaps them.
Containment clips a box measured in viewport units but cannot resize it, so
anything positioned that way is simply put somewhere outside the container and
disappears. That applied to the reactions overlay, a `100vw` by `100vh` box, and
to the reaction picker, which sits at `82vh` so as to appear near the footer it
belongs to. Both are positioned against whatever Element Call treats as its
root — `[data-overlay-container]` in the app, which is the size of the page, and
the host's container when embedded — so percentages mean the same thing there
and the right thing here. The earpiece overlay is `inset: 0` with no viewport
units, so containment is enough for it.
Three viewport-relative sizes remain, all of which need more than a change of
unit: the lobby's video preview is `50vh` on a flex item with an aspect ratio,
`--content-inset-*` ramps up to a desktop inset from the window width, and the
picker's `max-width` cap is the window's.
The clipping cuts both ways: a menu near the edge of a small container is
trimmed rather than overflowing into the host. That is the trade an embedded
component makes.
The component seeded its parameters with `computeUrlParams()`, which reads
`window.location`. Element Call is not the page any more, so what it found
there was the host's URL: no `widgetId`, therefore not a widget, therefore no
intent, therefore the standalone app's preset.
The visible symptom was the Element Call logo in the footer of a call embedded
in someone else's application, since `showLogo` follows `header ===
HeaderStyle.Standard`. The rest of that preset mattered more. A hosted call
defaulted to `perParticipantE2EE: false` and `confineToRoom: false` — so
unencrypted, and willing to take the user out of the room a host had put them
in — and a host could not correct either without naming every parameter
itself, nor even name `header`, since the enums were not exported.
So the intent presets move out of `computeUrlParams` into
`configurationForIntent`, and the component builds its parameters from an
intent plus the properties a host has no URL to supply. `intent` becomes a
prop, defaulting to joining an existing group call: the lobby first, confined
to the room, encrypted per participant, and no Element Call branding in
someone else's interface. A host that knows which button the user pressed
should say which.
One deliberate difference from the widget presets: the background defaults to
solid rather than the gradient, which is drawn by a `position: fixed`
pseudo-element and would escape the container to cover the host.
`UserIntent.Unknown` still means the standalone preset, so the app and widget
are unchanged — including a widget that names no intent, which has always been
given those defaults.
`pnpm dev:component` serves a page that stands in for a host application: it
signs in twice against the development backend and shows two calls side by
side, in resizable boxes, with furniture of its own around them. Two devices of
one account, so a real call happens between the two components and anything
Element Call keeps once per process rather than once per call shows itself.
The host bridge is driven by hand and reports both directions in a log along
the bottom, which is the first exercise the theme, hang-up and device-mute
requests have had outside widget mode. Each pane can be unmounted and remounted
to see what Element Call leaves behind, and there is a `position: fixed` dialog
belonging to the host to see whether it covers the calls. The page uses none of
Element Call's design tokens, so anything that looks styled outside a pane came
from Element Call reaching out of its container. It reaches Element Call only
through the component's public interface, which is how the exports missing from
that interface came to light.
Three things about the component build the harness turned up on the way, all
too small to be worth their own commits:
- It copied `public/` into `dist/`, including the developer's own gitignored
config.json, into output we would publish. `publicDir: false`, as the
embedded build already does. The sdk build has the same leak; untouched.
- `pnpm lint:externals` now exists, which the build config already claimed it
did. It reads the external list out of that config and fails if the source
imports React, the Matrix SDK or LiveKit by a path the list does not name.
Since the bundler silently ignores the pattern form of that option, an
unnamed subpath is bundled with no warning at all — which is how a host would
end up with a second React.
- `lint:oxlint` ran over `src playwright`, so nothing in `component/` had ever
been linted.
Serving a page also meant the shared plugin list could no longer inject the
app's HTML entry point unconditionally, so that is now optional — and off for
the library build too, which never had an HTML page to inject it into.
The component built and typechecked in the previous commit, but only because
nothing had rendered it. Everything Element Call needs that `src/main.tsx`
side-loads was missing from it.
Its stylesheet: only main.tsx imported index.css, so the library build emitted
CSS-module styles with every `--cpd-*` and `--font-size-*` unresolved. Split
into base.css, which both the app and the component import, and the rules that
are about owning a page, which only the app does. The split is a straight move
— comment-stripped and sorted, the old file and the two new ones differ by
exactly one line — and that line is the deliberate part: `.no-scroll-body`
becomes `body.no-scroll-body`. Element Call adds that class to whatever it
treats as its root, and since the root can now be a container, `position:
fixed` would have taken that container out of the host's layout. Pinning the
page is what it always meant.
Its translations: `initializeElementCall` called `i18n.init` with neither
resources nor a backend, so every key would have rendered as itself. English is
bundled in. The app fetches locale files from URLs its own build emits, which a
host serving the library from somewhere else could not resolve, so how a host
picks a language is left open.
And the types a host needs: `HostBridge` alone is not enough to implement
`HostBridge` — `HostRequest`, `DeviceMuteState`, `DeviceMuteRequest` and
`JoinCallData` all appear in its signatures, and `ConfigOptions` in
`initializeElementCall`'s.
toggleScreenSharing only had `.catch(logger.error)`, so a getDisplayMedia
request that hangs (element-call-rageshakes#17152: Element Desktop on
Windows, the user pressed the screen share button 14 times in 25 seconds
and the log shows nothing but the toggle lines and livekit-client's
"waiting for pending publication promise timed out") left the user with
a button that does nothing and us with no evidence of why.
Log when a toggle is requested and when it completes or fails, with the
elapsed time, so a hang is visible in the logs. Explicit failures other
than the user cancelling the picker show as a non-modal "Could not start
screen sharing" toast. Nothing is inferred from a toggle taking a long
time: the user may simply be choosing what to share.
If the camera track has already ended by the time the blur processor is
applied to it, MediaStreamTrackProcessor cannot be constructed and
setProcessor rejects. Both call sites used `void`, so this surfaced as
Unhandled promise rejection: TypeError: Failed to construct
'MediaStreamTrackProcessor': Input track cannot be ended
(element-call-rageshakes#17225). Skip ended tracks, and catch and log
processor attach/detach failures instead of leaking them.
When the ConnectionManager stops a Connection while livekitRoom.connect()
is still pending (which happens on every join, because the local
membership emits twice in quick succession and the connection set is
recomputed), livekit-client rejects the pending connect with "Client
initiated disconnect". Connection.start() then treated that as a failure:
it pushed an UnknownCallError into the state of a connection that was
already stopped and rethrew, and since start() is fire-and-forget the
throw surfaced as
Unhandled promise rejection: Error: Failed to connect to Livekit server
in every rageshake, right after a "livekitRoom.connect FAILED ... Client
initiated disconnect" line. Nothing was actually wrong: the replacement
connection connects fine a moment later.
Mark the connection as stopped before disconnecting, and have start()
return quietly when the rejection is the abort we asked for.
Adds component/index.tsx as a fourth build target: <ElementCall client
roomId /> and an initializeElementCall to await once beforehand. It gives
Element Call everything it would otherwise take from the page it is on —
the parameters, the host bridge, media devices, translations, a container
to confine itself to — and hands it the host's client rather than finding
one of its own.
React, the Matrix SDK and LiveKit stay external, since the host has them
and a second copy of any would not merely be wasteful: React would hold
two sets of hooks and the client would run two sync loops. Every subpath
has to be listed by name, because the pattern and callback forms of
rollupOptions.external are silently ignored here — a lesson worth the
comment that records it.
Element Call's own navigation runs in a MemoryRouter, so being embedded
cannot disturb the host's URL. ClientContext and GroupCallView both
navigate, so some router has to be present.
The bundle is not yet a reasonable size: library mode base64-inlines
assets referenced through import.meta.url, so MediaPipe's vision runtime
lands in it whole. Left for its own change, since the fix — loading the
background blur transformer lazily — is worth doing for the standalone app
too.
ElementCallView took muteStates as a prop, which would have meant an
embedder building one before it could show a call. It could not simply make
its own: RoomPage created one on mount whichever branch it went on to
render, and two would both report the user's mute state to the host,
talking over each other.
Move the construction into a useMuteStates hook, and give the lobby shown
while waiting to be let into a room its own component. Each lobby now holds
mute state only while it is on screen, so there is never a second one, and
the component can own the call's.
KnockLobbyView also takes the room summary and label handling that RoomPage
was assembling on its behalf, leaving the page with arriving at a call
rather than being in one.
Six selectors named `body` directly — the gradient backdrop and the
platform font overrides in index.css, and the iOS adjustments in
AppBar.module.css and Modal.module.css — so they only applied when Element
Call owned the page. A host mounting it into a container would have got an
interface decorated correctly and styled incorrectly, with nothing to show
that anything was wrong.
Mark the root element with data-element-call-root and match on that
instead. Scoping this way keeps the selectors more specific than they were,
rather than less: widening them to a bare [data-platform=…] would have
dropped specificity from (0,1,1) to (0,1,0) and changed which rules win.
The platform attribute moves with them, from the initializer's write onto
document.body to a layout effect on the root, alongside the theme — so it
still lands before anything is painted.
No visual change while Element Call owns the page: the root is the body,
which now carries the attribute, so every rewritten selector matches the
element it always did.
Being in a call and deciding to show one are different jobs, but RoomPage
did both: routing, authentication, resolving room aliases and knocking, and
then the call itself. Only the first set belongs to whatever is hosting
Element Call.
Add ElementCallView as the seam between them. It takes a client and a call
to join, and owns whether the user has joined — which is the call's own
business rather than its host's. RoomPage keeps everything about arriving
at a call and renders this for the call itself.
Nothing else moves yet. muteStates is still passed in, because the
standalone shell shares one with the lobby it shows while waiting to be let
into a room, and two instances would both report the user's mute state to
the host.
RootElementProvider was exported but never used: nothing supplies a root
element, so every consumer falls back to the document body. Knip reports
it, which is what is failing CI.
M1 adds a provider back along with the component that mounts Element Call
into a container. Until then there is nothing to provide.
The throttled flush callback returned this.flush instead of calling it
(regressed in #2607), so logs were only persisted to IndexedDB on
rageshake submission or beforeunload. When the host removes the widget
iframe at hangup, the whole call's logs were lost, so a rageshake filed
from a later call carries nothing from the affected one.
RootElementContext's documentation promised that Element Call confines its
decoration to the given element, but several selectors still name body
directly and the initializer writes data-platform onto it. A non-body root
is decorated correctly and styled incorrectly, with no error to show for
it, so the documentation should say so until the stylesheets are scoped.
close() sent io.element.close and then stopped the transport, so a
rejected send skipped the stop. Both call sites this replaced stopped it
unconditionally — ErrorView in a finally, GroupCallView outside its
try/catch — because a host that never acknowledges the request would
otherwise leave the messaging live and the close button doing nothing.
Restore that with a finally.
The SDK stopped sending join notifications: it threaded callIntent into
createCallViewModel$ but not its pair sendNotificationType, which
enterRTCSession used to read for itself, so an explicit
?sendNotificationType=ring — or an intent that implies one — no longer
reached joinRTCSession.
The mechanism is worth fixing rather than the instance. The defaults on
CallViewModelOptions describe a standalone Element Call, so a widget
caller that misses a field gets standalone behaviour rather than an error,
and the SDK is only ever a widget. Give both callers one shared mapping so
they cannot drift, and cover the whole chain from URL to options in tests.
autoLeaveWhenOthersLeft and waitForCallPickup stay out of it: the view
model never read those from the parameters, so enabling them for the SDK
would be a change in its behaviour rather than a fix.
Element Call reached the widget API through a mutable module-level
binding, which every consumer imported directly. Nothing outside the app
shell needs it any more, so hand it back from initializeWidget and thread
it through: the initializer returns it, main passes it to App, and App
uses it to build the host bridge and to await the client the host is
lending us.
ClientContext's loadClient is now only about restoring or creating a
session of Element Call's own, since a widget's client arrives as a prop
like any other host's would.
Also fixes an early return added in the previous commit, which skipped
starting the analytics settings listener when a client was supplied. That
was harmless until now, but would have stopped analytics following the
user's choices in widget mode.
sdk/main.ts asked the host to close by hand; it now uses the bridge, which
also stops the transport as the app does.
ClientProvider found its own client: from the widget API, or by restoring
or creating a session. A host that embeds Element Call already has one,
and owns the user's session, so accept it as a prop and skip all of that.
A supplied client seeds the state synchronously, since there is no session
of ours to restore and so nothing to wait for.
Also guard the broadcast that shuts down other instances of the app. It
protects Element Call's own session and crypto stores, which is why it was
already skipped in widget mode — a host's client has the same property, so
without this an embedded Element Call would close down the user's other
tabs on mount.
Adds the first tests for ClientContext, covering both
PosthogAnalytics read its own configuration out of the environment on
first use: the URL parameters, config.json, and the widget global. An
embedded Element Call has none of those to offer, and analytics that
configure themselves cannot be switched off by a host that does its own
reporting.
Take an AnalyticsConfig through PosthogAnalytics.configure() instead,
called from the initializer once the config has loaded. Unconfigured
analytics stay off.
Note the two halves of that config are decided differently, and have to
be: where the credentials come from depends on the package, but who owns
the user's analytics identity depends on how Element Call is running,
since the full package can be used as a widget too.
Drop the widget check around cryptoVersion, which never did anything —
widget mode never initialises crypto, so getCrypto() is already undefined
there.
Move the tests covering which package reads which credential source onto
analyticsConfigFromEnvironment, where that decision now lives.
The remaining reads of the widget global were all asking one of two
different questions, so they get two different answers.
The app shell — the auth hooks, automatic guest registration, the group
call loader's diagnostic and the initial mute state — wants to know
whether Element Call was launched as a widget. That is a property of the
URL it was launched with, so expose the isWidget that computeUrlParams
already computed internally, documented as being for shell use only.
The call interface — whether to offer the profile settings tab — wants to
know something about its host, so it asks the bridge. A host that can
dismiss Element Call owns the user's account, so their profile is not ours
to edit; this reuses the close capability as a proxy, with a TODO
alongside the others.
ClientContext also takes supportsReactions from the bridge rather than
checking four widget capabilities itself, which removes widgetApi from
InitResult — a field that was always null outside widget mode.
Note this changes behaviour for a malformed widget URL: one carrying a
widget ID and parent URL but missing the room, user, device or base URL
would previously have fallen back to registering a guest user, and will
now not.
MuteStates, CallViewModel and LocalMember reached the host through the
widget global. None of them are React components, so they take the bridge
as an explicit parameter: a constructor argument for MuteStates, a field
on CallViewModelOptions, and one on createLocalMembership$'s props.
src/state no longer refers to the widget API.
The conditionals around it mostly disappear: nullHostBridge's observables
are NEVER, so there is nothing to guard, and a request carries its own
reply rather than needing the transport and the original event.
CallViewModelWidget.test.ts drove hangup by emitting on the mocked
widget's action emitter, so it now injects a bridge instead, and checks
that the request is acknowledged.
ErrorView needed a widget only to decide whether to offer a close button
or a link home. That prop was threaded down through ErrorPage, RichError
and GroupCallErrorBoundary from seven call sites, to answer the single
question of whether the host can dismiss Element Call — which the host
bridge now answers directly through the presence of close().
Read the bridge from context in ErrorView and remove the prop, along with
the plumbing that carried it.
No functional change: the rendered output is unchanged, as the existing
snapshot confirms.
Move GroupCallView onto the host bridge
GroupCallView asked the widget API to keep it on screen, to close it, and
to tell it when a preloaded call should join. Route all three through the
host bridge and drop the widget prop.
useRoomEncryptionSystem is used from GroupCallView, inside the part of
Element Call that will become the embeddable component, but it reached for
getUrlParams() via getKeyForRoom(). Extract the lookup into a helper taking
the room ID and password explicitly: the hook supplies them from the
parameters context, while getKeyForRoom keeps reading the URL for its one
remaining caller in the app shell.
No functional change.
The view models reached for getUrlParams() — and so window.location —
from deep inside the call path: CallViewModel, MediaDevices, Publisher,
LocalMember and the footer view model. An embedded Element Call has no URL
of its own, so these values have to arrive as arguments instead.
Add the relevant options to CallViewModelOptions, to the MediaDevices and
Publisher constructors, to createLocalMembership$ and enterRTCSession, and
to createCallFooterViewModel. The remaining React consumers read the
context added in the previous commit. AppViewModel now takes its audio
output options too, moving that URL read out to main.tsx, where the app
shell can act as the adapter.
The new CallViewModelOptions fields are optional, defaulting to what the
URL parameters resolve to outside widget mode; the MediaDevices and
Publisher arguments are required, so that every construction site has to
be explicit.
useTheme.test.ts mocked the UrlParams module with a factory, so it needed
updating to mock the hook rather than getUrlParams.
No functional change.
Element Call's parameters come from its URL, which works while it owns
the page but leaves an embedder with nowhere to put them. Add a context
so they can be provided directly, falling back to parsing
window.location when no provider is present.
This also decouples the thirteen consumers from react-router: useUrlParams
called useLocation, so each of them required a router ancestor, which the
embedded build will not have. The standalone and widget builds keep their
URL-derived behaviour via useUrlParamsFromLocation, provided in App.
No functional change.
Element Call writes its theme classes, background and layout attributes
straight onto document.body, and portals its modals there too. That is
only correct while it owns the page; embedded in a host application it
has to confine itself to the container it was mounted into.
Add a RootElementContext, defaulting to document.body so that the
standalone and widget builds are unaffected, and point the theme classes,
data-background, no-scroll-body and the fullscreen target at it.
Give the Modal and Toast portals an explicit container as well. Radix and
vaul both default to document.body, so without this every modal, drawer
and toast would render outside the container and lose the theme and
platform attributes set on it.
No functional change: the root element is document.body until an embedder
provides otherwise.
Config.init() derives the location of config.json from window.location,
which only makes sense while Element Call owns the page. When it is
embedded in a host application the host owns the configuration, so add
Config.initWith() to accept it directly.
Share the defaulting and validation between both paths via resolveConfig(),
so that an injected config behaves identically to a fetched one, and mark
initialization as complete so that the init() calls already on the startup
path resolve immediately instead of fetching over the top of it.
initDefault() becomes initWith({}), which also stops it handing out a
shallow copy of DEFAULT_CONFIG whose nested objects were shared with the
module-level default.
Element Call configured the global i18next singleton. When Element Call
runs embedded in a host application rather than as its own page, that
singleton belongs to the host, so configuring it would clobber the
host's translations.
Create Element Call's own instance in utils/i18n.ts, configure it in the
initializer, and pass it to components via <I18nextProvider>. Drop
.use(initReactI18next) from the initializer: it registers the instance
as react-i18next's global default, which is the global we are trying to
avoid. Tests and stories keep using it, so that they do not need to wrap
every render in a provider.
Two modules imported `t` directly from "i18next" and so were bound to
the global instance: utils/errors.ts now calls i18n.t() on the instance
(reached at call time, since i18next only assigns `t` during init), and
QrCode uses useTranslation() like every other component.
No functional change.
This is the mode in which we sent membership events with the 'oldest membership' transport selection algorithm, which stopped being the default back in version 0.21.0. Users will no longer be able to select this mode in developer settings, and admins will no longer be able to select legacy mode through the config either. The app will still continue to support *receiving* membership events with the 'oldest membership' transport selection algorithm from others, however.
Support for events with an array of memberships from different devices was removed over a year ago in matrix-js-sdk ffd3c9575e9def576739baf6b1dc329b0db55c0c.
It turns out that for a blurry gradient background, low-res raster images look very convincing when scaled up and interpolated. That means that we can rasterize our gradient SVGs for improved paint performance when resizing the window, while still keeping them quite lightweight (just a couple of kilobytes each).
When someone starts or stops speaking, the layout will often be recomputed only to find out that there is ultimately no layout change. We can ignore these redundant updates to avoid re-rendering the InCallView and Grid components, which are relatively slow.
For that specific, common case, this reduces JS CPU usage by as much as 70% in my testing.
It's very uncommon to have this debug option enabled, and yet it currently causes the footer to re-render on every layout update, which is a small but avoidable cost.
I don't believe this affects the cost of running the useSprings hook, but it at least skips the animation frames that would otherwise follow, so it's worth a try.
It turns out that the timers which repeatedly poll the RTP stats of each video track all run independently of each other and can add up to a small but constant sink of CPU. Meanwhile we can replace these timers with HTMLVideoElement 'resize' event listeners, which is way more efficient and reacts instantly to orientation changes.
A change in a tile's speaking indicator could cause its entire tree of context menu components to re-render, which is expensive. Isolating the behavior subscriptions in their own component avoids this.
With the React Compiler, our component code is transformed at build time to automatically apply various forms of memoization. This changes the runtime semantics of our code a little bit and therefore could surface new bugs in the next release cycle in case any components fail to follow the rules of React. See https://react.dev/learn/react-compiler for more information.
This results in a small bundle size increase and modest performance gains, but I think it's worth it given the potential for larger performance gains once we tune the component structure a bit more.
* Update ghcr.io/element-hq/element-web:develop Docker digest to 0183c0a
* Update end-to-end tests to work with auto-collapsing room list
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Robin <robin@robin.town>
On the combined Storybook docs page for the call footer, the layout switches would not show the correct state until you hovered over them because they all shared the same input name and thus were interpreted as belonging to the same radio group.
It reduces confusion when reading snapshots if class names are still scoped to their respective CSS modules. It also discourages the use of class names in tests, which is a good thing. (https://testing-library.com/docs/guiding-principles)
Thanks to Johannes for the suggestion.
The rule should only care about enclosing function/class scopes. For example if an ObservableScope is received as a parameter to a function and then simply used inside an 'if' block (technically a different scope), that's not a problem.
Previously we were hiding the entire app bar on mobile phones in landscape orientation. However now that the app bar supports a small 'subtitle' element, we should show only the subtitle in this case to match the designs.
The subtitle still hides on tap, just like the footer.
On mobile, the ringing status indicator is supposed to display in the header rather than on a tile. The exact layout differs between Android and iOS. To get it right I had to refactor AppBar to use CSS grid templates.
(Also, I changed my mind about the exact ringing data I needed out of CallViewModel - sorry. A little move of the ringtone audio renderer into its own component was necessary to accommodate that.)
It was rather confusing that matrixLivekitMembers$ gives you objects of type RemoteMatrixLivekitMembers and yet the *local* member would often be among these. I've attempted to clear this up. To my knowledge this wasn't creating any bugs.
The rule of thumb to avoid resource leaks is that you should never call ObservableScope methods in a callback unless the ObservableScope is directly passed to or created inside that callback. I had a go at codifying this as a lint rule.
I noticed that calls to createDisplayNameBehavior$ and createAvatarUrlBehavior$ were technically leaking resources since they reused the ObservableScope from their outer scope, which in practice lasts for the entire lifetime of the CallViewModel. This would not have had any noticeable effect unless you had other participants leave and rejoin the same call many thousands of times.
I found our code's internal model of ringing a little overgrown (it had superfluous states like 'unknown') and difficult to extend with metadata or callbacks relating to ring attempts. By modeling ringing instead as a stream of ring attempts, where each attempt has an intent, a recipient, and an eventual outcome (accept/decline/timeout), I find it more natural to work with.
This makes room for a future 'try again' callback to allow ringing someone again after a timeout, and also forced me to look for a simpler solution to the duplicate leave sound effects. I exposed the intent of the ringing attempt to the call UI so I can later use it in the header.
The designs actually never want us to show the settings button in the footer if an app bar is in use (as in Element X mobile apps), so just avoid showing it at all in that case.
Because the type of header that we use in Element X (an 'app bar') lives in a different place in the document than the other headers, it needs a special branch to propagate the right insets.
* Posthog: drop $initial_person_info from outgoing events
* Posthog: migrate from sanitize_properties to before_send
* strip URL fields from $set / $set_once
* enable mask_personal_data_properties
* review
* update tests to check for `delete` (not anymore `=null`)
rename: `applyPrivacyFilters`->`santizeSensitiveData`
---------
Co-authored-by: Timo K <toger5@hotmail.de>
* Add Posthog events for Call reconnect including the reason
* Expose single trackCallReconnecting() entry point on PosthogAnalytics
* Track reconnect duration and align with existing analytics pattern
* Refactor combined$ to return [connected, reason] tuple
* Update firefoxUserPrefs to allow getUserMedia and enumerateDevices on CI
---------
Co-authored-by: Valere <bill.carson@valrsoft.com>
Co-authored-by: Robin <robin@robin.town>
Co-authored-by: Timo K <toger5@hotmail.de>
* add MeidaMuteAndSwitchButton
* User button in footer
* Add tests
* update styling (dark bg on menu open + chevron white + chevron up)
* fix tests
* add storybook to CI
only add storybook with storybook label
test names
another env name test
TestName
new default name
remove label condition
Update pr-deploy.yaml
* Update pr-deploy.yaml
* add toggle example to default component
* hook up footer select actions
* fix video audio button (swapped) and lable in story
* make `delayed_leave_event_delay_ms` and `network_error_retry_ms` mandatory
* Support delegation for legacy jwt request
* Calculate `maximumNetworkErrorRetryCount` based on timeouts
* add storybook to CI
only add storybook with storybook label
test names
another env name test
TestName
new default name
remove label condition
Update pr-deploy.yaml
* rename things to check if we actually run the latests ci
* cleanup things used for testing
* Update deploy-to-netlify.yaml
* use package instead of custom environment_name
* final cleanup
By adding viewport-fit=cover to the <meta name="viewport"> header, the page now requests to be displayed edge-to-edge across the entire screen. This gives us control over what we display around camera cut-outs and system navigation UI, if the user agent supports it. I then adjusted the styles of various UI elements to ensure that they still lie within the screen's safe area.
* Update compound-web to 9.3.0 and update Buttons from "sm" to "md".
* Clean up the liast bits
* Update snaps too
* fix test, need to dismiss a new dialog
---------
Co-authored-by: Valere <bill.carson@valrsoft.com>
Run `yarn i18n` to extract new translation keys for the media quality
settings, and update the DeveloperSettingsTab snapshot to include the
Camera quality, Screen sharing, and Audio processing sections.
Allows Element Call to be served from a subdirectory (e.g. /widgets/element-call/
in Element Web) without breaking dynamic imports for locales, workers, and other
assets that were previously using absolute paths.
Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
- Add camera video quality controls (resolution/framerate/bitrate/codec)
to Settings > Video, mirroring the screen share settings UI
- Add audio processing toggles (echo cancellation, noise suppression,
auto gain control) to Settings > Audio, replacing URL-param-only controls
- Display raw values inline on all sliders (framerate, bitrate, volume)
- Add config-seeded defaults: config.json media_quality values now seed
Setting defaults for users who haven't explicitly set preferences
- Camera settings are applied when joining a call via ConnectionFactory
Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
InputField only supports input/textarea, not select. Passing option
children caused React to crash rendering children inside a void input.
Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
Adds a "Screen sharing" section to Settings > Video with controls for:
- Resolution (576p to 4K)
- Framerate (5-60 fps slider)
- Bitrate (0.5-15 Mbps slider)
- Codec (VP8/VP9/H.264/AV1)
Gated behind an "Advanced screen share settings" toggle. When enabled,
settings are passed to LiveKit's setScreenShareEnabled as both capture
constraints and publish options. When disabled, falls back to
config.json media_quality defaults.
Settings are persisted in localStorage via the existing Setting<T>
system. The Slider component is extended with a tooltipFormatter prop
for custom tooltip display.
Inspired by pirosuki's advanced-screen-share-settings branch, but
reimplemented cleanly: settings are read directly in LocalMember.ts
(no signature changes), the existing Slider is extended (no component
duplication), and proper form components are used throughout.
Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
Add a `media_quality` section to config.json that allows self-hosters
to configure video codec, resolution, bitrate, framerate, and simulcast
layers for both camera and screen sharing.
This addresses the long-standing request in #249 for configurable media
quality settings. The LiveKit SDK already supports all of these options;
this change exposes them through the existing config system.
New config.json fields:
- media_quality.video_codec: preferred codec (vp8/vp9/h264/av1)
- media_quality.video: camera resolution, bitrate, framerate, simulcast layers
- media_quality.screen_share: screen share resolution, bitrate, framerate,
simulcast layers (enables 3+ layer simulcast for screen sharing)
All fields are optional and fall back to the existing defaults (VP8,
720p camera, 1080p screen share) when not specified.
Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
They were not properly being contained to where the MediaView is supposed to appear, causing them to all stack up on the first screen share in the spotlight tile.
6667fc54c0 changed this CSS selector that shows buttons on hover to only target the buttons in the bottom right corner of the spotlight tile, causing the forward/back buttons to stay invisible.
This approach is more flexible in that it allows even the local participant to share their screen in CallViewModel tests, and more rigorous in that it ensures that application code is reacting specifically to track publications.
If you manage to move your floating video tile to the bottom of the screen in a small landscape window, the footer obscures the tile when shown. The designs want us to smoothly move the floating tile out of the way in this case.
This adds two new intents: start_call_voice and join_existing_voice. I need the latter in order to implement Element Web's new incoming call toasts, in which you can turn off your video before joining a group call. The other one, start_call_voice, exists more for completeness than anything; we don't currently want to allow users to start voice calls in group chats in our messenger clients, but maybe Cinny would, for instance.
- use the new domain logic to discover the transport
- then try to authenticate
- Also fix the bug in multi sfu where active$ not updated on delayId change
If you are the only participant in the call, the expanded spotlight layout would redundantly show your media in both the spotlight and PiP tiles. This is a regression; in versions 0.16.1 and earlier we would avoid showing the same user twice.
Taking Valere's suggestion of giving them the 'switch' role. Also, the aria-label attributes were redundant (having tooltips already gives the buttons aria-labelledby).
Add comments on existing code
Extracted a specific android controller for isolation and better testing
lint fixes
Fix device update logic and more tests
better typescript
In landscape orientation the button would be buried underneath the footer, which would block interaction with it. This commit changes the footer to not show in cases where a button has been pressed.
Timo and I agreed previously that we should ditch the class pattern for view models and instead have them be interfaces which are simply created by functions. They're more straightforward to write, mock, and instantiate this way.
The code for media view models and media items is pretty much the last remaining instance of the class pattern. Since I was about to introduce a new media view model for ringing, I wanted to get this refactor out of the way first rather than add to the technical debt.
This refactor also makes things a little easier for https://github.com/element-hq/element-call/pull/3747 by extracting volume controls into their own module.
To correctly implement the legacy "oldest membership" mode, we need the code to be more nuanced about the local transport. Specifically, it needs to allow for the transport we advertise in our membership to be different from the transport that we connect to and publish media on. Otherwise, if these two are yoked together, members will resend their memberships whenever an SFU hop occurs, which an attacker could use to cause an amplified wave of state changes.
It's always worth having logs for when state holders are created or destroyed (these are often the most interesting things happening in the application), so I thought it would be nice to have generateItems always log for you when it's doing that.
once)
The local jwt token needs to be aquired via the right endpoint. The
endpoint defines how our rtcBackendIdentity is computed. Based on us
using sticky events or state events we also need to use the right
endpoint. This cannot be done generically in the connection manager. The
jwt token now is computed in the localTransport and the resolved sfu
config is passed to the connection manager.
Add JWT endpoint version and SFU config support Pin matrix-js-sdk to a
specific commit and update dev auth image tag. Propagate SFU config and
JWT endpoint choice through local transport, ConnectionManager and
Connection; add JwtEndpointVersion enum and LocalTransportWithSFUConfig
type. Add NO_MATRIX_2 auth error and locale string, thread
rtcBackendIdentity through UI props, and include related test, CSS and
minor imports updates
On second glance, the way that we determined a media tile to be 'waiting for media' was too implicit for my taste. It would appear on a surface reading to depend on whether a participant was currently publishing any video. But in reality, the 'video' object was always defined as long as a LiveKit participant existed, so in reality it depended on just the participant. We should show this relationship more explicitly by moving the computation into the view model, where it can depend on the participant directly.
We would show 'waiting for media' on participants that were connected but had no published tracks, because we were filtering them out of the remote participants list on connections. I believe this was done in an attempt to limit our view to only the participants that have a matching MatrixRTC membership. But that's fully redundant to the "Matrix-LiveKit members" module, which actually has the right information to do this (the MatrixRTC memberships).
This fixes a regression on the development branch: the layout switcher would not respond to input while the window mode is 'flat' (i.e. while a mobile phone is in landscape orientation). See https://github.com/element-hq/element-call/pull/3605#discussion_r2586226422 for more context.
I was having a little trouble interpreting the emergent behavior of the layout switching code, so I refactored it in the process into a form that I think is a more direct description of the behavior we want (while not making it as terse as my original implementation).
Semantically, behaviors are only meaningful for as long as their scope is running. Setting a behavior's value to an empty array once its scope ends is not guaranteed to work (as it depends on execution order of how the scope is ended), and subscribers should be robust enough to handle clean-up of all connections at the end of the scope either way.
Since we now bundle a trusted Element Call widget with our messenger applications and this widget reports analytics to an endpoint determined by the messenger app, there is no longer any reason to compute a different analytics ID from the one used by the messenger app.
While looking into what had regressed https://github.com/element-hq/element-call/issues/3588, I found that 28047217b8 had filled in a couple of behaviors with non-reactive default values, the "natural window mode" behavior being among them. This meant that the app would no longer determine the correct window mode upon joining a call, instead always guessing "normal" as the value. This change restores its reactivity.
* playwright: Fix error boundary mgmt or openId errors
* do not use tap for important logic
* fix lint
---------
Co-authored-by: Timo K <toger5@hotmail.de>
* Update docs for 'rageshake.submit_url' to use the dedicated subdomain
* Update Netlify preview deployment to use the dedicated subdomain for 'rageshake.submit_url'
Rename several classes/behaviors to factory-style creators and adapt
call wiring and tests accordingly:
- Replace ConnectionManager class with createConnectionManager$ which
returns transports$, connectionManagerData$, connections$
- Convert MatrixLivekitMerger to createMatrixLivekitMembers$
(matrixLivekitMerger$)
- Rename sessionBehaviors$, localMembership$, localTransport$ to
createSessionMembershipsAndTransports$, createLocalMembership$,
createLocalTransport$
- Adjust participant types and hook up connectOptions$; expose join via
localMembership.requestConnect
- Update tests to use the new factory APIs
- Replace MatrixLivekitItem with MatrixLivekitMember, add displayName$
and participantId, and use explicit LiveKit participant types
- Make sessionBehaviors$ accept a props object and return a typed
RxRtcSession
- Update CallViewModel to use the new session behaviors, rebuild media
items from matrixLivekitMembers, handle missing connections and use
participantId-based keys
- Change localMembership/localTransport to accept Behavior-based
options, read options.value for enterRTCSession, and fix advertised
transport selection order
- Update tests and minor UI adjustments (settings modal livekitRooms
stubbed) and fix JSON formatting in locales
Remove preferStickyEvents and multiSfu in favor of a MatrixRTCMode
enum/setting (Legacy, Compatibil, Matrix_2_0). Move session join/leave,
track pause/resume, and config error handling out of CallViewModel into
the localMembership module. Update developer settings UI, i18n strings,
and related RTC session helpers and wiring accordingly.
npm has recently limited the lifetime of all access tokens to 90 days (https://gh.io/npm-token-changes), so it would be a bit inconvenient to stick to our current access token-based method of publishing releases. Meanwhile npm has implemented a more secure publishing method based on OIDC in which you tell the registry that a particular GitHub Actions workflow should be a "trusted publisher" for a given package, and then the CLI will authenticate automatically. (https://docs.npmjs.com/trusted-publishers)
I've already set trusted publishing up on the registry side, and since we're already granting the job permission to generate ID tokens for provenance, there should be no additional lines of config needed to make it work. Let's take away the access token and see how this goes next time we release.
While we still ought to eventually port these tests in some way, the presence of these empty test files is causing a Vitest failure, so it's easiest to just let them go and refer to Git history when we do want to reference them next.
* add sticky event support
- use new js-sdk
- use custom synapse
- don't filter rooms by existing call state events
Signed-off-by: Timo K <toger5@hotmail.de>
* enable sticky events in the joinSessionConfig
Signed-off-by: Timo K <toger5@hotmail.de>
* Remove unused useNewMembershipmanager setting
* Add prefer sticky setting]
* Fixup call detection logic to allow sticky events
* lint
* update docker image
* More tidy
* update checksum
* bump js-sdk fix sticky events type
Signed-off-by: Timo K <toger5@hotmail.de>
* fix demo
Signed-off-by: Timo K <toger5@hotmail.de>
* always use multi sfu if we are using sticky events.
Signed-off-by: Timo K <toger5@hotmail.de>
* review
Signed-off-by: Timo K <toger5@hotmail.de>
* lint
Signed-off-by: Timo K <toger5@hotmail.de>
* Always consider multi-SFU mode enabled when using sticky events
CallViewModel would pass the wrong transport to enterRtcSession when the user enabled sticky events but didn't manually enable multi-SFU mode as well. This likely would've added some confusion to our attempts to test these modes.
* Fix test type errors
* add todo comment
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Half-Shot <will@half-shot.uk>
Co-authored-by: Robin <robin@robin.town>
The execution of certain Observables related to a local or remote connection would continue even after we stopped caring about said connection because we were failing to give these state holders a proper ObservableScope of their own, separate from the CallViewModel's longer-lived scope. With this commit they now have scopes managed by generateKeyed$.
Previously we had a ViewModel class which was responsible for little more than creating an ObservableScope. However, since this ObservableScope would be created implicitly upon view model construction, it became a tad bit harder for callers to remember to eventually end the scope (as you wouldn't just have to remember to end ObservableScopes, but also to destroy ViewModels). Requiring the scope to be specified explicitly by the caller also makes it possible for the caller to reuse the scope for other purposes, reducing the number of scopes mentally in flight that need tending to, and for all state holders (not just view models) to be handled uniformly by helper functions such as generateKeyed$.
Note that this effectively *reverts* 3ac2aa8526 because this branch now has what is a better UX (at least I think so): the in-call view is presented instantly when pressing the join button. Errors that occur during initial connection procedures will be surfaced more uniformly.
useCallViewKeyboardShortcuts() changed a param from `setMicrophoneMuted` to `setAudioEnabled`, the boolean arg of the callback is inverse tht it used to be
* Add media hints for notification events.
* Prevent showing calling view when disconnected from Livekit. (#3491)
* Refactor disconnection handling
* Use "unknown"
* Update signature
* Add tests
* Expose livekitConnectionState directly
* fix whoopsie
* Update dependency livekit-client to v2.15.7 (#3496)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Fix the interactivity of buttons while reconnecting or in earpiece mode (#3486)
* Fix the interactivity of buttons while reconnecting or in earpiece mode
When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles.
* Write a Playwright test for reconnecting
* fix lints
Signed-off-by: Timo K <toger5@hotmail.de>
* fix test
Signed-off-by: Timo K <toger5@hotmail.de>
* enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU
* remove rate limit for delayed events
* more time to connect to livekit SFU
* Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button
* adapt to most recent Element Web version
* Use the "End call" button as proofe for a started call
* Currrenty disabled due to recent Element Web
- not indicating the number of participants
- bypassing Lobby
* linting
* disable 'can only interact with header and footer while reconnecting' for firefox
---------
Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
Co-authored-by: Timo K <toger5@hotmail.de>
Co-authored-by: fkwp <github-fkwp@w4ve.de>
* Log when a track is unpublished or runs into an error (#3495)
* default mute states (unmuted!) in widget mode (embedded + intent) (#3494)
* default mute states (unmuted!) in widget mode (embedded + intent)
Signed-off-by: Timo K <toger5@hotmail.de>
* review
Signed-off-by: Timo K <toger5@hotmail.de>
* introduce a cache for the url params.
Signed-off-by: Timo K <toger5@hotmail.de>
* Add an option to skip the cache.
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
* Apply new hint code
* missed a bit
* fix intent
* Automatically update intent on mute change
* update packages
* lint
* Fix tests
* fix merge fails
---------
Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Robin <robin@robin.town>
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
Co-authored-by: Timo K <toger5@hotmail.de>
Co-authored-by: fkwp <github-fkwp@w4ve.de>
This ensures that we don't see a mistaken 'reconnecting' toast while we're hanging up (and also that the leave sound gets a chance to play in widgets once again).
* remove redis, since we dont use it
* update localhost TLS certificat to add *.othersite.m.localhost wildcard
* allow for federation
* Add services and config files for Matrix site othersite.m.localhost
* add element web instance app.othersite.m.localhost
* update README
* exclude synapse database for othersite.m.localhost
* linting
* make ring$ a behavior and add code comments to justify/explain the change.
Signed-off-by: Timo K <toger5@hotmail.de>
* Add test: reproduce "ring does not stop" race.
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
* Add ring notification to UserIntent.StartNewCallDM
Signed-off-by: Timo K <toger5@hotmail.de>
* Add more tests (refactor to compute + get)
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
* default mute states (unmuted!) in widget mode (embedded + intent)
Signed-off-by: Timo K <toger5@hotmail.de>
* review
Signed-off-by: Timo K <toger5@hotmail.de>
* introduce a cache for the url params.
Signed-off-by: Timo K <toger5@hotmail.de>
* Add an option to skip the cache.
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
* Fix the interactivity of buttons while reconnecting or in earpiece mode
When we're in one of these modes, we need to ensure that everything above the overlay (the header and footer buttons) is interactive, while everything obscured by the overlay (the media tiles) is non-interactive and removed from the accessibility tree. It's not a very easy task to trap focus *outside* an element, so the best solution I could come up with is to set tabindex="-1" manually on all interactive elements belonging to the media tiles.
* Write a Playwright test for reconnecting
* fix lints
Signed-off-by: Timo K <toger5@hotmail.de>
* fix test
Signed-off-by: Timo K <toger5@hotmail.de>
* enable http2 for matrx-rtc host to allow the jwt service to talk to the SFU
* remove rate limit for delayed events
* more time to connect to livekit SFU
* Due to a Firefox issue we set the start anchor for the tab test to the Mute microphone button
* adapt to most recent Element Web version
* Use the "End call" button as proofe for a started call
* Currrenty disabled due to recent Element Web
- not indicating the number of participants
- bypassing Lobby
* linting
* disable 'can only interact with header and footer while reconnecting' for firefox
---------
Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
Co-authored-by: Timo K <toger5@hotmail.de>
Co-authored-by: fkwp <github-fkwp@w4ve.de>
* add wait for pickup overlay
Signed-off-by: Timo K <toger5@hotmail.de>
* refactor and leave logic
Signed-off-by: Timo K <toger5@hotmail.de>
* recursive play sound logic
Signed-off-by: Timo K <toger5@hotmail.de>
* review
Signed-off-by: Timo K <toger5@hotmail.de>
* text color
Signed-off-by: Timo K <toger5@hotmail.de>
* overlay styling and interval fixes
Signed-off-by: Timo K <toger5@hotmail.de>
* fix permissions and styling
Signed-off-by: Timo K <toger5@hotmail.de>
* fix always getting pickup sound
Signed-off-by: Timo K <toger5@hotmail.de>
* Add sound effects for declined,timeout and ringtone
* better ringtone
* Integrate sounds
* Ensure leave sound does not play
* Remove unused blocked sound
* fix test
* Improve tests
* Loop ring sound inside Audio context for better perf.
* lint
* better ringtone
* Update to delay ringtone logic.
* lint + fix test
* Tidy up ring sync and add comments.
* lint
* Refactor onLeave to take a sound so we don't need to repeat the sound
* fix import
---------
Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Timo K <toger5@hotmail.de>
After a membership manager error, clicking the 'reconnect' button did nothing. This is because we were forgetting to clear the external error state, causing it to transition directly back to the same error state.
With this change I'm also taking care to not show the standard "Connection to the server has been lost" banner in the call view, since that is now covered by the 'reconnecting' message.
* bump js-sdk to Allow multiple rtc sessions per room (with different sessionDescriptions)
* Allow new state key string-packing format for widget mode
* bump js-sdk to latest version (with merged multi session PR)
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Timo K <toger5@hotmail.de>
* Introduce condigurable auto leave option
* Read url params for auto leave
* add tests
* rename url param to `autoLeave`
* lint
Signed-off-by: Timo K <toger5@hotmail.de>
* fix scope in CallViewModel tests
Signed-off-by: Timo K <toger5@hotmail.de>
* use auto leave in DM case
Signed-off-by: Timo K <toger5@hotmail.de>
* Make last once leave logic based on matrix user id (was participant id before)
Signed-off-by: Timo K <toger5@hotmail.de>
* add test for multi device auto leave
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
* Add a fullscreen button that uses the element request Fullscreen browser api
Signed-off-by: Timo K <toger5@hotmail.de>
* use body instead of root node
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
* Stop reading deprecated config options
* add PR-Breaking-Change as one of the possible PR- prefix labels
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
* Set available devices to empty map on safari.
Signed-off-by: Timo K <toger5@hotmail.de>
* better safari check
Signed-off-by: Timo K <toger5@hotmail.de>
---------
Signed-off-by: Timo K <toger5@hotmail.de>
I believe that the issue we were originally investigating using these increased timeouts was the fault of my earlier ISP in SW Virginia. I can't recall reproducing the exact issue on other networks.
0e0fba6575 added the ability to send call notification events when starting a call, but I forgot to give the widget the right capabilities to do this. The effect was that notifications just wouldn't send in widget mode.
* refactor UrlParams to use a preset intent system
* change defaults for intend headers
* add: getEnumParam to ParamParser
* remove deprecated url params
* only allow skip lobby in widget (more strict needs test adjustment)
* fix tests that now require the url to be a widget url
Co-authored-by: Robin <robin@robin.town>
---------
Co-authored-by: Robin <robin@robin.town>
* Send notification events when starting a call
Previously this has been the responsibility of the hosting application (Element Web / Element X), but I would like to move this responsibility to Element Call itself to make it even more lightweight to integrate Element Call into a widget-capable client.
* use RTCNotification event
* add url param
* bump to latest js-sdk
* remove everything decline related
* use notification type in url params
* fix url .md docs
* back to `head=develop` and using js-sdk with send notification feature
* format
---------
Co-authored-by: Timo <toger5@hotmail.de>
This hook is simpler in its implementation (therefore hopefully more correct & performant) and enforces a type-level distinction between raw Observables and Behaviors.
Rather than the 'share screen' button. Small screens are most likely to be mobile devices which wouldn't have the ability to share their screen, anyways.
* Add a global control for toggling earpiece mode
This will be used by Element X to show an earpiece toggle button in the header.
* Add an earpiece overlay
* Fix header
The header needs to be passed forward as a string to some components and as a bool (hideHeader) to others.
Also use a enum instead of string options.
* fix top clipping with header
* hide app bar in pip
* revert android overlay app_bar
* Modernize AppBarContext
* Style header icon color as desired and switch earpice/speaker icon
* fix initial selection when using controlled media
* Add "Back to video" button
* fix tests
* remove dead code
* add snapshot test
* fix back to video button
* Request capability to learn the room name
We now need the room name in order to implement the mobile (widget-based) designs with the app bar.
* Test the CallViewModel output switcher directly
---------
Co-authored-by: Timo <toger5@hotmail.de>
* bugfix: #3344 Reconnecting to the same SFU on membership change
* fixup! commit error
* Keep useActiveLivekitFocus from changing focus spuriously
* Remove redundant fix for spurious focus changes
We've now fixed it at the source by prohibiting state changes in useActiveLivekitFocus itself.
---------
Co-authored-by: Robin <robin@robin.town>
* Replace useContext with use
The docs recommend the use hook because it is simpler and allows itself to be called conditionally.
* Simplify our context providers
React 19 lets you omit the '.Provider' bit.
* Add `onBackButtonPressed` controls api
* Update docs/controls.md
Co-authored-by: Robin <robin@robin.town>
---------
Co-authored-by: Robin <robin@robin.town>
* Refactor media devices to live outside React as Observables
This moves the media devices state out of React to further our transition to a MVVM architecture in which we can more easily model and store complex application state. I have created an AppViewModel to act as the overarching state holder for any future non-React state we end up creating, and the MediaDevices reside within this. We should move more application logic (including the CallViewModel itself) there in the future.
* Address review feedback
* Fixes from ios debugging session: (#3342)
- dont use preferred vs selected concept in controlled media. Its not needed since we dont use the id for actual browser media devices (the id's are not even actual browser media devices)
- add more logging
- add more conditions to not accidently set a deviceId that is not a browser deviceId but one provided via controlled.
---------
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
echo "[yarn-linker] The pre-commit hook has disabled .links.yaml and MODIFIED the yarn.lock file. Review the staged changes (the hook added yarn.lock, was this desired?) and run \`git commit \` again if they look okay. The post-commit hook will re-enable your links."
# Checks if there currently is linking configured. Informs the user to disable linking before committing.
PNPMFILE=.pnpmfile.cjs
if test -f "$PNPMFILE"; then
echo "[pnpm-linker] The pre-commit hook detected $PNPMFILE which implies you have linked packages in your pnpm-lock.yaml. Run pnpm links:off and commit again. See also linking.md."
// TODO: Re-enable once oxlint supports lint rules that rely on TypeScript type-awareness.
// "eslint-plugin-rxjs"
],
"categories":{
"correctness":"error",
"perf":"error"
},
"options":{
"denyWarnings":true,
"typeAware":true
},
"env":{
"builtin":true
},
"rules":{
"element-call/copyright-header":[
"error",
"/*\nCopyright %%CURRENT_YEAR%% Element Creations Ltd.\n\nSPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial\nPlease see LICENSE in the repository root for full details.\n*/\n\n"
and [LiveKit](https://livekit.io/). It runs in multiple deployment contexts — as a
standalone web app and as a widget embedded in Element Web, Element X iOS, and
Element X Android. It is also the primary R&D foundation for MatrixRTC, which means
its architecture, maintainability, and flexibility are held to a high standard.
We welcome contributions from the community. This document explains how to
contribute effectively so that both you and the maintainers get the best outcome.
## Issue First Policy
> [!IMPORTANT]
> Before writing a single line of code for a new feature or UI change, you **must**
> open an issue and have the approach agreed with the maintainers.
>
> **We will not review or merge feature or UI pull requests that arrive without a
> corresponding, pre-approved issue.**
This is not gatekeeping — it's how we prevent wasted effort on both sides. Element
Call must work correctly across multiple deployment contexts and meet specific product
and design requirements. It is also a fast-moving codebase that underpins ongoing
MatrixRTC development. A PR that looks reasonable in isolation can easily conflict
with in-progress work, planned architecture changes, or design decisions that haven't
been publicly documented yet.
The issue is where we resolve all of that **before** anyone writes code.
**Bug fixes** are no exception — most confirmed bugs should already have an issue anyways, existing issues that are marked as bugs have an implicit maintainer approval. If the solution for the bug is controversial it is highly recommended to discuss the approach in the issue before opening a PR.
## Contribution Workflow
1.**Open an issue** using the [Enhancement request](https://github.com/element-hq/element-call/issues/new?template=enhancement.yml) template.
2.**Wait for feedback.** A maintainer will comment on the issue **within two weeks**. The use case and approach will get dicussed.
This may involve questions, suggestions, or a request to adjust scope.
This also allows to bring design and product into the loop before code gets created.
3.**Get a green light.** Wait for explicit approval from a maintainer before starting
implementation.
4.**Implement.** Write the code against the agreed approach.
5.**Open a PR.** Link to the issue in your PR description and satisfy the checklist
in the PR template.
## Code Quality
Element Call moves fast and the codebase must stay clean and maintainable.
- **Take responsibility for AI-generated code.** AI tools can be a useful aid, but we expect all the generated code to be understood and reasoned about by the contributor. Questions by the maintainers should be answered without just forwarding them to AI. The maintainers also have access to AI tools. If your contribution is just transporting messages between LLM <-> maintaines all our time is better used if the maintainers decide to interact with AI for this specific problem by themselves.
- **Think across deployment contexts.** Changes must work correctly in both standalone
and widget modes. Consider how your change interacts with Element Web, Element X
iOS, and Element X Android.
- **Write tests.** New functionality should be covered by tests. Where it is feasible,
existing uncovered code touched by your PR should also gain tests.
| 1 | Remove stray pause and improve logging for hung e2e tests | e2e | Applied as is: `playwright.config.ts`, `playwright/widget/federated-call.test.ts`. |
| 2 | Fix typo in the name of the createJWTToken Playwright fixture | e2e | Applied as is. |
| 3 | Advertise the loopback address from the dev LiveKit SFUs | backend | Applied as is: `backend/dev_livekit*.yaml`. |
| 4 | Match LiveKit participants against all backend identities | sdk | `RTCMember.backendIdentities` (new, see api.ts). `LivekitBackend.mediaFor` matches `member.backendIdentities.includes(p.identity)`; `RTCParticipation.data$` maps a packet's sender the same way. `mockRtcMembership` in both `sdk/src/utils/test.ts` and `src/utils/test.ts` takes the new 4-argument constructor and a `backendIdentities` override. |
| 5 | Model ConnectionState as a discriminated union | sdk | `Connection.ts` state is `{state: "initialized" \| "authenticating" \| "authenticated" (+ `livekitState`) \| "error" (+ `error`) \| "stopped"}`. Internal to the LiveKit backend: `connectionState.ts` maps it onto the backend-neutral `MediaConnectionState`, so nothing above the backend changes. `DataChannel.ts` and `ConnectionFactory.ts` follow. |
| 6 | Show the whole transport in debug UI and logs | sdk + src | SDK log lines use `JSON.stringify(transport)`. Element Call: `focusUrl$: Behavior<string>` on the member media view models becomes `transport$: Behavior<Transport>` (`TransportMetadata.raw`), carried through `GridTile`, `SpotlightTile`, `MediaView` to `RTCConnectionStats`, which prints the JSON. The old code showed `extractDomain(transport.id)`, which on this branch was already a JSON string, so this also fixes a latent bug. |
| 7 | Group livekitRoomItems$ by connection | n/a | `livekitRoomItems$` and `LivekitRoomAudioRenderer` do not exist on this branch: audio renders from `tracks$` (`MemberAudioRenderer`). Nothing to port. |
| 8 | Move SFU authentication into livekit/auth and choose the endpoint by role | sdk | `sdk/src/media-backend/livekit/auth/{index,openID,csApi,types}.ts` replace `openIDSFU.ts`. Publisher → legacy `/sfu/get`; subscriber → `/get_token`. `slotId` replaces the hard-coded `m.call#ROOM`: `MediaBackendContext.slotId`, taken from the js-sdk session in `RTCParticipation`. The `tokenEndpoint` option of `createLivekitBackend` and the derivation from `MatrixRTCMode` are gone. |
| 9 | Let the publishing Connection authenticate itself | sdk | Every `Connection` has a `role` and fetches its own token; `existingSFUConfig` is gone. `prepareLocalTransport` no longer fetches a token itself: it starts the publisher connection (through the connection map) and resolves once that connection reports `authenticated`, rejecting with its error otherwise. Since `LocalMembership` joins only after `preparedTransport$` resolves, the join waits for the publisher's token exactly as the PR's `authenticated$` gate does. |
| 10 | Reduce ConnectionManager to a ConnectionMap | sdk | `ConnectionMap.ts` replaces `ConnectionManager.ts`; `createConnectionMap$` takes a resolved local transport, and `LivekitBackend` wraps it in `mapScoped` over the local transport, as the PR's `CallViewModel` did. `LocalMedia.ts`, `DataChannel.ts` take `connectionMap$`. |
| 11 | Use the unstable LiveKit transport type | sdk + src | `UnstableLivekitTransport` / `isUnstableLivekitTransport` throughout the backend; `src/settings/settings.ts`, `DeveloperSettingsTab`, test helpers. Transport discovery in `sdk/src/matrixrtc/` is now backend neutral (filters by `type`, see commit 15). |
| 12 | Identify transports by server name as well as URLs | sdk | `TransportLocator {transport, serverName}` in `auth/types.ts`; `ConnectionMap.keyFor` keys on `(url, livekit_service_url, serverName)`. The server name is derived from the user id (`serverNameOf`) for remote members and from our own membership identity for the local transport, so neither `api.ts` nor the backend options need it. |
| 13 | Pin matrix-js-sdk to the member_id branch | superseded | By commit 19; the lockfile pins the PR's final `develop` commit. |
| 14 | Get SFU tokens and delegate delayed leaves through the MSC4195 CS API | sdk | `auth/csApi.ts` (`_unstable_getLivekitToken`, `_unstable_delegateDelayedLeave`), `auth/index.ts` facade with the fallback rules. `LivekitBackend.delegateDelayedLeave` goes through the facade; the delegation probe asks only the service the publisher flow uses. `LivekitBackendOptions.client` picks the two new client methods. |
| 15 | Let developers override the whole transport | sdk + src | `RTCParticipationOptions.transport` / `fallbackTransport` take a `Transport` instead of `transportUrl` / `fallbackTransportUrl` (see api.ts). Element Call: setting `custom-transport` (JSON) replaces `custom-livekit-url`; `DeveloperSettingsTab` edits JSON and validates through `authenticateWithTransport`; `InCallView` turns `config.livekit.livekit_service_url` into the fallback transport. Locale keys `developer_mode.custom_transport.*`. |
| 16 | Deprecate the livekit.livekit_service_url config option | src | Applied as is in `ConfigOptions.ts`. |
| 17 | Cover url transports end to end | e2e + backend | `playwright/url-transport.spec.ts`, `backend/app-service*.yaml`, `docker-compose-dev.yml` (lk-jwt-service 0.8.0) copied as is. |
| 18 | Retain compatibility in compatibility mode | superseded | Reverted by commit 23 inside the PR; only its e2e spec edits survive, in the final spec file. |
| 19 | Switch JS SDK back to develop | deps | `pnpm-lock.yaml` pins `4df2f2bf` and `matrix-widget-api` 1.20.0; `package.json` keeps `#develop`. |
| 20 | Fall back to get_token as subscriber on widget API errors | sdk | `isCsApiUnavailable` in `auth/index.ts`; `doNetworkOperationWithRetry(op, shouldRetry)` in `sdk/src/utils/network.ts` with `isNetworkError` in `csApi.ts`. |
| 21 | Share a connection between homeservers using the same JWT service | sdk | `keyFor` ignores the server name for transports without a `url`; `removeDuplicateTransports` keeps the first locator so the local one wins. |
| 22 | Avoid null in transport dev setting | src | `transportText()` in `DeveloperSettingsTab.tsx`. |
| 23 | Make the auth choice mode independent again | sdk | `chooseAuthFlow(transport, role)` has no mode parameter; `config.ts` documents that `MatrixRTCMode` decides how the membership is written, not how the transport is authenticated. |
| 24 | Cover identity matching and delegation across transport shapes | sdk + e2e | Unit tests in `LivekitBackend.test.ts` (see below); `playwright/url-transport.spec.ts` final version. |
| 25 | Cover the subscriber token path, widget fallback and server names | sdk + e2e | `auth/openID.test.ts` subscriber cases; `playwright/widget/url-transport.test.ts`; the server-name test lives in `LivekitBackend.test.ts` ("connects to a remote member's transport on the member's homeserver"). |
## Changes to `api.ts`
Three changes to the public SDK interface, all forced by the PR's behaviour.
### `RTCMember.backendIdentities` (new)
```ts
/** The membership's `rtcBackendIdentity`, `${userId}:${deviceId}`: a key and a debug label. */
rtcBackendIdentity: string;
/**
* Every identity the member may publish under on its transport: the
* participant identity the SFU sees, and the sender of its data packets.
* One per authentication flow the member's transport allows, so a transport
* with both a `url` and a `livekit_service_url` lists the hashed and the
* legacy identity, and the backend matches whichever shows up.
*/
backendIdentities: readonlystring[];
```
Commit 4 of the PR. A member on a transport that carries both a `url` and a
`livekit_service_url` may have authenticated through either flow and so has
two candidate identities; the js-sdk now lists them on the membership. The
backend matches media and data packets against the list.
`rtcBackendIdentity` stays as the stable per-membership key Element Call uses
for tile ids and the debug overlay, and its doc says that it is no longer
what the backend matches on. Read off the membership, like the single identity
before it; the backend will derive both once the identity scheme is its own.
### `RTCParticipationOptions.transport` / `fallbackTransport` replace the url options
```ts
/**
* Use this transport instead of asking the homeserver. Any shape the
* backend serves: for LiveKit a `url`, a `livekit_service_url` or both.
*/
transport?: Transport;
/** Use this transport when the homeserver advertises none of the backend's type. */
fallbackTransport?: Transport;
```
Commit 15 of the PR. The developer override has to carry a whole transport so
that a `url`-only or combined shape can be tested against a homeserver that
does not advertise one yet; a URL cannot say which shape it is. This also
removes the one piece of LiveKit knowledge that sat outside the backend:
`LocalTransport.ts` and `TransportAutoDiscovery.ts` no longer build a
`{type: "livekit", livekit_service_url}` object and filter by `transportType`
only, which `LocalTransport.ts` had marked as the moment to do this. The
host turns its config URL into a transport (`InCallView.rtcParticipationOptions`).
### `TransportMetadata` is unchanged, with one documented limit
`TransportMetadata.id` is still the raw transport serialised with sorted keys.
The LiveKit backend now keys its connections on the transport plus the server
name where the transport has a `url` (commits 12 and 21), so two homeservers
advertising the same `url` get two connections but one `TransportMetadata`.
`connectedTransports$` deduplicates, and `resolved$` reports the first
connection's resolution. This is diagnostics only; the media and data paths
go through the backend's own keys. Extending `TransportMetadata` with the
server name was left out to keep the type transport neutral; a developer panel
that needs it can be given a `serverName` on `BackendConnection` later.
### Related, outside `api.ts`
-`media-backend/api.ts`: `MediaBackendContext.slotId` (commit 8). The
client-server API issues a token for a slot, so the backend needs it;
`RTCParticipation` reads it from the js-sdk session.
-`sdk/index.ts`: `authenticateWithTransport` is now the auth facade's
| Endpoint choice | `chooseAuthFlow` in `src/livekit/auth` | Same function, `sdk/src/media-backend/livekit/auth/index.ts` |
| Join waits for the publisher's token | `authenticated$` gate in `LocalMember.ts` | `prepareLocalTransport` resolves on the publisher connection's `authenticated` state; `LocalMembership` joins after it |
| Delegation probe | The flow's service only, in `LocalMember.joinParams$` | The flow's service only, in `prepareLocalTransport` |
| Delegation handover | `delegateDelayedLeave` facade from `LocalMember` | Same facade from `LivekitBackend.delegateDelayedLeave`, with the delay timeout from the session timings instead of `Config` |
| Transport keys and connection sharing | `ConnectionMap.keyFor` on `TransportLocator` | Same, moved |
| Server names | `membershipsAndTransports$` from the user id; `getLocalTransport` from `client.getDomain()` | `serverNameOf(userId)` for members; our own membership identity's user id for the local transport |
| Identity matching | `membership.backendIdentities.includes(p.identity)` | `member.backendIdentities.includes(p.identity)` in `mediaFor` and for data packets |
| Dev transport override | `customTransport` setting → `getLocalTransport` | `customTransport` setting → `RTCParticipationOptions.transport` |
| Error for a `url` host without MSC4195 | `NoMatrix2AuthorizationService` from the publisher connection | Same error, surfaced through `prepareLocalTransport` → `state$ failed` → `fromMatrixRTCError` |
One deliberate difference: on `main` the publisher connection fetches its token
on every (re)creation and nothing else does; here it is the same, since the
up-front token fetch in `prepareLocalTransport` is gone. There is no second
| `src/livekit/auth/index.test.ts` | `sdk/src/media-backend/livekit/auth/index.test.ts`. Rewritten on the network layer (fetch-mock for the JWT service, mocked client methods) because a `vi.mock` of the sibling `openID` module does not reach the facade's own import under `sdk/`. Same cases, plus a table for `chooseAuthFlow`. |
| `ConnectionMap.test.ts` (incl. commit 21's sharing tests) | `sdk/src/media-backend/livekit/ConnectionMap.test.ts`, replacing `ConnectionManager.test.ts`; plus a role-tagging test |
| `Connection.test.ts` changes | `sdk/src/media-backend/livekit/Connection.test.ts`; plus a publisher-through-the-JWT-service case |
| `LocalMember.test.ts` "joins session with %s delegation support" (by transport shape) | `LivekitBackend.test.ts` "prepares the local transport and asks %s about delegation", "waits for the publisher connection to hold its token before the join", "fails the preparation with the publisher connection's error", "delegates the leave through the flow that authenticated the publisher" |
| `media-backend/livekit/Publisher.ts` | `Publisher.test.ts`: the cases from `main`'s `localMember/Publisher.test.ts` on a real `LocalParticipant` with faked device and signalling calls, plus publish, unpublish and destroy |
| `media-backend/livekit/LocalMedia.ts` | `LocalMedia.test.ts`: on a fake connection map; publish before and after the connection, republish on a new connection, mutes remembered, first publish error, output device |
| `RTCParticipation.ts` | `RTCParticipation.test.ts`: on a fake media backend; join options, state machine, publishing only while joined, delegation, member ids per mode, key replay, data sender check |
| `media-backend/livekit/ConnectionFactory.ts` | `ConnectionFactory.test.ts`: the room options from the requests and media quality (the cases from `main`'s `ECConnectionFactory.test.ts`), `resolved$` |
| `LivekitBackend.ts`›`mediaFor` | A member on another transport, a member changing transport, a member without one (from `main`'s `MatrixLivekitMembers.test.ts`) |
`ConnectionFactory.test.ts` reads the options off the real LiveKit room the
factory builds instead of mocking `livekit-client`: a `vi.mock` of the package
reached the factory's import on the package branch but not on this one.
The `LocalMedia` tests found a race in `ObservableScope.reconcile`: the loop was
marked as running until the promise of its last run had settled, a few
microtasks after it had last checked for new values, so a value arriving in
that window (a connection appearing at just that moment) was never processed.
Fixed in `[SDK] Keep a value that reaches reconcile as its loop winds down`,
with a test that drives the window microtask by microtask. Element Call's
`src/state/ObservableScope.ts` re-exports the SDK's class, so the fix applies
@@ -33,15 +34,16 @@ You can find the latest development version continuously deployed to
## ✨ Key Features
✅ **Decentralized & Federated**– No central authority; works across Matrix
homeservers.
✅ **End-to-End Encrypted**– Secure and private calls.
✅ **Standalone & Widget Mode**– Use as an independent app or embed in Matrix
clients.
✅ **WebRTC-based**– No additional software required.
homeservers.
✅ **End-to-End Encrypted**– Secure and private calls.
✅ **Standalone, Widget & Component Mode**– Use as an independent app, embed
in Matrix clients as a widget, or (experimentally) mount it as a React component
inside your own application.
✅ **WebRTC-based**– No additional software required.
✅ **Scalable with LiveKit**– Supports large meetings via SFU
([MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)).
([MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)).
✅ **Raise Hand**– Participants can signal when they want to speak, helping to
organize the flow of the meeting.
organize the flow of the meeting.
✅ **Emoji Reactions**– Users can react with emojis 👍️ 🎉 👏 🤘, adding
engagement and interactivity to the conversation.
@@ -65,7 +67,7 @@ requiring a separate Matrix client.
### 📲 In-App Calling (Widget Mode in Messenger Apps)
When used as a widget 🧩, Element Call is solely responsible on the core calling
When used as a widget 🧩, Element Call is solely responsible for the core calling
functionality (MatrixRTC). Authentication, event handling, and room state
updates (via the Client-Server API) are handled by the hosting client.
Communication between Element Call and the client is managed through the widget
@@ -89,7 +91,9 @@ and voice calls within Matrix rooms.
Element Call offers two packaging options: one for standalone or widget
deployment, and another for seamless widget-based integration into messenger
apps. Below is an overview of each option.
apps. A third, experimental option builds it as a React component library for
applications that want to render a call inside their own page rather than in an
iframe. Below is an overview of each option.
**Full Package**– Supports both **Standalone** and **Widget** mode. It is
hosted as a static web page and can be accessed via a URL when used as a widget.
@@ -106,57 +110,65 @@ recommended method for embedding Element Call.
"description":"Element Call as a React component. Consumed straight from the repository as a git dependency (github:element-hq/element-call#<ref>&path:/component): the host's package manager runs `prepare`, which builds `dist/`.",
The file is located at `backend/dev_tls_local-ca.crt`. Transfer it via:
- Matrix room
- AirDrop for iphone
### IOS Setup
**Install the certificate profile on iPhone**
- Open the `dev_tls_local-ca.crt` file on your iPhone
- You'll see "Profile Downloaded"
- Go to **Settings → General → VPN & Device Management** (or **Settings → General → Profiles**)
- Tap the "Element Call Dev CA" profile
- Tap **Install** and enter your passcode
- Confirm by tapping **Install** again
**Enable full trust (Critical!)**
- Go to **Settings → General → About → Certificate Trust Settings**
- Under "Enable Full Trust for Root Certificates"
- Toggle **ON** for "Element Call Dev CA"
- Confirm the security warning
**Access Element Call**
Find your laptop's IP address (e.g., `192.168.0.122`) and use one of these URLs in Safari to validate:
```
https://192-168-0-122.nip.io:3000/
```
**For Element X iOS Developer Tools**
In Element X's developer settings, set the Element Call URL to the nip.io url (replace . with - in the IP address):
```
https://192-168-0-122.nip.io:3000/room
```
### Android Setup
**Transfer the CA certificate to your Android device**
The file is located at `backend/dev_tls_local-ca.crt`.
**Install the certificate**
This might vary by Android version and manufacturer, but generally:
- Open **Settings** search for "CA Certificate"/"Certificate"
- Tap **Install a certificate** or **Install from storage**
- Select **CA certificate**
- Confirm the security warning
- Navigate to and select the `dev_tls_local-ca.crt` file
- Give it a name like "Element Call Dev CA"
**Access Element Call**
Find your laptop's IP address (e.g., `192.168.0.122`) and use one of these URLs in Chrome to validate:
```
https://192-168-0-122.nip.io:3000/
```
**For Element X Android Developer Tools**
In Element X's developer settings, set the Element Call URL to the nip.io url (replace . with - in the IP address):
```
https://192-168-0-122.nip.io:3000/room
```
### Why nip.io?
[nip.io](https://nip.io) is a free wildcard DNS service that automatically resolves domain names containing IP addresses. For example, `192-168-0-122.nip.io` automatically resolves to `192.168.0.122`. This means:
- No need to regenerate certificates when your laptop's IP changes
- Works from any device without DNS configuration
- iOS/Android treat it as a proper domain name, not an IP address
- One-time certificate setup works for all future IP addresses
> [!IMPORTANT]
> Make sure your network router doesn't enforce DNS rebinding protection (which will
> break nip.io). If it does, try allow-listing nip.io in your router's administration interface.
@@ -7,19 +7,28 @@ A few aspects of Element Call's interface can be controlled through a global API
-`controls.canEnterPip(): boolean` Determines whether it's possible to enter picture-in-picture mode.
-`controls.enablePip(): void` Puts the call interface into picture-in-picture mode. Throws if not in a call.
-`controls.disablePip(): void` Takes the call interface out of picture-in-picture mode, restoring it to its natural display mode. Throws if not in a call.
-`controls.onPipMediaOrientationUpdate: ((orientation: "landscape"|"portrait") => void) | undefined` Callback called whenever the PiP media orientation changes.
The client should track this value to already initiate the pip in the right orientation.
It should update the orientation of the current Pip window when called.
## Audio devices
On mobile platforms (iOS, Android), web views do not reliably support selecting audio output devices such as the main speaker, earpiece, or headset. To address this limitation, the following functions allow the hosting application (e.g., Element Web, Element X) to manage audio devices via exposed JavaScript interfaces. These functions must be enabled using the URL parameter `controlledAudioDevices` to take effect.
-`controls.setAvailableAudioDevices(devices: { id: string, name: string, forEarpiece?: boolean, isEarpiece?: boolean isSpeaker?: boolean, isExternalHeadset?, boolean; }[]): void` Sets the list of available audio outputs. `forEarpiece` is used on iOS only.
-`controls.setAvailableAudioDevices(devices: { id: string, name: string, forEarpiece?: boolean, isEarpiece?: boolean isSpeaker?: boolean, isExternalHeadset?: boolean }[]): void` Sets the list of available audio outputs. `forEarpiece` is used on iOS only.
It flags the device that should be used if the user selects earpiece mode. This should be the main stereo loudspeaker of the device.
-`controls.onAudioDeviceSelect: ((id: string) => void) | undefined` Callback called whenever the user or application selects a new audio output.
-`controls.setAudioDevice(id: string): void` Sets the selected audio device in Element Call's menu. This should be used if the OS decides to automatically switch to Bluetooth, for example.
-`controls.setAudioEnabled(enabled: boolean)` Enables/disables all audio output from the application. Output is enabled by default.
-`showNativeAudioDevicePicker: (() => void) | undefined`. Callback called whenever the user presses the output button in the settings menu.
This button is only shown on iOS. (`userAgent.includes("iPhone")`)
-`controls.onAudioPlaybackStarted: ((id: string) => void) | undefined`: This will be called the first time we start
playing audio in the webview. It can be helpful to do device setup on the native app when the webviews audio is ready.
In particular android is using it to setup the output channel so that the call volume can
be controlled by the hardware volume rocker.
## Element Call button delegation
Callbacks for buttons in EC that are handled by the native application
-`showNativeAudioDevicePicker: (() => void) | undefined`. Callback called whenever the user presses the output button in the settings menu.
This button is only shown on iOS. (`/iPad|iPhone|iPod|Mac/.test(navigator.userAgent)`)
-`onBackButtonPressed: (() => void) | undefined`. Callback when the webview detects a tab on the header's back button.
| **Responsibility for regulatory compliance** | The administrator that is deploying the app is responsible for compliance with any applicable regulations (e.g. privacy) | The developer of the messenger app is responsible for compliance |
| **Analytics consent** | Element Call will show a consent UI. | Element Call will not show a consent UI. The messenger app should only provide the embedded Element Call with the [analytics URL parameters](./url-params.md#embedded-only-parameters) if consent has been granted. |
| **Analytics consent** | Element Call will show a consent UI. | Element Call will not show a consent UI. The messenger app should only provide the embedded Element Call with the [analytics URL parameters](./url_params.md#embedded-only-parameters) if consent has been granted. |
| **Analytics data** | Element Call will send data to the Posthog, Sentry and Open Telemetry targets specified by the administrator in the `config.json` | Element Call will send data to the Posthog and Sentry targets specified in the URL parameters by the messenger app |
### Using the embedded package within a messenger app
@@ -25,8 +28,8 @@ The basics are:
1. Add the appropriate platform dependency as given for a [release](https://github.com/element-hq/element-call/releases), or use the embedded tarball. e.g. `npm install @element-hq/element-call-embedded@0.9.0`
2. Include the assets from the platform dependency in the build process. e.g. copy the assets during a [Webpack](https://github.com/element-hq/element-web/blob/247cd8d56d832d006d7dfb919d1042529d712b59/webpack.config.js#L677-L682) build.
3. Use the `index.html` entrypointof the imported assets when you are constructing the WebView or iframe. e.g. using a [relative path in a webapp](https://github.com/element-hq/element-web/blob/247cd8d56d832d006d7dfb919d1042529d712b59/src/models/Call.ts#L680), or on the the Android [WebViewAssetLoader](https://github.com/element-hq/element-x-android/blob/fe5aab6588ecdcf9354a3bfbd9e97c1b31175a8f/features/call/impl/src/main/kotlin/io/element/android/features/call/impl/utils/DefaultCallWidgetProvider.kt#L20)
4. Set any of the [embedded-only URL parameters](./url-params.md#embedded-only-parameters) that you need.
3. Use the `index.html` entrypointof the imported assets when you are constructing the WebView or iframe. e.g. using a [relative path in a webapp](https://github.com/element-hq/element-web/blob/247cd8d56d832d006d7dfb919d1042529d712b59/src/models/Call.ts#L680), or on the the Android [WebViewAssetLoader](https://github.com/element-hq/element-x-android/blob/fe5aab6588ecdcf9354a3bfbd9e97c1b31175a8f/features/call/impl/src/main/kotlin/io/element/android/features/call/impl/utils/DefaultCallWidgetProvider.kt#L20)
4. Set any of the [embedded-only URL parameters](./url_params.md#embedded-only-parameters) that you need.
## Widget vs standalone mode
@@ -35,5 +38,5 @@ Element Call is developed using the [js-sdk](https://github.com/matrix-org/matri
As a widget, the app only uses the core calling (MatrixRTC) parts. The rest (authentication, sending events, getting room state updates about calls) is done by the hosting client.
Element Call and the hosting client are connected via the widget API.
Element Call detects that it is run as a widget if a widgetId is defined in the url parameters. If `widgetId` is present then Element Call will try to connect to the client via the widget postMessage API using the parameters provided in [Url Format and parameters
](./url-params.md).
Element Call detects that it is run as a widget if `widgetId` is defined in the url parameters. If `widgetId` is present then Element Call will try to connect to the client via the widget postMessage API using the parameters provided in [Url Format and parameters
If you want to make changes to a package that Element Call depends on and see those changes applied in real time, you can create a link to a local copy of the package. Yarn has a command for this (`yarn link`), but it's not recommended to use it as it ends up modifying package.json with details specific to your development environment.
Run:
Instead, you can use our little 'linker' plugin. Create a file named `.links.yaml` in the Element Call project directory, listing the names and paths of any dependencies you want to link. For example:
Run: 'git commit' with links enabled to test the git pre-commit hook.
Run: 'pnpm links:off' to be able to commit again
Run: 'git config --local core.hooksPath ""' to allow committing with linking (not recommended)
Run: 'rm links.cjs' & 'git config --local core.hooksPath ""' to fully revert what this script did
```
# Developing with linked packages
If you want to make changes to a package that Element Call depends on and see those changes applied in real time, you can create a link to a local copy of the package. `pnpm` has a command for this (`pnpm link`), but it's not recommended to use it as it ends up modifying package.json with details specific to your development environment.
Instead, create a file named `.links.cjs` in the Element Call project directory (or run `./scripts/setup-linking.sh` to create a template), listing the names and paths of any dependencies you want to link. For example:
Then run `pnpm links:on`. (this will activate the pnpm file + run `pnpm install` to setup the linking)
## Hooks
Changes in `.links.yaml` will also update `yarn.lock` when `yarn` is executed. The lockfile will then contain the local
Changes in `.links.cjs` will also update `pnpm-lock.yaml` when `pnpm install` is executed. The lockfile will then contain the local
version of the package which would not work on others dev setups or the github CI.
One always needs to run:
One always needs to remove the pnpm `readPackage` script (the `.pnpmfile.cjs`) and run:
```bash
mv .links.yaml .links.disabled.yaml
yarn
pnpm install
```
before committing a change.
To make it more convenient to work with this linking system we added git hooks for your conviniece.
A `pre-commit` hook will run `mv .links.yaml .links.disabled.yaml`, `yarn` and `git add yarn.lock` if it detects
a `.links.yaml` file and abort the commit.
You will than need to check if the resulting changes are appropriate and commit again.
To make this less of a foot gun we added a git hook.
A `pre-commit` hook will check if linking is currently used. If it detects
a `.pnpmfile.cjs` file it will abort the commit with an explanatory message.
You will then need to run `pnpm links:off` and commit again.
A `post-commit` hook will setup the linking as it was
before if a `.links.disabled.yaml` is present. It runs `mv .links.disabled.yaml .links.yaml` and `yarn`.
To activate the hooks automatically configure git with
To activate the hooks configure git with (when using the setup script (`./scripts/setup-linking.sh`) this is already done):
```bash
git config --local core.hooksPath .githooks/
git config --local core.hooksPath .githooks
```
This will add the hook path for this repository only to .gihooks. which is a tracked (by git) folder containing the pre-commit hook.
## Testing against a matrix-js-sdk branch in CI
Linking covers local development. In CI, `scripts/layered.sh` plays the same
role for `matrix-js-sdk`: after the usual frozen install, `scripts/fetchdep.sh`
looks for a matrix-js-sdk branch with the same name as the pull request's head
branch (in the fork first, when the pull request comes from one). If there is
one, it is cloned, packed and installed over the locked version. If there is
none, the install stays as `pnpm-lock.yaml` pins it.
This means a change that needs a not yet merged matrix-js-sdk change can be tested
by giving both branches the same name. This mirrors how element-web tests against
matrix-js-sdk. Once the matrix-js-sdk branch is merged and deleted, the pull
request falls back to the lockfile and needs a lockfile bump to stay green,
which is also what keeps `main` consistent after the merge.
## Background
Information, why this approach is used can be found in the [linking concept reasoning](./linking_concept_reasoning.md) document.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Johannes Marbach
