From f0ec4b9add5a865a960decb03f8d77bc0c20b2ae Mon Sep 17 00:00:00 2001 From: Robin Date: Fri, 3 Apr 2026 18:47:13 +0200 Subject: [PATCH 01/10] Add intents for group voice calls This adds two new intents: start_call_voice and join_existing_voice. I need the latter in order to implement Element Web's new incoming call toasts, in which you can turn off your video before joining a group call. The other one, start_call_voice, exists more for completeness than anything; we don't currently want to allow users to start voice calls in group chats in our messenger clients, but maybe Cinny would, for instance. --- docs/url-params.md | 52 +++++++++++++++++++++---------------------- src/UrlParams.test.ts | 29 +++++++++++++++++++++++- src/UrlParams.ts | 11 +++++++++ 3 files changed, 65 insertions(+), 27 deletions(-) diff --git a/docs/url-params.md b/docs/url-params.md index e24e9823..ac6f2f7a 100644 --- a/docs/url-params.md +++ b/docs/url-params.md @@ -46,32 +46,32 @@ possible to support encryption. These parameters are relevant to both [widget](./embedded-standalone.md) and [standalone](./embedded-standalone.md) modes: -| Name | Values | Required for widget | Required for SPA | Description | -| ---------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `intent` | `start_call`, `join_existing`, `start_call_dm`, `join_existing_dm. | No, defaults to `start_call` | No, defaults to `start_call` | The intent is a special url parameter that defines the defaults for all the other parameters. In most cases it should be enough to only set the intent to setup element-call. | -| `allowIceFallback` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Allows use of fallback STUN servers for ICE if the user's homeserver doesn’t provide any. | -| `posthogUserId` | Posthog analytics ID | No | No | Available only with user's consent for sharing telemetry in Element Web. | -| `appPrompt` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Prompts the user to launch the native mobile app upon entering a room, applicable only on Android and iOS, and must be enabled in config. | -| `confineToRoom` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Keeps the user confined to the current call/room. | -| `displayName` | | No | No | Display name used for auto-registration. | -| `enableE2EE` (deprecated) | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Legacy flag to enable end-to-end encryption, not used in the `livekit` branch. | -| `fontScale` | A decimal number such as `0.9` | No, defaults to `1.0` | No, defaults to `1.0` | Factor by which to scale the interface's font size. | -| `fonts` | | No | No | Defines the font(s) used by the interface. Multiple font parameters can be specified: `?font=font-one&font=font-two...`. | -| `header` | `none`, `standard` or `app_bar` | No, defaults to `standard` | No, defaults to `standard` | The style of headers to show. `standard` is the default arrangement, `none` hides the header entirely, and `app_bar` produces a header with a back button like you might see in mobile apps. The callback for the back button is `window.controls.onBackButtonPressed`. | -| `hideScreensharing` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Hides the screen-sharing button. | -| `homeserver` | | Not applicable | No | Homeserver for registering a new (guest) user, configures non-default guest user server when creating a spa link. | -| `lang` | [BCP 47](https://www.rfc-editor.org/info/bcp47) code | No | No | The language the app should use. | -| `password` | | No | No | E2EE password when using a shared secret. (For individual sender keys in embedded mode this is not required.) | -| `perParticipantE2EE` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Enables per participant encryption with Keys exchanged over encrypted matrix room messages. | -| `controlledAudioDevices` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the [global JS controls for audio devices](./controls.md#audio-devices) should be enabled, allowing the list of audio devices to be controlled by the app hosting Element Call. | -| `roomId` | [Matrix Room ID](https://spec.matrix.org/v1.12/appendices/#room-ids) | Yes | No | Anything about what room we're pointed to should be from useRoomIdentifier which parses the path and resolves alias with respect to the default server name, however roomId is an exception as we need the room ID in embedded widget mode, and not the room alias (or even the via params because we are not trying to join it). This is also not validated, where it is in `useRoomIdentifier()`. | -| `showControls` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Displays controls like mute, screen-share, invite, and hangup buttons during a call. | -| `skipLobby` (deprecated: use `intent` instead) | `true` or `false` | No. If `intent` is explicitly `start_call` then defaults to `true`. Otherwise defaults to `false` | No, defaults to `false` | Skips the lobby to join a call directly, can be combined with preload in widget. When `true` the audio and video inputs will be muted by default. (This means there currently is no way to start without muted video if one wants to skip the lobby. Also not in widget mode.) | -| `theme` | One of: `light`, `dark`, `light-high-contrast`, `dark-high-contrast` | No, defaults to `dark` | No, defaults to `dark` | UI theme to use. | -| `viaServers` | Comma separated list of [Matrix Server Names](https://spec.matrix.org/v1.12/appendices/#server-name) | Not applicable | No | Homeserver for joining a room, non-empty value required for rooms not on the user’s default homeserver. | -| `sendNotificationType` | `ring` or `notification` | No | No | Will send a "ring" or "notification" `m.rtc.notification` event if the user is the first one in the call. | -| `autoLeaveWhenOthersLeft` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the app should automatically leave the call when there is no one left in the call. | -| `waitForCallPickup` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | When sending a notification, show UI that the app is awaiting an answer, play a dial tone, and (in widget mode) auto-close the widget once the notification expires. | +| Name | Values | Required for widget | Required for SPA | Description | +| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `intent` | `start_call`, `join_existing`, `start_call_voice`, `join_existing_voice`, `start_call_dm`, `join_existing_dm`, `start_call_dm_voice`, or `join_existing_dm_voice`. | No, defaults to `start_call` | No, defaults to `start_call` | The intent is a special url parameter that defines the defaults for all the other parameters. In most cases it should be enough to only set the intent to setup element-call. | +| `allowIceFallback` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Allows use of fallback STUN servers for ICE if the user's homeserver doesn’t provide any. | +| `posthogUserId` | Posthog analytics ID | No | No | Available only with user's consent for sharing telemetry in Element Web. | +| `appPrompt` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Prompts the user to launch the native mobile app upon entering a room, applicable only on Android and iOS, and must be enabled in config. | +| `confineToRoom` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Keeps the user confined to the current call/room. | +| `displayName` | | No | No | Display name used for auto-registration. | +| `enableE2EE` (deprecated) | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Legacy flag to enable end-to-end encryption, not used in the `livekit` branch. | +| `fontScale` | A decimal number such as `0.9` | No, defaults to `1.0` | No, defaults to `1.0` | Factor by which to scale the interface's font size. | +| `fonts` | | No | No | Defines the font(s) used by the interface. Multiple font parameters can be specified: `?font=font-one&font=font-two...`. | +| `header` | `none`, `standard` or `app_bar` | No, defaults to `standard` | No, defaults to `standard` | The style of headers to show. `standard` is the default arrangement, `none` hides the header entirely, and `app_bar` produces a header with a back button like you might see in mobile apps. The callback for the back button is `window.controls.onBackButtonPressed`. | +| `hideScreensharing` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Hides the screen-sharing button. | +| `homeserver` | | Not applicable | No | Homeserver for registering a new (guest) user, configures non-default guest user server when creating a spa link. | +| `lang` | [BCP 47](https://www.rfc-editor.org/info/bcp47) code | No | No | The language the app should use. | +| `password` | | No | No | E2EE password when using a shared secret. (For individual sender keys in embedded mode this is not required.) | +| `perParticipantE2EE` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Enables per participant encryption with Keys exchanged over encrypted matrix room messages. | +| `controlledAudioDevices` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the [global JS controls for audio devices](./controls.md#audio-devices) should be enabled, allowing the list of audio devices to be controlled by the app hosting Element Call. | +| `roomId` | [Matrix Room ID](https://spec.matrix.org/v1.12/appendices/#room-ids) | Yes | No | Anything about what room we're pointed to should be from useRoomIdentifier which parses the path and resolves alias with respect to the default server name, however roomId is an exception as we need the room ID in embedded widget mode, and not the room alias (or even the via params because we are not trying to join it). This is also not validated, where it is in `useRoomIdentifier()`. | +| `showControls` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Displays controls like mute, screen-share, invite, and hangup buttons during a call. | +| `skipLobby` (deprecated: use `intent` instead) | `true` or `false` | No. If `intent` is explicitly `start_call` then defaults to `true`. Otherwise defaults to `false` | No, defaults to `false` | Skips the lobby to join a call directly, can be combined with preload in widget. When `true` the audio and video inputs will be muted by default. (This means there currently is no way to start without muted video if one wants to skip the lobby. Also not in widget mode.) | +| `theme` | One of: `light`, `dark`, `light-high-contrast`, `dark-high-contrast` | No, defaults to `dark` | No, defaults to `dark` | UI theme to use. | +| `viaServers` | Comma separated list of [Matrix Server Names](https://spec.matrix.org/v1.12/appendices/#server-name) | Not applicable | No | Homeserver for joining a room, non-empty value required for rooms not on the user’s default homeserver. | +| `sendNotificationType` | `ring` or `notification` | No | No | Will send a "ring" or "notification" `m.rtc.notification` event if the user is the first one in the call. | +| `autoLeaveWhenOthersLeft` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the app should automatically leave the call when there is no one left in the call. | +| `waitForCallPickup` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | When sending a notification, show UI that the app is awaiting an answer, play a dial tone, and (in widget mode) auto-close the widget once the notification expires. | ### Widget-only parameters diff --git a/src/UrlParams.test.ts b/src/UrlParams.test.ts index ec92ee89..a18d10f1 100644 --- a/src/UrlParams.test.ts +++ b/src/UrlParams.test.ts @@ -271,7 +271,11 @@ describe("UrlParams", () => { computeUrlParams( "?intent=start_call&widgetId=1234&parentUrl=parent.org", ), - ).toMatchObject({ ...startNewCallDefaults("desktop"), skipLobby: false }); + ).toMatchObject({ + ...startNewCallDefaults("desktop"), + skipLobby: false, + callIntent: "video", + }); }); it("accepts start_call_dm mobile", () => { @@ -308,6 +312,29 @@ describe("UrlParams", () => { ), ).toMatchObject(joinExistingCallDefaults("desktop")); }); + + it("accepts start_call_voice", () => { + expect( + computeUrlParams( + "?intent=start_call_voice&widgetId=1234&parentUrl=parent.org", + ), + ).toMatchObject({ + ...startNewCallDefaults("desktop"), + skipLobby: false, + callIntent: "audio", + }); + }); + + it("accepts join_existing_voice", () => { + expect( + computeUrlParams( + "?intent=join_existing_voice&widgetId=1234&parentUrl=parent.org", + ), + ).toMatchObject({ + ...joinExistingCallDefaults("desktop"), + callIntent: "audio", + }); + }); }); describe("skipLobby", () => { diff --git a/src/UrlParams.ts b/src/UrlParams.ts index 31101197..3a972119 100644 --- a/src/UrlParams.ts +++ b/src/UrlParams.ts @@ -29,6 +29,8 @@ interface RoomIdentifier { export enum UserIntent { StartNewCall = "start_call", JoinExistingCall = "join_existing", + StartNewCallVoice = "start_call_voice", + JoinExistingCallVoice = "join_existing_voice", StartNewCallDM = "start_call_dm", StartNewCallDMVoice = "start_call_dm_voice", JoinExistingCallDM = "join_existing_dm", @@ -414,6 +416,15 @@ export const computeUrlParams = (search = "", hash = ""): UrlParams => { intentPreset.skipLobby = false; intentPreset.callIntent = "video"; break; + case UserIntent.StartNewCallVoice: + intentPreset.skipLobby = false; + intentPreset.callIntent = "audio"; + break; + case UserIntent.JoinExistingCallVoice: + // On desktop this will be overridden based on which button was used to join the call + intentPreset.skipLobby = false; + intentPreset.callIntent = "audio"; + break; case UserIntent.StartNewCallDMVoice: intentPreset.callIntent = "audio"; // Fall through From 58d377c3f749a65772be85f8ce406f68b37cb277 Mon Sep 17 00:00:00 2001 From: Valere Date: Wed, 8 Apr 2026 11:04:57 +0200 Subject: [PATCH 02/10] test: Add publishing bug reproduction test --- .../federation-oldest-membership-bug.spec.ts | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 playwright/widget/federation-oldest-membership-bug.spec.ts diff --git a/playwright/widget/federation-oldest-membership-bug.spec.ts b/playwright/widget/federation-oldest-membership-bug.spec.ts new file mode 100644 index 00000000..70442e05 --- /dev/null +++ b/playwright/widget/federation-oldest-membership-bug.spec.ts @@ -0,0 +1,92 @@ +/* +Copyright 2026 Element Creations Ltd. + +SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial +Please see LICENSE in the repository root for full details. +*/ + +import { expect, test } from "@playwright/test"; + +import { widgetTest } from "../fixtures/widget-user"; +import { HOST1, HOST2, TestHelpers } from "./test-helpers"; + +widgetTest( + "Bug new joiner was not publishing on correct SFU", + async ({ addUser, browserName }) => { + test.skip( + browserName === "firefox", + "This is a bug in the old widget, not a browser problem.", + ); + + test.slow(); + + // 2 users in federation + const florian = await addUser("floriant", HOST1); + const timo = await addUser("timo", HOST2); + + // Florian creates a room and invites Timo to it + const roomName = "Call Room"; + await TestHelpers.createRoom(roomName, florian.page, [timo.mxId]); + + // Timo joins the room + await TestHelpers.acceptRoomInvite(roomName, timo.page); + + // Ensure we are in legacy mode (should be the default) + await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget( + florian.page, + "legacy", + ); + await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget( + timo.page, + "legacy", + ); + + // Let timo create a call + await TestHelpers.startCallInCurrentRoom(timo.page, false); + await TestHelpers.joinCallFromLobby(timo.page); + + // We want to simulate that the oldest membership authentication is way slower than + // the preffered auth. + // In this setup, timo advertised$ transport will be it's own, and the active will be the one from florian + await florian.page.route( + "**/matrix-rtc.othersite.m.localhost/livekit/jwt/**", + async (route) => { + await new Promise((resolve) => setTimeout(resolve, 2000)); // 5 second delay + await route.continue(); + }, + ); + + // Florian joins the call + await expect(florian.page.getByTestId("join-call-button")).toBeVisible(); + await florian.page.getByTestId("join-call-button").click(); + await TestHelpers.joinCallFromLobby(florian.page); + + await florian.page.waitForTimeout(3000); + await timo.page.waitForTimeout(3000); + + // We should see 2 video tiles everywhere now + for (const user of [timo, florian]) { + const frame = user.page + .locator('iframe[title="Element Call"]') + .contentFrame(); + await expect(frame.getByTestId("videoTile")).toHaveCount(2); + + // No one should be waiting for media + await expect(frame.getByText("Waiting for media...")).not.toBeVisible(); + + // There should be 2 video elements, visible and autoplaying + const videoElements = await frame.locator("video").all(); + expect(videoElements.length).toBe(2); + + const blockDisplayCount = await frame + .locator("video") + .evaluateAll( + (videos: Element[]) => + videos.filter( + (v: Element) => window.getComputedStyle(v).display === "block", + ).length, + ); + expect(blockDisplayCount).toBe(2); + } + }, +); From 08006d640aa89756fb4777a336dc22b0ffb0faca Mon Sep 17 00:00:00 2001 From: Valere Date: Wed, 8 Apr 2026 11:10:27 +0200 Subject: [PATCH 03/10] fix: Ensure to publish on the correct transport --- src/state/CallViewModel/CallViewModel.ts | 25 +++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/src/state/CallViewModel/CallViewModel.ts b/src/state/CallViewModel/CallViewModel.ts index 6dca08dc..2cb59810 100644 --- a/src/state/CallViewModel/CallViewModel.ts +++ b/src/state/CallViewModel/CallViewModel.ts @@ -120,6 +120,7 @@ import { type LocalMatrixLivekitMember, type RemoteMatrixLivekitMember, type MatrixLivekitMember, + areLivekitTransportsEqual, } from "./remoteMembers/MatrixLivekitMembers.ts"; import { type AutoLeaveReason, @@ -517,6 +518,26 @@ export function createCallViewModel$( ), ); + // Observe the transport we should publish + const publishingTransport$ = localTransport$.pipe( + // observe the active$ transport + switchMap((t) => { + return combineLatest([t.active$, t.advertised$]).pipe( + map(([active, advertised]) => { + if (active?.transport) { + // use the active one (oldest member transport) + return active.transport; + } else { + // There is no active transport, we might just be the first member in the call + // so use the advertised to start + return advertised; + } + }), + ); + }), + distinctUntilChanged(areLivekitTransportsEqual), + ); + const localMembership = createLocalMembership$({ scope, homeserverConnected: createHomeserverConnected$( @@ -546,9 +567,7 @@ export function createCallViewModel$( }, connectionManager, matrixRTCSession, - localTransport$: scope.behavior( - localTransport$.pipe(switchMap((t) => t.advertised$)), - ), + localTransport$: scope.behavior(publishingTransport$), logger: logger.getChild(`[${Date.now()}]`), }); From 40dacd523b472a55fa488d8156daf94b4d021e3b Mon Sep 17 00:00:00 2001 From: Valere Date: Thu, 9 Apr 2026 15:22:10 +0200 Subject: [PATCH 04/10] review: Move the all advertised/active down to the LocalMember And let the local member use it properly to send membership event and publish media --- src/state/CallViewModel/CallViewModel.ts | 23 +----- .../localMember/LocalMember.test.ts | 70 +++++++++++++++---- .../CallViewModel/localMember/LocalMember.ts | 59 ++++++++++------ .../localMember/LocalTransport.ts | 2 +- 4 files changed, 94 insertions(+), 60 deletions(-) diff --git a/src/state/CallViewModel/CallViewModel.ts b/src/state/CallViewModel/CallViewModel.ts index 2cb59810..711e5e7e 100644 --- a/src/state/CallViewModel/CallViewModel.ts +++ b/src/state/CallViewModel/CallViewModel.ts @@ -120,7 +120,6 @@ import { type LocalMatrixLivekitMember, type RemoteMatrixLivekitMember, type MatrixLivekitMember, - areLivekitTransportsEqual, } from "./remoteMembers/MatrixLivekitMembers.ts"; import { type AutoLeaveReason, @@ -518,26 +517,6 @@ export function createCallViewModel$( ), ); - // Observe the transport we should publish - const publishingTransport$ = localTransport$.pipe( - // observe the active$ transport - switchMap((t) => { - return combineLatest([t.active$, t.advertised$]).pipe( - map(([active, advertised]) => { - if (active?.transport) { - // use the active one (oldest member transport) - return active.transport; - } else { - // There is no active transport, we might just be the first member in the call - // so use the advertised to start - return advertised; - } - }), - ); - }), - distinctUntilChanged(areLivekitTransportsEqual), - ); - const localMembership = createLocalMembership$({ scope, homeserverConnected: createHomeserverConnected$( @@ -567,7 +546,7 @@ export function createCallViewModel$( }, connectionManager, matrixRTCSession, - localTransport$: scope.behavior(publishingTransport$), + localTransport$, logger: logger.getChild(`[${Date.now()}]`), }); diff --git a/src/state/CallViewModel/localMember/LocalMember.test.ts b/src/state/CallViewModel/localMember/LocalMember.test.ts index e5e9f327..bfcd7167 100644 --- a/src/state/CallViewModel/localMember/LocalMember.test.ts +++ b/src/state/CallViewModel/localMember/LocalMember.test.ts @@ -40,6 +40,10 @@ import { ConnectionManagerData } from "../remoteMembers/ConnectionManager"; import { ConnectionState, type Connection } from "../remoteMembers/Connection"; import { type Publisher } from "./Publisher"; import { initializeWidget } from "../../../widget"; +import { + type LocalTransport, + type LocalTransportWithSFUConfig, +} from "./LocalTransport"; initializeWidget(); @@ -214,7 +218,7 @@ describe("LocalMembership", () => { }; it("throws error on missing RTC config error", () => { - withTestScheduler(({ scope, hot, expectObservable }) => { + withTestScheduler(({ scope, hot, behavior, expectObservable }) => { const localTransport$ = scope.behavior( hot("1ms #", {}, new MatrixRTCTransportMissingError("domain.com")), null, @@ -230,11 +234,16 @@ describe("LocalMembership", () => { ), }; + const aLocalTransport: LocalTransport = { + advertised$: localTransport$, + active$: behavior("a", { a: null }), + }; + const localMembership = createLocalMembership$({ scope, ...defaultCreateLocalMemberValues, connectionManager: mockConnectionManager, - localTransport$, + localTransport$: behavior("a", { a: aLocalTransport }), }); localMembership.requestJoinAndPublish(); @@ -248,7 +257,11 @@ describe("LocalMembership", () => { it("logs if callIntent cannot be updated", async () => { const scope = new ObservableScope(); - const localTransport$ = new BehaviorSubject(aTransport); + const aLocalTransport: LocalTransport = { + advertised$: new BehaviorSubject(aTransport), + active$: new BehaviorSubject(aTransportWithSFUConfig), + }; + const mockConnectionManager = { transports$: constant(new Epoch([])), connectionManagerData$: constant(new Epoch(new ConnectionManagerData())), @@ -264,7 +277,7 @@ describe("LocalMembership", () => { leaveRoomSession: vi.fn(), }, connectionManager: mockConnectionManager, - localTransport$, + localTransport$: new BehaviorSubject(aLocalTransport), }); const expextedLog = "'not connected yet' while updating the call intent (this is expected on startup)"; @@ -279,6 +292,17 @@ describe("LocalMembership", () => { const aTransport = { livekit_service_url: "a", } as LivekitTransportConfig; + + const aTransportWithSFUConfig = { + transport: aTransport, + sfuConfig: { + jwt: "foo", + livekitAlias: "bar", + livekitIdentity: "baz", + url: "bro", + }, + } as LocalTransportWithSFUConfig; + const bTransport = { livekit_service_url: "b", } as LivekitTransportConfig; @@ -307,7 +331,11 @@ describe("LocalMembership", () => { it("recreates publisher if new connection is used, always unpublish and end tracks", async () => { const scope = new ObservableScope(); - const localTransport$ = new BehaviorSubject(aTransport); + const activeTransport$ = new BehaviorSubject(aTransportWithSFUConfig); + const aLocalTransport: LocalTransport = { + advertised$: new BehaviorSubject(aTransport), + active$: activeTransport$, + }; const publishers: Publisher[] = []; let seed = 0; @@ -343,10 +371,13 @@ describe("LocalMembership", () => { connectionManager: { connectionManagerData$: constant(new Epoch(connectionManagerData)), }, - localTransport$, + localTransport$: new BehaviorSubject(aLocalTransport), }); await flushPromises(); - localTransport$.next(bTransport); + activeTransport$.next({ + ...aTransportWithSFUConfig, + transport: bTransport, + }); await flushPromises(); expect(publisherFactory).toHaveBeenCalledTimes(2); @@ -368,8 +399,6 @@ describe("LocalMembership", () => { it("only start tracks if requested", async () => { const scope = new ObservableScope(); - const localTransport$ = new BehaviorSubject(aTransport); - const publishers: Publisher[] = []; const tracks$ = new BehaviorSubject([]); @@ -396,6 +425,11 @@ describe("LocalMembership", () => { typeof vi.fn >; + const aLocalTransport: LocalTransport = { + advertised$: new BehaviorSubject(aTransport), + active$: new BehaviorSubject(aTransportWithSFUConfig), + }; + const connectionManagerData = new ConnectionManagerData(); connectionManagerData.add(connectionTransportAConnected, []); // connectionManagerData.add(connectionTransportB, []); @@ -405,7 +439,7 @@ describe("LocalMembership", () => { connectionManager: { connectionManagerData$: constant(new Epoch(connectionManagerData)), }, - localTransport$, + localTransport$: new BehaviorSubject(aLocalTransport), }); await flushPromises(); expect(publisherFactory).toHaveBeenCalledOnce(); @@ -428,9 +462,15 @@ describe("LocalMembership", () => { const scope = new ObservableScope(); const connectionManagerData = new ConnectionManagerData(); - const localTransport$ = new BehaviorSubject( - null, - ); + + const activeTransport$ = + new BehaviorSubject(null); + + const aLocalTransport: LocalTransport = { + advertised$: new BehaviorSubject(aTransport), + active$: activeTransport$, + }; + const connectionManagerData$ = new BehaviorSubject( new Epoch(connectionManagerData), ); @@ -470,14 +510,14 @@ describe("LocalMembership", () => { connectionManager: { connectionManagerData$, }, - localTransport$, + localTransport$: new BehaviorSubject(aLocalTransport), }); await flushPromises(); expect(localMembership.localMemberState$.value).toStrictEqual( TransportState.Waiting, ); - localTransport$.next(aTransport); + activeTransport$.next(aTransportWithSFUConfig); await flushPromises(); expect(localMembership.localMemberState$.value).toStrictEqual({ matrix: RTCMemberStatus.Connected, diff --git a/src/state/CallViewModel/localMember/LocalMember.ts b/src/state/CallViewModel/localMember/LocalMember.ts index eb641ca7..ec4a2082 100644 --- a/src/state/CallViewModel/localMember/LocalMember.ts +++ b/src/state/CallViewModel/localMember/LocalMember.ts @@ -62,6 +62,8 @@ import { } from "../remoteMembers/Connection.ts"; import { type HomeserverConnected } from "./HomeserverConnected.ts"; import { and$ } from "../../../utils/observable.ts"; +import { type LocalTransport } from "./LocalTransport.ts"; +import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers.ts"; export enum TransportState { /** Not even a transport is available to the LocalMembership */ @@ -127,7 +129,7 @@ interface Props { createPublisherFactory: (connection: Connection) => Publisher; joinMatrixRTC: (transport: LivekitTransportConfig) => void; homeserverConnected: HomeserverConnected; - localTransport$: Behavior; + localTransport$: Behavior; matrixRTCSession: Pick< MatrixRTCSession, "updateCallIntent" | "leaveRoomSession" @@ -160,7 +162,7 @@ interface Props { export const createLocalMembership$ = ({ scope, connectionManager, - localTransport$: localTransportCanThrow$, + localTransport$, homeserverConnected, createPublisherFactory, joinMatrixRTC, @@ -205,23 +207,34 @@ export const createLocalMembership$ = ({ const logger = parentLogger.getChild("[LocalMembership]"); logger.debug(`Creating local membership..`); + // We consider error on the transport as fatal. + // Whether it is the active transport or the preferred transport. + const handleTransportError = (e: unknown): Observable => { + let error: ElementCallError; + if (e instanceof ElementCallError) { + error = e; + } else { + error = new UnknownCallError( + e instanceof Error ? e : new Error("Unknown error from localTransport"), + ); + } + setTransportError(error); + return of(null); + }; + + // This is the transport that we will advertise in our membership. + const advertisedTransport$ = localTransport$.pipe( + switchMap((lt) => lt.advertised$), + catchError(handleTransportError), + distinctUntilChanged(areLivekitTransportsEqual), + ); + // Unwrap the local transport and set the state of the LocalMembership to error in case the transport is an error. - const localTransport$ = scope.behavior( - localTransportCanThrow$.pipe( - catchError((e: unknown) => { - let error: ElementCallError; - if (e instanceof ElementCallError) { - error = e; - } else { - error = new UnknownCallError( - e instanceof Error - ? e - : new Error("Unknown error from localTransport"), - ); - } - setTransportError(error); - return of(null); - }), + const activeTransport$ = scope.behavior( + localTransport$.pipe( + switchMap((lt) => lt.active$.pipe(map((t) => t?.transport ?? null))), + catchError(handleTransportError), + distinctUntilChanged(areLivekitTransportsEqual), ), ); @@ -229,7 +242,7 @@ export const createLocalMembership$ = ({ const localConnection$ = scope.behavior( combineLatest([ connectionManager.connectionManagerData$, - localTransport$, + activeTransport$, ]).pipe( map(([{ value: connectionData }, localTransport]) => { if (localTransport === null) { @@ -398,7 +411,7 @@ export const createLocalMembership$ = ({ const mediaState$: Behavior = scope.behavior( combineLatest([ localConnectionState$, - localTransport$, + activeTransport$, joinAndPublishRequested$, from(trackStartRequested.promise).pipe( map(() => true), @@ -537,9 +550,11 @@ export const createLocalMembership$ = ({ }); }); - // Keep matrix rtc session in sync with localTransport$, connectRequested$ + // Keep matrix rtc session in sync with advertisedTransport$, connectRequested$ scope.reconcile( - scope.behavior(combineLatest([localTransport$, joinAndPublishRequested$])), + scope.behavior( + combineLatest([advertisedTransport$, joinAndPublishRequested$]), + ), async ([transport, shouldConnect]) => { if (!transport) return; // if shouldConnect=false we will do the disconnect as the cleanup from the previous reconcile iteration. diff --git a/src/state/CallViewModel/localMember/LocalTransport.ts b/src/state/CallViewModel/localMember/LocalTransport.ts index 037b6a0b..10ea79c4 100644 --- a/src/state/CallViewModel/localMember/LocalTransport.ts +++ b/src/state/CallViewModel/localMember/LocalTransport.ts @@ -106,7 +106,7 @@ export function isLocalTransportWithSFUConfig( return "transport" in obj && "sfuConfig" in obj; } -interface LocalTransport { +export interface LocalTransport { /** * The transport to be advertised in our MatrixRTC membership. `null` when not * yet fetched/validated. From aea5815dabafa653f1220ecf8cdafa767a0d5fde Mon Sep 17 00:00:00 2001 From: Valere Date: Fri, 10 Apr 2026 09:24:34 +0200 Subject: [PATCH 05/10] Ensure we don't publish to any transport until our own transport is ok --- .../localMember/LocalMember.test.ts | 64 ++++++++++++++++++- .../CallViewModel/localMember/LocalMember.ts | 11 +++- 2 files changed, 73 insertions(+), 2 deletions(-) diff --git a/src/state/CallViewModel/localMember/LocalMember.test.ts b/src/state/CallViewModel/localMember/LocalMember.test.ts index bfcd7167..c8cc45f8 100644 --- a/src/state/CallViewModel/localMember/LocalMember.test.ts +++ b/src/state/CallViewModel/localMember/LocalMember.test.ts @@ -33,7 +33,10 @@ import { PublishState, TrackState, } from "./LocalMember"; -import { MatrixRTCTransportMissingError } from "../../../utils/errors"; +import { + FailToGetOpenIdToken, + MatrixRTCTransportMissingError, +} from "../../../utils/errors"; import { Epoch, ObservableScope } from "../../ObservableScope"; import { constant } from "../../Behavior"; import { ConnectionManagerData } from "../remoteMembers/ConnectionManager"; @@ -254,6 +257,55 @@ describe("LocalMembership", () => { }); }); + it("Should not publish to active transport if advertised has errors", () => { + withTestScheduler(({ scope, hot, behavior, expectObservable }) => { + const advertised$ = scope.behavior( + hot("--#", {}, new FailToGetOpenIdToken(new Error("foo"))), + null, + ); + + // Populate a connection for active + const connectionManagerData = new ConnectionManagerData(); + connectionManagerData.add(connectionTransportBConnected, []); + const mockConnectionManager = { + transports$: constant(new Epoch([bTransport])), + connectionManagerData$: constant(new Epoch(connectionManagerData)), + }; + + const aLocalTransport: LocalTransport = { + advertised$, + active$: behavior("a", { n: null, a: bTransportWithSFUConfig }), + }; + + defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation( + () => { + return {} as unknown as Publisher; + }, + ); + const publisherFactory = + defaultCreateLocalMemberValues.createPublisherFactory as ReturnType< + typeof vi.fn + >; + + const localMembership = createLocalMembership$({ + scope, + ...defaultCreateLocalMemberValues, + connectionManager: mockConnectionManager, + localTransport$: behavior("a", { a: aLocalTransport }), + }); + localMembership.requestJoinAndPublish(); + + + expectObservable(localMembership.localMemberState$).toBe("n-e", { + n: TransportState.Waiting, + e: expect.toSatisfy((e) => e instanceof FailToGetOpenIdToken), + }); + + // Should not have created any publisher + expect(publisherFactory).toHaveBeenCalledTimes(0); + }); + }); + it("logs if callIntent cannot be updated", async () => { const scope = new ObservableScope(); @@ -307,6 +359,16 @@ describe("LocalMembership", () => { livekit_service_url: "b", } as LivekitTransportConfig; + const bTransportWithSFUConfig = { + transport: bTransport, + sfuConfig: { + jwt: "foo2", + livekitAlias: "bar2", + livekitIdentity: "baz2", + url: "bro2", + }, + } as LocalTransportWithSFUConfig; + const connectionTransportAConnected = { livekitRoom: mockLivekitRoom({ localParticipant: { diff --git a/src/state/CallViewModel/localMember/LocalMember.ts b/src/state/CallViewModel/localMember/LocalMember.ts index ec4a2082..188dc543 100644 --- a/src/state/CallViewModel/localMember/LocalMember.ts +++ b/src/state/CallViewModel/localMember/LocalMember.ts @@ -232,7 +232,16 @@ export const createLocalMembership$ = ({ // Unwrap the local transport and set the state of the LocalMembership to error in case the transport is an error. const activeTransport$ = scope.behavior( localTransport$.pipe( - switchMap((lt) => lt.active$.pipe(map((t) => t?.transport ?? null))), + switchMap((lt) => { + return combineLatest([lt.active$, lt.advertised$]).pipe( + map(([active, advertised]) => { + // Our policy is to not publish to another transport if our prefered transport is miss-configured + if (advertised == null) return null; + + return active?.transport ?? null; + }), + ); + }), catchError(handleTransportError), distinctUntilChanged(areLivekitTransportsEqual), ), From 208184909e0138028d4b706b738e97b2599af4aa Mon Sep 17 00:00:00 2001 From: Valere Date: Fri, 10 Apr 2026 10:08:57 +0200 Subject: [PATCH 06/10] fixup: prettier --- src/state/CallViewModel/localMember/LocalMember.test.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/src/state/CallViewModel/localMember/LocalMember.test.ts b/src/state/CallViewModel/localMember/LocalMember.test.ts index c8cc45f8..6eaaa0b0 100644 --- a/src/state/CallViewModel/localMember/LocalMember.test.ts +++ b/src/state/CallViewModel/localMember/LocalMember.test.ts @@ -295,7 +295,6 @@ describe("LocalMembership", () => { }); localMembership.requestJoinAndPublish(); - expectObservable(localMembership.localMemberState$).toBe("n-e", { n: TransportState.Waiting, e: expect.toSatisfy((e) => e instanceof FailToGetOpenIdToken), From 9ac42b240e303aac9d899f13eac7610496bdc954 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Mon, 13 Apr 2026 01:27:40 +0000 Subject: [PATCH 07/10] Update Compound --- package.json | 4 ++-- yarn.lock | 22 +++++++++++----------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/package.json b/package.json index 8aa37499..32476d10 100644 --- a/package.json +++ b/package.json @@ -80,8 +80,8 @@ "@typescript-eslint/eslint-plugin": "^8.31.0", "@typescript-eslint/parser": "^8.31.0", "@use-gesture/react": "^10.2.11", - "@vector-im/compound-design-tokens": "^9.0.0", - "@vector-im/compound-web": "^8.0.0", + "@vector-im/compound-design-tokens": "^10.0.0", + "@vector-im/compound-web": "^9.0.0", "@vitejs/plugin-react": "^4.0.1", "@vitest/coverage-v8": "^4.0.18", "babel-plugin-transform-vite-meta-env": "^1.0.3", diff --git a/yarn.lock b/yarn.lock index 25fecdea..3b057236 100644 --- a/yarn.lock +++ b/yarn.lock @@ -6455,9 +6455,9 @@ __metadata: languageName: node linkType: hard -"@vector-im/compound-design-tokens@npm:^9.0.0": - version: 9.0.0 - resolution: "@vector-im/compound-design-tokens@npm:9.0.0" +"@vector-im/compound-design-tokens@npm:^10.0.0": + version: 10.1.0 + resolution: "@vector-im/compound-design-tokens@npm:10.1.0" peerDependencies: "@types/react": "*" react: ^17 || ^18 || ^19.0.0 @@ -6466,13 +6466,13 @@ __metadata: optional: true react: optional: true - checksum: 10c0/6c53770bfba512d8a9f330ca2d0c481806e35f40d98f53815716e41ddac74d6fc3c4788fcda2e33907d62d2c5c04e64db62176c04513fbee41c7c436730081ce + checksum: 10c0/ffd8223195eac66bcddd85eb5d6cf64ba5ffa521d3673c1caa5346d0346cfe165819d4744b7c1ad6289f8074684a4c4e8f972a54d34ffa91625f1e377e732ad5 languageName: node linkType: hard -"@vector-im/compound-web@npm:^8.0.0": - version: 8.4.0 - resolution: "@vector-im/compound-web@npm:8.4.0" +"@vector-im/compound-web@npm:^9.0.0": + version: 9.0.1 + resolution: "@vector-im/compound-web@npm:9.0.1" dependencies: "@floating-ui/react": "npm:^0.27.0" "@radix-ui/react-context-menu": "npm:^2.2.16" @@ -6487,12 +6487,12 @@ __metadata: "@fontsource/inconsolata": ^5 "@fontsource/inter": ^5 "@types/react": "*" - "@vector-im/compound-design-tokens": ">=1.6.1 <7.0.0" + "@vector-im/compound-design-tokens": ">=1.6.1 <11.0.0" react: ^18 || ^19.0.0 peerDependenciesMeta: "@types/react": optional: true - checksum: 10c0/31b73555c47b373d4250872bfe863a030b487197bf1198e3cf3a1ec344f2b02f0c72c1513bb598c1cbd7a91d3c6a334d0c8ae37bd7c90d4859c864fc223e059a + checksum: 10c0/4ae90e1518001a8a4a2a76e86cfc4fc2bba9b2e556c6e4b56df2c7b04e0e2cdb213b70ee7c7ba4288250c726e8182a473d3d5fdf95890451675239c957251eee languageName: node linkType: hard @@ -8666,8 +8666,8 @@ __metadata: "@typescript-eslint/eslint-plugin": "npm:^8.31.0" "@typescript-eslint/parser": "npm:^8.31.0" "@use-gesture/react": "npm:^10.2.11" - "@vector-im/compound-design-tokens": "npm:^9.0.0" - "@vector-im/compound-web": "npm:^8.0.0" + "@vector-im/compound-design-tokens": "npm:^10.0.0" + "@vector-im/compound-web": "npm:^9.0.0" "@vitejs/plugin-react": "npm:^4.0.1" "@vitest/coverage-v8": "npm:^4.0.18" babel-plugin-transform-vite-meta-env: "npm:^1.0.3" From 0842b2b472c0d4885ea71359777454d7fa65a71b Mon Sep 17 00:00:00 2001 From: Robin Date: Mon, 13 Apr 2026 13:02:15 +0200 Subject: [PATCH 08/10] Fix placement of reaction and raised hand indicators --- src/tile/MediaView.module.css | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/tile/MediaView.module.css b/src/tile/MediaView.module.css index 1183f7c5..e972443f 100644 --- a/src/tile/MediaView.module.css +++ b/src/tile/MediaView.module.css @@ -82,9 +82,9 @@ unconditionally select the container so we can use cqmin units */ ); inset: var(--fg-inset); display: grid; - grid-template-columns: 30px 1fr 30px; + grid-template-columns: 1fr auto; grid-template-rows: 1fr auto; - grid-template-areas: "status status reactions" "nameTag nameTag button"; + grid-template-areas: "status reactions" "nameTag button"; gap: var(--cpd-space-1x); place-items: start; } @@ -125,6 +125,7 @@ unconditionally select the container so we can use cqmin units */ .reactions { grid-area: reactions; + place-self: start end; display: flex; gap: var(--cpd-space-1x); } @@ -192,4 +193,5 @@ unconditionally select the container so we can use cqmin units */ .fg > button:first-of-type { grid-area: button; + place-self: end; } From 5daa110f7ee9127d104c2bd65c37d53d616f5ff7 Mon Sep 17 00:00:00 2001 From: Robin Date: Mon, 13 Apr 2026 13:13:09 +0200 Subject: [PATCH 09/10] Make the footer move floating tiles out of the way when shown If you manage to move your floating video tile to the bottom of the screen in a small landscape window, the footer obscures the tile when shown. The designs want us to smoothly move the floating tile out of the way in this case. --- src/room/InCallView.module.css | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/room/InCallView.module.css b/src/room/InCallView.module.css index 70f7c73a..685cc35c 100644 --- a/src/room/InCallView.module.css +++ b/src/room/InCallView.module.css @@ -55,9 +55,8 @@ Please see LICENSE in the repository root for full details. } .footer.overlay { - position: absolute; - inset-block-end: 0; - inset-inline: 0; + /* Note that the footer is still position: sticky in this case so that certain + tiles can move up out of the way of the footer when visible. */ opacity: 1; transition: opacity 0.15s; } @@ -66,6 +65,11 @@ Please see LICENSE in the repository root for full details. display: grid; opacity: 0; pointer-events: none; + /* Switch to position: absolute so the footer takes up no space in the layout + when hidden. */ + position: absolute; + inset-block-end: 0; + inset-inline: 0; } .footer.overlay:has(:focus-visible) { From 9df60310e44a2da0c4d623827f950f6d594672d7 Mon Sep 17 00:00:00 2001 From: Robin Date: Mon, 13 Apr 2026 15:13:19 +0200 Subject: [PATCH 10/10] Update test snapshots --- .../GroupCallErrorBoundary.test.tsx.snap | 35 +++++++++++-------- .../__snapshots__/InCallView.test.tsx.snap | 7 ++-- 2 files changed, 25 insertions(+), 17 deletions(-) diff --git a/src/room/__snapshots__/GroupCallErrorBoundary.test.tsx.snap b/src/room/__snapshots__/GroupCallErrorBoundary.test.tsx.snap index 3f02a49a..0d2d39bc 100644 --- a/src/room/__snapshots__/GroupCallErrorBoundary.test.tsx.snap +++ b/src/room/__snapshots__/GroupCallErrorBoundary.test.tsx.snap @@ -108,8 +108,9 @@ exports[`ConnectionLostError: Action handling should reset error state 1`] = ` class="error" >
rendering > renders 1`] = ` data-show="false" >