Compare commits

..
Author SHA1 Message Date
Johannes Marbach b7bc218b19 Merge pull request #4367 from RaphaelFakhri/RaphaelFakhri/shared-key-storage-key
Read and write the room shared key under its single-prefixed storage key
2026-10-28 16:21:11 +01:00
Raphael Fakhri 85424f76b1 Ignore the URL password when the parameters are for another room 2026-10-28 14:33:07 +00:00
renovate[bot] 6793a4b46e Update ghcr.io/element-hq/element-web:develop Docker digest to ee8ce55 (#4380)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-28 02:32:09 +00:00
Robin 619f5ac3a2 Merge pull request #4375 from element-hq/robin/stats-crash
Fix crash when joining call with connection stats enabled
2026-10-27 18:08:31 +01:00
Robin 5780597b83 Fix crash when joining call with connection stats enabled
As demonstrated by the test. This crashed with "Behavior failed to synchronously emit an initial value."
2026-10-27 16:17:58 +01:00
Robin 76da59be40 Merge pull request #4301 from element-hq/floating-tile
Make drag & drop of floating tiles less janky
2026-10-27 12:16:12 +01:00
Robin c6ad8b2ac7 Use the right comparators 2026-10-27 11:45:35 +01:00
Robin d33a1eb059 Merge branch 'floating-tile' of github.com:element-hq/element-call into floating-tile 2026-10-27 11:04:12 +01:00
Robin 9fcba5a704 Skip redundant tile wrapper updates 2026-10-27 10:44:33 +01:00
RaphaelFakhri dc1459e45e Prefer the stored shared key when the parameters are for another room
keyForRoom returned the password from the parameters even when they were for a different room, and useRoomSharedKey then saved it for this room. Use the stored key first in that case, so another room's password can't replace it.
2026-10-26 21:13:23 +00:00
Johannes Marbach 96333dbc3a Merge branch 'main' into floating-tile 2026-10-26 13:46:54 +01:00
Robin ed6a0e1341 Merge pull request #4371 from element-hq/ellipsize-title
Ellipsise app bar text on mobile
2026-10-26 13:45:06 +01:00
renovate[bot]andJohannes Marbach 90ccf62d5c Update ghcr.io/element-hq/element-web:develop Docker digest to c9ffe3e (#4356)
* Update ghcr.io/element-hq/element-web:develop Docker digest to 79e47d8

* Adapt to new recovery popup button title

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
2026-10-26 12:44:39 +00:00
Robin dde07f6d54 Ellipsise app bar text on mobile
We already had text-overflow: ellipsis set on the title and subtitle elements of the app bar, but due to an annoying quirk of place-items: center, this had no effect. place-items: center stretch seems to give the text elements the right size constraints.
2026-10-26 13:25:13 +01:00
Robin 9ec0cef9c5 Merge pull request #4370 from element-hq/reintroduce-livekit-alias
Reintroduce livekit_alias in published transports
2026-10-26 12:39:14 +01:00
Robin c248ff4280 Don't add livekit_alias to newer transport format 2026-10-26 11:41:45 +01:00
Robin 0f1d0941d0 Reference matrix-rust-sdk PR making livekit_alias optional, too 2026-10-26 11:17:16 +01:00
Robin 7e6e1698c0 Reintroduce livekit_alias in published transports
Some Ruma-based clients still expect it to be present. We should remove it again as soon as https://github.com/ruma/ruma/pull/2642 is widely deployed.
2026-10-26 10:57:49 +01:00
Johannes Marbach de05b82cfe Merge pull request #4369 from element-hq/renovate/npm-storybook-vulnerability
Update dependency storybook to v10.6.2 [SECURITY]
2026-10-26 10:16:21 +01:00
Johannes Marbach b0e985c014 Update related storybook dependencies 2026-10-26 09:57:30 +01:00
Johannes Marbach 33368ab8da Merge pull request #4361 from element-hq/renovate/compound
Update dependency @vector-im/compound-design-tokens to v11.3.1
2026-10-26 09:49:54 +01:00
Johannes Marbach d85a4a1929 Merge pull request #4360 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.23.0
2026-10-26 09:48:08 +01:00
renovate[bot] a6930c5aa9 Update dependency storybook to v10.6.2 [SECURITY] 2026-10-26 08:31:56 +00:00
Johannes Marbach e83093feb2 Merge pull request #4365 from element-hq/renovate/npm-vite-vulnerability
Update dependency vite to v8.2.4 [SECURITY]
2026-10-26 09:30:23 +01:00
renovate[bot] c50127ab62 Update dependency vite to v8.2.4 [SECURITY] 2026-10-22 17:44:29 +00:00
Raphael Fakhri bda7b86c39 Read and write the room shared key under its single-prefixed storage key 2026-10-22 12:24:39 +00:00
renovate[bot] c3ae1c516b Update dependency livekit-client to v2.23.0 2026-10-21 14:25:29 +00:00
renovate[bot] b975dce53e Update dependency @vector-im/compound-design-tokens to v11.3.1 2026-10-19 01:30:27 +00:00
Johannes Marbach 8be229a438 Merge pull request #4354 from element-hq/toger5/johennes/rtc-remote-publish
Fall back to publishing on a remote SFU if possible
2026-10-15 16:27:36 +02:00
Timo K. ef7dce7b07 Update url-transport.test.ts 2026-10-15 16:10:02 +02:00
Timo K. 2de5ee2df5 Update test to consider EWs support for cs-api 2026-10-15 12:53:37 +02:00
Timo K. a427206792 Only return null if there is no endpoint available. 2026-10-15 12:42:36 +02:00
Timo K. 1913a099ab bump EW to inlcude the new widget capabilities 2026-10-15 11:28:45 +02:00
TimoandJohannes Marbach 93b893527e Update sdk/src/media-backend/livekit/RemotePublishTransport.ts
Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
2026-10-15 09:28:40 +02:00
Timo K. 2b26fd472f fmt 2026-10-14 19:03:22 +02:00
Timo K. 281fff0e9f array -> prop|null in discoverLocalTransport 2026-10-14 19:01:14 +02:00
Johannes Marbach bc557dfb5d Revert "Temporarily enable full output to debug the Claude action"
This reverts commit 4827ab5fdc.
2026-10-14 17:55:35 +02:00
Johannes Marbach 4827ab5fdc Temporarily enable full output to debug the Claude action 2026-10-14 17:50:20 +02:00
Timo K. 7af6525f01 fix lints 2026-10-14 16:44:31 +02:00
Timo K. 3b763d87f2 make the transport discovery lk edgecase a EC exclusive workaround
Dont add it to the sdk.
2026-10-14 16:40:27 +02:00
Timo K.andClaude Fable 5.1 8638d4f159 Fall back to publishing on a remote SFU if possible
Port of edf1feae from johannes/rtc-remote-publish onto the SDK split,
where
the MatrixRTC logic lives in sdk/ rather than src/state/CallViewModel/.

Where the original lands:

- getLocalTransport's remote fallback: the LiveKit backend's new
  findRemotePublishTransport (sdk/src/media-backend/livekit/
  RemotePublishTransport.ts), which tries the candidates from the
  js-sdk's
  getRemoteLivekitPublishCandidates oldest first and keeps the first
  whose
  homeserver issues a token with the publish grant.
  discoverLocalTransport
  asks the backend for it after the host's fallback, with a snapshot of
  the
  session's memberships from RTCParticipation, and throws
  MatrixRTCTransportMissingError where that finds nothing too. Transport
  discovery stays backend neutral: the MSC4195 token check is the
  backend's.
- LocalMember's "no delegation on a remote SFU" and the custom
  transport's
  server_name: LivekitBackend.prepareLocalTransport locates the
  transport on
  the homeserver it names (serverNameOfTransport) and skips the
  delegation
  probe where that is not our own.
- SessionBehaviors' membership.getTransportServerName: the same helper
  in
  LivekitBackend.mediaFor.
- decodeSFUJWTPayload, SFUJWTPayload.video.canPublish and the exported
  isCsApiUnavailable: sdk/src/media-backend/livekit/auth/.
- DeveloperSettingsTab, the widget capabilities, the locale string and
  the
  js-sdk pin (9405287c, 44.0.0): applied as is. The error boundary
  snapshots follow the string.

Tests: the LocalTransport cases go to RemotePublishTransport.test.ts and
LocalTransport.test.ts, the LocalMember remote case and the
SessionBehaviors
server-name case to LivekitBackend.test.ts, plus the memberships
snapshot in
RTCParticipation.test.ts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-14 16:09:11 +02:00
Robin 4441bb5e8c Add snapshot coverage for all call layouts
Snapshotting the layout templates so that they get reviewed as a whole feels a little more straightforward than the existing tests.
2026-09-29 03:20:14 +02:00
Robin f2f49315eb Make drag & drop performant in all layouts
While also giving the logic for drag & drop of a floating tile a common home in the view model layer.
2026-09-28 19:53:30 +02:00
Robin 99256fa01c Make drag & drop of floating tiles less janky
This fixes an issue where upon dragging a floating tile to a different corner of the interface, it would briefly move toward its original position before jerking back in the intended direction.
2026-09-28 19:48:49 +02:00
55 changed files with 1989 additions and 892 deletions
+2 -2
View File
@@ -19,7 +19,7 @@ services:
- ./backend/playwright_homeserver-othersite.yaml:/data/cfg/homeserver.yaml:Z
element-web:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:e464b2aca73e248195b24b8c6f62cd331100302284d60cc735405807fb2730be
image: ghcr.io/element-hq/element-web:develop@sha256:ee8ce55012dda2c097147859aece32802ce7a9174e40c3711c56f75ccfe1b169
element-web-1:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:e464b2aca73e248195b24b8c6f62cd331100302284d60cc735405807fb2730be
image: ghcr.io/element-hq/element-web:develop@sha256:ee8ce55012dda2c097147859aece32802ce7a9174e40c3711c56f75ccfe1b169
+1 -2
View File
@@ -88,7 +88,6 @@
},
"mute_all_audio": "Mute all audio (participants, reactions, join sounds)",
"show_connection_stats": "Show connection statistics",
"throw_error_button": "Throw test error",
"url_params": "URL parameters"
},
"disconnected_banner": "Connectivity to the server has been lost.",
@@ -107,7 +106,7 @@
"insufficient_capacity_description": "The server has reached its maximum capacity and you cannot join the call at this time. Try again later, or contact your server admin if the problem persists.",
"livekit_connection_error": "Failed to connect to Livekit server",
"livekit_connection_error_description": "An error occurred while connecting to the Livekit server (<1>Reason:</1> <2>{{ reason }}</2>).",
"matrix_rtc_transport_missing": "The server is not configured to work with {{brand}}. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",
"matrix_rtc_transport_missing": "The server is not configured to work with {{brand}} and no other call participant provides a usable transport. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",
"membership_manager": "Membership Manager Error",
"membership_manager_description": "The Membership Manager had to shut down. This is caused by many consecutive failed network requests.",
"no_matrix_2_authorization_service": "The authorization service for your media server (SFU) is out of date.",
+1 -1
View File
@@ -107,7 +107,7 @@
"livekit-client": "^2.18.1",
"lodash-es": "^4.17.21",
"loglevel": "^1.9.1",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#develop",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#3f05dc4d104a3dfc753c5e3549a18c74fab4b076",
"matrix-widget-api": "^1.18.0",
"node-stdlib-browser": "^1.3.1",
"normalize.css": "^8.0.1",
@@ -1,64 +0,0 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user.ts";
import { TestHelpers } from "./test-helpers.ts";
widgetTest.skip(
({ browserName }) => browserName === "firefox",
"Widget specs only run on Chromium, see simple-create.spec.ts",
);
widgetTest(
"Switching rooms away from a crashed call dismisses it for good",
async ({ asWidget }) => {
test.slow();
const { brooks } = asWidget;
const page = brooks.page;
const call = page.locator('iframe[title="Element Call"]');
const callFrame = call.contentFrame();
// Somewhere to switch to once the call has crashed
await TestHelpers.createRoom("Other Room", page);
await TestHelpers.switchToRoomNamed(page, "Welcome Room");
await TestHelpers.startCallInCurrentRoom(page);
// A dev server compiles the widget on first load
await expect(callFrame.getByTestId("lobby_joinCall")).toBeVisible({
timeout: 30000,
});
await TestHelpers.joinCallFromLobby(page);
await expect(
callFrame.getByRole("button", { name: "End call" }),
).toBeVisible();
// Crash the call from its developer settings
await callFrame.getByRole("button", { name: "Settings" }).click();
await callFrame.getByRole("tab", { name: "Preferences" }).click();
await callFrame.getByText("Developer mode", { exact: true }).check();
await callFrame.getByRole("tab", { name: "Developer" }).click();
await callFrame.getByRole("button", { name: "Throw test error" }).click();
await expect(callFrame.getByText("Something went wrong")).toBeVisible();
// Showing the call collapses the room list
await TestHelpers.expandRoomList(page);
await TestHelpers.switchToRoomNamed(page, "Other Room");
await expect(
page.getByRole("banner").getByRole("heading", { name: "Other Room" }),
).toBeVisible();
// The crashed call is gone and stays gone: no restart, no reconnect
await expect(call).toHaveCount(0);
await page.waitForTimeout(5000);
await expect(call).toHaveCount(0);
await expect(
page.getByRole("banner").getByRole("heading", { name: "Other Room" }),
).toBeVisible();
},
);
+6 -2
View File
@@ -159,7 +159,7 @@ export class TestHelpers {
public static async dismissStartupToasts(page: Page): Promise<void> {
const expectedToasts = [
{ title: "Failed to load service worker", button: "OK" },
{ title: "Back up your chats", button: "Dismiss" },
{ title: "Back up your chats", button: "Not now" },
{ title: "Turn on key storage", button: "Dismiss" },
{ title: "Element does not support this browser", button: "Dismiss" },
];
@@ -178,7 +178,11 @@ export class TestHelpers {
);
if (toastConfig) {
await toast.getByRole("button", { name: toastConfig.button }).click();
// Bounded so that a renamed button leaves the toast in place and the
// next step reports it, instead of hanging until the test timeout.
await toast
.getByRole("button", { name: toastConfig.button })
.click({ timeout: 5000 });
} else {
// Unknown toast. We don't want to act on unknown toasts
break;
+2 -6
View File
@@ -66,13 +66,9 @@ widgetTest(
await TestHelpers.expectVisibleVideoCount(frame, 2);
}
// Element Web does not expose the MSC4195 endpoints to widgets yet, so
// neither side reached the CS API. The host published through the JWT
// service's legacy endpoint and subscribed to the guest through its
// default one; the guest published the same way and fell back to the
// default endpoint for the host's transport.
// Element Web used for Playwright does support MSC4195 for widgets.
expect(hostFlows).toEqual(new Set(["service-legacy", "service-default"]));
expect(guestFlows).toEqual(new Set(["service-legacy", "service-default"]));
expect(guestFlows).toEqual(new Set(["service-legacy", "cs-api"]));
},
);
+541 -270
View File
File diff suppressed because it is too large Load Diff
+6 -1
View File
@@ -101,6 +101,8 @@ LiveKit participants.
│ create ───▶│ │ │
│ │ GET rtc/transports (session) ────────▶│ │
│ │◀── first transport of the backend's type │
│ │ (none, LiveKit only: a member's SFU │ │
│ │ that lets us publish, get_token) ─▶│ │
│ │ prepareLocalTransport (backend): │ │
│ │ publisher token: CS API get_token, │ │
│ │ or OpenID token then /sfu/get ─────▶│ │
@@ -556,7 +558,10 @@ export interface MediaBackend {
```
What stays on the client's side: discovery of the local transport from the
homeserver's list (a homeserver endpoint, so not a backend's), the membership
homeserver's list (a homeserver endpoint, so not a backend's) with, for the
LiveKit backend only, a last resort of another member's SFU that lets us
publish (`findRemotePublishLivekitTransport`, wired in by `RTCParticipation` rather
than by the backend interface, since it is an MSC4195 matter), the membership
state machine in `LocalMembership.ts` and its combination with
`local.connectionState$` into `state$`, the choice of delayed leave timings,
mapping a data packet's sender to a member, and the transport registry, which keys on the raw
+9
View File
@@ -0,0 +1,9 @@
lockfileVersion: '9.0'
settings:
autoInstallPeers: false
excludeLinksFromLockfile: false
importers:
.: {}
+61 -6
View File
@@ -8,9 +8,11 @@ Please see LICENSE in the repository root for full details.
import { type MatrixClient, type Room, SyncState } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import {
type CallMembership,
MatrixRTCSessionEvent,
MembershipManagerEvent,
type Transport,
type UnstableLivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, EMPTY, Subject } from "rxjs";
import { describe, expect, it, type Mock, vi } from "vitest";
@@ -38,13 +40,32 @@ import { constant } from "./reactive/Behavior";
import { ObservableScope } from "./reactive/ObservableScope";
import { createRTCParticipation } from "./RTCParticipation";
import { transportId } from "./matrixrtc/Transports";
import { flushPromises, mockEmitter, MockRTCSession } from "./utils/test";
import {
flushPromises,
mockEmitter,
mockRtcMembership,
MockRTCSession,
} from "./utils/test";
const transport: Transport = {
const transport: UnstableLivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
};
const transportWithAlias: Transport = {
livekit_alias: "!room:example.org",
...transport,
};
const userId = "@alice:example.org";
/** A token with the publish grant, as a homeserver issues it for its own SFU. */
const publisherJWT = [
{},
{ sub: "identity", video: { room: "room", canPublish: true } },
{},
]
.map((d) => global.btoa(JSON.stringify(d)))
.join(".");
const deviceId = "AAAA";
const legacyMemberId = `${userId}:${deviceId}`;
@@ -91,6 +112,13 @@ interface FakeBackend {
function setup(
options: Partial<RTCParticipationOptions> = {},
slotOptions: { matrixRTCMode?: MatrixRTCMode } = {},
{
homeserverTransports = [transport],
memberships = [],
}: {
homeserverTransports?: Transport[];
memberships?: CallMembership[];
} = {},
): {
scope: ObservableScope;
participation: ReturnType<typeof createRTCParticipation>["participation"];
@@ -103,7 +131,7 @@ function setup(
} {
const scope = new ObservableScope();
const room = { roomId: "!room:example.org" } as Room;
const session = new MockRTCSession(room);
const session = new MockRTCSession(room, memberships);
const mockedSession = session.asMockedSession();
const join = vi.spyOn(session, "join");
const client = {
@@ -112,7 +140,8 @@ function setup(
getDeviceId: () => deviceId,
getDomain: () => "example.org",
getSyncState: () => SyncState.Syncing,
_unstable_getRTCTransports: vi.fn().mockResolvedValue([transport]),
_unstable_getRTCTransports: vi.fn().mockResolvedValue(homeserverTransports),
_unstable_getLivekitToken: vi.fn().mockResolvedValue({ jwt: publisherJWT }),
} as unknown as MatrixClient;
const members$ = new BehaviorSubject<RTCMember[]>([]);
@@ -201,7 +230,7 @@ describe("createRTCParticipation", () => {
await flushPromises();
expect(join).toHaveBeenCalledWith(
{ userId, deviceId, memberId: legacyMemberId },
[transport],
[transportWithAlias],
expect.objectContaining({
manageMediaKeys: true,
unstableSendStickyEvents: false,
@@ -220,6 +249,32 @@ describe("createRTCParticipation", () => {
});
});
it("publishes on another member's SFU where the homeserver advertises none", async () => {
const bobOnSfu = mockRtcMembership("@bob:example.org", "BOB", {
fociPreferred: [{ type: "livekit", url: "wss://sfu.example.org" }],
});
const { join, fake } = setup(
{},
{},
{ homeserverTransports: [], memberships: [bobOnSfu] },
);
fake.prepared.resolve({ canDelegateDelayedLeave: false });
await flushPromises();
await flushPromises();
expect(join).toHaveBeenCalledWith(
expect.anything(),
[
{
type: "livekit",
url: "wss://sfu.example.org",
server_name: "example.org",
},
],
expect.anything(),
);
});
it("publishes only while joined", async () => {
const { participation, session, fake, scope } = setup();
const ended = vi.fn();
@@ -271,7 +326,7 @@ describe("createRTCParticipation", () => {
await flushPromises();
expect(join).toHaveBeenCalledWith(
expect.anything(),
[transport],
[transportWithAlias],
expect.objectContaining({ delayedLeaveEventDelayMs: 7_200_000 }),
);
expect(fake.delegateDelayedLeave).not.toHaveBeenCalled();
+43 -14
View File
@@ -11,6 +11,7 @@ import {
type MatrixRTCSession as JsSdkRTCSession,
MatrixRTCSessionEvent,
MembershipManagerEvent,
type Transport,
} from "matrix-js-sdk/lib/matrixrtc";
import { type IMembershipManager } from "matrix-js-sdk/lib/matrixrtc/IMembershipManager";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
@@ -50,6 +51,7 @@ import {
type MediaKey,
} from "./media-backend/api";
import { createLivekitBackend } from "./media-backend/livekit/LivekitBackend";
import { findRemotePublishLivekitTransport } from "./media-backend/livekit/RemotePublishTransport";
import { createHomeserverConnected$ } from "./matrixrtc/HomeserverConnected";
import { joinJsSdkSession } from "./matrixrtc/joinJsSdkSession";
import {
@@ -130,13 +132,34 @@ export function createRTCParticipation(
logger,
});
// If the backend is LiveKit, we check it for transports as well, as a
// fallback: another member's SFU may let us publish (MSC4195). LiveKit only,
// so it stays out of the backend interface
const transportFromBackends = async (): Promise<Transport | null> =>
backend.transportType === "livekit"
? findRemotePublishLivekitTransport({
client,
// A snapshot: if nobody with a usable SFU is here yet, the user has
// to come back later rather than wait in a half-working lobby
memberships: jsSdkSession.memberships,
ownMembershipIdentity,
roomId: room.roomId,
slotId: jsSdkSession.slotId,
logger,
})
: null;
const preparedTransport$ = from(
discoverLocalTransport(client, backend.transportType, options, logger).then(
async (transport): Promise<PreparedTransport> => ({
transport,
...(await backend.prepareLocalTransport(transport)),
}),
),
discoverLocalTransport(
client,
backend.transportType,
options,
transportFromBackends,
logger,
).then(async (transport): Promise<PreparedTransport> => ({
transport,
...(await backend.prepareLocalTransport(transport)),
})),
);
const localMembership = createLocalMembership$({
@@ -151,14 +174,20 @@ export function createRTCParticipation(
timings.syncDisconnectGracePeriodMs,
),
joinMatrixRTC: (transport, delayedLeave) =>
joinJsSdkSession(jsSdkSession, ownMembershipIdentity, transport, {
encryptMedia: encryptionSystem.kind !== E2eeType.NONE,
matrixRTCMode,
sendNotificationType: options.sendNotificationType,
applicationData: options.applicationData,
timings,
delayedLeave,
}),
joinJsSdkSession(
room.roomId,
jsSdkSession,
ownMembershipIdentity,
transport,
{
encryptMedia: encryptionSystem.kind !== E2eeType.NONE,
matrixRTCMode,
sendNotificationType: options.sendNotificationType,
applicationData: options.applicationData,
timings,
delayedLeave,
},
),
membershipManagerError$: fromEvent(
jsSdkSession,
MatrixRTCSessionEvent.MembershipManagerError,
+36 -1
View File
@@ -20,6 +20,11 @@ const hostTransport: Transport = {
type: "livekit",
livekit_service_url: "https://host.example.org",
};
const bobsTransport: Transport = {
type: "livekit",
url: "wss://sfu.bob.com",
server_name: "bob.com",
};
function client(
advertised: Transport[],
@@ -30,6 +35,12 @@ function client(
};
}
function fromBackends(
transport: Transport | null = null,
): ReturnType<typeof vi.fn<() => Promise<Transport | null>>> {
return vi.fn(async () => Promise.resolve(transport));
}
describe("discoverLocalTransport", () => {
it("takes the transport the host names without asking the homeserver", async () => {
const c = client([homeserverTransport]);
@@ -39,6 +50,7 @@ describe("discoverLocalTransport", () => {
c,
"livekit",
{ transport: hostTransport },
fromBackends(),
logger,
),
).resolves.toBe(hostTransport);
@@ -51,6 +63,7 @@ describe("discoverLocalTransport", () => {
client([homeserverTransport]),
"livekit",
{ fallbackTransport: hostTransport },
fromBackends(),
logger,
),
).resolves.toEqual(homeserverTransport);
@@ -62,13 +75,35 @@ describe("discoverLocalTransport", () => {
client([{ type: "not_livekit" }]),
"livekit",
{ fallbackTransport: hostTransport },
fromBackends(),
logger,
),
).resolves.toEqual(hostTransport);
});
it("prefers a local transport over the backends' without asking them", async () => {
const backends = fromBackends(bobsTransport);
await expect(
discoverLocalTransport(
client([homeserverTransport]),
"livekit",
{},
backends,
logger,
),
).resolves.toEqual(homeserverTransport);
expect(backends).not.toHaveBeenCalled();
});
it("fails naming the homeserver where there is no transport at all", async () => {
const promise = discoverLocalTransport(client([]), "livekit", {}, logger);
const promise = discoverLocalTransport(
client([]),
"livekit",
{},
fromBackends(),
logger,
);
await expect(promise).rejects.toBeInstanceOf(
MatrixRTCTransportMissingError,
+8 -2
View File
@@ -15,7 +15,10 @@ import { TransportAutoDiscovery } from "./TransportAutoDiscovery";
/**
* The transport the local member advertises: the one the host names, else the
* homeserver's first of the type the backend serves, else the host's fallback.
* homeserver's first of the type the backend serves, else the host's fallback,
* else the first the backends can provide, which are asked only then.
*
* @throws {MatrixRTCTransportMissingError} where none of these exists.
*/
export async function discoverLocalTransport(
client: Pick<MatrixClient, "getDomain" | "_unstable_getRTCTransports">,
@@ -24,6 +27,7 @@ export async function discoverLocalTransport(
transport: override,
fallbackTransport,
}: Pick<RTCParticipationOptions, "transport" | "fallbackTransport">,
transportFromBackends: () => Promise<Transport | null>,
logger: Logger,
): Promise<Transport> {
const transport =
@@ -33,7 +37,9 @@ export async function discoverLocalTransport(
transportType,
fallbackTransport,
logger,
}).discoverPreferredTransport());
}).discoverPreferredTransport()) ??
(await transportFromBackends()) ??
null;
if (transport === null)
throw new MatrixRTCTransportMissingError(client.getDomain() ?? "");
return transport;
+35 -6
View File
@@ -6,6 +6,7 @@ Please see LICENSE in the repository root for full details.
*/
import {
type UnstableLivekitTransport,
type MatrixRTCSession,
type Transport,
} from "matrix-js-sdk/lib/matrixrtc";
@@ -15,11 +16,16 @@ import { defaultSessionTimings, MatrixRTCMode } from "../config";
import { ownMemberMock } from "../utils/test";
import { joinJsSdkSession } from "./joinJsSdkSession";
const transport: Transport = {
const transport: UnstableLivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
};
const transportWithAlias: Transport = {
livekit_alias: "!room:example.org",
...transport,
};
/** A session that only records the join; nothing else of it is touched. */
function session(): MatrixRTCSession & {
join: Mock<MatrixRTCSession["join"]>;
@@ -39,7 +45,7 @@ describe("joinJsSdkSession", () => {
keyRotationParticipantLimit: 20,
};
joinJsSdkSession(s, ownMemberMock, transport, {
joinJsSdkSession("!room:example.org", s, ownMemberMock, transport, {
encryptMedia: true,
matrixRTCMode: MatrixRTCMode.Compatibility,
sendNotificationType: "ring",
@@ -51,7 +57,7 @@ describe("joinJsSdkSession", () => {
},
});
expect(s.join).toHaveBeenCalledWith(ownMemberMock, [transport], {
expect(s.join).toHaveBeenCalledWith(ownMemberMock, [transportWithAlias], {
notificationType: "ring",
callIntent: undefined,
applicationData: {},
@@ -72,7 +78,7 @@ describe("joinJsSdkSession", () => {
it("gives up on the network once the delayed leave has probably been sent", () => {
const s = session();
joinJsSdkSession(s, ownMemberMock, transport, {
joinJsSdkSession("!room:example.org", s, ownMemberMock, transport, {
encryptMedia: false,
matrixRTCMode: MatrixRTCMode.Compatibility,
timings: { ...defaultSessionTimings, networkErrorRetryMs: 4_000 },
@@ -81,7 +87,7 @@ describe("joinJsSdkSession", () => {
expect(s.join).toHaveBeenCalledWith(
ownMemberMock,
[transport],
[transportWithAlias],
expect.objectContaining({
manageMediaKeys: false,
maximumNetworkErrorRetryCount: 4,
@@ -92,7 +98,30 @@ describe("joinJsSdkSession", () => {
it("lifts the call intent out of the application data and sends sticky events in Matrix 2.0 mode", () => {
const s = session();
joinJsSdkSession(s, ownMemberMock, transport, {
joinJsSdkSession("!room:example.org", s, ownMemberMock, transport, {
encryptMedia: true,
matrixRTCMode: MatrixRTCMode.Matrix_2_0,
applicationData: { "m.call.intent": "audio", custom: 1 },
timings: defaultSessionTimings,
delayedLeave: defaultSessionTimings.delayedLeave,
});
expect(s.join).toHaveBeenCalledWith(
ownMemberMock,
[transportWithAlias],
expect.objectContaining({
callIntent: "audio",
applicationData: { custom: 1 },
unstableSendStickyEvents: true,
}),
);
});
it("does not add livekit_alias to newer transport format", () => {
const s = session();
const transport = { type: "livekit", url: "wss://lk.example.org" };
joinJsSdkSession("!room:example.org", s, ownMemberMock, transport, {
encryptMedia: true,
matrixRTCMode: MatrixRTCMode.Matrix_2_0,
applicationData: { "m.call.intent": "audio", custom: 1 },
+34 -16
View File
@@ -35,6 +35,7 @@ interface Options {
* @throws If the join could not be started.
*/
export function joinJsSdkSession(
roomId: string,
session: JsSdkRTCSession,
ownMembershipIdentity: CallMembershipIdentityParts,
transport: Transport,
@@ -58,20 +59,37 @@ export function joinJsSdkSession(
timings.syncDisconnectGracePeriodMs,
delayedLeave.delay_ms,
);
session.join(ownMembershipIdentity, [transport], {
notificationType: sendNotificationType,
callIntent,
applicationData: rest,
manageMediaKeys: encryptMedia,
delayedLeaveEventRestartMs: delayedLeave.restart_ms,
delayedLeaveEventDelayMs: delayedLeave.delay_ms,
delayedLeaveEventRestartLocalTimeoutMs: delayedLeave.restart_timeout_ms,
networkErrorRetryMs: timings.networkErrorRetryMs,
makeKeyDelay: timings.waitForKeyRotationMs,
membershipEventExpiryMs: timings.membershipEventExpiryMs,
keyRotationParticipantLimit: timings.keyRotationParticipantLimit,
unstableSendStickyEvents: matrixRTCMode === MatrixRTCMode.Matrix_2_0,
maximumNetworkErrorRetryCount:
Math.ceil(maxWaitMs / timings.networkErrorRetryMs) + 1,
});
session.join(
ownMembershipIdentity,
// TODO: `livekit_alias` should be unused these days, but some Ruma-based
// clients still expect it to be present. Remove from `publishedTransports`
// as soon as https://github.com/ruma/ruma/pull/2642 and
// https://github.com/matrix-org/matrix-rust-sdk/pull/7335 are widely
// deployed.
[
{
...("livekit_service_url" in transport
? { livekit_alias: roomId }
: {}),
...transport,
},
],
{
notificationType: sendNotificationType,
callIntent,
applicationData: rest,
manageMediaKeys: encryptMedia,
delayedLeaveEventRestartMs: delayedLeave.restart_ms,
delayedLeaveEventDelayMs: delayedLeave.delay_ms,
delayedLeaveEventRestartLocalTimeoutMs: delayedLeave.restart_timeout_ms,
networkErrorRetryMs: timings.networkErrorRetryMs,
makeKeyDelay: timings.waitForKeyRotationMs,
membershipEventExpiryMs: timings.membershipEventExpiryMs,
keyRotationParticipantLimit: timings.keyRotationParticipantLimit,
unstableSendStickyEvents: matrixRTCMode === MatrixRTCMode.Matrix_2_0,
maximumNetworkErrorRetryCount:
Math.ceil(maxWaitMs / timings.networkErrorRetryMs) + 1,
},
);
}
@@ -258,6 +258,23 @@ describe("createLivekitBackend", () => {
},
);
it("prepares a remote SFU's transport on its homeserver, where delegation is not possible", async () => {
// Our own homeserver would delegate, so only the SFU being remote can
// explain delegation being skipped
fetchMock.post(homeserverEndpoint, 401);
const remoteTransport: UnstableLivekitTransport = {
type: "livekit",
url: "wss://sfu.sample.com",
server_name: "sample.com",
};
const capabilities = await backend().prepareLocalTransport(remoteTransport);
expect(capabilities).toEqual({ canDelegateDelayedLeave: false });
expect(connections[0].connection.role).toBe("publisher");
expect(connections[0].connection.serverName).toBe("sample.com");
});
it("waits for the publisher connection to hold its token before the join", async () => {
fetchMock.post(homeserverEndpoint, 404);
fetchMock.post(`${exampleServiceUrl}/delegate_delayed_leave`, 404);
@@ -488,6 +505,27 @@ describe("createLivekitBackend", () => {
expect(remote?.connection.transport).toEqual(carlsTransport);
expect(remote?.connection.serverName).toBe("sample.com");
});
it("connects to a remote publisher's transport on the SFU's homeserver", async () => {
const b = await prepared();
// Dave has no SFU of his own and publishes on sample.com's
const remoteTransport: UnstableLivekitTransport = {
type: "livekit",
url: "wss://sfu.sample.com",
server_name: "sample.com",
};
b.mediaFor(testScope(), {
...bob(remoteTransport, ["dave-hashed"]),
userId: "@dave:example.org",
});
await flushPromises();
const remote = connections.find(
(c) => c.connection.role === "subscriber",
);
expect(remote?.connection.transport).toEqual(remoteTransport);
expect(remote?.connection.serverName).toBe("sample.com");
});
});
});
+31 -16
View File
@@ -34,7 +34,7 @@ import {
trackEpoch,
} from "../../reactive/ObservableScope";
import { mapScoped } from "../../utils/mapScoped";
import { serverNameOf } from "../../utils/serverName";
import { serverNameOf, serverNameOfTransport } from "../../utils/serverName";
import {
type BackendConnection,
type MediaBackend,
@@ -146,26 +146,38 @@ export function createLivekitBackend(
logger,
);
const ownServerName = serverNameOf(ownMembershipIdentity.userId);
const prepareLocalTransport = async (
transport: Transport,
): Promise<TransportCapabilities> => {
if (!isUnstableLivekitTransport(transport))
throw new MatrixRTCError(`Not a LiveKit transport: ${transport.type}`);
const locator: TransportLocator = {
const serverName = serverNameOfTransport(
transport,
serverName: serverNameOf(ownMembershipIdentity.userId),
};
// Delegation goes through whichever service authenticates us as a
// publisher, so only that service's support counts
const flow = chooseAuthFlow(transport, "publisher");
const canDelegate =
flow.kind === "openID"
? supportsDelegation(
`${flow.serviceUrl}/delegate_delayed_leave`,
`transport ${flow.serviceUrl}`,
logger,
)
: homeserverSupportsDelegation;
ownMembershipIdentity.userId,
);
const locator: TransportLocator = { transport, serverName };
let canDelegate: Promise<boolean>;
if (serverName !== ownServerName) {
// Our homeserver only takes over leaves for its own SFUs
logger.info(
`Publishing on a remote SFU of ${serverName}, delegation is not possible`,
);
canDelegate = Promise.resolve(false);
} else {
// Delegation goes through whichever service authenticates us as a
// publisher, so only that service's support counts
const flow = chooseAuthFlow(transport, "publisher");
canDelegate =
flow.kind === "openID"
? supportsDelegation(
`${flow.serviceUrl}/delegate_delayed_leave`,
`transport ${flow.serviceUrl}`,
logger,
)
: homeserverSupportsDelegation;
}
localTransport$.next(locator);
// The membership is only sent once the publisher connection holds its
// token, so that the LiveKit room exists before anyone subscribes to it:
@@ -204,7 +216,10 @@ export function createLivekitBackend(
map((transport): TransportLocator | undefined => {
const raw = transport?.raw;
return isUnstableLivekitTransport(raw)
? { transport: raw, serverName: serverNameOf(member.userId) }
? {
transport: raw,
serverName: serverNameOfTransport(raw, member.userId),
}
: undefined;
}),
distinctUntilChanged((a, b) => (a && keyFor(a)) === (b && keyFor(b))),
@@ -0,0 +1,172 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { MatrixError } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { beforeEach, describe, expect, it, type Mocked, vi } from "vitest";
import { mockRtcMembership, ownMemberMock } from "../../utils/test";
import { type ClientGetTokenParts } from "./auth/csApi";
import { findRemotePublishLivekitTransport } from "./RemotePublishTransport";
/** A JWT whose payload states whether the issuing server lets us publish. */
function jwtWithPublishGrant(canPublish: boolean): string {
return [
{}, // header
{ sub: "identity", video: { room: "room", canPublish } }, // payload
{}, // signature
]
.map((d) => global.btoa(JSON.stringify(d)))
.join(".");
}
const bobSfu = { type: "livekit", url: "wss://sfu.bob.com" };
const carolSfu = { type: "livekit", url: "wss://sfu.carol.com" };
const bobsTransport = { ...bobSfu, server_name: "bob.com" };
const carolsTransport = { ...carolSfu, server_name: "carol.com" };
// Bob is the oldest member, Carol joined after him
const bob = mockRtcMembership("@bob:bob.com", "BOB", {
fociPreferred: [bobSfu],
membership: { created_ts: 100 },
});
const carol = mockRtcMembership("@carol:carol.com", "CAROL", {
fociPreferred: [carolSfu],
membership: { created_ts: 200 },
});
// Bob's other device, and Dave who publishes on Bob's SFU remotely
const bobsLaptop = mockRtcMembership("@bob:bob.com", "LAPTOP", {
fociPreferred: [bobSfu],
membership: { created_ts: 150 },
});
const dave = mockRtcMembership("@dave:dave.com", "DAVE", {
fociPreferred: [bobsTransport],
membership: { created_ts: 160 },
});
const legacyErin = mockRtcMembership("@erin:erin.com", "ERIN", {
fociPreferred: [
{ type: "livekit", livekit_service_url: "https://jwt.erin.com" },
],
membership: { created_ts: 50 },
});
const ourselves = mockRtcMembership(
ownMemberMock.userId,
ownMemberMock.deviceId,
{
fociPreferred: [{ type: "livekit", url: "wss://sfu.example.org" }],
membership: { created_ts: 10 },
},
);
const csApiMissing = new MatrixError(
{ errcode: "M_UNRECOGNIZED", error: "" },
404,
);
describe("findRemotePublishLivekitTransport", () => {
let client: Mocked<ClientGetTokenParts>;
const find = async (
memberships: CallMembership[],
): ReturnType<typeof findRemotePublishLivekitTransport> =>
findRemotePublishLivekitTransport({
client,
memberships,
ownMembershipIdentity: ownMemberMock,
roomId: "!room:example.org",
slotId: "m.call#ROOM",
logger,
});
beforeEach(() => {
client = {
_unstable_getLivekitToken: vi
.fn()
.mockResolvedValue({ jwt: jwtWithPublishGrant(true) }),
};
});
it("publishes on the oldest member's SFU with its server_name", async () => {
await expect(find([carol, bob])).resolves.toEqual(bobsTransport);
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
expect(client._unstable_getLivekitToken).toHaveBeenCalledWith({
url: "wss://sfu.bob.com",
server_name: "bob.com",
room_id: "!room:example.org",
slot_id: "m.call#ROOM",
member_id: ownMemberMock.memberId,
});
});
it("asks about an SFU once however many members are on it", async () => {
client._unstable_getLivekitToken.mockResolvedValue({
jwt: jwtWithPublishGrant(false),
});
await expect(find([dave, bobsLaptop, bob])).resolves.toBeNull();
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
});
it("skips our own membership", async () => {
await expect(find([ourselves, bob])).resolves.toEqual(bobsTransport);
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
});
it("ignores members publishing through a legacy JWT service", async () => {
await expect(find([legacyErin, bob])).resolves.toEqual(bobsTransport);
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
});
it("moves on when a homeserver does not let us publish", async () => {
client._unstable_getLivekitToken.mockImplementation(
async ({ server_name }) =>
Promise.resolve({
jwt: jwtWithPublishGrant(server_name === "carol.com"),
}),
);
await expect(find([bob, carol])).resolves.toEqual(carolsTransport);
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(2);
});
it("moves on when a homeserver refuses to issue a token", async () => {
client._unstable_getLivekitToken.mockImplementation(
async ({ server_name }) =>
server_name === "bob.com"
? Promise.reject(
new MatrixError(
{ errcode: "M_FORBIDDEN", error: "Not joined" },
403,
),
)
: Promise.resolve({ jwt: jwtWithPublishGrant(true) }),
);
await expect(find([bob, carol])).resolves.toEqual(carolsTransport);
});
it("finds nothing once all candidates are exhausted", async () => {
client._unstable_getLivekitToken.mockResolvedValue({
jwt: jwtWithPublishGrant(false),
});
await expect(find([bob, carol])).resolves.toBeNull();
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(2);
});
it("gives up immediately if our homeserver lacks the CS API", async () => {
client._unstable_getLivekitToken.mockRejectedValue(csApiMissing);
await expect(find([bob, carol])).resolves.toBeNull();
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
});
it("finds nothing where there are no other members", async () => {
await expect(find([ourselves])).resolves.toBeNull();
expect(client._unstable_getLivekitToken).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,105 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type Logger } from "matrix-js-sdk/lib/logger";
import {
type CallMembership,
isUnstableLivekitTransport,
type Transport,
} from "matrix-js-sdk/lib/matrixrtc";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { isCsApiUnavailable } from "./auth";
import * as csApi from "./auth/csApi";
import { decodeSFUJWTPayload } from "./auth/types";
export interface RemotePublishTransportParams {
client: csApi.ClientGetTokenParts;
/** The members whose SFUs are the candidates. */
memberships: CallMembership[];
ownMembershipIdentity: CallMembershipIdentityParts;
roomId: string;
slotId: string;
logger: Logger;
}
/**
* The SFU of another member that its homeserver lets us publish on, as the
* transport to advertise, or null where none does. Candidates are tried
* oldest membership first, each SFU once, and each is checked by asking for a
* publisher token through our homeserver (MSC4195): a homeserver may restrict
* remote publishers, which shows as a token without the publish grant.
*/
export async function findRemotePublishLivekitTransport({
client,
memberships,
ownMembershipIdentity: own,
roomId,
slotId,
logger,
}: RemotePublishTransportParams): Promise<Transport | null> {
logger.info(
"No local transport available, looking for another member's SFU to publish on",
);
const tried = new Set<string>();
const byAge = [...memberships].sort((a, b) => a.createdTs() - b.createdTs());
for (const membership of byAge) {
if (
membership.userId === own.userId &&
membership.deviceId === own.deviceId
)
continue;
for (const transport of membership.transports) {
// Publishing remotely goes through the client-server API, which a JWT
// service cannot stand in for, so only a `url` qualifies
if (!isUnstableLivekitTransport(transport) || !("url" in transport))
continue;
const { url } = transport;
const serverName = membership.getTransportServerName(transport);
const key = JSON.stringify([url, serverName]);
if (tried.has(key)) continue;
tried.add(key);
try {
const { jwt } = await csApi.getSFUConfig({
client,
membership: own,
url,
serverName,
roomId,
slotId,
});
try {
// The SFU treats an absent canPublish grant as publishing allowed.
if (decodeSFUJWTPayload(jwt).video.canPublish !== false) {
logger.info(`Publishing on remote SFU ${url} of ${serverName}`);
return { type: "livekit", url, server_name: serverName };
}
logger.info(
`${serverName} does not let us publish on ${url}, skipping`,
);
continue;
} catch (e) {
logger.info(
`${serverName}: failed to decode SFUJWT payload: ${e}, skipping`,
);
continue;
}
} catch (e) {
if (isCsApiUnavailable(e)) {
// Without the endpoint no further candidate can work
logger.warn(
`Cannot use the MSC4195 get_token endpoint (${e}), so no remote SFU can be used`,
);
return null;
}
logger.warn(`Failed to get a token for ${url} of ${serverName}:`, e);
continue;
}
}
}
return null;
}
+1 -1
View File
@@ -48,7 +48,7 @@ function isEndpointUnsupported(e: unknown): boolean {
* (MSC4533), which the widget API reports as a plain error. A connection
* error is neither: the endpoint may well exist.
*/
function isCsApiUnavailable(e: unknown): boolean {
export function isCsApiUnavailable(e: unknown): boolean {
if (isEndpointUnsupported(e)) return true;
return !(e instanceof MatrixError) && !(e instanceof ConnectionError);
}
+10 -3
View File
@@ -34,11 +34,13 @@ export interface SFUConfig {
}
/** The parts of the JWT payload the backend reads. */
interface SFUJWTPayload {
export interface SFUJWTPayload {
/** The subject: the participant identity the token carries. */
sub: string;
video: {
room: string;
/** Whether the token lets us publish; a homeserver may deny this to remote publishers. */
canPublish?: boolean;
};
}
@@ -47,8 +49,7 @@ export function extractFullConfigFromToken(sfuConfig: {
url: string;
jwt: string;
}): SFUConfig {
const [, payloadStr] = sfuConfig.jwt.split(".");
const payload = JSON.parse(global.atob(payloadStr)) as SFUJWTPayload;
const payload = decodeSFUJWTPayload(sfuConfig.jwt);
return {
jwt: sfuConfig.jwt,
url: sfuConfig.url,
@@ -56,3 +57,9 @@ export function extractFullConfigFromToken(sfuConfig: {
livekitIdentity: payload.sub,
};
}
/** Decodes the payload of an SFU JWT, without verifying its signature. */
export function decodeSFUJWTPayload(jwt: string): SFUJWTPayload {
const [, payloadStr] = jwt.split(".");
return JSON.parse(global.atob(payloadStr)) as SFUJWTPayload;
}
+16
View File
@@ -5,7 +5,23 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type Transport } from "matrix-js-sdk/lib/matrixrtc";
/** The homeserver a Matrix user id belongs to: everything after the first colon. */
export function serverNameOf(userId: string): string {
return userId.replace(/^.*?:/, "");
}
/**
* The homeserver that operates a transport: the one the transport names, else
* the one of the member advertising it. A member publishing on another
* homeserver's SFU (MSC4195) names that homeserver in `server_name`.
*/
export function serverNameOfTransport(
transport: Transport,
userId: string,
): string {
return typeof transport.server_name === "string"
? transport.server_name
: serverNameOf(userId);
}
+2 -1
View File
@@ -7,6 +7,7 @@
padding-top: env(safe-area-inset-top);
opacity: 1;
transition: opacity 0.15s;
text-align: center;
}
/* Pseudo-element for the gradient background */
@@ -52,7 +53,7 @@
". . ."
"primaryButton title secondaryButton"
". . .";
place-items: center;
place-items: center stretch;
column-gap: var(--cpd-space-2x);
}
+1 -6
View File
@@ -6,12 +6,11 @@ Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, vi } from "vitest";
import { BehaviorSubject } from "rxjs";
import { testScope, mockMuteStates, mockMediaDevices } from "../utils/test";
import { constant } from "../state/Behavior";
import type { CallViewModel } from "../state/CallViewModel/CallViewModel";
import type { Alignment, Layout } from "../state/layout-types";
import type { Layout } from "../state/layout-types";
import type { SpotlightTileViewModel } from "../state/TileViewModel";
import type { DeviceLabel } from "../state/MediaDevices";
import { createCallFooterViewModel } from "./CallFooterViewModel";
@@ -65,10 +64,6 @@ function buildMinimalCallViewModel(layout: Layout): CallViewModel {
const gridLayout: Layout = {
type: "grid",
grid: [],
spotlightAlignment$: new BehaviorSubject<Alignment>({
inline: "end",
block: "end",
}),
setVisibleTiles: (_: number) => {},
};
+69 -2
View File
@@ -5,11 +5,24 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, describe, expect, test } from "vitest";
import { afterEach, describe, expect, test, vi } from "vitest";
import { act, renderHook } from "@testing-library/react";
import { E2eeType } from "@element-hq/matrixrtc-sdk";
import { getKeyForRoom, saveKeyForRoom } from "./sharedKeyManagement";
import {
getKeyForRoom,
saveKeyForRoom,
useRoomEncryptionSystem,
} from "./sharedKeyManagement";
vi.mock("../ClientContext", () => {
const room = { hasEncryptionStateEvent: (): boolean => false };
const client = { getRoom: (): typeof room => room };
return { useClient: (): { client: typeof client } => ({ client }) };
});
const roomId = "!room:example.org";
const otherRoomId = "!other:example.org";
describe("getKeyForRoom", () => {
afterEach(() => {
@@ -33,4 +46,58 @@ describe("getKeyForRoom", () => {
test("has no key to offer for a room it has never seen", () => {
expect(getKeyForRoom(roomId)).toBeNull();
});
test("prefers the stored key when the parameters are for another room", () => {
saveKeyForRoom(roomId, "stored");
window.location.hash = `#?roomId=${encodeURIComponent(otherRoomId)}&password=other-room`;
expect(getKeyForRoom(roomId)).toBe("stored");
});
test("ignores the password in the parameters for another room", () => {
window.location.hash = `#?roomId=${encodeURIComponent(otherRoomId)}&password=other-room`;
expect(getKeyForRoom(roomId)).toBeNull();
});
});
describe("useRoomEncryptionSystem", () => {
afterEach(() => {
window.location.hash = "#";
localStorage.clear();
});
test("picks up a key saved for the room after it rendered", () => {
const { result } = renderHook(() => useRoomEncryptionSystem(roomId));
expect(result.current).toEqual({ kind: E2eeType.NONE });
act(() => saveKeyForRoom(roomId, "stored"));
expect(result.current).toEqual({
kind: E2eeType.SHARED_KEY,
secret: "stored",
});
});
test("stores a key from the parameters under the room's key only", () => {
window.location.hash = `#?roomId=${encodeURIComponent(roomId)}&password=from-the-link`;
renderHook(() => useRoomEncryptionSystem(roomId));
expect(Object.keys(localStorage)).toEqual([`room-shared-key-${roomId}`]);
expect(getKeyForRoom(roomId)).toBe("from-the-link");
});
test("keeps the stored key when the parameters are for another room", () => {
saveKeyForRoom(roomId, "stored");
window.location.hash = `#?roomId=${encodeURIComponent(otherRoomId)}&password=other-room`;
const { result } = renderHook(() => useRoomEncryptionSystem(roomId));
expect(result.current).toEqual({
kind: E2eeType.SHARED_KEY,
secret: "stored",
});
expect(localStorage.getItem(`room-shared-key-${roomId}`)).toBe("stored");
});
});
+11 -10
View File
@@ -59,22 +59,27 @@ const useRoomSharedKey = (
/**
* The shared key for a room, preferring one supplied in the parameters Element
* Call was started with over whatever is in local storage.
* Call was started with over whatever is in local storage. If the parameters
* are for another room, only the stored key is used, because the password in
* the parameters belongs to that other room.
*/
function keyForRoom(
roomId: string,
paramsRoomId: string | null,
password: string | null,
): string | null {
if (roomId !== paramsRoomId)
const storedKey = localStorage.getItem(
getRoomSharedKeyLocalStorageKey(roomId),
);
if (roomId !== paramsRoomId) {
logger.warn(
"requested key for a roomId which is not the current call room id (from the URL)",
roomId,
paramsRoomId,
);
return (
password ?? localStorage.getItem(getRoomSharedKeyLocalStorageKey(roomId))
);
return storedKey;
}
return password ?? storedKey;
}
export function getKeyForRoom(roomId: string): string | null {
@@ -89,11 +94,7 @@ export function useRoomEncryptionSystem(roomId: string): EncryptionSystem {
const { roomId: paramsRoomId, password } = useUrlParams();
const [storedPassword] = useRoomSharedKey(
// TODO: this passes an already-prefixed key where a room ID is expected, so
// the local storage key ends up prefixed twice and never matches what
// saveKeyForRoom writes. Preserved as-is here to keep this commit a pure
// refactor; the reactive read is effectively dead until it is fixed.
getRoomSharedKeyLocalStorageKey(roomId),
roomId,
keyForRoom(roomId, paramsRoomId, password) ?? undefined,
);
+100 -150
View File
@@ -5,28 +5,25 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { act, render, screen } from "@testing-library/react";
import { BehaviorSubject } from "rxjs";
import { describe, expect, it, vi } from "vitest";
import { render } from "@testing-library/react";
import { expect, test, vi } from "vitest";
import { arrangeTiles, type Bounds } from "./CallLayout";
import { type CallLayout } from "./CallLayout";
import { makeGridLayout } from "./GridLayout";
import { makeSpotlightLandscapeLayout } from "./SpotlightLandscapeLayout";
import { makeSpotlightPortraitLayout } from "./SpotlightPortraitLayout";
import { type DragCallback, type LayoutProps } from "./Grid";
import { type LayoutProps } from "./Grid";
import { constant } from "../state/Behavior";
import {
type Alignment,
type GridLayout,
type SpotlightLandscapeLayout,
type SpotlightPortraitLayout,
} from "../state/layout-types";
import { type Alignment } from "../state/layout-types";
import { type UserMediaViewModel } from "../state/media/UserMediaViewModel";
import {
GridTileViewModel,
SpotlightTileViewModel,
type TileViewModel,
} from "../state/TileViewModel";
import { makeOneOnOneMobileLayout } from "./OneOnOneMobileLayout";
import { makeSpotlightExpandedLayout } from "./SpotlightExpandedLayout";
import { makeOneOnOneDesktopLayout } from "./OneOnOneDesktopLayout";
// Grid supplies these hooks to the layouts it renders; on their own the
// layouts have no grid to report to
@@ -36,152 +33,105 @@ vi.mock("./Grid", async (importOriginal) => ({
useVisibleTiles: (): void => {},
}));
describe("grid layout", () => {
it("sizes the tiles from the minimum bounds", () => {
const minBounds$ = new BehaviorSubject<Bounds>({ width: 800, height: 600 });
const { scrolling: Scrolling } = makeGridLayout({ minBounds$ });
render(<Scrolling model={gridModel()} Slot={Slot} />);
const layer = screen.getByTestId("slot-0").parentElement!;
expect(layer.style.width).toBe("800px");
expect(layer.style.getPropertyValue("--width")).toBe(
`${Math.floor(arrangeTiles(800, 600, 3).tileWidth)}px`,
);
const minBounds$ = constant({ width: 800, height: 800 });
act(() => minBounds$.next({ width: 400, height: 300 }));
expect(layer.style.width).toBe("400px");
expect(layer.style.getPropertyValue("--width")).toBe(
`${Math.floor(arrangeTiles(400, 300, 3).tileWidth)}px`,
);
});
it("moves the spotlight only when a drag reaches another corner", () => {
const model = gridModel();
const { fixed: Fixed } = makeGridLayout({ minBounds$: bounds });
render(<Fixed model={model} Slot={Slot} />);
const alignments: Alignment[] = [];
model.spotlightAlignment$.subscribe((a) => alignments.push(a));
const drag = drags.get("spotlight")!;
// Still in the bottom right corner
act(() => drag({ x: 0, y: 0, xRatio: 0.9, yRatio: 0.9 }));
// Over to the bottom left
act(() => drag({ x: 0, y: 0, xRatio: 0.1, yRatio: 0.9 }));
// And a little further, still bottom left
act(() => drag({ x: 0, y: 0, xRatio: 0.2, yRatio: 0.8 }));
// Up to the top left
act(() => drag({ x: 0, y: 0, xRatio: 0.2, yRatio: 0.1 }));
expect(alignments).toEqual([
{ inline: "end", block: "end" },
{ inline: "start", block: "end" },
{ inline: "start", block: "start" },
]);
const slot = screen.getByTestId("slot-spotlight");
expect(slot.getAttribute("data-inline-alignment")).toBe("start");
expect(slot.getAttribute("data-block-alignment")).toBe("start");
});
});
describe("spotlight landscape layout", () => {
it("puts the spotlight in the fixed layer and the grid in the scrolling one", () => {
const { fixed: Fixed, scrolling: Scrolling } = makeSpotlightLandscapeLayout(
{ minBounds$: bounds },
);
const model = landscapeModel();
const { unmount } = render(<Fixed model={model} Slot={Slot} />);
expect(screen.getByTestId("slot-spotlight")).toBeInTheDocument();
unmount();
render(<Scrolling model={model} Slot={Slot} />);
expect(slotIds()).toEqual(model.grid.map((tile) => tile.id));
});
});
describe("spotlight portrait layout", () => {
it("sizes the grid tiles from the minimum width", () => {
const minBounds$ = new BehaviorSubject<Bounds>({ width: 300, height: 700 });
const { scrolling: Scrolling } = makeSpotlightPortraitLayout({
minBounds$,
});
const model = portraitModel();
render(<Scrolling model={model} Slot={Slot} />);
expect(slotIds()).toEqual(model.grid.map((tile) => tile.id));
const layer = screen.getByTestId(`slot-${model.grid[0].id}`).parentElement!
.parentElement!;
expect(layer.style.getPropertyValue("--grid-tile-width")).toBe(
`${Math.floor(arrangeTiles(300, 300, 3).tileWidth)}px`,
);
act(() => minBounds$.next({ width: 500, height: 700 }));
expect(layer.style.getPropertyValue("--grid-tile-width")).toBe(
`${Math.floor(arrangeTiles(500, 500, 3).tileWidth)}px`,
);
});
});
const bounds = constant<Bounds>({ width: 800, height: 600 });
// The layouts only pass tile view models through to their slots
const media = {} as UserMediaViewModel;
const spotlight = new SpotlightTileViewModel(
constant([]),
constant(false),
constant("solid"),
const spotlight = {
vm: new SpotlightTileViewModel(
constant([]),
constant(false),
constant("solid"),
),
alignment$: constant<Alignment>({ inline: "end", block: "end" }),
onDrag: () => {},
};
const grid = Array.from(
{ length: 3 },
() => new GridTileViewModel(constant({} as UserMediaViewModel)),
);
const tiles = (): GridTileViewModel[] =>
Array.from({ length: 3 }, () => new GridTileViewModel(constant(media)));
const pip = {
alignment$: constant<Alignment>({ inline: "end", block: "end" }),
onDrag: () => {},
vm: { id: "pip" } as GridTileViewModel,
size$: constant("sm" as const),
};
function gridModel(): GridLayout {
return {
type: "grid",
spotlight,
grid: tiles(),
spotlightAlignment$: new BehaviorSubject<Alignment>({
inline: "end",
block: "end",
}),
setVisibleTiles: () => {},
};
}
function landscapeModel(): SpotlightLandscapeLayout {
return {
type: "spotlight-landscape",
spotlight,
grid: tiles(),
setVisibleTiles: () => {},
};
}
function portraitModel(): SpotlightPortraitLayout {
return {
type: "spotlight-portrait",
spotlight,
grid: tiles(),
setVisibleTiles: () => {},
};
}
// Slots record their drag callback so a test can drag them, and show the id
// of the tile they hold
const drags = new Map<string, DragCallback | undefined>();
const Slot: LayoutProps<unknown, TileViewModel, HTMLDivElement>["Slot"] = ({
id,
model,
onDrag,
children,
...props
}) => {
drags.set(id, onDrag);
return (
<div data-testid={`slot-${id}`} {...props}>
{model instanceof GridTileViewModel ? model.id : "spotlight"}
</div>
);
};
}) => (
<div data-testid={`slot-${id}`} {...props}>
{children}
</div>
);
function slotIds(): string[] {
return screen
.getAllByTestId(/^slot-/)
.map((slot) => slot.textContent ?? "")
.filter((id) => id !== "spotlight");
function snapshotLayout<Model>(layout: CallLayout<Model>, model: Model): void {
const {
foreground,
scrolling: Scrolling,
fixed: Fixed,
} = layout({ minBounds$ });
const scrollingLayer = <Scrolling model={model} Slot={Slot} />;
const fixedLayer = <Fixed model={model} Slot={Slot} />;
const { container } = render(
foreground === "fixed" ? (
<>
{scrollingLayer}
{fixedLayer}
</>
) : (
<>
{fixedLayer}
{scrollingLayer}
</>
),
);
expect(container).toMatchSnapshot();
}
test("grid layout", () =>
snapshotLayout(makeGridLayout, {
type: "grid",
spotlight,
grid,
setVisibleTiles: () => {},
}));
test("spotlight landscape layout", () =>
snapshotLayout(makeSpotlightLandscapeLayout, {
type: "spotlight-landscape",
spotlight,
grid,
setVisibleTiles: () => {},
}));
test("spotlight portrait layout", () =>
snapshotLayout(makeSpotlightPortraitLayout, {
type: "spotlight-portrait",
spotlight,
grid,
setVisibleTiles: () => {},
}));
test("spotlight expanded layout", () =>
snapshotLayout(makeSpotlightExpandedLayout, {
type: "spotlight-expanded",
spotlight: spotlight.vm,
pip,
}));
test("one-on-one desktop layout", () =>
snapshotLayout(makeOneOnOneDesktopLayout, {
type: "one-on-one-desktop",
spotlight: { id: "spotlight" } as GridTileViewModel,
pip,
}));
test("one-on-one mobile layout", () =>
snapshotLayout(makeOneOnOneMobileLayout, {
type: "one-on-one-mobile",
spotlight: spotlight.vm,
pip,
}));
+12 -29
View File
@@ -40,6 +40,7 @@ import { useMergedRefs } from "../useMergedRefs";
import { TileWrapper } from "./TileWrapper";
import { usePrefersReducedMotion } from "../usePrefersReducedMotion";
import { useInitial } from "../useInitial";
import { type DragCallback } from "../state/layout-types";
const MAX_ANIMATED_TILES = 50; // Capped for performance reasons
@@ -83,7 +84,10 @@ interface DragState {
cursorY: number;
}
interface SlotProps<Model> extends Omit<ComponentProps<"div">, "onDrag"> {
export interface SlotProps<Model> extends Omit<
ComponentProps<"div">,
"onDrag"
> {
id: string;
model: Model;
onDrag?: DragCallback;
@@ -181,27 +185,6 @@ export interface TileProps<Model, R extends HTMLElement> {
model: Model;
}
interface Drag {
/**
* The X coordinate of the dragged tile in grid space.
*/
x: number;
/**
* The Y coordinate of the dragged tile in grid space.
*/
y: number;
/**
* The X coordinate of the dragged tile, as a scalar of the grid width.
*/
xRatio: number;
/**
* The Y coordinate of the dragged tile, as a scalar of the grid height.
*/
yRatio: number;
}
export type DragCallback = (drag: Drag) => void;
interface LayoutMemoProps<
LayoutModel,
TileModel,
@@ -464,13 +447,13 @@ export function Grid<
)
.catch(logger.error);
if (endOfGesture)
callback({
x: tileX,
y: tileY,
xRatio: tileX / (gridBounds.width - tile.width),
yRatio: tileY / (gridBounds.height - tile.height),
});
callback({
x: tileX,
y: tileY,
xRatio: tileX / (gridBounds.width - tile.width),
yRatio: tileY / (gridBounds.height - tile.height),
endOfGesture,
});
};
// Callback for useDrag. We could call useDrag here, but the default
+30 -25
View File
@@ -8,15 +8,21 @@ Please see LICENSE in the repository root for full details.
import {
type CSSProperties,
type ReactNode,
useCallback,
useMemo,
type FC,
type ComponentType,
} from "react";
import { type GridLayout as GridLayoutModel } from "../state/layout-types.ts";
import styles from "./GridLayout.module.css";
import { type CallLayout, arrangeTiles } from "./CallLayout";
import { type DragCallback, useUpdateLayout, useVisibleTiles } from "./Grid";
import { type SlotProps, useUpdateLayout, useVisibleTiles } from "./Grid";
import { useBehavior } from "../useBehavior";
import {
type SpotlightTileViewModel,
type TileViewModel,
} from "../state/TileViewModel.ts";
import { type FloatingTile } from "../state/FloatingTile.ts";
interface GridCSSProperties extends CSSProperties {
"--gap": string;
@@ -24,6 +30,27 @@ interface GridCSSProperties extends CSSProperties {
"--height": string;
}
interface SpotlightSlotProps {
model: FloatingTile & { vm: SpotlightTileViewModel };
Slot: ComponentType<SlotProps<TileViewModel>>;
}
const SpotlightSlot: FC<SpotlightSlotProps> = ({ model, Slot }) => {
useUpdateLayout();
const alignment = useBehavior(model.alignment$);
return (
<Slot
className={styles.slot}
id="spotlight"
model={model.vm}
onDrag={model.onDrag}
data-block-alignment={alignment.block}
data-inline-alignment={alignment.inline}
/>
);
};
/**
* An implementation of the "grid" layout, in which all participants are shown
* together in a scrolling grid.
@@ -37,32 +64,10 @@ export const makeGridLayout: CallLayout<GridLayoutModel> = ({
// lives
fixed: function GridLayoutFixed({ ref, model, Slot }): ReactNode {
useUpdateLayout();
const alignment = useBehavior(model.spotlightAlignment$);
const onDragSpotlight: DragCallback = useCallback(
({ xRatio, yRatio }) => {
const block = yRatio < 0.5 ? "start" : "end";
const inline = xRatio < 0.5 ? "start" : "end";
// A drag reports the same alignment on every move; only a change is
// worth a re-render
const current = model.spotlightAlignment$.value;
if (current.block !== block || current.inline !== inline)
model.spotlightAlignment$.next({ block, inline });
},
[model.spotlightAlignment$],
);
return (
<div ref={ref} className={styles.fixed}>
{model.spotlight && (
<Slot
className={styles.slot}
id="spotlight"
model={model.spotlight}
onDrag={onDragSpotlight}
data-block-alignment={alignment.block}
data-inline-alignment={alignment.inline}
/>
<SpotlightSlot model={model.spotlight} Slot={Slot} />
)}
</div>
);
+6 -15
View File
@@ -6,13 +6,13 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type ReactNode, useCallback, useMemo } from "react";
import { type ReactNode, useMemo } from "react";
import classNames from "classnames";
import { type OneOnOneDesktopLayout as OneOnOneDesktopLayoutModel } from "../state/layout-types.ts";
import { type CallLayout, arrangeTiles } from "./CallLayout";
import styles from "./OneOnOneDesktopLayout.module.css";
import { type DragCallback, useUpdateLayout } from "./Grid";
import { useUpdateLayout } from "./Grid";
import { useBehavior } from "../useBehavior";
/**
@@ -37,21 +37,12 @@ export const makeOneOnOneDesktopLayout: CallLayout<
}): ReactNode {
useUpdateLayout();
const { width, height } = useBehavior(minBounds$);
const pipAlignment = useBehavior(model.pipAlignment$);
const pipAlignment = useBehavior(model.pip.alignment$);
const { tileWidth, tileHeight } = useMemo(
() => arrangeTiles(width, height, 1),
[width, height],
);
const onDragLocalTile: DragCallback = useCallback(
({ xRatio, yRatio }) =>
model.pipAlignment$.next({
block: yRatio < 0.5 ? "start" : "end",
inline: xRatio < 0.5 ? "start" : "end",
}),
[model.pipAlignment$],
);
return (
<div ref={ref} className={styles.layer}>
<Slot
@@ -62,9 +53,9 @@ export const makeOneOnOneDesktopLayout: CallLayout<
>
<Slot
className={classNames(styles.slot, styles.local)}
id={model.pip.id}
model={model.pip}
onDrag={onDragLocalTile}
id={model.pip.vm.id}
model={model.pip.vm}
onDrag={model.pip.onDrag}
data-block-alignment={pipAlignment.block}
data-inline-alignment={pipAlignment.inline}
/>
+32 -24
View File
@@ -6,14 +6,43 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type ReactNode, useCallback } from "react";
import { type ComponentType, type FC, type ReactNode } from "react";
import classNames from "classnames";
import { type OneOnOneMobileLayout as OneOnOneMobileLayoutModel } from "../state/layout-types.ts";
import { type CallLayout } from "./CallLayout";
import styles from "./OneOnOneMobileLayout.module.css";
import { type DragCallback, useUpdateLayout } from "./Grid";
import { type SlotProps, useUpdateLayout } from "./Grid";
import { useBehavior } from "../useBehavior";
import { type FloatingTile } from "../state/FloatingTile.ts";
import {
type GridTileViewModel,
type TileViewModel,
} from "../state/TileViewModel.ts";
import { type Behavior } from "../state/Behavior.ts";
interface PipSlotProps {
model: FloatingTile & { vm: GridTileViewModel; size$: Behavior<"sm" | "lg"> };
Slot: ComponentType<SlotProps<TileViewModel>>;
}
const PipSlot: FC<PipSlotProps> = ({ model, Slot }) => {
useUpdateLayout();
const size = useBehavior(model.size$);
const alignment = useBehavior(model.alignment$);
return (
<Slot
className={classNames(styles.pip)}
id={model.vm.id}
model={model.vm}
onDrag={model.onDrag}
data-size={size}
data-block-alignment={alignment.block}
data-inline-alignment={alignment.inline}
/>
);
};
/**
* An implementation of the "one-on-one" layout for mobile platforms, in which
@@ -44,30 +73,9 @@ export const makeOneOnOneMobileLayout: CallLayout<
Slot,
}): ReactNode {
useUpdateLayout();
const pipSize = useBehavior(model.pipSize$);
const pipAlignment = useBehavior(model.pipAlignment$);
const onDragLocalTile: DragCallback = useCallback(
({ xRatio, yRatio }) =>
model.pipAlignment$.next({
block: yRatio < 0.5 ? "start" : "end",
inline: xRatio < 0.5 ? "start" : "end",
}),
[model.pipAlignment$],
);
return (
<div ref={ref} className={styles.layer}>
{model.pip && (
<Slot
className={classNames(styles.pip)}
id={model.pip.id}
model={model.pip}
onDrag={onDragLocalTile}
data-size={pipSize}
data-block-alignment={pipAlignment.block}
data-inline-alignment={pipAlignment.inline}
/>
)}
{model.pip && <PipSlot model={model.pip} Slot={Slot} />}
</div>
);
},
+29 -24
View File
@@ -5,13 +5,39 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type ReactNode, useCallback } from "react";
import { type ReactNode, type ComponentType, type FC } from "react";
import { type SpotlightExpandedLayout as SpotlightExpandedLayoutModel } from "../state/layout-types.ts";
import { type CallLayout } from "./CallLayout";
import { type DragCallback, useUpdateLayout } from "./Grid";
import { type SlotProps, useUpdateLayout } from "./Grid";
import styles from "./SpotlightExpandedLayout.module.css";
import { useBehavior } from "../useBehavior";
import { type FloatingTile } from "../state/FloatingTile.ts";
import {
type GridTileViewModel,
type TileViewModel,
} from "../state/TileViewModel.ts";
interface PipSlotProps {
model: FloatingTile & { vm: GridTileViewModel };
Slot: ComponentType<SlotProps<TileViewModel>>;
}
const PipSlot: FC<PipSlotProps> = ({ model, Slot }) => {
useUpdateLayout();
const alignment = useBehavior(model.alignment$);
return (
<Slot
className={styles.pip}
id={model.vm.id}
model={model.vm}
onDrag={model.onDrag}
data-block-alignment={alignment.block}
data-inline-alignment={alignment.inline}
/>
);
};
/**
* An implementation of the "expanded spotlight" layout, in which the spotlight
@@ -28,7 +54,6 @@ export const makeSpotlightExpandedLayout: CallLayout<
Slot,
}): ReactNode {
useUpdateLayout();
return (
<div ref={ref} className={styles.layer}>
<Slot
@@ -46,29 +71,9 @@ export const makeSpotlightExpandedLayout: CallLayout<
Slot,
}): ReactNode {
useUpdateLayout();
const pipAlignment = useBehavior(model.pipAlignment$);
const onDragPip: DragCallback = useCallback(
({ xRatio, yRatio }) =>
model.pipAlignment$.next({
block: yRatio < 0.5 ? "start" : "end",
inline: xRatio < 0.5 ? "start" : "end",
}),
[model.pipAlignment$],
);
return (
<div ref={ref} className={styles.layer}>
{model.pip && (
<Slot
className={styles.pip}
id={model.pip.id}
model={model.pip}
onDrag={onDragPip}
data-block-alignment={pipAlignment.block}
data-inline-alignment={pipAlignment.inline}
/>
)}
{model.pip && <PipSlot model={model.pip} Slot={Slot} />}
</div>
);
},
+2 -2
View File
@@ -40,7 +40,7 @@ export const makeSpotlightLandscapeLayout: CallLayout<
<Slot
className={styles.slot}
id="spotlight"
model={model.spotlight}
model={model.spotlight.vm}
/>
</div>
<div className={styles.grid} />
@@ -59,7 +59,7 @@ export const makeSpotlightLandscapeLayout: CallLayout<
return (
<div ref={ref} className={styles.layer}>
<SpotlightSlot media$={model.spotlight.media$} />
<SpotlightSlot media$={model.spotlight.vm.media$} />
<div className={styles.grid}>
{model.grid.map((m) => (
<Slot key={m.id} className={styles.slot} id={m.id} model={m} />
+2 -2
View File
@@ -43,7 +43,7 @@ export const makeSpotlightPortraitLayout: CallLayout<
<Slot
className={styles.slot}
id="spotlight"
model={model.spotlight}
model={model.spotlight.vm}
/>
</div>
</div>
@@ -65,7 +65,7 @@ export const makeSpotlightPortraitLayout: CallLayout<
width,
model.grid.length,
);
const withIndicators = useBehavior(model.spotlight.media$).length > 1;
const withIndicators = useBehavior(model.spotlight.vm.media$).length > 1;
return (
<div
@@ -0,0 +1,186 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
exports[`grid layout 1`] = `
<div>
<div
class="_scrolling_e8555d"
style="width: 800px; --gap: 20px; --width: 370px; --height: 277px;"
>
<div
class="_slot_e8555d"
data-testid="slot-0"
/>
<div
class="_slot_e8555d"
data-testid="slot-1"
/>
<div
class="_slot_e8555d"
data-testid="slot-2"
/>
</div>
<div
class="_fixed_e8555d"
>
<div
class="_slot_e8555d"
data-block-alignment="end"
data-inline-alignment="end"
data-testid="slot-spotlight"
/>
</div>
</div>
`;
exports[`one-on-one desktop layout 1`] = `
<div>
<div
class="_layer_185815"
>
<div
class="_container_185815"
data-testid="slot-spotlight"
style="width: 760px; height: 570px;"
>
<div
class="_slot_185815 _local_185815"
data-block-alignment="end"
data-inline-alignment="end"
data-testid="slot-pip"
/>
</div>
</div>
<div />
</div>
`;
exports[`one-on-one mobile layout 1`] = `
<div>
<div
class="_layer_c81c30"
>
<div
class="_spotlight_c81c30"
data-testid="slot-spotlight"
/>
</div>
<div
class="_layer_c81c30"
>
<div
class="_pip_c81c30"
data-block-alignment="end"
data-inline-alignment="end"
data-size="sm"
data-testid="slot-pip"
/>
</div>
</div>
`;
exports[`spotlight expanded layout 1`] = `
<div>
<div
class="_layer_e70191"
>
<div
class="_spotlight_e70191"
data-testid="slot-spotlight"
/>
</div>
<div
class="_layer_e70191"
>
<div
class="_pip_e70191"
data-block-alignment="end"
data-inline-alignment="end"
data-testid="slot-pip"
/>
</div>
</div>
`;
exports[`spotlight landscape layout 1`] = `
<div>
<div
class="_layer_29809c"
>
<div
class="_spotlight_29809c"
>
<div
class="_slot_29809c"
data-testid="slot-spotlight"
/>
</div>
<div
class="_grid_29809c"
/>
</div>
<div
class="_layer_29809c"
>
<div
class="_spotlight_29809c"
/>
<div
class="_grid_29809c"
>
<div
class="_slot_29809c"
data-testid="slot-0"
/>
<div
class="_slot_29809c"
data-testid="slot-1"
/>
<div
class="_slot_29809c"
data-testid="slot-2"
/>
</div>
</div>
</div>
`;
exports[`spotlight portrait layout 1`] = `
<div>
<div
class="_layer_065bbc"
style="--grid-gap: 20px; --grid-tile-width: 370px; --grid-tile-height: 277px;"
>
<div
class="_spotlight_065bbc"
/>
<div
class="_grid_065bbc"
>
<div
class="_slot_065bbc"
data-testid="slot-0"
/>
<div
class="_slot_065bbc"
data-testid="slot-1"
/>
<div
class="_slot_065bbc"
data-testid="slot-2"
/>
</div>
</div>
<div
class="_layer_065bbc"
>
<div
class="_spotlight_065bbc"
>
<div
class="_slot_065bbc"
data-testid="slot-spotlight"
/>
</div>
</div>
</div>
`;
@@ -1228,7 +1228,7 @@ exports[`should have a close button when the host can dismiss us 1`] = `
Call is not supported
</h1>
<p>
The server is not configured to work with Element Call. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
The server is not configured to work with Element Call and no other call participant provides a usable transport. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
</p>
<button
class="_button_1nw83_8"
@@ -1381,7 +1381,7 @@ exports[`should render the error page with link back to home 1`] = `
Call is not supported
</h1>
<p>
The server is not configured to work with Element Call. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
The server is not configured to work with Element Call and no other call participant provides a usable transport. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
</p>
<button
class="_button_1nw83_8"
@@ -1534,7 +1534,7 @@ exports[`should report correct error for 'Call is not supported' 1`] = `
Call is not supported
</h1>
<p>
The server is not configured to work with Element Call. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
The server is not configured to work with Element Call and no other call participant provides a usable transport. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
</p>
<button
class="_button_1nw83_8"
@@ -30,9 +30,6 @@ import {
} from "@element-hq/matrixrtc-sdk";
import type * as MatrixRTCSdk from "@element-hq/matrixrtc-sdk";
import { exampleTransport, mockConfig } from "../utils/test";
import { GroupCallErrorBoundary } from "../room/GroupCallErrorBoundary";
import { LeaveToHomeProvider } from "../LeaveToHomeContext";
import { ElementCallError, ErrorCode } from "../utils/errors";
// Mock url params hook to avoid environment-dependent snapshot churn.
vi.mock("../UrlParams", () => ({
@@ -496,38 +493,4 @@ describe("DeveloperSettingsTab", () => {
);
});
});
describe("throw test error", () => {
it("throws an ElementCallError that the error boundary shows", async () => {
const user = userEvent.setup();
const client = createMockMatrixClient();
const onError = vi.fn();
render(
<LeaveToHomeProvider value={vi.fn()}>
<GroupCallErrorBoundary
onError={onError}
recoveryActionHandler={vi.fn()}
>
<TooltipProvider>
<DeveloperSettingsTab
client={client}
env={{} as unknown as ImportMetaEnv}
vm={outOfCallDeveloperSettingsTabViewModel}
/>
</TooltipProvider>
</GroupCallErrorBoundary>
</LeaveToHomeProvider>,
);
await user.click(
screen.getByRole("button", { name: "Throw test error" }),
);
expect(await screen.findByText("Something went wrong")).toBeVisible();
expect(onError).toHaveBeenCalledOnce();
const error = onError.mock.calls[0][0] as unknown;
expect(error).toBeInstanceOf(ElementCallError);
expect((error as ElementCallError).code).toBe(ErrorCode.UNKNOWN_ERROR);
});
});
});
+1 -23
View File
@@ -22,7 +22,6 @@ import {
import { logger } from "matrix-js-sdk/lib/logger";
import { type UnstableLivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import {
Button,
EditInPlace,
ErrorMessage,
Root as Form,
@@ -73,7 +72,6 @@ import {
} from "@element-hq/matrixrtc-sdk";
import { useBehavior } from "../useBehavior";
import { type ViewModel } from "../state/ViewModel.ts";
import { type ElementCallError, UnknownCallError } from "../utils/errors.ts";
/**
* The state of MatrixRTC's media key rotation.
@@ -191,22 +189,6 @@ export const DeveloperSettingsTab: FC<Props> = ({
const urlParams = useUrlParams();
// An error thrown from a click handler never reaches an error boundary, so
// the button only records it and the next render throws it instead.
const [errorToThrow, setErrorToThrow] = useState<ElementCallError | null>(
null,
);
const throwTestError = useCallback(
() =>
setErrorToThrow(
new UnknownCallError(
new Error("Test error thrown from the developer settings"),
),
),
[],
);
if (errorToThrow !== null) throw errorToThrow;
const MediaQualitySettings: React.FC<{
id: string;
header: string;
@@ -535,7 +517,7 @@ export const DeveloperSettingsTab: FC<Props> = ({
client,
membership: { userId, deviceId, memberId: "" },
transport,
serverName,
serverName: transport.server_name ?? serverName,
roomId,
slotId: "org.example.fake-slot#room",
role: "publisher",
@@ -667,10 +649,6 @@ export const DeveloperSettingsTab: FC<Props> = ({
<Separator />
<AudioProcessingSettings />
<Separator />
<Button kind="secondary" destructive onClick={throwTestError}>
{t("developer_mode.throw_error_button")}
</Button>
<Separator />
<p>{t("developer_mode.environment_variables")}</p>
<pre className={styles.pre}>{JSON.stringify(env, null, 2)}</pre>
<p>{t("developer_mode.url_params")}</p>
@@ -635,21 +635,6 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
data-orientation="horizontal"
role="separator"
/>
<button
class="_button_1nw83_8 _destructive_1nw83_110"
data-kind="secondary"
data-size="lg"
role="button"
tabindex="0"
>
Throw test error
</button>
<div
class="_separator_13qwf_8"
data-kind="primary"
data-orientation="horizontal"
role="separator"
/>
<p>
Environment variables
</p>
+10 -10
View File
@@ -155,7 +155,7 @@ function summarizeLayout$(l$: Observable<Layout>): Observable<LayoutSummary> {
case "grid":
return combineLatest(
[
l.spotlight?.media$ ?? constant(undefined),
l.spotlight?.vm.media$ ?? constant(undefined),
...l.grid.map((vm) => vm.media$),
],
(spotlight, ...grid) => ({
@@ -167,7 +167,7 @@ function summarizeLayout$(l$: Observable<Layout>): Observable<LayoutSummary> {
case "spotlight-landscape":
case "spotlight-portrait":
return combineLatest(
[l.spotlight.media$, ...l.grid.map((vm) => vm.media$)],
[l.spotlight.vm.media$, ...l.grid.map((vm) => vm.media$)],
(spotlight, ...grid) => ({
type: l.type,
spotlight: spotlight.map((vm) => vm.id),
@@ -176,7 +176,7 @@ function summarizeLayout$(l$: Observable<Layout>): Observable<LayoutSummary> {
);
case "spotlight-expanded":
return combineLatest(
[l.spotlight.media$, l.pip?.media$ ?? constant(undefined)],
[l.spotlight.media$, l.pip?.vm.media$ ?? constant(undefined)],
(spotlight, pip) => ({
type: l.type,
spotlight: spotlight.map((vm) => vm.id),
@@ -185,7 +185,7 @@ function summarizeLayout$(l$: Observable<Layout>): Observable<LayoutSummary> {
);
case "one-on-one-desktop":
return combineLatest(
[l.spotlight.media$, l.pip.media$],
[l.spotlight.media$, l.pip.vm.media$],
(spotlight, pip) => ({
type: l.type,
spotlight: spotlight.id,
@@ -196,8 +196,8 @@ function summarizeLayout$(l$: Observable<Layout>): Observable<LayoutSummary> {
return combineLatest(
[
l.spotlight.media$,
l.pip?.media$ ?? constant(undefined),
l.pipSize$,
l.pip?.vm.media$ ?? constant(undefined),
l.pip?.size$ ?? constant(undefined),
],
(spotlight, pip, pipSize) => ({
type: l.type,
@@ -506,7 +506,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
type: "one-on-one-mobile",
spotlight: [`${aliceId}:0`],
pip: undefined,
pipSize: "lg",
pipSize: undefined,
},
b: {
type: "one-on-one-mobile",
@@ -544,7 +544,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
type: "one-on-one-mobile",
spotlight: [`${aliceId}:0`],
pip: undefined,
pipSize: "lg",
pipSize: undefined,
},
});
// It wouldn't be clear whether Alice or Bob is the remote video tile,
@@ -592,7 +592,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
type: "one-on-one-mobile",
spotlight: [`${aliceId}:0`],
pip: undefined,
pipSize: "sm",
pipSize: undefined,
},
b: {
type: "spotlight-landscape",
@@ -791,7 +791,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
type: "one-on-one-mobile",
spotlight: [`${aliceId}:0`],
pip: undefined,
pipSize: "lg",
pipSize: undefined,
},
b: {
// In a larger window, expect the one-on-one desktop layout
+32 -32
View File
@@ -78,14 +78,13 @@ import {
type ReactionInfo,
type ReactionOption,
} from "../../reactions";
import { shallowEquals as shallowArrayEquals } from "../../utils/array";
import { shallowEquals as shallowObjectEquals } from "../../utils/object";
import { shallowEquals as arrayShallowEquals } from "../../utils/array";
import { type MediaDevice, type MediaDevices } from "../MediaDevices";
import { type Handler, type MuteStates } from "../MuteStates";
import { HeaderStyle, type UrlParams } from "../../UrlParams";
import { type HostBridge, nullHostBridge } from "../../HostBridge";
import {
type Alignment,
layoutShallowEquals,
type Layout,
type LayoutMedia,
type WindowMode,
@@ -126,6 +125,7 @@ import {
type RingingMediaViewModel,
} from "../media/RingingMediaViewModel.ts";
import { type GridTileViewModel } from "../TileViewModel.ts";
import { createFloatingTile } from "../FloatingTile.ts";
/** How a call captures and encodes the tracks it publishes. */
export interface CaptureSettings {
@@ -830,7 +830,7 @@ export function createCallViewModel$(
bins.sort(([, bin1], [, bin2]) => bin1 - bin2).map(([m]) => m),
);
}),
distinctUntilChanged(shallowArrayEquals),
distinctUntilChanged(arrayShallowEquals),
),
);
@@ -862,13 +862,18 @@ export function createCallViewModel$(
[ringingMedia$, screenShares$, spotlightSpeaker$, localUserMediaForPip$],
(ringing, screenShares, speaker, localPip) =>
computeSpotlight({ ringing, screenShares, speaker, localPip }),
).pipe(distinctUntilChanged(shallowObjectEquals)),
).pipe(
distinctUntilChanged(
(a, b) =>
arrayShallowEquals(a.spotlight, b.spotlight) && a.pip === b.pip,
),
),
);
const spotlight$ = scope.behavior<MediaViewModel[]>(
spotlightAndPip$.pipe(
map(({ spotlight }) => spotlight),
distinctUntilChanged<MediaViewModel[]>(shallowArrayEquals),
distinctUntilChanged<MediaViewModel[]>(arrayShallowEquals),
),
);
@@ -1029,7 +1034,7 @@ export function createCallViewModel$(
desktop: platform === "desktop",
}),
),
distinctUntilChanged(shallowObjectEquals<LayoutMedia>),
distinctUntilChanged(layoutShallowEquals),
),
);
@@ -1221,32 +1226,28 @@ export function createCallViewModel$(
);
/**
* The alignment of the floating spotlight tile, if present.
* State of the floating spotlight tile, if present.
*/
const spotlightAlignment$ = new BehaviorSubject<Alignment>({
const floatingSpotlightTile = createFloatingTile(scope, {
inline: "end",
block: "end",
});
/**
* The size of the small picture-in-picture tile, if present, when in portrait.
* State of the small picture-in-picture tile, if present, when in landscape.
*/
const portraitPipSize$ = scope.behavior(
showFooter$.pipe(map((showFooter) => (showFooter ? "lg" : "sm"))),
);
/**
* The alignment of the small picture-in-picture tile, if present, when in portrait.
*/
const portraitPipAlignment$ = new BehaviorSubject<Alignment>({
inline: "end",
block: "end",
});
/**
* The alignment of the small picture-in-picture tile, if present, when in landscape.
*/
const landscapePipAlignment$ = new BehaviorSubject<Alignment>({
const landscapePipTile = createFloatingTile(scope, {
inline: "end",
block: "start",
});
/**
* State of the small picture-in-picture tile, if present, when in portrait.
*/
const portraitPipTile = {
...createFloatingTile(scope, { inline: "end", block: "end" }),
size$: scope.behavior(
showFooter$.pipe(map((showFooter) => (showFooter ? "lg" : "sm"))),
),
};
// There is a cyclical dependency here: the layout algorithms want to know
// which tiles are on screen, but to know which tiles are on screen we have to
@@ -1276,7 +1277,7 @@ export function createCallViewModel$(
case "spotlight-portrait":
[layout, newTiles] = gridLikeLayout(
media,
spotlightAlignment$,
floatingSpotlightTile,
visibleTiles,
setVisibleTiles,
prevTiles,
@@ -1286,26 +1287,25 @@ export function createCallViewModel$(
case "spotlight-expanded":
[layout, newTiles] = spotlightExpandedLayout(
media,
landscapePipAlignment$,
landscapePipTile,
prevTiles,
);
break;
case "one-on-one-desktop":
[layout, newTiles] = oneOnOneDesktopLayout(
media,
landscapePipAlignment$,
landscapePipTile,
prevTiles,
);
pip = layout.pip;
pip = layout.pip.vm;
break;
case "one-on-one-mobile":
[layout, newTiles] = oneOnOneMobileLayout(
media,
portraitPipSize$,
portraitPipAlignment$,
portraitPipTile,
prevTiles,
);
pip = layout.pip;
pip = layout.pip?.vm;
break;
case "pip":
[layout, newTiles] = pipLayout(media, prevTiles);
@@ -1330,7 +1330,7 @@ export function createCallViewModel$(
layoutInternals$.pipe(
map(({ layout }) => layout),
// Drop redundant layout updates before they would hit React.
distinctUntilChanged(shallowObjectEquals<Layout>),
distinctUntilChanged(layoutShallowEquals),
),
);
+38
View File
@@ -0,0 +1,38 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { test } from "vitest";
import { withTestScheduler } from "../utils/test";
import { createFloatingTile } from "./FloatingTile";
test("FloatingTile changes alignment on drag", () =>
withTestScheduler(({ scope, expectObservable, schedule }) => {
// Tile starts in the top left
const { alignment$, onDrag } = createFloatingTile(scope, {
inline: "start",
block: "start",
});
// Drag to all four corners, and then repeatedly into the top left.
schedule("-↗↘↙↖↖", {
"↗": () =>
onDrag({ x: 100, y: 0, xRatio: 1, yRatio: 0, endOfGesture: false }),
"↘": () =>
onDrag({ x: 100, y: 100, xRatio: 1, yRatio: 1, endOfGesture: false }),
"↙": () =>
onDrag({ x: 0, y: 100, xRatio: 0, yRatio: 1, endOfGesture: false }),
"↖": () =>
onDrag({ x: 0, y: 0, xRatio: 0, yRatio: 0, endOfGesture: false }),
});
// Alignment should change accordingly. Notably, the alignment should *not*
// update when repeatedly dragged into the same corner, for performance.
expectObservable(alignment$).toBe("↖↗↘↙↖-", {
"↗": { inline: "end", block: "start" },
"↘": { inline: "end", block: "end" },
"↙": { inline: "start", block: "end" },
"↖": { inline: "start", block: "start" },
});
}));
+43
View File
@@ -0,0 +1,43 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { distinctUntilChanged, map, startWith, Subject } from "rxjs";
import { type Behavior } from "./Behavior";
import { type DragCallback, type Alignment, type Drag } from "./layout-types";
import { type ObservableScope } from "./ObservableScope";
/**
* Represents a floating tile which may be dragged to different corners of the
* call interface.
*/
export interface FloatingTile {
alignment$: Behavior<Alignment>;
onDrag: DragCallback;
}
export function createFloatingTile(
scope: ObservableScope,
defaultAlignment: Alignment,
): FloatingTile {
const dragEvents$ = new Subject<Drag>();
return {
alignment$: scope.behavior(
dragEvents$.pipe(
map<Drag, Alignment>(({ xRatio, yRatio }) => ({
inline: xRatio < 0.5 ? "start" : "end",
block: yRatio < 0.5 ? "start" : "end",
})),
startWith(defaultAlignment),
distinctUntilChanged(
(a1, a2) => a1.inline === a2.inline && a1.block === a2.block,
),
),
),
onDrag: (drag) => dragEvents$.next(drag),
};
}
+7 -10
View File
@@ -5,14 +5,9 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type BehaviorSubject } from "rxjs";
import {
type Alignment,
type Layout,
type LayoutMedia,
} from "./layout-types.ts";
import { type Layout, type LayoutMedia } from "./layout-types.ts";
import { type TileStore } from "./TileStore";
import { type FloatingTile } from "./FloatingTile.ts";
export type GridLikeLayoutType =
| "grid"
@@ -25,7 +20,7 @@ export type GridLikeLayoutType =
*/
export function gridLikeLayout(
media: LayoutMedia & { type: GridLikeLayoutType },
spotlightAlignment$: BehaviorSubject<Alignment>,
floatingSpotlightTile: FloatingTile,
visibleTiles: number,
setVisibleTiles: (value: number) => void,
prevTiles: TileStore,
@@ -42,9 +37,11 @@ export function gridLikeLayout(
return [
{
type: media.type,
spotlight: tiles.spotlightTile,
spotlight: tiles.spotlightTile && {
vm: tiles.spotlightTile,
...floatingSpotlightTile,
},
grid: tiles.gridTiles,
spotlightAlignment$,
setVisibleTiles,
} as Layout & { type: GridLikeLayoutType },
tiles,
+3 -6
View File
@@ -6,21 +6,19 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type BehaviorSubject } from "rxjs";
import {
type Alignment,
type OneOnOneDesktopLayout,
type OneOnOneDesktopLayoutMedia,
} from "./layout-types";
import { type TileStore } from "./TileStore";
import { type FloatingTile } from "./FloatingTile";
/**
* Produces a one-on-one desktop layout with the given media.
*/
export function oneOnOneDesktopLayout(
media: OneOnOneDesktopLayoutMedia,
pipAlignment$: BehaviorSubject<Alignment>,
pipTile: FloatingTile,
prevTiles: TileStore,
): [OneOnOneDesktopLayout, TileStore] {
const update = prevTiles.from(2);
@@ -32,8 +30,7 @@ export function oneOnOneDesktopLayout(
{
type: media.type,
spotlight: tiles.gridTilesByMedia.get(media.spotlight)!,
pip: tiles.gridTilesByMedia.get(media.pip)!,
pipAlignment$,
pip: { vm: tiles.gridTilesByMedia.get(media.pip)!, ...pipTile },
},
tiles,
];
+4 -8
View File
@@ -6,23 +6,20 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type BehaviorSubject } from "rxjs";
import {
type Alignment,
type OneOnOneMobileLayout,
type OneOnOneMobileLayoutMedia,
} from "./layout-types";
import { type TileStore } from "./TileStore";
import { type Behavior } from "./Behavior";
import { type FloatingTile } from "./FloatingTile";
/**
* Produces a one-on-one mobile layout with the given media.
*/
export function oneOnOneMobileLayout(
media: OneOnOneMobileLayoutMedia,
pipSize$: Behavior<"sm" | "lg">,
pipAlignment$: BehaviorSubject<Alignment>,
pipTile: FloatingTile & { size$: Behavior<"sm" | "lg"> },
prevTiles: TileStore,
): [OneOnOneMobileLayout, TileStore] {
const update = prevTiles.from(media.pip === undefined ? 0 : 1);
@@ -30,13 +27,12 @@ export function oneOnOneMobileLayout(
if (media.pip !== undefined) update.registerGridTile(media.pip);
const tiles = update.build();
const pipVm = media.pip && tiles.gridTilesByMedia.get(media.pip);
return [
{
type: media.type,
spotlight: tiles.spotlightTile!,
pip: media.pip && tiles.gridTilesByMedia.get(media.pip),
pipSize$,
pipAlignment$,
pip: pipVm && { vm: pipVm, ...pipTile },
},
tiles,
];
+4 -6
View File
@@ -5,21 +5,19 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type BehaviorSubject } from "rxjs";
import {
type Alignment,
type SpotlightExpandedLayout,
type SpotlightExpandedLayoutMedia,
} from "./layout-types";
import { type TileStore } from "./TileStore";
import { type FloatingTile } from "./FloatingTile";
/**
* Produces an expanded spotlight layout with the given media.
*/
export function spotlightExpandedLayout(
media: SpotlightExpandedLayoutMedia,
pipAlignment$: BehaviorSubject<Alignment>,
pipTile: FloatingTile,
prevTiles: TileStore,
): [SpotlightExpandedLayout, TileStore] {
const update = prevTiles.from(1);
@@ -27,12 +25,12 @@ export function spotlightExpandedLayout(
if (media.pip !== undefined) update.registerPipTile(media.pip);
const tiles = update.build();
const pipVm = tiles.gridTiles.at(0);
return [
{
type: media.type,
spotlight: tiles.spotlightTile!,
pip: tiles.gridTiles.at(0),
pipAlignment$,
pip: pipVm && { vm: pipVm, ...pipTile },
},
tiles,
];
+82
View File
@@ -0,0 +1,82 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { test, expect } from "vitest";
import { layoutShallowEquals, type Layout } from "./layout-types";
import {
type SpotlightTileViewModel,
type GridTileViewModel,
} from "./TileViewModel";
import { type FloatingTile } from "./FloatingTile";
const spotlightTile = {} as unknown as SpotlightTileViewModel;
const gridTile = {} as unknown as GridTileViewModel;
const pipTile = {} as unknown as FloatingTile & { vm: GridTileViewModel };
const spotlightExpanded: Layout = {
type: "spotlight-expanded",
spotlight: spotlightTile,
};
const spotlightPortrait: Layout = {
type: "spotlight-portrait",
spotlight: { vm: spotlightTile },
grid: [gridTile],
setVisibleTiles: () => {},
};
test("layoutShallowEquals considers a layout to be equal to its shallow clone", () =>
expect(layoutShallowEquals(spotlightExpanded, { ...spotlightExpanded })).toBe(
true,
));
test("layoutShallowEquals detects a missing key", () => {
expect(
layoutShallowEquals(spotlightExpanded, {
...spotlightExpanded,
pip: pipTile,
}),
).toBe(false);
expect(
layoutShallowEquals(
{ ...spotlightExpanded, pip: pipTile },
spotlightExpanded,
),
).toBe(false);
});
test("layoutShallowEquals considers tile wrappers with equal contents to be equal", () =>
expect(
layoutShallowEquals(spotlightPortrait, {
...spotlightPortrait,
spotlight: { vm: spotlightTile },
}),
).toBe(true));
test("layoutShallowEquals detects tile wrappers with different contents", () =>
expect(
layoutShallowEquals(spotlightPortrait, {
...spotlightPortrait,
spotlight: { vm: {} as unknown as SpotlightTileViewModel },
}),
).toBe(false));
test("layoutShallowEquals considers grid arrays with equal contents to be equal", () =>
expect(
layoutShallowEquals(spotlightPortrait, {
...spotlightPortrait,
grid: [...spotlightPortrait.grid],
}),
).toBe(true));
test("layoutShallowEquals detects grid arrays with different contents", () =>
expect(
layoutShallowEquals(spotlightPortrait, {
...spotlightPortrait,
grid: [...spotlightPortrait.grid, gridTile],
}),
).toBe(false));
+65 -13
View File
@@ -5,8 +5,6 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type BehaviorSubject } from "rxjs";
import { type LocalUserMediaViewModel } from "./media/LocalUserMediaViewModel.ts";
import { type MediaViewModel } from "./media/MediaViewModel.ts";
import { type RingingMediaViewModel } from "./media/RingingMediaViewModel.ts";
@@ -16,6 +14,9 @@ import {
type SpotlightTileViewModel,
} from "./TileViewModel.ts";
import { type Behavior } from "./Behavior.ts";
import { type FloatingTile } from "./FloatingTile.ts";
import { shallowEquals } from "../utils/object.ts";
import { shallowEquals as arrayShallowEquals } from "../utils/array.ts";
/**
* The general shape of the space the call is drawn in. Called a window because
@@ -86,24 +87,48 @@ export interface Alignment {
block: "start" | "end";
}
export interface Drag {
/**
* The X offset of the dragged tile relative to its container.
*/
x: number;
/**
* The Y offset of the dragged tile relative to its container.
*/
y: number;
/**
* The X coordinate of the dragged tile, as a scalar of the container's width.
*/
xRatio: number;
/**
* The Y coordinate of the dragged tile, as a scalar of the container's height.
*/
yRatio: number;
/**
* Whether this event ends the drag gesture.
*/
endOfGesture: boolean;
}
export type DragCallback = (drag: Drag) => void;
export interface GridLayout {
type: "grid";
spotlight?: SpotlightTileViewModel;
spotlight?: FloatingTile & { vm: SpotlightTileViewModel };
grid: GridTileViewModel[];
spotlightAlignment$: BehaviorSubject<Alignment>;
setVisibleTiles: (value: number) => void;
}
export interface SpotlightLandscapeLayout {
type: "spotlight-landscape";
spotlight: SpotlightTileViewModel;
spotlight: { vm: SpotlightTileViewModel };
grid: GridTileViewModel[];
setVisibleTiles: (value: number) => void;
}
export interface SpotlightPortraitLayout {
type: "spotlight-portrait";
spotlight: SpotlightTileViewModel;
spotlight: { vm: SpotlightTileViewModel };
grid: GridTileViewModel[];
setVisibleTiles: (value: number) => void;
}
@@ -111,23 +136,19 @@ export interface SpotlightPortraitLayout {
export interface SpotlightExpandedLayout {
type: "spotlight-expanded";
spotlight: SpotlightTileViewModel;
pip?: GridTileViewModel;
pipAlignment$: BehaviorSubject<Alignment>;
pip?: FloatingTile & { vm: GridTileViewModel };
}
export interface OneOnOneDesktopLayout {
type: "one-on-one-desktop";
spotlight: GridTileViewModel;
pip: GridTileViewModel;
pipAlignment$: BehaviorSubject<Alignment>;
pip: FloatingTile & { vm: GridTileViewModel };
}
export interface OneOnOneMobileLayout {
type: "one-on-one-mobile";
spotlight: SpotlightTileViewModel;
pip?: GridTileViewModel;
pipSize$: Behavior<"sm" | "lg">;
pipAlignment$: BehaviorSubject<Alignment>;
pip?: FloatingTile & { vm: GridTileViewModel; size$: Behavior<"sm" | "lg"> };
}
export interface PipLayout {
@@ -147,3 +168,34 @@ export type Layout =
| OneOnOneDesktopLayout
| OneOnOneMobileLayout
| PipLayout;
/**
* Tests whether the top-level properties, tile wrapper properties, and array
* elements of layout `a` are equal to those of layout `b`. Useful for deduping
* redundant layout updates.
*/
export function layoutShallowEquals<T extends Layout | LayoutMedia>(
a: T,
b: T,
): boolean {
return shallowEquals<T>(a, b, (aValue, bValue) => {
// Special case for tile wrapper objects so we can detect when the
// spotlight or PiP tile is essentially the same
if (
typeof aValue === "object" &&
aValue !== null &&
"vm" in aValue &&
typeof bValue === "object" &&
bValue !== null &&
"vm" in bValue
)
return shallowEquals(aValue, bValue);
// Special case for arrays so we can detect when the grid tile arrays are
// essentially the same
if (Array.isArray(aValue) && Array.isArray(bValue))
return arrayShallowEquals(aValue, bValue);
return aValue === bValue;
});
}
+24 -11
View File
@@ -235,22 +235,35 @@ test("audio and video follow the tracks' mute state", () => {
test("user media polls stream stats only while the setting is on", () => {
onTestFinished(() => showConnectionStats.setValue(false));
withTestScheduler(({ cold, expectObservable, schedule }) => {
const stats = { type: "inbound-rtp" } as RTCInboundRtpStreamStats;
const vm = mockRemoteMedia(rtcMembership, {}, [
mockAudioTrack({
stats$: () => cold("-s", { s: stats }),
}),
mockAudioTrack({ stats$: () => cold("-s", { s: { ...stats } }) }),
]);
schedule("-a-b", {
a() {
showConnectionStats.setValue(true);
},
b() {
showConnectionStats.setValue(false);
},
schedule("-t-f-t", {
t: () => showConnectionStats.setValue(true),
f: () => showConnectionStats.setValue(false),
});
expectObservable(vm.audioStreamStats$).toBe("u-su", {
expectObservable(vm.audioStreamStats$).toBe("u-s---s", {
u: undefined,
s: stats,
});
});
});
test("stream stats can be enabled before creating media", () => {
showConnectionStats.setValue(true);
onTestFinished(() => showConnectionStats.setValue(false));
withTestScheduler(({ cold, expectObservable }) => {
const stats = { type: "inbound-rtp" } as RTCInboundRtpStreamStats;
const vm = mockRemoteMedia(rtcMembership, {}, [
mockAudioTrack({ stats$: () => cold("-s", { s: stats }) }),
]);
expectObservable(vm.audioStreamStats$).toBe("us", {
u: undefined,
s: stats,
});
+4 -3
View File
@@ -16,7 +16,9 @@ import {
BehaviorSubject,
combineLatest,
map,
NEVER,
of,
startWith,
Subject,
switchMap,
} from "rxjs";
@@ -105,10 +107,9 @@ export function createBaseUserMedia(
showConnectionStats.value$,
]).pipe(
switchMap(([track, show]) =>
track !== undefined && show
? track.stats$(statsIntervalMs)
: of(undefined),
track !== undefined && show ? track.stats$(statsIntervalMs) : NEVER,
),
startWith<MediaStreamStats>(undefined),
),
);
+19 -4
View File
@@ -7,6 +7,7 @@ Please see LICENSE in the repository root for full details.
import { test, expect } from "vitest";
import { shallowEquals } from "./object";
import { shallowEquals as arrayShallowEquals } from "./array";
interface Thing {
type: string;
@@ -29,12 +30,26 @@ test("shallowEquals detects a missing key", () => {
test("shallowEquals detects a differing value", () =>
expect(shallowEquals(withArray, { ...withArray, value: 2 })).toBe(false));
test("shallowEquals considers arrays with equal contents to be equal", () =>
const comparator = (aValue: unknown, bValue: unknown): boolean => {
if (Array.isArray(aValue) && Array.isArray(bValue))
return arrayShallowEquals(aValue, bValue);
return aValue === bValue;
};
test("shallowEquals with comparator considers arrays with equal contents to be equal", () =>
expect(
shallowEquals(withArray, { ...withArray, items: [...withArray.items] }),
shallowEquals(
withArray,
{ ...withArray, items: [...withArray.items] },
comparator,
),
).toBe(true));
test("shallowEquals detects arrays with different contents", () =>
test("shallowEquals with comparator detects arrays with different contents", () =>
expect(
shallowEquals(withArray, { ...withArray, items: [...withArray.items, {}] }),
shallowEquals(
withArray,
{ ...withArray, items: [...withArray.items, {}] },
comparator,
),
).toBe(false));
+6 -8
View File
@@ -5,17 +5,17 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { shallowEquals as arrayShallowEquals } from "./array.ts";
/**
* Determine whether two objects are equal by shallow comparison of their
* top-level properties. Array-valued properties are themselves compared
* shallowly, so two objects holding distinct arrays with the same elements are
* considered equal.
* top-level properties, using the given `comparator`.
*/
export function shallowEquals<T extends object>(
a: Readonly<T>,
b: Readonly<T>,
comparator: (aValue: unknown, bValue: unknown) => boolean = (
aValue,
bValue,
) => aValue === bValue,
): boolean {
// If a and b have the same number of keys and every key in a is also in b,
// then they have the same keys.
@@ -29,9 +29,7 @@ export function shallowEquals<T extends object>(
// Now check that they have the same values.
const aValue = (a as any)[key];
const bValue = (b as any)[key];
if (Array.isArray(aValue) && Array.isArray(bValue)) {
if (!arrayShallowEquals(aValue, bValue)) return false;
} else if (aValue !== bValue) return false;
if (!comparator(aValue, bValue)) return false;
}
return true;
+2
View File
@@ -111,6 +111,8 @@ describe("widget", () => {
receiveToDevice: sendRecvToDevice,
turnServers: false,
rtcTransports: true,
rtcLivekitGetToken: true,
rtcLivekitDelegateDelayedLeave: true,
sendDelayedEvents: true,
updateDelayedEvents: true,
sendSticky: true,
+2
View File
@@ -175,6 +175,8 @@ export const initializeWidget = (
receiveToDevice: sendRecvToDevice,
turnServers: false,
rtcTransports: true,
rtcLivekitGetToken: true,
rtcLivekitDelegateDelayedLeave: true,
sendDelayedEvents: true,
updateDelayedEvents: true,
sendSticky: true,