Compare commits

...

99 Commits

Author SHA1 Message Date
Robin
ede29bdf1b Remove MatrixRTC legacy mode
This is the mode in which we sent membership events with the 'oldest membership' transport selection algorithm, which stopped being the default back in version 0.21.0. Users will no longer be able to select this mode in developer settings, and admins will no longer be able to select legacy mode through the config either. The app will still continue to support *receiving* membership events with the 'oldest membership' transport selection algorithm from others, however.
2026-08-26 19:51:12 +02:00
Robin
b5b2bd2193 Merge pull request #4207 from element-hq/legacy-member-events
Remove feature_use_device_session_member_events
2026-08-26 19:49:20 +02:00
Robin
21ca382306 Fix tests 2026-08-26 19:14:19 +02:00
Robin
25e379546e Retry CI 2026-08-26 18:32:14 +02:00
Robin
0443ef55d1 Remove feature_use_device_session_member_events
Support for events with an array of memberships from different devices was removed over a year ago in matrix-js-sdk ffd3c9575e9def576739baf6b1dc329b0db55c0c.
2026-08-26 17:05:38 +02:00
renovate[bot]
465c413691 Update ghcr.io/element-hq/element-web:develop Docker digest to 0332028 (#4204)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-26 09:54:28 +02:00
Johannes Marbach
6cf0e1df51 Merge pull request #4196 from element-hq/renovate/compound
Update dependency @vector-im/compound-design-tokens to v10.2.4
2026-08-24 11:00:01 +02:00
renovate[bot]
bfa20ef747 Update ghcr.io/element-hq/element-web:develop Docker digest to ccee84c (#4195)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-24 09:35:45 +02:00
Johannes Marbach
5955d22c1b Merge pull request #4197 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.22.0
2026-08-24 08:51:19 +02:00
renovate[bot]
ce4426858b Update dependency livekit-client to v2.22.0 2026-08-24 00:37:56 +00:00
renovate[bot]
4ec75ecda4 Update dependency @vector-im/compound-design-tokens to v10.2.4 2026-08-24 00:37:19 +00:00
Timo
b51a33c4e1 Merge pull request #4194 from element-hq/toger5/rtc-mode-documentation
Add docs about currently used matrixRTC mode
2026-08-21 17:44:10 +02:00
Timo K.
c3837464a3 Add docs about currently used matrixRTC mode 2026-08-21 12:04:24 +02:00
Timo
e7372d4c03 Merge pull request #4190 from element-hq/toger5/update-branche-ci-main
`livekit` -> `main` Update branches used in ci and tooling to main
2026-08-21 11:55:32 +02:00
renovate[bot]
69eddd7846 Update ghcr.io/element-hq/element-web:develop Docker digest to ee5b90c (#4193)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-21 07:10:30 +02:00
Timo K.
de3e4c6a78 update branches used in ci and tooling to main 2026-08-20 15:44:13 +02:00
Robin
3abc5a0435 Merge pull request #4185 from element-hq/pip-no-modals
Hide reactions interface in PiP mode
2026-08-20 15:16:22 +02:00
Johannes Marbach
8e20d48b1d Merge pull request #4187 from element-hq/johannes/label-sync-pat
Switch label sync workflow to LABEL_SYNC_GITHUB_TOKEN
2026-08-20 10:05:12 +02:00
Johannes Marbach
ff57c217aa Switch label sync workflow to LABEL_SYNC_GITHUB_TOKEN 2026-08-20 09:21:42 +02:00
renovate[bot]
b60e7452b7 Update ghcr.io/element-hq/element-web:develop Docker digest to a416986 (#4186)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-20 07:14:58 +02:00
Robin
1c03ad335c Hide reactions interface in PiP mode 2026-08-19 17:44:45 +02:00
Robin
87873b726f Merge pull request #4183 from element-hq/msc4519-docs
Update docs to reflect transports endpoint moving to MSC4519
2026-08-19 15:58:28 +02:00
Robin
3448a3a2ef Update docs to reflect transports endpoint moving to MSC4519
This is linked from the 0.24.0 release notes, so even more important for it to be up to date.
2026-08-19 11:48:48 +02:00
Robin
c625fb8845 Merge pull request #4180 from element-hq/background-pngs
Performance: Use PNGs for the background gradients
2026-08-19 10:40:07 +02:00
Robin
cab9698fc7 Undo breakpoint change made during testing 2026-08-19 10:24:26 +02:00
Robin
7410bd8a82 Merge pull request #4181 from element-hq/lobby-video-border
Fix lobby video preview losing its rounded corners in Firefox
2026-08-19 08:54:57 +02:00
renovate[bot]
f17670b731 Update ghcr.io/element-hq/element-web:develop Docker digest to 9ac7cde (#4182)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-19 08:09:58 +02:00
Robin
33a0653ea5 Fix lobby video preview losing its rounded corners in Firefox
Same fix as e1bc4a096b, but for the video preview in the lobby.
2026-08-18 17:49:14 +02:00
Robin
b1c29e9a41 Performance: Use PNGs for the background gradients
It turns out that for a blurry gradient background, low-res raster images look very convincing when scaled up and interpolated. That means that we can rasterize our gradient SVGs for improved paint performance when resizing the window, while still keeping them quite lightweight (just a couple of kilobytes each).
2026-08-18 17:30:12 +02:00
Robin
ad92892063 Merge pull request #4177 from element-hq/ignore-layout-updates
Performance: Ignore redundant layout updates
2026-08-18 15:39:49 +02:00
Robin
1a827e0930 Merge pull request #4178 from element-hq/tiles-debug-performance
Performance: Isolate tile store debug info in its own component
2026-08-18 13:05:02 +02:00
Robin
a1b37bb3c9 Performance: Ignore redundant layout updates
When someone starts or stops speaking, the layout will often be recomputed only to find out that there is ultimately no layout change. We can ignore these redundant updates to avoid re-rendering the InCallView and Grid components, which are relatively slow.

For that specific, common case, this reduces JS CPU usage by as much as 70% in my testing.
2026-08-18 08:42:59 +02:00
renovate[bot]
061d25a54c Update ghcr.io/element-hq/element-web:develop Docker digest to 073e6a4 (#4176)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-18 07:48:41 +02:00
Robin
b554fa63d1 Performance: Isolate tile store debug info in its own component
It's very uncommon to have this debug option enabled, and yet it currently causes the footer to re-render on every layout update, which is a small but avoidable cost.
2026-08-17 16:45:26 +02:00
Robin
b5bcb9aeed Merge pull request #4160 from alexmaras/room-joining/assume-public
Room joining: if join_rule is absent, assume public
2026-08-17 14:40:07 +02:00
Timo
3d3e6dcdcc Merge pull request #4171 from element-hq/toger5/fix-unclose-able-error-screen
Fix: error screen cannot be closed in embedded mode.
2026-08-17 14:16:37 +02:00
Timo K.
43ce68e80a Fix: error screen cannot be closed in embedded mode. 2026-08-17 12:08:30 +02:00
Robin
6d5f698357 Merge pull request #4151 from element-hq/more-more-performance
Performance: Fit video to frame without polling RTP stats
2026-08-17 11:36:37 +02:00
Robin
64a5b2ee5c Clarify how an invalid width and height manifest 2026-08-17 11:19:43 +02:00
Robin
c0d5f2d470 Merge branch 'livekit' into more-more-performance 2026-08-17 11:12:17 +02:00
Robin
4a5141406e Merge pull request #4152 from element-hq/more-more-more-performance
Performance: Disable animations when there are a large number of tiles
2026-08-17 11:11:39 +02:00
Robin
b036d1c0c0 Merge pull request #4169 from element-hq/renovate/major-compound
Update dependency @vector-im/compound-web to v10
2026-08-17 09:25:18 +02:00
Robin
0cfe6cb03d Merge pull request #4168 from element-hq/renovate/livekit-components
Update LiveKit components
2026-08-17 09:21:33 +02:00
renovate[bot]
8be8cf08bf Update ghcr.io/element-hq/element-web:develop Docker digest to abd6473 (#4165)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-17 09:19:40 +02:00
renovate[bot]
9a772131b6 Update dependency @vector-im/compound-web to v10 2026-08-17 00:38:58 +00:00
renovate[bot]
1cdd5d8db0 Update LiveKit components 2026-08-17 00:38:39 +00:00
renovate[bot]
fc9e13c922 Update ghcr.io/element-hq/element-web:develop Docker digest to 6176839 (#4163)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-14 12:49:32 +02:00
Robin
580897730a Merge pull request #4150 from element-hq/more-performance
Performance: Avoid re-rendering context menus so often
2026-08-12 17:30:13 +02:00
Robin
6f7dac31d6 Merge pull request #4153 from element-hq/valere/rtc/remove_well_known_fallback
Remove .well-known transport discovery
2026-08-11 18:24:03 +02:00
Robin
78e4881c37 Enable MatrixRTC transports endpoint in dev and testing configs 2026-08-11 18:03:28 +02:00
Alex Maras
1bb342e9e1 Room joining: if join_rule is absent, assume public 2026-08-11 21:24:38 +08:00
Robin
7ca68fb5bd Merge branch 'livekit' into valere/rtc/remove_well_known_fallback 2026-08-11 12:29:36 +02:00
Timo
72e7e0f406 Merge pull request #3736 from emmick4/livekit
feat: advanced media quality settings UI and config-driven defaults
2026-08-11 12:27:12 +02:00
Robin
94a1d87813 Merge pull request #4158 from element-hq/spotlight-performance
Performance: Avoid re-rendering entire spotlight layout so often
2026-08-11 12:24:42 +02:00
Robin
fee54a09c3 Merge branch 'livekit' into valere/rtc/remove_well_known_fallback 2026-08-11 11:45:26 +02:00
Robin
72a327f12f Merge pull request #4135 from element-hq/valere/widget_rtc_transport_action
Use widget API action to get transports in widget mode
2026-08-11 11:41:35 +02:00
Robin
7f8d8027dc Request capability to get RTC transports over widget API 2026-08-11 11:23:50 +02:00
Ryan Emmick
cbc12b0365 Merge remote-tracking branch 'upstream/livekit' into livekit 2026-08-11 02:50:16 -05:00
Robin
414ae266ab Merge branch 'livekit' into valere/widget_rtc_transport_action 2026-08-11 09:48:22 +02:00
renovate[bot]
ecca5e1e13 Update ghcr.io/element-hq/element-web:develop Docker digest to f25f10f (#4159)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-11 09:25:55 +02:00
Ryan Emmick
1d12e10628 Merge remote-tracking branch 'upstream/livekit' into livekit 2026-08-10 12:12:18 -05:00
Ryan Emmick
89cabf9fa2 test: cover rageshake media quality metadata and preset edge cases 2026-08-10 11:41:26 -05:00
Robin
7ca8059b0f Remove unused access token mocks 2026-08-10 17:10:31 +02:00
Timo K.
29ea90f800 add rageshake metadata and log 2026-08-10 15:48:53 +02:00
Robin
426ea30fe1 Remove all references to .well-known transport advertisement 2026-08-10 15:10:49 +02:00
Robin
7d6a0da6a9 Remove outdated tests 2026-08-10 14:33:14 +02:00
Robin
ea31f073a7 Merge branch 'valere/widget_rtc_transport_action' into valere/rtc/remove_well_known_fallback 2026-08-10 14:32:44 +02:00
Timo K.
b0d08cb0b5 update order in developer settings tab 2026-08-10 14:22:04 +02:00
Timo K.
88e5a5badb More resolution tests. 2026-08-10 14:21:49 +02:00
Robin
296c735500 Merge branch 'livekit' into valere/widget_rtc_transport_action 2026-08-10 13:43:42 +02:00
Robin
341c508a42 Remove outdated tests 2026-08-10 13:22:37 +02:00
Robin
78e585c5f4 Update matrix-js-sdk to support RTC transport discovery over widget API
Pulls in commit 0d8558de2c682a8cd719d385bfa0b5c01f2fb69e from matrix-js-sdk
2026-08-10 13:14:45 +02:00
Robin
81da4c9093 Remove need to approve builds of matrix-js-sdk after every update 2026-08-10 13:13:58 +02:00
Robin
064a88b9c7 Update pnpm to 11.21.0 2026-08-10 13:11:14 +02:00
Robin
14fd8c179b Fix formatting 2026-08-10 12:55:29 +02:00
renovate[bot]
4a146ebada Update ghcr.io/element-hq/element-web:develop Docker digest to 5c9a7f0 (#4157)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-08-10 11:41:51 +02:00
Ryan Emmick
662259154a Merge remote-tracking branch 'upstream/livekit' into livekit 2026-08-07 03:16:12 -05:00
Ryan Emmick
3ace4ab5a3 Merge remote-tracking branch 'upstream/livekit' into livekit 2026-08-06 02:53:13 -05:00
Ryan Emmick
87aee1c02c test: update developer settings snapshot after merge 2026-08-05 12:05:41 -05:00
Ryan Emmick
f7cf674152 Merge remote-tracking branch 'upstream/livekit' into livekit 2026-08-05 11:55:38 -05:00
Valere
9d5ca297e3 Remove well-known discovery 2026-08-05 16:12:41 +02:00
Robin
c90d1d8dda Expand GridTile test coverage 2026-08-05 12:08:18 +02:00
Robin
dd59519a85 Performance: Disable animations when there are a large number of tiles
I don't believe this affects the cost of running the useSprings hook, but it at least skips the animation frames that would otherwise follow, so it's worth a try.
2026-08-05 11:49:55 +02:00
Robin
33fc8997f8 Performance: Use a shared timer for polling RTP stats
This is now only relevant in case the user has enabled the developer option to show advanced media statistics, but still an easy performance fix.
2026-08-05 11:18:25 +02:00
Robin
8aead74ab9 Performance: Lower the RTP stats refresh interval
Because we no longer use it for computing the fit to frame setting; it's only for developer tools.
2026-08-05 11:18:22 +02:00
Robin
1e128c4da0 Performance: Fit video to frame without polling RTP stats
It turns out that the timers which repeatedly poll the RTP stats of each video track all run independently of each other and can add up to a small but constant sink of CPU. Meanwhile we can replace these timers with HTMLVideoElement 'resize' event listeners, which is way more efficient and reacts instantly to orientation changes.
2026-08-05 11:18:22 +02:00
Robin
e1df71817a Performance: Avoid re-rendering context menus so often
A change in a tile's speaking indicator could cause its entire tree of context menu components to re-render, which is expensive. Isolating the behavior subscriptions in their own component avoids this.
2026-08-04 17:57:25 +02:00
Valere
ffb6b0a5f6 Use rtc transport action with widgets 2026-07-30 10:43:09 +02:00
Ryan Emmick
9ef3499418 test: update developer settings snapshot after merge 2026-06-11 22:15:05 -05:00
Ryan Emmick
08d9982981 Merge upstream/livekit into branch 2026-06-11 22:11:21 -05:00
Ryan Emmick
79b9335192 fix: update i18n translations and developer settings snapshot
Run `yarn i18n` to extract new translation keys for the media quality
settings, and update the DeveloperSettingsTab snapshot to include the
Camera quality, Screen sharing, and Audio processing sections.
2026-04-23 09:51:01 -05:00
Ryan Emmick
dedf4e158e refactor: move media settings to developer tab, centralize config defaults 2026-04-23 09:49:14 -05:00
Ryan Emmick
8642646fa7 Add unit tests, revert vite base path change
Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
2026-04-23 09:47:58 -05:00
Ryan Emmick
98f63c7172 Clean up: deduplicate settings UI, fix backupCodec, remove dead export
Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
2026-04-23 09:47:58 -05:00
Ryan Emmick
297470ac2e Use relative base path for vite build
Allows Element Call to be served from a subdirectory (e.g. /widgets/element-call/
in Element Web) without breaking dynamic imports for locales, workers, and other
assets that were previously using absolute paths.

Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
2026-04-23 09:47:58 -05:00
Ryan Emmick
b043230680 feat: camera quality settings, audio processing toggles, config-seeded defaults
- Add camera video quality controls (resolution/framerate/bitrate/codec)
  to Settings > Video, mirroring the screen share settings UI
- Add audio processing toggles (echo cancellation, noise suppression,
  auto gain control) to Settings > Audio, replacing URL-param-only controls
- Display raw values inline on all sliders (framerate, bitrate, volume)
- Add config-seeded defaults: config.json media_quality values now seed
  Setting defaults for users who haven't explicitly set preferences
- Camera settings are applied when joining a call via ConnectionFactory

Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
2026-04-23 09:47:58 -05:00
Ryan Emmick
bc38d15b6d fix: replace InputField type="select" with native select elements
InputField only supports input/textarea, not select. Passing option
children caused React to crash rendering children inside a void input.

Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
2026-04-23 09:46:47 -05:00
Ryan Emmick
342b61f633 feat: add user-configurable screen share quality settings UI
Adds a "Screen sharing" section to Settings > Video with controls for:
- Resolution (576p to 4K)
- Framerate (5-60 fps slider)
- Bitrate (0.5-15 Mbps slider)
- Codec (VP8/VP9/H.264/AV1)

Gated behind an "Advanced screen share settings" toggle. When enabled,
settings are passed to LiveKit's setScreenShareEnabled as both capture
constraints and publish options. When disabled, falls back to
config.json media_quality defaults.

Settings are persisted in localStorage via the existing Setting<T>
system. The Slider component is extended with a tooltipFormatter prop
for custom tooltip display.

Inspired by pirosuki's advanced-screen-share-settings branch, but
reimplemented cleanly: settings are read directly in LocalMember.ts
(no signature changes), the existing Slider is extended (no component
duplication), and proper form components are used throughout.

Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
2026-04-23 09:46:47 -05:00
Ryan Emmick
390dc22c96 feat: configurable media quality via config.json
Add a `media_quality` section to config.json that allows self-hosters
to configure video codec, resolution, bitrate, framerate, and simulcast
layers for both camera and screen sharing.

This addresses the long-standing request in #249 for configurable media
quality settings. The LiveKit SDK already supports all of these options;
this change exposes them through the existing config system.

New config.json fields:
- media_quality.video_codec: preferred codec (vp8/vp9/h264/av1)
- media_quality.video: camera resolution, bitrate, framerate, simulcast layers
- media_quality.screen_share: screen share resolution, bitrate, framerate,
  simulcast layers (enables 3+ layer simulcast for screen sharing)

All fields are optional and fall back to the existing defaults (VP8,
720p camera, 1080p screen share) when not specified.

Signed-off-by: Ryan Emmick <ryanemmick4@gmail.com>
2026-04-23 09:46:47 -05:00
83 changed files with 2402 additions and 1929 deletions

View File

@@ -9,7 +9,7 @@ on:
- opened
- labeled
push:
branches: [livekit]
branches: [main]
jobs:
versioning:

View File

@@ -4,12 +4,12 @@ on:
push:
branches:
- livekit
- main
paths:
- .github/labels.yml
- .github/workflows/sync-labels.yml
permissions: {} # We use ELEMENT_BOT_TOKEN instead
permissions: {} # We use LABEL_SYNC_GITHUB_TOKEN instead
jobs:
sync-labels:
@@ -20,4 +20,4 @@ jobs:
DELETE: true
WET: true
secrets:
ELEMENT_BOT_TOKEN: ${{ secrets.ELEMENT_BOT_TOKEN }}
ELEMENT_BOT_TOKEN: ${{ secrets.LABEL_SYNC_GITHUB_TOKEN }}

View File

@@ -2,7 +2,7 @@ name: Test
on:
pull_request: {}
push:
branches: [livekit]
branches: [main]
jobs:
vitest:
name: Run unit tests

View File

@@ -2,7 +2,7 @@ name: Upload translation files to Localazy
on:
push:
branches:
- livekit
- main
paths-ignore:
- ".github/**"

View File

@@ -2,7 +2,7 @@ name: GitHub Actions Security Analysis with zizmor 🌈
on:
push:
branches: ["livekit", "full-mesh"]
branches: ["main", "full-mesh"]
pull_request: {}
permissions: {}

View File

@@ -115,30 +115,33 @@ For more details on the packages, see the
For operating and deploying Element Call on your own server, refer to the
[**Self-Hosting Guide**](./docs/self_hosting.md).
## 🧭 MatrixRTC Backend Discovery and Selection
## MatrixRTC Transports
For proper Element Call operation each site deployment needs a MatrixRTC backend
setup as outlined in the [Self-Hosting Guide](./docs/self_hosting.md). A typical
federated site deployment for three different sites A, B and C is depicted below.
For proper operation of Element Call, each deployment needs to set up a
MatrixRTC transport in the form of a LiveKit server as outlined in the
[Self-Hosting Guide](./docs/self_hosting.md). A typical federated site
deployment for three different sites A, B and C is depicted below.
<p align="center">
<img src="./docs/Federated_Setup.drawio.png" alt="Element Call federated setup">
</p>
### Backend Discovery
### Transport Discovery
The MatrixRTC backend (according to
[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143)) is
announced by the Matrix site's `.well-known/matrix/client` file and discovered
via the `org.matrix.msc4143.rtc_foci` key, e.g.:
Element Call discovers the available MatrixRTC transports (as defined by
[MSC4519](https://github.com/matrix-org/matrix-spec-proposals/pull/4519)) by
hitting the `GET /_matrix/client/unstable/org.matrix.msc4143/rtc/transports`
endpoint of the Client-Server API. An example response:
```json
"org.matrix.msc4143.rtc_foci": [
{
"rtc_transports": [
{
"type": "livekit",
"livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt"
},
]
"type": "livekit",
"livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt"
}
]
}
```
where the format for MatrixRTC using LiveKit backend is defined in
@@ -149,7 +152,7 @@ via `livekit_service_url`.
### Backend Selection
- Each call participant proposes their discovered MatrixRTC backend from
- Each call participant proposes their discovered MatrixRTC transport from
`org.matrix.msc4143.rtc_foci` in their `org.matrix.msc3401.call.member` state event.
- For the **LiveKit** MatrixRTC backend
([MSC4195](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)),
@@ -244,10 +247,9 @@ pnpm backend
> [!NOTE]
> To ensure your local development frontend functions properly, youll need to
> add certificate exceptions in your browser for `https://localhost:3000`,
> `https://matrix-rtc.m.localhost/livekit/jwt/healthz` and
> `https://synapse.m.localhost/.well-known/matrix/client`. This can be either
> done by adding the minimum localhost CA
> add certificate exceptions in your browser for `https://localhost:3000` and
> `https://matrix-rtc.m.localhost/livekit/jwt/healthz`. This can be done either
> by adding the minimum localhost CA
> ([./backend/dev_tls_local-ca.crt](./backend/dev_tls_local-ca.crt)) to your web
> browser's trusted certificates or by simply copying and pasting each URL into
> your browsers address bar and follow the prompts to add the exception.

View File

@@ -40,6 +40,8 @@ experimental_features:
msc4222_enabled: true
# sticky events for MatrixRTC user state
msc4354_enabled: true
# MatrixRTC
msc4143_enabled: true
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no
@@ -54,7 +56,6 @@ enable_registration_without_verification: true
registration_shared_secret: "test_shared_secret_for_local_dev_only"
report_stats: false
serve_server_wellknown: true
# Ratelimiting settings for client actions (registration, login, messaging).
#
@@ -67,3 +68,8 @@ rc_message:
# Currently the heart-beat is every 5 seconds which translates into a rate of 0.2s
per_second: 0.5
burst_count: 30
matrix_rtc:
transports:
- type: livekit
livekit_service_url: https://matrix-rtc.othersite.m.localhost/livekit/jwt

View File

@@ -40,6 +40,8 @@ experimental_features:
msc4222_enabled: true
# sticky events for MatrixRTC user state
msc4354_enabled: true
# MatrixRTC
msc4143_enabled: true
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no
@@ -54,7 +56,6 @@ enable_registration_without_verification: true
registration_shared_secret: "test_shared_secret_for_local_dev_only"
report_stats: false
serve_server_wellknown: true
# Ratelimiting settings for client actions (registration, login, messaging).
#
@@ -67,3 +68,8 @@ rc_message:
# Currently the heart-beat is every 5 seconds which translates into a rate of 0.2s
per_second: 0.5
burst_count: 30
matrix_rtc:
transports:
- type: livekit
livekit_service_url: https://matrix-rtc.m.localhost/livekit/jwt

View File

@@ -1,4 +1,4 @@
# Synapse reverse proxy including .well-known/matrix/client
# Synapse reverse proxy
# domain synapse.m.localhost
server {
listen 80;
@@ -11,18 +11,6 @@ server {
ssl_certificate /root/ssl/cert.pem;
ssl_certificate_key /root/ssl/key.pem;
# well-known config adding rtc_foci backend
# Note well-known is currently not effective due to:
# https://spec.matrix.org/v1.12/client-server-api/#well-known-uri the spec
# says it must be at https://$server_name/... (implied port 443) Hence, we
# currently rely for local development environment on deprecated config.json
# setting for livekit_service_url
location /.well-known/matrix/client {
add_header Access-Control-Allow-Origin *;
return 200 '{"m.homeserver": {"base_url": "https://synapse.m.localhost"}, "org.matrix.msc4143.rtc_foci": [{"type": "livekit", "livekit_service_url": "https://matrix-rtc.m.localhost/livekit/jwt"}]}';
default_type application/json;
}
# Reverse proxy for Matrix Synapse Homeserver
# This is also required for development environment.
# Reason: the lk-jwt-service uses the federation API for the openid token
@@ -44,7 +32,7 @@ server {
}
# Synapse reverse proxy including .well-known/matrix/client
# Synapse reverse proxy
# domain synapse.othersite.m.localhost
server {
listen 80;
@@ -57,18 +45,6 @@ server {
ssl_certificate /root/ssl/cert.pem;
ssl_certificate_key /root/ssl/key.pem;
# well-known config adding rtc_foci backend
# Note well-known is currently not effective due to:
# https://spec.matrix.org/v1.12/client-server-api/#well-known-uri the spec
# says it must be at https://$server_name/... (implied port 443) Hence, we
# currently rely for local development environment on deprecated config.json
# setting for livekit_service_url
location /.well-known/matrix/client {
add_header Access-Control-Allow-Origin *;
return 200 '{"m.homeserver": {"base_url": "https://synapse.othersite.m.localhost"}, "org.matrix.msc4143.rtc_foci": [{"type": "livekit", "livekit_service_url": "https://matrix-rtc.othersite.m.localhost/livekit/jwt"}]}';
default_type application/json;
}
# Reverse proxy for Matrix Synapse Homeserver
# This is also required for development environment.
# Reason: the lk-jwt-service uses the federation API for the openid token

View File

@@ -40,6 +40,8 @@ experimental_features:
msc4222_enabled: true
# sticky events for MatrixRTC user state
msc4354_enabled: true
# MatrixRTC
msc4143_enabled: true
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no
@@ -54,7 +56,6 @@ enable_registration_without_verification: true
registration_shared_secret: "test_shared_secret_for_local_dev_only"
report_stats: false
serve_server_wellknown: true
# Ratelimiting settings for client actions (registration, login, messaging).
#
@@ -84,3 +85,8 @@ rc_login:
rc_registration:
per_second: 10000
burst_count: 10000
matrix_rtc:
transports:
- type: livekit
livekit_service_url: https://matrix-rtc.othersite.m.localhost/livekit/jwt

View File

@@ -40,6 +40,8 @@ experimental_features:
msc4222_enabled: true
# sticky events for MatrixRTC user state
msc4354_enabled: true
# MatrixRTC
msc4143_enabled: true
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no
@@ -54,7 +56,6 @@ enable_registration_without_verification: true
registration_shared_secret: "test_shared_secret_for_local_dev_only"
report_stats: false
serve_server_wellknown: true
# Ratelimiting settings for client actions (registration, login, messaging).
#
@@ -84,3 +85,8 @@ rc_login:
rc_registration:
per_second: 10000
burst_count: 10000
matrix_rtc:
transports:
- type: livekit
livekit_service_url: https://matrix-rtc.m.localhost/livekit/jwt

View File

@@ -5,9 +5,6 @@
"server_name": "synapse.m.localhost"
}
},
"features": {
"feature_use_device_session_member_events": true
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,

View File

@@ -8,9 +8,6 @@
"livekit": {
"livekit_service_url": "https://livekit-jwt.mydomain.com"
},
"features": {
"feature_use_device_session_member_events": true
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_mode": "compatibility",
"matrix_rtc_session": {

View File

@@ -13,7 +13,7 @@ services:
- ./backend/playwright_homeserver-othersite.yaml:/data/cfg/homeserver.yaml:Z
element-web:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:a23cced80588306c66bc7cfa859c591be03ece31f25d0a6a2e1e935943a35c92
image: ghcr.io/element-hq/element-web:develop@sha256:0332028836603e91689053f157847b4deef9342bd19e3195eeb81793b7e4046e
element-web-1:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:a23cced80588306c66bc7cfa859c591be03ece31f25d0a6a2e1e935943a35c92
image: ghcr.io/element-hq/element-web:develop@sha256:0332028836603e91689053f157847b4deef9342bd19e3195eeb81793b7e4046e

View File

@@ -5,5 +5,6 @@ This folder contains documentation for setup, usage, and development of Element
- [Embedded vs standalone mode](./embedded_standalone.md)
- [Url format and parameters](./url_params.md)
- [Global JS controls](./controls.md)
- [MatrixRTC modes](./matrix_rtc_modes.md)
- [Self-Hosting](./self_hosting.md)
- [Developing with linked packages](./linking.md)

48
docs/matrix_rtc_modes.md Normal file
View File

@@ -0,0 +1,48 @@
# MatrixRTC modes
Element Call is in the middle of a transition of how a call session is
represented: from room _state_ events (`org.matrix.msc3401.call.member`) to
_sticky_ events
([MSC4354](https://github.com/matrix-org/matrix-spec-proposals/pull/4354)),
which are a much better fit for the short lived, per-device nature of call
memberships.
Not every homeserver supports sticky events yet. The two MatrixRTC modes
controls whether Element Call uses them.
## The modes
| Mode | Membership events | SFU selection | JWT endpoint |
| --------------- | ----------------- | ------------- | ---------------------------- |
| `compatibility` | state events | multi SFU | legacy |
| `matrix_2_0` | sticky events | multi SFU | Matrix 2.0 (hashed identity) |
**`compatibility`** — multi SFU, but still state events. Use it when the
homeserver does not support sticky events. This is the default.
**`matrix_2_0`** — the target state. Requires a homeserver that advertises
MSC4354 and all clients on v0.17.0 or later. The local membership requests its
token from the Matrix 2.0 JWT endpoint of the
[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)
and identifies the room by a hashed identity instead of a `livekit_alias`.
(Remote memberships always try the new endpoint first and fall back to the
legacy one, so remote participants can be on either.)
## Selecting a mode
Users can choose a mode under **Settings → Developer → MatrixRTC mode**. The
Matrix 2.0 option is disabled if the homeserver does not support sticky events.
A deployment can pin the mode for all its clients in `config.json`, which
disables the Developer Settings choice:
```json
{
"matrix_rtc_mode": "compatibility"
}
```
Valid values are `compatibility` and `matrix_2_0`; an invalid value is
ignored (with a warning) and the user's choice applies. Pinning `matrix_2_0` on a
homeserver without sticky event support makes joining fail with a "sticky events
required" error.

View File

@@ -16,7 +16,7 @@
The following [MSCs](https://github.com/matrix-org/matrix-spec-proposals) are
required for Element Call to work properly:
- **[MSC3266](https://github.com/deepbluev7/matrix-doc/blob/room-summaries/proposals/3266-room-summary.md):
- **[MSC3266](https://github.com/deepbluev7/matrix-doc/blob/room-summaries/proposals/3266-room-summary.md)
Room Summary API**: In Standalone mode Element Call is able to join rooms
over federation using knocking. In this context MSC3266 is required as it
allows to request a room summary of rooms you are not joined. The summary
@@ -29,19 +29,26 @@ required for Element Call to work properly:
signalling. If disabled it is very likely that you end up with stuck calls in
Matrix rooms.
- **[MSC4519](https://github.com/matrix-org/matrix-spec-proposals/blob/travis/msc/voip-transports-registry/proposals/4519-rtc-transports-registry.md)
MatrixRTC Transports Registry**: Defines an endpoint that clients can use to
query the available MatrixRTC transports (i.e. find your LiveKit SFU).
- **[MSC4222](https://github.com/matrix-org/matrix-spec-proposals/blob/erikj/sync_v2_state_after/proposals/4222-sync-v2-state-after.md)
Adding `state_after` to sync v2**: Allow clients to opt-in to a change of the
sync v2 API that allows them to correctly track the state of the room. This is
required by Element Call to track room state reliably.
If you're using [Synapse](https://github.com/element-hq/synapse/) as your
homeserver, you'll need to additionally add the following config items to
`homeserver.yaml` to comply with Element Call:
homeserver, you can configure these features by adding the following entries to
`homeserver.yaml`:
```yaml
experimental_features:
# MSC3266: Room summary API. Used for knocking over federation
msc3266_enabled: true
# MSC4143: MatrixRTC. For historical reasons this flag enables the transports
# endpoint defined in MSC4519.
msc4143_enabled: true
# MSC4222 needed for syncv2 state_after. This allow clients to
# correctly track the state of the room.
msc4222_enabled: true
@@ -61,6 +68,15 @@ rc_delayed_event_mgmt:
# Currently the heart-beat is every 5 seconds which translates into a rate of 0.2Hz
per_second: 1
burst_count: 20
matrix_rtc:
transports:
# The transport you specify will be made available to clients over the
# /_matrix/client/unstable/org.matrix.msc4143/rtc/transports endpoint as
# defined in MSC4519.
- type: livekit
# Replace this with the actual URL of your MatrixRTC Authorization Service
livekit_service_url: https://matrix-rtc.example.com/livekit/jwt
```
As a prerequisite for the
@@ -68,10 +84,10 @@ As a prerequisite for the
make sure that your Synapse server has either a `federation` or `openid`
[listener configured](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#listeners).
### MatrixRTC Backend
### LiveKit backend
In order to **guarantee smooth operation** of Element Call, a MatrixRTC backend is
required for each site deployment.
In order to **guarantee smooth operation** of Element Call, a dedicated LiveKit
backend is required for each site deployment.
![MSC4195 compatible setup](MSC4195_setup.drawio.png)
@@ -165,7 +181,7 @@ Using Haproxy, you can achieve this by:
use_backend mxrtc_auth_backend if is_mxrtc_auth matrixrtc_domain
# Backend
## MatrixRTC backend
## LiveKit backend
backend sfu_backend
server livekit 127.0.0.1:7880
http-request set-path %[path,regsub(^/livekit/sfu/,/)]
@@ -187,44 +203,6 @@ backend mxrtc_auth_backend
```
#### MatrixRTC backend announcement
> [!IMPORTANT]
> As defined in
> [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143),
> the MatrixRTC backend(s) must be announced to the client via your **Matrix site's
> `.well-known/matrix/client`** file (e.g.
> `example.com/.well-known/matrix/client` matching the site deployment example
> from above). The configuration is a list of Foci configs:
```json
"org.matrix.msc4143.rtc_foci": [
{
"type": "livekit",
"livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt"
},
{
"type": "livekit",
"livekit_service_url": "https://matrix-rtc-2.example.com/livekit/jwt"
}
]
```
Make sure this file is served with the correct MIME type (`application/json`).
Additionally, ensure the appropriate CORS headers are set to allow web clients
to access it across origins. For more details, refer to the
[Matrix Client-Server API: 2. Web Browser Clients](https://spec.matrix.org/latest/client-server-api/#web-browser-clients).
```
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: X-Requested-With, Content-Type, Authorization
```
> [!NOTE]
> Most `org.matrix.msc4143.rtc_foci` configurations will only have one entry in
> the array.
## Building Element Call
> [!NOTE]
@@ -278,7 +256,7 @@ runtime. Documentation and default values for `public/config.json` can be found
in [ConfigOptions.ts](../src/config/ConfigOptions.ts).
> [!CAUTION]
> Please note configuring MatrixRTC backend via `config.json` of
> Please note configuring LiveKit backend via `config.json` of
> Element Call is only available for developing and debug purposes. Relying on
> it might break Element Call going forward!

View File

@@ -61,7 +61,7 @@
"crypto_version": "Crypto version: {{version}}",
"custom_livekit_url": {
"current_url": "Currently set to: ",
"from_config": "Currently, no overwrite is set. Url from well-known or config is used.",
"from_config": "Currently, no overwrite is set. Url from config is used.",
"label": "Custom Livekit-url",
"reset": "Reset overwrite",
"save": "Save",
@@ -80,10 +80,6 @@
"description": "Compatible with homeservers that do not support sticky events (but all other EC clients are v0.17.0 or later)",
"label": "Compatibility: state events & multi SFU"
},
"Legacy": {
"description": "Compatible with old versions of EC that do not support multi SFU",
"label": "Legacy: state events & oldest membership SFU"
},
"Matrix_2_0": {
"description": "Compatible only with homservers supporting sticky events and all EC clients v0.17.0 or later",
"label": "Matrix 2.0: sticky events & multi SFU"
@@ -196,13 +192,23 @@
"room_auth_view_ssla_caption": "By clicking \"Join call now\", you agree to our <2>Software and Services License Agreement (SSLA)</2>",
"screenshare_button_label": "Share screen",
"settings": {
"advanced_camera_description": "Configure resolution, framerate, bitrate, and codec for camera video. Changes apply on next call join.",
"advanced_camera_label": "Advanced camera settings",
"advanced_screen_share_description": "Configure resolution, framerate, bitrate, and codec for screen sharing",
"advanced_screen_share_label": "Advanced screen share settings",
"audio_processing_description": "Changes apply on next call join.",
"audio_processing_header": "Audio processing",
"audio_tab": {
"effect_volume_description": "Adjust the volume at which reactions and hand raised effects play.",
"effect_volume_label": "Sound effect volume"
},
"auto_gain_control_label": "Automatic gain control",
"background_blur_header": "Background",
"background_blur_label": "Blur the background of the video",
"bitrate_label": "Bitrate",
"blur_not_supported_by_browser": "(Background blur is not supported by this device.)",
"camera_header": "Camera quality",
"codec_label": "Codec",
"developer_tab_title": "Developer",
"devices": {
"activating": "Activating…",
@@ -218,12 +224,15 @@
"speaker": "Speaker",
"speaker_numbered": "Speaker {{n}}"
},
"echo_cancellation_label": "Echo cancellation",
"feedback_tab_body": "If you are experiencing issues or simply would like to provide some feedback, please send us a short description below.",
"feedback_tab_description_label": "Your feedback",
"feedback_tab_h4": "Submit feedback",
"feedback_tab_send_logs_label": "Include debug logs",
"feedback_tab_thank_you": "Thanks, we received your feedback!",
"feedback_tab_title": "Feedback",
"framerate_label": "Framerate",
"noise_suppression_label": "Noise suppression",
"opt_in_description": "<0></0><1></1>You may withdraw consent by unchecking this box. If you are currently in a call, this setting will take effect at the end of the call.",
"preferences_tab": {
"developer_mode_label": "Developer mode",
@@ -235,7 +244,9 @@
"reactions_show_label": "Show reactions",
"show_hand_raised_timer_description": "Show a timer when a participant raises their hand",
"show_hand_raised_timer_label": "Show hand raise duration"
}
},
"resolution_label": "Resolution",
"screen_share_header": "Screen sharing"
},
"star_rating_input_label_one": "{{count}} star",
"star_rating_input_label_other": "{{count}} stars",

View File

@@ -77,7 +77,7 @@
"@typescript-eslint/utils": "^8.61.0",
"@use-gesture/react": "^10.2.11",
"@vector-im/compound-design-tokens": "^10.0.0",
"@vector-im/compound-web": "^9.3.0",
"@vector-im/compound-web": "^10.0.0",
"@vitejs/plugin-react": "^6.0.2",
"@vitest/browser-playwright": "^4.1.5",
"@vitest/coverage-v8": "^4.0.18",
@@ -97,7 +97,7 @@
"lodash-es": "^4.17.21",
"loglevel": "^1.9.1",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#develop",
"matrix-widget-api": "^1.16.1",
"matrix-widget-api": "^1.18.0",
"node-stdlib-browser": "^1.3.1",
"normalize.css": "^8.0.1",
"observable-hooks": "^4.2.3",
@@ -131,5 +131,5 @@
"vitest": "^4.1.5",
"vitest-axe": "^1.0.0-pre.3"
},
"packageManager": "pnpm@11.6.0+sha512.9a36518224080c6fe5165afdcfe79bfa118c29be703f3f462b1e32efe1e98e47e8750b148e08286250aad4113cc7993ca413c4e2cd447752708c2ee5751bc95f"
"packageManager": "pnpm@11.21.0+sha512.521705bce689924eac72f5a3587122f362689ef6571e55ba80076fd637c11132ecffada26fad4ea79c485bfddbfd3d5a2a5b05805a77e893de71ec8a6cca3bb1"
}

View File

@@ -101,9 +101,7 @@ async function setRtcModeFromSettings(
// Move to Developer tab now
await page.getByRole("tab", { name: "Developer" }).click();
if (mode == "legacy") {
await page.getByText("Legacy: state events").click();
} else if (mode == "2_0") {
if (mode == "2_0") {
await page.getByText("Matrix 2.0").click();
} else {
// compat

View File

@@ -12,9 +12,7 @@ import { HOST1, HOST2, type RtcMode, TestHelpers } from "./test-helpers";
const modePairs: [RtcMode, RtcMode][] = [
["compat", "compat"],
["legacy", "legacy"],
["legacy", "compat"],
["compat", "legacy"],
// TODO: Compatibility + Matrix 2.0?
];
modePairs.forEach(([rtcMode1, rtcMode2]) => {

View File

@@ -1,85 +0,0 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user";
import { HOST1, HOST2, TestHelpers } from "./test-helpers";
widgetTest(
"Bug new joiner was not publishing on correct SFU",
async ({ addUser, browserName }) => {
test.skip(
browserName === "firefox",
"This is a bug in the old widget, not a browser problem.",
);
test.slow();
// 2 users in federation
const florian = await addUser("floriant", HOST1);
const timo = await addUser("timo", HOST2);
// Florian creates a room and invites Timo to it
const roomName = "Call Room";
await TestHelpers.createRoom(roomName, florian.page, [timo.mxId]);
// Timo joins the room
await TestHelpers.acceptRoomInvite(roomName, timo.page);
// Ensure we are in legacy mode (should be the default)
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
florian.page,
"legacy",
);
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
timo.page,
"legacy",
);
// Let timo create a call
await TestHelpers.startCallInCurrentRoom(timo.page, false);
await TestHelpers.joinCallFromLobby(timo.page);
// We want to simulate that the oldest membership authentication is way slower than
// the preffered auth.
// In this setup, timo advertised$ transport will be it's own, and the active will be the one from florian
await florian.page.route(
"**/matrix-rtc.othersite.m.localhost/livekit/jwt/**",
async (route) => {
await new Promise((resolve) => setTimeout(resolve, 2000)); // 5 second delay
await route.continue();
},
);
// Florian joins the call
await expect(florian.page.getByTestId("join-call-button")).toBeVisible();
await florian.page.getByTestId("join-call-button").click();
await TestHelpers.joinCallFromLobby(florian.page);
await florian.page.waitForTimeout(3000);
await timo.page.waitForTimeout(3000);
// We should see 2 video tiles everywhere now
for (const user of [timo, florian]) {
const frame = user.page
.locator('iframe[title="Element Call"]')
.contentFrame();
await expect(frame.getByTestId("videoTile")).toHaveCount(2);
// No one should be waiting for media
await expect(frame.getByText("Waiting for media...")).not.toBeVisible();
// There should be 2 video elements, visible and autoplaying
await expect(frame.locator("video")).toHaveCount(2, {
timeout: 10000,
});
await TestHelpers.expectVisibleVideoCount(frame, 2);
}
},
);

View File

@@ -1,91 +0,0 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user";
import { HOST1, HOST2, TestHelpers } from "./test-helpers";
// ## Issue
// This test reproduces an issue with the publisher.
// When switching local focus, we need to recreate the publisher.
// This failed because of a dead lock in the old publishers destruction.
//
// There are numerus ways to enforece this situation:
// - oldest member swap (manually set the oldest member focus and leave with the prev oldest member)
// This almost never happens in the real worls since clients will set their preferredFoci list to what the oldest member is.
// - switch from oldest member to multi sfu as the NOT the first joiner + the first joiner is on a different sfu than your preferred sfu.
//
// This test uses the "switch from oldest member to multi sfu" approach.
//
// It is a copy of federated-call.test.ts in the `["legacy", "legacy"]` setup,
// which once connected will make the second user switch to multi sfu.
widgetTest(
`Test swapping publisher from ${HOST1} to ${HOST2}`,
async ({ addUser, browserName }) => {
test.slow();
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
const florian = await addUser("floriant", HOST1);
const timo = await addUser("timo", HOST2);
const roomName = "Call Room";
await TestHelpers.createRoom(roomName, florian.page, [timo.mxId]);
await TestHelpers.acceptRoomInvite(roomName, timo.page);
await florian.page.pause();
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
florian.page,
"legacy",
);
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
timo.page,
"legacy",
);
await TestHelpers.startCallInCurrentRoom(florian.page, false);
await TestHelpers.joinCallFromLobby(florian.page);
// timo joins
await TestHelpers.joinCallInCurrentRoom(timo.page);
// We should see 2 video tiles everywhere now
for (const user of [timo, florian]) {
const frame = user.page
.locator('iframe[title="Element Call"]')
.contentFrame();
await expect(frame.getByTestId("videoTile")).toHaveCount(2);
// Wait for "Waiting for media..." to disappear (with timeout)
await expect(frame.getByText("Waiting for media...")).not.toBeVisible({
timeout: 10000, // Maximum time to wait
});
// There should be 2 video elements, visible and autoplaying
await expect(frame.locator("video")).toHaveCount(2, {
timeout: 10000,
});
await TestHelpers.expectVisibleVideoCount(frame, 2);
}
// now we switch the mode for timo (second joiner on multi-sfu HOST2 but currently HOST1)
await TestHelpers.setEmbeddedElementCallRtcMode(timo.page, "compat");
await timo.page.waitForTimeout(3000);
await TestHelpers.expectVisibleVideoCount(
timo.page.locator('iframe[title="Element Call"]').contentFrame(),
2,
);
},
);

View File

@@ -21,7 +21,7 @@ const PASSWORD = "foobarbaz1!";
export const HOST1 = "https://app.m.localhost/#/welcome";
export const HOST2 = "https://app.othersite.m.localhost/#/welcome";
export type RtcMode = "legacy" | "compat" | "2_0";
export type RtcMode = "compat" | "2_0";
export class TestHelpers {
public static async startCallInCurrentRoom(
@@ -309,9 +309,7 @@ export class TestHelpers {
// Move to Developer tab now
await iframe.getByRole("tab", { name: "Developer" }).click();
if (mode == "legacy") {
await iframe.getByText("Legacy: state events").click();
} else if (mode == "2_0") {
if (mode == "2_0") {
await iframe.getByText("Matrix 2.0").click();
} else {
// compat

410
pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff

View File

@@ -9,7 +9,7 @@ allowBuilds:
"@swc/core": true
"core-js": true
"esbuild": true
"matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/8c95727b6278fe7942c20d0b9485f984dd0694b7": true
"matrix-js-sdk@git+https://github.com/matrix-org/matrix-js-sdk.git": true
"protobufjs": true
overrides:
# We need compatible versions of RxJS in our dependencies and LiveKit's dependencies, but

View File

@@ -31,6 +31,11 @@ interface Props {
max: number;
step: number;
disabled?: boolean;
/**
* Custom formatter for the tooltip label. If not provided, the value is
* displayed as a percentage.
*/
tooltipFormatter?: (value: number) => string;
}
/**
@@ -46,6 +51,7 @@ export const Slider: FC<Props> = ({
max,
step,
disabled,
tooltipFormatter,
}) => {
const onValueChange = useCallback(
([v]: number[]) => onValueChangeProp(v),
@@ -71,7 +77,14 @@ export const Slider: FC<Props> = ({
<Range className={styles.highlight} />
</Track>
{/* Note: This is expected not to be visible on mobile.*/}
<Tooltip placement="top" label={Math.round(value * 100).toString() + "%"}>
<Tooltip
placement="top"
label={
tooltipFormatter
? tooltipFormatter(value)
: Math.round(value * 100).toString() + "%"
}
>
<Thumb className={styles.handle} aria-label={label} />
</Tooltip>
</Root>

View File

@@ -127,6 +127,7 @@ export const Default: Story = {
showFooter: true,
hideControls: false,
asOverlay: false,
showModals: true,
sharingScreen: false,
audioOutputSwitcher: undefined,
reactionIdentifier: undefined,

View File

@@ -26,6 +26,7 @@ import {
MediaMuteAndSwitchButton,
type MenuOptions,
} from "./MediaMuteAndSwitchButton";
import { type Behavior } from "../state/Behavior";
import { type ViewModel } from "../state/ViewModel";
import { useBehavior } from "../useBehavior";
import { type LayoutSwitchViewModel } from "../state/LayoutSwitchViewModel";
@@ -76,6 +77,7 @@ export interface FooterState {
/** The footer should be used as an overlay.
* (Over the Call Grid) This saves spaces on small screens. */
asOverlay: boolean;
showModals: boolean;
buttonSize: "md" | "lg";
showLogo: boolean;
@@ -120,6 +122,7 @@ export const CallFooter: FC<FooterProps> = ({
const asOverlay = useBehavior(vm.asOverlay$);
const showFooter = useBehavior(vm.showFooter$);
const hideControls = useBehavior(vm.hideControls$);
const showModals = useBehavior(vm.showModals$);
const layoutSwitchVm = useBehavior(vm.layoutSwitchVm$);
const openSettings = useBehavior(vm.openSettings$);
const audioEnabled = useBehavior(vm.audioEnabled$);
@@ -135,7 +138,6 @@ export const CallFooter: FC<FooterProps> = ({
const audioOutputSwitcher = useBehavior(vm.audioOutputSwitcher$);
const hangup = useBehavior(vm.hangup$);
const debugTileLayout = useBehavior(vm.debugTileLayout$);
const tileStoreGeneration = useBehavior(vm.tileStoreGeneration$);
const videoOptions = useBehavior(vm.videoOptions$);
const selectedVideo = useBehavior(vm.selectedVideo$);
const audioOptions = useBehavior(vm.audioOptions$);
@@ -235,7 +237,8 @@ export const CallFooter: FC<FooterProps> = ({
);
}
if (reactionIdentifier && reactionData) {
// Reaction button contains a pretty large menu, so treat it like a modal
if (reactionIdentifier && reactionData && showModals) {
buttons.push(
<ReactionToggleButton
size={buttonSize}
@@ -283,7 +286,9 @@ export const CallFooter: FC<FooterProps> = ({
/>
</>
)}
{debugTileLayout ? `Tiles generation: ${tileStoreGeneration}` : undefined}
{debugTileLayout ? (
<TilesDebugInfo generation$={vm.tileStoreGeneration$} />
) : undefined}
</div>
);
@@ -316,3 +321,14 @@ export const CallFooter: FC<FooterProps> = ({
</div>
);
};
interface TilesDebugInfoProps {
generation$: Behavior<number | undefined>;
}
// Isolated in its own component since the layout generation updates frequently
// and we can avoid re-rendering the footer this way
const TilesDebugInfo: FC<TilesDebugInfoProps> = ({ generation$ }) => {
const generation = useBehavior(generation$);
return `Tiles generation: ${generation}`;
};

View File

@@ -49,6 +49,7 @@ function buildMinimalCallViewModel(layout: Layout): CallViewModel {
reactions$: constant({}),
tileStoreGeneration$: constant(0),
showFooter$: constant(true),
showModals$: constant(true),
settingsOpen$: constant(false),
setSettingsOpen$: constant(() => {}),
} as unknown as CallViewModel;

View File

@@ -161,6 +161,7 @@ export function createCallFooterViewModel(
// candidat to move into the FooterViewModel
showFooter$: callModel.showFooter$,
hideControls$: constant(!showControls),
showModals$: callModel.showModals$,
asOverlay$: callModel.edgeToEdge$,
buttonSize$: scope.behavior(
isPip$.pipe(map<boolean, "md" | "lg">((pip) => (pip ? "md" : "lg"))),
@@ -168,12 +169,12 @@ export function createCallFooterViewModel(
openSettings$: scope.behavior(
combineLatest([
isPip$,
callModel.showModals$,
callModel.showHeader$,
callModel.setSettingsOpen$,
]).pipe(
map(([isPip, showHeader, setSettingsOpen]) =>
!isPip && headerStyle !== HeaderStyle.AppBar && showControls
map(([showModals, showHeader, setSettingsOpen]) =>
showModals && headerStyle !== HeaderStyle.AppBar && showControls
? (): void => setSettingsOpen(true)
: undefined,
),
@@ -239,6 +240,7 @@ export function createLobbyFooterViewModel(
showLogo,
hideControls: false,
asOverlay: false,
showModals: true,
buttonSize: "lg",
openSettings,
hangup,

View File

@@ -12,9 +12,7 @@ Please see LICENSE in the repository root for full details.
* Settings, or pinned for a deployment via `matrix_rtc_mode` in config.json.
*/
export enum MatrixRTCMode {
/** Legacy single-SFU + user-keyed memberships + legacy JWT endpoint. */
Legacy = "legacy",
/** Multi-SFU transport, legacy JWT endpoint, no sticky events. */
/** Multi-SFU transport, legacy JWT endpoint, state events. */
Compatibility = "compatibility",
/**
* Multi-SFU transport with:
@@ -74,10 +72,7 @@ export interface ConfigOptions {
livekit?: {
// The link to the service that returns a livekit url and token to use it.
// This is a fallback link in case the homeserver in use does not advertise
// a livekit service url in the client well-known.
// The well known needs to be formatted like so:
// {"type":"livekit", "livekit_service_url":"https://livekit.example.com"}
// and stored under the key: "org.matrix.msc4143.rtc_foci"
// a livekit service url over the transports endpoint.
livekit_service_url: string;
};
@@ -89,15 +84,6 @@ export interface ConfigOptions {
* Allow to join group calls without audio and video.
*/
feature_group_calls_without_video_and_audio?: boolean;
/**
* Send device-specific call session membership state events instead of
* legacy user-specific call membership state events.
* This setting has no effect when the user joins an active call with
* legacy state events. For compatibility, Element Call will always join
* active legacy calls with legacy state events.
*/
feature_use_device_session_member_events?: boolean;
};
/**
@@ -105,6 +91,61 @@ export interface ConfigOptions {
*/
ssla?: string;
/**
* Media quality settings for video and screen sharing.
* These override the hardcoded LiveKit defaults.
*/
media_quality?: {
/**
* Video codec preference. The server must also have the codec enabled.
* @default "vp8"
*/
video_codec?: "vp8" | "vp9" | "h264" | "av1";
/**
* Camera video settings.
*/
video?: {
/** Max resolution height in pixels (e.g. 720, 1080, 1440). @default 720 */
max_resolution?: number;
/** Max bitrate in bits per second. @default 1700000 */
max_bitrate?: number;
/** Max framerate. @default 30 */
max_framerate?: number;
/**
* Simulcast layers as an array of {height, bitrate} objects,
* ordered from lowest to highest quality.
* @default [{height: 180, bitrate: 160000}, {height: 360, bitrate: 450000}]
*/
simulcast_layers?: Array<{
height: number;
bitrate: number;
}>;
};
/**
* Screen share settings.
*/
screen_share?: {
/** Max resolution height in pixels. @default 1080 */
max_resolution?: number;
/** Max bitrate in bits per second. @default 5000000 */
max_bitrate?: number;
/** Max framerate. @default 30 */
max_framerate?: number;
/**
* Simulcast layers for screen sharing as an array of {height, bitrate, framerate} objects,
* ordered from lowest to highest quality. If omitted, LiveKit SDK defaults apply (1 extra
* layer at half resolution).
*/
simulcast_layers?: Array<{
height: number;
bitrate: number;
framerate?: number;
}>;
};
};
media_devices?: {
/**
* Defines whether participants should start with audio enabled by default.
@@ -127,7 +168,8 @@ export interface ConfigOptions {
/**
* Pins the {@link MatrixRTCMode} for all clients on this deployment,
* overriding any per-user choice from the Developer Settings. If unset,
* the user's Developer Settings choice (or its default of `Legacy`) wins.
* the user's Developer Settings choice (or its default of `Compatibility`)
* wins.
*/
matrix_rtc_mode?: MatrixRTCMode;
@@ -185,6 +227,24 @@ export interface ConfigOptions {
export interface ResolvedConfigOptions extends ConfigOptions {
sync_disconnect_grace_period_ms: number;
ssla: string;
media_quality: Required<
Pick<NonNullable<ConfigOptions["media_quality"]>, "video_codec">
> & {
video: Required<
Pick<
NonNullable<NonNullable<ConfigOptions["media_quality"]>["video"]>,
"max_resolution" | "max_bitrate" | "max_framerate"
>
>;
screen_share: Required<
Pick<
NonNullable<
NonNullable<ConfigOptions["media_quality"]>["screen_share"]
>,
"max_resolution" | "max_bitrate" | "max_framerate"
>
>;
};
matrix_rtc_session: {
wait_for_key_rotation_ms?: number;
delayed_leave_event_delay_ms: number;
@@ -196,11 +256,21 @@ export interface ResolvedConfigOptions extends ConfigOptions {
}
export const DEFAULT_CONFIG: ResolvedConfigOptions = {
features: {
feature_use_device_session_member_events: true,
},
sync_disconnect_grace_period_ms: 10000,
ssla: "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
media_quality: {
video_codec: "vp8",
video: {
max_resolution: 720,
max_bitrate: 1_700_000,
max_framerate: 30,
},
screen_share: {
max_resolution: 1080,
max_bitrate: 5_000_000,
max_framerate: 30,
},
},
matrix_rtc_session: {
delayed_leave_event_delay_ms: 10000,
network_error_retry_ms: 1000,

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 KiB

View File

@@ -1,16 +0,0 @@
<svg width="1440" height="800" viewBox="0 0 1440 800" fill="none" xmlns="http://www.w3.org/2000/svg">
<g filter="url(#filter0_f_4162_80259)">
<path d="M-37.0486 666.028C267.76 138.867 944.304 -46.1945 1477.05 260.929" stroke="url(#paint0_linear_4162_80259)" stroke-width="192" stroke-linecap="round"/>
</g>
<defs>
<filter id="filter0_f_4162_80259" x="-333.118" y="-183.694" width="2106.24" height="1145.68" filterUnits="userSpaceOnUse" color-interpolation-filters="sRGB">
<feFlood flood-opacity="0" result="BackgroundImageFix"/>
<feBlend mode="normal" in="SourceGraphic" in2="BackgroundImageFix" result="shape"/>
<feGaussianBlur stdDeviation="100" result="effect1_foregroundBlur_4162_80259"/>
</filter>
<linearGradient id="paint0_linear_4162_80259" x1="1255.31" y1="320.254" x2="184.899" y2="607.497" gradientUnits="userSpaceOnUse">
<stop stop-color="#00C59E"/>
<stop offset="0.75" stop-color="#0044A5"/>
</linearGradient>
</defs>
</svg>

Before

Width:  |  Height:  |  Size: 938 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 KiB

View File

@@ -1,86 +0,0 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
width="1200"
height="285"
viewBox="0 0 1200 285"
fill="none"
version="1.1"
id="svg5"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<g
filter="url(#filter0_f_3970_9366)"
id="g2"
transform="translate(422.00201)">
<path
d="m -164.342,495.134 c 188.2881,-188.288 494.684,-190 684.684,0"
stroke="url(#paint0_linear_3970_9366)"
stroke-width="235.517"
stroke-linecap="round"
id="path1"
style="stroke:url(#paint0_linear_3970_9366)" />
<path
d="m -164.342,495.134 c 188.2881,-188.288 494.684,-190 684.684,0"
stroke="url(#paint1_linear_3970_9366)"
style="mix-blend-mode:overlay;stroke:url(#paint1_linear_3970_9366)"
stroke-width="235.517"
stroke-linecap="round"
id="path2" />
</g>
<defs
id="defs5">
<filter
id="filter0_f_3970_9366"
x="-517.617"
y="-0.00012207"
width="1391.23"
height="848.409"
filterUnits="userSpaceOnUse"
color-interpolation-filters="sRGB">
<feFlood
flood-opacity="0"
result="BackgroundImageFix"
id="feFlood2" />
<feBlend
mode="normal"
in="SourceGraphic"
in2="BackgroundImageFix"
result="shape"
id="feBlend2" />
<feGaussianBlur
stdDeviation="117.758"
result="effect1_foregroundBlur_3970_9366"
id="feGaussianBlur2" />
</filter>
<linearGradient
id="paint0_linear_3970_9366"
x1="349.17099"
y1="323.96301"
x2="6.82898"
y2="666.30499"
gradientUnits="userSpaceOnUse">
<stop
stop-color="#0D5CBD"
id="stop2" />
<stop
offset="0.730863"
stop-color="#0DBDA8"
id="stop3" />
</linearGradient>
<linearGradient
id="paint1_linear_3970_9366"
x1="349.17099"
y1="323.96301"
x2="6.82898"
y2="666.30499"
gradientUnits="userSpaceOnUse">
<stop
stop-color="#0D5CBD"
id="stop4" />
<stop
offset="0.730863"
stop-color="#0DBDA8"
id="stop5" />
</linearGradient>
</defs>
</svg>

Before

Width:  |  Height:  |  Size: 2.2 KiB

View File

@@ -41,6 +41,8 @@ import { TileWrapper } from "./TileWrapper";
import { usePrefersReducedMotion } from "../usePrefersReducedMotion";
import { useInitial } from "../useInitial";
const MAX_ANIMATED_TILES = 50; // Capped for performance reasons
interface Rect {
x: number;
y: number;
@@ -285,7 +287,6 @@ export function Grid<
const [visibleTilesCallback, setVisibleTilesCallback] =
useState<VisibleTilesCallback | null>(null);
const tiles = useInitial(() => new Map<string, Tile<TileModel>>());
const prefersReducedMotion = usePrefersReducedMotion();
const Slot: FC<SlotProps<TileModel>> = useMemo(
() =>
@@ -372,6 +373,10 @@ export function Grid<
// react-spring's imperative API during gestures to improve responsiveness
const dragState = useRef<DragState | null>(null);
// If true, disables animations
const immediate =
usePrefersReducedMotion() || placedTiles.length > MAX_ANIMATED_TILES;
const [tileTransitions, springRef] = useTransition(
placedTiles,
() => ({
@@ -389,9 +394,9 @@ export function Grid<
y,
width,
height,
immediate: prefersReducedMotion,
immediate,
}),
enter: { opacity: 1, scale: 1, immediate: prefersReducedMotion },
enter: { opacity: 1, scale: 1, immediate },
update: ({
id,
x,
@@ -406,9 +411,9 @@ export function Grid<
y,
width,
height,
immediate: prefersReducedMotion,
immediate,
},
leave: { opacity: 0, scale: 0, immediate: prefersReducedMotion },
leave: { opacity: 0, scale: 0, immediate },
config: { mass: 0.7, tension: 252, friction: 25 },
}),
// react-spring's types are bugged and can't infer the spring type
@@ -441,8 +446,7 @@ export function Grid<
y: tile.y,
width: tile.width,
height: tile.height,
immediate:
prefersReducedMotion || ((key): boolean => key === "zIndex"),
immediate: immediate || ((key): boolean => key === "zIndex"),
// Allow the tile's position to settle before pushing its
// z-index back down
delay: (key): number => (key === "zIndex" ? 500 : 0),
@@ -453,7 +457,7 @@ export function Grid<
x: tileX,
y: tileY,
immediate:
prefersReducedMotion ||
immediate ||
((key): boolean =>
key === "zIndex" || key === "x" || key === "y"),
},

View File

@@ -78,16 +78,19 @@ body {
body[data-background="gradient"]::before {
content: "";
position: fixed;
inset: 0;
background-image: url("graphics/mobile-gradient.svg");
background-size: auto;
/* Chromium abruptly fades our images to fully transparent at the edge of
the element. If we just make the element a little bigger than the viewport,
this is no longer visible. */
inset: -20px;
background-image: url("graphics/mobile-gradient.png");
background-size: 1400px 305px;
background-position: bottom;
background-repeat: no-repeat;
}
body[data-background="gradient"][data-platform="desktop"]::before {
background-image: url("graphics/desktop-gradient.svg");
background-size: calc(max(1440px, 100vw)) calc(max(800px, 100vh));
background-image: url("graphics/desktop-gradient.png");
background-size: max(1440px, 100vw) max(1440px, 100vh);
background-position: center;
}
}

View File

@@ -30,6 +30,7 @@ import {
import { getUrlParams } from "./UrlParams";
import { Config } from "./config/Config";
import { seedSettingsFromConfig } from "./settings/settings";
import { platform } from "./Platform";
import { isFailure } from "./utils/fetch";
import { initializeWidget } from "./widget";
@@ -237,6 +238,7 @@ export class Initializer {
this.loadStates.config = LoadState.Loading;
Config.init().then(
() => {
seedSettingsFromConfig(Config.get().media_quality);
this.loadStates.config = LoadState.Loaded;
this.initStep(resolve);
},

213
src/livekit/options.test.ts Normal file
View File

@@ -0,0 +1,213 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, vi } from "vitest";
import { VideoPresets, type VideoPreset } from "livekit-client";
import { buildLiveKitOptions, getLiveKitOptions } from "./options";
import { Config } from "../config/Config";
vi.mock("../config/Config", () => ({
Config: {
get: vi.fn(),
},
}));
describe("buildLiveKitOptions", () => {
it("returns sensible defaults with no config", () => {
const opts = buildLiveKitOptions();
expect(opts.adaptiveStream).toBe(true);
expect(opts.dynacast).toBe(true);
expect(opts.videoCaptureDefaults?.resolution).toEqual(
VideoPresets.h720.resolution,
);
expect(opts.publishDefaults?.videoCodec).toBe("vp8");
expect(opts.publishDefaults?.videoEncoding).toEqual({
maxBitrate: 1_700_000,
maxFramerate: 30,
});
expect(opts.publishDefaults?.screenShareEncoding).toEqual({
maxBitrate: 5_000_000,
maxFramerate: 30,
});
expect(opts.publishDefaults?.videoSimulcastLayers).toEqual([
VideoPresets.h180,
VideoPresets.h360,
]);
});
it("applies video codec from config", () => {
const opts = buildLiveKitOptions({ video_codec: "vp9" });
expect(opts.publishDefaults?.videoCodec).toBe("vp9");
});
it("applies video resolution and encoding from config", () => {
const baseVideoConfig = {
max_resolution: 1080,
max_bitrate: 3_000_000,
max_framerate: 60,
};
const opts1080 = buildLiveKitOptions({
video: baseVideoConfig,
});
const opts1440 = buildLiveKitOptions({
video: { ...baseVideoConfig, max_resolution: 1440 },
});
const opts2160 = buildLiveKitOptions({
video: { ...baseVideoConfig, max_resolution: 2160 },
});
expect(opts1080.videoCaptureDefaults?.resolution).toEqual(
VideoPresets.h1080.resolution,
);
expect(opts1440.videoCaptureDefaults?.resolution).toEqual(
VideoPresets.h1440.resolution,
);
expect(opts2160.videoCaptureDefaults?.resolution).toEqual(
VideoPresets.h2160.resolution,
);
expect(opts1080.publishDefaults?.videoEncoding).toEqual({
maxBitrate: 3_000_000,
maxFramerate: 60,
});
});
it("applies screen share encoding from config", () => {
const opts = buildLiveKitOptions({
screen_share: {
max_bitrate: 8_000_000,
max_framerate: 15,
},
});
expect(opts.publishDefaults?.screenShareEncoding).toEqual({
maxBitrate: 8_000_000,
maxFramerate: 15,
});
});
it("uses DEFAULT_CONFIG defaults when only resolution is set", () => {
const opts = buildLiveKitOptions({
screen_share: {
max_resolution: 720,
},
});
// Bitrate and framerate fall back to DEFAULT_CONFIG, not the preset
expect(opts.publishDefaults?.screenShareEncoding).toEqual({
maxBitrate: 5_000_000,
maxFramerate: 30,
});
});
it("maps low resolutions to the closest preset, rounding up", () => {
const expectations: [number, VideoPreset][] = [
[180, VideoPresets.h180],
[360, VideoPresets.h360],
[480, VideoPresets.h540],
[540, VideoPresets.h540],
[720, VideoPresets.h720],
];
for (const [height, preset] of expectations) {
const opts = buildLiveKitOptions({ video: { max_resolution: height } });
expect(opts.videoCaptureDefaults?.resolution).toEqual(preset.resolution);
}
});
it("screen share layers fall back to max_framerate, then 30", () => {
const fromMax = buildLiveKitOptions({
screen_share: {
max_framerate: 15,
simulcast_layers: [{ height: 540, bitrate: 1_000_000 }],
},
});
expect(
fromMax.publishDefaults?.screenShareSimulcastLayers?.[0],
).toMatchObject({ encoding: { maxFramerate: 15 } });
const fromDefault = buildLiveKitOptions({
screen_share: {
simulcast_layers: [{ height: 540, bitrate: 1_000_000 }],
},
});
expect(
fromDefault.publishDefaults?.screenShareSimulcastLayers?.[0],
).toMatchObject({ encoding: { maxFramerate: 30 } });
});
it("applies custom video simulcast layers", () => {
const opts = buildLiveKitOptions({
video: {
simulcast_layers: [
{ height: 180, bitrate: 100_000 },
{ height: 360, bitrate: 300_000 },
{ height: 540, bitrate: 600_000 },
],
max_framerate: 24,
},
});
const layers = opts.publishDefaults?.videoSimulcastLayers;
expect(layers).toHaveLength(3);
expect(layers?.[0]).toMatchObject({
width: 320,
height: 180,
encoding: { maxBitrate: 100_000, maxFramerate: 24 },
});
expect(layers?.[2]).toMatchObject({
width: 960,
height: 540,
encoding: { maxBitrate: 600_000, maxFramerate: 24 },
});
});
it("applies custom screen share simulcast layers", () => {
const opts = buildLiveKitOptions({
screen_share: {
simulcast_layers: [{ height: 540, bitrate: 1_000_000, framerate: 5 }],
},
});
const layers = opts.publishDefaults?.screenShareSimulcastLayers;
expect(layers).toHaveLength(1);
expect(layers?.[0]).toMatchObject({
width: 960,
height: 540,
encoding: { maxBitrate: 1_000_000, maxFramerate: 5 },
});
});
it("does not include screenShareSimulcastLayers when not configured", () => {
const opts = buildLiveKitOptions();
expect(opts.publishDefaults?.screenShareSimulcastLayers).toBeUndefined();
});
it("backupCodec always uses stock VP8 720p encoding", () => {
const opts = buildLiveKitOptions({
video_codec: "av1",
video: { max_bitrate: 10_000_000, max_framerate: 60 },
});
const backup = opts.publishDefaults?.backupCodec as {
codec: string;
encoding: { maxBitrate: number; maxFramerate: number };
};
expect(backup.codec).toBe("vp8");
expect(backup.encoding).toEqual(VideoPresets.h720.encoding);
});
});
describe("getLiveKitOptions", () => {
it("reads from Config singleton", () => {
vi.mocked(Config.get).mockReturnValue({
media_quality: { video_codec: "h264" },
} as ReturnType<typeof Config.get>);
const opts = getLiveKitOptions();
expect(opts.publishDefaults?.videoCodec).toBe("h264");
});
it("throws when Config is not initialized", () => {
vi.mocked(Config.get).mockImplementation(() => {
throw new Error("Config not initialized");
});
expect(() => getLiveKitOptions()).toThrow("Config not initialized");
});
});

View File

@@ -9,46 +9,144 @@ import {
AudioPresets,
DefaultReconnectPolicy,
type RoomOptions,
ScreenSharePresets,
type TrackPublishDefaults,
type VideoPreset,
VideoPresets,
VideoPreset as VideoPresetClass,
} from "livekit-client";
const defaultLiveKitPublishOptions: TrackPublishDefaults = {
audioPreset: AudioPresets.music,
dtx: true,
// disable red because the livekit server strips out red packets for clients
// that don't support it (firefox) but of course that doesn't work with e2ee.
red: false,
forceStereo: false,
simulcast: true,
videoSimulcastLayers: [VideoPresets.h180, VideoPresets.h360] as VideoPreset[],
screenShareEncoding: ScreenSharePresets.h1080fps30.encoding,
stopMicTrackOnMute: false,
videoCodec: "vp8",
videoEncoding: VideoPresets.h720.encoding,
backupCodec: { codec: "vp8", encoding: VideoPresets.h720.encoding },
} as const;
import { Config } from "../config/Config";
import { DEFAULT_CONFIG, type ConfigOptions } from "../config/ConfigOptions";
export const defaultLiveKitOptions: RoomOptions = {
// automatically manage subscribed video quality
adaptiveStream: true,
/**
* Find the closest matching VideoPreset for a given height.
*/
function videoPresetForHeight(height: number): VideoPreset {
if (height <= 180) return VideoPresets.h180;
if (height <= 360) return VideoPresets.h360;
if (height <= 540) return VideoPresets.h540;
if (height <= 720) return VideoPresets.h720;
if (height <= 1080) return VideoPresets.h1080;
if (height <= 1440) return VideoPresets.h1440;
return VideoPresets.h2160;
}
// optimize publishing bandwidth and CPU for published tracks
dynacast: true,
/**
* Build LiveKit publish options from config, falling back to sensible defaults.
*/
function buildPublishOptions(
mediaQuality: ConfigOptions["media_quality"],
): TrackPublishDefaults {
const defaults = DEFAULT_CONFIG.media_quality;
const videoConf = mediaQuality?.video;
const screenConf = mediaQuality?.screen_share;
const codec = mediaQuality?.video_codec ?? defaults.video_codec;
// capture settings
videoCaptureDefaults: {
resolution: VideoPresets.h720.resolution,
},
// Camera video encoding
const videoEncoding = {
maxBitrate: videoConf?.max_bitrate ?? defaults.video.max_bitrate,
maxFramerate: videoConf?.max_framerate ?? defaults.video.max_framerate,
};
// publish settings
publishDefaults: defaultLiveKitPublishOptions,
// Camera simulcast layers
let videoSimulcastLayers: VideoPreset[];
if (videoConf?.simulcast_layers) {
videoSimulcastLayers = videoConf.simulcast_layers.map(
(layer) =>
new VideoPresetClass(
Math.round((layer.height * 16) / 9),
layer.height,
layer.bitrate,
videoConf?.max_framerate ?? defaults.video.max_framerate,
),
);
} else {
videoSimulcastLayers = [VideoPresets.h180, VideoPresets.h360];
}
// default LiveKit options that seem to be sane
stopLocalTrackOnUnpublish: true,
reconnectPolicy: new DefaultReconnectPolicy(),
disconnectOnPageLeave: true,
webAudioMix: false,
};
// Screen share encoding
const screenShareEncoding = {
maxBitrate: screenConf?.max_bitrate ?? defaults.screen_share.max_bitrate,
maxFramerate:
screenConf?.max_framerate ?? defaults.screen_share.max_framerate,
};
// Screen share simulcast layers
let screenShareSimulcastLayers: VideoPreset[] | undefined;
if (screenConf?.simulcast_layers) {
screenShareSimulcastLayers = screenConf.simulcast_layers.map(
(layer) =>
new VideoPresetClass(
Math.round((layer.height * 16) / 9),
layer.height,
layer.bitrate,
layer.framerate ?? screenConf?.max_framerate ?? 30,
),
);
}
return {
audioPreset: AudioPresets.music,
dtx: true,
// disable red because the livekit server strips out red packets for clients
// that don't support it (firefox) but of course that doesn't work with e2ee.
red: false,
forceStereo: false,
simulcast: true,
videoSimulcastLayers: videoSimulcastLayers as VideoPreset[],
screenShareEncoding,
...(screenShareSimulcastLayers && {
screenShareSimulcastLayers: screenShareSimulcastLayers as VideoPreset[],
}),
stopMicTrackOnMute: false,
videoCodec: codec,
videoEncoding,
backupCodec: {
codec: "vp8",
encoding: VideoPresets.h720.encoding,
},
} as TrackPublishDefaults;
}
/**
* Build LiveKit RoomOptions from config.
* Call this after Config.init() has resolved.
*/
export function buildLiveKitOptions(
mediaQuality?: ConfigOptions["media_quality"],
): RoomOptions {
const videoHeight =
mediaQuality?.video?.max_resolution ??
DEFAULT_CONFIG.media_quality.video.max_resolution;
const basePreset = videoPresetForHeight(videoHeight);
return {
// automatically manage subscribed video quality
adaptiveStream: true,
// optimize publishing bandwidth and CPU for published tracks
dynacast: true,
// capture settings
videoCaptureDefaults: {
resolution: basePreset.resolution,
},
// publish settings
publishDefaults: buildPublishOptions(mediaQuality),
// default LiveKit options that seem to be sane
stopLocalTrackOnUnpublish: true,
reconnectPolicy: new DefaultReconnectPolicy(),
disconnectOnPageLeave: true,
webAudioMix: false,
};
}
/**
* Get LiveKit options, reading from the loaded Config singleton.
* Requires Config.init() to have resolved first.
*/
export function getLiveKitOptions(): RoomOptions {
return buildLiveKitOptions(Config.get().media_quality);
}

View File

@@ -555,6 +555,9 @@ export const GroupCallView: FC<Props> = ({
}}
onError={(_error) => {
if (rtcSession.isJoined()) onLeft("error");
// If there is an error we need to be able to close the widget. This is done in `onLeft` as well
// We need it here explicitly in case rtcSession.isJoined is false.
void widget?.api.setAlwaysOnScreen(false);
}}
>
{body}

View File

@@ -261,6 +261,7 @@ export const InCallView: FC<InCallViewProps> = ({
const overflowing = useBehavior(vm.overflowing$);
const showNameTags = useBehavior(vm.showNameTags$);
const showHeader = useBehavior(vm.showHeader$);
const showModals = useBehavior(vm.showModals$);
const settingsOpen = useBehavior(vm.settingsOpen$);
const setSettingsOpen = useBehavior(vm.setSettingsOpen$);
const earpieceMode = useBehavior(vm.earpieceMode$);
@@ -622,7 +623,7 @@ export const InCallView: FC<InCallViewProps> = ({
{earpieceOverlay}
<ReactionsOverlay vm={vm} />
{footer}
{layout.type !== "pip" && (
{showModals && (
<>
<RageshakeRequestModal {...rageshakeRequestModalProps} />
<SettingsModal

View File

@@ -22,6 +22,9 @@ Please see LICENSE in the repository root for full details.
height: 100%;
object-fit: cover;
background-color: var(--video-tile-background);
/* In FF if you add a transform: scale/translate/matrix filter on an element,
it'll ignore the parents' border-radius, so force back the radius to avoid UI glitch*/
border-radius: inherit;
}
video.mirror {

View File

@@ -280,7 +280,7 @@ export const useLoadGroupCall = (
);
}
if (
roomSummary === undefined ||
roomSummary?.join_rule === undefined ||
roomSummary.join_rule === JoinRule.Public
) {
room = await client.joinRoom(roomId, {

View File

@@ -317,19 +317,15 @@ describe("DeveloperSettingsTab", () => {
describe("matrix rtc mode", () => {
afterEach(() => {
matrixRTCModeSetting.setValue(MatrixRTCMode.Legacy);
matrixRTCModeSetting.setValue(MatrixRTCMode.Compatibility);
vi.restoreAllMocks();
});
function getModeRadios(): {
legacy: HTMLInputElement;
compatibility: HTMLInputElement;
matrix20: HTMLInputElement;
} {
return {
legacy: screen.getByDisplayValue(
MatrixRTCMode.Legacy,
) as HTMLInputElement,
compatibility: screen.getByDisplayValue(
MatrixRTCMode.Compatibility,
) as HTMLInputElement,
@@ -359,27 +355,21 @@ describe("DeveloperSettingsTab", () => {
const radios = getModeRadios();
expect(radios.compatibility).toBeChecked();
expect(radios.legacy).not.toBeChecked();
expect(radios.matrix20).not.toBeChecked();
// None are disabled by config; only Matrix_2_0 may be disabled by sticky-events support.
expect(radios.legacy).not.toBeDisabled();
expect(radios.compatibility).not.toBeDisabled();
});
it.each([
MatrixRTCMode.Legacy,
MatrixRTCMode.Compatibility,
MatrixRTCMode.Matrix_2_0,
])(
it.each([MatrixRTCMode.Compatibility, MatrixRTCMode.Matrix_2_0])(
"disables all radios and shows the config value (%s) as checked when matrix_rtc_mode is set",
async (configMode) => {
mockConfig({ matrix_rtc_mode: configMode });
// Local setting is intentionally different from the config value to
// prove config wins.
matrixRTCModeSetting.setValue(
configMode === MatrixRTCMode.Legacy
? MatrixRTCMode.Compatibility
: MatrixRTCMode.Legacy,
configMode === MatrixRTCMode.Compatibility
? MatrixRTCMode.Matrix_2_0
: MatrixRTCMode.Compatibility,
);
const client = createMockMatrixClient();
@@ -397,13 +387,11 @@ describe("DeveloperSettingsTab", () => {
);
const radios = getModeRadios();
expect(radios.legacy).toBeDisabled();
expect(radios.compatibility).toBeDisabled();
expect(radios.matrix20).toBeDisabled();
const checkedValue = (
{
[MatrixRTCMode.Legacy]: radios.legacy,
[MatrixRTCMode.Compatibility]: radios.compatibility,
[MatrixRTCMode.Matrix_2_0]: radios.matrix20,
} as const

View File

@@ -8,6 +8,7 @@ Please see LICENSE in the repository root for full details.
import {
type ChangeEvent,
type FC,
type ReactNode,
useCallback,
useEffect,
useMemo,
@@ -29,12 +30,14 @@ import {
InlineField,
Label,
RadioControl,
Separator,
} from "@vector-im/compound-web";
import { type Room as LivekitRoom } from "livekit-client";
import { FieldRow, InputField } from "../input/Input";
import { Config } from "../config/Config";
import {
type Setting,
useSetting,
duplicateTiles as duplicateTilesSetting,
debugTileLayout as debugTileLayoutSetting,
@@ -43,10 +46,26 @@ import {
alwaysShowIphoneEarpiece as alwaysShowIphoneEarpieceSetting,
matrixRTCMode as matrixRTCModeSetting,
customLivekitUrl as customLivekitUrlSetting,
advancedScreenShare as advancedScreenShareSetting,
screenShareResolution as screenShareResolutionSetting,
screenShareFramerate as screenShareFramerateSetting,
screenShareBitrate as screenShareBitrateSetting,
screenShareCodec as screenShareCodecSetting,
advancedCamera as advancedCameraSetting,
cameraResolution as cameraResolutionSetting,
cameraFramerate as cameraFramerateSetting,
cameraBitrate as cameraBitrateSetting,
cameraCodec as cameraCodecSetting,
echoCancellationSetting,
noiseSuppressionSetting,
autoGainControlSetting,
type VideoCodec,
enableExtendedLivekitLogs as enableExtendedLivekitLogsSetting,
} from "./settings";
import { MatrixRTCMode } from "../config/ConfigOptions";
import styles from "./DeveloperSettingsTab.module.css";
import settingsStyles from "./SettingsModal.module.css";
import { Slider } from "../Slider";
import { useUrlParams } from "../UrlParams";
import { getSFUConfigWithOpenID } from "../livekit/openIDSFU";
@@ -138,6 +157,185 @@ export const DeveloperSettingsTab: FC<Props> = ({
return null;
}, [livekitRooms]);
const MediaQualitySettings: React.FC<{
id: string;
header: string;
toggleLabel: string;
description: string;
toggleSetting: Setting<boolean>;
resolutionSetting: Setting<string>;
framerateSetting: Setting<number>;
bitrateSetting: Setting<number>;
codecSetting: Setting<VideoCodec>;
resolutionOptions: { value: string; label: string }[];
bitrateRange: { min: number; max: number; step: number };
}> = ({
id,
header,
toggleLabel,
description,
toggleSetting,
resolutionSetting,
framerateSetting,
bitrateSetting,
codecSetting,
resolutionOptions,
bitrateRange,
}): ReactNode => {
const [advancedEnabled, setAdvancedEnabled] = useSetting(toggleSetting);
const [resolution, setResolution] = useSetting(resolutionSetting);
const [framerate, setFramerate] = useSetting(framerateSetting);
const [framerateRaw, setFramerateRaw] = useState(framerate);
const [bitrate, setBitrate] = useSetting(bitrateSetting);
const [bitrateRaw, setBitrateRaw] = useState(bitrate);
const [codec, setCodec] = useSetting(codecSetting);
return (
<>
<h4>{header}</h4>
<FieldRow>
<InputField
id={`${id}Toggle`}
label={toggleLabel}
description={description}
type="checkbox"
checked={advancedEnabled}
onChange={(e): void => setAdvancedEnabled(e.target.checked)}
/>
</FieldRow>
{advancedEnabled && (
<>
<div className={settingsStyles.volumeSlider}>
<label htmlFor={`${id}Resolution`}>
{t("settings.resolution_label", "Resolution")}
</label>
<select
id={`${id}Resolution`}
value={resolution}
onChange={(e): void => setResolution(e.target.value)}
>
{resolutionOptions.map((opt) => (
<option key={opt.value} value={opt.value}>
{opt.label}
</option>
))}
</select>
</div>
<div className={settingsStyles.volumeSlider}>
<label>
{t("settings.framerate_label", "Framerate")}
{": "}
<span className={settingsStyles.settingValue}>
{framerateRaw} fps
</span>
</label>
<Slider
label={t("settings.framerate_label", "Framerate")}
value={framerateRaw}
onValueChange={setFramerateRaw}
onValueCommit={setFramerate}
min={5}
max={60}
step={5}
tooltipFormatter={(v): string => `${v} fps`}
/>
</div>
<div className={settingsStyles.volumeSlider}>
<label>
{t("settings.bitrate_label", "Bitrate")}
{": "}
<span className={settingsStyles.settingValue}>
{(bitrateRaw / 1_000_000).toFixed(1)} Mbps
</span>
</label>
<Slider
label={t("settings.bitrate_label", "Bitrate")}
value={bitrateRaw}
onValueChange={setBitrateRaw}
onValueCommit={setBitrate}
min={bitrateRange.min}
max={bitrateRange.max}
step={bitrateRange.step}
tooltipFormatter={(v): string =>
`${(v / 1_000_000).toFixed(1)} Mbps`
}
/>
</div>
<div className={settingsStyles.volumeSlider}>
<label htmlFor={`${id}Codec`}>
{t("settings.codec_label", "Codec")}
</label>
<select
id={`${id}Codec`}
value={codec}
onChange={(e): void => setCodec(e.target.value as VideoCodec)}
>
<option value="vp8">VP8</option>
<option value="vp9">VP9</option>
<option value="h264">H.264</option>
<option value="av1">AV1</option>
</select>
</div>
</>
)}
</>
);
};
const AudioProcessingSettings: React.FC = (): ReactNode => {
const [echoCancellation, setEchoCancellation] = useSetting(
echoCancellationSetting,
);
const [noiseSuppression, setNoiseSuppression] = useSetting(
noiseSuppressionSetting,
);
const [autoGainControl, setAutoGainControl] = useSetting(
autoGainControlSetting,
);
return (
<>
<h4>{t("settings.audio_processing_header", "Audio processing")}</h4>
<p>
{t(
"settings.audio_processing_description",
"Changes apply on next call join.",
)}
</p>
<FieldRow>
<InputField
id="echoCancellation"
label={t("settings.echo_cancellation_label", "Echo cancellation")}
type="checkbox"
checked={echoCancellation}
onChange={(e): void => setEchoCancellation(e.target.checked)}
/>
</FieldRow>
<FieldRow>
<InputField
id="noiseSuppression"
label={t("settings.noise_suppression_label", "Noise suppression")}
type="checkbox"
checked={noiseSuppression}
onChange={(e): void => setNoiseSuppression(e.target.checked)}
/>
</FieldRow>
<FieldRow>
<InputField
id="autoGainControl"
label={t(
"settings.auto_gain_control_label",
"Automatic gain control",
)}
type="checkbox"
checked={autoGainControl}
onChange={(e): void => setAutoGainControl(e.target.checked)}
/>
</FieldRow>
</>
);
};
return (
<>
<p>
@@ -166,6 +364,7 @@ export const DeveloperSettingsTab: FC<Props> = ({
id: client.getDeviceId() || "unknown",
})}
</p>
<Separator />
<FieldRow>
<InputField
id="duplicateTiles"
@@ -315,27 +514,12 @@ export const DeveloperSettingsTab: FC<Props> = ({
<ErrorMessage>{customLivekitUrlUpdateError}</ErrorMessage>
)}
</EditInPlace>
<Separator />
<Heading as="h3" type="body" weight="semibold" size="lg">
{t("developer_mode.matrixRTCMode.title")}
</Heading>
{matrixRTCModeForced && <p>Your deployment overrides the mode.</p>}
<Form>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={effectiveMatrixRTCMode === MatrixRTCMode.Legacy}
value={MatrixRTCMode.Legacy}
disabled={matrixRTCModeForced}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Legacy.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Legacy.description")}
</HelpMessage>
</InlineField>
<InlineField
name={matrixRTCModeRadioGroup}
control={
@@ -403,6 +587,61 @@ export const DeveloperSettingsTab: FC<Props> = ({
</ul>
</div>
))}
<Separator />
<MediaQualitySettings
id="camera"
header={t("settings.camera_header", "Camera quality")}
toggleLabel={t(
"settings.advanced_camera_label",
"Advanced camera settings",
)}
description={t(
"settings.advanced_camera_description",
"Configure resolution, framerate, bitrate, and codec for camera video. Changes apply on next call join.",
)}
toggleSetting={advancedCameraSetting}
resolutionSetting={cameraResolutionSetting}
framerateSetting={cameraFramerateSetting}
bitrateSetting={cameraBitrateSetting}
codecSetting={cameraCodecSetting}
resolutionOptions={[
{ value: "640x360", label: "360p" },
{ value: "960x540", label: "540p" },
{ value: "1280x720", label: "720p" },
{ value: "1920x1080", label: "1080p" },
{ value: "2560x1440", label: "1440p" },
]}
bitrateRange={{ min: 200_000, max: 8_000_000, step: 100_000 }}
/>
<Separator />
<MediaQualitySettings
id="screenShare"
header={t("settings.screen_share_header", "Screen sharing")}
toggleLabel={t(
"settings.advanced_screen_share_label",
"Advanced screen share settings",
)}
description={t(
"settings.advanced_screen_share_description",
"Configure resolution, framerate, bitrate, and codec for screen sharing",
)}
toggleSetting={advancedScreenShareSetting}
resolutionSetting={screenShareResolutionSetting}
framerateSetting={screenShareFramerateSetting}
bitrateSetting={screenShareBitrateSetting}
codecSetting={screenShareCodecSetting}
resolutionOptions={[
{ value: "1024x576", label: "576p" },
{ value: "1280x720", label: "720p" },
{ value: "1920x1080", label: "1080p" },
{ value: "2560x1440", label: "1440p" },
{ value: "3840x2160", label: "4K" },
]}
bitrateRange={{ min: 500_000, max: 15_000_000, step: 500_000 }}
/>
<Separator />
<AudioProcessingSettings />
<Separator />
<p>{t("developer_mode.environment_variables")}</p>
<pre>{JSON.stringify(env, null, 2)}</pre>
<p>{t("developer_mode.url_params")}</p>

View File

@@ -33,3 +33,27 @@ Please see LICENSE in the repository root for full details.
.volumeSlider > p {
color: var(--cpd-color-text-secondary);
}
.volumeSlider > select {
display: block;
width: 100%;
padding: 8px 12px;
margin-top: var(--cpd-space-1x);
border: 1px solid var(--cpd-color-border-interactive-primary);
border-radius: 4px;
background-color: var(--cpd-color-bg-canvas-default);
color: var(--cpd-color-text-primary);
font-size: var(--font-size-body);
font-family: inherit;
cursor: pointer;
}
.volumeSlider > select:focus {
outline: none;
border-color: var(--cpd-color-text-link-external);
}
.settingValue {
font-weight: normal;
color: var(--cpd-color-text-secondary);
}

View File

@@ -152,7 +152,13 @@ export const SettingsModal: FC<Props> = ({
/>
<div className={styles.volumeSlider}>
<label>{t("settings.audio_tab.effect_volume_label")}</label>
<label>
{t("settings.audio_tab.effect_volume_label")}
{": "}
<span className={styles.settingValue}>
{Math.round(soundVolumeRaw * 100)}%
</span>
</label>
<p>{t("settings.audio_tab.effect_volume_description")}</p>
<Slider
label={t("video_tile.volume")}

View File

@@ -17,6 +17,12 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
<p>
Device ID: DEVICE123
</p>
<div
class="_separator_13qwf_8"
data-kind="primary"
data-orientation="horizontal"
role="separator"
/>
<div
class="_fieldRow_1bd8c0"
>
@@ -251,10 +257,16 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
class="_message_1o4d9_86 _help-message_1o4d9_92"
id="radix-_r_8_"
>
Currently, no overwrite is set. Url from well-known or config is used.
Currently, no overwrite is set. Url from config is used.
</span>
</div>
</form>
<div
class="_separator_13qwf_8"
data-kind="primary"
data-orientation="horizontal"
role="separator"
/>
<h3
class="_typography_6v6n8_153 _font-body-lg-semibold_6v6n8_74"
>
@@ -273,50 +285,10 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
class="_container_1ug7n_10"
>
<input
aria-describedby="radix-_r_a_ radix-_r_c_ radix-_r_e_"
class="_input_1ug7n_18"
id="radix-_r_9_"
name="_r_0_"
title=""
type="radio"
value="legacy"
/>
<div
class="_ui_1ug7n_19"
/>
</div>
</div>
<div
class="_inline-field-body_1o4d9_39"
>
<label
class="_label_1o4d9_60"
for="radix-_r_9_"
>
Legacy: state events & oldest membership SFU
</label>
<span
class="_message_1o4d9_86 _help-message_1o4d9_92"
id="radix-_r_a_"
>
Compatible with old versions of EC that do not support multi SFU
</span>
</div>
</div>
<div
class="_inline-field_1o4d9_33"
>
<div
class="_inline-field-control_1o4d9_45"
>
<div
class="_container_1ug7n_10"
>
<input
aria-describedby="radix-_r_a_ radix-_r_c_ radix-_r_e_"
aria-describedby="radix-_r_a_ radix-_r_c_"
checked=""
class="_input_1ug7n_18"
id="radix-_r_b_"
id="radix-_r_9_"
name="_r_0_"
title=""
type="radio"
@@ -332,13 +304,13 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
>
<label
class="_label_1o4d9_60"
for="radix-_r_b_"
for="radix-_r_9_"
>
Compatibility: state events & multi SFU
</label>
<span
class="_message_1o4d9_86 _help-message_1o4d9_92"
id="radix-_r_c_"
id="radix-_r_a_"
>
Compatible with homeservers that do not support sticky events (but all other EC clients are v0.17.0 or later)
</span>
@@ -354,9 +326,9 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
class="_container_1ug7n_10"
>
<input
aria-describedby="radix-_r_a_ radix-_r_c_ radix-_r_e_"
aria-describedby="radix-_r_a_ radix-_r_c_"
class="_input_1ug7n_18"
id="radix-_r_d_"
id="radix-_r_b_"
name="_r_0_"
title=""
type="radio"
@@ -372,13 +344,13 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
>
<label
class="_label_1o4d9_60"
for="radix-_r_d_"
for="radix-_r_b_"
>
Matrix 2.0: sticky events & multi SFU
</label>
<span
class="_message_1o4d9_86 _help-message_1o4d9_92"
id="radix-_r_e_"
id="radix-_r_c_"
>
Compatible only with homservers supporting sticky events and all EC clients v0.17.0 or later
</span>
@@ -463,6 +435,242 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
</p>
<ul />
</div>
<div
class="_separator_13qwf_8"
data-kind="primary"
data-orientation="horizontal"
role="separator"
/>
<h4>
Camera quality
</h4>
<div
class="_fieldRow_1bd8c0"
>
<div
class="_field_1bd8c0 _checkboxField_1bd8c0"
>
<input
aria-describedby="_r_i_"
id="cameraToggle"
type="checkbox"
/>
<label
for="cameraToggle"
>
<div
class="_checkbox_1bd8c0"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Advanced camera settings
</label>
<p
class="_description_1bd8c0"
id="_r_i_"
>
Configure resolution, framerate, bitrate, and codec for camera video. Changes apply on next call join.
</p>
</div>
</div>
<div
class="_separator_13qwf_8"
data-kind="primary"
data-orientation="horizontal"
role="separator"
/>
<h4>
Screen sharing
</h4>
<div
class="_fieldRow_1bd8c0"
>
<div
class="_field_1bd8c0 _checkboxField_1bd8c0"
>
<input
aria-describedby="_r_j_"
id="screenShareToggle"
type="checkbox"
/>
<label
for="screenShareToggle"
>
<div
class="_checkbox_1bd8c0"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Advanced screen share settings
</label>
<p
class="_description_1bd8c0"
id="_r_j_"
>
Configure resolution, framerate, bitrate, and codec for screen sharing
</p>
</div>
</div>
<div
class="_separator_13qwf_8"
data-kind="primary"
data-orientation="horizontal"
role="separator"
/>
<h4>
Audio processing
</h4>
<p>
Changes apply on next call join.
</p>
<div
class="_fieldRow_1bd8c0"
>
<div
class="_field_1bd8c0 _checkboxField_1bd8c0"
>
<input
aria-describedby="_r_k_"
checked=""
id="echoCancellation"
type="checkbox"
/>
<label
for="echoCancellation"
>
<div
class="_checkbox_1bd8c0"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Echo cancellation
</label>
</div>
</div>
<div
class="_fieldRow_1bd8c0"
>
<div
class="_field_1bd8c0 _checkboxField_1bd8c0"
>
<input
aria-describedby="_r_l_"
checked=""
id="noiseSuppression"
type="checkbox"
/>
<label
for="noiseSuppression"
>
<div
class="_checkbox_1bd8c0"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Noise suppression
</label>
</div>
</div>
<div
class="_fieldRow_1bd8c0"
>
<div
class="_field_1bd8c0 _checkboxField_1bd8c0"
>
<input
aria-describedby="_r_m_"
checked=""
id="autoGainControl"
type="checkbox"
/>
<label
for="autoGainControl"
>
<div
class="_checkbox_1bd8c0"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Automatic gain control
</label>
</div>
</div>
<div
class="_separator_13qwf_8"
data-kind="primary"
data-orientation="horizontal"
role="separator"
/>
<p>
Environment variables
</p>

View File

@@ -0,0 +1,111 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, beforeEach } from "vitest";
import {
Setting,
parseResolution,
seedSettingsFromConfig,
screenShareCodec,
screenShareResolution,
screenShareFramerate,
screenShareBitrate,
cameraCodec,
cameraResolution,
cameraFramerate,
cameraBitrate,
} from "./settings";
beforeEach(() => {
localStorage.clear();
});
describe("parseResolution", () => {
it("parses a WIDTHxHEIGHT string", () => {
expect(parseResolution("1920x1080")).toEqual({
width: 1920,
height: 1080,
});
});
it("parses non-standard resolutions", () => {
expect(parseResolution("640x360")).toEqual({ width: 640, height: 360 });
});
});
describe("Setting", () => {
describe("seedFromConfig", () => {
it("updates value when no localStorage value exists", () => {
const setting = new Setting<string>("test-seed", "default");
setting.seedFromConfig("from-config");
expect(setting.getValue()).toBe("from-config");
});
it("does not override an existing localStorage value", () => {
localStorage.setItem(
"matrix-setting-test-seed-existing",
JSON.stringify("user-set"),
);
const setting = new Setting<string>("test-seed-existing", "default");
expect(setting.getValue()).toBe("user-set");
setting.seedFromConfig("from-config");
expect(setting.getValue()).toBe("user-set");
});
});
});
describe("seedSettingsFromConfig", () => {
it("does nothing when config is undefined", () => {
const codecBefore = screenShareCodec.getValue();
seedSettingsFromConfig(undefined);
expect(screenShareCodec.getValue()).toBe(codecBefore);
});
it("seeds video codec to both camera and screen share", () => {
seedSettingsFromConfig({ video_codec: "av1" });
expect(screenShareCodec.getValue()).toBe("av1");
expect(cameraCodec.getValue()).toBe("av1");
});
it("seeds screen share settings from config", () => {
seedSettingsFromConfig({
screen_share: {
max_resolution: 720,
max_framerate: 15,
max_bitrate: 2_000_000,
},
});
expect(screenShareResolution.getValue()).toBe("1280x720");
expect(screenShareFramerate.getValue()).toBe(15);
expect(screenShareBitrate.getValue()).toBe(2_000_000);
});
it("seeds camera/video settings from config", () => {
seedSettingsFromConfig({
video: {
max_resolution: 1080,
max_framerate: 60,
max_bitrate: 4_000_000,
},
});
expect(cameraResolution.getValue()).toBe("1920x1080");
expect(cameraFramerate.getValue()).toBe(60);
expect(cameraBitrate.getValue()).toBe(4_000_000);
});
it("only seeds provided fields, leaves others at defaults", () => {
const defaultFramerate = screenShareFramerate.getValue();
seedSettingsFromConfig({
screen_share: {
max_bitrate: 3_000_000,
},
});
expect(screenShareBitrate.getValue()).toBe(3_000_000);
expect(screenShareFramerate.getValue()).toBe(defaultFramerate);
});
});

View File

@@ -21,6 +21,7 @@ export class Setting<T> {
this.key = `matrix-setting-${key}`;
const storedValue = localStorage.getItem(this.key);
this.hasStoredValue = storedValue !== null;
let initialValue = defaultValue;
if (storedValue !== null) {
try {
@@ -40,6 +41,7 @@ export class Setting<T> {
}
private readonly key: string;
private readonly hasStoredValue: boolean;
private readonly _value$: BehaviorSubject<T>;
private readonly _lastUpdateReason$: BehaviorSubject<string | null>;
@@ -54,6 +56,17 @@ export class Setting<T> {
public readonly getValue = (): T => {
return this._value$.getValue();
};
/**
* Update the setting's value from a config source, but only if the user
* hasn't explicitly set a value in localStorage. This lets admins set
* org-wide defaults in config.json that users can override.
*/
public seedFromConfig(value: T): void {
if (!this.hasStoredValue) {
this._value$.next(value);
}
}
}
/**
@@ -144,3 +157,130 @@ export const customLivekitUrl = new Setting<string | null>(
"custom-livekit-url",
null,
);
export type VideoCodec = "vp8" | "vp9" | "h264" | "av1";
/**
* Parse a "WIDTHxHEIGHT" resolution string into numeric width and height.
*/
export function parseResolution(res: string): {
width: number;
height: number;
} {
const [w, h] = res.split("x").map(Number);
return { width: w, height: h };
}
export const advancedScreenShare = new Setting<boolean>(
"advanced-screen-share",
false,
);
export const screenShareResolution = new Setting<string>(
"screen-share-resolution",
"1920x1080",
);
export const screenShareFramerate = new Setting<number>(
"screen-share-framerate",
30,
);
export const screenShareBitrate = new Setting<number>(
"screen-share-bitrate",
5_000_000,
);
export const screenShareCodec = new Setting<VideoCodec>(
"screen-share-codec",
"vp9",
);
// Camera video quality settings
export const advancedCamera = new Setting<boolean>("advanced-camera", false);
export const cameraResolution = new Setting<string>(
"camera-resolution",
"1280x720",
);
export const cameraFramerate = new Setting<number>("camera-framerate", 30);
export const cameraBitrate = new Setting<number>("camera-bitrate", 1_700_000);
export const cameraCodec = new Setting<VideoCodec>("camera-codec", "vp8");
// Audio processing settings
export const echoCancellationSetting = new Setting<boolean>(
"echo-cancellation",
true,
);
export const noiseSuppressionSetting = new Setting<boolean>(
"noise-suppression",
true,
);
export const autoGainControlSetting = new Setting<boolean>(
"auto-gain-control",
true,
);
/**
* Seed setting defaults from config.json's media_quality section.
* Call this after Config.init() has resolved.
* Only updates settings that the user hasn't explicitly set in localStorage.
*/
export function seedSettingsFromConfig(
mediaQuality:
| {
video_codec?: VideoCodec;
video?: {
max_resolution?: number;
max_bitrate?: number;
max_framerate?: number;
};
screen_share?: {
max_resolution?: number;
max_bitrate?: number;
max_framerate?: number;
};
}
| undefined,
): void {
if (!mediaQuality) return;
const codec = mediaQuality.video_codec;
if (codec) {
screenShareCodec.seedFromConfig(codec);
cameraCodec.seedFromConfig(codec);
}
const screen = mediaQuality.screen_share;
if (screen) {
if (screen.max_resolution) {
const width = Math.round((screen.max_resolution * 16) / 9);
screenShareResolution.seedFromConfig(`${width}x${screen.max_resolution}`);
}
if (screen.max_framerate) {
screenShareFramerate.seedFromConfig(screen.max_framerate);
}
if (screen.max_bitrate) {
screenShareBitrate.seedFromConfig(screen.max_bitrate);
}
}
const video = mediaQuality.video;
if (video) {
if (video.max_resolution) {
const width = Math.round((video.max_resolution * 16) / 9);
cameraResolution.seedFromConfig(`${width}x${video.max_resolution}`);
}
if (video.max_framerate) {
cameraFramerate.seedFromConfig(video.max_framerate);
}
if (video.max_bitrate) {
cameraBitrate.seedFromConfig(video.max_bitrate);
}
}
}

View File

@@ -19,7 +19,10 @@ import { useClient } from "../ClientContext";
import { Config } from "../config/Config";
import { type RageshakeRequestModal } from "../room/RageshakeRequestModal";
import { getUrlParams } from "../UrlParams";
import { deepCompare } from "matrix-js-sdk/lib/utils";
import { advancedCamera as advancedCameraSetting } from "./settings";
import { advancedScreenShare as advancedScreenShareSetting } from "./settings";
import { DEFAULT_CONFIG } from "../config/ConfigOptions";
const gzip = async (text: string): Promise<Blob> => {
// pako is relatively large (200KB), so we only import it when needed
const { gzip: pakoGzip } = await import("pako");
@@ -245,6 +248,20 @@ export function useSubmitRageshake(
}
}
// Add custom media related information to the rageshake issue description.
// Used to quickly identify issues due to untested configurations.
if (
!deepCompare(Config.get().media_quality, DEFAULT_CONFIG.media_quality)
) {
body.append("custom_media_quality_in_config", "true");
}
if (advancedCameraSetting.getValue()) {
body.append("devTools_advancedCameraSettings", "true");
}
if (advancedScreenShareSetting.getValue()) {
body.append("devTools_advancedScreenShareSetting", "true");
}
if (navigator.storage && navigator.storage.estimate) {
try {
const estimate: {

View File

@@ -22,6 +22,8 @@ import { useSubmitRageshake, getRageshakeSubmitUrl } from "./submit-rageshake";
import { ClientContextProvider } from "../ClientContext";
import { getUrlParams } from "../UrlParams";
import { mockConfig } from "../utils/test";
import { DEFAULT_CONFIG } from "../config/ConfigOptions";
import { advancedCamera, advancedScreenShare } from "./settings";
vi.mock("../UrlParams", () => ({ getUrlParams: vi.fn() }));
@@ -201,6 +203,60 @@ describe("useSubmitRageshake", () => {
});
});
describe("media quality metadata", () => {
const submitAndGetBody = async (): Promise<FormData> => {
const fetchFn = vi.fn().mockResolvedValue({
status: 200,
});
vi.stubGlobal("fetch", fetchFn);
renderWithMockClient(() => "https://rageshake.localhost/foo", false);
screen.getByTestId("submit").click();
await waitFor(() => {
expect(screen.getByTestId("sent").textContent).toBe("true");
});
return fetchFn.mock.calls[0][1].body as FormData;
};
beforeEach(() => {
vi.unstubAllGlobals();
});
afterEach(() => {
advancedCamera.setValue(advancedCamera.defaultValue);
advancedScreenShare.setValue(advancedScreenShare.defaultValue);
vi.clearAllMocks();
});
it("omits media quality fields when config and settings are default", async () => {
mockConfig({});
const body = await submitAndGetBody();
expect(body.get("custom_media_quality_in_config")).toBeNull();
expect(body.get("devTools_advancedCameraSettings")).toBeNull();
expect(body.get("devTools_advancedScreenShareSetting")).toBeNull();
});
it("includes custom_media_quality_in_config when media_quality differs from default", async () => {
mockConfig({
media_quality: {
...DEFAULT_CONFIG.media_quality,
video_codec: "h264",
},
});
const body = await submitAndGetBody();
expect(body.get("custom_media_quality_in_config")).toBe("true");
});
it("includes devTools flags when advanced media settings are enabled", async () => {
mockConfig({});
advancedCamera.setValue(true);
advancedScreenShare.setValue(true);
const body = await submitAndGetBody();
expect(body.get("devTools_advancedCameraSettings")).toBe("true");
expect(body.get("devTools_advancedScreenShareSetting")).toBe("true");
});
});
describe("when rageshake is not available", () => {
it("starts unsent", () => {
renderWithMockClient(() => undefined, false);

View File

@@ -258,11 +258,9 @@ function mockRingEvent(
} as unknown as { event_id: string } & IRTCNotificationContent;
}
describe.each([
[MatrixRTCMode.Legacy],
[MatrixRTCMode.Compatibility],
[MatrixRTCMode.Matrix_2_0],
])("CallViewModel (%s mode)", (mode) => {
const modes = [[MatrixRTCMode.Compatibility], [MatrixRTCMode.Matrix_2_0]];
describe.each(modes)("CallViewModel (%s mode)", (mode) => {
const withCallViewModel = withCallViewModelInMode(mode);
test("participants are retained during a focus switch", () => {

View File

@@ -79,7 +79,7 @@ import {
type ReactionInfo,
type ReactionOption,
} from "../../reactions";
import { shallowEquals } from "../../utils/array";
import { shallowEquals as shallowArrayEquals } from "../../utils/array";
import { type MediaDevices } from "../MediaDevices";
import { constant, type Behavior } from "../Behavior";
import { E2eeType } from "../../e2ee/e2eeType";
@@ -89,6 +89,7 @@ import { getUrlParams, HeaderStyle } from "../../UrlParams";
import { type ProcessorState } from "../../livekit/TrackProcessorContext";
import { ElementWidgetActions, widget } from "../../widget";
import {
layoutShallowEquals,
type Alignment,
type GridLayoutMedia,
type Layout,
@@ -365,6 +366,11 @@ export interface CallViewModel {
*/
overflowing$: Behavior<boolean>;
/**
* Whether modals such as settings and reactions should be accessible at all.
*/
showModals$: Behavior<boolean>;
settingsOpen$: Behavior<boolean>;
setSettingsOpen$: Behavior<(open: boolean) => void>;
@@ -435,7 +441,7 @@ export function createCallViewModel$(
const matrixRTCMode$ =
configMatrixRTCMode !== undefined
? constant(configMatrixRTCMode)
: (options.matrixRTCMode$ ?? constant(MatrixRTCMode.Legacy));
: (options.matrixRTCMode$ ?? constant(MatrixRTCMode.Compatibility));
// Each hbar seperates a block of input variables required for the CallViewModel to function.
// The outputs of this block is written under the hbar.
@@ -497,7 +503,6 @@ export function createCallViewModel$(
mode === MatrixRTCMode.Matrix_2_0
? JwtEndpointVersion.Matrix_2_0
: JwtEndpointVersion.Legacy,
useOldestMember: mode === MatrixRTCMode.Legacy,
}),
),
),
@@ -513,8 +518,6 @@ export function createCallViewModel$(
livekitKeyProvider,
getUrlParams().controlledAudioDevices,
options.livekitRoomFactory,
getUrlParams().echoCancellation,
getUrlParams().noiseSuppression,
);
const connectionManager = createConnectionManager$({
@@ -943,7 +946,7 @@ export function createCallViewModel$(
bins.sort(([, bin1], [, bin2]) => bin1 - bin2).map(([m]) => m),
);
}),
distinctUntilChanged(shallowEquals),
distinctUntilChanged(shallowArrayEquals),
),
);
@@ -1002,7 +1005,7 @@ export function createCallViewModel$(
const spotlight$ = scope.behavior<MediaViewModel[]>(
spotlightAndPip$.pipe(
map(({ spotlight }) => spotlight),
distinctUntilChanged<MediaViewModel[]>(shallowEquals),
distinctUntilChanged<MediaViewModel[]>(shallowArrayEquals),
),
);
@@ -1211,6 +1214,7 @@ export function createCallViewModel$(
}
return layout;
}),
distinctUntilChanged(),
scope.bind(),
)
.subscribe((orientation) => {
@@ -1450,6 +1454,11 @@ export function createCallViewModel$(
map((naturallyShowFooter) => naturallyShowFooter && showFooterUrlParams),
),
);
const showModals$ = scope.behavior(
windowMode$.pipe(map((mode) => mode !== "pip")),
);
const settingsOpen$ = new BehaviorSubject(false);
const setSettingsOpen$ = constant((open: boolean) => {
settingsOpen$.next(open);
@@ -1576,7 +1585,11 @@ export function createCallViewModel$(
* The layout of tiles in the call interface.
*/
const layout$ = scope.behavior<Layout>(
layoutInternals$.pipe(map(({ layout }) => layout)),
layoutInternals$.pipe(
map(({ layout }) => layout),
// Drop redundant layout updates before they would hit React.
distinctUntilChanged<Layout>(layoutShallowEquals),
),
);
const overflowing$ = scope.behavior<boolean>(
@@ -1822,6 +1835,7 @@ export function createCallViewModel$(
showNameTags$,
showHeader$: showHeader$,
showFooter$: showFooter$,
showModals$,
settingsOpen$: settingsOpen$,
setSettingsOpen$: setSettingsOpen$,
edgeToEdge$,

View File

@@ -138,9 +138,6 @@ export function withCallViewModel(mode: MatrixRTCMode) {
public getSyncState(): SyncState {
return syncState;
}
public getAccessToken(): string | null {
return "a-token";
}
})() as Partial<MatrixClient> as MatrixClient,
getMembers: () => roomMembers,
getMembersWithMembership: () => roomMembers,

View File

@@ -20,7 +20,6 @@ import {
afterAll,
beforeEach,
} from "vitest";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { BehaviorSubject, map, of } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { type LocalParticipant, type LocalTrack } from "livekit-client";
@@ -60,7 +59,7 @@ import {
initializeWidget();
const MATRIX_RTC_MODE = MatrixRTCMode.Legacy;
const MATRIX_RTC_MODE = MatrixRTCMode.Compatibility;
const getUrlParams = vi.hoisted(() => vi.fn(() => ({})));
vi.mock("../../../UrlParams", () => ({ getUrlParams }));
vi.mock("@livekit/components-core", () => ({
@@ -72,40 +71,10 @@ vi.mock("@livekit/components-core", () => ({
describe("LocalMembership", () => {
describe("enterRTCSession", () => {
it("It joins the correct Session", () => {
const focusFromOlderMembership = {
type: "livekit",
livekit_service_url: "http://my-oldest-member-service-url.com",
livekit_alias: "my-oldest-member-service-alias",
};
const focusConfigFromWellKnown = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
};
const focusConfigFromWellKnown2 = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url2.com",
};
const clientWellKnown = {
"org.matrix.msc4143.rtc_foci": [
focusConfigFromWellKnown,
focusConfigFromWellKnown2,
],
};
mockConfig({
livekit: { livekit_service_url: "http://my-default-service-url.com" },
});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockImplementation(
async (domain) => {
if (domain === "example.org") {
return Promise.resolve(clientWellKnown);
}
return Promise.resolve({});
},
);
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
@@ -120,10 +89,6 @@ describe("LocalMembership", () => {
},
},
memberships: [],
getFocusInUse: vi.fn().mockReturnValue(focusFromOlderMembership),
getOldestMembership: vi.fn().mockReturnValue({
getPreferredFoci: vi.fn().mockReturnValue([focusFromOlderMembership]),
}),
joinRTCSession: vi.fn(),
}) as unknown as MatrixRTCSession;
@@ -132,7 +97,7 @@ describe("LocalMembership", () => {
ownMemberMock,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
livekit_service_url: "http://my-livekit-service-url.com",
type: "livekit",
},
{
@@ -147,62 +112,13 @@ describe("LocalMembership", () => {
memberId: "@alice:example.org:DEVICE",
userId: "@alice:example.org",
},
[
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
],
undefined,
expect.objectContaining({
manageMediaKeys: true,
useLegacyMemberEvents: false,
}),
);
});
it("It should not fail with configuration error if homeserver config has livekit url but not fallback", () => {
mockConfig({});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({
"org.matrix.msc4143.rtc_foci": [
{
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
},
],
});
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn(),
joinRTCSession: vi.fn(),
}) as unknown as MatrixRTCSession;
enterRTCSession(
mockedSession,
ownMemberMock,
[],
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
livekit_service_url: "http://my-livekit-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
expect.objectContaining({ manageMediaKeys: true }),
);
});
});

View File

@@ -10,6 +10,7 @@ import {
ParticipantEvent,
type LocalParticipant,
type ScreenShareCaptureOptions,
type TrackPublishOptions,
RoomEvent,
MediaDeviceFailure,
} from "livekit-client";
@@ -53,6 +54,14 @@ import {
import { ElementWidgetActions, widget } from "../../../widget.ts";
import { getUrlParams } from "../../../UrlParams.ts";
import { PosthogAnalytics } from "../../../analytics/PosthogAnalytics.ts";
import {
advancedScreenShare,
screenShareResolution,
screenShareFramerate,
screenShareBitrate,
screenShareCodec,
parseResolution,
} from "../../../settings/settings.ts";
import { MatrixRTCMode } from "../../../config/ConfigOptions.ts";
import { Config } from "../../../config/Config.ts";
import {
@@ -108,8 +117,6 @@ export type LocalMemberState =
};
/*
* - get well known
* - get oldest membership
* - get transport to use
* - get openId + jwt token
* - wait for createTrack() call
@@ -719,6 +726,43 @@ export const createLocalMembership$ = ({
surfaceSwitching: "include",
systemAudio: "include",
};
let publishOptions: TrackPublishOptions | undefined;
if (advancedScreenShare.getValue()) {
// User has advanced screen share settings enabled
const { width, height } = parseResolution(
screenShareResolution.getValue(),
);
const fps = screenShareFramerate.getValue();
const bps = screenShareBitrate.getValue();
const codec = screenShareCodec.getValue();
screenshareSettings.resolution = {
width,
height,
frameRate: fps,
};
publishOptions = {
screenShareEncoding: {
maxBitrate: bps,
maxFramerate: fps,
},
videoCodec: codec,
};
} else {
// Fall back to config.json settings if available
const screenConf = Config.get().media_quality?.screen_share;
if (screenConf?.max_resolution) {
screenshareSettings.resolution = {
width: Math.round((screenConf.max_resolution * 16) / 9),
height: screenConf.max_resolution,
frameRate: screenConf.max_framerate ?? 30,
};
}
}
const targetScreenshareState = !sharingScreen$.value;
logger.info(
`toggleScreenSharing called. Switching ${
@@ -734,7 +778,11 @@ export const createLocalMembership$ = ({
// is still initializing or publishing tracks, because there's no
// technical reason to disallow this. LiveKit will publish if it can.
participant$.value
?.setScreenShareEnabled(targetScreenshareState, screenshareSettings)
?.setScreenShareEnabled(
targetScreenshareState,
screenshareSettings,
publishOptions,
)
.catch(logger.error);
};
}
@@ -802,9 +850,7 @@ export function enterRTCSession(
// have started tracking by the time calls start getting created.
// groupCallOTelMembership?.onJoinCall();
const { features, matrix_rtc_session: matrixRtcSessionConfig } = Config.get();
const useDeviceSessionMemberEvents =
features?.feature_use_device_session_member_events;
const { matrix_rtc_session: matrixRtcSessionConfig } = Config.get();
const { sendNotificationType: notificationType, callIntent } = getUrlParams();
const multiSFU =
matrixRTCMode === MatrixRTCMode.Compatibility ||
@@ -846,9 +892,6 @@ export function enterRTCSession(
notificationType,
callIntent,
manageMediaKeys: encryptMedia,
...(useDeviceSessionMemberEvents !== undefined && {
useLegacyMemberEvents: !useDeviceSessionMemberEvents,
}),
delayedLeaveEventRestartMs:
matrixRtcSessionConfig?.delayed_leave_event_restart_ms,
delayedLeaveEventDelayMs:

View File

@@ -13,7 +13,6 @@ import {
it,
type MockedObject,
vi,
type MockInstance,
} from "vitest";
import {
type CallMembership,
@@ -26,7 +25,6 @@ import {
mockConfig,
flushPromises,
ownMemberMock,
mockRtcMembership,
testScope,
} from "../../../utils/test";
import {
@@ -35,7 +33,7 @@ import {
type LocalTransportWithSFUConfig,
} from "./LocalTransport";
import { constant } from "../../Behavior";
import { Epoch, ObservableScope, trackEpoch } from "../../ObservableScope";
import { Epoch, ObservableScope } from "../../ObservableScope";
import {
MatrixRTCTransportMissingError,
FailToGetOpenIdToken,
@@ -58,13 +56,11 @@ describe("LocalTransport", () => {
const { advertised$, active$ } = createLocalTransport$({
scope: testScope(),
roomId: "!room:example.org",
useOldestMember: false,
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getAccessToken: vi.fn().mockReturnValue("access_token"),
getDomain: () => "",
getDomain: () => "example.org",
baseUrl: "example.org",
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
@@ -77,9 +73,11 @@ describe("LocalTransport", () => {
await flushPromises();
expect(() => advertised$.value).toThrow(
new MatrixRTCTransportMissingError(""),
new MatrixRTCTransportMissingError("example.org"),
);
expect(() => active$.value).toThrow(
new MatrixRTCTransportMissingError("example.org"),
);
expect(() => active$.value).toThrow(new MatrixRTCTransportMissingError(""));
});
it("throws FailToGetOpenIdToken when OpenID fetch fails", async () => {
@@ -100,13 +98,10 @@ describe("LocalTransport", () => {
const { advertised$, active$ } = createLocalTransport$({
scope,
roomId: "!example_room_id",
useOldestMember: false,
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
baseUrl: "https://lk.example.org",
// Use empty domain to skip .well-known and use config directly
getDomain: () => "",
getAccessToken: vi.fn().mockReturnValue("access_token"),
baseUrl: "https://example.org",
getDomain: () => "example.org",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getOpenIdToken: vi.fn(),
@@ -145,16 +140,14 @@ describe("LocalTransport", () => {
const { advertised$, active$ } = createLocalTransport$({
scope: testScope(),
roomId: "!room:example.org",
useOldestMember: false,
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getDomain: () => "",
getDomain: () => "example.org",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
baseUrl: "https://lk.example.org",
getAccessToken: vi.fn().mockReturnValue("access_token"),
baseUrl: "https://example.org",
},
ownMembershipIdentity: ownMemberMock,
forceJwtEndpoint: JwtEndpointVersion.Legacy,
@@ -187,129 +180,6 @@ describe("LocalTransport", () => {
});
});
describe("oldest member mode", () => {
const aliceTransport: LivekitTransportConfig = {
type: "livekit",
livekit_service_url: "https://alice.example.org",
};
const bobTransport: LivekitTransportConfig = {
type: "livekit",
livekit_service_url: "https://bob.example.org",
};
const aliceMembership = mockRtcMembership("@alice:example.org", "AAA", {
fociPreferred: [aliceTransport],
});
const bobMembership = mockRtcMembership("@bob:example.org", "BBB", {
fociPreferred: [bobTransport],
});
let openIdSpy: MockInstance<(typeof openIDSFU)["getSFUConfigWithOpenID"]>;
beforeEach(() => {
openIdSpy = vi
.spyOn(openIDSFU, "getSFUConfigWithOpenID")
.mockResolvedValue(openIdResponse);
});
it("updates active transport when oldest member changes", async () => {
// Initially, Alice is the only member
const memberships$ = new BehaviorSubject([aliceMembership]);
const scope = testScope();
const { advertised$, active$ } = createLocalTransport$({
scope,
roomId: "!example_room_id",
useOldestMember: true,
memberships$: scope.behavior(memberships$.pipe(trackEpoch())),
client: {
getDomain: () => "",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getAccessToken: vi.fn().mockReturnValue("access_token"),
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
baseUrl: "https://lk.example.org",
},
ownMembershipIdentity: ownMemberMock,
forceJwtEndpoint: JwtEndpointVersion.Legacy,
delayId$: constant("delay_id_mock"),
});
expect(active$.value).toBe(null);
await flushPromises();
// SFU config should've been fetched
expect(openIdSpy).toHaveBeenCalled();
// Alice's transport should be active and advertised
expect(active$.value?.transport).toStrictEqual(aliceTransport);
expect(advertised$.value).toStrictEqual(aliceTransport);
// Now Bob joins the call, but Alice is still the oldest member
openIdSpy.mockClear();
memberships$.next([aliceMembership, bobMembership]);
await flushPromises();
// No new SFU config should've been fetched
expect(openIdSpy).not.toHaveBeenCalled();
// Alice's transport should still be active and advertised
expect(active$.value?.transport).toStrictEqual(aliceTransport);
expect(advertised$.value).toStrictEqual(aliceTransport);
// Now Bob takes Alice's place as the oldest member
openIdSpy.mockClear();
memberships$.next([bobMembership, aliceMembership]);
// Active transport should reset to null until we have Bob's SFU config
expect(active$.value).toStrictEqual(null);
await flushPromises();
// Bob's SFU config should've been fetched
expect(openIdSpy).toHaveBeenCalled();
// Bob's transport should be active, but Alice's should remain advertised
// (since we don't want the change in oldest member to cause a wave of new
// state events)
expect(active$.value?.transport).toStrictEqual(bobTransport);
expect(advertised$.value).toStrictEqual(aliceTransport);
});
it("advertises preferred transport when no other member exists", async () => {
// Initially, there are no members
const memberships$ = new BehaviorSubject<CallMembership[]>([]);
const scope = testScope();
const { advertised$, active$ } = createLocalTransport$({
scope,
roomId: "!example_room_id",
useOldestMember: true,
memberships$: scope.behavior(memberships$.pipe(trackEpoch())),
client: {
getDomain: () => "",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () =>
Promise.resolve([aliceTransport]),
getAccessToken: vi.fn().mockReturnValue("access_token"),
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
baseUrl: "https://lk.example.org",
},
ownMembershipIdentity: ownMemberMock,
forceJwtEndpoint: JwtEndpointVersion.Legacy,
delayId$: constant("delay_id_mock"),
});
expect(active$.value).toBe(null);
await flushPromises();
// Our own preferred transport should be advertised
expect(advertised$.value).toStrictEqual(aliceTransport);
// No transport should be active however (there is still no oldest member)
expect(active$.value).toBe(null);
// Now Bob joins the call and becomes the oldest member
memberships$.next([bobMembership]);
await flushPromises();
// We should still advertise our own preferred transport (to avoid
// unnecessary state changes)
expect(advertised$.value).toStrictEqual(aliceTransport);
// Bob's transport should become active
expect(active$.value?.transport).toBe(bobTransport);
});
});
type LocalTransportProps = Parameters<typeof createLocalTransport$>[0];
describe("transport configuration mechanisms", () => {
@@ -324,16 +194,14 @@ describe("LocalTransport", () => {
ownMembershipIdentity: ownMemberMock,
scope: testScope(),
roomId: "!example_room_id",
useOldestMember: false,
forceJwtEndpoint: JwtEndpointVersion.Legacy,
delayId$: constant(null),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
baseUrl: "https://example.org",
getDomain: vi.fn().mockReturnValue(""),
getDomain: vi.fn().mockReturnValue("example.org"),
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: vi.fn().mockResolvedValue([]),
getAccessToken: vi.fn().mockReturnValue("access_token"),
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
@@ -421,42 +289,6 @@ describe("LocalTransport", () => {
});
});
it("Should not call _unstable_getRTCTransports in widget mode but use well-known", async () => {
mockConfig({
livekit: { livekit_service_url: "https://do-not-use.lk.example.org" },
});
localTransportOpts.client.getDomain.mockReturnValue("example.org");
fetchMock.getOnce("https://example.org/.well-known/matrix/client", {
"org.matrix.msc4143.rtc_foci": [
{
type: "livekit",
livekit_service_url: "https://use-me.jwt.call.example.org",
},
],
});
localTransportOpts.client.getAccessToken.mockReturnValue(null);
const { advertised$, active$ } =
createLocalTransport$(localTransportOpts);
openIdResolver.resolve?.(openIdResponse);
expect(advertised$.value).toBe(null);
expect(active$.value).toBe(null);
await flushPromises();
expect(
localTransportOpts.client._unstable_getRTCTransports,
).not.toHaveBeenCalled();
const expectedTransport = {
type: "livekit",
livekit_service_url: "https://use-me.jwt.call.example.org",
};
expect(advertised$.value).toStrictEqual(expectedTransport);
});
it("fails fast if the openID request fails for backend config", async () => {
localTransportOpts.client._unstable_getRTCTransports.mockResolvedValue([
{ type: "livekit", livekit_service_url: "https://lk.example.org" },
@@ -469,66 +301,19 @@ describe("LocalTransport", () => {
).rejects.toThrow(expect.any(FailToGetOpenIdToken));
});
it("supports getting transport via well-known", async () => {
localTransportOpts.client.getDomain.mockReturnValue("example.org");
fetchMock.getOnce("https://example.org/.well-known/matrix/client", {
"org.matrix.msc4143.rtc_foci": [
{ type: "livekit", livekit_service_url: "https://lk.example.org" },
],
});
const { advertised$, active$ } =
createLocalTransport$(localTransportOpts);
openIdResolver.resolve?.(openIdResponse);
expect(advertised$.value).toBe(null);
expect(active$.value).toBe(null);
await flushPromises();
const expectedTransport = {
livekit_service_url: "https://lk.example.org",
type: "livekit",
};
expect(advertised$.value).toStrictEqual(expectedTransport);
expect(active$.value).toStrictEqual({
transport: expectedTransport,
sfuConfig: {
jwt: "e30=.eyJzdWIiOiJAbWU6ZXhhbXBsZS5vcmc6QUJDREVGIiwidmlkZW8iOnsicm9vbSI6IiFleGFtcGxlX3Jvb21faWQifX0=.e30=",
livekitAlias: "Akph4alDMhen",
livekitIdentity: "@lk_user:ABCDEF",
url: "https://lk.example.org",
},
});
expect(fetchMock.done()).toEqual(true);
});
it("fails fast if the openId request fails for the well-known config", async () => {
localTransportOpts.client.getDomain.mockReturnValue("example.org");
fetchMock.getOnce("https://example.org/.well-known/matrix/client", {
"org.matrix.msc4143.rtc_foci": [
{ type: "livekit", livekit_service_url: "https://lk.example.org" },
],
});
openIdResolver.reject(
new FailToGetOpenIdToken(new Error("Test driven error")),
);
await expect(async () =>
lastValueFrom(createLocalTransport$(localTransportOpts).active$),
).rejects.toThrow(expect.any(FailToGetOpenIdToken));
});
it("throws if no options are available", async () => {
const { advertised$, active$ } = createLocalTransport$({
scope: testScope(),
ownMembershipIdentity: ownMemberMock,
roomId: "!example_room_id",
useOldestMember: false,
forceJwtEndpoint: JwtEndpointVersion.Legacy,
delayId$: constant(null),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "",
getDomain: () => "example.org",
baseUrl: "https://example.org",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getAccessToken: vi.fn().mockReturnValue("access_token"),
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
@@ -537,10 +322,10 @@ describe("LocalTransport", () => {
await flushPromises();
expect(() => advertised$.value).toThrow(
new MatrixRTCTransportMissingError(""),
new MatrixRTCTransportMissingError("example.org"),
);
expect(() => active$.value).toThrow(
new MatrixRTCTransportMissingError(""),
new MatrixRTCTransportMissingError("example.org"),
);
});
});
@@ -560,16 +345,14 @@ describe("LocalTransport", () => {
ownMembershipIdentity: ownMemberMock,
roomId: "!example_room_id",
// We want multi-sdu
useOldestMember: false,
forceJwtEndpoint: JwtEndpointVersion.Legacy,
delayId$: delayId$,
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "",
getDomain: () => "example.org",
baseUrl: "https://example.org",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getAccessToken: vi.fn().mockReturnValue("access_token"),
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),

View File

@@ -7,26 +7,18 @@ Please see LICENSE in the repository root for full details.
import {
type CallMembership,
isLivekitTransportConfig,
type LivekitTransportConfig,
} from "matrix-js-sdk/lib/matrixrtc";
import { type MatrixClient } from "matrix-js-sdk";
import {
catchError,
combineLatest,
distinctUntilChanged,
first,
from,
map,
merge,
type Observable,
of,
startWith,
switchMap,
tap,
} from "rxjs";
import { logger as rootLogger, type Logger } from "matrix-js-sdk/lib/logger";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { type Behavior } from "../../Behavior.ts";
@@ -48,8 +40,7 @@ import { RtcTransportAutoDiscovery } from "./RtcTransportAutoDiscovery.ts";
/*
* It figures out “which LiveKit focus URL/alias the local user should use,”
* optionally aligning with the oldest member, and ensures the SFU path is primed
* before advertising that choice.
* and ensures the SFU path is primed before advertising that choice.
*/
interface Props {
scope: ObservableScope;
@@ -57,12 +48,11 @@ interface Props {
memberships$: Behavior<Epoch<CallMembership[]>>;
client: Pick<
MatrixClient,
"getDomain" | "baseUrl" | "_unstable_getRTCTransports" | "getAccessToken"
"getDomain" | "baseUrl" | "_unstable_getRTCTransports"
> &
OpenIDClientParts;
// Used by the jwt service to create the livekit room and compute the livekit alias.
roomId: string;
useOldestMember: boolean;
forceJwtEndpoint: JwtEndpointVersion;
delayId$: Behavior<string | null>;
}
@@ -120,8 +110,6 @@ export interface LocalTransport {
/**
* Connects to the JWT service and determines the transports that the local member should use.
*
* @prop useOldestMember Whether to use the same transport as the oldest member.
* This will only update once the first oldest member appears. Will not recompute if the oldest member leaves.
* @prop useOldJwtEndpoint Whether to set forceOldJwtEndpoint on the returned transport and to use the old JWT endpoint.
* This is used when the connection manager needs to know if it has to use the legacy endpoint which implies a string concatenated rtcBackendIdentity.
* (which is expected for non sticky event based rtc member events)
@@ -134,23 +122,14 @@ export const createLocalTransport$ = ({
ownMembershipIdentity,
client,
roomId,
useOldestMember,
forceJwtEndpoint,
delayId$,
}: Props): LocalTransport => {
const logger = rootLogger.getChild("[LocalTransport]");
// The LiveKit transport in use by the oldest RTC membership. `null` when the
// oldest member has no such transport.
const oldestMemberTransport$ = observerOldestMembership$(
scope,
memberships$,
logger,
);
const transportDiscovery = new RtcTransportAutoDiscovery({
client: client,
resolvedConfig: Config.get(),
wellKnownFetcher: AutoDiscovery.getRawClientConfig.bind(AutoDiscovery),
logger: logger,
});
@@ -205,19 +184,6 @@ export const createLocalTransport$ = ({
}),
);
if (useOldestMember) {
return observeLocalTransportForOldestMembership(
scope,
oldestMemberTransport$,
preferredTransport$,
client,
ownMembershipIdentity,
roomId,
logger,
);
}
// --- Multi-SFU mode ---
// Always publish on and advertise the preferred transport.
return {
advertised$: scope.behavior(
@@ -245,47 +211,6 @@ export const createLocalTransport$ = ({
};
};
/**
* Observes the oldest member in the room and returns the transport that it uses if it is a livekit transport.
* @param scope - The observable scope.
* @param memberships$ - The observable of the call's memberships.'
*/
function observerOldestMembership$(
scope: ObservableScope,
memberships$: Behavior<Epoch<CallMembership[]>>,
logger: Logger,
): Behavior<LivekitTransportConfig | null> {
return scope.behavior<LivekitTransportConfig | null>(
memberships$.pipe(
map((memberships) => {
const oldestMember = memberships.value[0];
if (oldestMember === undefined) {
logger.info("Oldest member: not found");
return null;
}
const transport = oldestMember.getTransport(oldestMember);
if (transport === undefined) {
logger.warn(
`Oldest member: ${oldestMember.userId}|${oldestMember.deviceId}|${oldestMember.memberId} has no transport`,
);
return null;
}
if (!isLivekitTransportConfig(transport)) {
logger.warn(
`Oldest member: ${oldestMember.userId}|${oldestMember.deviceId}|${oldestMember.memberId} has invalid transport`,
);
return null;
}
logger.info(
"Oldest member: ${oldestMember.userId}|${oldestMember.deviceId}|${oldestMember.memberId} has valid transport",
);
return transport;
}),
distinctUntilChanged(areLivekitTransportsEqual),
),
);
}
/**
* Utility to ensure the user can authenticate with the SFU.
* We will call `getSFUConfigWithOpenID` once per transport here as it's our
@@ -309,7 +234,7 @@ async function doOpenIdAndJWTFromUrl(
roomId: string,
client: Pick<
MatrixClient,
"getDomain" | "baseUrl" | "_unstable_getRTCTransports" | "getAccessToken"
"getDomain" | "baseUrl" | "_unstable_getRTCTransports"
> &
OpenIDClientParts,
delayId?: string,
@@ -333,85 +258,6 @@ async function doOpenIdAndJWTFromUrl(
};
}
function observeLocalTransportForOldestMembership(
scope: ObservableScope,
oldestMemberTransport$: Behavior<LivekitTransportConfig | null>,
preferredTransport$: Observable<LocalTransportWithSFUConfig>,
client: Pick<
MatrixClient,
"getDomain" | "baseUrl" | "_unstable_getRTCTransports" | "getAccessToken"
> &
OpenIDClientParts,
ownMembershipIdentity: CallMembershipIdentityParts,
roomId: string,
logger: Logger,
): LocalTransport {
// Ensure we can authenticate with the SFU.
const authenticatedOldestMemberTransport$ = oldestMemberTransport$.pipe(
switchMap((transport) => {
// Oldest member not available -we are first- (or invalid SFU config).
if (transport === null) return of(null);
// Whenever there is transport change we want to revert
// to no transport while we do the authentication.
// So do a from(promise) here to be able to startWith(null)
return from(
doOpenIdAndJWTFromUrl(
transport,
JwtEndpointVersion.Legacy,
ownMembershipIdentity,
roomId,
client,
undefined,
logger,
),
).pipe(
catchError((e: unknown) => {
logger.error(
`Failed to authenticate to transport ${transport.livekit_service_url}`,
e,
);
throw mapAuthErrorToUserFriendlyError(e);
}),
startWith(null),
);
}),
);
// --- Oldest member mode ---
return {
// Never update the transport that we advertise in our membership. Just
// take the first valid oldest member or preferred transport that we learn
// about, and stick with that. This avoids unnecessary SFU hops and room
// state changes.
advertised$: scope.behavior(
merge(
authenticatedOldestMemberTransport$.pipe(
map((t) => t?.transport ?? null),
),
preferredTransport$.pipe(map((t) => t.transport)),
).pipe(
first((t) => t !== null),
tap((t) =>
logger.info(`Advertise transport: ${t.livekit_service_url}`),
),
),
null,
),
// Publish on the transport used by the oldest member.
active$: scope.behavior(
authenticatedOldestMemberTransport$.pipe(
tap((t) =>
logger.info(
`Publish on transport: ${t?.transport.livekit_service_url}`,
),
),
),
null,
),
};
}
function mapAuthErrorToUserFriendlyError(e: unknown): Error {
if (
e instanceof FailToGetOpenIdToken ||

View File

@@ -13,7 +13,7 @@ import {
type MockedObject,
vi,
} from "vitest";
import { type IClientWellKnown, MatrixError } from "matrix-js-sdk";
import { MatrixError } from "matrix-js-sdk";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import {
type LivekitTransportConfig,
@@ -33,9 +33,9 @@ const backendTransport: LivekitTransportConfig = {
livekit_service_url: "https://backend.example.org",
};
const wellKnownTransport: LivekitTransportConfig = {
const configTransport: LivekitTransportConfig = {
type: "livekit",
livekit_service_url: "https://well-known.example.org",
livekit_service_url: "https://config.example.org",
};
function makeClient(): MockedObject<DiscoveryClient> {
@@ -43,7 +43,6 @@ function makeClient(): MockedObject<DiscoveryClient> {
getDomain: vi.fn().mockReturnValue("example.org"),
baseUrl: "https://matrix.example.org",
_unstable_getRTCTransports: vi.fn().mockResolvedValue([]),
getAccessToken: vi.fn().mockReturnValue("access_token"),
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
} as unknown as MockedObject<DiscoveryClient>;
@@ -59,12 +58,6 @@ function makeResolvedConfig(livekitServiceUrl?: string): ResolvedConfigOptions {
} as ResolvedConfigOptions;
}
function makeWellKnown(rtcFoci?: Transport[]): IClientWellKnown {
return {
"org.matrix.msc4143.rtc_foci": rtcFoci,
} as unknown as IClientWellKnown;
}
describe("RtcTransportAutoDiscovery", () => {
beforeEach(() => {
vi.clearAllMocks();
@@ -75,33 +68,27 @@ describe("RtcTransportAutoDiscovery", () => {
{ transports: [{ type: "not_livekit" }, backendTransport] },
];
it.each(VALID_TEST_CASES)(
"prefers backend transport over well-known and app config $transports",
"prefers backend transport other app config $transports",
async ({ transports }) => {
// it("prefers backend transport over well-known and app config", async () => {
const client = makeClient();
client._unstable_getRTCTransports.mockResolvedValue(transports);
const wellKnownFetcher = vi
.fn<(domain: string) => Promise<IClientWellKnown>>()
.mockResolvedValue(makeWellKnown([wellKnownTransport]));
const discovery = new RtcTransportAutoDiscovery({
client,
resolvedConfig: makeResolvedConfig("https://config.example.org"),
wellKnownFetcher,
resolvedConfig: makeResolvedConfig(configTransport.livekit_service_url),
logger: rootLogger,
});
await expect(
discovery.discoverPreferredTransport(),
).resolves.toStrictEqual(backendTransport);
const discoveredTransport = await discovery.discoverPreferredTransport();
expect(discoveredTransport).toStrictEqual(backendTransport);
expect(discoveredTransport).not.toStrictEqual(configTransport);
expect(client._unstable_getRTCTransports).toHaveBeenCalledTimes(1);
expect(wellKnownFetcher).not.toHaveBeenCalled();
},
);
it("Retries limit_exceeded backend transport over well-known", async () => {
it("Retries limit_exceeded backend transport", async () => {
const client = makeClient();
client._unstable_getRTCTransports
.mockRejectedValueOnce(
@@ -116,14 +103,9 @@ describe("RtcTransportAutoDiscovery", () => {
)
.mockResolvedValue([backendTransport]);
const wellKnownFetcher = vi
.fn<(domain: string) => Promise<IClientWellKnown>>()
.mockResolvedValue(makeWellKnown([wellKnownTransport]));
const discovery = new RtcTransportAutoDiscovery({
client,
resolvedConfig: makeResolvedConfig("https://config.example.org"),
wellKnownFetcher,
logger: rootLogger,
});
@@ -132,7 +114,6 @@ describe("RtcTransportAutoDiscovery", () => {
);
expect(client._unstable_getRTCTransports).toHaveBeenCalledTimes(2);
expect(wellKnownFetcher).not.toHaveBeenCalled();
});
const INVALID_TEST_CASES: Array<{ transports: Transport[] }> = [
@@ -140,91 +121,30 @@ describe("RtcTransportAutoDiscovery", () => {
{ transports: [{ type: "not_livekit" }] },
];
it.each(INVALID_TEST_CASES)(
"falls back to well-known when backend has no (valid) livekit transports $transports",
"falls back to config when backend has no (valid) livekit transports $transports",
async ({ transports }) => {
const client = makeClient();
client._unstable_getRTCTransports.mockResolvedValue(transports);
const wellKnownFetcher = vi
.fn<(domain: string) => Promise<IClientWellKnown>>()
.mockResolvedValue(makeWellKnown([wellKnownTransport]));
const discovery = new RtcTransportAutoDiscovery({
client,
resolvedConfig: makeResolvedConfig("https://config.example.org"),
wellKnownFetcher,
resolvedConfig: makeResolvedConfig(configTransport.livekit_service_url),
logger: rootLogger,
});
await expect(
discovery.discoverPreferredTransport(),
).resolves.toStrictEqual(wellKnownTransport);
expect(wellKnownFetcher).toHaveBeenCalledWith("example.org");
const discoveredTransport = await discovery.discoverPreferredTransport();
expect(discoveredTransport).not.toStrictEqual(backendTransport);
expect(discoveredTransport).toStrictEqual(configTransport);
},
);
it("skips backend discovery in widget mode and uses well-known", async () => {
const client = makeClient();
// widget mode is detected by the absence of an access token
client.getAccessToken.mockReturnValue(null);
const wellKnownFetcher = vi
.fn<(domain: string) => Promise<IClientWellKnown>>()
.mockResolvedValue(makeWellKnown([wellKnownTransport]));
const discovery = new RtcTransportAutoDiscovery({
client,
resolvedConfig: makeResolvedConfig("https://config.example.org"),
wellKnownFetcher,
logger: rootLogger,
});
await expect(discovery.discoverPreferredTransport()).resolves.toStrictEqual(
wellKnownTransport,
);
expect(client._unstable_getRTCTransports).not.toHaveBeenCalled();
expect(wellKnownFetcher).toHaveBeenCalledWith("example.org");
});
it("falls back to app config when backend fails and well-known has no rtc_foci", async () => {
const client = makeClient();
client._unstable_getRTCTransports.mockRejectedValue(
new MatrixError({ errcode: "M_UNKNOWN" }, 404),
);
const wellKnownFetcher = vi
.fn<(domain: string) => Promise<IClientWellKnown>>()
.mockResolvedValue({} as IClientWellKnown);
const discovery = new RtcTransportAutoDiscovery({
client,
resolvedConfig: makeResolvedConfig("https://config.example.org"),
wellKnownFetcher,
logger: rootLogger,
});
await expect(discovery.discoverPreferredTransport()).resolves.toStrictEqual(
{
type: "livekit",
livekit_service_url: "https://config.example.org",
},
);
});
it("returns null when backend, well-known and config are all unavailable", async () => {
it("returns null when backend and config are all unavailable", async () => {
const client = makeClient();
client._unstable_getRTCTransports.mockResolvedValue([]);
const wellKnownFetcher = vi
.fn<(domain: string) => Promise<IClientWellKnown>>()
.mockResolvedValue({} as IClientWellKnown);
const discovery = new RtcTransportAutoDiscovery({
client,
resolvedConfig: makeResolvedConfig(undefined),
wellKnownFetcher,
logger: rootLogger,
});

View File

@@ -8,7 +8,7 @@ import {
isLivekitTransportConfig,
type LivekitTransportConfig,
} from "matrix-js-sdk/lib/matrixrtc";
import { type IClientWellKnown, type MatrixClient } from "matrix-js-sdk";
import { type MatrixClient } from "matrix-js-sdk";
import { type Logger } from "matrix-js-sdk/lib/logger";
import type { ResolvedConfigOptions } from "../../../config/ConfigOptions.ts";
@@ -16,33 +16,27 @@ import { doNetworkOperationWithRetry } from "../../../utils/matrix.ts";
type TransportDiscoveryClient = Pick<
MatrixClient,
"getDomain" | "_unstable_getRTCTransports" | "getAccessToken"
"getDomain" | "_unstable_getRTCTransports"
>;
export interface RtcTransportAutoDiscoveryProps {
client: TransportDiscoveryClient;
resolvedConfig: ResolvedConfigOptions;
wellKnownFetcher: (domain: string) => Promise<IClientWellKnown>;
logger: Logger;
}
export class RtcTransportAutoDiscovery {
private readonly client: TransportDiscoveryClient;
private readonly resolvedConfig: ResolvedConfigOptions;
private readonly wellKnownFetcher: (
domain: string,
) => Promise<IClientWellKnown>;
private readonly logger: Logger;
public constructor({
client,
resolvedConfig,
wellKnownFetcher,
logger,
}: RtcTransportAutoDiscoveryProps) {
this.client = client;
this.resolvedConfig = resolvedConfig;
this.wellKnownFetcher = wellKnownFetcher;
this.logger = logger.getChild("[RtcTransportAutoDiscovery]");
}
@@ -56,21 +50,7 @@ export class RtcTransportAutoDiscovery {
return backendTransport;
}
this.logger.info("No backend transport found, falling back to well-known");
// 2) .well-known transports
const wellKnownTransport = await this.tryWellKnownTransports();
if (wellKnownTransport) {
this.logger.info(
`Found .well-known transport: ${wellKnownTransport.livekit_service_url}`,
);
return wellKnownTransport;
}
this.logger.info(
"No .well-known transport found, falling back to app config",
);
// 3) app config URL
// 2) app config URL
const configTransport = this.tryConfigTransport();
if (configTransport) {
this.logger.info(
@@ -90,72 +70,23 @@ export class RtcTransportAutoDiscovery {
private async tryBackendTransports(): Promise<LivekitTransportConfig | null> {
const client = this.client;
// MSC4143: Attempt to fetch transports from backend.
// TODO: Workaround for an issue in the js-sdk RoomWidgetClient that
// is not yet implementing _unstable_getRTCTransports properly (via widget API new action).
// For now we just skip this call if we are in a widget.
// In widget mode the client is a `RoomWidgetClient` which has no access token (it is using the widget API).
// Could be removed once the js-sdk is fixed (https://github.com/matrix-org/matrix-js-sdk/issues/5245)
const isSPA = !!client.getAccessToken();
if (isSPA && "_unstable_getRTCTransports" in client) {
this.logger.info("First try to use getRTCTransports end point ...");
try {
const transportList = await doNetworkOperationWithRetry(async () =>
client._unstable_getRTCTransports(),
);
const first = transportList.find(isLivekitTransportConfig);
if (first) {
return first;
} else {
this.logger.info(
`No livekit transport found in getRTCTransports end point`,
transportList,
);
}
} catch (ex) {
this.logger.info(`Failed to use getRTCTransports end point: ${ex}`);
}
} else {
this.logger.debug(`getRTCTransports end point not available`);
}
return null;
}
/**
* Fetches the first rtc_foci from the .well-known/matrix/client.
* This will not throw errors, but instead just log them and return null if the expected config is not found or malformed.
* @private
*/
private async tryWellKnownTransports(): Promise<LivekitTransportConfig | null> {
// Legacy MSC4143 (to be removed) WELL_KNOWN: Prioritize the .well-known/matrix/client, if available.
const client = this.client;
const domain = client.getDomain();
if (domain) {
// we use AutoDiscovery instead of relying on the MatrixClient having already
// been fully configured and started
const wellKnownFoci = await this.wellKnownFetcher(domain);
const fociConfig = wellKnownFoci["org.matrix.msc4143.rtc_foci"];
if (fociConfig) {
if (!Array.isArray(fociConfig)) {
this.logger.warn(
`org.matrix.msc4143.rtc_foci is not an array in .well-known`,
);
} else {
return fociConfig[0];
}
this.logger.info("First try to use getRTCTransports end point ...");
try {
const transportList = await doNetworkOperationWithRetry(async () =>
client._unstable_getRTCTransports(),
);
const first = transportList.find(isLivekitTransportConfig);
if (first) {
return first;
} else {
this.logger.info(
`No .well-known "org.matrix.msc4143.rtc_foci" found for ${domain}`,
wellKnownFoci,
`No livekit transport found in getRTCTransports end point`,
transportList,
);
}
} else {
// Should never happen, but just in case
this.logger.warn(`No domain configured for client`);
} catch (ex) {
this.logger.info(`Failed to use getRTCTransports end point: ${ex}`);
}
return null;
}

View File

@@ -36,7 +36,6 @@ import {
SFURoomCreationRestrictedError,
UnknownCallError,
} from "../../../utils/errors.ts";
import { type JwtEndpointVersion } from "../localMember/LocalTransport.ts";
export interface ConnectionOpts {
/**
@@ -44,11 +43,6 @@ export interface ConnectionOpts {
* On top the local transport will send additional data to the jwt server to use delayed event delegation.
*/
existingSFUConfig?: SFUConfig;
/**
* For local connections that use the oldest member pattern. here we have not prefetched the sfuConfig
* and hence we need to let the connection do the jwt token fetching.
*/
forceJwtEndpoint?: JwtEndpointVersion;
/** The identity parts to use on this connection */
ownMembershipIdentity: CallMembershipIdentityParts;
/** The media transport to connect to. */

View File

@@ -12,7 +12,7 @@ import {
type E2EEManagerOptions,
type BaseE2EEManager,
} from "livekit-client";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { logger, type Logger } from "matrix-js-sdk/lib/logger";
// imported as inline to support worker when loaded from a cdn (cross domain)
import E2EEWorker from "livekit-client/e2ee-worker?worker&inline";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
@@ -27,7 +27,18 @@ import type {
import type { MediaDevices } from "../../MediaDevices.ts";
import type { Behavior } from "../../Behavior.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { defaultLiveKitOptions } from "../../../livekit/options.ts";
import { getLiveKitOptions } from "../../../livekit/options.ts";
import {
advancedCamera,
cameraResolution,
cameraFramerate,
cameraBitrate,
cameraCodec,
parseResolution,
echoCancellationSetting,
noiseSuppressionSetting,
autoGainControlSetting,
} from "../../../settings/settings.ts";
// TODO evaluate if this should be done like the Publisher Factory
export interface ConnectionFactory {
@@ -53,8 +64,6 @@ export class ECConnectionFactory implements ConnectionFactory {
* @param livekitKeyProvider - Optional key provider for end-to-end encryption.
* @param controlledAudioDevices - Option to indicate whether audio output device is controlled externally (native mobile app).
* @param livekitRoomFactory - Optional factory function (for testing) to create LivekitRoom instances. If not provided, a default factory is used.
* @param echoCancellation - Whether to enable echo cancellation for audio capture.
* @param noiseSuppression - Whether to enable noise suppression for audio capture.
*/
public constructor(
private client: OpenIDClientParts,
@@ -64,25 +73,22 @@ export class ECConnectionFactory implements ConnectionFactory {
livekitKeyProvider: BaseKeyProvider | undefined,
private controlledAudioDevices: boolean,
livekitRoomFactory?: () => LivekitRoom,
echoCancellation: boolean = true,
noiseSuppression: boolean = true,
) {
const defaultFactory = (): LivekitRoom =>
new LivekitRoom(
generateRoomOption({
devices: this.devices,
processorState: this.processorState$.value,
e2eeLivekitOptions: livekitKeyProvider && {
keyProvider: livekitKeyProvider,
// It's important that every room use a separate E2EE worker.
// They get confused if given streams from multiple rooms.
worker: new E2EEWorker(),
},
controlledAudioDevices: this.controlledAudioDevices,
echoCancellation,
noiseSuppression,
}),
);
const defaultFactory = (): LivekitRoom => {
const roomOptions = generateRoomOption({
devices: this.devices,
processorState: this.processorState$.value,
e2eeLivekitOptions: livekitKeyProvider && {
keyProvider: livekitKeyProvider,
// It's important that every room use a separate E2EE worker.
// They get confused if given streams from multiple rooms.
worker: new E2EEWorker(),
},
controlledAudioDevices: this.controlledAudioDevices,
});
logger.info("[ECConnectionFactory] livekit room options: ", roomOptions);
return new LivekitRoom(roomOptions);
};
this.livekitRoomFactory = livekitRoomFactory ?? defaultFactory;
}
@@ -119,14 +125,13 @@ export class ECConnectionFactory implements ConnectionFactory {
/**
* Generate the initial LiveKit RoomOptions based on the current media devices and processor state.
* Reads audio processing and camera quality settings directly from Settings.
*/
function generateRoomOption({
devices,
processorState,
e2eeLivekitOptions,
controlledAudioDevices,
echoCancellation,
noiseSuppression,
}: {
devices: MediaDevices;
processorState: ProcessorState;
@@ -135,21 +140,44 @@ function generateRoomOption({
| { e2eeManager: BaseE2EEManager }
| undefined;
controlledAudioDevices: boolean;
echoCancellation: boolean;
noiseSuppression: boolean;
}): RoomOptions {
const liveKitOptions = getLiveKitOptions();
// Apply advanced camera settings if enabled
let videoCaptureDefaults = {
...liveKitOptions.videoCaptureDefaults,
deviceId: devices.videoInput.selected$.value?.id,
processor: processorState.processor,
};
let publishDefaults = liveKitOptions.publishDefaults;
if (advancedCamera.getValue()) {
const { width, height } = parseResolution(cameraResolution.getValue());
const fps = cameraFramerate.getValue();
const bps = cameraBitrate.getValue();
const codec = cameraCodec.getValue();
videoCaptureDefaults = {
...videoCaptureDefaults,
resolution: { width, height, frameRate: fps },
};
publishDefaults = {
...publishDefaults,
videoEncoding: { maxBitrate: bps, maxFramerate: fps },
videoCodec: codec,
};
}
return {
...defaultLiveKitOptions,
videoCaptureDefaults: {
...defaultLiveKitOptions.videoCaptureDefaults,
deviceId: devices.videoInput.selected$.value?.id,
processor: processorState.processor,
},
...liveKitOptions,
videoCaptureDefaults,
publishDefaults,
audioCaptureDefaults: {
...defaultLiveKitOptions.audioCaptureDefaults,
...liveKitOptions.audioCaptureDefaults,
deviceId: devices.audioInput.selected$.value?.id,
echoCancellation,
noiseSuppression,
echoCancellation: echoCancellationSetting.getValue(),
noiseSuppression: noiseSuppressionSetting.getValue(),
autoGainControl: autoGainControlSetting.getValue(),
},
audioOutput: {
// When using controlled audio devices, we don't want to set the

View File

@@ -22,6 +22,16 @@ import {
} from "../../../utils/test.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { constant } from "../../Behavior";
import {
echoCancellationSetting,
noiseSuppressionSetting,
autoGainControlSetting,
advancedCamera,
cameraResolution,
cameraFramerate,
cameraBitrate,
cameraCodec,
} from "../../../settings/settings.ts";
// At the top of your test file, after imports
vi.mock("livekit-client", async (importOriginal) => {
@@ -58,11 +68,14 @@ describe("ECConnectionFactory - Audio inputs options", () => {
{ echo: false, noise: true },
{ echo: false, noise: false },
])(
"it sets echoCancellation=$echo and noiseSuppression=$noise based on constructor parameters",
"it sets echoCancellation=$echo and noiseSuppression=$noise based on settings",
({ echo, noise }) => {
// test("it sets echoCancellation and noiseSuppression based on constructor parameters", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
// Set audio processing settings
echoCancellationSetting.setValue(echo);
noiseSuppressionSetting.setValue(noise);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
@@ -73,9 +86,6 @@ describe("ECConnectionFactory - Audio inputs options", () => {
}),
undefined,
false,
undefined,
echo,
noise,
);
ecConnectionFactory.createConnection(
testScope,
@@ -101,9 +111,13 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
test.each([{ controlled: true }, { controlled: false }])(
"it sets controlledAudioDevice=$controlled then uses deviceId accordingly",
({ controlled }) => {
// test("it sets echoCancellation and noiseSuppression based on constructor parameters", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
// Explicitly set audio settings so the test doesn't depend on defaults
echoCancellationSetting.setValue(true);
noiseSuppressionSetting.setValue(true);
autoGainControlSetting.setValue(true);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
@@ -120,9 +134,6 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
}),
undefined,
controlled,
undefined,
false,
false,
);
ecConnectionFactory.createConnection(
testScope,
@@ -143,6 +154,114 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
);
});
describe("ECConnectionFactory - Camera quality settings", () => {
test("it uses default video options when advancedCamera is disabled", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
advancedCamera.setValue(false);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
);
ecConnectionFactory.createConnection(
testScope,
exampleTransport,
ownMemberMock,
logger,
);
// publishDefaults should use config defaults (vp8), not custom settings
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
publishDefaults: expect.objectContaining({
videoCodec: "vp8",
}),
}),
);
});
test("it applies custom camera resolution, encoding, and codec when advancedCamera is enabled", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
advancedCamera.setValue(true);
cameraResolution.setValue("1920x1080");
cameraFramerate.setValue(60);
cameraBitrate.setValue(4_000_000);
cameraCodec.setValue("vp9");
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
);
ecConnectionFactory.createConnection(
testScope,
exampleTransport,
ownMemberMock,
logger,
);
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
videoCaptureDefaults: expect.objectContaining({
resolution: { width: 1920, height: 1080, frameRate: 60 },
}),
publishDefaults: expect.objectContaining({
videoEncoding: { maxBitrate: 4_000_000, maxFramerate: 60 },
videoCodec: "vp9",
}),
}),
);
});
test("it applies autoGainControl from settings", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
autoGainControlSetting.setValue(false);
echoCancellationSetting.setValue(true);
noiseSuppressionSetting.setValue(true);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
);
ecConnectionFactory.createConnection(
testScope,
exampleTransport,
ownMemberMock,
logger,
);
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
audioCaptureDefaults: expect.objectContaining({
autoGainControl: false,
}),
}),
);
});
});
afterEach(() => {
testScope.end();
fetchMock.reset();

View File

@@ -35,11 +35,7 @@ vi.mock("../widget", () => ({
},
}));
it.each([
[MatrixRTCMode.Legacy],
[MatrixRTCMode.Compatibility],
[MatrixRTCMode.Matrix_2_0],
])(
it.each([[MatrixRTCMode.Compatibility], [MatrixRTCMode.Matrix_2_0]])(
"expect leave when ElementWidgetActions.HangupCall is called (%s mode)",
async (mode) => {
const pr = Promise.withResolvers<string>();

View File

@@ -0,0 +1,74 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { test, expect } from "vitest";
import {
type Alignment,
layoutShallowEquals,
type Layout,
} from "./layout-types";
import {
type SpotlightTileViewModel,
type GridTileViewModel,
} from "./TileViewModel";
import { BehaviorSubject } from "rxjs";
const spotlightTile = {} as unknown as SpotlightTileViewModel;
const gridTile = {} as unknown as GridTileViewModel;
const pipAlignment$ = new BehaviorSubject<Alignment>({
inline: "end",
block: "end",
});
const spotlightExpanded: Layout = {
type: "spotlight-expanded",
spotlight: spotlightTile,
pipAlignment$,
};
const spotlightPortrait: Layout = {
type: "spotlight-portrait",
spotlight: spotlightTile,
grid: [gridTile],
setVisibleTiles: () => {},
};
test("layoutShallowEquals considers a layout to be equal to its shallow clone", () =>
expect(layoutShallowEquals(spotlightExpanded, { ...spotlightExpanded })).toBe(
true,
));
test("layoutShallowEquals detects a missing key", () => {
expect(
layoutShallowEquals(spotlightExpanded, {
...spotlightExpanded,
pip: gridTile,
}),
).toBe(false);
expect(
layoutShallowEquals(
{ ...spotlightExpanded, pip: gridTile },
spotlightExpanded,
),
).toBe(false);
});
test("layoutShallowEquals considers grid arrays with equal contents to be equal", () =>
expect(
layoutShallowEquals(spotlightPortrait, {
...spotlightPortrait,
grid: [...spotlightPortrait.grid],
}),
).toBe(true));
test("layoutShallowEquals detects grid arrays with different contents", () =>
expect(
layoutShallowEquals(spotlightPortrait, {
...spotlightPortrait,
grid: [...spotlightPortrait.grid, gridTile],
}),
).toBe(false));

View File

@@ -16,6 +16,7 @@ import {
type SpotlightTileViewModel,
} from "./TileViewModel.ts";
import { type Behavior } from "./Behavior.ts";
import { shallowEquals as arrayShallowEquals } from "../utils/array.ts";
export interface GridLayoutMedia {
type: "grid";
@@ -140,3 +141,30 @@ export type Layout =
| OneOnOneDesktopLayout
| OneOnOneMobileLayout
| PipLayout;
/**
* Tests whether the top-level properties and array elements of layout `a` are
* equal to those of layout `b`. Useful for deduping redundant layout updates.
*/
export function layoutShallowEquals(a: Layout, b: Layout): boolean {
// If a and b have the same number of keys and every key in a is also in b,
// then they have the same keys.
const aKeys = Object.keys(a);
const bKeys = Object.keys(b);
if (aKeys.length !== bKeys.length) return false;
for (const key of aKeys) {
if (!(key in b)) return false;
// Now check that they have the same values.
const aValue = (a as any)[key];
const bValue = (b as any)[key];
if (Array.isArray(aValue) && Array.isArray(bValue)) {
// Special case for arrays so we can detect when the grid tiles arrays are
// essentially the same.
if (!arrayShallowEquals(aValue, bValue)) return false;
} else if (aValue !== bValue) return false;
}
return true;
}

View File

@@ -33,7 +33,6 @@ import { type RemoteUserMediaViewModel } from "./RemoteUserMediaViewModel";
import { type ObservableScope } from "../ObservableScope";
import { showConnectionStats } from "../../settings/settings";
import { observeRtpStreamStats$ } from "./observeRtpStreamStats";
import { videoFit$, videoSizeFromParticipant$ } from "../../utils/videoFit.ts";
/**
* A participant's user media (i.e. their microphone and camera feed).
@@ -47,7 +46,6 @@ export interface BaseUserMediaViewModel extends BaseMemberMediaViewModel {
speaking$: Behavior<boolean>;
audioEnabled$: Behavior<boolean>;
videoEnabled$: Behavior<boolean>;
videoFit$: Behavior<"cover" | "contain">;
videoOrientation$: Behavior<"landscape" | "portrait">;
toggleCropVideo: () => void;
/**
@@ -63,12 +61,9 @@ export interface BaseUserMediaViewModel extends BaseMemberMediaViewModel {
RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined
>;
/**
* Set the target dimensions of the HTML element (final dimension after anim).
* This can be used to determine the best video fit (fit to frame / keep ratio).
* @param targetWidth - The target width of the HTML element displaying the video.
* @param targetHeight - The target height of the HTML element displaying the video.
* Set the aspect ratio of the video track to determine the orientation.
*/
setTargetDimensions: (targetWidth: number, targetHeight: number) => void;
setVideoAspectRatio: (ratio: number) => void;
}
export interface BaseUserMediaInputs extends Omit<
@@ -98,14 +93,8 @@ export function createBaseUserMedia(
),
);
const toggleCropVideo$ = new Subject<void>();
const videoAspectRatio$ = new BehaviorSubject(NaN);
// The target size of the video element, used to determine the best video fit.
// The target size is the final size of the HTML element after any animations have completed.
const targetSize$ = new BehaviorSubject<
{ width: number; height: number } | undefined
>(undefined);
const videoSize$ = videoSizeFromParticipant$(participant$);
return {
...createMemberMedia(scope, {
...inputs,
@@ -132,13 +121,11 @@ export function createBaseUserMedia(
media$.pipe(map((m) => m?.cameraTrack?.isMuted === false)),
),
videoOrientation$: scope.behavior(
videoSize$.pipe(
map((s) => (s ? s.width / s.height : 1)),
videoAspectRatio$.pipe(
map((aspect) => (aspect > 1 ? "landscape" : "portrait")),
),
"portrait",
),
videoFit$: videoFit$(scope, videoSize$, targetSize$),
toggleCropVideo: () => toggleCropVideo$.next(),
rtcBackendIdentity,
handRaised$,
@@ -162,8 +149,6 @@ export function createBaseUserMedia(
return observeRtpStreamStats$(p, Track.Source.Camera, statsType);
}),
),
setTargetDimensions: (targetWidth: number, targetHeight: number): void => {
targetSize$.next({ width: targetWidth, height: targetHeight });
},
setVideoAspectRatio: (ratio) => videoAspectRatio$.next(ratio),
};
}

View File

@@ -19,10 +19,15 @@ import {
startWith,
switchMap,
map,
share,
} from "rxjs";
import { observeTrackReference$ } from "../observeTrackReference";
// Use a shared timer for all the stats observers so that we don't clog up the
// event loop with hundreds of timers in large calls
const refreshStats$ = interval(1000).pipe(share());
export function observeRtpStreamStats$(
participant: Participant,
source: Track.Source,
@@ -32,9 +37,7 @@ export function observeRtpStreamStats$(
> {
return combineLatest([
observeTrackReference$(participant, source),
// The update frequency is high because we use this value to update the PiP orientation and the fit/fill video tile props based on that
// We want it to be responsive. For just the debug tools 1s would be sufficient.
interval(350).pipe(startWith(0)),
refreshStats$.pipe(startWith(0)),
]).pipe(
switchMap(async ([trackReference]) => {
const track = trackReference?.publication?.track;
@@ -69,12 +72,3 @@ export function observeInboundRtpStreamStats$(
map((x) => x as RTCInboundRtpStreamStats | undefined),
);
}
export function observeOutboundRtpStreamStats$(
participant: Participant,
source: Track.Source,
): Observable<RTCOutboundRtpStreamStats | undefined> {
return observeRtpStreamStats$(participant, source, "outbound-rtp").pipe(
map((x) => x as RTCOutboundRtpStreamStats | undefined),
);
}

View File

@@ -5,7 +5,10 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type RemoteTrackPublication } from "livekit-client";
import {
type LocalTrackPublication,
type RemoteTrackPublication,
} from "livekit-client";
import { test, expect } from "vitest";
import { act, render, screen } from "@testing-library/react";
import { axe } from "vitest-axe";
@@ -17,6 +20,9 @@ import {
mockRtcMembership,
mockRemoteMedia,
mockRemoteParticipant,
mockLocalMedia,
mockLocalParticipant,
mockMediaDevices,
} from "../utils/test";
import { GridTileViewModel } from "../state/TileViewModel";
import { ReactionsSenderProvider } from "../reactions/useReactionsSender";
@@ -54,7 +60,7 @@ const callVm = {
handsRaised$: constant({}),
} as Partial<CallViewModel> as CallViewModel;
test("GridTile is accessible", async () => {
test("GridTile displays remote media", async () => {
const vm = mockRemoteMedia(
mockRtcMembership("@alice:example.org", "AAAA"),
{
@@ -88,6 +94,40 @@ test("GridTile is accessible", async () => {
screen.getByText("Alice");
});
test("GridTile displays local media", async () => {
const vm = mockLocalMedia(
mockRtcMembership("@alice:example.org", "AAAA"),
{
rawDisplayName: "Alice",
getMxcAvatarUrl: () => "mxc://adfsg",
},
mockLocalParticipant({
getTrackPublication: () =>
({}) as Partial<LocalTrackPublication> as LocalTrackPublication,
}),
mockMediaDevices({}),
);
const { container } = render(
<ReactionsSenderProvider vm={callVm} rtcSession={fakeRtcSession}>
<GridTile
vm={new GridTileViewModel(constant(vm))}
onOpenProfile={() => {}}
targetWidth={300}
targetHeight={200}
showSpeakingIndicators
showNameTags
showRingingStatus
showOutline
focusable
/>
</ReactionsSenderProvider>,
);
expect(await axe(container)).toHaveNoViolations();
// Name should be visible
screen.getByText("Alice");
});
test("GridTile displays ringing media", async () => {
const pickupState$ = new BehaviorSubject<
RingingMediaViewModel["pickupState$"]["value"]

View File

@@ -11,9 +11,9 @@ import {
type ReactNode,
type Ref,
useCallback,
useEffect,
useRef,
useState,
useMemo,
} from "react";
import { type animated } from "@react-spring/web";
import classNames from "classnames";
@@ -91,7 +91,6 @@ const RingingMediaTile: FC<RingingMediaTileProps> = ({
}
avatarStyle="translucent"
videoEnabled={false}
videoFit="cover"
mirror={false}
{...props}
/>
@@ -105,20 +104,22 @@ interface UserMediaTileProps extends TileProps {
playbackMuted: boolean;
waitingForMedia?: boolean;
primaryButton?: ReactNode;
menuStart?: ReactNode;
menuEnd?: ReactNode;
focusUrl: string | undefined;
}
const UserMediaTile: FC<UserMediaTileProps> = ({
/**
* A user media tile without a context menu.
*/
// The context menu is kept separate from this component for performance
// reasons (c.f. UserMediaTile)
const UserMediaTileInner: FC<UserMediaTileProps & { menu: ReactNode }> = ({
ref,
vm,
showSpeakingIndicators,
playbackMuted,
waitingForMedia,
primaryButton,
menuStart,
menuEnd,
menu,
className,
focusUrl,
displayName,
@@ -141,19 +142,11 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
const audioEnabled = useBehavior(vm.audioEnabled$);
const videoEnabled = useBehavior(vm.videoEnabled$);
const speaking = useBehavior(vm.speaking$);
const videoFit = useBehavior(vm.videoFit$);
const rtcBackendIdentity = vm.rtcBackendIdentity;
const handRaised = useBehavior(vm.handRaised$);
const reaction = useBehavior(vm.reaction$);
// Whenever bounds change, inform the viewModel
useEffect(() => {
if (targetWidth > 0 && targetHeight > 0) {
vm.setTargetDimensions(targetWidth, targetHeight);
}
}, [targetWidth, targetHeight, vm]);
const AudioIcon = playbackMuted
? VolumeOffSolidIcon
: audioEnabled
@@ -166,31 +159,32 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
: t("microphone_off");
const [menuOpen, setMenuOpen] = useState(false);
const menu = (
<>
{menuStart}
{/*
No additional menu item (used to be the manual fit to frame.
Placeholder for future menu items that should be placed here.
*/}
{menuEnd}
</>
const menuTrigger = useMemo(
() => (
<button
aria-label={t("common.options")}
tabIndex={focusable ? undefined : -1}
>
<OverflowHorizontalIcon aria-hidden width={20} height={20} />
</button>
),
[t, focusable],
);
const raisedHandOnClick = vm.local
? (): void => void toggleRaisedHand()
: undefined;
const raisedHandOnClick = useMemo(
() => (vm.local ? (): void => void toggleRaisedHand() : undefined),
[vm.local, toggleRaisedHand],
);
const showSpeaking = showSpeakingIndicators && speaking;
const tile = (
return (
<MediaView
ref={ref}
video={video}
userId={vm.userId}
unencryptedWarning={unencryptedWarning}
videoEnabled={videoEnabled}
videoFit={videoFit}
className={classNames(className, styles.tile, {
[styles.speaking]: showSpeaking,
[styles.handRaised]: !showSpeaking && handRaised,
@@ -213,14 +207,7 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
open={menuOpen}
onOpenChange={setMenuOpen}
title={displayName}
trigger={
<button
aria-label={t("common.options")}
tabIndex={focusable ? undefined : -1}
>
<OverflowHorizontalIcon aria-hidden width={20} height={20} />
</button>
}
trigger={menuTrigger}
side="left"
align="start"
>
@@ -233,6 +220,7 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
raisedHandOnClick={raisedHandOnClick}
waitingForMedia={waitingForMedia}
focusUrl={focusUrl}
setVideoAspectRatio={vm.setVideoAspectRatio}
audioStreamStats={audioStreamStats}
videoStreamStats={videoStreamStats}
rtcBackendIdentity={rtcBackendIdentity}
@@ -241,9 +229,37 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
{...props}
/>
);
};
/**
* A user media tile enhanced with a context menu.
*/
const UserMediaTile: FC<
UserMediaTileProps & { menuStart?: ReactNode; menuEnd?: ReactNode }
> = ({ menuStart, menuEnd, ...props }) => {
const menu = useMemo(
() => (
<>
{menuStart}
{/*
No additional menu item (used to be the manual fit to frame.
Placeholder for future menu items that should be placed here.
*/}
{menuEnd}
</>
),
[menuStart, menuEnd],
);
// ContextMenu is expensive to render, so we avoid subscribing to any
// frequently-changing behaviors here and instead keep them isolated in the
// UserMediaTileInner component
return (
<ContextMenu title={displayName} trigger={tile} hasAccessibleAlternative>
<ContextMenu
title={props.displayName}
trigger={<UserMediaTileInner {...props} menu={menu} />}
hasAccessibleAlternative
>
{menu}
</ContextMenu>
);
@@ -279,6 +295,29 @@ const LocalUserMediaTile: FC<LocalUserMediaTileProps> = ({
[vm, latestAlwaysShow],
);
const menuStart = useMemo(
() => (
<ToggleMenuItem
Icon={VisibilityOnIcon}
label={t("video_tile.always_show")}
checked={alwaysShow}
onSelect={onSelectAlwaysShow}
/>
),
[t, alwaysShow, onSelectAlwaysShow],
);
const menuEnd = useMemo(
() =>
onOpenProfile && (
<MenuItem
Icon={UserProfileIcon}
label={t("common.profile")}
onSelect={onOpenProfile}
/>
),
[t, onOpenProfile],
);
return (
<UserMediaTile
ref={ref}
@@ -297,23 +336,8 @@ const LocalUserMediaTile: FC<LocalUserMediaTileProps> = ({
</button>
)
}
menuStart={
<ToggleMenuItem
Icon={VisibilityOnIcon}
label={t("video_tile.always_show")}
checked={alwaysShow}
onSelect={onSelectAlwaysShow}
/>
}
menuEnd={
onOpenProfile && (
<MenuItem
Icon={UserProfileIcon}
label={t("common.profile")}
onSelect={onOpenProfile}
/>
)
}
menuStart={menuStart}
menuEnd={menuEnd}
focusable={focusable}
focusUrl={focusUrl}
{...props}

View File

@@ -33,7 +33,6 @@ describe("MediaView", () => {
const baseProps: ComponentProps<typeof MediaView> = {
displayName: "some name",
videoEnabled: true,
videoFit: "contain",
targetWidth: 300,
targetHeight: 200,
mirror: false,

View File

@@ -7,7 +7,13 @@ Please see LICENSE in the repository root for full details.
import { type TrackReferenceOrPlaceholder } from "@livekit/components-core";
import { animated } from "@react-spring/web";
import { type FC, type ComponentProps, type ReactNode } from "react";
import {
type FC,
type ComponentProps,
type ReactNode,
type SyntheticEvent,
useState,
} from "react";
import { useTranslation } from "react-i18next";
import classNames from "classnames";
import { VideoTrack } from "@livekit/components-react";
@@ -26,6 +32,7 @@ import { type ReactionOption } from "../reactions";
import { ReactionIndicator } from "../reactions/ReactionIndicator";
import { RTCConnectionStats } from "../RTCConnectionStats";
import videoPlaceholder from "../graphics/video-placeholder.gif";
import { autoVideoFit } from "../utils/videoFit";
interface Props extends ComponentProps<typeof animated.div> {
className?: string;
@@ -33,7 +40,11 @@ interface Props extends ComponentProps<typeof animated.div> {
targetWidth: number;
targetHeight: number;
video: TrackReferenceOrPlaceholder | undefined;
videoFit: "cover" | "contain";
/**
* How to fit the video content inside the tile. When undefined, MediaView
* chooses a smart default based on the aspect ratios of the tile and video.
*/
videoFit?: "cover" | "contain";
mirror: boolean;
soundWaves?: boolean;
userId: string;
@@ -55,8 +66,15 @@ interface Props extends ComponentProps<typeof animated.div> {
audioStreamStats?: RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats;
videoStreamStats?: RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats;
rtcBackendIdentity?: string;
// The focus url, mainly for debugging purposes
/**
* The focus url, mainly for debugging purposes.
*/
focusUrl?: string;
/**
* Called whenever the aspect ratio of the video content becomes known or
* otherwise changes.
*/
setVideoAspectRatio?: (ratio: number) => void;
}
export const MediaView: FC<Props> = ({
@@ -89,6 +107,7 @@ export const MediaView: FC<Props> = ({
videoStreamStats,
rtcBackendIdentity,
focusUrl,
setVideoAspectRatio: setTheirVideoAspectRatio,
...props
}) => {
const { t } = useTranslation();
@@ -100,6 +119,22 @@ export const MediaView: FC<Props> = ({
(soundWaves === undefined ? 0.5 : 0.38),
);
const [videoAspectRatio, setOurVideoAspectRatio] = useState<number>(NaN);
const tileAspectRatio = targetWidth / targetHeight;
// Propagate video dimensions
const setVideoAspectRatio = (ratio: number) => {
setOurVideoAspectRatio(ratio);
setTheirVideoAspectRatio?.(ratio);
};
const videoRef = (el: HTMLVideoElement | null) => {
if (el !== null) setVideoAspectRatio(el.videoWidth / el.videoHeight);
};
const onResize = (ev: SyntheticEvent<HTMLVideoElement>) =>
setVideoAspectRatio(
ev.currentTarget.videoWidth / ev.currentTarget.videoHeight,
);
const warnings = unencryptedWarning && (
<Tooltip
label={t("common.unencrypted")}
@@ -126,7 +161,9 @@ export const MediaView: FC<Props> = ({
ref={ref}
data-testid="videoTile"
data-video-enabled={video && videoEnabled}
data-video-fit={videoFit}
data-video-fit={
videoFit ?? autoVideoFit(videoAspectRatio, tileAspectRatio)
}
data-background={background}
{...props}
>
@@ -158,6 +195,8 @@ export const MediaView: FC<Props> = ({
// Set the placeholder to a small transparent image. (On Android web
// views the default poster image is particularly ugly.)
poster={videoPlaceholder}
ref={videoRef}
onResize={onResize}
/>
)}
</div>

View File

@@ -68,6 +68,7 @@ interface SpotlightItemBaseProps {
background: "solid" | "transparent";
focusable: boolean;
"aria-hidden"?: boolean;
setVideoAspectRatio?: (ratio: number) => void;
}
interface SpotlightMemberMediaItemBaseProps extends SpotlightItemBaseProps {
@@ -77,7 +78,6 @@ interface SpotlightMemberMediaItemBaseProps extends SpotlightItemBaseProps {
}
interface SpotlightUserMediaItemBaseProps extends SpotlightMemberMediaItemBaseProps {
videoFit: "contain" | "cover";
videoEnabled: boolean;
soundWaves: boolean | undefined;
}
@@ -120,20 +120,12 @@ const SpotlightUserMediaItem: FC<SpotlightUserMediaItemProps> = ({
targetHeight,
...props
}) => {
const videoFit = useBehavior(vm.videoFit$);
const videoEnabled = useBehavior(vm.videoEnabled$);
const speaking = useBehavior(vm.speaking$);
// Whenever target bounds change, inform the viewModel
useEffect(() => {
if (targetWidth > 0 && targetHeight > 0) {
vm.setTargetDimensions(targetWidth, targetHeight);
}
}, [targetWidth, targetHeight, vm]);
const baseProps: SpotlightUserMediaItemBaseProps &
RefAttributes<HTMLDivElement> = {
videoFit,
setVideoAspectRatio: vm.setVideoAspectRatio,
videoEnabled,
soundWaves: props.background === "transparent" ? speaking : undefined,
targetWidth,
@@ -227,7 +219,6 @@ const SpotlightRingingMediaItem: FC<SpotlightRingingMediaItemProps> = ({
}
avatarStyle="translucent"
videoEnabled={false}
videoFit="cover"
mirror={false}
{...props}
/>

View File

@@ -12,7 +12,7 @@ import { i18nKey } from "./i18n";
export enum ErrorCode {
/**
* Configuration problem due to no MatrixRTC backend/SFU is exposed via .well-known and no fallback configured.
* Configuration problem due to no MatrixRTC transport provided by homeserver and no fallback configured.
*/
MISSING_MATRIX_RTC_TRANSPORT = "MISSING_MATRIX_RTC_TRANSPORT",
CONNECTION_LOST_ERROR = "CONNECTION_LOST_ERROR",
@@ -67,7 +67,7 @@ export class ElementCallError extends Error {
}
/**
* Configuration problem due to no MatrixRTC backend/SFU is exposed via .well-known and no fallback configured.
* Configuration problem due to no MatrixRTC transport provided by homeserver and no fallback configured.
*/
export class MatrixRTCTransportMissingError extends ElementCallError {
public domain: string;

View File

@@ -171,7 +171,7 @@ export function getBasicCallViewModelEnvironment(
setE2EEEnabled: async () => Promise.resolve(),
}),
connectionState$: constant(ConnectionState.Connected),
matrixRTCMode$: constant(MatrixRTCMode.Legacy),
matrixRTCMode$: constant(MatrixRTCMode.Compatibility),
...callViewModelOptions,
},
handRaisedSubject$,

View File

@@ -237,7 +237,7 @@ export function mockRtcMembership(
fociPreferred: [exampleTransport],
focusActive: {
type: "livekit" as const,
focus_selection: "oldest_membership" as const,
focus_selection: "multi_sfu" as const,
},
callId: "",
membership: {},
@@ -463,9 +463,6 @@ export class MockRTCSession extends TypedEventEmitter<
session.reemitEncryptionKeys = vi
.fn<() => void>()
.mockReturnValue(undefined);
session.getOldestMembership = vi
.fn<() => CallMembership | undefined>()
.mockReturnValue(this.memberships[0]);
return session;
}

View File

@@ -5,259 +5,106 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, test, vi } from "vitest";
import {
LocalTrack,
type LocalTrackPublication,
type RemoteTrackPublication,
Track,
} from "livekit-client";
import { describe, expect, test } from "vitest";
import { ObservableScope } from "../state/ObservableScope";
import { videoFit$, videoSizeFromParticipant$ } from "./videoFit";
import { constant } from "../state/Behavior";
import {
flushPromises,
mockLocalParticipant,
mockRemoteParticipant,
} from "./test";
import { autoVideoFit } from "./videoFit";
describe("videoFit$ defaults", () => {
test.each([
{
videoSize: { width: 1920, height: 1080 },
tileSize: undefined,
videoAspectRatio: 1920 / 1080,
tileAspectRatio: NaN,
},
{
videoSize: { width: 1080, height: 1920 },
tileSize: undefined,
videoAspectRatio: 1080 / 1920,
tileAspectRatio: NaN,
},
{
videoSize: undefined,
tileSize: { width: 1920, height: 1080 },
videoAspectRatio: NaN,
tileAspectRatio: 1920 / 1080,
},
{
videoSize: undefined,
tileSize: { width: 1080, height: 1920 },
videoAspectRatio: NaN,
tileAspectRatio: 1080 / 1920,
},
])(
"videoFit$ returns `cover` when videoSize is $videoSize and tileSize is $tileSize",
({ videoSize, tileSize }) => {
const scope = new ObservableScope();
const videoSize$ = constant(videoSize);
const tileSize$ = constant(tileSize);
const fit = videoFit$(scope, videoSize$, tileSize$);
expect(fit.value).toBe("cover");
},
"videoFit$ returns `cover` when videoAspectRatio is $videoAspectRatio and tileAspectRatio is $tileAspectRatio",
({ videoAspectRatio, tileAspectRatio }) =>
expect(autoVideoFit(videoAspectRatio, tileAspectRatio)).toBe("cover"),
);
});
const VIDEO_480_L = { width: 640, height: 480 };
const VIDEO_720_L = { width: 1280, height: 720 };
const VIDEO_1080_L = { width: 1920, height: 1080 };
const VIDEO_480_L = 640 / 480;
const VIDEO_720_L = 1280 / 720;
const VIDEO_1080_L = 1920 / 1080;
// Some sizes from real world testing, which don't match the standard video sizes exactly
const TILE_SIZE_1_L = { width: 180, height: 135 };
const TILE_SIZE_3_P = { width: 379, height: 542 };
const TILE_SIZE_4_L = { width: 957, height: 542 };
const TILE_SIZE_1_L = 180 / 135;
const TILE_SIZE_3_P = 379 / 542;
const TILE_SIZE_4_L = 957 / 542;
// This is the size of an iPhone Xr in portrait mode
const TILE_SIZE_5_P = { width: 414, height: 896 };
const TILE_SIZE_5_P = 414 / 896;
export function invertSize(size: { width: number; height: number }): {
width: number;
height: number;
} {
return {
width: size.height,
height: size.width,
};
function inverse(ratio: number): number {
return 1 / ratio;
}
test.each([
{
videoSize: VIDEO_480_L,
tileSize: TILE_SIZE_1_L,
videoAspectRatio: VIDEO_480_L,
tileAspectRatio: TILE_SIZE_1_L,
expected: "cover",
},
{
videoSize: invertSize(VIDEO_480_L),
tileSize: TILE_SIZE_1_L,
videoAspectRatio: inverse(VIDEO_480_L),
tileAspectRatio: TILE_SIZE_1_L,
expected: "contain",
},
{
videoSize: VIDEO_720_L,
tileSize: TILE_SIZE_4_L,
videoAspectRatio: VIDEO_720_L,
tileAspectRatio: TILE_SIZE_4_L,
expected: "cover",
},
{
videoSize: invertSize(VIDEO_720_L),
tileSize: TILE_SIZE_4_L,
videoAspectRatio: inverse(VIDEO_720_L),
tileAspectRatio: TILE_SIZE_4_L,
expected: "contain",
},
{
videoSize: invertSize(VIDEO_1080_L),
tileSize: TILE_SIZE_3_P,
videoAspectRatio: inverse(VIDEO_1080_L),
tileAspectRatio: TILE_SIZE_3_P,
expected: "cover",
},
{
videoSize: VIDEO_1080_L,
tileSize: TILE_SIZE_5_P,
videoAspectRatio: VIDEO_1080_L,
tileAspectRatio: TILE_SIZE_5_P,
expected: "contain",
},
{
videoSize: invertSize(VIDEO_1080_L),
tileSize: TILE_SIZE_5_P,
videoAspectRatio: inverse(VIDEO_1080_L),
tileAspectRatio: TILE_SIZE_5_P,
expected: "cover",
},
{
// square video
videoSize: { width: 400, height: 400 },
tileSize: VIDEO_480_L,
videoAspectRatio: 400 / 400,
tileAspectRatio: VIDEO_480_L,
expected: "contain",
},
{
// Should default to cover if the initial size is 0:0.
// Or else it will cause a flash of "contain" mode until the real size is loaded, which can be jarring.
videoSize: VIDEO_480_L,
tileSize: { width: 0, height: 0 },
videoAspectRatio: VIDEO_480_L,
tileAspectRatio: 0 / 0,
expected: "cover",
},
{
videoSize: { width: 0, height: 0 },
tileSize: VIDEO_480_L,
videoAspectRatio: 0 / 0,
tileAspectRatio: VIDEO_480_L,
expected: "cover",
},
])(
"videoFit$ returns $expected when videoSize is $videoSize and tileSize is $tileSize",
({ videoSize, tileSize, expected }) => {
const scope = new ObservableScope();
const videoSize$ = constant(videoSize);
const tileSize$ = constant(tileSize);
const fit = videoFit$(scope, videoSize$, tileSize$);
expect(fit.value).toBe(expected);
},
"videoFit$ returns $expected when videoAspectRatio is $videoAspectRatio and tileAspectRatio is $tileAspectRatio",
({ videoAspectRatio, tileAspectRatio, expected }) =>
expect(autoVideoFit(videoAspectRatio, tileAspectRatio)).toBe(expected),
);
describe("extracting video size from participant stats", () => {
function createMockRtpStats(
isInbound: boolean,
props: Partial<RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats> = {},
): RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats {
const baseStats = {
id: "mock-stats-id",
timestamp: Date.now(),
type: isInbound ? "inbound-rtp" : "outbound-rtp",
kind: "video",
...props,
};
return baseStats as RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats;
}
test("get stats for local user", async () => {
const localParticipant = mockLocalParticipant({
identity: "@local:example.org:AAAAAA",
});
const mockReport: RTCStatsReport = new Map([
[
"OT01V639885149",
createMockRtpStats(false, {
frameWidth: 1280,
frameHeight: 720,
}),
],
]);
const track = {
source: Track.Source.Camera,
getRTCStatsReport: vi
.fn()
.mockImplementation(async () => Promise.resolve(mockReport)),
} as Partial<LocalTrack> as LocalTrack;
// Set up the prototype chain (there is an instanceof check in getRTCStatsReport)
Object.setPrototypeOf(track, LocalTrack.prototype);
localParticipant.getTrackPublication = vi
.fn()
.mockImplementation((source: Track.Source) => {
if (source === Track.Source.Camera) {
return {
track,
} as unknown as LocalTrackPublication;
} else {
return undefined;
}
});
const videoDimensions$ = videoSizeFromParticipant$(
constant(localParticipant),
);
const publishedDimensions: { width: number; height: number }[] = [];
videoDimensions$.subscribe((dimensions) => {
if (dimensions) publishedDimensions.push(dimensions);
});
await flushPromises();
const dimension = publishedDimensions.pop();
expect(dimension).toEqual({ width: 1280, height: 720 });
});
test("get stats for remote user", async () => {
// vi.useFakeTimers()
const remoteParticipant = mockRemoteParticipant({
identity: "@bob:example.org:AAAAAA",
});
const mockReport: RTCStatsReport = new Map([
[
"OT01V639885149",
createMockRtpStats(true, {
frameWidth: 480,
frameHeight: 640,
}),
],
]);
const track = {
source: Track.Source.Camera,
getRTCStatsReport: vi
.fn()
.mockImplementation(async () => Promise.resolve(mockReport)),
} as Partial<LocalTrack> as LocalTrack;
// Set up the prototype chain (there is an instanceof check in getRTCStatsReport)
Object.setPrototypeOf(track, LocalTrack.prototype);
remoteParticipant.getTrackPublication = vi
.fn()
.mockImplementation((source: Track.Source) => {
if (source === Track.Source.Camera) {
return {
track,
} as unknown as RemoteTrackPublication;
} else {
return undefined;
}
});
const videoDimensions$ = videoSizeFromParticipant$(
constant(remoteParticipant),
);
const publishedDimensions: { width: number; height: number }[] = [];
videoDimensions$.subscribe((dimensions) => {
if (dimensions) publishedDimensions.push(dimensions);
});
await flushPromises();
const dimension = publishedDimensions.pop();
expect(dimension).toEqual({ width: 480, height: 640 });
});
});

View File

@@ -5,107 +5,27 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { combineLatest, map, type Observable, of, switchMap } from "rxjs";
import {
type LocalParticipant,
type RemoteParticipant,
Track,
} from "livekit-client";
import { type ObservableScope } from "../state/ObservableScope.ts";
import { type Behavior } from "../state/Behavior.ts";
import {
observeInboundRtpStreamStats$,
observeOutboundRtpStreamStats$,
} from "../state/media/observeRtpStreamStats";
type Size = {
width: number;
height: number;
};
/**
* Computes the appropriate video fit mode ("cover" or "contain") based on the aspect ratios of the video and the tile.
* - If the video and tile have the same orientation (both landscape or both portrait), we use "cover" to fill the tile, even if it means cropping.
* - If the video and tile have different orientations, we use "contain" to ensure the entire video is visible, even if it means letterboxing (black bars).
* @param scope - the ObservableScope to create the Behavior in
* @param videoSize$ - an Observable of the video size (width and height) or undefined if the size is not yet known (no data yet received).
* @param tileSize$ - an Observable of the tile size (width and height) or undefined if the size is not yet known (not yet rendered).
*/
export function videoFit$(
scope: ObservableScope,
videoSize$: Observable<Size | undefined>,
tileSize$: Observable<Size | undefined>,
): Behavior<"cover" | "contain"> {
const fit$ = combineLatest([videoSize$, tileSize$]).pipe(
map(([videoSize, tileSize]) => {
if (!videoSize || !tileSize) {
// If we don't have the sizes, default to cover to avoid black bars.
// This is a reasonable default as it will ensure the video fills the tile, even if it means cropping.
return "cover";
}
if (
videoSize.width === 0 ||
videoSize.height === 0 ||
tileSize.width === 0 ||
tileSize.height === 0
) {
// If we have invalid sizes (e.g. width or height is 0), default to cover to avoid black bars.
return "cover";
}
const videoAspectRatio = videoSize.width / videoSize.height;
const tileAspectRatio = tileSize.width / tileSize.height;
export function autoVideoFit(
videoAspectRatio: number,
tileAspectRatio: number,
): "cover" | "contain" {
if (Number.isNaN(videoAspectRatio) || Number.isNaN(tileAspectRatio)) {
// If we have invalid sizes (e.g. useMeasure returns 0×0 on an initial render),
// default to cover to avoid black bars.
return "cover";
}
// If video is landscape (ratio > 1) and tile is portrait (ratio < 1) or vice versa,
// we want to use "contain" (fit) mode to avoid excessive cropping
const videoIsLandscape = videoAspectRatio > 1;
const tileIsLandscape = tileAspectRatio > 1;
// If video is landscape (ratio > 1) and tile is portrait (ratio < 1) or vice versa,
// we want to use "contain" (fit) mode to avoid excessive cropping
const videoIsLandscape = videoAspectRatio > 1;
const tileIsLandscape = tileAspectRatio > 1;
// If the orientations are the same, use the cover mode (Preserves the aspect ratio, and the image fills the container.)
// If they're not the same orientation, use the contain mode (Preserves the aspect ratio, but the image is letterboxed - black bars- to fit within the container.)
return videoIsLandscape === tileIsLandscape ? "cover" : "contain";
}),
);
return scope.behavior(fit$, "cover");
}
/**
* Helper function to get the video size from a participant.
* It observes the participant's video track stats and extracts the frame width and height.
* @param participant$ - an Observable of a LocalParticipant or RemoteParticipant, or null if no participant is selected.
* @returns an Observable of the video size (width and height) or undefined if the size cannot be determined.
*/
export function videoSizeFromParticipant$(
participant$: Observable<LocalParticipant | RemoteParticipant | null>,
): Observable<{ width: number; height: number } | undefined> {
return participant$
.pipe(
// If we have a participant, observe their video track stats. If not, return undefined.
switchMap((p) => {
if (!p) return of(undefined);
if (p.isLocal) {
return observeOutboundRtpStreamStats$(p, Track.Source.Camera);
} else {
return observeInboundRtpStreamStats$(p, Track.Source.Camera);
}
}),
)
.pipe(
// Extract the frame width and height from the stats. If we don't have valid stats, return undefined.
map((stats) => {
if (!stats) return undefined;
if (
// For video tracks, frameWidth and frameHeight should be numbers. If they're not, we can't determine the size.
typeof stats.frameWidth !== "number" ||
typeof stats.frameHeight !== "number"
) {
return undefined;
}
return {
width: stats.frameWidth,
height: stats.frameHeight,
};
}),
);
// If the orientations are the same, use the cover mode (Preserves the aspect ratio, and the image fills the container.)
// If they're not the same orientation, use the contain mode (Preserves the aspect ratio, but the image is letterboxed - black bars- to fit within the container.)
return videoIsLandscape === tileIsLandscape ? "cover" : "contain";
}

View File

@@ -19,6 +19,7 @@ const createRoomWidgetClientSpy = vi.mocked(createRoomWidgetClient);
vi.mock("./config/Config", () => ({
Config: {
init: vi.fn().mockImplementation(async () => Promise.resolve()),
get: vi.fn().mockReturnValue({}),
},
}));
const configInitSpy = vi.mocked(Config.init);
@@ -109,6 +110,7 @@ describe("widget", () => {
sendToDevice: sendRecvToDevice,
receiveToDevice: sendRecvToDevice,
turnServers: false,
rtcTransports: true,
sendDelayedEvents: true,
updateDelayedEvents: true,
sendSticky: true,

View File

@@ -21,6 +21,7 @@ import type { IWidgetApiRequest } from "matrix-widget-api";
import { LazyEventEmitter } from "./LazyEventEmitter";
import { getUrlParams } from "./UrlParams";
import { Config } from "./config/Config";
import { seedSettingsFromConfig } from "./settings/settings";
import { ElementCallReactionEventType } from "./reactions";
// Subset of the actions in element-web
@@ -173,6 +174,7 @@ export const initializeWidget = (
sendToDevice: sendRecvToDevice,
receiveToDevice: sendRecvToDevice,
turnServers: false,
rtcTransports: true,
sendDelayedEvents: true,
updateDelayedEvents: true,
sendSticky: true,
@@ -195,7 +197,8 @@ export const initializeWidget = (
// Wait for the config file to be ready (we load very early on so it might not
// be otherwise)
await Config.init();
await client.startClient({ clientWellKnownPollPeriod: 60 * 10 });
seedSettingsFromConfig(Config.get().media_quality);
await client.startClient();
return client;
};