Compare commits

..
Author SHA1 Message Date
Johannes Marbach b53b2122a5 Merge branch 'johannes/matrix2-test' into robin/livekit-updates 2026-10-13 07:38:49 +02:00
Johannes Marbach ee7dcdee7e Cover the subscriber token path, widget fallback and server names 2026-10-12 21:28:04 +02:00
Johannes Marbach 8d1ed31b0b Cover identity matching and delegation across transport shapes 2026-10-12 21:09:15 +02:00
Johannes Marbach 54e948fdc0 Make the auth choice mode independent again 2026-10-12 20:37:14 +02:00
Johannes Marbach 37d25d4293 Merge branch 'main' into robin/livekit-updates 2026-10-12 16:29:13 +02:00
Timo 079185b232 Merge pull request #4347 from element-hq/toger5/CI-playwright-improvements
[CI Playwirght] Make reporter more verbose + fail faster
2026-10-12 16:18:13 +02:00
Johannes Marbach 022415e98e Avoid null in transport dev setting 2026-10-12 16:11:03 +02:00
Johannes Marbach a1575cb554 Share a connection between homeservers using the same JWT service 2026-10-12 16:08:56 +02:00
Johannes Marbach 97b5e61ad8 Fall back to get_token as subscriber on widget API errors 2026-10-12 16:03:00 +02:00
Johannes Marbach f8258f6a1d Switch JS SDK back to develop 2026-10-12 15:43:44 +02:00
Johannes Marbach 68107d119e Retain compatiblity in compatiblity mode 2026-10-12 15:40:27 +02:00
Timo e96f578b63 Merge pull request #4346 from element-hq/renovate/compound
Update dependency @vector-im/compound-design-tokens to v11.2.0
2026-10-12 14:12:06 +02:00
renovate[bot] cfaae8f7e2 Update dependency @vector-im/compound-design-tokens to v11.2.0 2026-10-12 11:01:00 +00:00
Timo 1a90e813e8 Merge pull request #4338 from element-hq/toger5/compound-portal-root
[ElementCallComponent] Use compound-web's PortalRoot in EC.
2026-10-12 12:59:28 +02:00
Timo K.andClaude Fable 5.1 9d0a12c38a Fix PortalRoot hiding the component's menus and tooltips
compound-web 10.4.0 treats a PortalRoot whose root is null as one whose root
is not available yet, so floating-ui never mounts the portal and every tooltip
label, and with it every icon button's accessible name, is missing. The
component defaulted portalRoot to null and wrapped the call unconditionally.

Wrap the call only when the host supplied a root, keeping null as the
not-yet-available case Compound defines, and say so on the prop. Cover it in
CallPortalRoot.test.tsx. Drop the git allowBuilds entry for compound-web now
that it is a release again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 12:43:29 +02:00
Timo K. a1c98ac4e6 make reporter more verbose + let CI playwright fail faster 2026-10-12 12:38:10 +02:00
Timo K. b72493f47f bump compound-web to release 10.4.0 2026-10-12 11:15:03 +02:00
Johannes Marbach 49223e720b Cover url transports end to end
The specs replace the transports the e2e homeserver advertises with each
shape a migrating deployment may produce and record which token endpoints
each side used: a url-only or combined transport on both sides goes
through the CS API alone, a legacy guest joining a url host uses the
legacy endpoint to publish and the CS API to subscribe, a subscriber falls
back to the JWT service when the homeserver lacks MSC4195, a publisher
gets an error instead, a delayed leave delegated through the CS API fires
on disconnect, and subscribers get tokens for a remote homeserver's SFU
over federation.

The JWT service's app service registration has to cover users of the e2e
Synapse's real server name for the get_token flow, and lk-jwt-service
0.8.0 implements the endpoints.
2026-10-12 10:33:58 +02:00
Robin c5044119ce Deprecate the livekit.livekit_service_url config option
Homeservers should advertise their transports over the MSC4143
rtc/transports endpoint instead of relying on a fallback in the app
config.
2026-10-12 10:33:38 +02:00
Robin f366e7cbe2 Let developers override the whole transport
The developer setting that overrode the LiveKit service URL becomes a
JSON transport, so that a `url`-only or combined transport can be tested
against a homeserver that does not advertise one yet. The settings modal
shows each LiveKit room's role instead of a never-populated isLocal flag.
2026-10-12 10:33:19 +02:00
Robin e336ee5cfa Get SFU tokens and delegate delayed leaves through the MSC4195 CS API
src/livekit/auth now fronts two flows. For a transport with a LiveKit SFU
`url`, tokens come from our own homeserver's MSC4195 get_token endpoint
and delayed leaves are delegated through delegate_delayed_leave, both via
the js-sdk's unstable client methods. Transports with only a
livekit_service_url keep using the OpenID flow against the JWT service.

A transport advertising both shapes lets a subscriber fall back to the
OpenID flow when the homeserver answers get_token with a 404 (Synapse
reports M_UNRECOGNIZED for unknown routes rather than M_NOT_FOUND), so a
client can still join calls hosted on servers that have not deployed
MSC4195 yet. Publishers never fall back: a homeserver that advertises a
`url` but does not implement the endpoints is misconfigured and the
admin should hear about it, so they get NoMatrix2AuthorizationService.
Delegation uses the same flow as the token so that the participant
identity the service computes matches.

The local member only probes a transport's delegate_delayed_leave
endpoint when it has a service URL. The developer settings tab validates
a custom service URL through the same facade.
2026-10-12 10:33:00 +02:00
Robin 17c3a7cb5d Pin matrix-js-sdk to the member_id branch
Temporary, until matrix-org/matrix-js-sdk#5588 merges and this can go
back to a develop bump. The branch only changes the get_token request
body from a member object to a member_id, so this is kept separate from
the code that uses it and can be dropped without touching that code.
2026-10-12 10:32:13 +02:00
Robin 53a0e273d6 Identify transports by server name as well as URLs
A transport with only a LiveKit SFU `url` has to be authenticated through
our own homeserver, which needs to know the name of the server owning the
SFU. Carry that name alongside the transport as a TransportLocator,
derived from the member's user ID for remote transports and from the
client's domain for our own, and key the connection map on the SFU URL,
the service URL and the server name together so that `url`, service URL
and combined transport shapes are deduplicated consistently.
2026-10-12 10:31:36 +02:00
Robin 83e1396bdd Use the unstable LiveKit transport type
UnstableLivekitTransport covers every shape a homeserver may advertise
while MSC4195 evolves: a LiveKit SFU `url`, a `livekit_service_url`, or
both. Switching to it means a service URL is no longer guaranteed, so the
remaining code that relies on one checks for it explicitly: delegation
probing skips transports without one, the connection map keys transports
by whichever URL they carry, and the OpenID flow rejects transports
without a service URL until the client-server API flow lands.
2026-10-12 10:30:52 +02:00
Robin d5bbb43e3a Reduce ConnectionManager to a ConnectionMap
createConnectionMap$ now returns the behavior of connections and their
participants directly and takes an already resolved local transport
instead of an observable of it. The CallViewModel wraps it with mapScoped
so that a new map is built, in a fresh scope, whenever the local transport
changes, and swallows local transport errors there since the local member
already surfaces them in the UI.
2026-10-12 10:26:54 +02:00
Robin 24b6a632dd Let the publishing Connection authenticate itself
getLocalTransport no longer authenticates with the SFU up front and hands
the resulting config to the connection. Instead the ConnectionManager tags
the connection whose transport equals the local one as the publisher, and
every Connection fetches its own SFU config for its role.

The local member now only joins the MatrixRTC session once its connection
is authenticated, so that the LiveKit room exists before remote members
try to get a subscriber token for it. Subscribers authenticating with the
OpenID flow lack permission to create the room and would otherwise fail
without retrying, causing a cascade of leaves and rejoins at the start of
a call.
2026-10-12 10:22:33 +02:00
Robin 17c3223883 Move SFU authentication into livekit/auth and choose the endpoint by role
The OpenID flow moves to src/livekit/auth/openID.ts with the SFU config
types in a sibling file, takes an options object, and replaces the
MatrixRTC mode with the role we take on the transport plus the session's
slot ID instead of a hard-coded m.call#ROOM.

This changes behaviour: publishers on a livekit_service_url transport
always use the legacy /sfu/get endpoint, so that subscribers can predict
their user:device participant identity, and subscribers always use the
default /get_token endpoint. The MatrixRTC mode no longer influences which
endpoint is used.
2026-10-12 10:17:39 +02:00
Timo K. c79675e6a9 bump to compound-web main branch 2026-10-12 10:14:49 +02:00
Robin f260755f46 Group livekitRoomItems$ by connection
The old reducer keyed items by service URL and mutated the accumulator's
entries in place, so a transport without a service URL could not be
represented and items were not safe to compare. Group members by their
Connection object instead and carry the whole transport on the item, which
the audio renderer now receives in place of a URL.
2026-10-12 10:11:54 +02:00
Robin 3927c2b7dc Show the whole transport in debug UI and logs
A transport may soon carry a LiveKit SFU `url` instead of, or as well as,
a `livekit_service_url`, so a service URL alone no longer identifies it.
Pass the whole transport to the tiles' debug stats and print it as JSON in
log lines instead of picking out the service URL.
2026-10-12 10:09:50 +02:00
Robin 4e4ab23cda Model ConnectionState as a discriminated union
The enum mixed our own lifecycle states with a copy of livekit-client's
connection states, and errors were smuggled through the same observable as
a separate union member. A discriminated union keeps the LiveKit state
nested under "authenticated", carries the error as data, and lets the
local member represent the time before a connection exists as an
"initializing" state instead of null.
2026-10-12 10:08:22 +02:00
Robin 3f030e64dd Match LiveKit participants against all backend identities
A member authenticated through the legacy JWT endpoint publishes under a
`user:device` identity, while one authenticated through the default
endpoint publishes under a hashed identity. Both are listed in the
membership's backend identities, so match against the whole list rather
than only the primary one.
2026-10-12 10:04:47 +02:00
Johannes Marbach d8becaf868 Advertise the loopback address from the dev LiveKit SFUs
The container IP is not routable from the host when Docker runs in a VM
(colima, Docker Desktop), so calls against the dev stack drop once ICE
fails. Advertising 127.0.0.1 instead lets the browser reach the SFU through
the published ports. Firefox ignores loopback candidates, so this only
works in Chromium-based browsers.
2026-10-12 10:03:03 +02:00
Robin 994cd36c44 Fix typo in the name of the createJWTToken Playwright fixture 2026-10-12 10:03:03 +02:00
Johannes Marbach 7ac5373af9 Remove stray pause and improve logging for hung e2e tests
The list reporter is the only progress a hung CI run leaves behind, and a
global timeout under the job's 60 minute limit means a hung run still ends
with a report instead of being cancelled by GitHub.
2026-10-12 10:03:03 +02:00
renovate[bot] 32fcede65e Update ghcr.io/element-hq/element-web:develop Docker digest to d43e8d2 (#4345)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-12 02:40:40 +00:00
renovate[bot] 6438c18b73 Update ghcr.io/element-hq/element-web:develop Docker digest to 88eeb34 (#4344)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-11 00:49:17 +00:00
renovate[bot] f07e9f5c70 Update ghcr.io/element-hq/element-web:develop Docker digest to 1488790 (#4343)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-10 02:09:04 +00:00
Johannes Marbach c97e3bf7a6 Merge pull request #4337 from element-hq/johannes/remove-per-participant-e2ee-param
Remove perParticipantE2EE
2026-10-09 12:01:45 +02:00
Timo K. 674692c926 Fix breaking Ci tests due to compound-web 10.1.0 -> 10.3.1 bump 2026-10-09 11:09:41 +02:00
Timo K. c3d07b89bd review: unnecassary matchMedia overwrite on test start. 2026-10-09 11:08:03 +02:00
renovate[bot] f89cbc140e Update ghcr.io/element-hq/element-web:develop Docker digest to 181a526 (#4339)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-09 01:18:17 +00:00
Timo K. 3bd892ebd3 bump compound-web 2026-10-08 21:26:34 +02:00
Timo K. f322af2742 [ElementCallComponent] Use compound-web's PortalRoot in EC. 2026-10-08 19:14:01 +02:00
Johannes Marbach a9fa48f68c Remove perParticipantE2EE 2026-10-08 17:15:19 +02:00
Robin 471c76c0e7 Merge pull request #4336 from element-hq/robin/vite-warning
Fix Vite warning
2026-10-08 16:40:45 +02:00
Johannes Marbach 4881fd5fc6 Merge pull request #4280 from element-hq/johannes/claude
Add workflow for automated Claude review of pull requests
2026-10-08 15:27:13 +02:00
Johannes Marbach 49c29f160a Add workflow for automated Claude review of pull requests 2026-10-08 14:57:19 +02:00
Robin aa8f73e632 Fix Vite warning 2026-10-08 14:23:59 +02:00
Johannes Marbach 14c561af8d Merge pull request #4300 from element-hq/johannes/pure-layout-functions
Make the layout decisions pure / testable functions
2026-10-08 13:20:44 +02:00
Johannes Marbach 4fd2f66949 Explain why we use merge 2026-10-08 11:14:26 +02:00
Johannes Marbach 53e038b09b layoutShallowEquals -> shallowEquals 2026-10-08 10:57:03 +02:00
renovate[bot] 3b7ccb63e2 Update ghcr.io/element-hq/element-web:develop Docker digest to 53a4fe6 (#4334)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-08 02:17:28 +00:00
Johannes Marbach 5d34f57567 Set up jig for testing Matrix 2.0 2026-10-07 16:16:21 +02:00
Johannes Marbach 1f07f949b5 Merge pull request #4330 from element-hq/renovate/npm-postcss-selector-parser-vulnerability
Update dependency postcss-selector-parser to v7.1.6 [SECURITY]
2026-10-07 14:59:28 +02:00
renovate[bot] a60036b3d7 Update dependency postcss-selector-parser to v7.1.6 [SECURITY] 2026-10-07 12:37:14 +00:00
Timo 2eb3a2fd2d Merge pull request #4329 from element-hq/toger5/remove-lk-client-from-external-deps
Remove livekit-client from the list of external dependencies
2026-10-07 14:34:47 +02:00
Timo K. 8b6adf067e remove unnecassary (confused) comment 2026-10-07 14:20:42 +02:00
Timo K. b408610277 Remove livekit-client from the list of external dependencies 2026-10-07 12:01:21 +02:00
renovate[bot] 2faa6c3dac Update ghcr.io/element-hq/element-web:develop Docker digest to 589c490 (#4328)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-07 01:42:16 +00:00
renovate[bot] 9b3c3569ec Update ghcr.io/element-hq/element-web:develop Docker digest to 39c94f0 (#4327)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-06 02:57:18 +00:00
Johannes Marbach a4654d9d00 Merge pull request #4315 from element-hq/johannes/workflow-call
Allow workflow_call on lint and test workflows
2026-10-05 14:07:07 +02:00
Johannes Marbach 6e5f71c796 Revert back to negative flags 2026-10-05 13:53:52 +02:00
Johannes Marbach 067f3d3038 Fix repo selection and flags 2026-10-05 13:44:48 +02:00
Johannes Marbach 6d70221fe6 Flip boolean flags 2026-10-05 13:04:34 +02:00
renovate[bot] 427376f78b Update ghcr.io/element-hq/element-web:develop Docker digest to 573daa2 (#4323)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-05 09:47:57 +00:00
Johannes Marbach ccfd36923d Merge pull request #4324 from element-hq/renovate/compound
Update dependency @vector-im/compound-design-tokens to v11.1.0
2026-10-05 08:41:57 +02:00
Johannes Marbach 89567726f8 Fix tests 2026-10-05 08:13:58 +02:00
Robin 9cd111b5f2 Merge pull request #4318 from element-hq/redact-posthog-user-id
Redact analytics ID from rageshakes
2026-10-05 07:44:16 +02:00
renovate[bot] 096aaac9f3 Update dependency @vector-im/compound-design-tokens to v11.1.0 2026-10-05 01:27:09 +00:00
renovate[bot] 8835a6d75d Update ghcr.io/element-hq/element-web:develop Docker digest to 3cf23ff (#4322)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-04 09:34:33 +00:00
renovate[bot] bfb868aec3 Update ghcr.io/element-hq/element-web:develop Docker digest to 5371503 (#4321)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-10-03 01:48:09 +00:00
Robin 4e8fb6e4cb Redact analytics ID from rageshakes
To improve our minimization of external data we collect in rageshakes.
2026-10-02 16:51:00 +02:00
Robin 7c53a6be5a Merge pull request #4317 from element-hq/remove-wait-for-rotation
Remove wait_for_key_rotation_ms config option
2026-10-02 15:14:14 +02:00
Robin 64b932f128 Remove unused vite-plugin-generate-file dependency 2026-10-02 14:57:34 +02:00
Robin bfad85a3af Remove wait_for_key_rotation_ms config option
I learned that makeKeyDelay has been deprecated in matrix-js-sdk ever since the switch to the new encryption manager, so this config option has no effect.
2026-10-02 14:49:44 +02:00
Johannes Marbach be822adec8 Allow workflow_call on lint and test workflows 2026-10-02 13:33:29 +02:00
Johannes Marbach 4c0f02ef47 Merge pull request #4309 from element-hq/johannes/layered
Set up a layered build for CI similar to what EW does
2026-10-02 13:22:19 +02:00
Johannes Marbach ca6319f06d Remove layered boolean 2026-10-02 13:07:09 +02:00
Timo 1d57a8eee1 Merge pull request #4308 from element-hq/toger5/fix-component-sounds
[ElementCall Component] Decode inlined sounds directly instead of fetching them
2026-10-02 12:45:31 +02:00
Timo K. 0bb9632b60 review play sounds does not throw 2026-10-02 11:46:54 +02:00
Timo K. cbe195d449 review audio data loading 2026-10-02 11:46:40 +02:00
Robin e800fec74d Merge pull request #4314 from element-hq/renovate/element-web
Update ghcr.io/element-hq/element-web:develop Docker digest to 588f0b4
2026-10-02 11:29:05 +02:00
Robin 9d0725ad31 Fix Element Web integration tests 2026-10-02 10:51:31 +02:00
renovate[bot] 241412f29b Update ghcr.io/element-hq/element-web:develop Docker digest to 588f0b4 2026-10-02 02:54:17 +00:00
Timo K. d96d00b64a Merge branch 'main' into toger5/fix-component-sounds 2026-10-01 19:07:39 +02:00
Robin cde88a7c93 Merge pull request #4312 from element-hq/js-sdk-develop
Reset matrix-js-sdk to develop branch, adopt new join/leave signatures
2026-10-01 18:55:16 +02:00
Robin d5733e8274 Reset matrix-js-sdk to develop branch, adopt new join/leave signatures 2026-10-01 17:11:24 +02:00
Timo K. 3befe3d34b Merge branch 'main' into toger5/fix-component-sounds 2026-10-01 13:58:23 +02:00
Timo 3ad778e7e8 Merge pull request #4306 from element-hq/toger5/fix-component-tooltips
[ElementCall Component] Make compound a peer dependency with EW. (fix tooltips in embedded mode)
2026-10-01 13:57:58 +02:00
Timo K. 9a286032f2 shorten readme 2026-10-01 13:41:07 +02:00
Timo K. 363a8feab2 Stop bundling Compound, let the host supply it 2026-09-30 17:10:17 +02:00
Johannes Marbach bd8c3142a6 Set up a layered build for CI similar to what EW does 2026-09-30 11:35:06 +02:00
Timo K. 16c60c8b6b Warn when the audio context fails to set up. 2026-09-30 11:22:19 +02:00
Timo K. 63ac1ba03e Decode inlined sounds directly instead of fetching them, with the test. 2026-09-30 11:22:15 +02:00
Robin 9e190e9e57 Merge pull request #4307 from element-hq/update-js-sdk
Adapt to matrix-js-sdk removal of livekit_alias and oldest_membership
2026-09-30 10:41:37 +02:00
Robin f287fb3800 Adapt to matrix-js-sdk removal of oldest_membership support 2026-09-30 00:45:19 +02:00
Robin f9b601504b Stop sending livekit_alias in memberships
It's been many versions since anything used the value.
2026-09-30 00:45:19 +02:00
Timo K. 9140365251 Merge branch 'main' into toger5/fix-component-tooltips 2026-09-29 16:25:15 +02:00
Timo 81233e5cfb Merge pull request #4269 from element-hq/toger5/component-api-entry
Export EC component related types
2026-09-29 16:24:26 +02:00
Timo K.andClaude Fable 5.1 e296d88969 Keep the component's tooltips inside its root
Compound's Tooltip floats its bubble through floating-ui's FloatingPortal,
into the body: outside the element the component build confines its
stylesheet to, so a label tooltip shows as bare text at all times. An
enclosing FloatingPortal rooted at the container makes the nested portals
land inside it instead.

The call goes in an isolating wrapper there, standing in for the standalone
page's #root, so that its footer layer no longer paints above the modals.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 12:08:44 +02:00
Robin fbce73ae55 Merge pull request #4302 from element-hq/always-multi-sfu
Remove multi-SFU check which is always true
2026-09-29 09:14:42 +02:00
renovate[bot] ce79274ccc Update ghcr.io/element-hq/element-web:develop Docker digest to cbb7be9 (#4303)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-29 01:07:59 +00:00
Robin 8db2e8ac41 Remove multi-SFU check which is always true
Compatibility and Matrix 2.0 are the only MatrixRTC modes now.
2026-09-28 22:03:16 +02:00
Johannes Marbach 12352b94f5 Merge pull request #4299 from element-hq/johannes/drop-observable-hooks
Drop observable-hooks in favor of useBehavior
2026-09-28 18:12:43 +02:00
Johannes MarbachandRobin e393b6a984 Swap conditional ordering
Co-authored-by: Robin <robin@robin.town>
2026-09-28 17:56:08 +02:00
Johannes Marbach 3f63a6fc3c Drop observable-hooks in favor of useBehavior
With this change, views now read every reactive value through `useBehavior`, as
`architecture.md` already prescribes. Where a view reads a plain stream, the view
model now exposes a `Behavior`.
2026-09-28 17:54:50 +02:00
Johannes Marbach 6600c891f3 Extract the layout decision into a pure function 2026-09-28 17:36:24 +02:00
Johannes Marbach 33cfd2d8ad Build the layout media from one snapshot of its inputs 2026-09-28 17:34:22 +02:00
Johannes Marbach 5a8714805c Extract the spotlight and PiP choice into a pure function 2026-09-28 17:29:41 +02:00
Johannes Marbach 232e3f7ad4 Extract the spotlight speaker choice into a pure function 2026-09-28 17:28:14 +02:00
Johannes Marbach 10adf6658b Let layoutShallowEquals compare any layout-shaped object 2026-09-28 17:27:04 +02:00
Johannes Marbach 29742006df Move WindowMode next to the layout types to prevent cyclic import 2026-09-28 17:26:11 +02:00
Johannes Marbach addc1b10a0 Merge pull request #4297 from element-hq/renovate/compound
Update dependency @vector-im/compound-web to v10.1.0
2026-09-28 08:23:32 +02:00
renovate[bot] ee0d7dc0cb Update ghcr.io/element-hq/element-web:develop Docker digest to cfbe7ce (#4296)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-28 01:14:27 +00:00
renovate[bot] 7b1f3f28ed Update dependency @vector-im/compound-web to v10.1.0 2026-09-28 01:03:27 +00:00
renovate[bot] 8c44ecd61b Update ghcr.io/element-hq/element-web:develop Docker digest to 8301993 (#4295)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-27 01:25:15 +00:00
renovate[bot] 7d92ddf0c9 Update ghcr.io/element-hq/element-web:develop Docker digest to d97061c (#4294)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-26 02:21:38 +00:00
Johannes Marbach 5c497cc6a8 Fix build on Windows 2026-09-25 16:15:39 +02:00
fkwpandClaude Opus 5 b7bd7d18b2 [Feature] Quick audio menu (#4275)
* Give the microphone level its own observable
* Add a microphone level meter
* Show speakers and microphones in the quick audio menu
* Wire speaker selection through the call footer
* Add stories for the meter and the device menu
* Add end-to-end specs for the quick audio menu
* Keep the footer while a menu opened from it is open
* Use the shared audio capture stub in the lobby test
---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 09:52:02 +00:00
fkwpandClaude Opus 5.5 b4b72432b9 Tighten the comment guidance in AGENTS.md (#4279)
* Say what earns a comment its place

- Add a comments section to the code style: no comment by default, the reason
  only, and the kinds that go stale.
- Point to it from the PR checklist in AGENTS.md.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 08:44:04 +00:00
Robin ac1f5e2c8c Merge pull request #4277 from element-hq/vitest-warning
Fix Vitest import warning
2026-09-23 16:04:47 +02:00
renovate[bot] 98d77cc816 Update ghcr.io/element-hq/element-web:develop Docker digest to 0636bbe (#4276)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-23 01:55:54 +00:00
Robin d5812ddd34 Fix Vitest import warning
Fixes the following warning:

[warn] (!) Your Vite config uses features that are unsupported by `configLoader: 'native'`, which is planned to become the default in a future major version of Vite:
  - import "./vite.config" without a file extension (vitest.config.ts:3:35). Add the file extension
Set `VITE_CONFIG_NATIVE_IGNORE_WARNING=true` to suppress this warning.
2026-09-22 23:34:19 +02:00
Johannes Marbach cf7c275fb7 Merge pull request #4274 from element-hq/hs/bell
Bell reaction sound.
2026-09-22 12:32:38 +02:00
Half-Shot 93b2661513 Switch to 🛎️ 2026-09-22 11:21:11 +01:00
Robin 1f84cb9c5c Merge pull request #4273 from element-hq/simplify-local-transport
Simplify local transport code
2026-09-22 12:15:34 +02:00
Half-Shot 4269aaa163 update snap 2026-09-22 10:04:12 +01:00
Half-Shot bc62bb9565 lint 2026-09-22 10:01:22 +01:00
Half-Shot be56bbd05b lint 2026-09-22 10:01:10 +01:00
Half-Shot eb44fd545d Bell reaction sound. 2026-09-22 09:57:43 +01:00
Robin c9f911b7c9 Swallow local transport errors in connection manager 2026-09-22 10:01:14 +02:00
Robin 3a189d155b Remove references to outdated variables or types 2026-09-22 09:56:02 +02:00
Robin 0211d5c367 Remove unused shareReplay operator 2026-09-22 09:45:48 +02:00
Robin c88f884fc9 Simplify local transport code
As of the removal of 'Legacy' mode, there can no longer be a difference between the transport you advertise in your membership and the transport you publish media on, so the local transport code can be simplified considerably. For instance, the function which gets the local transport can simply return a promise rather than being reactive.

I kept the local transport as an Observable in other modules so that they could easily be tested with existing marble tests.
2026-09-21 12:51:28 +02:00
Johannes Marbach 9f7c35cd8f Merge pull request #4270 from element-hq/renovate/major-compound
Update dependency @vector-im/compound-design-tokens to v11
2026-09-21 07:51:35 +02:00
renovate[bot] ac81a7c0ab Update dependency @vector-im/compound-design-tokens to v11 2026-09-21 02:56:00 +00:00
Timo K. 8ebb0b5a90 Export component related types. 2026-09-19 08:46:30 +02:00
183 changed files with 6626 additions and 7938 deletions
+3
View File
@@ -102,6 +102,9 @@
- name: "backport-candidate"
description: "Something that is a candidate for backport to a particular release branch"
color: "0B8D85"
- name: "Claude-Review"
description: "Request code review from Claude"
color: "e57157"
- name: "customer-retainer"
color: "F44A5F"
- name: "dependencies"
+3 -1
View File
@@ -44,7 +44,9 @@ jobs:
node-version-file: ".node-version"
- name: Install dependencies
# ignore-pnpmfile should never be commited. Make CI crash if it happened (`pnpmfileChecksum` is present)
run: "pnpm install --frozen-lockfile --ignore-pnpmfile"
run: "scripts/layered.sh --ignore-pnpmfile"
env:
PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
- name: Build Element Call
run: pnpm run build:"$PACKAGE":"$BUILD_MODE"
env:
+90
View File
@@ -0,0 +1,90 @@
name: Claude
on:
pull_request:
types: [labeled]
permissions:
# The action exchanges the GitHub OIDC token for a short-lived Anthropic
# access token. GitHub API access goes through CLAUDE_GH_TOKEN instead.
id-token: write
jobs:
review:
name: Review
# Only run on labelling and skip forks.
if: github.event.label.name == 'Claude-Review' && github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
# The review looks at the individual commits, not just the combined diff
fetch-depth: 0
persist-credentials: false
- name: Review
uses: anthropics/claude-code-action@6fed3ca145920b639991cb756090506e1bcaf515 # v1.0.245
with:
anthropic_federation_rule_id: fdrl_01XQmWo4nka1ZTqZdsdeScyr
anthropic_organization_id: dfbc9a23-d636-4653-8970-035204ef60fa
anthropic_service_account_id: svac_018Ay5qaA7ZW8X7XsbdekqUM
anthropic_workspace_id: wrkspc_01Aw8LHjBS6fZysu9yqzYifW
# The token used here must grant read access to code, issues, and metadata as well as
# read and write access to pull requests.
github_token: ${{ secrets.CLAUDE_GH_TOKEN }}
use_sticky_comment: true
track_progress: true
prompt: |
Review this pull request under the following (and no further) angles:
1. Scope:
- Does the PR do one thing?
- Is the PR free of unrelated fixes and drive-by refactorings?
- If the PR is large, could it reasonably be split into smaller PRs
that still deliver value?
2. Commits:
- Is the work split into self-contained commits of reasonable size?
- Are the commits arranged in a sensible order?
- Have required refactorings been split out into separate commits?
- Is the history free of "wip", checkpoint and fixup-style commits
which only correct an earlier commit in the same PR?
3. Code:
- Are the code changes functionally correct?
- Do the changes introduce new security vulnerabilities or metadata
leaks?
- Is the new code performant enough?
- Don't check anything that is already validated via other GitHub
workflows like linting, formatting or tests passing.
Read the commits themselves with `git log` and `git diff` against
the base branch of the PR.
Drop any findings that are outside the above review angles or that don't
require resolution. Grade the severity of each finding as either "🟡 low",
"🟠 medium" or "🔴 high". Devise at least one possible resolution for each
finding.
Report the graded findings and resolutions from the "Code" angle as inline
comments. Add a footer "_-- Finding from automated review_" to each inline
comment.
Finally, report your overall result as PR comment. Start your message with
"# Automated review". Then add one section per review angle with:
- A verdict of "✅ PASS" or "❌ NEEDS WORK"
- A bullet list summarizing the remaining non-inlined graded findings and
resolutions (as nested bullets).
Be brief and concise in all your communication. Do not mention what you have
not covered and don't bring up issues that have already been fixed.
Use Markdown links for anything that can be navigated to on GitHub such
as files, commits, issues, pull requests, workflow runs, etc.
claude_args: |
--model claude-fable-5-1
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(git log:*),Bash(git diff:*),Bash(git show:*)"
--max-turns 50
+12 -1
View File
@@ -1,6 +1,12 @@
name: Lint, format & type check
on:
pull_request: {}
workflow_call:
inputs:
matrix-js-sdk-sha:
type: string
required: false
description: The matrix-js-sdk commit to build against instead of the locked one
jobs:
lint:
name: Lint, format & type check
@@ -9,6 +15,8 @@ jobs:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
# Inside a called workflow, github.repository is the caller so we need to make sure to still check out the EC repo here.
repository: ${{ inputs.matrix-js-sdk-sha && 'element-hq/element-call' || github.repository }}
persist-credentials: false
- name: Enable Corepack
run: corepack enable
@@ -19,7 +27,10 @@ jobs:
node-version-file: ".node-version"
- name: Install dependencies
# ignore-pnpmfile should never be commited. Make CI crash if it happened (`pnpmfileChecksum` is present)
run: "pnpm install --frozen-lockfile --ignore-pnpmfile"
run: "scripts/layered.sh --ignore-pnpmfile"
env:
PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
JS_SDK_GITHUB_BASE_REF: ${{ inputs.matrix-js-sdk-sha }}
- name: Formatting
run: "pnpm run format:check"
- name: i18n
+36 -3
View File
@@ -3,6 +3,25 @@ on:
pull_request: {}
push:
branches: [main]
workflow_call:
inputs:
matrix-js-sdk-sha:
type: string
required: false
description: The matrix-js-sdk commit to build against instead of the locked one
disable_coverage:
type: boolean
required: false
default: false
description: Skip generating and uploading coverage reports
disable_e2e_tests:
type: boolean
required: false
default: false
description: Skip the end-to-end tests
secrets:
CODECOV_TOKEN:
required: false
jobs:
vitest:
name: Run unit tests
@@ -11,6 +30,8 @@ jobs:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
# Inside a called workflow, github.repository is the caller so we need to make sure to still check out the EC repo here.
repository: ${{ inputs.matrix-js-sdk-sha && 'element-hq/element-call' || github.repository }}
persist-credentials: false
- name: Enable Corepack
run: corepack enable
@@ -21,7 +42,10 @@ jobs:
node-version-file: ".node-version"
- name: Install dependencies
# ignore-pnpmfile should never be commited. Make CI crash if it happened (`pnpmfileChecksum` is present)
run: "pnpm install --frozen-lockfile --ignore-pnpmfile"
run: "scripts/layered.sh --ignore-pnpmfile"
env:
PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
JS_SDK_GITHUB_BASE_REF: ${{ inputs.matrix-js-sdk-sha }}
- name: Get Playwright version
run: echo "PLAYWRIGHT_VERSION=$(pnpm list @playwright/test --depth=0 --json | jq -r '.[0].devDependencies["@playwright/test"].version')" >> $GITHUB_ENV
- name: Cache Playwright binaries
@@ -35,8 +59,11 @@ jobs:
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: pnpm playwright install --with-deps
- name: Vitest
run: "pnpm run test:coverage"
run: pnpm vitest --coverage.enabled="$ENABLE_COVERAGE"
env:
ENABLE_COVERAGE: ${{ !inputs.disable_coverage }}
- name: Upload to codecov
if: ${{ !inputs.disable_coverage }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
@@ -45,11 +72,14 @@ jobs:
fail_ci_if_error: true
playwright:
name: Run end-to-end tests
if: ${{ !inputs.disable_e2e_tests }}
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
# Inside a called workflow, github.repository is the caller so we need to make sure to still check out the EC repo here.
repository: ${{ inputs.matrix-js-sdk-sha && 'element-hq/element-call' || github.repository }}
persist-credentials: false
- name: Enable Corepack
run: corepack enable
@@ -59,7 +89,10 @@ jobs:
node-version-file: ".node-version"
- name: Install dependencies
# ignore-pnpmfile should never be commited. Make CI crash if it happened (`pnpmfileChecksum` is present)
run: pnpm install --frozen-lockfile --ignore-pnpmfile
run: scripts/layered.sh --ignore-pnpmfile
env:
PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
JS_SDK_GITHUB_BASE_REF: ${{ inputs.matrix-js-sdk-sha }}
- name: Get Playwright version
run: echo "PLAYWRIGHT_VERSION=$(pnpm list @playwright/test --depth=0 --json | jq -r '.[0].devDependencies["@playwright/test"].version')" >> $GITHUB_ENV
- name: Cache Playwright binaries
+4
View File
@@ -2,6 +2,7 @@ node_modules
.DS_Store
.env
dist
dist-embedded
dist-ssr
*.local
*.bkp
@@ -41,3 +42,6 @@ yarn-error.log
*storybook.log
storybook-static
# Nested checkout made by scripts/layered.sh
/matrix-js-sdk/
+2
View File
@@ -48,6 +48,8 @@ pnpm i18n:check
- View model + marble tests, thin view, a story per state, an e2e spec.
- Any new shared component called out explicitly, with why nothing existing fit.
- Template filled for real: what, why, before/after screenshots, repro steps.
- Comments only where the code can't give the reason — see
[code-style.md](docs/agents/code-style.md#comments).
- Anything that redraws continuously — an animation frame, a stream — says what it
costs, or better, counts the redraws in a test.
- Branch `<handle>/<topic>`. Plain imperative commit subjects, no prefixes.
+57 -10
View File
@@ -34,16 +34,16 @@ You can find the latest development version continuously deployed to
## ✨ Key Features
✅ **Decentralized & Federated** – No central authority; works across Matrix
homeservers.
✅ **End-to-End Encrypted** – Secure and private calls.
homeservers.
✅ **End-to-End Encrypted** – Secure and private calls.
✅ **Standalone, Widget & Component Mode** – Use as an independent app, embed
in Matrix clients as a widget, or (experimentally) mount it as a React component
inside your own application.
✅ **WebRTC-based** – No additional software required.
inside your own application.
✅ **WebRTC-based** – No additional software required.
✅ **Scalable with LiveKit** – Supports large meetings via SFU
([MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)).
([MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)).
✅ **Raise Hand** – Participants can signal when they want to speak, helping to
organize the flow of the meeting.
organize the flow of the meeting.
✅ **Emoji Reactions** – Users can react with emojis 👍️ 🎉 👏 🤘, adding
engagement and interactivity to the conversation.
@@ -297,8 +297,8 @@ The package is not published yet. A host installs it as a git dependency on the
```
whose `prepare` script runs the build on install. That build needs pnpm (via
Corepack) on the host's machine, runs a full `pnpm install` of this repository
and is memory-hungry, since it inherits the `--max-old-space-size` setting of
Corepack) on the host's machine, runs a full `pnpm install` of this repository,
since it inherits the `--max-old-space-size` setting of
the app build; the host's pnpm also has to allow it to run at all
(`allowBuilds` in its `pnpm-workspace.yaml`). Note that `component/` is a pnpm
project of its own for this reason, so pnpm commands run from inside that
@@ -306,8 +306,28 @@ directory target it rather than the repository; run them from the repository
root. The host imports the component from
`@element-hq/element-call-component` and the stylesheet from
`@element-hq/element-call-component/style.css`, and has to provide `react`,
`react-dom`, `matrix-js-sdk` and `livekit-client` itself, since the bundle leaves
them external.
`react-dom`, `matrix-js-sdk`, `@vector-im/compound-web` and
`@vector-im/compound-design-tokens` itself, since the bundle leaves them
external.
Compound's stylesheet are the host's to load too: the component shares
the host's copy of Compound, (required for example for tooltips)
The component is large, and a host will usually load it lazily, only once a
call is shown. Everything a host needs in order to talk about a call before
then is also exported
from `@element-hq/element-call-component/api` (a small entry point that does not
load the component).
- props and the types they are made of
- the handle
- the host bridge
- `UserIntent`
- `HeaderStyle`
- `BackgroundStyle` enums
- `configurationForIntent`, the defaults each intent implies
Import from it wherever a value such as
`BackgroundStyle.Solid` is needed on a path that must stay light; the main
entry point re-exports all of it too.
### Backend
@@ -341,6 +361,33 @@ pnpm backend
# podman-compose -f docker-compose-dev.yml up
```
The two Element Web instances load the Element Call widget from
`./dist-embedded`, which is mounted over the copy bundled in the Element Web
image. Build it from the current sources before starting the backend, and keep
it rebuilding on change:
```sh
VITE_MATRIX_RTC_MODE=matrix_2_0 pnpm build:embedded:development --outDir dist-embedded --watch
```
Build it at least once before the first `pnpm backend`, since Docker otherwise
creates an empty `dist-embedded` directory that the build cannot write to.
Reload the page after a rebuild to pick up the new widget. The widget has no
`config.json` in this setup, so `VITE_MATRIX_RTC_MODE` sets the default
MatrixRTC mode of the build instead (see
[docs/matrix_rtc_modes.md](docs/matrix_rtc_modes.md)); it can still be changed
under Settings → Developer. Do not delete `dist-embedded` while the backend is
running: the containers hold a mount of the directory, and a new directory of
the same name is not picked up until they are recreated.
> [!NOTE]
> When Docker runs in a VM (colima, Docker Desktop), the SFUs' container IPs
> are not routable from the host, so they advertise `127.0.0.1` as their ICE
> candidate instead and the browser reaches them through the published ports.
> This only works in Chromium-based browsers: Firefox ignores loopback
> candidates, and calls drop about fifteen seconds after joining. Media runs
> over ICE-TCP, since the UDP port ranges are not forwarded out of the VM.
> [!NOTE]
> To ensure your local development frontend functions properly, you’ll need to
> add certificate exceptions in your browser for `https://localhost:3000` and
+1 -1
View File
@@ -5,7 +5,7 @@ sender_localpart: "_lk_jwt_service"
namespaces:
users:
- exclusive: false
regex: "@.*:homeserver-1" # Cover all local users
regex: "@.*:synapse\\.othersite\\.m\\.localhost" # Cover all local users
# Required for Synapse to call POST /_matrix/app/v1/ping on this service.
url: "http://auth-server-1:16080"
# Enable membership look-ups via /is_joined.
+1 -1
View File
@@ -5,7 +5,7 @@ sender_localpart: "_lk_jwt_service"
namespaces:
users:
- exclusive: false
regex: "@.*:homeserver" # Cover all local users
regex: "@.*:synapse\\.m\\.localhost" # Cover all local users
# Required for Synapse to call POST /_matrix/app/v1/ping on this service.
url: "http://auth-server:6080"
# Enable membership look-ups via /is_joined.
+5
View File
@@ -6,6 +6,11 @@ rtc:
port_range_start: 50300
port_range_end: 50400
use_external_ip: false
# Advertise the loopback address instead of the container IP, which is not
# routable from the host when Docker runs in a VM (colima, Docker Desktop).
# The browser then reaches the SFU through the published ports. Firefox
# ignores loopback candidates, so this only works in Chromium-based browsers.
node_ip: 127.0.0.1
turn:
enabled: false
domain: localhost
+5
View File
@@ -6,6 +6,11 @@ rtc:
port_range_start: 50100
port_range_end: 50200
use_external_ip: false
# Advertise the loopback address instead of the container IP, which is not
# routable from the host when Docker runs in a VM (colima, Docker Desktop).
# The browser then reaches the SFU through the published ports. Firefox
# ignores loopback candidates, so this only works in Chromium-based browsers.
node_ip: 127.0.0.1
turn:
enabled: false
domain: localhost
+57
View File
@@ -0,0 +1,57 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "vitest";
import { render, screen } from "@testing-library/react";
import { type ReactNode } from "react";
import { Tooltip, TooltipProvider } from "@vector-im/compound-web";
import { CallPortalRoot } from "./CallPortalRoot";
// A button whose accessible name comes from its tooltip, as every icon button
// in the call does: the name is lost whenever the tooltip has nowhere to go.
const LabelledButton = (): ReactNode => (
<TooltipProvider>
<Tooltip label="Microphone">
<button>🎤</button>
</Tooltip>
</TooltipProvider>
);
test("floats into the host document with no root", () => {
render(
<CallPortalRoot root={undefined}>
<LabelledButton />
</CallPortalRoot>,
);
expect(screen.getByRole("button", { name: "Microphone" })).toBeTruthy();
});
test("floats into the root it is given", () => {
const root = document.createElement("div");
document.body.appendChild(root);
render(
<CallPortalRoot root={root}>
<LabelledButton />
</CallPortalRoot>,
);
const button = screen.getByRole("button", { name: "Microphone" });
const label = document.getElementById(
button.getAttribute("aria-labelledby")!,
);
expect(root.contains(label)).toBe(true);
root.remove();
});
test("waits for a root that is not available yet", () => {
render(
<CallPortalRoot root={null}>
<LabelledButton />
</CallPortalRoot>,
);
expect(screen.queryByRole("button", { name: "Microphone" })).toBeNull();
});
+26
View File
@@ -0,0 +1,26 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC, type ReactNode } from "react";
import { PortalRoot } from "@vector-im/compound-web";
interface Props {
root: HTMLElement | null | undefined;
children: ReactNode;
}
/**
* Where Compound floats the call's menus and tooltips. With no `root` the call
* is not wrapped at all and they go into the host document's body; `null` is
* Compound's "not available yet", and the floating parts wait for it.
*/
export const CallPortalRoot: FC<Props> = ({ root, children }): ReactNode =>
root === undefined ? (
children
) : (
<PortalRoot root={root}>{children}</PortalRoot>
);
+13 -5
View File
@@ -34,10 +34,18 @@ component rather than a page makes. */
-webkit-tap-highlight-color: transparent;
}
/* Compound's overlay container, which holds tooltips and popovers, has to fill
the container for the elements inside it to be positioned against it. The
standalone page does the same for the container under `#root`. */
.root > [data-overlay-container] {
position: relative;
/* The call itself. It is a stacking context of its own, as the standalone
page's `#root` is, so that the layers the call view uses for its header, footer
and overlays stay below the modals and toasts, which are portalled into the
root and carry no layer of their own.
What Compound floats — tooltips, menus — is not in here at all: it goes into
the host's body, as it would for any other component on the host's page, and is
styled and layered by the host's copy of Compound. */
.content {
display: flex;
flex-direction: column;
block-size: 100%;
position: relative;
isolation: isolate;
}
+138
View File
@@ -0,0 +1,138 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* EXPERIMENTAL
*
* What a host needs in order to talk about Element Call, without Element Call:
* the component's props and the types they are made of, the handle and the
* host bridge, the intents a call can be started with and the configuration
* each implies.
*
* This is an entry point of its own, `@element-hq/element-call-component/api`,
* so that a host can import it on a path that must stay light — a call model
* that needs `BackgroundStyle.Solid` as a value, say — and load the component
* itself, and everything it brings, only when a call is shown. The main entry
* point re-exports all of it, so a host that does not care may import
* everything from there.
*
* Nothing in here may reach anything that is not a type or a constant: the
* point of this module is what it does not import.
*/
import type { Ref } from "react";
import type { MatrixClient } from "matrix-js-sdk";
import type { UrlProperties } from "../src/UrlParams";
import type { ElementCallHandle, ElementCallHostBridge } from "./host";
import {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
} from "../src/UrlConfiguration";
// How the host and Element Call talk to each other, and what they say
export type { ElementCallHandle, ElementCallHostBridge } from "./host";
export type { DeviceMuteRequest, DeviceMuteState } from "../src/HostBridge";
export type { JoinCallData } from "../src/widget";
// The deployment-wide configuration, as distinct from ElementCallConfiguration
// below, which is per call
export type { ConfigOptions } from "../src/config/ConfigOptions";
// The values that appear in ElementCallConfiguration and in the intent, and
// what each intent means by default
export {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
};
/**
* How Element Call should behave. Everything is optional; anything left out
* takes the default that {@link ElementCallProps.intent} implies.
*
* This is the behaviour a widget can be configured with through its URL, plus
* the one fact about the call a host has a say in here, the background. The
* rest of what a widget's URL carries — who the user is, how to reach the
* homeserver, where to report analytics, the shared secret of a room that is
* encrypted with one — a component host supplies by other routes, or not at
* all; and what can change while the call is running, the theme and the
* language, is a prop of its own.
*/
export type ElementCallConfiguration = Partial<UrlConfiguration> &
Partial<Pick<UrlProperties, "background">>;
export interface ElementCallProps {
/**
* The client to place the call with. Element Call does not authenticate
* anyone or manage a session of its own; this one is the host's.
*/
client: MatrixClient;
/** The room to call in. The host's client must already know about it. */
roomId: string;
/**
* What the user asked for — whether they started the call or joined one that
* was already running, and whether it is a call in a group or a DM. Element
* Call decides what each of those means: whether to show the lobby first,
* whether to ring, and so on.
*
* Defaults to joining an existing group call, which is the most conservative
* reading, but a host that knows which button the user pressed should say so.
*/
intent?: UserIntent;
/**
* How Element Call should behave, overriding whatever {@link intent} implies.
* A host that finds itself setting a lot of these probably wants a different
* intent instead.
*
* Compared by value, so it is fine to write this inline; only a change to
* what it says restarts anything.
*/
config?: ElementCallConfiguration;
/**
* What Element Call tells the host while the call is running: that the user
* has joined or hung up, that it would like to be kept on screen, and so on.
* Without one, Element Call assumes nobody is listening.
*/
hostBridge?: ElementCallHostBridge;
/**
* What the host tells Element Call: to hang up, to mute, to join. Available
* once the component has rendered.
*/
ref?: Ref<ElementCallHandle>;
/**
* The theme to show Element Call in, `light` or `dark`. Left out, Element
* Call picks. Changes take effect at once, and cost nothing else.
*/
theme?: string;
/**
* The language to show Element Call in, as a BCP 47 tag: one of the
* `supportedLanguages` the main entry point exports, or something that falls
* back to one (`de-AT` to `de`). Left out, the browser's language is used.
*
* Translations are one thing shared by every Element Call on the page, so
* the most recently set language wins for all of them.
*/
language?: string;
/**
* Where the parts of Element Call that float above it — its menus and
* tooltips, which Compound portals out of the call — are rendered. Left out,
* they go into the host document's body, as they do for any other component
* on the host's page.
*
* A host that moves Element Call's DOM into another document, such as a
* Document Picture-in-Picture window, passes that document's body here so
* that a menu opens in the window its button is in. Changes take effect at
* once. `null` means the root is not available yet, as from a callback ref
* that has not fired: menus and tooltips wait for it rather than falling
* back to the body.
*/
portalRoot?: HTMLElement | null;
}
+8 -7
View File
@@ -40,10 +40,10 @@ const IS_ROOT = `:where(${ROOT_SELECTOR})`;
* Element Call as a component.
*
* As a page of its own, Element Call can style the document: normalize.css and
* Compound speak of `html`, `body` and bare elements, and the design tokens are
* declared on `:root`. As a component, all of that would land on the host's
* document too. This rewrites every selector so that it matches only the root
* or its descendants:
* its own base styles speak of `html`, `body` and bare elements, and its custom
* properties are declared on `:root`. As a component, all of that would land on
* the host's document too. This rewrites every selector so that it matches only
* the root or its descendants:
*
* - `html`, `body` and `:root` become the root element, which is what stands in
* for the document inside a host.
@@ -55,9 +55,10 @@ const IS_ROOT = `:where(${ROOT_SELECTOR})`;
* CSS modules are scoped by their class names already, so only their selectors
* that would match by element alone — `pre` rather than `.pre` — are touched.
*
* The root's fonts and design tokens are still inherited by everything inside
* it, the way they were from `body` and `:root`, and `@font-face` declarations
* stay global, which they are by nature.
* The root's custom properties are still inherited by everything inside it,
* the way they were from `:root`. Compound's own stylesheet — the design
* tokens, the component styles, the fonts — is not in this build at all: the
* host supplies it, unscoped, along with its copy of Compound.
*/
export function scopeStylesToRoot(): Plugin {
return {
+10
View File
@@ -9,6 +9,16 @@ import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import { logger } from "matrix-js-sdk/lib/logger";
// What a host provides along with its copy of Compound: its stylesheet, the
// design tokens and the fonts they name. The component does not bundle them.
import "@fontsource/inter/400.css";
import "@fontsource/inter/500.css";
import "@fontsource/inter/600.css";
import "@fontsource/inter/700.css";
import "@fontsource/inconsolata/400.css";
import "@fontsource/inconsolata/700.css";
import "@vector-im/compound-design-tokens/assets/web/css/compound-design-tokens.css";
import "@vector-im/compound-web/dist/style.css";
import { type ConfigOptions, initializeElementCall } from "../index";
import { Harness } from "./Harness";
// After Element Call's, so that the host has the last word on its own page
+51 -127
View File
@@ -17,33 +17,37 @@ Please see LICENSE in the repository root for full details.
* host instead, or is confined to the container it is mounted in.
*/
// The design tokens, fonts and element defaults every Element Call stylesheet
// The element defaults and custom properties every Element Call stylesheet
// builds on. Written for a page, they speak of `html`, `body` and bare
// elements; the component build confines them, and every other stylesheet in
// this bundle, to the root element below (see build/scopeStylesToRoot.ts), so
// that the host's document is left as it was.
//
// Compound is not in here. The component uses the host's copy of it (a peer
// dependency, left external by the build), and the host loads Compound's
// stylesheet, design tokens and fonts along with it: what Compound floats into
// the host's body — tooltips, menus — is then styled by the same stylesheet as
// everything else there, and Element Call's root takes the tokens and fonts of
// the theme class it carries, as any element on the host's page would.
//
// Where these land relative to the component stylesheets is the bundler's
// choice — the standalone app puts them first, this build puts them in the
// middle — so nothing in base.css may depend on winning or losing against a
// component's own rules at equal specificity. It currently does not: what it
// declares unlayered is custom properties on Element Call's root, which
// components inherit rather than compete with, and everything from Compound
// sits in a `@layer`, which loses to unlayered rules either way.
// components inherit rather than compete with.
import "../src/base.css";
import {
type FC,
type JSX,
type ReactNode,
type Ref,
useEffect,
useLayoutEffect,
useMemo,
useRef,
useState,
} from "react";
import { type MatrixClient } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import { I18nextProvider } from "react-i18next";
import { TooltipProvider } from "@vector-im/compound-web";
@@ -62,10 +66,8 @@ import { RootElementProvider, useRootElement } from "../src/RootElementContext";
import {
configurationForIntent,
componentProperties,
type UrlConfiguration,
type UrlParams,
UrlParamsProvider,
type UrlProperties,
UserIntent,
useUrlParams,
} from "../src/UrlParams";
@@ -79,102 +81,17 @@ import { i18n } from "../src/utils/i18n";
import { useTheme } from "../src/useTheme";
import { useStableValue } from "../src/useStableValue";
import styles from "./ElementCall.module.css";
import {
type ElementCallHandle,
type ElementCallHostBridge,
useComponentHostBridge,
} from "./host";
import { useComponentHostBridge } from "./host";
import { CallPortalRoot } from "./CallPortalRoot";
import { type ElementCallProps } from "./api";
import { supportedLanguages, translationsBackend } from "./localization";
// The languages Element Call can be shown in
export { supportedLanguages } from "./localization";
// How the host and Element Call talk to each other, and what they say
export { type ElementCallHandle, type ElementCallHostBridge } from "./host";
export {
type DeviceMuteRequest,
type DeviceMuteState,
} from "../src/HostBridge";
export { type JoinCallData } from "../src/widget";
// The deployment-wide configuration, as distinct from ElementCallConfiguration
// above, which is per call
export { type ConfigOptions } from "../src/config/ConfigOptions";
// The values that appear in ElementCallConfiguration and in the intent
export {
BackgroundStyle,
HeaderStyle,
UserIntent,
type UrlConfiguration,
} from "../src/UrlParams";
/**
* How Element Call should behave. Everything is optional; anything left out
* takes the default that {@link ElementCallProps.intent} implies.
*
* This is the behaviour a widget can be configured with through its URL, plus
* the one fact about the call a host has a say in here, the background. The
* rest of what a widget's URL carries — who the user is, how to reach the
* homeserver, where to report analytics, the shared secret of a room that is
* encrypted with one — a component host supplies by other routes, or not at
* all; and what can change while the call is running, the theme and the
* language, is a prop of its own.
*/
export type ElementCallConfiguration = Partial<UrlConfiguration> &
Partial<Pick<UrlProperties, "background">>;
export interface ElementCallProps {
/**
* The client to place the call with. Element Call does not authenticate
* anyone or manage a session of its own; this one is the host's.
*/
client: MatrixClient;
/** The room to call in. The host's client must already know about it. */
roomId: string;
/**
* What the user asked for — whether they started the call or joined one that
* was already running, and whether it is a call in a group or a DM. Element
* Call decides what each of those means: whether to show the lobby first,
* whether to ring, and so on.
*
* Defaults to joining an existing group call, which is the most conservative
* reading, but a host that knows which button the user pressed should say so.
*/
intent?: UserIntent;
/**
* How Element Call should behave, overriding whatever {@link intent} implies.
* A host that finds itself setting a lot of these probably wants a different
* intent instead.
*
* Compared by value, so it is fine to write this inline; only a change to
* what it says restarts anything.
*/
config?: ElementCallConfiguration;
/**
* What Element Call tells the host while the call is running: that the user
* has joined or hung up, that it would like to be kept on screen, and so on.
* Without one, Element Call assumes nobody is listening.
*/
hostBridge?: ElementCallHostBridge;
/**
* What the host tells Element Call: to hang up, to mute, to join. Available
* once the component has rendered.
*/
ref?: Ref<ElementCallHandle>;
/**
* The theme to show Element Call in, `light` or `dark`. Left out, Element
* Call picks. Changes take effect at once, and cost nothing else.
*/
theme?: string;
/**
* The language to show Element Call in, as a BCP 47 tag: one of
* {@link supportedLanguages}, or something that falls back to one (`de-AT`
* to `de`). Left out, the browser's language is used.
*
* Translations are one thing shared by every Element Call on the page, so
* the most recently set language wins for all of them.
*/
language?: string;
}
// Everything a host needs to talk about Element Call — the props, the handle,
// the host bridge, the intents and what each means — also available without
// the component itself from `@element-hq/element-call-component/api`
export * from "./api";
/**
* Prepares the things Element Call needs before it can be shown: translations,
@@ -238,6 +155,7 @@ export const ElementCall: FC<ElementCallProps> = ({
ref,
theme,
language,
portalRoot,
}): ReactNode => {
const hostBridge = useComponentHostBridge(suppliedHostBridge, ref, theme);
@@ -321,34 +239,40 @@ export const ElementCall: FC<ElementCallProps> = ({
<UrlParamsProvider value={params}>
<div ref={setContainer} className={styles.root}>
{ready && (
<RootElementProvider value={container}>
{/* Whatever goes wrong in here is shown in here. Left to
propagate, an error would unmount the host's own tree. */}
<ErrorBoundary
fallback={(error) => <ErrorPage error={error} />}
// A broken call should not hold the host on screen
onError={() => void hostBridge.setAlwaysOnScreen(false)}
>
<Decoration>
<TooltipProvider>
<ClientProvider client={client}>
<MediaDevicesContext value={mediaDevices}>
<ProcessorProvider>
<CallView
client={client}
rtcSession={rtcSession}
isPasswordlessUser={false}
confineToRoom={params.confineToRoom}
preload={params.preload}
skipLobby={params.skipLobby}
/>
</ProcessorProvider>
</MediaDevicesContext>
</ClientProvider>
</TooltipProvider>
</Decoration>
</ErrorBoundary>
</RootElementProvider>
// Stands in for the standalone page's `#root`: see `.content`
// in ElementCall.module.css
<div className={styles.content}>
<CallPortalRoot root={portalRoot}>
<RootElementProvider value={container}>
{/* Whatever goes wrong in here is shown in here. Left to
propagate, an error would unmount the host's own tree. */}
<ErrorBoundary
fallback={(error) => <ErrorPage error={error} />}
// A broken call should not hold the host on screen
onError={() => void hostBridge.setAlwaysOnScreen(false)}
>
<Decoration>
<TooltipProvider>
<ClientProvider client={client}>
<MediaDevicesContext value={mediaDevices}>
<ProcessorProvider>
<CallView
client={client}
rtcSession={rtcSession}
isPasswordlessUser={false}
confineToRoom={params.confineToRoom}
preload={params.preload}
skipLobby={params.skipLobby}
/>
</ProcessorProvider>
</MediaDevicesContext>
</ClientProvider>
</TooltipProvider>
</Decoration>
</ErrorBoundary>
</RootElementProvider>
</CallPortalRoot>
</div>
)}
</div>
</UrlParamsProvider>
+6 -1
View File
@@ -25,6 +25,10 @@
"types": "./dist/types/component/index.d.ts",
"default": "./dist/element-call.js"
},
"./api": {
"types": "./dist/types/component/api.d.ts",
"default": "./dist/api.js"
},
"./style.css": "./dist/element-call.css"
},
"sideEffects": [
@@ -34,7 +38,8 @@
"prepare": "cd .. && pnpm install --frozen-lockfile && pnpm build:component"
},
"peerDependencies": {
"livekit-client": "^2.18.1",
"@vector-im/compound-design-tokens": "^11.0.0",
"@vector-im/compound-web": "^10.3.1",
"matrix-js-sdk": "*",
"react": "^19",
"react-dom": "^19"
+3 -3
View File
@@ -13,8 +13,8 @@
"rootDir": "..",
"outDir": "./dist/types"
},
// The entry point, plus the ambient declarations (CSS modules, `?react` SVGs,
// `import.meta.env`, …) that the sources it reaches rely on.
"include": ["./index.tsx", "../src/@types/*.d.ts"],
// The entry points, plus the ambient declarations (CSS modules, `?react` SVGs,
// `import.meta.env`, …) that the sources they reach rely on.
"include": ["./index.tsx", "./api.ts", "../src/@types/*.d.ts"],
"exclude": []
}
-1
View File
@@ -7,7 +7,6 @@
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"network_error_retry_ms": 100
}
-1
View File
@@ -11,7 +11,6 @@
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_mode": "compatibility",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"network_error_retry_ms": 100
}
-1
View File
@@ -7,7 +7,6 @@
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"network_error_retry_ms": 100
},
-1
View File
@@ -7,7 +7,6 @@
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"network_error_retry_ms": 100
}
+8 -2
View File
@@ -3,7 +3,7 @@ networks:
services:
auth-service:
image: ghcr.io/element-hq/lk-jwt-service:0.7.0
image: ghcr.io/element-hq/lk-jwt-service:0.8.0
pull_policy: always
hostname: auth-server
environment:
@@ -33,7 +33,7 @@ services:
- ecbackend
auth-service-1:
image: ghcr.io/element-hq/lk-jwt-service:0.7.0
image: ghcr.io/element-hq/lk-jwt-service:0.8.0
pull_policy: always
hostname: auth-server-1
environment:
@@ -157,6 +157,9 @@ services:
pull_policy: always
volumes:
- ./backend/ew.test.config.json:/app/config.json:Z
# Replaces the Element Call widget bundled in the image with the embedded
# build of the current sources. See "Backend" in README.md.
- ./dist-embedded:/app/widgets/element-call:ro,Z
environment:
ELEMENT_WEB_PORT: 8081
ports:
@@ -169,6 +172,9 @@ services:
pull_policy: always
volumes:
- ./backend/ew.test.othersite.config.json:/app/config.json:Z
# Replaces the Element Call widget bundled in the image with the embedded
# build of the current sources. See "Backend" in README.md.
- ./dist-embedded:/app/widgets/element-call:ro,Z
environment:
ELEMENT_WEB_PORT: 18081
ports:
+2 -2
View File
@@ -19,7 +19,7 @@ services:
- ./backend/playwright_homeserver-othersite.yaml:/data/cfg/homeserver.yaml:Z
element-web:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:3993d53bc2c784c0376b86e00230aef0ee2c9a705b2166e9229555117e8db167
image: ghcr.io/element-hq/element-web:develop@sha256:d43e8d26361b546a9ceed13602cc13892ff212df7024b2282147ee8ec370977e
element-web-1:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:3993d53bc2c784c0376b86e00230aef0ee2c9a705b2166e9229555117e8db167
image: ghcr.io/element-hq/element-web:develop@sha256:d43e8d26361b546a9ceed13602cc13892ff212df7024b2282147ee8ec370977e
+9 -1
View File
@@ -76,4 +76,12 @@ Code that builds in only one is a bug.
- `build:component` — `@element-hq/element-call-component`, sources in `component/`
(its own pnpm project; run pnpm from the repo root). Host API is in the README;
`pnpm lint:externals` rejects an import of a `react` / `react-dom` /
`matrix-js-sdk` / `livekit-client` subpath the externals list omits.
`matrix-js-sdk` / `@vector-im/compound-web` /
`@vector-im/compound-design-tokens` subpath the externals list omits.
`livekit-client` is bundled on purpose: no host imports it, and the pinned
version travels with the component.
Compound's stylesheets are loaded by `src/index.css` for the page and by the
host for the component, never by `src/base.css`. Two entry
points: `index.tsx` (the component) and `api.ts` (types, enums and
`configurationForIntent`, for a host that must not load the component). `api.ts`
may only reach types and constants — `src/UrlConfiguration.ts`, not `UrlParams`.
+22 -1
View File
@@ -31,7 +31,28 @@ copy, the PR says so.
- **Suites.** Test cases first, helpers below. Older suites invert this; follow the
rule in new ones, don't reorder old ones.
- **Names.** What a thing means, not what it is made of: `naturalLayout$`, not
`computedLayout$`. Comments explain why, never what.
`computedLayout$`.
## Comments
No comment is the default. One earns its place by giving a reason the code
cannot: a platform quirk, an outside constraint, a trap that looks like a
simplification.
- Say the reason, then stop. Test each sentence: does the reason still stand
without it? If so, cut it. Most reasons fit on a line, so a long comment is
worth a second look — it is usually carrying something that isn't one.
- Write about the line the comment sits on, in terms of what the code does
there — leave layout to the stylesheet and measurements to the design.
- Name the condition ("where the platform can't route audio"), not the browser
or the spec entry.
- Put history — what was tried, removed or rejected, the dev journal — in the
commit or the PR description.
- In tests, comment only structure that looks wrong but isn't: batching, timers,
why this scroll position.
Before keeping one: would it still be true after a CSS change or a refactor of
its caller? If not, cut it.
## TypeScript, React and RxJS idiom win on a clash
+15
View File
@@ -60,6 +60,21 @@ git config --local core.hooksPath .githooks
This will add the hook path for this repository only to .gihooks. which is a tracked (by git) folder containing the pre-commit hook.
## Testing against a matrix-js-sdk branch in CI
Linking covers local development. In CI, `scripts/layered.sh` plays the same
role for `matrix-js-sdk`: after the usual frozen install, `scripts/fetchdep.sh`
looks for a matrix-js-sdk branch with the same name as the pull request's head
branch (in the fork first, when the pull request comes from one). If there is
one, it is cloned, packed and installed over the locked version. If there is
none, the install stays as `pnpm-lock.yaml` pins it.
This means a change that needs a not yet merged matrix-js-sdk change can be tested
by giving both branches the same name. This mirrors how element-web tests against
matrix-js-sdk. Once the matrix-js-sdk branch is merged and deleted, the pull
request falls back to the lockfile and needs a lockfile bump to stay green,
which is also what keeps `main` consistent after the merge.
## Background
Information, why this approach is used can be found in the [linking concept reasoning](./linking_concept_reasoning.md) document.
+9 -1
View File
@@ -24,7 +24,7 @@ homeserver does not support sticky events. This is the default.
MSC4354 and all clients on v0.17.0 or later. The local membership requests its
token from the Matrix 2.0 JWT endpoint of the
[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)
and identifies the room by a hashed identity instead of a `livekit_alias`.
and identifies the room by a hashed identity instead of the Matrix `room_id`.
(Remote memberships always try the new endpoint first and fall back to the
legacy one, so remote participants can be on either.)
@@ -46,3 +46,11 @@ Valid values are `compatibility` and `matrix_2_0`; an invalid value is
ignored (with a warning) and the user's choice applies. Pinning `matrix_2_0` on a
homeserver without sticky event support makes joining fail with a "sticky events
required" error.
A build can change the default mode, for setups without a `config.json` by setting
`VITE_MATRIX_RTC_MODE` at build time:
```sh
VITE_MATRIX_RTC_MODE=matrix_2_0 pnpm build:embedded
```
-1
View File
@@ -61,7 +61,6 @@ These parameters are relevant to both [widget](./embedded_standalone.md) and [st
| `homeserver` | | Not applicable | No | Homeserver for registering a new (guest) user, configures non-default guest user server when creating a spa link. |
| `lang` | [BCP 47](https://www.rfc-editor.org/info/bcp47) code | No | No | The language the app should use. |
| `password` | | No | No | E2EE password when using a shared secret. (For individual sender keys in embedded mode this is not required.) |
| `perParticipantE2EE` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Enables per participant encryption with Keys exchanged over encrypted matrix room messages. |
| `controlledAudioDevices` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the [global JS controls for audio devices](./controls.md#audio-devices) should be enabled, allowing the list of audio devices to be controlled by the app hosting Element Call. |
| `roomId` | [Matrix Room ID](https://spec.matrix.org/v1.12/appendices/#room-ids) | Yes | No | Anything about what room we're pointed to should be from useRoomIdentifier which parses the path and resolves alias with respect to the default server name, however roomId is an exception as we need the room ID in embedded widget mode, and not the room alias (or even the via params because we are not trying to join it). This is also not validated, where it is in `useRoomIdentifier()`. |
| `showControls` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Displays controls like mute, screen-share, invite, and hangup buttons during a call. |
+4 -16
View File
@@ -3,10 +3,6 @@
"user_menu": "User menu"
},
"action": {
"add_background_image": "Add image",
"background_effect_blur": "Blur",
"background_effect_none": "None",
"background_effect_numbered": "Background {{n}}",
"blur_background": "Blur background",
"close": "Close",
"copy_link": "Copy link",
@@ -27,9 +23,6 @@
"upload_file": "Upload file"
},
"analytics_notice": "By participating in this beta, you consent to the collection of anonymous data, which we use to improve the product. You can find more information about which data we track in our <2>Privacy Policy</2> and our <6>Cookie Policy</6>.",
"background_effects": {
"slow_in_this_browser": "This browser runs background effects slowly, so other people may see your video stutter."
},
"call_ended_view": {
"create_account_button": "Create account",
"create_account_prompt": "<0>Why not finish by setting up a password to keep your account?</0><1>You'll be able to keep your name and set an avatar for use on future calls</1>",
@@ -66,10 +59,10 @@
"developer_mode": {
"always_show_iphone_earpiece": "Show iPhone earpiece option on all platforms",
"crypto_version": "Crypto version: {{version}}",
"custom_livekit_url": {
"current_url": "Currently set to: ",
"custom_transport": {
"current": "Currently set to: ",
"from_config": "Currently, no overwrite is set. Url from config is used.",
"label": "Custom Livekit-url",
"label": "Custom transport JSON",
"reset": "Reset overwrite",
"save": "Save",
"saving": "Saving..."
@@ -99,10 +92,6 @@
},
"disconnected_banner": "Connectivity to the server has been lost.",
"error": {
"background_animated": "Animated images cannot be used as a background",
"background_not_an_image": "That file is not an image",
"background_not_kept": "That background could not be saved",
"background_undecodable": "That image could not be opened",
"call_is_not_supported": "Call is not supported",
"call_not_found": "Call not found",
"call_not_found_description": "<0>That link doesn't appear to belong to any existing call. Check that you have the right link, or <2>create a new one</2>.</0>",
@@ -177,7 +166,7 @@
"microphone_level": {
"label": "Microphone level",
"no_device": "No microphone found. Connect one, then try again.",
"permission_denied": "Element Call cannot use your microphone. Allow access in your browser settings.",
"permission_denied": "Microphone access is blocked. Allow it, then try again.",
"value": "{{level}} of {{max}}"
},
"microphone_off": "Microphone off",
@@ -223,7 +212,6 @@
"auto_gain_control_label": "Automatic gain control",
"background_blur_header": "Background",
"background_blur_label": "Blur the background of the video",
"background_effects_header": "Background effects",
"bitrate_label": "Bitrate",
"blur_not_supported_by_browser": "(Background blur is not supported by this device.)",
"camera_header": "Camera quality",
+3 -5
View File
@@ -8,7 +8,7 @@
"dev:embedded": "vite --config vite-embedded.config.js",
"dev:component": "vite --config vite-component-dev.config.ts",
"build": "pnpm build:full",
"build:full": "NODE_OPTIONS=--max-old-space-size=16384 vite build",
"build:full": "node --max-old-space-size=16384 node_modules/vite/bin/vite.js build",
"build:full:production": "pnpm build:full",
"build:full:development": "pnpm build:full --mode development",
"build:embedded": "pnpm build:full --config vite-embedded.config.js",
@@ -83,8 +83,8 @@
"@types/sdp-transform": "^2.4.5",
"@typescript-eslint/utils": "^8.61.0",
"@use-gesture/react": "^10.2.11",
"@vector-im/compound-design-tokens": "^10.0.0",
"@vector-im/compound-web": "^10.0.0",
"@vector-im/compound-design-tokens": "^11.0.0",
"@vector-im/compound-web": "^10.4.0",
"@vitejs/plugin-react": "^6.0.2",
"@vitest/browser-playwright": "^4.1.5",
"@vitest/coverage-v8": "^4.0.18",
@@ -107,7 +107,6 @@
"matrix-widget-api": "^1.18.0",
"node-stdlib-browser": "^1.3.1",
"normalize.css": "^8.0.1",
"observable-hooks": "^4.2.3",
"oxfmt": "^0.66.0",
"oxlint": "^1.70.0",
"oxlint-tsgolint": "^0.25.0",
@@ -130,7 +129,6 @@
"uuid": "^14.0.0",
"vaul": "^1.0.0",
"vite": "^8.0.0",
"vite-plugin-generate-file": "^0.3.0",
"vite-plugin-html": "^3.2.2",
"vite-plugin-node-polyfills": "^0.28.0",
"vite-plugin-node-stdlib-browser": "^0.2.1",
+14 -2
View File
@@ -38,8 +38,20 @@ export default defineConfig({
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: "html",
/**
* Reporter to use. See https://playwright.dev/docs/test-reporters
*
* On CI, `list` prints a line per test as it finishes, which keeps the job
* log moving; `github` annotates failures on the PR; `html` is uploaded as
* an artifact and must never try to open a browser there.
*/
reporter: process.env.CI
? [["list"], ["github"], ["html", { open: "never" }]]
: [["list"], ["html"]],
/* Stop a broken run early on CI instead of waiting out every retry. */
maxFailures: process.env.CI ? 5 : undefined,
/* Under the job's 60 minute limit, so a hung run still ends with a report. */
globalTimeout: process.env.CI ? 50 * 60 * 1000 : undefined,
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
+15 -39
View File
@@ -18,11 +18,9 @@ test.describe("the quick audio menu", () => {
await joinACall(page, "Menu user", "Audio menu");
await openAudioMenu(page);
// The speaker list is what the settings modal used to be the only home of.
await expect(page.getByRole("group", { name: "Speaker" })).toBeVisible();
await expect(page.getByRole("group", { name: "Microphone" })).toBeVisible();
// Named rather than counted: the browser contributes its own fake output
// and a "Default" entry, so a total would be a fact about the browser.
// By name, not count: the browser adds its own fake output and a Default.
for (const n of [1, 2, 3])
await expect(
page
@@ -30,8 +28,6 @@ test.describe("the quick audio menu", () => {
.getByRole("menuitemradio", { name: `Fake Speaker ${n}` }),
).toBeVisible();
// Only one entry of a kind is marked, and the meter reports a number
// rather than a colour.
await expect(
page.getByRole("menuitemradio", { checked: true }),
).toHaveCount(2);
@@ -40,10 +36,7 @@ test.describe("the quick audio menu", () => {
await expect(meter).toHaveAttribute("aria-valuenow", /\d+/);
await expect(meter).toHaveAttribute("aria-valuetext", /\d+ of \d+/);
// Both browsers in the matrix can route audio to a chosen output, so the
// section offers a real choice. The case where a platform cannot — Safari,
// and anything without setSinkId — is covered by a unit check, since no
// browser here can reach it.
// Both browsers here can route audio; the platform that can't is a unit check.
await expect(
page
.getByRole("group", { name: "Speaker" })
@@ -55,7 +48,7 @@ test.describe("the quick audio menu", () => {
test("moves the microphone and the speaker without disturbing the call", async ({
browser,
}) => {
// Two browsers, two joins and a real call between them.
// Two browsers and a real call.
test.slow();
const hostContext = await browser.newContext({ reducedMotion: "reduce" });
const host = await hostContext.newPage();
@@ -73,9 +66,7 @@ test.describe("the quick audio menu", () => {
await openAudioMenu(host);
await selectDevice(host, "Speaker", "Fake Speaker 2");
// The point of the criterion: the switch is not a rejoin. Neither side
// sees the call drop, and the guest still has both tiles — so the host
// never left and came back.
// Not a rejoin: neither side drops, and the guest still has both tiles.
await expect(
host.getByRole("dialog", { name: "Reconnecting…" }),
).not.toBeVisible();
@@ -92,7 +83,7 @@ test.describe("the quick audio menu", () => {
test("keeps the meter moving while muted, and sends nothing", async ({
browser,
}) => {
// Two browsers, two joins and a real call between them.
// Two browsers and a real call.
test.slow();
const hostContext = await browser.newContext({ reducedMotion: "reduce" });
const host = await hostContext.newPage();
@@ -110,15 +101,12 @@ test.describe("the quick audio menu", () => {
await expect(mute).toHaveAttribute("aria-checked", "false");
await openAudioMenu(host);
// The microphone is held open while muted, so the meter still reports the
// hardware. The mute control is what says nothing is being transmitted.
const meter = host.getByRole("meter", { name: "Microphone level" });
await expect(meter).toBeVisible();
// Queried by test id, not by role: the menu is modal, so Radix takes the
// rest of the call out of the accessibility tree while it is open.
// By test id: the modal menu hides the rest of the call from the a11y tree.
await expect(mute).toHaveAttribute("aria-checked", "false");
await expect(mute).toBeVisible();
// And the listener is told so, rather than being left to guess from silence.
// And the guest is shown the mute.
await expect(
guest.getByTestId("videoTile").filter({ hasText: "Muted host" }),
).toBeVisible();
@@ -137,9 +125,8 @@ test.describe("the quick audio menu", () => {
const meter = page.getByRole("meter", { name: "Microphone level" });
await expect(meter).toBeVisible();
// Scrolled so the microphones start at the top of the list and run past its
// bottom: the position that tells a pinned meter from one that merely
// happens to be last.
// Scrolled so the microphones start at the top: only there does a pinned
// meter differ from one that is simply last.
const list = page.locator("[role='menu'] div[role='none']").first();
await list.evaluate((element) => {
const group = element.querySelector("[role='group'][aria-label*='icro']");
@@ -150,7 +137,6 @@ test.describe("the quick audio menu", () => {
await expect(meter).toBeInViewport();
await expectPinnedInside(meter, list);
// Every entry stays reachable, which is what the scroll is for.
await expect(
page.getByRole("menuitemradio", { name: "Fake Microphone 20" }),
).toBeVisible();
@@ -169,16 +155,14 @@ test.describe("the quick audio menu", () => {
await openAudioMenu(page);
const first = page.getByRole("menuitemradio").first();
// Opened by pointer, so no ring, even though Radix has moved focus into the
// menu already.
// Opened by pointer: no ring, though Radix has moved focus into the menu.
await expect.poll(async () => outlineWidth(first)).toBe(0);
await page.keyboard.press("ArrowDown");
const focused = page.locator("[role='menuitemradio']:focus");
await expect.poll(async () => outlineWidth(focused)).toBeGreaterThan(0);
// The pointer takes it away again: the menu focuses whatever it is over, so
// a ring that followed focus alone would trail the mouse.
// And the pointer takes it away again.
await first.hover();
await expect.poll(async () => outlineWidth(focused)).toBe(0);
});
@@ -193,8 +177,7 @@ async function joinACall(
await page.goto("/");
await SpaHelpers.createCall(page, userName, callName, true);
await expect(page.getByTestId("name_tag")).toContainText(userName);
// The media controls stay disabled until the devices have enumerated, and
// every test here drives them.
// The media controls stay disabled until devices have enumerated.
await expect(page.getByTestId("incall_mute")).toBeEnabled({
timeout: 10_000,
});
@@ -214,19 +197,12 @@ async function selectDevice(
.getByRole("group", { name: section })
.getByRole("menuitemradio", { name });
await item.click();
// Selecting does not close the menu — the component prevents the default so
// the list survives a mis-click — so it is dismissed explicitly.
// Selecting doesn't close the menu, so dismiss it.
await page.keyboard.press("Escape");
await expect(page.getByRole("menu")).not.toBeVisible();
}
/**
* Asserts the meter sits within the scrollport, and inside the menu's frame.
*
* The meter is the one opaque element in the menu, so it is the one thing that
* can paint over the border. Whether it actually does needs a screenshot; this
* pins the geometry that decides it.
*/
/** Asserts the meter sits within the scrollport and inside the menu's frame. */
async function expectPinnedInside(
meter: Locator,
list: Locator,
@@ -243,7 +219,7 @@ async function expectPinnedInside(
expect(meterBox.x + meterBox.width).toBeLessThan(frame.x + frame.width);
}
/** The painted outline width in pixels, however the stylesheet spells it. */
/** Painted outline width, in px. */
async function outlineWidth(item: Locator): Promise<number> {
if ((await item.count()) === 0) return 0;
return item.first().evaluate((element) => {
+17 -47
View File
@@ -11,22 +11,12 @@ import { createUserAndRoom, resizeContainer, startHarness } from "./harness.ts";
import { installFakeDevices } from "../utils/fake-devices.ts";
/**
* The device menu where Element Call is a component in a host's page rather
* than the whole of one.
*
* This is the case the stylesheets cannot describe: the menu is portalled to
* the document, so a container query and a viewport unit both measure the wrong
* thing — the first has no container to resolve against out there, the second
* measures a page Element Call does not own. The menu has to be sized against
* the space the call is actually drawn in.
*
* Driven from the lobby rather than a joined call. The footer builds the same
* menu from the same device behaviours in both, and the container is the same
* size either way, so joining would only add two connections' worth of flake.
* The device menu with Element Call as a component in a host page, where the
* portalled menu must be sized against the call, not the window. Driven from
* the lobby: it builds the same menu, without two connections' worth of flake.
*/
// Signing in, setting up crypto and syncing happen twice before anything is on
// screen, as in component-call.spec.ts
// Sign-in, crypto setup and sync happen twice before anything shows.
test.describe.configure({ timeout: 180_000 });
test("sizes the device list against the call, not the window", async ({
@@ -37,8 +27,7 @@ test("sizes the device list against the call, not the window", async ({
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
// A short call in a much taller page: the difference between measuring the
// call and measuring the window.
// A short call in a much taller page.
const container = pane.getByTestId("call-container");
await resizeContainer(container, { width: 900, height: 400 });
await expect(pane.getByTestId("lobby_joinCall")).toBeVisible({
@@ -50,8 +39,6 @@ test("sizes the device list against the call, not the window", async ({
const windowHeight = page.viewportSize()!.height;
const listHeight = (await list.boundingBox())!.height;
// Sized against the call. Were it sized against the window the list would be
// half as tall again, and the assertion below would not be able to tell.
expect(callHeight).toBeLessThan(windowHeight * 0.75);
expect(listHeight).toBeLessThanOrEqual(callHeight);
expect(listHeight).toBeLessThan(windowHeight * 0.6);
@@ -72,9 +59,7 @@ test("follows the call area when the host resizes it", async ({ page }) => {
const list = await openDeviceList(page, pane);
const whenShort = (await list.boundingBox())!.height;
// The host grows the space Element Call is drawn in while the menu is open —
// a panel opening, a window dragged, a phone turned. A bound taken once on
// opening would still describe the smaller call.
// The host grows the call while the menu is open.
await resizeContainer(container, { width: 900, height: 700 });
await expect
.poll(async () => (await list.boundingBox())!.height)
@@ -88,15 +73,13 @@ test("keeps every device reachable in a small container", async ({ page }) => {
const pane = panes.first();
const container = pane.getByTestId("call-container");
// Narrow as well as short, which is where entries get pushed out of reach.
await resizeContainer(container, { width: 400, height: 360 });
await expect(pane.getByTestId("lobby_joinCall")).toBeVisible({
timeout: 60_000,
});
const list = await openDeviceList(page, pane);
// More devices than the space allows, so the list has to scroll rather than
// put entries somewhere they cannot be got at.
// More devices than fit, so the list must scroll.
expect(await list.evaluate((el) => el.scrollHeight > el.clientHeight)).toBe(
true,
);
@@ -104,14 +87,12 @@ test("keeps every device reachable in a small container", async ({ page }) => {
const last = page.getByRole("menuitemradio", { name: "Fake Microphone 20" });
await last.scrollIntoViewIfNeeded();
await expect(last).toBeInViewport();
// Reachable means usable, not merely painted: D11 accepts that the menu may
// be drawn outside the call area, so this asserts reach rather than
// containment.
// Reachable means clickable: the menu may be drawn outside the call area.
await last.click();
await expect(last).toHaveAttribute("aria-checked", "true");
});
test("tracks the focus modality of its own call, not the page", async ({
test("tracks the focus source of its own call, not the page", async ({
page,
}) => {
await installFakeDevices(page);
@@ -124,42 +105,31 @@ test("tracks the focus modality of its own call, not the page", async ({
timeout: 60_000,
});
await openDeviceList(page, pane);
// The menu owns the modality, because every item it can focus has to answer
// to it — the device rows and the camera menu's blur toggle alike.
const menu = page.getByRole("menu");
// Asserted on the attribute rather than the painted ring, which cannot be
// read here: the menu is portalled outside the call root, and the component
// build scopes the stylesheet to it, so neither the ring nor the rule that
// suppresses the browser's own reaches this menu. The paint is asserted
// standalone instead — in the story and in audio-menu.spec.ts. What is on
// trial here is which call the tracking answers for.
await expect(menu).toHaveAttribute("data-focus-modality", "pointer");
// Asserted on the attribute: the component build's scoped stylesheet doesn't
// reach the portalled menu, so its ring can't be read here.
await expect(menu).toHaveAttribute("data-focus-source", "pointer");
// A key pressed in the other call on this page — or anywhere in the host's
// own page — says nothing about how this menu is being used.
// A key pressed in the other call says nothing about this menu.
await other.evaluate((element) =>
element.dispatchEvent(
new KeyboardEvent("keydown", { key: "ArrowDown", bubbles: true }),
),
);
await expect(menu).toHaveAttribute("data-focus-modality", "pointer");
await expect(menu).toHaveAttribute("data-focus-source", "pointer");
// A key pressed in this menu does.
await page.keyboard.press("ArrowDown");
await expect(menu).toHaveAttribute("data-focus-modality", "keyboard");
await expect(menu).toHaveAttribute("data-focus-source", "keyboard");
});
/**
* Opens the microphone menu of one component and returns its scrolling device
* list, which lives outside the component: the menu is portalled to the page.
*/
/** Opens a component's microphone menu and returns its device list. */
async function openDeviceList(page: Page, pane: Locator): Promise<Locator> {
await pane
.getByRole("button", { name: "Microphone" })
.click({ timeout: 60_000 });
await expect(page.getByRole("menu")).toBeVisible();
const list = page.locator("[role='menu'] div[role='none']").first();
const list = page.getByRole("menu").getByTestId("device-list");
await expect(list).toBeVisible();
return list;
}
+6
View File
@@ -92,6 +92,12 @@ export async function expectWithin(
outer: Locator,
): Promise<void> {
await expect(inner).toBeVisible();
// Wait for animations to settle.
await inner.evaluate(async (element) => {
for (const animation of element.getAnimations({ subtree: true }))
await animation.finished;
});
const innerBox = await inner.boundingBox();
const outerBox = await outer.boundingBox();
if (innerBox === null || outerBox === null)
+2 -2
View File
@@ -7,7 +7,7 @@ Please see LICENSE in the repository root for full details.
import { expect, test } from "@playwright/test";
import { createJTWToken } from "./fixtures/jwt-token";
import { createJWTToken } from "./fixtures/jwt-token";
test("Should show error screen if fails to get JWT token", async ({ page }) => {
await page.goto("/");
@@ -95,7 +95,7 @@ test("Should show error screen if call creation is restricted", async ({
contentType: "application/json",
body: JSON.stringify({
url: "wss://badurltotricktest/livekit/sfu",
jwt: createJTWToken("@fake:user", "!fake:room"),
jwt: createJWTToken("@fake:user", "!fake:room"),
}),
}),
);
+1 -1
View File
@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
export function createJTWToken(sub: string, room: string): string {
export function createJWTToken(sub: string, room: string): string {
return [
{}, // header
{
+1 -1
View File
@@ -132,7 +132,7 @@ export const widgetTest = test.extend<MyFixtures>({
await ewPage1
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.getByRole("button", { name: "New", exact: true })
.click();
await ewPage1.getByRole("menuitem", { name: "Start chat" }).click();
+493
View File
@@ -0,0 +1,493 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Page, test } from "@playwright/test";
import { SpaHelpers } from "./spa-helpers";
import { type RtcMode } from "./widget/test-helpers.ts";
const sfuUrl = "wss://matrix-rtc.m.localhost/livekit/sfu";
const serviceUrl = "https://matrix-rtc.m.localhost/livekit/jwt";
const otherHomeserver = {
base_url: "https://synapse.othersite.m.localhost",
server_name: "synapse.othersite.m.localhost",
};
const otherSfuUrl = "wss://matrix-rtc.othersite.m.localhost/livekit/sfu";
interface Transport {
type: "livekit";
url?: string;
livekit_service_url?: string;
}
/**
* The transport shapes a homeserver may advertise while migrating to the
* latest MSC4195. `legacy` is what the e2e Synapse advertises on its own.
*/
const shapes = {
url: { type: "livekit", url: sfuUrl },
both: { type: "livekit", url: sfuUrl, livekit_service_url: serviceUrl },
legacy: null,
} satisfies Record<string, Transport | null>;
type Shape = keyof typeof shapes;
/** Which endpoint a page asks for its SFU tokens. */
type Flow = "cs-api" | "service-legacy" | "service-default";
const cases: {
hostShape: Shape;
guestShape: Shape;
/** The MatrixRTC mode both sides use; the default is compatibility mode. */
mode?: RtcMode;
hostFlows: Flow[];
guestFlows: Flow[];
}[] = [
{
hostShape: "url",
guestShape: "url",
hostFlows: ["cs-api"],
guestFlows: ["cs-api"],
},
{
// Publishers keep the legacy participant identity that clients predating
// MSC4195 expect, which only the JWT service assigns.
hostShape: "both",
guestShape: "both",
hostFlows: ["service-legacy"],
guestFlows: ["service-legacy"],
},
{
// The MatrixRTC mode has no bearing on the choice.
hostShape: "both",
guestShape: "both",
mode: "2_0",
hostFlows: ["service-legacy"],
guestFlows: ["service-legacy"],
},
{
hostShape: "url",
guestShape: "legacy",
hostFlows: ["cs-api", "service-default"],
guestFlows: ["service-legacy", "cs-api"],
},
];
for (const { hostShape, guestShape, mode, hostFlows, guestFlows } of cases) {
test(`One to one call with ${hostShape} host and ${guestShape} guest transport${mode === "2_0" ? " in Matrix 2.0 mode" : ""}`, async ({
browser,
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host starts the call.
await advertiseTransport(page, shapes[hostShape]);
const hostRequests = recordAuthRequests(page);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true, mode);
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// The guest joins.
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guestPage = await guestContext.newPage();
await advertiseTransport(guestPage, shapes[guestShape]);
const guestRequests = recordAuthRequests(guestPage);
await SpaHelpers.joinCallFromInviteLink(
guestPage,
inviteLink,
"Pevara",
mode,
);
// Wait for the call to connect and render.
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
// The host and guest ran through the expected flows.
expect(flowsOf(hostRequests)).toEqual(new Set(hostFlows));
expect(flowsOf(guestRequests)).toEqual(new Set(guestFlows));
// All requests succeeded and the SFU URL was set on C-S requests.
for (const request of [...hostRequests, ...guestRequests]) {
expect(request.status).toBe(200);
if (request.flow === "cs-api") expect(request.sfuUrl).toBe(sfuUrl);
// Clients predating MSC4195 match a publisher by the legacy identity,
// which only the JWT service's legacy endpoint assigns. The other flows
// hash it.
expect(request.identity).toMatch(
request.flow === "service-legacy" ? legacyIdentity : hashedIdentity,
);
}
});
}
test("Subscriber falls back to the JWT service when the homeserver lacks MSC4195", async ({
browser,
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host starts the call with both transport shapes.
await advertiseTransport(page, shapes.both);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true);
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// The guest joins but their C-S requests fail.
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guestPage = await guestContext.newPage();
await rejectCsApiTokens(guestPage);
const guestRequests = recordAuthRequests(guestPage);
await SpaHelpers.joinCallFromInviteLink(guestPage, inviteLink, "Pevara");
// Wait for the call to connect and render.
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
// The guest ran through all flows.
expect(flowsOf(guestRequests)).toEqual(
new Set<Flow>(["cs-api", "service-legacy", "service-default"]),
);
// The C-S token requests failed.
for (const request of guestRequests) {
expect(request.status).toBe(request.flow === "cs-api" ? 404 : 200);
}
// The C-S token requests preceeded the fallback request.
const flows = guestRequests.map((r) => r.flow);
expect(flows.indexOf("service-default")).toBeGreaterThan(
flows.indexOf("cs-api"),
);
});
test("Publisher shows an error when the homeserver lacks MSC4195", async ({
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host tries to start a call with only the C-S transport shape available but
// their C-S requests fail.
await advertiseTransport(page, shapes.url);
await rejectCsApiTokens(page);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true);
// We land on an error page.
await expect(
page.getByRole("heading", { name: "Something went wrong" }),
).toBeVisible();
await expect(
page.getByText(
"The authorization service for your media server (SFU) is out of date.",
),
).toBeVisible();
});
/**
* Delegation follows the flow that authenticated the publisher: the homeserver
* for a `url`-only transport, the JWT service wherever there is one.
*/
const delegationCases: { shape: Shape; via: Delegation["via"] }[] = [
{ shape: "url", via: "homeserver" },
{ shape: "legacy", via: "service" },
{ shape: "both", via: "service" },
];
for (const { shape, via } of delegationCases) {
test(`Delayed leave event delegated through the ${via} for a ${shape} transport gets sent on disconnect`, async ({
browser,
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host starts the call.
await advertiseTransport(page, shapes[shape]);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true);
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// The guest joins.
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guestPage = await guestContext.newPage();
await advertiseTransport(guestPage, shapes[shape]);
const delayedLeave = recordDelayedLeave(guestPage);
const delegations = recordDelegations(guestPage);
await SpaHelpers.joinCallFromInviteLink(guestPage, inviteLink, "Pevara");
// Wait for the call to connect and render.
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
// The guest handed its delayed leave over, after probing for delegation
// support with bodiless requests, which count as supported unless they
// 404. The homeserver is always probed; the JWT service only when there
// is one.
await expect
.poll(() => delegations.at(-1))
.toEqual({
via,
status: 200,
delayId: delayedLeave.delayId,
...(via === "homeserver" && { sfuUrl }),
});
const probes = delegations.slice(0, -1);
expect(probes.some((probe) => probe.via === via)).toBe(true);
for (const probe of probes) {
expect(probe.status).toBeGreaterThanOrEqual(400);
expect(probe.status).not.toBe(404);
}
// The delay is long enough that a leave within the window below cannot
// come from the delayed event expiring on its own.
expect(delayedLeave.delayMs).toBeGreaterThan(60_000);
// The guest drops without leaving.
await guestPage.reload();
// The host eventually sees the guest go once the delayed event fires.
await expect(page.getByTestId("videoTile")).toHaveCount(1, {
timeout: 30_000,
});
});
}
test("Subscribers get tokens for a remote homeserver's SFU over federation", async ({
browser,
page,
browserName,
}) => {
skipOnFirefox(browserName);
// The host lives on the other homeserver and publishes on its SFU.
await useHomeserver(page, otherHomeserver);
await advertiseTransport(page, { type: "livekit", url: otherSfuUrl });
const hostRequests = recordAuthRequests(page);
await page.goto("/");
await SpaHelpers.createCall(page, "Androl", "HelloCall", true);
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// The guest lives on the default homeserver and joins over federation.
const guestContext = await browser.newContext({ reducedMotion: "reduce" });
const guestPage = await guestContext.newPage();
await advertiseTransport(guestPage, shapes.url);
const guestRequests = recordAuthRequests(guestPage);
await SpaHelpers.joinCallFromInviteLink(
guestPage,
`${inviteLink}&viaServers=${otherHomeserver.server_name}`,
"Pevara",
);
// Wait for the call to connect and render.
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
// Each side got a token for its own SFU and for the other side's SFU, all
// through its own homeserver.
for (const requests of [hostRequests, guestRequests]) {
expect(flowsOf(requests)).toEqual(new Set<Flow>(["cs-api"]));
expect(new Set(requests.map((r) => r.sfuUrl))).toEqual(
new Set([sfuUrl, otherSfuUrl]),
);
for (const request of requests) expect(request.status).toBe(200);
}
});
/** Skips the calling test if we're running on Firefox. */
function skipOnFirefox(browserName: string): void {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
}
/** Points this page's Element Call at another homeserver. */
async function useHomeserver(
page: Page,
homeserver: typeof otherHomeserver,
): Promise<void> {
await page.route("**/config.json", async (route) => {
const response = await route.fetch();
const config = (await response.json()) as Record<string, unknown>;
await route.fulfill({
response,
json: {
...config,
default_server_config: { "m.homeserver": homeserver },
},
});
});
}
/**
* Maps recorded requests to a set of flows. Publisher and subscriber connections
* start concurrently so using arrays for comparison isn't convenient.
*/
function flowsOf(requests: AuthRequest[]): Set<Flow> {
return new Set(requests.map((r) => r.flow));
}
/**
* Replaces the transports the homeserver advertises to this page. The real
* response is fetched first so its CORS headers carry over.
*/
async function advertiseTransport(
page: Page,
transport: Transport | null,
): Promise<void> {
if (transport === null) return;
await page.route("**/org.matrix.msc4143/rtc/transports", async (route) => {
if (route.request().method() !== "GET") return route.continue();
const response = await route.fetch();
await route.fulfill({ response, json: { rtc_transports: [transport] } });
});
}
/**
* Makes the homeserver look like one without MSC4195: its `get_token`
* endpoint answers as Synapse does for an unknown route.
*/
async function rejectCsApiTokens(page: Page): Promise<void> {
await page.route("**/rtc/livekit/get_token", async (route) => {
if (route.request().method() !== "POST") return route.continue();
await route.fulfill({
status: 404,
json: { errcode: "M_UNRECOGNIZED", error: "Unrecognized request" },
headers: { "Access-Control-Allow-Origin": "*" },
});
});
}
/** A recorded delayed leave event. */
interface DelayedLeave {
delayMs?: number;
delayId?: string;
}
/** Records the delayed leave event the page schedules on joining. */
function recordDelayedLeave(page: Page): DelayedLeave {
const delayedLeave: DelayedLeave = {};
page.on("response", (response) => {
const delay = new URL(response.request().url()).searchParams.get(
"org.matrix.msc4140.delay",
);
if (delay === null || response.status() !== 200) return;
delayedLeave.delayMs = Number(delay);
response
.json()
.then((body: { delay_id: string }) => {
delayedLeave.delayId = body.delay_id;
})
.catch(() => {});
});
return delayedLeave;
}
/** A recorded delegation request for a delayed leave event. */
interface Delegation {
/** Who was asked to send the leave event on the guest's behalf. */
via: "homeserver" | "service";
status: number;
sfuUrl?: string;
delayId?: string;
}
/**
* Records requests that delegate a delayed leave event, or probe for support:
* the MSC4195 `delegate_delayed_leave` endpoint of the homeserver, and the JWT
* service's `delegate_delayed_leave` probe and `/sfu/get` with delay fields.
*/
function recordDelegations(page: Page): Delegation[] {
const delegations: Delegation[] = [];
page.on("response", (response) => {
const request = response.request();
if (request.method() !== "POST") return;
const url = request.url();
const body = request.postDataJSON() as {
url?: string;
delay_id?: string;
} | null;
if (url.endsWith("/rtc/livekit/delegate_delayed_leave")) {
delegations.push({
via: "homeserver",
status: response.status(),
...(body && { sfuUrl: body.url, delayId: body.delay_id }),
});
} else if (url.endsWith("/livekit/jwt/delegate_delayed_leave")) {
delegations.push({ via: "service", status: response.status() });
} else if (url.endsWith("/livekit/jwt/sfu/get") && body?.delay_id) {
delegations.push({
via: "service",
status: response.status(),
delayId: body.delay_id,
});
}
});
return delegations;
}
/** A recorded token request. */
interface AuthRequest {
flow: Flow;
status: number;
/** The SFU a request to the MSC4195 `get_token` endpoint asked for. */
sfuUrl?: string;
/** The participant identity the issued token carries. */
identity?: string;
}
/** The legacy `<user_id>:<device_id>` participant identity. */
const legacyIdentity = /^@[^:]+:[^:]+:[A-Z]+$/;
/** A hashed participant identity: unpadded base64 of a SHA-256. */
const hashedIdentity = /^[A-Za-z0-9+/]{43}$/;
/** Reads the subject of a JWT without verifying it. */
function jwtSubject(jwt: string): string {
const [, payload] = jwt.split(".");
return (
JSON.parse(Buffer.from(payload, "base64url").toString()) as {
sub: string;
}
).sub;
}
/** Records token requests, including the identity each token carries. */
function recordAuthRequests(page: Page): AuthRequest[] {
const requests: AuthRequest[] = [];
page.on("response", (response) => {
const request = response.request();
if (request.method() !== "POST") return;
const url = request.url();
let entry: AuthRequest;
if (url.endsWith("/rtc/livekit/get_token")) {
entry = {
flow: "cs-api",
status: response.status(),
sfuUrl: (request.postDataJSON() as { url: string }).url,
};
} else if (url.endsWith("/livekit/jwt/sfu/get")) {
entry = { flow: "service-legacy", status: response.status() };
} else if (url.endsWith("/livekit/jwt/get_token")) {
entry = { flow: "service-default", status: response.status() };
} else {
return;
}
requests.push(entry);
if (entry.status !== 200) return;
response
.json()
.then((body: { jwt: string }) => {
entry.identity = jwtSubject(body.jwt);
})
.catch(() => {});
});
return requests;
}
+7 -20
View File
@@ -8,20 +8,10 @@ Please see LICENSE in the repository root for full details.
import { type Page } from "@playwright/test";
/**
* Gives the browser more fake devices than it ships with.
*
* A headless browser's fake capture offers one microphone and one speaker,
* which is one short of what a device menu is for: with no choice to make, every
* entry renders disabled. These are synthetic entries on top of the real fake
* device, so the menu has a list to show and a selection to move, and the app
* runs its real device pipeline against them.
*
* What they do not do is route audio: every entry is backed by the same capture,
* and `setSinkId` is accepted rather than honoured. A test can prove that
* choosing a device changes the app's state and does not disturb the call. That
* a listener hears the change needs hardware, and stays a manual check.
*
* Must be called before the page navigates.
* Adds synthetic devices beside the browser's one fake microphone and speaker,
* so the menu has a choice to show. They share one capture and `setSinkId` is
* accepted but not honoured: routing still needs hardware and a manual check.
* Call before the page navigates.
*/
export async function installFakeDevices(
page: Page,
@@ -58,8 +48,7 @@ export async function installFakeDevices(
...synthetic("audiooutput", speakers, "Fake Speaker"),
];
// Our ids name no hardware, so an exact-device constraint on one would be
// rejected. Drop it and let the one real fake device answer.
// Our ids name no hardware, so drop the exact-device constraint.
const getUserMedia = devices.getUserMedia.bind(devices);
devices.getUserMedia = async (
constraints?: MediaStreamConstraints,
@@ -77,10 +66,8 @@ export async function installFakeDevices(
return getUserMedia(constraints);
};
// Routing to a device that does not exist would reject, and the app
// treats that as a failed switch. Both sinks are patched: Element Call
// routes its own AudioContext as well as the media elements, and leaving
// that one alone logs a NotFoundError for every switch.
// Accept routing to our ids on both media elements and the AudioContext,
// which the app also routes.
for (const proto of [
HTMLMediaElement.prototype,
AudioContext.prototype,
-4
View File
@@ -37,8 +37,6 @@ modePairs.forEach(([rtcMode1, rtcMode2]) => {
await TestHelpers.acceptRoomInvite(roomName, timo.page);
await florian.page.pause();
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
florian.page,
rtcMode1,
@@ -74,8 +72,6 @@ modePairs.forEach(([rtcMode1, rtcMode2]) => {
await TestHelpers.expectVisibleVideoCount(frame, 2);
}
// await florian.page.pause();
},
);
});
+1 -1
View File
@@ -223,7 +223,7 @@ export class TestHelpers {
await page
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.getByRole("button", { name: "New", exact: true })
.click();
await page.getByRole("menuitem", { name: "New Room" }).click({
+108
View File
@@ -0,0 +1,108 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Page, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user.ts";
import { HOST1, TestHelpers } from "./test-helpers.ts";
const sfuUrl = "wss://matrix-rtc.m.localhost/livekit/sfu";
const serviceUrl = "https://matrix-rtc.m.localhost/livekit/jwt";
/** Which endpoint a page asks for its SFU tokens. */
type Flow = "cs-api" | "service-legacy" | "service-default";
widgetTest(
"Call between a url + livekit_service_url transport and a legacy one",
async ({ addUser, browserName }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
test.slow();
const [host, guest] = await Promise.all([
addUser("Androl", HOST1),
addUser("Pevara", HOST1),
]);
// The host's homeserver has started advertising a `url` next to its JWT
// service; the guest's still advertises the JWT service alone.
await advertiseTransport(host.page, {
type: "livekit",
url: sfuUrl,
livekit_service_url: serviceUrl,
});
// Element Web answers the widget from a cache it filled at login, so make
// it fetch the transports again now that they are rerouted
await host.clientHandle.evaluate(async (client) => {
await client.cachedRtcTransports.refresh();
});
const hostFlows = recordFlows(host.page);
const guestFlows = recordFlows(guest.page);
const roomName = "Transport Room";
await TestHelpers.createRoom(roomName, host.page, [guest.mxId]);
await TestHelpers.acceptRoomInvite(roomName, guest.page);
await TestHelpers.startCallInCurrentRoom(host.page, false);
await TestHelpers.joinCallFromLobby(host.page);
await TestHelpers.joinCallInCurrentRoom(guest.page);
for (const user of [host, guest]) {
const frame = user.page
.locator('iframe[title="Element Call"]')
.contentFrame();
await expect(frame.getByTestId("videoTile")).toHaveCount(2, {
timeout: 10000,
});
await expect(frame.getByText("Waiting for media...")).not.toBeVisible({
timeout: 10000,
});
await TestHelpers.expectVisibleVideoCount(frame, 2);
}
// Element Web does not expose the MSC4195 endpoints to widgets yet, so
// neither side reached the CS API. The host published through the JWT
// service's legacy endpoint and subscribed to the guest through its
// default one; the guest published the same way and fell back to the
// default endpoint for the host's transport.
expect(hostFlows).toEqual(new Set(["service-legacy", "service-default"]));
expect(guestFlows).toEqual(new Set(["service-legacy", "service-default"]));
},
);
/**
* Replaces the transports the homeserver advertises to this page, including
* to the widget host, which fetches them on the widget's behalf. The real
* response is fetched first so its CORS headers carry over.
*/
async function advertiseTransport(
page: Page,
transport: { type: "livekit"; url?: string; livekit_service_url?: string },
): Promise<void> {
await page.route("**/org.matrix.msc4143/rtc/transports", async (route) => {
if (route.request().method() !== "GET") return route.continue();
const response = await route.fetch();
await route.fulfill({ response, json: { rtc_transports: [transport] } });
});
}
/** Records which token endpoints the page's widget asks. */
function recordFlows(page: Page): Set<Flow> {
const flows = new Set<Flow>();
page.on("response", (response) => {
const request = response.request();
if (request.method() !== "POST") return;
const url = request.url();
if (url.endsWith("/rtc/livekit/get_token")) flows.add("cs-api");
else if (url.endsWith("/livekit/jwt/sfu/get")) flows.add("service-legacy");
else if (url.endsWith("/livekit/jwt/get_token"))
flows.add("service-default");
});
return flows;
}
+60 -111
View File
@@ -131,11 +131,11 @@ importers:
specifier: ^10.2.11
version: 10.3.1(react@19.2.8)
'@vector-im/compound-design-tokens':
specifier: ^10.0.0
version: 10.2.4(@types/react@19.2.18)(react@19.2.8)
specifier: ^11.0.0
version: 11.2.0(@types/react@19.2.18)(react@19.2.8)
'@vector-im/compound-web':
specifier: ^10.0.0
version: 10.0.1(@fontsource/inconsolata@5.3.0)(@fontsource/inter@5.3.0)(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(@vector-im/compound-design-tokens@10.2.4(@types/react@19.2.18)(react@19.2.8))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
specifier: ^10.4.0
version: 10.4.0(@fontsource/inconsolata@5.3.0)(@fontsource/inter@5.3.0)(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(@vector-im/compound-design-tokens@11.2.0(@types/react@19.2.18)(react@19.2.8))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@vitejs/plugin-react':
specifier: ^6.0.2
version: 6.1.1(@rolldown/plugin-babel@0.2.3(@babel/core@7.29.7(supports-color@7.2.0))(@babel/runtime@8.0.0)(rolldown@1.2.7)(vite@8.2.2(@types/node@24.13.3)(esbuild@0.28.2)(jiti@2.7.0)(sass@1.103.1)(terser@5.46.1)(yaml@2.9.0)))(babel-plugin-react-compiler@1.0.0)(vite@8.2.2(@types/node@24.13.3)(esbuild@0.28.2)(jiti@2.7.0)(sass@1.103.1)(terser@5.46.1)(yaml@2.9.0))
@@ -192,19 +192,16 @@ importers:
version: 1.9.2
matrix-js-sdk:
specifier: github:matrix-org/matrix-js-sdk#develop
version: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/24929be0e741be6a5d0a7226f1c682e245263b8a
version: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/4df2f2bfb5826c25f05d9535a8556acecaf2e570
matrix-widget-api:
specifier: ^1.18.0
version: 1.19.0
version: 1.20.0
node-stdlib-browser:
specifier: ^1.3.1
version: 1.3.1
normalize.css:
specifier: ^8.0.1
version: 8.0.1
observable-hooks:
specifier: ^4.2.3
version: 4.2.4(react-dom@19.2.8(react@19.2.8))(react@19.2.8)(rxjs@7.8.2)
oxfmt:
specifier: ^0.66.0
version: 0.66.0
@@ -225,7 +222,7 @@ importers:
version: 10.6.1(postcss@8.5.26)
postcss-selector-parser:
specifier: ^7.1.1
version: 7.1.3
version: 7.1.6
posthog-js:
specifier: 1.408.2
version: 1.408.2
@@ -271,9 +268,6 @@ importers:
vite:
specifier: ^8.0.0
version: 8.2.2(@types/node@24.13.3)(esbuild@0.28.2)(jiti@2.7.0)(sass@1.103.1)(terser@5.46.1)(yaml@2.9.0)
vite-plugin-generate-file:
specifier: ^0.3.0
version: 0.3.1
vite-plugin-html:
specifier: ^3.2.2
version: 3.2.2(vite@8.2.2(@types/node@24.13.3)(esbuild@0.28.2)(jiti@2.7.0)(sass@1.103.1)(terser@5.46.1)(yaml@2.9.0))
@@ -1197,8 +1191,8 @@ packages:
'@types/dom-mediacapture-transform': ^0.1.9
livekit-client: ^1.12.0 || ^2.1.0
'@matrix-org/matrix-sdk-crypto-wasm@18.5.0':
resolution: {integrity: sha512-E826Hy1rG26LanPjtSsOiVRcVoHfSgPgj2r2Xsb5RPScpaKi9XJADQ0u3dNjRCitZPX4oyLNl1FDd5AfDlmIwQ==}
'@matrix-org/matrix-sdk-crypto-wasm@18.9.0':
resolution: {integrity: sha512-NtLzNDTEfNASBjNCTGZHuk7N0jrQhS6Zx2CM857rCHZYYh/rpbKGQNrJ7azktm4S1P5Jy3WU3nwJTGohGw26qQ==}
engines: {node: '>= 18'}
'@mdx-js/react@3.1.1':
@@ -3229,8 +3223,8 @@ packages:
peerDependencies:
react: '>= 16.8.0'
'@vector-im/compound-design-tokens@10.2.4':
resolution: {integrity: sha512-2aCjhSvJxktPWnvh5s9Xdc0Rd76AfHqbDbN4QjOxo2VIac6zYdLyQbxMUQ5ZeoR9G3vAxILd0H1UvApRfMilpQ==}
'@vector-im/compound-design-tokens@11.2.0':
resolution: {integrity: sha512-XGwO63ZqFVXdkhCVLm3wa/tpx2Lw8pyaQYskObrLZIUkjs3aUmgq8CNhGcjkqRJXYGsLNJE2azov4DaM74h2PQ==}
peerDependencies:
'@adobe/leonardo-contrast-colors': ^1.0.0
'@types/react': '*'
@@ -3246,13 +3240,13 @@ packages:
react:
optional: true
'@vector-im/compound-web@10.0.1':
resolution: {integrity: sha512-CT3aGMalPSf53o3JUbBF7EA10Rfp+auftIuBwOUSna+hfZ/zPxGLTFcuCuOJ7L8hrzijM+V4I8L0PUkfyOczpg==}
'@vector-im/compound-web@10.4.0':
resolution: {integrity: sha512-Tt5hsNK7ulhGdRuy9lzSm5C+T8M/4O4kLDFLkaU+Hi1kVWj6tj/5awW+U+GvtpWRh5llbHN2HRCh79+gQhek+A==}
peerDependencies:
'@fontsource/inconsolata': ^5
'@fontsource/inter': ^5
'@types/react': '*'
'@vector-im/compound-design-tokens': '>=1.6.1 <11.0.0'
'@vector-im/compound-design-tokens': '>=1.6.1 <12.0.0'
react: ^18 || ^19.0.0
peerDependenciesMeta:
'@types/react':
@@ -3662,8 +3656,8 @@ packages:
constants-browserify@1.0.0:
resolution: {integrity: sha512-xFxOwqIzR/e1k1gLiWEophSCMqXcwVHIH7akf7b/vxcUeGunlj3hvZaaqxwHsTgn+IndtkQJgSztIDWeumWJDQ==}
content-type@3.0.0:
resolution: {integrity: sha512-AIi5H6p0xk5uknXcN3/rmhP8jgp69OfSe/JuKiQAFprJ7UGw7mwj7m4XcmDzlrnJDG+cGpphAINGdU3g3g7kDw==}
content-type@3.1.1:
resolution: {integrity: sha512-GW4qUsfFo59d0HbUibDlWv5wPz+vAAcaTWbKIuKCf0JkC7wWkSyf8f13IpXn5JkeMlB2P8iTSSIjwXljorg2vA==}
engines: {node: '>=22'}
convert-source-map@2.0.0:
@@ -4468,10 +4462,6 @@ packages:
js-tokens@4.0.0:
resolution: {integrity: sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==}
js-yaml@4.3.0:
resolution: {integrity: sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==}
hasBin: true
js-yaml@4.3.1:
resolution: {integrity: sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==}
hasBin: true
@@ -4683,13 +4673,13 @@ packages:
matrix-events-sdk@0.0.1:
resolution: {integrity: sha512-1QEOsXO+bhyCroIe2/A5OwaxHvBm7EsSQ46DEDn8RBIfQwN5HWBpFvyWWR4QY0KHPPnnJdI99wgRiAl7Ad5qaA==}
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/24929be0e741be6a5d0a7226f1c682e245263b8a:
resolution: {gitHosted: true, integrity: sha512-0EGwbXzvG88sBOxq4ZlCRwT4VpQj2xkizfyy0A3bTSK0ysb6/gXJ/o1WPFoqCjRGMDgPq4gAsk92xGtHA7EhPg==, tarball: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/24929be0e741be6a5d0a7226f1c682e245263b8a}
version: 42.2.0
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/4df2f2bfb5826c25f05d9535a8556acecaf2e570:
resolution: {gitHosted: true, integrity: sha512-b1o6ywaPY9jakj+3GIgx3samM9AbqohzPQ5nZKbyOGYB16IHQtNMyjEo/PY0fW/s0uwBVApEHNb/MMtN40c/rQ==, tarball: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/4df2f2bfb5826c25f05d9535a8556acecaf2e570}
version: 43.0.0
engines: {node: '>=22.0.0'}
matrix-widget-api@1.19.0:
resolution: {integrity: sha512-Qe7L6G1mQJphBC47/Bzp5ljPIK1UARgLiuGgS+/KmEs/eaNHkQ3+ArD0/0kvMV+d6kCGAZ9OKRqsnuPjizI+tw==}
matrix-widget-api@1.20.0:
resolution: {integrity: sha512-qwFP37rgns7mPesM5/vvZgknzb+kvyRB5BISv5sCuBxsGMceMr+Kogf7OkjRxBXdMCPBFuf6gIuYRe5S+lpzQw==}
md5.js@1.3.5:
resolution: {integrity: sha512-xitP+WxNPcTTOgnTJcrhM0xvdPepipPSf3I8EIpGKeFLjt3PlJLIDG3u8EX53ZIubkb+5U2+3rELYpEhHhzdkg==}
@@ -4706,14 +4696,6 @@ packages:
resolution: {integrity: sha512-115fLhvZVqWwHPbClyntxEVfVDfl9DLLTuJvq3g2O/Oxi8AiNouAHvDSzHS0viUJc+V5vm3eq91Xwqn9dp4jRA==}
hasBin: true
mime-db@1.52.0:
resolution: {integrity: sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==}
engines: {node: '>= 0.6'}
mime-types@2.1.35:
resolution: {integrity: sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==}
engines: {node: '>= 0.6'}
mimic-function@5.0.1:
resolution: {integrity: sha512-VP79XUPxV2CigYP3jWwAUFSku2aKqBH7uTAapFWCBqutsbmDo96KY5o8uh6U+/YSIn5OxJnXp73beVkpqMIGhA==}
engines: {node: '>=18'}
@@ -4816,13 +4798,6 @@ packages:
resolution: {integrity: sha512-nK28WOo+QIjBkDduTINE4JkF/UJJKyf2EJxvJKfblDpyg0Q+pkOHNTL0Qwy6NP6FhE/EnzV73BxxqcJaXY9anw==}
engines: {node: '>= 0.4'}
observable-hooks@4.2.4:
resolution: {integrity: sha512-FdTQgyw1h5bG/QHCBIqctdBSnv9VARJCEilgpV6L2qlw1yeLqFIwPm4U15dMtl5kDmNN0hSt+Nl6iYbLFwEcQA==}
peerDependencies:
react: '>=16.8.0'
react-dom: '>=16.8.0'
rxjs: '>=6.0.0'
obug@2.1.4:
resolution: {integrity: sha512-4a+OsYv9UktOJKE+l1A4OufDgdRF9PifWj+tJnHURo/P+WOxpG4GzUFL9qCalmWauao6ogiG+QvnCovwPoyAWA==}
engines: {node: '>=12.20.0'}
@@ -4905,8 +4880,8 @@ packages:
resolution: {integrity: sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw==}
engines: {node: '>=10'}
p-retry@8.0.0:
resolution: {integrity: sha512-kFVqH1HxOHp8LupNsOys7bSV09VYTRLxarH/mokO4Rqhk6wGi70E0jh4VzvVGXfEVNggHoHLAMWsQqHyU1Ey9A==}
p-retry@8.0.1:
resolution: {integrity: sha512-NAigyu8hfvJe7MsS18mnc4is0MZtau3QMdBklaJKK23oBQb6occO3UPM3vHmTckW8KhfyjMZaUOqdTFEYliHgg==}
engines: {node: '>=22'}
p-try@2.2.0:
@@ -5185,8 +5160,8 @@ packages:
peerDependencies:
postcss: ^8.4
postcss-selector-parser@7.1.3:
resolution: {integrity: sha512-ajnd7iZnqjJDkyHNfznl/ZVO0lWqvBmQXfKKENx9/p/bEiF/L3eHwdydNUg9RXZx6xfZWOCmXmBa5oeB+YrAPQ==}
postcss-selector-parser@7.1.6:
resolution: {integrity: sha512-7qASPzhKF2l2KLboRZux8CCTRMdGiV08vWmyKzPz22qZ7ZjQBOeY7rNzNoCLSUiftJ7HUq0GERHmxw/t0dCdMw==}
engines: {node: '>=4'}
postcss-value-parser@4.2.0:
@@ -5914,9 +5889,6 @@ packages:
react: ^16.8 || ^17.0 || ^18.0 || ^19.0.0 || ^19.0.0-rc
react-dom: ^16.8 || ^17.0 || ^18.0 || ^19.0.0 || ^19.0.0-rc
vite-plugin-generate-file@0.3.1:
resolution: {integrity: sha512-tiA3gkPM21MS2+RyqsBMT33GSlM9LM1TJjf6vGvV/e/ml3e3vTKfuH3l2N0NpUgcayvj1fXnmlo5YBuahA6bsg==}
vite-plugin-html@3.2.2:
resolution: {integrity: sha512-vb9C9kcdzcIo/Oc3CLZVS03dL5pDlOFuhGlZYDCJ840BhWl/0nGeZWf3Qy7NlOayscY4Cm/QRgULCQkEZige5Q==}
peerDependencies:
@@ -6425,9 +6397,9 @@ snapshots:
'@csstools/postcss-cascade-layers@5.0.2(postcss@8.5.26)':
dependencies:
'@csstools/selector-specificity': 5.0.0(postcss-selector-parser@7.1.3)
'@csstools/selector-specificity': 5.0.0(postcss-selector-parser@7.1.6)
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
'@csstools/postcss-color-function-display-p3-linear@1.0.1(postcss@8.5.26)':
dependencies:
@@ -6533,9 +6505,9 @@ snapshots:
'@csstools/postcss-is-pseudo-class@5.0.3(postcss@8.5.26)':
dependencies:
'@csstools/selector-specificity': 5.0.0(postcss-selector-parser@7.1.3)
'@csstools/selector-specificity': 5.0.0(postcss-selector-parser@7.1.6)
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
'@csstools/postcss-light-dark-function@2.0.11(postcss@8.5.26)':
dependencies:
@@ -6637,7 +6609,7 @@ snapshots:
'@csstools/postcss-scope-pseudo-class@4.0.1(postcss@8.5.26)':
dependencies:
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
'@csstools/postcss-sign-functions@1.1.4(postcss@8.5.26)':
dependencies:
@@ -6681,13 +6653,13 @@ snapshots:
dependencies:
postcss: 8.5.26
'@csstools/selector-resolve-nested@3.1.0(postcss-selector-parser@7.1.3)':
'@csstools/selector-resolve-nested@3.1.0(postcss-selector-parser@7.1.6)':
dependencies:
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
'@csstools/selector-specificity@5.0.0(postcss-selector-parser@7.1.3)':
'@csstools/selector-specificity@5.0.0(postcss-selector-parser@7.1.6)':
dependencies:
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
'@csstools/utilities@2.0.0(postcss@8.5.26)':
dependencies:
@@ -7111,7 +7083,7 @@ snapshots:
'@types/dom-mediacapture-transform': 0.1.11
livekit-client: 2.22.3(@types/dom-mediacapture-record@1.0.22)
'@matrix-org/matrix-sdk-crypto-wasm@18.5.0': {}
'@matrix-org/matrix-sdk-crypto-wasm@18.9.0': {}
'@mdx-js/react@3.1.1(@types/react@19.2.18)(react@19.2.8)':
dependencies:
@@ -8695,12 +8667,12 @@ snapshots:
'@use-gesture/core': 10.3.1
react: 19.2.8
'@vector-im/compound-design-tokens@10.2.4(@types/react@19.2.18)(react@19.2.8)':
'@vector-im/compound-design-tokens@11.2.0(@types/react@19.2.18)(react@19.2.8)':
optionalDependencies:
'@types/react': 19.2.18
react: 19.2.8
'@vector-im/compound-web@10.0.1(@fontsource/inconsolata@5.3.0)(@fontsource/inter@5.3.0)(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(@vector-im/compound-design-tokens@10.2.4(@types/react@19.2.18)(react@19.2.8))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
'@vector-im/compound-web@10.4.0(@fontsource/inconsolata@5.3.0)(@fontsource/inter@5.3.0)(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(@vector-im/compound-design-tokens@11.2.0(@types/react@19.2.18)(react@19.2.8))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@floating-ui/react': 0.27.20(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@floating-ui/react-dom': 2.1.9(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
@@ -8712,7 +8684,7 @@ snapshots:
'@radix-ui/react-progress': 1.1.16(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@radix-ui/react-separator': 1.1.15(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@radix-ui/react-slot': 1.3.3(@types/react@19.2.18)(react@19.2.8)
'@vector-im/compound-design-tokens': 10.2.4(@types/react@19.2.18)(react@19.2.8)
'@vector-im/compound-design-tokens': 11.2.0(@types/react@19.2.18)(react@19.2.8)
classnames: 2.5.1
react: 19.2.8
vaul: 1.1.2(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
@@ -9173,7 +9145,7 @@ snapshots:
constants-browserify@1.0.0: {}
content-type@3.0.0: {}
content-type@3.1.1: {}
convert-source-map@2.0.0: {}
@@ -9244,13 +9216,13 @@ snapshots:
css-blank-pseudo@7.0.1(postcss@8.5.26):
dependencies:
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
css-has-pseudo@7.0.3(postcss@8.5.26):
dependencies:
'@csstools/selector-specificity': 5.0.0(postcss-selector-parser@7.1.3)
'@csstools/selector-specificity': 5.0.0(postcss-selector-parser@7.1.6)
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-value-parser: 4.2.0
css-prefers-color-scheme@10.0.0(postcss@8.5.26):
@@ -10044,10 +10016,6 @@ snapshots:
js-tokens@4.0.0: {}
js-yaml@4.3.0:
dependencies:
argparse: 2.0.1
js-yaml@4.3.1:
dependencies:
argparse: 2.0.1
@@ -10259,21 +10227,21 @@ snapshots:
matrix-events-sdk@0.0.1: {}
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/24929be0e741be6a5d0a7226f1c682e245263b8a:
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/4df2f2bfb5826c25f05d9535a8556acecaf2e570:
dependencies:
'@babel/runtime': 8.0.0
'@matrix-org/matrix-sdk-crypto-wasm': 18.5.0
'@matrix-org/matrix-sdk-crypto-wasm': 18.9.0
another-json: 0.2.0
bs58: 6.0.0
content-type: 3.0.0
content-type: 3.1.1
loglevel: 1.9.2
matrix-events-sdk: 0.0.1
matrix-widget-api: 1.19.0
p-retry: 8.0.0
matrix-widget-api: 1.20.0
p-retry: 8.0.1
sdp-transform: 3.0.0
unhomoglyph: 1.0.6
matrix-widget-api@1.19.0:
matrix-widget-api@1.20.0:
dependencies:
'@types/events': 3.0.3
events: 3.3.0
@@ -10296,12 +10264,6 @@ snapshots:
bn.js: 4.12.3
brorand: 1.1.0
mime-db@1.52.0: {}
mime-types@2.1.35:
dependencies:
mime-db: 1.52.0
mimic-function@5.0.1: {}
min-indent@1.0.1: {}
@@ -10410,12 +10372,6 @@ snapshots:
has-symbols: 1.1.0
object-keys: 1.1.1
observable-hooks@4.2.4(react-dom@19.2.8(react@19.2.8))(react@19.2.8)(rxjs@7.8.2):
dependencies:
react: 19.2.8
react-dom: 19.2.8(react@19.2.8)
rxjs: 7.8.2
obug@2.1.4: {}
once@1.4.0:
@@ -10600,7 +10556,7 @@ snapshots:
dependencies:
p-limit: 3.1.0
p-retry@8.0.0:
p-retry@8.0.1:
dependencies:
is-network-error: 1.3.2
@@ -10710,7 +10666,7 @@ snapshots:
postcss-attribute-case-insensitive@7.0.1(postcss@8.5.26):
dependencies:
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-clamp@4.1.0(postcss@8.5.26):
dependencies:
@@ -10761,12 +10717,12 @@ snapshots:
'@csstools/css-parser-algorithms': 3.0.5(@csstools/css-tokenizer@3.0.4)
'@csstools/css-tokenizer': 3.0.4
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-dir-pseudo-class@9.0.1(postcss@8.5.26):
dependencies:
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-double-position-gradients@6.0.4(postcss@8.5.26):
dependencies:
@@ -10778,12 +10734,12 @@ snapshots:
postcss-focus-visible@10.0.1(postcss@8.5.26):
dependencies:
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-focus-within@9.0.1(postcss@8.5.26):
dependencies:
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-font-variant@5.0.0(postcss@8.5.26):
dependencies:
@@ -10815,10 +10771,10 @@ snapshots:
postcss-nesting@13.0.2(postcss@8.5.26):
dependencies:
'@csstools/selector-resolve-nested': 3.1.0(postcss-selector-parser@7.1.3)
'@csstools/selector-specificity': 5.0.0(postcss-selector-parser@7.1.3)
'@csstools/selector-resolve-nested': 3.1.0(postcss-selector-parser@7.1.6)
'@csstools/selector-specificity': 5.0.0(postcss-selector-parser@7.1.6)
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-opacity-percentage@3.0.0(postcss@8.5.26):
dependencies:
@@ -10916,7 +10872,7 @@ snapshots:
postcss-pseudo-class-any-link@10.0.1(postcss@8.5.26):
dependencies:
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-replace-overflow-wrap@4.0.0(postcss@8.5.26):
dependencies:
@@ -10925,9 +10881,9 @@ snapshots:
postcss-selector-not@8.0.1(postcss@8.5.26):
dependencies:
postcss: 8.5.26
postcss-selector-parser: 7.1.3
postcss-selector-parser: 7.1.6
postcss-selector-parser@7.1.3:
postcss-selector-parser@7.1.6:
dependencies:
cssesc: 3.0.0
util-deprecate: 1.0.2
@@ -11668,13 +11624,6 @@ snapshots:
- '@types/react'
- '@types/react-dom'
vite-plugin-generate-file@0.3.1:
dependencies:
ejs: 3.1.10
js-yaml: 4.3.0
mime-types: 2.1.35
picocolors: 1.1.1
vite-plugin-html@3.2.2(vite@8.2.2(@types/node@24.13.3)(esbuild@0.28.2)(jiti@2.7.0)(sass@1.103.1)(terser@5.46.1)(yaml@2.9.0)):
dependencies:
'@rollup/pluginutils': 4.2.1
+11 -4
View File
@@ -9,9 +9,11 @@ Please see LICENSE in the repository root for full details.
* Checks that the component build leaves the packages a host must supply to
* the host.
*
* A host application already has React, the Matrix SDK and LiveKit, and a
* second copy of any of them is worse than dead weight: React would hold two
* sets of hooks, and the Matrix client would run two sync loops. So the
* A host application already has React, the Matrix SDK and Compound,
* and a second copy of any of them is worse than dead weight: React would hold
* two sets of hooks, the Matrix client would run two sync loops, and a second
* Compound would style the tooltips it floats into the host's body with class
* names the host's stylesheet does not know. So the
* component build lists them as external — but that list has to name every
* subpath, since the bundler silently ignores the pattern and callback forms
* of the option, and an import it does not cover is bundled with no warning at
@@ -33,13 +35,17 @@ import { loadConfigFromFile } from "vite";
const CONFIG = "vite-component.config.ts";
const SOURCES = ["src", "component"];
// The development harness is a host, not part of the component: it is the
// one that imports what a host supplies (Compound's stylesheets, say).
const EXCLUDED = ["component/dev"];
/** The packages whose duplication would break a host, rather than merely enlarge it. */
const MUST_BE_EXTERNAL = [
"react",
"react-dom",
"matrix-js-sdk",
"livekit-client",
"@vector-im/compound-web",
"@vector-im/compound-design-tokens",
];
const isTestFile = (name) =>
@@ -49,6 +55,7 @@ const isTestFile = (name) =>
async function* sourceFiles(dir) {
for (const entry of await readdir(dir, { withFileTypes: true })) {
const path = join(dir, entry.name);
if (EXCLUDED.includes(path)) continue;
if (entry.isDirectory()) yield* sourceFiles(path);
else if (/\.(ts|tsx)$/.test(entry.name) && !isTestFile(entry.name))
yield path;
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env bash
# Clones the branch of a dependency's repository that has the same name as
# the pull request branch under test, so a change spanning both repositories
# can be tested before either side is merged. Inspired by scripts/fetchdep.sh
# element-web.
#
# Usage: scripts/fetchdep.sh <org> <repo> [ref]
#
# The clone lands in ./<repo>. When the pull request comes from a fork and
# PR_HEAD_REPO_OWNER was set, the fork's own copy of the repository is tried
# before <org>'s. Exits 0 when a branch was cloned and 1 when there is none,
# so the caller can keep the locked version in that case. Without a ref,
# nothing is closed outside a pull request.
set -eu
org=$1
repo=$2
ref=${3:-}
rm -rf "$repo"
# If a ref was passed, try to clone it or die.
if [ -n "$ref" ]; then
echo "fetchdep.sh: Using $org/$repo at $ref"
git clone --depth 1 "https://github.com/$org/$repo.git" "$repo"
git -C "$repo" fetch --depth 1 origin "$ref"
git -C "$repo" -c advice.detachedHead=false checkout FETCH_HEAD
git -C "$repo" log -1
exit 0
fi
# Otherwise ensure that we're on a pull request.
head_ref=${GITHUB_HEAD_REF:-}
if [ -z "$head_ref" ]; then
echo "fetchdep.sh: Not a pull request, nothing to match"
exit 1
fi
# Bots name their branches the same way in every repository, so a match on
# one of theirs would be a coincidence rather than a paired change.
case "$head_ref" in
renovate/* | dependabot/*)
echo "fetchdep.sh: Not matching bot branch $head_ref"
exit 1
;;
esac
# Clones a branch if it exists and ends the script on success.
clone() {
local try_org=$1
local branch=$2
echo "fetchdep.sh: Trying to use $try_org/$repo#$branch"
# Disable auth prompts: https://serverfault.com/a/665959
if GIT_TERMINAL_PROMPT=0 git clone "https://github.com/$try_org/$repo.git" "$repo" --branch "$branch" --depth 1; then
git -C "$repo" log -1
# Surface the match on the workflow run summary.
echo "::notice title=Matched $repo branch::Using $try_org/$repo#$branch at $(git -C "$repo" rev-parse HEAD)"
exit 0
fi
}
# Try cloning from the fork, then from the org.
head_owner=${PR_HEAD_REPO_OWNER:-}
if [ -n "$head_owner" ] && [ "$head_owner" != "${GITHUB_REPOSITORY_OWNER:-}" ]; then
clone "$head_owner" "$head_ref"
fi
clone "$org" "$head_ref"
# Nothing worked. Die.
echo "fetchdep.sh: No branch named $head_ref in $org/$repo"
exit 1
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Installs dependencies, then swaps matrix-js-sdk for a branch of the same
# name as the pull request branch under test if one exists. This is what
# lets CI test an Element Call branch together with a matrix-js-sdk branch
# before either is merged. Inspired by scripts/layered.sh in element-web.
#
# Usage: scripts/layered.sh [pnpm install flags]
#
# The layering is only performed when package.json uses the develop branch of
# matrix-js-sdk and the run is for a pull request whose branch has a matrix-js-sdk
# counterpart. Otherwise the dependency is left as pnpm-lock.yaml pins it.
#
# The matrix-js-sdk checkout is nested inside this directory because some CI
# systems do not allow moving above the primary checkout. It is git-ignored.
#
# Set JS_SDK_GITHUB_BASE_REF to a ref to use that exact ref of matrix-js-sdk
# instead of looking for a matching branch.
#
# This script is only meant to be used in CI. For local development use
# see docs/linking.md.
set -ex
# Paths below are relative to the repository root.
cd "$(dirname "$0")/.."
pnpm install --frozen-lockfile "$@"
js_sdk_dep=$(jq -r '.devDependencies["matrix-js-sdk"]' < package.json)
# Only layer when package.json follows the branch head rather than pinning a
# release, so a release build keeps its pinned version.
if [ "$js_sdk_dep" != "github:matrix-org/matrix-js-sdk#develop" ]; then
echo "layered.sh: Skipping matrix-js-sdk fetch and install as package.json pins $js_sdk_dep"
exit 0
fi
if ! scripts/fetchdep.sh matrix-org matrix-js-sdk "${JS_SDK_GITHUB_BASE_REF:-}"; then
echo "layered.sh: No matching matrix-js-sdk branch, keeping the locked version"
exit 0
fi
# matrix-js-sdk pins its pnpm version in `devEngines` only. Under corepack,
# pnpm does not switch versions on its own and corepack, finding no
# `packageManager` field there, would walk up to this project's and use
# ours. Pinning it in the clone makes every pnpm invocation inside that
# directory, including the `prepare` script run by pack, use the right one.
js_sdk_pnpm="pnpm@$(jq -r '.devEngines.packageManager.version' < matrix-js-sdk/package.json)"
jq --arg pm "$js_sdk_pnpm" '.packageManager = $pm' matrix-js-sdk/package.json > matrix-js-sdk/package.json.tmp
mv matrix-js-sdk/package.json.tmp matrix-js-sdk/package.json
export COREPACK_ENABLE_DOWNLOAD_PROMPT=0
# Install matrix-js-sdk's build dependencies. `prepare` runs on pack below,
# so it is skipped here.
(cd matrix-js-sdk && pnpm install --frozen-lockfile --ignore-scripts)
# Rather than `pnpm link`, build matrix-js-sdk into a tarball and install
# that. This better reflects a clean install: dependencies may be hoisted
# and shared, and there is no tsconfig.json to pick up sources from.
(cd matrix-js-sdk && pnpm pack --out matrix-js-sdk.tgz)
pnpm install "$(pwd)/matrix-js-sdk/matrix-js-sdk.tgz" "$@"
+2 -3
View File
@@ -26,7 +26,7 @@ This file needs to be hosted. Locally (via `npx serve -l 81234 --cors`) or on a
Now you just need to add the widget to element web via:
```
/addwidget http://localhost:3000?widgetId=$matrix_widget_id&perParticipantE2EE=true&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
/addwidget http://localhost:3000?widgetId=$matrix_widget_id&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
```
## Widgets
@@ -42,7 +42,6 @@ by the matrixRTCSdk to start the postmessage widget api (communication between t
```
widgetId = $matrix_widget_id
perParticipantE2EE = true
userId = $matrix_user_id
deviceId = $org.matrix.msc3819.matrix_device_id
baseUrl = $org.matrix.msc4039.matrix_base_url
@@ -53,7 +52,7 @@ baseUrl = $org.matrix.msc4039.matrix_base_url
Full template use as `<widgetUrl>`:
```
http://localhost:3000?widgetId=$matrix_widget_id&perParticipantE2EE=true&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
http://localhost:3000?widgetId=$matrix_widget_id&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
```
the `$` prefixed variables will be replaced by EW on widget instantiation. (e.g. `$matrix_user_id` -> `@user:example.com` (url encoding will also be applied automatically by EW) -> `%40user%3Aexample.com`)
+1 -1
View File
@@ -231,7 +231,7 @@ export async function createMatrixRTCSdk(
complete: () => {
logger.info("Livekit room items subscription completed");
for (const item of callViewModel.livekitRoomItems$.value) {
logger.info("unregistering room item from room", item.url);
logger.info("unregistering room item from room", item.transport);
item.livekitRoom.unregisterTextStreamHandler(TEXT_LK_TOPIC);
}
},
-11
View File
@@ -22,8 +22,6 @@ import {
} from "react-router-dom";
import * as Sentry from "@sentry/react";
import { TooltipProvider } from "@vector-im/compound-web";
import { BackgroundEffectsBench } from "./livekit/BackgroundEffectsBench";
import { logger } from "matrix-js-sdk/lib/logger";
import { type MatrixClient } from "matrix-js-sdk";
import { I18nextProvider } from "react-i18next";
@@ -158,15 +156,6 @@ export const App: FC<Props> = ({ vm, widget }) => {
<SentryRoute path="/" element={<HomePage />} />
<SentryRoute path="/login" element={<LoginPage />} />
<SentryRoute path="/register" element={<RegisterPage />} />
{/* Exploration only: measures whether this device can run
background effects, on a device the app refuses to run them
on. One path segment, not two: the config is fetched
relative to the page, so a nested path looks for it in a
directory that does not exist and the app never starts. */}
<SentryRoute
path="/background-effects-bench"
element={<BackgroundEffectsBench />}
/>
<SentryRoute path="*" element={<RoomPage />} />
</Routes>
</Sentry.ErrorBoundary>
+2 -2
View File
@@ -6,9 +6,9 @@ Please see LICENSE in the repository root for full details.
*/
import { createContext, use, useMemo } from "react";
import { useObservableEagerState } from "observable-hooks";
import { type MediaDevices } from "./state/MediaDevices";
import { useBehavior } from "./useBehavior";
export const MediaDevicesContext = createContext<MediaDevices | undefined>(
undefined,
@@ -35,7 +35,7 @@ export const useEarpieceAudioConfig = (): {
volume: number;
} => {
const devices = useMediaDevices();
const audioOutput = useObservableEagerState(devices.audioOutput.selected$);
const audioOutput = useBehavior(devices.audioOutput.selected$);
const isVirtualEarpiece = audioOutput?.virtualEarpiece ?? false;
return {
// We use only the right speaker (pan = 1) for the earpiece.
+29
View File
@@ -11,6 +11,7 @@ import { type ReactNode, useState } from "react";
import userEvent from "@testing-library/user-event";
import { Modal } from "./Modal";
import { RootElementProvider } from "./RootElementContext";
const originalMatchMedia = window.matchMedia;
afterEach(() => {
@@ -70,3 +71,31 @@ test("the modal renders as a drawer in mobile viewports", () => {
);
expect(queryByRole("dialog")).toMatchSnapshot();
});
test("the modal can be closed by clicking the backdrop, in another window too", async () => {
// A dialog moved into another window, as a host does for a Document
// Picture-in-Picture window: its nodes are not `Node`s of this window
const frame = document.createElement("iframe");
document.body.appendChild(frame);
const otherBody = frame.contentDocument!.body;
function ModalFn(): ReactNode {
const [isOpen, setOpen] = useState(true);
return (
<RootElementProvider value={otherBody}>
<Modal title="My modal" open={isOpen} onDismiss={() => setOpen(false)}>
<p>This is the content.</p>
</Modal>
</RootElementProvider>
);
}
render(<ModalFn />);
const dialog = otherBody.querySelector('[role="dialog"]');
expect(dialog).not.toBeNull();
const backdrop = dialog!.previousElementSibling as HTMLElement;
const user = userEvent.setup({ document: frame.contentDocument! });
await user.click(backdrop);
expect(otherBody.querySelector('[role="dialog"]')).toBeNull();
frame.remove();
});
+6
View File
@@ -160,6 +160,12 @@ export const Modal: FC<Props> = ({
<DialogPortal container={rootElement}>
<DialogOverlay
className={classNames(overlayStyles.bg, overlayStyles.animate)}
// Radix dismisses on a pointer down outside the dialog, but only
// for event targets that are a `Node` of the window it was loaded
// in. A host can move the dialog into another window (a Document
// Picture-in-Picture window), whose nodes fail that check, so the
// backdrop closes the dialog itself.
onClick={() => onDismiss?.()}
/>
<DialogContent
asChild
+7 -16
View File
@@ -16,30 +16,21 @@ import classNames from "classnames";
import { Modal } from "./Modal";
import styles from "./RTCConnectionStats.module.css";
import mediaViewStyles from "../src/tile/MediaView.module.css";
import { type UnstableLivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
interface Props {
audio?: RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats;
video?: RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats;
focusUrl?: string;
transport?: UnstableLivekitTransport;
rtcBackendIdentity?: string;
}
const extractDomain = (url: string): string => {
try {
const parsedUrl = new URL(url);
return parsedUrl.hostname; // Returns "kdk.cpm"
} catch (error) {
console.error("Invalid URL:", error);
return url;
}
};
// This is only used in developer mode for debugging purposes, so we don't need full localization
export const RTCConnectionStats: FC<Props> = ({
audio,
video,
focusUrl,
transport,
rtcBackendIdentity,
...rest
}) => {
const [showModal, setShowModal] = useState(false);
const [modalContents, setModalContents] = useState<
@@ -76,10 +67,10 @@ export const RTCConnectionStats: FC<Props> = ({
<Text as="span" size="xs" title="rtcBackendIdentity">
rtcBackendIdentity:{rtcBackendIdentity}
</Text>
{focusUrl && (
{transport && (
<div>
<Text as="span" size="xs" title="focusURL">
&nbsp;{extractDomain(focusUrl)}
<Text as="span" size="xs" title="transport">
&nbsp;{JSON.stringify(transport)}
</Text>
</div>
)}
+235
View File
@@ -0,0 +1,235 @@
/*
Copyright 2022-2024 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* How Element Call can be told to behave, and what each intent means.
*
* Kept apart from {@link UrlParams} on purpose: everything here is constants,
* types and one function of them, so that a host of the component can import
* these — the enums it needs as values, the defaults an intent implies — without
* loading Element Call itself. UrlParams, which reads them from a URL, needs
* the router, the config and the rest of the app; this module must not.
* `component/api.ts` is built from it as an entry point of its own.
*/
import {
type RTCCallIntent,
type RTCNotificationType,
} from "matrix-js-sdk/lib/matrixrtc";
import { platform } from "./Platform";
export enum UserIntent {
StartNewCall = "start_call",
JoinExistingCall = "join_existing",
StartNewCallVoice = "start_call_voice",
JoinExistingCallVoice = "join_existing_voice",
StartNewCallDM = "start_call_dm",
StartNewCallDMVoice = "start_call_dm_voice",
JoinExistingCallDM = "join_existing_dm",
JoinExistingCallDMVoice = "join_existing_dm_voice",
Unknown = "unknown",
}
export enum HeaderStyle {
None = "none",
Standard = "standard",
AppBar = "app_bar",
}
export enum BackgroundStyle {
Solid = "solid",
Gradient = "gradient",
}
/**
* The configuration for the app, which can be set via URL parameters.
* Those property are different to the UrlProperties, since they are all optional
* and configure the behavior of the app. Their value is the same if EC is used in
* the same context but with different accounts/users.
*
* Their defaults can be controlled by the `intent` property.
*/
export interface UrlConfiguration {
/**
* Whether the app should keep the user confined to the current call/room.
*/
confineToRoom: boolean;
/**
* Whether the app should pause before joining the call until it sees an
* io.element.join widget action, allowing it to be preloaded.
*/
preload: boolean;
/**
* The style of headers to show. "standard" is the default arrangement, "none"
* hides the header entirely, and "app_bar" produces a header with a back
* button like you might see in mobile apps. The callback for the back button
* is window.controls.onBackButtonPressed.
*/
header: HeaderStyle;
/**
* Whether the controls should be shown. For screen recording no controls can be desired.
*/
showControls: boolean;
/**
* Whether to hide the screen-sharing button.
*/
hideScreensharing: boolean;
/**
* Whether the app is allowed to use fallback STUN servers for ICE in case the
* user's homeserver doesn't provide any.
*/
allowIceFallback: boolean;
/**
* Whether the global JS controls for audio output devices should be enabled,
* allowing the list of output devices to be controlled by the app hosting
* Element Call.
*/
controlledAudioDevices: boolean;
/**
* Setting this flag skips the lobby and brings you in the call directly.
* In the widget this can be combined with preload to pass the device settings
* with the join widget action.
*/
skipLobby: boolean;
/**
* Setting this flag makes element call show the lobby after leaving a call.
* This is useful for video rooms.
*/
returnToLobby: boolean;
/**
* Whether and what type of notification EC should send, when the user joins the call.
*/
sendNotificationType?: RTCNotificationType;
/**
* Whether the app should automatically leave the call when there
* is no one left in the call.
* This is one part to make the call matrixRTC session behave like a telephone call.
*/
autoLeaveWhenOthersLeft: boolean;
/**
* If the client should behave like it is awaiting an answer if a notification was sent (wait for call pick up).
* This is a no-op if not combined with sendNotificationType.
*
* This entails:
* - show ui that it is awaiting an answer
* - play a sound that indicates that it is awaiting an answer
* - auto-dismiss the call widget once the notification lifetime expires on the receivers side.
*/
waitForCallPickup: boolean;
/**
* Whether to enable echo cancellation for audio capture.
* Defaults to true.
*/
echoCancellation?: boolean;
/**
* Whether to enable noise suppression for audio capture.
* Defaults to true.
*/
noiseSuppression?: boolean;
callIntent?: RTCCallIntent;
}
// If you need to add a new flag to this interface, prefer a name that describes
// a specific behavior (such as 'confineToRoom'), rather than one that describes
// the situations that call for this behavior ('isEmbedded'). This makes it
// clearer what each flag means, and helps us avoid coupling Element Call's
// behavior to the needs of specific consumers.
/**
* The configuration implied by what the user meant to do — if they pressed a
* Start Call button this would be `start_call`, and if they pressed Join Call,
* `join_existing`.
*
* These are platform-specific defaults, so that a host can start a call by
* saying what the user asked for rather than by setting every parameter itself,
* and so that what each intent means is Element Call's decision, made in one
* place. A host that wants something else states it alongside the intent.
*
* {@link UserIntent.Unknown} means no intent was stated, and gives the
* standalone app's defaults: Element Call owns the whole page, so it offers the
* way out of the room that a hosted call must not.
*/
export function configurationForIntent(intent: UserIntent): UrlConfiguration {
// Only constants and `platform` here, so that this depends on nothing but
// the intent.
let preset: UrlConfiguration = {
confineToRoom: true,
preload: false,
header: platform === "desktop" ? HeaderStyle.None : HeaderStyle.AppBar,
showControls: true,
hideScreensharing: false,
allowIceFallback: true,
controlledAudioDevices: platform === "desktop" ? false : true,
skipLobby: true,
returnToLobby: false,
sendNotificationType: "notification",
autoLeaveWhenOthersLeft: false,
waitForCallPickup: false,
};
switch (intent) {
case UserIntent.StartNewCall:
preset.skipLobby = false;
preset.callIntent = "video";
break;
case UserIntent.JoinExistingCall:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = false;
preset.callIntent = "video";
break;
case UserIntent.StartNewCallVoice:
preset.skipLobby = false;
preset.callIntent = "audio";
break;
case UserIntent.JoinExistingCallVoice:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = false;
preset.callIntent = "audio";
break;
case UserIntent.StartNewCallDMVoice:
preset.callIntent = "audio";
// Fall through
case UserIntent.StartNewCallDM:
preset.skipLobby = true;
preset.sendNotificationType = "ring";
preset.autoLeaveWhenOthersLeft = true;
preset.waitForCallPickup = true;
preset.callIntent = preset.callIntent ?? "video";
break;
case UserIntent.JoinExistingCallDMVoice:
preset.callIntent = "audio";
// Fall through
case UserIntent.JoinExistingCallDM:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = true;
preset.autoLeaveWhenOthersLeft = true;
preset.callIntent = preset.callIntent ?? "video";
break;
// Non widget usecase defaults
default:
preset = {
confineToRoom: false,
preload: false,
header: HeaderStyle.Standard,
showControls: true,
hideScreensharing: false,
allowIceFallback: false,
controlledAudioDevices: false,
skipLobby: false,
returnToLobby: false,
sendNotificationType: undefined,
autoLeaveWhenOthersLeft: false,
waitForCallPickup: false,
};
}
return preset;
}
-5
View File
@@ -230,7 +230,6 @@ describe("UrlParams", () => {
showControls: true,
hideScreensharing: false,
allowIceFallback: false,
perParticipantE2EE: false,
controlledAudioDevices: false,
skipLobby: false,
returnToLobby: false,
@@ -243,7 +242,6 @@ describe("UrlParams", () => {
showControls: true,
hideScreensharing: false,
allowIceFallback: true,
perParticipantE2EE: true,
controlledAudioDevices: platform === "desktop" ? false : true,
skipLobby: true,
returnToLobby: false,
@@ -256,7 +254,6 @@ describe("UrlParams", () => {
showControls: true,
hideScreensharing: false,
allowIceFallback: true,
perParticipantE2EE: true,
controlledAudioDevices: platform === "desktop" ? false : true,
skipLobby: false,
returnToLobby: false,
@@ -499,7 +496,6 @@ describe("UrlParams", () => {
// A host owns navigation, so Element Call must not offer a way out of
// the room
confineToRoom: true,
perParticipantE2EE: true,
// The lobby first, so that the user picks their devices rather than
// being thrown into the call by the act of being rendered
skipLobby: false,
@@ -512,7 +508,6 @@ describe("UrlParams", () => {
expect(configurationForIntent(UserIntent.Unknown)).toMatchObject({
confineToRoom: false,
header: HeaderStyle.Standard,
perParticipantE2EE: false,
});
});
});
+25 -237
View File
@@ -9,47 +9,36 @@ Please see LICENSE in the repository root for full details.
import { createContext, use, useMemo } from "react";
import { useLocation } from "react-router-dom";
import { logger } from "matrix-js-sdk/lib/logger";
import {
type RTCCallIntent,
type RTCNotificationType,
} from "matrix-js-sdk/lib/matrixrtc";
import { pickBy } from "lodash-es";
import { Config } from "./config/Config";
import { type EncryptionSystem } from "./e2ee/sharedKeyManagement";
import { E2eeType } from "./e2ee/e2eeType";
import { platform } from "./Platform";
import {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
} from "./UrlConfiguration";
import { redact } from "./utils/redact";
// Moved to their own module, so that a host can import them without the rest
// of this one; re-exported here as they always were
export {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
} from "./UrlConfiguration";
interface RoomIdentifier {
roomAlias: string | null;
roomId: string | null;
viaServers: string[];
}
export enum UserIntent {
StartNewCall = "start_call",
JoinExistingCall = "join_existing",
StartNewCallVoice = "start_call_voice",
JoinExistingCallVoice = "join_existing_voice",
StartNewCallDM = "start_call_dm",
StartNewCallDMVoice = "start_call_dm_voice",
JoinExistingCallDM = "join_existing_dm",
JoinExistingCallDMVoice = "join_existing_dm_voice",
Unknown = "unknown",
}
export enum HeaderStyle {
None = "none",
Standard = "standard",
AppBar = "app_bar",
}
export enum BackgroundStyle {
Solid = "solid",
Gradient = "gradient",
}
/**
* The UrlProperties are used to pass required data to the widget.
* Those are different in different rooms, users, devices. They do not configure the behavior of the
@@ -166,109 +155,6 @@ export interface UrlProperties {
*/
background: BackgroundStyle;
}
/**
* The configuration for the app, which can be set via URL parameters.
* Those property are different to the UrlProperties, since they are all optional
* and configure the behavior of the app. Their value is the same if EC is used in
* the same context but with different accounts/users.
*
* Their defaults can be controlled by the `intent` property.
*/
export interface UrlConfiguration {
/**
* Whether the app should keep the user confined to the current call/room.
*/
confineToRoom: boolean;
/**
* Whether the app should pause before joining the call until it sees an
* io.element.join widget action, allowing it to be preloaded.
*/
preload: boolean;
/**
* The style of headers to show. "standard" is the default arrangement, "none"
* hides the header entirely, and "app_bar" produces a header with a back
* button like you might see in mobile apps. The callback for the back button
* is window.controls.onBackButtonPressed.
*/
header: HeaderStyle;
/**
* Whether the controls should be shown. For screen recording no controls can be desired.
*/
showControls: boolean;
/**
* Whether to hide the screen-sharing button.
*/
hideScreensharing: boolean;
/**
* Whether the app is allowed to use fallback STUN servers for ICE in case the
* user's homeserver doesn't provide any.
*/
allowIceFallback: boolean;
/**
* Whether the app should use per participant keys for E2EE.
*/
perParticipantE2EE: boolean;
/**
* Whether the global JS controls for audio output devices should be enabled,
* allowing the list of output devices to be controlled by the app hosting
* Element Call.
*/
controlledAudioDevices: boolean;
/**
* Setting this flag skips the lobby and brings you in the call directly.
* In the widget this can be combined with preload to pass the device settings
* with the join widget action.
*/
skipLobby: boolean;
/**
* Setting this flag makes element call show the lobby after leaving a call.
* This is useful for video rooms.
*/
returnToLobby: boolean;
/**
* Whether and what type of notification EC should send, when the user joins the call.
*/
sendNotificationType?: RTCNotificationType;
/**
* Whether the app should automatically leave the call when there
* is no one left in the call.
* This is one part to make the call matrixRTC session behave like a telephone call.
*/
autoLeaveWhenOthersLeft: boolean;
/**
* If the client should behave like it is awaiting an answer if a notification was sent (wait for call pick up).
* This is a no-op if not combined with sendNotificationType.
*
* This entails:
* - show ui that it is awaiting an answer
* - play a sound that indicates that it is awaiting an answer
* - auto-dismiss the call widget once the notification lifetime expires on the receivers side.
*/
waitForCallPickup: boolean;
/**
* Whether to enable echo cancellation for audio capture.
* Defaults to true.
*/
echoCancellation?: boolean;
/**
* Whether to enable noise suppression for audio capture.
* Defaults to true.
*/
noiseSuppression?: boolean;
callIntent?: RTCCallIntent;
}
// If you need to add a new flag to this interface, prefer a name that describes
// a specific behavior (such as 'confineToRoom'), rather than one that describes
// the situations that call for this behavior ('isEmbedded'). This makes it
// clearer what each flag means, and helps us avoid coupling Element Call's
// behavior to the needs of specific consumers.
export interface UrlParams extends UrlProperties, UrlConfiguration {}
class ParamParser {
@@ -354,97 +240,6 @@ export const getUrlParams = (
return params;
};
/**
* The configuration implied by what the user meant to do — if they pressed a
* Start Call button this would be `start_call`, and if they pressed Join Call,
* `join_existing`.
*
* These are platform-specific defaults, so that a host can start a call by
* saying what the user asked for rather than by setting every parameter itself,
* and so that what each intent means is Element Call's decision, made in one
* place. A host that wants something else states it alongside the intent.
*
* {@link UserIntent.Unknown} means no intent was stated, and gives the
* standalone app's defaults: Element Call owns the whole page, so it offers the
* way out of the room that a hosted call must not.
*/
export function configurationForIntent(intent: UserIntent): UrlConfiguration {
// Only constants and `platform` here, so that this depends on nothing but
// the intent.
let preset: UrlConfiguration = {
confineToRoom: true,
preload: false,
header: platform === "desktop" ? HeaderStyle.None : HeaderStyle.AppBar,
showControls: true,
hideScreensharing: false,
allowIceFallback: true,
perParticipantE2EE: true,
controlledAudioDevices: platform === "desktop" ? false : true,
skipLobby: true,
returnToLobby: false,
sendNotificationType: "notification",
autoLeaveWhenOthersLeft: false,
waitForCallPickup: false,
};
switch (intent) {
case UserIntent.StartNewCall:
preset.skipLobby = false;
preset.callIntent = "video";
break;
case UserIntent.JoinExistingCall:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = false;
preset.callIntent = "video";
break;
case UserIntent.StartNewCallVoice:
preset.skipLobby = false;
preset.callIntent = "audio";
break;
case UserIntent.JoinExistingCallVoice:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = false;
preset.callIntent = "audio";
break;
case UserIntent.StartNewCallDMVoice:
preset.callIntent = "audio";
// Fall through
case UserIntent.StartNewCallDM:
preset.skipLobby = true;
preset.sendNotificationType = "ring";
preset.autoLeaveWhenOthersLeft = true;
preset.waitForCallPickup = true;
preset.callIntent = preset.callIntent ?? "video";
break;
case UserIntent.JoinExistingCallDMVoice:
preset.callIntent = "audio";
// Fall through
case UserIntent.JoinExistingCallDM:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = true;
preset.autoLeaveWhenOthersLeft = true;
preset.callIntent = preset.callIntent ?? "video";
break;
// Non widget usecase defaults
default:
preset = {
confineToRoom: false,
preload: false,
header: HeaderStyle.Standard,
showControls: true,
hideScreensharing: false,
allowIceFallback: false,
perParticipantE2EE: false,
controlledAudioDevices: false,
skipLobby: false,
returnToLobby: false,
sendNotificationType: undefined,
autoLeaveWhenOthersLeft: false,
waitForCallPickup: false,
};
}
return preset;
}
/**
* The {@link UrlProperties} for Element Call running as a component inside a
* host application.
@@ -555,7 +350,6 @@ export const computeUrlParams = (search = "", hash = ""): UrlParams => {
showControls: parser.getFlag("showControls"),
hideScreensharing: parser.getFlag("hideScreensharing"),
allowIceFallback: parser.getFlag("allowIceFallback"),
perParticipantE2EE: parser.getFlag("perParticipantE2EE"),
controlledAudioDevices: parser.getFlag("controlledAudioDevices"),
skipLobby: isWidget ? parser.getFlag("skipLobby") : false,
// In SPA mode the user should always exit to the home screen when hanging
@@ -578,7 +372,7 @@ export const computeUrlParams = (search = "", hash = ""): UrlParams => {
"intent:",
intent,
"\nproperties:",
redact(properties, "password"),
redact(properties, "password", "posthogUserId"),
"configuration:",
configuration,
);
@@ -706,20 +500,14 @@ export function generateUrlSearchParams(
const params = new URLSearchParams();
// The password shouldn't need URL encoding here (we generate URL-safe ones) but encode
// it in case it came from another client that generated a non url-safe one
switch (encryptionSystem?.kind) {
case E2eeType.SHARED_KEY: {
const encodedPassword = encodeURIComponent(encryptionSystem.secret);
if (encodedPassword !== encryptionSystem.secret) {
logger.info(
"Encoded call password used non URL-safe chars: buggy client?",
);
}
params.set("password", encodedPassword);
break;
if (encryptionSystem?.kind === E2eeType.SHARED_KEY) {
const encodedPassword = encodeURIComponent(encryptionSystem.secret);
if (encodedPassword !== encryptionSystem.secret) {
logger.info(
"Encoded call password used non URL-safe chars: buggy client?",
);
}
case E2eeType.PER_PARTICIPANT:
params.set("perParticipantE2EE", "true");
break;
params.set("password", encodedPassword);
}
params.set("roomId", roomId);
viaServers?.forEach((s) => params.set("viaServers", s));
+2 -2
View File
@@ -15,7 +15,7 @@ exports[`AppBar > renders 1`] = `
tabindex="0"
>
<div
class="_indicator-icon_147l5_17"
class="_indicator-icon_j097f_17"
style="--cpd-icon-button-size: 100%;"
>
<svg
@@ -58,7 +58,7 @@ exports[`AppBar > renders with title and subtitle 1`] = `
tabindex="0"
>
<div
class="_indicator-icon_147l5_17"
class="_indicator-icon_j097f_17"
style="--cpd-icon-button-size: 100%;"
>
<svg
+15 -19
View File
@@ -5,19 +5,24 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/* The styles Element Call needs wherever it is shown: the design tokens, fonts
and element defaults its own stylesheets build on top of.
/* The styles Element Call needs wherever it is shown: the element defaults and
custom properties its own stylesheets build on top of.
Split out from index.css so that Element Call as a component can have these
without also being given the standalone page's layout, which would style the
host's own document. What remains here still speaks of the
document — normalize.css and the typography below use bare element selectors,
and the custom properties are declared on `:root` — which is right for the
page, and is why the component build rewrites it: there every selector is
confined to Element Call's root element (see component/build/scopeStylesToRoot.ts),
with `html`, `body` and `:root` becoming that element. Nothing needs to be
written differently here for that to work, but nothing here may rely on
reaching the host's document either.
host's own document. Compound's stylesheets — the design tokens, the component
styles and the fonts they use — are not here either: as a component, Element
Call shares the host's copy of Compound (it is a peer dependency, so that what
Compound floats into the host's body, tooltips and menus, is styled by the
host's stylesheet), and index.css loads it for the page.
What remains here still speaks of the document — normalize.css and the
typography below use bare element selectors, and the custom properties are
declared on `:root` — which is right for the page, and is why the component
build rewrites it: there every selector is confined to Element Call's root
element (see component/build/scopeStylesToRoot.ts), with `html`, `body` and
`:root` becoming that element. Nothing needs to be written differently here for
that to work, but nothing here may rely on reaching the host's document either.
Nothing here should depend on where it lands relative to Element Call's
component stylesheets: the bundler decides that, and it decides differently for
@@ -25,16 +30,7 @@ the app and for the component build. */
@layer normalize, compound-legacy, compound;
@import url("@fontsource/inter/400.css");
@import url("@fontsource/inter/500.css");
@import url("@fontsource/inter/600.css");
@import url("@fontsource/inter/700.css");
@import url("@fontsource/inconsolata/400.css");
@import url("@fontsource/inconsolata/700.css");
@import url("normalize.css/normalize.css") layer(normalize);
@import url("@vector-im/compound-design-tokens/assets/web/css/compound-design-tokens.css") layer(compound);
@import url("@vector-im/compound-web/dist/style.css") layer(compound.components);
:root {
--font-scale: 1;
+1 -1
View File
@@ -83,7 +83,7 @@ div.reactionPopupMenuRoot.reactionPopupMenuModal > div > div {
flex: 0 0 auto;
}
.reactionButton {
.reactionPopupMenu .reactionsMenu .reactionButton {
padding: var(--reaction-button-padding);
border-radius: var(--cpd-radius-pill-effect);
font-size: var(--reaction-button-fontsize);
@@ -9,7 +9,7 @@ exports[`Can close reaction dialog 1`] = `
aria-disabled="false"
aria-expanded="true"
aria-haspopup="true"
aria-labelledby="_r_bb_"
aria-labelledby="_r_bh_"
class="_button_1nw83_8 _has-icon_1nw83_60 _icon-only_1nw83_53"
data-kind="primary"
data-size="lg"
+4 -1
View File
@@ -43,7 +43,10 @@ Please see LICENSE in the repository root for full details.
inset-inline: 0;
}
.footer.overlay:has(:focus-visible) {
/* Keep the footer while it is in use: focus inside it, or a menu opened from
it. The menu is portalled out, so the trigger's aria-expanded tracks it. */
.footer.overlay:has(:focus-visible),
.footer.overlay:has([aria-expanded="true"]) {
opacity: 1;
pointer-events: initial;
}
+27 -12
View File
@@ -12,8 +12,8 @@ import { Link } from "@vector-im/compound-web";
import type { Meta, StoryObj } from "@storybook/react-vite";
import { CallFooter, type FooterSnapshot } from "./CallFooter";
import { ProcessorProvider } from "../livekit/TrackProcessorContext";
import inCallViewStyles from "../room/InCallView.module.css";
import styles from "./CallFooter.module.css";
import { useStaticViewModel } from "../state/ViewModel";
import { ReactionsSenderContext } from "../reactions/useReactionsSender";
import { type ReactionOption } from "../reactions";
@@ -83,15 +83,6 @@ const fnArgType = {
const meta = {
component: CallFooterStoryWrapper,
// The footer reads the backgrounds this device keeps from the processor
// provider, the same way the lobby and the call do.
decorators: [
(Story): JSX.Element => (
<ProcessorProvider>
<Story />
</ProcessorProvider>
),
],
argTypes: {
layout: {
control: "radio",
@@ -134,8 +125,6 @@ export const Default: Story = {
toggleScreenSharing: fn(),
toggleBlur: fn(),
videoBlurEnabled: true,
backgroundEffect: "none",
selectBackgroundEffect: fn(),
hangup: fn(),
buttonSize: "lg",
showFooter: true,
@@ -187,6 +176,32 @@ export const WithAudioAndVideoOptions: Story = {
},
};
export const StaysWhileAMenuIsOpen: Story = {
...Default,
args: {
...WithAudioAndVideoOptions.args,
// In a short window the footer overlays the call and hides itself.
asOverlay: true,
showFooter: true,
},
play: async ({ canvasElement }): Promise<void> => {
const footer = canvasElement.querySelector<HTMLElement>(
'[data-testid="footer-container"]',
)!;
await userEvent.click(
within(canvasElement).getByRole("button", { name: "Microphone" }),
);
await expect(document.body.querySelector('[role="menu"]')).not.toBeNull();
// The call hides the footer with this class.
footer.classList.add(styles.hidden);
// Read after the 0.15s fade, not mid-transition.
await new Promise((settled) => setTimeout(settled, 400));
await expect(getComputedStyle(footer).opacity).toBe("1");
},
};
export const AudioBusy: Story = {
...Default,
args: {
+6 -143
View File
@@ -5,17 +5,8 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type FC,
type JSX,
type Ref,
useCallback,
useMemo,
useState,
} from "react";
import { type FC, type JSX, type Ref, useMemo } from "react";
import classNames from "classnames";
import { useTranslation } from "react-i18next";
import { logger } from "matrix-js-sdk/lib/logger";
import LogoMark from "../icons/LogoMark.svg?react";
import LogoType from "../icons/LogoType.svg?react";
@@ -32,25 +23,9 @@ import {
} from "../button";
import styles from "./CallFooter.module.css";
import {
type BackgroundEffectOption,
MediaMuteAndSwitchButton,
type MenuOptions,
} from "./MediaMuteAndSwitchButton";
import {
parseEffect,
serializeEffect,
shippedBackgrounds,
} from "../livekit/backgroundEffects";
import {
maxAddedBackgrounds,
UnusableImage,
} from "../livekit/backgroundImages";
import {
useAddedBackgrounds,
useBackgroundProcessing,
} from "../livekit/TrackProcessorContext";
import { usesFallbackProcessing } from "../livekit/backgroundProcessing";
import { SelfPreview } from "./SelfPreview";
import { type Behavior } from "../state/Behavior";
import { type ViewModel } from "../state/ViewModel";
import { useBehavior } from "../useBehavior";
@@ -82,8 +57,6 @@ export interface FooterActions {
/** Also controls if the videoMute button is disabled */
toggleVideo: (() => void) | undefined;
toggleBlur: (() => void) | undefined;
/** Undefined where background processing is unavailable. */
selectBackgroundEffect: ((id: string) => void) | undefined;
toggleScreenSharing: (() => void) | undefined;
/** Also controls if the settings button is visible */
openSettings: (() => void) | undefined;
@@ -97,8 +70,6 @@ export interface FooterState {
videoEnabled: boolean;
videoBusy: boolean;
videoBlurEnabled: boolean;
/** The chosen background effect, in its stored form. */
backgroundEffect: string;
showFooter: boolean;
/* This is needed for WindowMode = "flat" */
@@ -128,7 +99,7 @@ export interface FooterState {
/** Providing no options `[]` or `undefined` will imply that we dont have a audio fast switcher */
audioOptions: MenuOptions[];
/** Output devices shown as their own section in the audio menu. */
/** Output (speaker) devices. */
audioOutputOptions: MenuOptions[];
/** Providing no options `[]` or `undefined` will imply that we dont have a audio fast switcher */
videoOptions: MenuOptions[];
@@ -152,7 +123,6 @@ export const CallFooter: FC<FooterProps> = ({
children,
vm,
}) => {
const { t } = useTranslation();
const asOverlay = useBehavior(vm.asOverlay$);
const showFooter = useBehavior(vm.showFooter$);
const hideControls = useBehavior(vm.hideControls$);
@@ -181,102 +151,8 @@ export const CallFooter: FC<FooterProps> = ({
const selectedAudioOutput = useBehavior(vm.selectedAudioOutput$);
const selectAudioOutputOption = useBehavior(vm.selectAudioOutputOption$);
const selectVideoButtonOption = useBehavior(vm.selectVideoButtonOption$);
const backgroundEffect = useBehavior(vm.backgroundEffect$);
const selectBackgroundEffect = useBehavior(vm.selectBackgroundEffect$);
const { added, addBackground, removeBackground } = useAddedBackgrounds();
const { settling, cameraTrack } = useBackgroundProcessing();
// Said, not decided. Where only the slow path exists the effect still works,
// it costs frames — and the user is the only one who knows whether they would
// rather show the room they are sitting in. It stays put while the pipeline
// is being built: that wait is shown on the tile that was pressed, where a
// message that came and went would only flash once the assets are cached.
const backgroundEffectNotice = useMemo(
() =>
usesFallbackProcessing()
? t("background_effects.slow_in_this_browser")
: undefined,
[t],
);
const [backgroundEffectError, setBackgroundEffectError] = useState<
string | undefined
>(undefined);
// Spelled out rather than built from the reason, so the extractor can find
// every string it has to translate.
const whyRefused = useCallback(
(e: unknown): string => {
if (e instanceof UnusableImage)
switch (e.reason) {
case "not-an-image":
return t("error.background_not_an_image");
case "animated":
return t("error.background_animated");
case "undecodable":
return t("error.background_undecodable");
}
return t("error.background_not_kept");
},
[t],
);
const onRemoveBackgroundEffect = useCallback(
(id: string): void => {
const effect = parseEffect(id);
if (effect.kind !== "added") return;
removeBackground(effect.id).catch((e) =>
logger.warn("Could not remove that background", e),
);
},
[removeBackground],
);
const onAddBackgroundImage = useCallback(
(file: File): void => {
setBackgroundEffectError(undefined);
// Added, not put on — before joining and during a call alike. It joins
// the backgrounds on offer and is in force once the user chooses it:
// choosing a file and choosing to wear it are two decisions, and one
// rule for both places is one less thing to know.
addBackground(file).catch((e) => {
setBackgroundEffectError(whyRefused(e));
logger.warn(
e instanceof UnusableImage
? `Cannot use that file as a background: ${e.reason}`
: "Could not keep that background",
e,
);
});
},
[addBackground, whyRefused],
);
// The catalogue is named here rather than in the view model: the names are
// for reading, and a view model has no business holding translated text.
const backgroundEffects = useMemo(
(): BackgroundEffectOption[] => [
{ id: "none", kind: "none", label: t("action.background_effect_none") },
{ id: "blur", kind: "blur", label: t("action.background_effect_blur") },
...shippedBackgrounds.map((background, i) => ({
id: serializeEffect({ kind: "shipped", id: background.id }),
kind: "image" as const,
// Numbered rather than named: the images are stand-ins, and naming
// them here would invent names the design has not given them.
label: t("action.background_effect_numbered", { n: i + 1 }),
imageUrl: background.imagePath,
})),
...added.map((background, i) => ({
id: serializeEffect({ kind: "added", id: background.id }),
kind: "image" as const,
removable: true,
label: t("action.background_effect_numbered", {
n: shippedBackgrounds.length + i + 1,
}),
imageUrl: background.url,
})),
],
[t, added],
);
const toggleBlur = useBehavior(vm.toggleBlur$);
const videoBlurEnabled = useBehavior(vm.videoBlurEnabled$);
const buttonSize = useBehavior(vm.buttonSize$);
const showLogo = useBehavior(vm.showLogo$);
@@ -338,21 +214,8 @@ export const CallFooter: FC<FooterProps> = ({
options={videoOptions}
selectedOption={selectedVideo}
onSelect={selectVideoButtonOption}
backgroundEffects={backgroundEffects}
selectedBackgroundEffect={backgroundEffect}
onSelectBackgroundEffect={selectBackgroundEffect}
// Withheld once the device keeps as many as it will, which is what
// renders the add tile unavailable rather than letting it fail.
onAddBackgroundImage={
selectBackgroundEffect && added.length < maxAddedBackgrounds
? onAddBackgroundImage
: undefined
}
onRemoveBackgroundEffect={onRemoveBackgroundEffect}
backgroundEffectError={backgroundEffectError}
backgroundEffectNotice={backgroundEffectNotice}
backgroundEffectSettling={settling}
selfPreview={<SelfPreview track={cameraTrack ?? null} />}
videoBlurToggleClick={toggleBlur}
videoBlurEnabled={videoBlurEnabled}
/>,
);
} else {
+6 -87
View File
@@ -14,10 +14,7 @@ import type { CallViewModel } from "../state/CallViewModel/CallViewModel";
import type { Alignment, Layout } from "../state/layout-types";
import type { SpotlightTileViewModel } from "../state/TileViewModel";
import type { DeviceLabel } from "../state/MediaDevices";
import {
createCallFooterViewModel,
createLobbyFooterViewModel,
} from "./CallFooterViewModel";
import { createCallFooterViewModel } from "./CallFooterViewModel";
import { HeaderStyle } from "../UrlParams";
const platformMock = vi.hoisted(() => vi.fn(() => "desktop"));
@@ -27,12 +24,10 @@ vi.mock("../Platform", () => ({
},
}));
// The SDK's own check needs WebGL and canvas APIs jsdom does not have. The
// tests below drive it directly, because what they are about is the answer the
// app gives on top of it.
const sdkSupportMock = vi.hoisted(() => vi.fn(() => true));
// Prevent supportsBackgroundProcessors from throwing in jsdom – it is not
// exercised by these tests (only used in `videoToggles`, not `videoOptions`).
vi.mock("@livekit/track-processors", () => ({
supportsBackgroundProcessors: (): boolean => sdkSupportMock(),
supportsBackgroundProcessors: (): boolean => false,
}));
const outputSelectionMock = vi.hoisted(() => vi.fn(() => true));
@@ -121,7 +116,6 @@ describe("createCallFooterViewModel", () => {
it("is withheld where the platform cannot route audio to a chosen device", () => {
outputSelectionMock.mockReturnValue(false);
// Undefined is what renders the speaker section disabled.
expect(buildFooterVm().selectAudioOutputOption$.value).toBeUndefined();
});
@@ -150,9 +144,7 @@ describe("createCallFooterViewModel", () => {
selected$: constant(undefined),
select: vi.fn(),
},
// Safari enumerates no output devices whatsoever. Reproduced by the
// condition rather than by the browser, so it is checked on the
// Linux CI runners that have no Safari to check it with.
// As Safari: no outputs listed.
audioOutput: {
available$: constant(new Map<string, DeviceLabel>()),
selected$: constant(undefined),
@@ -163,10 +155,7 @@ describe("createCallFooterViewModel", () => {
{ showControls: true, header: HeaderStyle.Standard },
);
// Empty rather than undefined: undefined means this menu has no notion
// of outputs at all, as the camera menu has none, and hides the section.
// Empty means there are none to list, and the menu still shows the
// section with a default in it, disabled.
// Empty, not undefined: undefined would hide the section.
expect(vm.audioOutputOptions$.value).toEqual([]);
});
});
@@ -233,74 +222,4 @@ describe("createCallFooterViewModel", () => {
]);
});
});
describe("background effects", () => {
// The lobby, deliberately: in a call the whole switcher is already
// withheld on a phone, so a check there passes whatever the verdict says.
// The lobby keeps its switcher, which is where a phone browser was offered
// every effect, given none of them, and told nothing.
function lobbyFor(
platform: string,
): ReturnType<typeof createLobbyFooterViewModel> {
platformMock.mockReturnValue(platform);
return createLobbyFooterViewModel(
testScope(),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
/* openSettings */ undefined,
/* hangup */ undefined,
/* showLogo */ false,
);
}
// A phone is offered them, and that is the point of asking the browser
// rather than the platform: measured, a phone on the fast path held 99% of
// its frame rate while a desktop on the slow one held 73%, so refusing by
// platform refused the better device and allowed the worse.
it("offers them on a phone whose browser can run them", () => {
sdkSupportMock.mockReturnValue(true);
const vm = lobbyFor("ios");
expect(vm.selectBackgroundEffect$.value).toBeDefined();
expect(vm.toggleBlur$.value).toBeDefined();
});
it("offers them where the pipeline will honour them", () => {
sdkSupportMock.mockReturnValue(true);
const vm = lobbyFor("desktop");
expect(vm.selectBackgroundEffect$.value).toBeDefined();
expect(vm.toggleBlur$.value).toBeDefined();
});
it("offers nothing where the browser itself cannot run them", () => {
sdkSupportMock.mockReturnValue(false);
const vm = lobbyFor("desktop");
expect(vm.selectBackgroundEffect$.value).toBeUndefined();
expect(vm.toggleBlur$.value).toBeUndefined();
});
it("availability is the same before and during a call", () => {
for (const supported of [true, false]) {
sdkSupportMock.mockReturnValue(supported);
platformMock.mockReturnValue("desktop");
const inCall = createCallFooterViewModel(
testScope(),
buildMinimalCallViewModel(gridLayout),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
/* reactionIdentifier */ undefined,
{ showControls: true, header: HeaderStyle.Standard },
);
const lobby = lobbyFor("desktop");
const offeredInCall =
inCall.selectBackgroundEffect$.value !== undefined;
const offeredInLobby =
lobby.selectBackgroundEffect$.value !== undefined;
expect(offeredInLobby).toBe(offeredInCall);
expect(offeredInLobby).toBe(supported);
}
});
});
});
+3 -21
View File
@@ -6,16 +6,14 @@ Please see LICENSE in the repository root for full details.
*/
import { combineLatest, map, type Observable, switchMap } from "rxjs";
import { supportsBackgroundProcessors } from "@livekit/track-processors";
import { supportsAudioOutputSelection } from "livekit-client";
import { supportsBackgroundProcessors } from "../livekit/backgroundProcessing";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { type MenuOptions } from "./MediaMuteAndSwitchButton";
import { type MediaDevices } from "../state/MediaDevices";
import {
backgroundBlur as backgroundBlurSettings,
backgroundEffect as backgroundEffectSetting,
debugTileLayout as debugTileLayoutSetting,
} from "../settings/settings";
import { type Behavior, constant } from "../state/Behavior";
@@ -78,8 +76,6 @@ function buildDeviceBehaviors(
| "selectVideoButtonOption$"
| "toggleBlur$"
| "videoBlurEnabled$"
| "backgroundEffect$"
| "selectBackgroundEffect$"
> {
const options$ = (
available$: Behavior<Map<string, MenuOptions["label"]>>,
@@ -108,8 +104,8 @@ function buildDeviceBehaviors(
selectedAudioOutput$: scope.behavior(
mediaDevices.audioOutput.selected$.pipe(map((s) => s?.id)),
),
// Safari and most Firefox builds cannot route audio to a chosen device at
// all. Withholding the callback is what renders the section disabled.
// Withheld where the platform can't route audio to a chosen device, which
// disables the speaker section.
selectAudioOutputOption$: constant(
supportsAudioOutputSelection()
? mediaDevices.audioOutput.select
@@ -132,20 +128,6 @@ function buildDeviceBehaviors(
),
),
videoBlurEnabled$: backgroundBlurSettings.value$,
backgroundEffect$: backgroundEffectSetting.value$,
// Withholding the callback is what renders the section disabled, the same
// way the speaker section is disabled where no output can be chosen.
selectBackgroundEffect$: scope.behavior(
disableSwitcher$.pipe(
map((switcherDisabled) =>
!switcherDisabled && supportsBackgroundProcessors()
? (id: string): void => {
backgroundEffectSetting.setValue(id);
}
: undefined,
),
),
),
};
}
@@ -1,88 +0,0 @@
{/**
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
**/}
{/**
A custom doc page, replacing the default autodocs for this component, so that
the background effects states can be reviewed together rather than hunted for
in the sidebar. Same filename as the component, which is how Storybook picks
it up.
**/}
import {
Meta,
Primary,
Controls,
Stories,
Title,
Subtitle,
Canvas,
} from "@storybook/addon-docs/blocks";
import * as MenuStories from "./MediaMuteAndSwitchButton.stories";
<Meta of={MenuStories} />
<Title> Device menu </Title>
The menu behind the chevron on the microphone and camera buttons: the devices
to choose from, and — on the camera — the background effects.
<Subtitle> Background effects: the four states worth reviewing </Subtitle>
**Offered.** The ordinary case. No effect, blur, the images we ship, the ones
the user added, and a tile for adding another.
<Canvas of={MenuStories.BackgroundEffects} />
**Offered, with what it costs.** Browsers without the fast path — Safari and
Firefox, on every platform — draw each frame through a canvas instead.
Measured across four of them that costs between a sixth and a quarter of the
frame rate, on a phone and on a desktop alike.
It is said rather than enforced. The cost is smoothness; the reason to pay it
is privacy; and only the person in the room knows which they would rather
have. Everything stays choosable.
Drawn quietly on purpose, to the design: the subtle ground, an outline mark and
the text at its ordinary weight, held the same sixteen pixels from the menu's
sides as the grid above it. Not an alert — an alert is for something that just
happened, and this was true before the menu opened and will be true after it
closes.
<Canvas of={MenuStories.BackgroundEffectsSlowInThisBrowser} />
**Being set up.** Only on the first effect of a session. The tile that was
pressed spins where its tick will go, until a frame has actually come out of
the pipeline — about three seconds of building, and on Safari another twelve to
fifteen while the page holds still. The spinner turns by rotating, which a
browser composites, so it keeps moving through a pause that has stopped
everything else: the difference between looking busy and looking crashed.
On the tile rather than in a message, because once the assets are cached the
wait is a fraction of a second, and a message that appears and vanishes that
fast is noise. A spinner that brief just reads as a control responding. The
notice about what the browser costs stays put throughout — it is true before,
during and after.
This story plays the sequence rather than posing it, shortened to a second and
a half so it is watchable. Later effects skip it entirely: switching a built
pipeline costs nothing.
<Canvas of={MenuStories.BackgroundEffectsSettling} />
**Not possible at all.** Where the browser cannot run background processing on
any path, the tiles are shown and disabled — except no effect, which needs no
processing and stays choosable, so there is always a way back to plain video.
<Canvas of={MenuStories.BackgroundEffectsUnavailable} />
<Subtitle> Mobile layouts </Subtitle>
This component is reactive. To check the mobile layout, open the story from the
sidebar: the summaries on this page do not render it.
<Primary />
<Controls of={MenuStories.Default} />
<Stories />
@@ -41,549 +41,54 @@ Please see LICENSE in the repository root for full details.
flex-direction: column;
}
/* Only the device lists scroll; the level meter stays put beneath them.
The bound comes from the measured height of the call area, set by the
component: the menu is portalled outside the root, so neither a container
query nor a viewport unit describes the space it is allowed to fill. */
/* Only the device lists scroll, bounded by the measured call area: the menu is
portalled out of the root, so CSS can't size it against the call. */
.deviceList {
/* Held one border width clear of the frame on each side. The menu draws its
frame as an outline just inside its own edge, and once this list has
enough in it to scroll, Safari paints it — rows and all — above that
outline, so the frame vanished beside every device; measured on the
microphone menu with many devices, 384 pixels of it. Everything the list
holds is inside this, so nothing in it needs to keep clear on its own. */
margin-inline: var(--cpd-border-width-1);
/* The camera menu's width, set by the component, less that clearance so the
menu still comes to it. Without the variable the calc is invalid and the
width falls back to auto, which is what every other menu wants. */
inline-size: calc(
var(--device-list-inline-size) - 2 * var(--cpd-border-width-1)
);
overflow-y: auto;
min-block-size: 0;
max-block-size: var(--device-list-max-height);
/* The heading stands over the head of this list and the level meter over its
foot. Scrolling a row flush to either edge would put it underneath one of
them, which is how a row reached by the keyboard ends up half-readable;
this keeps both their heights clear for anything the browser scrolls to.
Set by the component, which is the only thing that can measure them. */
/* Keeps a row reached by keyboard clear of the sticky heading and meter. */
scroll-padding-block: var(--device-list-scroll-padding-start, 0)
var(--device-list-scroll-padding-end, 0);
}
/* Each section is headed by its own rule, which Compound draws under a menu
heading already. The heading spans the whole menu — the menu has no padding
of its own — so that rule runs edge to edge, stopping only where the frame
is, rather than inset as a separator between the sections would be.
Compound's spacing around it is the design's too, bar one thing: the design
leaves a section's first device further below the rule than Compound does —
measured off the mock, 27px from the rule to the top of the control, against
the 16px that Compound's 8px margin and the row's own 8px padding give. The
difference goes below the rule, so the rule stays tight under its own text
where Compound put it. */
/* Compound's heading rule heads each section; the design adds space below it. */
.menu h3 {
margin-block-end: var(--cpd-space-5x);
}
/* One section is set much further from the one above it than Compound's 8px
heading margin allows: measured off the mock, 41px from the last device's
control to the next heading's text, against the 22px we had. Only between
sections — the first heading keeps Compound's spacing, because the menu's
own padding is already above it. */
/* And more above a heading that follows another section. */
.deviceList [role="group"] + [role="group"] h3 {
margin-block-start: var(--cpd-space-7x);
}
/* Each section's heading stays at the top of the scrollport while any of that
section is still in view, so a long list never leaves you wondering which
kind of device you are looking at. It leaves with its own section, because
sticky only holds while the group it belongs to is in view.
Opaque, and held a border width clear of the sides, for the same reason as
the meter below: this is a positioned element, so it paints above the
outline the menu draws its frame with, and would swallow it. */
/* Sticky within its own section. Opaque and inset a border width so it doesn't
paint over the menu's outline. */
.sectionHeading {
position: sticky;
inset-block-start: 0;
/* Over what scrolls beneath it. Without a layer of its own it is painted in
order with everything else positioned, and the effect tiles — positioned,
to carry their tick and the remove cross — come after it, so the grid
slid over the heading instead of under it. */
z-index: 1;
background: var(--cpd-color-bg-canvas-default);
/* Clear of the frame's top edge; the list it sits in already keeps it clear
of the sides. */
margin-inline: var(--cpd-border-width-1);
margin-block-start: var(--cpd-border-width-1);
}
/* The meter belongs to the microphone section: it stays at the bottom of the
scrollport while that section is in view, and leaves with it when the list
is scrolled up to the speakers. The wrapper is deliberately unpositioned —
a positioned one paints above the menu's outline and swallows the frame
along this whole section. Sticky is resolved against the scroll container,
so it does not need one. */
/* Sticky at the foot of the microphone section; opaque and inset as above. */
.stickyMeter {
position: sticky;
inset-block-end: 0;
/* Opaque, so the list does not show through it as it scrolls past, and held
clear of the frame's bottom edge. The list it sits in already keeps it
clear of the sides. */
background: var(--cpd-color-bg-canvas-default);
margin-inline: var(--cpd-border-width-1);
margin-block-end: var(--cpd-border-width-1);
}
/* Radix focuses whatever the pointer is over, so the browser's own ring marks
the item under the mouse. It cannot distinguish the two modalities here, so
it is suppressed and replaced by one that can.
Every kind of item the menu can focus, not only the device rows: the camera
menu's blur toggle is a checkbox item and a child of the menu rather than of
the list, and left out it kept the browser's own ring and followed the
pointer with it. */
/* Radix focuses whatever the pointer is over, so the browser's ring would
follow the mouse. Replaced by one shown only for keyboard focus. */
.menu [role^="menuitem"]:focus,
.menu [role^="menuitem"]:focus-visible {
outline: none;
}
/* Shown only when the keyboard is what moved the focus. */
.menu[data-focus-modality="keyboard"] [role^="menuitem"]:focus {
.menu[data-focus-source="keyboard"] [role^="menuitem"]:focus {
outline: var(--cpd-border-width-2) solid var(--cpd-color-border-focused);
outline-offset: calc(-1 * var(--cpd-border-width-2));
}
/*
* The Background effects grid.
*
* Three across matches the design and keeps a thumbnail readable at a glance;
* with no effect, blur and two shipped images, the five a user may add fill it
* to a whole three-by-three at the limit, where the add tile goes away, and
* four and the add tile fill it one short of it.
*
* minmax(0, 1fr) rather than 1fr: a grid track will not shrink below its
* content's minimum width by default, and a tile carrying a label is wider
* than a third of the menu, so plain 1fr lays out three full-width columns and
* clips two of them.
*/
.effectGrid {
display: grid;
/* Thirds of a menu whose width is set, so the tiles are the size design
draws them — about eighty pixels — whatever the device names are. Zero
rather than the content as the floor, so a long label wraps inside its
tile instead of pushing the grid wider than the menu. */
grid-template-columns: repeat(3, minmax(0, 1fr));
/* Every tile the same height, set by the tallest — the ones carrying a
label. Forcing a ratio instead clips the label as soon as the menu is
narrow enough that the ratio cannot hold it. */
grid-auto-rows: 1fr;
/* Measured against the design, where the gap is about a sixth of a tile's
width. A step less than this read as tight, and left the remove cross on
the corner with little room of its own. */
gap: var(--cpd-space-3x);
/* The same inset a menu heading and a device row carry, so the tiles line
their left edge up with the "B" of the heading above them and with the
radio buttons above that. Matching the surrounding boxes instead puts the
tiles to the left of both, since a heading insets its own text. */
/* The sides carry the full inset to the menu's frame. Above, the grid is
the first control of its section, so it starts where a device row's
control starts under its own rule: the heading's spacing supplies most of
that, and this adds the same step a device row holds above its control.
Below, the menu's own padding is already the inset, so nothing is set. */
/* Sixteen from the menu's edge, as the notice below it is and as design
draws both: the list this sits in is already held a border width in from
the frame, so that much less here. */
padding: var(--cpd-space-2x)
calc(var(--cpd-space-4x) - var(--cpd-border-width-1)) 0;
}
/*
* A menu item lays itself out as a grid of chevron, icon and label in a row.
* A tile centres its glyph with the label beneath instead, so this has to
* outweigh that rule rather than sit beside it.
*
* The tiles are a little wider than tall, as the design draws them, and hold
* that shape whatever the menu measures.
*/
.effectGrid .effectTile {
position: relative;
display: flex;
flex-direction: column;
align-items: center;
/* Glyph and label centre together as one group. Pinning the label to the
floor instead leaves the glyph riding high in the space above it. */
justify-content: center;
/* Measured against the design: the glyph sits a quarter of the tile wide,
with about a seventh of the tile's height between its ink and the label's.
The label's own leading is part of that distance, so the step here is
smaller than the ink gap it produces. */
gap: var(--cpd-space-1x);
block-size: auto;
min-block-size: 0;
min-inline-size: 0;
/* More room above and below than at the sides: the tile is wider than tall,
and the padding is what carries that proportion now the gap is set by the
design's spacing rather than by filling the box. */
padding: var(--cpd-space-3x) var(--cpd-space-2x);
overflow: hidden;
border: 2px solid var(--cpd-color-border-interactive-secondary);
border-radius: var(--cpd-space-2x);
}
/* The selection is drawn here and announced by the item's aria-checked, so it
never rests on colour alone. Matched to the specificity of the base tile
rule above, which also sets a border-color and would otherwise win.
Colour alone, at the width every tile already carries: thickening it with an
inset shadow drew it inside the padding box, where an image tile's picture
sits, so the picture covered it and an added background wore a thinner frame
than no effect did. */
.effectGrid .effectTileSelected {
border-color: var(--cpd-color-border-accent);
}
.effectGrid .effectTile > * {
/* A menu item spaces its icon and label for a row, with the gap carried as a
right margin on each. Stacked in a tile that margin is dead space on one
side, which shifts the glyph and the label off the tile's centre. */
margin: 0;
min-inline-size: 0;
max-inline-size: 100%;
text-align: center;
overflow-wrap: anywhere;
}
.effectSwatch {
flex: 0 0 auto;
inline-size: 100%;
display: flex;
align-items: center;
justify-content: center;
color: var(--cpd-color-icon-secondary);
}
/* An image tile is the picture: it fills the tile rather than sitting in a box
inside one, which is what the design draws and what makes a background
recognisable at this size. */
.effectThumb {
position: absolute;
inset: 0;
inline-size: 100%;
block-size: 100%;
object-fit: cover;
}
/* Blur has no icon in the design system, and a hatch reads as "softened" in a
way a glyph does not. */
.effectBlurGlyph {
/* Smaller than the glyph beside it, and than the design's own hatch measures.
The design's is an icon whose strokes shorten towards the corners, so it
leaves them empty and reads lighter; a uniform hatch fills its box and
reads heavier at the same size. */
inline-size: 18px;
block-size: 18px;
border-radius: var(--cpd-space-1x);
background-image: repeating-linear-gradient(
135deg,
var(--cpd-color-icon-secondary) 0 1.2px,
transparent 1.2px 3.15px
);
}
/* The tick marking the tile in force, standing where the glyph would.
Positioned, so it paints above the image, which is positioned itself and
would otherwise cover anything left in normal flow. */
.effectCheck {
position: relative;
color: var(--cpd-color-icon-accent-primary);
}
/* On a picture it needs its own ground: a tick alone disappears into whatever
the image happens to be behind it. */
.effectCheckOnImage {
border-radius: 50%;
background: var(--cpd-color-bg-canvas-default);
box-shadow: 0 0 0 var(--cpd-border-width-2) var(--cpd-color-bg-canvas-default);
}
/* A menu item's label is sized for a row; in a tile the design sets it a step
smaller, against the glyph above it. */
.effectLabel {
font: var(--cpd-font-body-sm-medium);
/* A menu item labels itself in the secondary colour, which reads as muted
under a glyph the design draws at full strength. */
color: var(--cpd-color-text-primary);
}
/* Named for assistive technology, not drawn: an image labels itself, and the
design gives the add tile a plus alone. */
.effectLabelUnseen {
position: absolute;
inline-size: 1px;
block-size: 1px;
overflow: hidden;
clip-path: inset(50%);
white-space: nowrap;
}
/*
* Unavailable. Opacity alone is not enough on an image tile: a washed-out
* photograph still reads as a photograph you could pick. Draining the colour
* as well says plainly that there is nothing to choose here.
*/
.effectGrid .effectTile[aria-disabled="true"] {
opacity: 0.55;
}
.effectGrid .effectTile[aria-disabled="true"] .effectThumb {
filter: grayscale(1);
}
/*
* The cross that removes one of the user's own backgrounds.
*
* Shown on hover, and whenever the tile has the keyboard's attention, so the
* two ways of reaching it look the same. It carries its own ground for the
* same reason the tick does: it sits on a picture nobody chose.
*/
.effectRemove {
position: absolute;
/* On the tile's top-right corner, where Zoom and Teams both put it, and
clear of the middle: centred, it sat inside the target for choosing the
tile, so the one destructive action lived where the ordinary one does. */
inset-block-start: 0;
inset-inline-end: 0;
/* It overhangs the corner by about a fifth of itself, as the design draws
it: enough to read as sitting on the tile rather than in it, little
enough that it never reaches the tile beside it. */
translate: 20% -20%;
display: none;
align-items: center;
justify-content: center;
inline-size: 28px;
block-size: 28px;
border-radius: 50%;
background: var(--cpd-color-bg-canvas-default);
/* Red at rest, not only under the pointer. The design says what the button
does before you are on it, and the ground carries the hover instead. */
color: var(--cpd-color-icon-critical-primary);
/* Raised off the tile rather than outlined against it — Compound's own value
for a small raised surface, there being no shadow token. In a dark theme
it falls on a near-black ground and reads as nothing, which is how the
design draws it there too. */
box-shadow: 0 1.2px 2.4px 0 rgb(0 0 0 / 15%);
cursor: pointer;
}
/* Under the pointer the ground turns critical and the shadow gives way to a
ring, so the button reads as pressable and as destructive at once. */
.effectRemove:hover {
background: var(--cpd-color-bg-critical-subtle-hovered);
box-shadow: 0 0 0 var(--cpd-border-width-1)
var(--cpd-color-border-critical-subtle);
}
.effectGrid .effectTileWrap:hover .effectRemove,
.effectGrid .effectTileWrap:focus-within .effectRemove {
display: flex;
}
/* Holds a tile that can be removed, so the cross can sit over it without
being inside the menu item. */
.effectTileWrap {
position: relative;
display: flex;
min-inline-size: 0;
}
.effectGrid .effectTileWrap > .effectTile {
flex: 1;
min-inline-size: 0;
}
/* Sits under the tiles, inside the same margin they keep from the frame, and
with the same distance above it as below: the menu's own bottom padding
supplies the lower half, so only the upper half is set here. Without it the
message sits flush against the tiles with a gap twice its size underneath. */
.effectError {
/* For the same reason as the notice below: a refusal's title is a sentence,
and would otherwise set the menu's width rather than fit inside it. */
contain: inline-size;
/* A step short of the margin it keeps at the sides: the menu is a column
with a gap of its own, which makes up the difference above. */
padding: var(--cpd-space-3x) var(--cpd-space-4x) 0;
}
/* The spinner stands exactly where the tick will, so the tile does not shift
when one becomes the other. */
.effectBusy {
display: inline-flex;
align-items: center;
justify-content: center;
inline-size: 20px;
block-size: 20px;
}
/*
* What the browser costs, said quietly.
*
* Measured off the design: the subtle secondary ground, held the same sixteen
* pixels from the menu's sides as the grid above it, with the mark and the
* text on one line's baseline and the text at its ordinary weight. Not an
* alert — an alert is for something that just happened, and this was true
* before the menu opened and will be true after it closes.
*/
.effectNotice {
/* Takes the width the menu has rather than lending it one. The menu is as
wide as its widest content, and a sentence asks for its whole length on
one line: without this the notice widened the camera menu from 296 to 340
pixels, past the width the grid and the preview are drawn for. */
contain: inline-size;
display: flex;
align-items: flex-start;
gap: var(--cpd-space-3x);
/* Nothing below: the menu's own bottom padding is already the inset, and
setting one here as well made the gap under the notice twice the gap
beside it. The same reason the grid leaves its own bottom alone. */
margin: var(--cpd-space-2x) var(--cpd-space-4x) 0;
padding: var(--cpd-space-3x);
border-radius: var(--cpd-space-2x);
background: var(--cpd-color-bg-subtle-secondary);
color: var(--cpd-color-text-primary);
font: var(--cpd-font-body-md-regular);
}
.effectNotice > svg {
flex-shrink: 0;
color: var(--cpd-color-icon-secondary);
}
/*
* How the user looks, at the top of the camera menu.
*
* Edge to edge and flush with the top, as the design draws it: it takes the
* menu's own top padding, which is why it pulls itself up over it. Held one
* border width clear of the frame and rounded to match what is left of the
* frame's corner, because a video paints on a layer of its own and would
* otherwise draw over the outline the menu is framed with.
*
* Sixteen by nine across the camera menu's fixed width, less its frame, and
* written as a height rather than a ratio: with a ratio and a cap Safari kept
* the ratio by narrowing the box, and the preview stopped short of the menu's
* right edge. Cropped rather than letterboxed where the call is narrow enough
* to narrow the menu — a face cut at the edge reads as a self-view, a face in
* bars reads as a broken one.
*/
.selfPreview {
position: relative;
display: flex;
align-items: center;
justify-content: center;
margin-block-end: var(--cpd-space-2x);
block-size: 166px;
overflow: hidden;
border-start-start-radius: calc(
var(--cpd-space-3x) - var(--cpd-border-width-1)
);
border-start-end-radius: calc(
var(--cpd-space-3x) - var(--cpd-border-width-1)
);
background: var(--cpd-color-bg-subtle-secondary);
color: var(--cpd-color-icon-secondary);
}
/* Above the list: it takes the menu's own top padding, which is why it pulls
itself up over it, and it keeps clear of the frame's sides itself, since it
is not inside the list that otherwise does. */
.selfPreviewPinned {
margin-block-start: calc(
-1 * var(--cpd-space-5x) + var(--cpd-border-width-1)
);
margin-inline: var(--cpd-border-width-1);
}
/* Inside the list, where there was no room to pin it. A scrolling container
cuts off whatever is pulled above its top, so it is the list that moves up
over the menu's padding instead, to bring the preview flush with the top.
That puts the list's corners where the frame's are rounded, so they are
rounded too: square, a stuck heading would show past the curve. */
.deviceListLeadsWithPreview {
margin-block-start: calc(
-1 * var(--cpd-space-5x) + var(--cpd-border-width-1)
);
border-start-start-radius: calc(
var(--cpd-space-3x) - var(--cpd-border-width-1)
);
border-start-end-radius: calc(
var(--cpd-space-3x) - var(--cpd-border-width-1)
);
}
/* Laid over the box rather than in it. In flow, a sixteen-by-nine picture
166 pixels tall asks for 295 of width, and the menu, sized to its widest
content, grew by the pixel that asked for. Out of flow it only fills what
the menu's width already gives it. */
.selfPreview > video,
.selfPreview > img {
position: absolute;
inset: 0;
inline-size: 100%;
block-size: 100%;
object-fit: cover;
}
/* The camera faces the user, so the picture is what a mirror would show —
the same as the pre-join preview, or the two would disagree. */
.selfPreview > .mirrored {
transform: scaleX(-1);
}
/*
* A fade at each edge of the list while there is more that way.
*
* In place of the scrollbar, which the platform fades out after a moment: with
* a trackpad on a Mac there was otherwise nothing to say that the list scrolls,
* and it looked as though the backgrounds simply stopped. Sticky, so each holds
* its edge of what is in view, and taking no room: each gives back its own
* height in a margin. Over the content, so the rows and tiles fade into it
* rather than past it, since they are opaque and a background would not show.
*
* The top one stands below whichever heading holds the top of the list, which
* is where the content is coming out from under.
*/
.scrollEdge {
position: sticky;
z-index: 1;
block-size: var(--cpd-space-6x);
pointer-events: none;
opacity: 0;
}
@media (prefers-reduced-motion: no-preference) {
.scrollEdge {
transition: opacity 150ms ease-out;
}
}
.scrollEdgeShown {
opacity: 1;
}
.scrollEdgeTop {
inset-block-start: var(--device-list-stuck-heading-height, 0);
margin-block-end: calc(-1 * var(--cpd-space-6x));
background: linear-gradient(
to bottom,
var(--cpd-color-bg-canvas-default),
transparent
);
}
.scrollEdgeBottom {
inset-block-end: 0;
margin-block-start: calc(-1 * var(--cpd-space-6x));
background: linear-gradient(
to top,
var(--cpd-color-bg-canvas-default),
transparent
);
}
File diff suppressed because it is too large Load Diff
+168 -63
View File
@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, test, vi } from "vitest";
import { afterEach, describe, expect, test, vi } from "vitest";
import { axe } from "vitest-axe";
import {
act,
@@ -15,37 +15,51 @@ import {
type RenderResult,
} from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { type JSX, useState, type ReactNode } from "react";
import { Profiler, type JSX, useState, type ReactNode } from "react";
import { TooltipProvider } from "@vector-im/compound-web";
import { MediaMuteAndSwitchButton } from "./MediaMuteAndSwitchButton";
import {
MediaMuteAndSwitchButton,
type MenuOptions,
} from "./MediaMuteAndSwitchButton";
import { MediaDevicesContext } from "../MediaDevicesContext";
import { type MediaDevices } from "../state/MediaDevices";
import { restoreAudioCapture, stubAudioCapture } from "../utils/test";
interface RenderOptions {
requestDeviceNames: () => void;
}
function renderComponent(
function withProviders(
component: ReactNode,
{ requestDeviceNames = (): void => {} }: Partial<RenderOptions> = {},
): RenderResult {
return render(
): JSX.Element {
return (
<TooltipProvider>
<MediaDevicesContext
value={{ requestDeviceNames } as unknown as MediaDevices}
>
{component}
</MediaDevicesContext>
</TooltipProvider>,
</TooltipProvider>
);
}
function renderComponent(
component: ReactNode,
options: Partial<RenderOptions> = {},
): RenderResult {
return render(withProviders(component, options));
}
describe("MediaMuteAndSwitchButton", () => {
// Only one test stubs the capture; don't let it leak into the rest.
afterEach(restoreAudioCapture);
test("renders", () => {
const { container } = renderComponent(
<TooltipProvider>
<MediaMuteAndSwitchButton title={"Switcher"} iconsAndLabels={"audio"} />
<MediaMuteAndSwitchButton iconsAndLabels={"audio"} />
</TooltipProvider>,
);
expect(container).toMatchSnapshot();
@@ -57,11 +71,7 @@ describe("MediaMuteAndSwitchButton", () => {
enabled: boolean,
): RenderResult => {
return renderComponent(
<MediaMuteAndSwitchButton
title={"Switcher"}
iconsAndLabels={type}
enabled={enabled}
/>,
<MediaMuteAndSwitchButton iconsAndLabels={type} enabled={enabled} />,
);
};
const renderAudioEndabled = renderLabels("audio", true);
@@ -88,7 +98,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onMute = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title={"Switcher"}
onMuteClick={onMute}
iconsAndLabels="audio"
enabled={true}
@@ -105,7 +114,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onMute = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title={"Switcher"}
onMuteClick={onMute}
iconsAndLabels="audio"
enabled={true}
@@ -126,7 +134,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onMute = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title={"Switcher"}
onMuteClick={onMute}
iconsAndLabels="video"
enabled={true}
@@ -146,11 +153,7 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const requestDeviceNames = vi.fn();
renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled
/>,
<MediaMuteAndSwitchButton iconsAndLabels="audio" enabled />,
{ requestDeviceNames },
);
@@ -164,7 +167,6 @@ describe("MediaMuteAndSwitchButton", () => {
renderComponent(
<>
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled
options={[
@@ -174,7 +176,6 @@ describe("MediaMuteAndSwitchButton", () => {
selectedOption="mic1"
/>
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="video"
enabled
options={[
@@ -200,7 +201,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onSelect = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -224,7 +224,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onSelect = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -253,7 +252,6 @@ describe("MediaMuteAndSwitchButton", () => {
const [selectedOption, setSelectedOption] = useState("mic1");
return (
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -313,7 +311,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onVideoBlurToggle = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="video"
enabled={true}
videoBlurToggleClick={onVideoBlurToggle}
@@ -338,7 +335,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -362,7 +358,6 @@ describe("MediaMuteAndSwitchButton", () => {
const [selectedOption, setSelectedOption] = useState("mic1");
return (
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -389,8 +384,7 @@ describe("MediaMuteAndSwitchButton", () => {
screen.getByRole("menuitemradio", { name: "Microphone 2" }),
);
// In flight: nothing else can be picked, in either section, so a second
// request cannot overtake the first.
// In flight: nothing else can be picked, in either section.
for (const name of ["Microphone 1", "Speakers", "Headset"]) {
expect(screen.getByRole("menuitemradio", { name })).toHaveAttribute(
"aria-disabled",
@@ -403,7 +397,7 @@ describe("MediaMuteAndSwitchButton", () => {
await promise;
});
// Settled: choosable again.
// Settled: selectable again.
expect(
screen.getByRole("menuitemradio", { name: "Microphone 1" }),
).not.toHaveAttribute("aria-disabled", "true");
@@ -412,11 +406,144 @@ describe("MediaMuteAndSwitchButton", () => {
).not.toHaveAttribute("aria-disabled", "true");
});
test("lets go of a device switch that never arrives", async () => {
const user = userEvent.setup();
// onSelect never reports back, as when a device is removed mid-switch.
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
iconsAndLabels="audio"
enabled={true}
options={[
{ label: { type: "name", name: "Microphone 1" }, id: "mic1" },
{ label: { type: "name", name: "Microphone 2" }, id: "mic2" },
]}
selectedOption="mic1"
onSelect={vi.fn()}
outputOptions={[
{ label: { type: "name", name: "Speakers" }, id: "spk1" },
{ label: { type: "name", name: "Headset" }, id: "spk2" },
]}
selectedOutputOption="spk1"
onSelectOutput={vi.fn()}
/>,
);
await user.click(getByRole("button", { name: "Microphone" }));
await user.click(
screen.getByRole("menuitemradio", { name: "Microphone 2" }),
);
await user.keyboard("{Escape}");
await user.click(getByRole("button", { name: "Microphone" }));
for (const name of ["Microphone 1", "Speakers", "Headset"]) {
expect(screen.getByRole("menuitemradio", { name })).not.toHaveAttribute(
"aria-disabled",
"true",
);
}
});
test("lets go of a device switch whose device is unplugged", async () => {
const user = userEvent.setup();
const mics: MenuOptions[] = [
{ label: { type: "name", name: "Microphone 1" }, id: "mic1" },
{ label: { type: "name", name: "Microphone 2" }, id: "mic2" },
];
const menu = (options: MenuOptions[]): JSX.Element => (
<MediaMuteAndSwitchButton
iconsAndLabels="audio"
enabled={true}
options={options}
selectedOption="mic1"
onSelect={vi.fn()}
outputOptions={[
{ label: { type: "name", name: "Speakers" }, id: "spk1" },
{ label: { type: "name", name: "Headset" }, id: "spk2" },
]}
selectedOutputOption="spk1"
onSelectOutput={vi.fn()}
/>
);
const { getByRole, rerender } = renderComponent(menu(mics));
await user.click(getByRole("button", { name: "Microphone" }));
await user.click(
screen.getByRole("menuitemradio", { name: "Microphone 2" }),
);
// The second microphone is unplugged before the switch lands.
rerender(withProviders(menu(mics.slice(0, 1))));
// Selectable again without closing the menu.
expect(
screen.getByRole("menuitemradio", { name: "Headset" }),
).not.toHaveAttribute("aria-disabled", "true");
});
test("moves the level without re-rendering anything", async () => {
// The level is drawn into the DOM, so a moving level commits nothing.
const capture = stubAudioCapture();
const user = userEvent.setup();
let commits = 0;
const { getByRole } = renderComponent(
<Profiler
id="menu"
onRender={(): void => {
commits++;
}}
>
<MediaMuteAndSwitchButton
iconsAndLabels="audio"
enabled={true}
options={[
{ label: { type: "name", name: "Microphone 1" }, id: "mic1" },
{ label: { type: "name", name: "Microphone 2" }, id: "mic2" },
]}
selectedOption="mic1"
onSelect={vi.fn()}
outputOptions={[
{ label: { type: "name", name: "Speakers" }, id: "spk1" },
{ label: { type: "name", name: "Headset" }, id: "spk2" },
]}
selectedOutputOption="spk1"
onSelectOutput={vi.fn()}
/>
</Profiler>,
);
await user.click(getByRole("button", { name: "Microphone" }));
capture.grant();
await vi.waitFor(() => expect(capture.contexts).toHaveLength(1));
const meter = await screen.findByRole("meter");
// The capture's arrival is one render: the idle level swapped for its own.
await act(async () => {});
const settled = commits;
// The meter smooths by elapsed time, so hand-driven frames need a clock.
let elapsed = performance.now();
const clock = vi
.spyOn(performance, "now")
.mockImplementation(() => (elapsed += 16));
// One frame per task, as a browser delivers them: one act() would batch them.
for (let step = 1; step <= 8; step++) {
capture.speak(step / 8);
await act(async () => {
capture.drawFrames(1);
await Promise.resolve();
});
}
clock.mockRestore();
expect(meter.getAttribute("aria-valuenow")).not.toBe("0");
expect(commits - settled).toBe(0);
});
test("camera menu uses the same selection pattern and keeps the blur toggle", async () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="video"
enabled={true}
options={[
@@ -431,10 +558,8 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Camera" }));
// Same selection pattern as the microphone menu.
screen.getByRole("menuitemradio", { name: "Camera 1", checked: true });
screen.getByRole("menuitemradio", { name: "Camera 2", checked: false });
// And background blur is still reachable from here.
expect(
screen.getByRole("menuitemcheckbox", { name: "Blur background" }),
).toBeInTheDocument();
@@ -444,7 +569,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -459,27 +583,25 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Microphone" }));
const list = screen
.getByRole("menuitemradio", { name: "Microphone 1" })
.closest("[data-focus-modality]");
.closest("[data-focus-source]");
// The menu focuses whatever the pointer is over, so focus alone says
// nothing about how someone is navigating.
expect(list).toHaveAttribute("data-focus-modality", "pointer");
// The menu focuses whatever the pointer is over, so focus alone says nothing.
expect(list).toHaveAttribute("data-focus-source", "pointer");
await user.keyboard("{ArrowDown}");
expect(list).toHaveAttribute("data-focus-modality", "keyboard");
expect(list).toHaveAttribute("data-focus-source", "keyboard");
await user.pointer({
target: screen.getByRole("menuitemradio", { name: "Microphone 2" }),
coords: { clientX: 10, clientY: 10 },
});
expect(list).toHaveAttribute("data-focus-modality", "pointer");
expect(list).toHaveAttribute("data-focus-source", "pointer");
});
test("marks the selected device with the accent fill", async () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -497,8 +619,7 @@ describe("MediaMuteAndSwitchButton", () => {
.getByRole("menuitemradio", { name: "Microphone 2" })
.querySelector("input[type=radio]");
expect(selected).toBeChecked();
// A read-only control is painted muted, which loses the accent fill that
// marks the selection and makes the menu differ from settings.
// readOnly would paint the selected radio muted.
expect(selected).not.toHaveAttribute("readonly");
});
@@ -506,7 +627,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -526,8 +646,6 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Microphone" }));
// Includes the menu's own structure: wrappers between the menu and its
// items break the relationship the roles describe.
const menu = document.querySelector('[role="menu"]');
expect(await axe(menu as HTMLElement)).toHaveNoViolations();
});
@@ -536,7 +654,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -570,7 +687,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -607,7 +723,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onSelectOutput = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -635,7 +750,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onSelect = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -648,7 +762,7 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Microphone" }));
// Shown rather than hidden, so the menu keeps its shape, but not choosable.
// Shown, but not selectable.
const only = screen.getByRole("menuitemradio", { name: "Microphone 1" });
expect(only).toHaveAttribute("aria-disabled", "true");
});
@@ -657,7 +771,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -666,8 +779,7 @@ describe("MediaMuteAndSwitchButton", () => {
]}
selectedOption="mic1"
onSelect={vi.fn()}
// Safari enumerates no output devices at all, and offers no way to
// choose one.
// As Safari: no outputs listed.
outputOptions={[]}
selectedOutputOption={undefined}
onSelectOutput={undefined}
@@ -676,8 +788,6 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Microphone" }));
// A heading with nothing under it says the feature is broken. Audio is
// playing somewhere, so the section names that somewhere and disables it.
const speakers = screen
.getAllByRole("group")
.find((group) => group.getAttribute("aria-label") === "Speaker")!;
@@ -685,8 +795,6 @@ describe("MediaMuteAndSwitchButton", () => {
expect(entries).toHaveLength(1);
expect(entries[0]).toHaveAccessibleName("Default");
expect(entries[0]).toHaveAttribute("aria-disabled", "true");
// And marked as the selection: it is where audio is going, so an unchecked
// lone entry would read as nothing being chosen at all.
expect(entries[0]).toHaveAttribute("aria-checked", "true");
expect(
within(entries[0]).getByRole("radio", { hidden: true }),
@@ -697,7 +805,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -711,7 +818,6 @@ describe("MediaMuteAndSwitchButton", () => {
{ label: { type: "name", name: "Headset" }, id: "spk2" },
]}
selectedOutputOption="spk1"
// No callback: nothing can be picked here.
onSelectOutput={undefined}
/>,
);
@@ -724,7 +830,6 @@ describe("MediaMuteAndSwitchButton", () => {
expect(
screen.getByRole("menuitemradio", { name: "Headset" }),
).toHaveAttribute("aria-disabled", "true");
// The microphone section is unaffected.
expect(
screen.getByRole("menuitemradio", { name: "Microphone 2" }),
).not.toHaveAttribute("aria-disabled", "true");
+86 -663
View File
@@ -11,31 +11,20 @@ import {
type CSSProperties,
type FC,
useEffect,
useRef,
type ReactElement,
type ReactNode,
useLayoutEffect,
} from "react";
import {
Alert,
Button,
Menu,
MenuItem,
InlineSpinner,
MenuTitle,
RadioInput,
Tooltip,
ToggleMenuItem,
} from "@vector-im/compound-web";
import {
ChevronUpIcon,
ChevronDownIcon,
SpinnerIcon,
BlockIcon,
PlusIcon,
CheckCircleSolidIcon,
CloseIcon,
InfoIcon,
} from "@vector-im/compound-design-tokens/assets/web/icons";
import classNames from "classnames";
import { useTranslation } from "react-i18next";
@@ -50,32 +39,14 @@ import {
import { useMediaDevices } from "../MediaDevicesContext";
import { useRootElement } from "../RootElementContext";
import { observeElementSize$ } from "../utils/elementSize";
import { MicrophoneLevelMeter } from "./MicrophoneLevelMeter";
import { useMicrophoneLevel } from "./useMicrophoneLevel";
import { LiveMicrophoneLevelMeter } from "./MicrophoneLevelMeter";
export interface MenuOptions {
label: DeviceLabel | AudioOutputDeviceLabel;
id: string;
}
/** One choice in the camera menu's Background effects section. */
export interface BackgroundEffectOption {
id: string;
/** Shown under the tile, and the item's accessible name. */
label: string;
kind: "none" | "blur" | "image";
/** The thumbnail, for kind "image". Blur and no effect draw their own. */
imageUrl?: string;
/** Whether this one is the user's to remove. */
removable?: boolean;
}
export interface MediaMuteAndSwitchButtonProps {
/**
* The accessible name of the menu. Defaults to a translated name for the
* media kind. Never shown: each section carries its own heading.
*/
title?: string;
/** If the Mute button is enabled */
enabled?: boolean;
/** Callback if the mute button is clicked */
@@ -87,71 +58,14 @@ export interface MediaMuteAndSwitchButtonProps {
options?: MenuOptions[];
/** The option that will currently be rendered as the selected option */
selectedOption?: string;
/**
* Output (speaker) devices, shown as their own section above the input
* section. Audio menu only; omitted entirely for video.
*/
/** Output (speaker) devices. Audio menu only. */
outputOptions?: MenuOptions[];
/** The output option currently rendered as selected */
selectedOutputOption?: string;
/**
* Called when an output device is picked. Undefined means no output can be
* chosen here, and the section renders disabled.
*/
/** Picks an output device. Undefined disables the speaker section. */
onSelectOutput?: (id: string) => void;
videoBlurToggleClick?: () => void;
videoBlurEnabled?: boolean;
/**
* Background effects, shown as a grid of tiles under the camera list. Camera
* menu only; omitted entirely for audio. An empty list leaves the section out.
*/
backgroundEffects?: BackgroundEffectOption[];
/** The effect currently in force. */
selectedBackgroundEffect?: string;
/**
* Called when an effect is chosen. Undefined means no effect can be chosen
* here, and the section renders disabled, so the menu keeps the same shape
* wherever background processing is unavailable.
*/
onSelectBackgroundEffect?: (id: string) => void;
/**
* Called with the file the user chose from the add tile. Omit to leave that
* tile out. The picker lives here rather than with the caller because
* opening it takes the focus, which would otherwise dismiss the menu.
*/
onAddBackgroundImage?: (file: File) => void;
/**
* Called to remove one of the user's own backgrounds. Omit to offer no
* removal. The one in force is never offered: it is what the user is
* wearing, and taking it away would leave them with nothing chosen.
*/
onRemoveBackgroundEffect?: (id: string) => void;
/**
* Why the last file the user offered could not be used, if it could not.
* Shown with the grid, where they chose it.
*/
backgroundEffectError?: string;
/**
* What the user should know before choosing, if anything — that effects run
* slowly in this browser, say. Told, not decided for them: the cost is
* smoothness and the reason to pay it is privacy, and only they know which
* they would rather have.
*/
backgroundEffectNotice?: string;
/**
* Whether the pipeline is still being built, which happens once a session on
* the first effect chosen. Shown on that effect's own tile, in place of its
* tick, rather than as a message: it belongs to what the user just pressed,
* and where the wait is short a message would only flash.
*/
backgroundEffectSettling?: boolean;
/**
* How the user looks right now, with whatever effect is in force: drawn at
* the top of the camera menu, where choosing a background can be judged by
* its result rather than by a thumbnail of it. Shown only where the call
* leaves room for it, and scrolled beneath rather than scrolled away.
*/
selfPreview?: ReactNode;
/**
* For any toggle and option this method will be called.
* So toggles need to be implemented by listening here and setting the right toggle item to `enabled`
@@ -161,58 +75,16 @@ export interface MediaMuteAndSwitchButtonProps {
const BLUR_ID = "blur";
/**
* Stands for wherever the platform is sending audio, where it will not say.
*
* Not a device id the browser would recognise: nothing can be selected on a
* platform that lists no outputs, so this is only ever shown, never sent.
*/
/** Id of the placeholder "Default" row, shown when the platform lists no outputs. */
const DEFAULT_OUTPUT_ID = "default";
/**
* The share of the call area the device list may fill.
*
* The menu carries its headings and the level meter as well, and a list that
* took the whole call would hide the call it belongs to.
*/
/** Largest share of the call area's height the device list may take. */
const LIST_SHARE_OF_CALL = 0.6;
/**
* The shortest the device list may be, whatever the call measures.
*
* A share alone collapses in a small call to a list that shows one device and
* gives no sign that there are others. Scrolling a short list is the better
* failure.
*/
/** Smallest device list height in px, so a short call still shows more than one device. */
const MIN_LIST_HEIGHT = 160;
/**
* The most the self-preview may take, which is also what it costs the list.
*
* Its fixed height: sixteen by nine across the camera menu's width, less the
* frame, so it never costs more than this. The
* preview is paid for out of the list's share rather than on top of it, so the
* menu is never taller for having one: where the list could not keep its floor
* after paying, there is no preview at all.
*/
const PREVIEW_BLOCK = 166;
/**
* The camera menu's width, as design sets it.
*
* Fixed rather than sized to its content, so a long device name wraps instead
* of widening the menu over more of the picture the user is trying to judge,
* and so the self-preview is always the sixteen by nine it is drawn at. Less
* only where the call area itself is narrower — the menu has to stay inside
* it — which is why it is measured rather than written into the stylesheet.
*/
const CAMERA_MENU_WIDTH = 296;
/** Clear space kept between the menu and the call area's sides. */
const CAMERA_MENU_MARGIN = 16;
export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
title,
enabled,
busy,
onMuteClick,
@@ -224,81 +96,36 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
onSelectOutput,
videoBlurEnabled,
videoBlurToggleClick,
backgroundEffects,
selectedBackgroundEffect,
onSelectBackgroundEffect,
onAddBackgroundImage,
onRemoveBackgroundEffect,
backgroundEffectError,
backgroundEffectNotice,
backgroundEffectSettling,
selfPreview,
onSelect,
}) => {
// Which device we have asked for but not yet been given. Carries the kind as
// well as the id, because an input and an output can share an id: "default"
// names both on Chrome.
// Held open across the file picker: a native dialog takes the focus, and the
// menu would take that as a click elsewhere and close behind it.
const [choosingFile, setChoosingFile] = useState(false);
// The refusal belongs to the attempt that caused it. It stays while the menu
// is open, can be dismissed, and is gone by the time the menu is opened
// again: a message about a file chosen minutes ago explains nothing.
const [refusalSeen, setRefusalSeen] = useState(false);
useEffect(() => setRefusalSeen(false), [backgroundEffectError]);
const chooseFile = useRef<HTMLInputElement>(null);
useEffect(() => {
const input = chooseFile.current;
if (!input) return;
// Dismissing the picker without choosing fires `cancel`, which React does
// not type, so it is listened for directly. Without it the menu would
// stay pinned open after a cancelled pick.
const done = (): void => setChoosingFile(false);
input.addEventListener("cancel", done);
return (): void => input.removeEventListener("cancel", done);
}, [onAddBackgroundImage]);
// Requested but not yet selected. Keyed by kind too, since Chrome uses
// "default" for both an input and an output.
const [plannedSelection, setPlannedSelection] = useState<{
kind: "input" | "output";
id: string;
} | null>(null);
const [menuOpen, setMenuOpen] = useState(false);
const onOpenChange = useCallback((open: boolean): void => {
setMenuOpen(open);
// Drop a request that never arrived.
if (!open) setPlannedSelection(null);
}, []);
const isBusy = busy ?? false;
const { t } = useTranslation();
const devices = useMediaDevices();
// Radix focuses whatever the pointer is over, so the browser's own
// :focus-visible cannot tell us whether a person is navigating by keyboard:
// Chromium answers yes to everything after any key press, Firefox answers no
// to programmatic focus. Track it ourselves and let the styling follow.
/**
* Tracks which modality moved the focus, for as long as the list is mounted.
*
* A ref rather than an effect on `menuOpen`: that state is ours, the open
* menu is Radix's, and the two do not commit together — an effect keyed on
* ours can run before Radix has mounted the content, with nothing to attach
* to. The list existing is the honest signal that the menu is open.
*
* Recorded on the menu rather than held in state, because every item the menu
* can focus has to answer to it — the device rows and the camera menu's blur
* toggle, which is the menu's child and not the list's — and because which
* modality someone is using changes nothing that has to be rendered again.
* Records on the menu whether the keyboard or the pointer moved focus, for
* the focus ring. `:focus-visible` can't tell: Radix focuses whatever the
* pointer is over. Listened for on the menu, not the document, so a second
* Element Call on the page doesn't answer for this one.
*/
const trackFocusModality = useCallback(
const trackFocusSource = useCallback(
(list: HTMLDivElement | null): (() => void) | undefined => {
// Watched on the menu, not on the document. Element Call can be mounted
// more than once in a host's page, and the menu is portalled out of the
// call root, so a document listener would also answer for a key pressed
// in the other instance, or in the host's own page. The menu rather than
// the list, because the first arrow key arrives while the menu itself
// holds focus, above anything we render.
const menu = list?.closest<HTMLElement>('[role="menu"]');
if (menu === null || menu === undefined) return;
// Each opening starts over: the modality belongs to whoever is using this
// menu now, not to whoever last used it.
const record = (modality: "keyboard" | "pointer"): void => {
menu.dataset.focusModality = modality;
const record = (source: "keyboard" | "pointer"): void => {
menu.dataset.focusSource = source;
};
record("pointer");
const usedKeyboard = (): void => record("keyboard");
@@ -313,150 +140,34 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
[],
);
// The menu is portalled outside the call root, so nothing in the stylesheets
// can size it against the call. Measure the call area rather than the window,
// or the menu is wrong wherever Element Call is not the whole page.
// Measured on the call area: CSS can't size the portalled menu against it.
const rootElement = useRootElement();
const [listShare, setListShare] = useState<number>();
const [listMaxHeight, setListMaxHeight] = useState<number>();
useEffect(() => {
if (!menuOpen) return;
// Followed rather than measured once: a host can resize the space Element
// Call is drawn in while the menu is open — a panel animating, a window
// dragged, a phone turned — and a bound taken on opening then describes a
// call area that no longer exists. Quantised before it reaches React, so a
// resize re-renders only when the bound itself moves.
// Followed, since a host can resize the call while the menu is open.
const subscription = observeElementSize$(rootElement)
.pipe(
map(({ height }) => Math.round(height * LIST_SHARE_OF_CALL)),
distinctUntilChanged(),
)
.subscribe(setListShare);
return (): void => subscription.unsubscribe();
}, [menuOpen, rootElement]);
// Whether the list has more to show above or below what is in view, and how
// tall the heading stuck at its top is. Said with a fade at each edge rather
// than left to the scrollbar, which the platform fades away after a moment —
// with a trackpad on a Mac there is otherwise nothing to say the list scrolls
// at all. Worked out from the list itself, on scrolling, on resizing and
// after every render, since adding or removing a background changes its
// length without doing either.
const [listElement, setListElement] = useState<HTMLDivElement | null>(null);
const listRef = useCallback(
(list: HTMLDivElement | null): (() => void) | undefined => {
setListElement(list);
return trackFocusModality(list);
},
[trackFocusModality],
);
const [edges, setEdges] = useState({ above: false, below: false, stuck: 0 });
const measureEdges = useCallback((): void => {
if (listElement === null) return;
const above = listElement.scrollTop > 0;
const below =
listElement.scrollTop + listElement.clientHeight <
listElement.scrollHeight - 1;
// The heading holding the top is the one at the list's own top edge, give
// or take the border width it keeps clear of the frame.
const top = listElement.getBoundingClientRect().top;
let stuck = 0;
if (above)
for (const heading of listElement.querySelectorAll<HTMLElement>(
`.${styles.sectionHeading}`,
)) {
const box = heading.getBoundingClientRect();
// How far down it reaches, to the fraction: a heading's line height
// lands it on half pixels, and rounding left the fade half a pixel
// short of it or over it.
if (Math.abs(box.top - top) < 2) stuck = box.bottom - top;
}
setEdges((previous) =>
previous.above === above &&
previous.below === below &&
previous.stuck === stuck
? previous
: { above, below, stuck },
);
}, [listElement]);
useLayoutEffect(measureEdges);
useEffect(() => {
if (listElement === null) return;
listElement.addEventListener("scroll", measureEdges, { passive: true });
const subscription =
observeElementSize$(listElement).subscribe(measureEdges);
return (): void => {
listElement.removeEventListener("scroll", measureEdges);
subscription.unsubscribe();
};
}, [listElement, measureEdges]);
const [cameraMenuWidth, setCameraMenuWidth] = useState(CAMERA_MENU_WIDTH);
useEffect(() => {
if (!menuOpen || iconsAndLabels !== "video") return;
const subscription = observeElementSize$(rootElement)
.pipe(
map(({ width }) =>
Math.min(
CAMERA_MENU_WIDTH,
Math.round(width - 2 * CAMERA_MENU_MARGIN),
),
map(({ height }) =>
Math.max(MIN_LIST_HEIGHT, Math.round(height * LIST_SHARE_OF_CALL)),
),
distinctUntilChanged(),
)
.subscribe(setCameraMenuWidth);
.subscribe(setListMaxHeight);
return (): void => subscription.unsubscribe();
}, [menuOpen, iconsAndLabels, rootElement]);
}, [menuOpen, rootElement]);
// Only while the menu is open, so nothing holds a second capture of the
// microphone for the length of a call.
// The meter sits over the foot of the scrolling list, so the list has to
// keep that much of itself clear: a row scrolled to by the keyboard would
// otherwise arrive underneath it, half-read. Its own height, measured,
// because the failure states are two lines where a level is one.
const [meterHeight, setMeterHeight] = useState<number>();
const meter = useCallback(
(element: HTMLDivElement | null): (() => void) | undefined => {
if (element === null) return;
const subscription = observeElementSize$(element)
.pipe(
map(({ height }) => height),
distinctUntilChanged(),
)
.subscribe(setMeterHeight);
return (): void => subscription.unsubscribe();
},
[],
);
// Kept clear at the list's foot, so a row reached by keyboard isn't under
// the meter.
const [meterHeight, meter] = useMeasuredHeight();
// The headings stand over the head of the list, so it has to keep their
// height clear too — the same bargain as the meter, at the other end. One
// measurement serves both: the sections are headed alike.
const [headingHeight, setHeadingHeight] = useState<number>();
const heading = useCallback(
(element: HTMLDivElement | null): (() => void) | undefined => {
if (element === null) return;
const subscription = observeElementSize$(element)
.pipe(
map(({ height }) => height),
distinctUntilChanged(),
)
.subscribe(setHeadingHeight);
return (): void => subscription.unsubscribe();
},
[],
);
const microphoneState = useMicrophoneLevel(
selectedOption,
menuOpen && iconsAndLabels === "audio",
);
// Likewise at its head, for the sticky headings.
const [headingHeight, heading] = useMeasuredHeight();
useEffect(() => {
if (menuOpen) devices.requestDeviceNames(); // No-op after the first call
}, [menuOpen, devices]);
// The mute control differs between the two only in which button it is and
// what it is called; how it behaves is the same, and was worth saying once.
const MuteButton = iconsAndLabels === "audio" ? MicButton : VideoButton;
const button = (
<MuteButton
@@ -474,7 +185,6 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
/>
);
// Only the camera menu carries a toggle, and only when the caller offers one.
const toggles =
iconsAndLabels === "video" && videoBlurToggleClick !== undefined
? [
@@ -487,24 +197,23 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
: [];
let optionsButtonLabel: string;
let defaultMenuTitle: string;
let menuTitle: string;
let numberedLabel: (number: number) => string;
switch (iconsAndLabels) {
case "video":
optionsButtonLabel = t("settings.devices.camera");
defaultMenuTitle = t("settings.devices.camera_source");
menuTitle = t("settings.devices.camera_source");
numberedLabel = (n): string =>
t("settings.devices.camera_numbered", { n });
break;
case "audio":
optionsButtonLabel = t("settings.devices.microphone");
defaultMenuTitle = t("settings.devices.mic_source");
menuTitle = t("settings.devices.mic_source");
numberedLabel = (n): string =>
t("settings.devices.microphone_numbered", { n });
break;
}
/** The text shown for a device, whichever kind of label it carries. */
const labelText = (
label: MenuOptions["label"],
numbered: (n: number) => string,
@@ -525,26 +234,24 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
}
};
// A device we asked for that has not arrived yet. Until it does, nothing in
// the menu can be picked, so a second request cannot overtake the first.
// Nothing can be picked while a requested device is on offer but not yet
// selected, so a second request can't overtake it.
const plannedOutput = plannedSelection?.kind === "output";
const selectedOfPlannedKind = plannedOutput
? selectedOutputOption
: selectedOption;
const offeredOfPlannedKind = plannedOutput ? outputOptions : options;
const settling =
plannedSelection !== null &&
plannedSelection.id !==
(plannedSelection.kind === "output"
? selectedOutputOption
: selectedOption);
plannedSelection.id !== selectedOfPlannedKind &&
offeredOfPlannedKind?.some(({ id }) => id === plannedSelection.id) === true;
// Safari enumerates no output devices at all, and offers no way to choose
// one, so the list arrives empty. The section is shown all the same — audio
// is playing somewhere — naming that somewhere and disabling it like any
// single entry. A heading with nothing beneath it reads as a broken feature,
// and leaves the menu a different shape on one browser.
// Safari lists no outputs: show a disabled, selected Default rather than an
// empty section.
const noOutputsListed = outputOptions?.length === 0;
const speakerOptions: MenuOptions[] | undefined = noOutputsListed
? [{ id: DEFAULT_OUTPUT_ID, label: { type: "default", name: null } }]
: outputOptions;
// And it is the selection, not merely the only row: it is where audio is
// going. An unchecked lone entry reads as nothing being chosen at all.
const selectedSpeaker = noOutputsListed
? DEFAULT_OUTPUT_ID
: selectedOutputOption;
@@ -557,30 +264,23 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
numbered: (n: number) => string,
): ReactElement[] => {
const list = items ?? [];
// Shown but not choosable when nothing can be picked here, or when there is
// only one device. The entry stays visible so the menu keeps the same shape
// on every platform.
// Disabled rather than hidden, so the menu keeps its shape.
const disabled = select === undefined || list.length <= 1 || settling;
return list.map(({ label, id }) => (
<MenuItem
// A radio input inside a button is invalid, and the menu needs an
// element it can give menuitemradio semantics to.
// A radio input may not sit inside a button.
as="div"
hideChevron
disabled={disabled}
label={labelText(label, numbered)}
Icon={
// Inert, not aria-hidden: a form control inside a menu item must be
// out of the focus order and out of the accessibility tree, and
// aria-hidden alone leaves it focusable. The item's aria-checked is
// what conveys the selection.
// Inert, not aria-hidden: aria-hidden alone leaves it focusable.
<span inert>
<RadioInput
checked={selected === id}
disabled={disabled}
// Not readOnly: that styles the control as muted, losing the
// accent fill that marks the selection. The menu item owns the
// interaction, so the change handler has nothing to do.
// Not readOnly, which mutes the selected fill. The item handles
// the click.
onChange={(): void => {}}
/>
</span>
@@ -609,209 +309,6 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
));
};
// The camera menu's Background effects section. Tiles rather than rows, but
// menuitemradio like the device list above, so the menu stays one keyboard
// model throughout and the selection is announced rather than only drawn.
const effectTiles = (): ReactElement[] => {
const list = backgroundEffects ?? [];
// Shown but not choosable where background processing is unavailable, so
// the menu keeps the same shape on every platform. No effect is the
// exception: it needs no processing, so it stays choosable and stays the
// one in force.
const unavailable = onSelectBackgroundEffect === undefined;
// The one in force is never offered for removal: FR-025. Everything else
// the user added is theirs to take away.
const canRemove = (effect: BackgroundEffectOption): boolean =>
effect.removable === true &&
onRemoveBackgroundEffect !== undefined &&
selectedBackgroundEffect !== effect.id;
const tiles = list.map((effect) => {
const tile = (
<MenuItem
as="div"
hideChevron
disabled={unavailable && effect.kind !== "none"}
// The cross is drawn, not focusable: a control inside a menu item is
// invalid, and the item is what the arrow keys walk. Delete reaches
// the same action from the keyboard, announced by aria-keyshortcuts.
aria-keyshortcuts={canRemove(effect) ? "Delete" : undefined}
onKeyDown={
canRemove(effect)
? (e: React.KeyboardEvent): void => {
if (e.key !== "Delete" && e.key !== "Backspace") return;
e.preventDefault();
onRemoveBackgroundEffect?.(effect.id);
}
: undefined
}
className={classNames(styles.effectTile, {
[styles.effectTileSelected]: selectedBackgroundEffect === effect.id,
})}
label={effect.label}
// An image is its own label, so its name is carried for assistive
// technology alone rather than drawn over the picture.
labelProps={{
className:
effect.kind === "image"
? styles.effectLabelUnseen
: styles.effectLabel,
}}
Icon={
<span aria-hidden className={styles.effectSwatch}>
{effect.kind === "image" && (
<img
className={styles.effectThumb}
src={effect.imageUrl}
alt=""
/>
)}
{selectedBackgroundEffect === effect.id &&
backgroundEffectSettling ? (
// Before the tick, on the tile that was pressed: the feedback
// belongs to the thing the user acted on. It spins by rotating,
// which a browser composites, so it keeps turning through the
// pause where the page itself has stopped — the difference
// between looking busy and looking crashed.
<span
className={classNames(styles.effectCheck, styles.effectBusy, {
[styles.effectCheckOnImage]: effect.kind === "image",
})}
>
<InlineSpinner size={20} />
</span>
) : selectedBackgroundEffect === effect.id ? (
// The tick stands where the glyph would, and on a picture it
// carries its own ground so it reads against whatever is behind
// it.
<CheckCircleSolidIcon
className={classNames(styles.effectCheck, {
[styles.effectCheckOnImage]: effect.kind === "image",
})}
width={20}
height={20}
/>
) : (
<>
{effect.kind === "none" && (
<BlockIcon width={20} height={20} />
)}
{effect.kind === "blur" && (
<span className={styles.effectBlurGlyph} />
)}
</>
)}
</span>
}
onSelect={(e) => {
e.preventDefault();
if (effect.id === selectedBackgroundEffect) return;
onSelectBackgroundEffect?.(effect.id);
}}
key={effect.id}
role="menuitemradio"
aria-checked={selectedBackgroundEffect === effect.id}
/>
);
// The cross sits beside the item rather than inside it. Inside, its
// press and its click both reach the item first however they are
// handled, so the tile was chosen and the cross taken away before it
// could act: by mouse nothing happened, while the keyboard, which never
// touches it, worked.
return canRemove(effect) ? (
<div className={styles.effectTileWrap} key={effect.id}>
{tile}
{/* The cross carries no words, so hovering it names what it does.
Interactive as far as the tooltip is concerned — that keeps it
from wrapping the cross in a tab stop of its own, which is the
one thing this must not grow. */}
<Tooltip label={t("action.remove")}>
{/* eslint-disable-next-line jsx-a11y/click-events-have-key-events,
jsx-a11y/no-static-element-interactions --
Deliberately not a control: aria-hidden, so assistive technology
never meets it, and the keyboard reaches the same action through
Delete on the tile. A role and a tab stop here would add a stop
the menu's arrow keys know nothing about. */}
<span
aria-hidden
className={styles.effectRemove}
onClick={(): void => onRemoveBackgroundEffect?.(effect.id)}
>
<CloseIcon width={20} height={20} />
</span>
</Tooltip>
</div>
) : (
tile
);
});
// Adding is a command, not a choice, so it is a plain item among the tiles
// rather than another radio.
if (onAddBackgroundImage !== undefined)
tiles.push(
<MenuItem
as="div"
hideChevron
disabled={unavailable}
className={styles.effectTile}
label={t("action.add_background_image")}
// The design draws a plus alone; the name is kept for assistive
// technology, which has nothing else to go on.
labelProps={{ className: styles.effectLabelUnseen }}
Icon={
<span aria-hidden className={styles.effectSwatch}>
<PlusIcon width={24} height={24} />
</span>
}
onSelect={(e) => {
e.preventDefault();
setChoosingFile(true);
chooseFile.current?.click();
}}
key="add-background-image"
/>,
);
return tiles;
};
// The preview is always there in the camera menu; what the call's height
// decides is whether it stays put. Pinned above the list, it is paid for out
// of the list's share, and only where the list keeps its floor after paying,
// so a menu with one is no taller than a menu without. Where the list could
// not afford it, it goes into the list instead, first, and scrolls away with
// everything else: seen on opening, out of the way once the user is choosing.
const hasPreview =
iconsAndLabels === "video" &&
selfPreview !== undefined &&
listShare !== undefined;
const previewPinned =
hasPreview && listShare - PREVIEW_BLOCK >= MIN_LIST_HEIGHT;
const listMaxHeight =
listShare === undefined
? undefined
: Math.max(
MIN_LIST_HEIGHT,
listShare - (previewPinned ? PREVIEW_BLOCK : 0),
);
const preview = hasPreview && (
// Decoration to assistive technology — the choice is announced by the
// items, and a picture of the user tells them nothing new.
<div
aria-hidden
className={classNames(styles.selfPreview, {
[styles.selfPreviewPinned]: previewPinned,
})}
>
{backgroundEffectSettling ? (
// The same wait the pressed tile shows, where the picture will be, so
// the eye does not have to go looking for why it is not.
<InlineSpinner size={32} />
) : (
selfPreview
)}
</div>
);
return (
<div
className={classNames({
@@ -821,35 +318,13 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
>
{/* The mute button lives inside */}
{button}
{onAddBackgroundImage !== undefined && (
<input
ref={chooseFile}
type="file"
accept="image/*"
hidden
onChange={(e) => {
const file = e.target.files?.[0];
// Cleared so the same file can be chosen twice in a row.
e.target.value = "";
setChoosingFile(false);
if (file) onAddBackgroundImage(file);
}}
/>
)}
<Menu
className={styles.menu}
title={title ?? defaultMenuTitle}
// Each section carries its own heading, so the menu's own title would
// sit on top of the first one. Kept for the accessible name only.
// Named for screen readers only: each section has its own heading.
title={menuTitle}
showTitle={false}
open={menuOpen}
onOpenChange={(open) => {
// Ignore the close the file picker provokes by taking the focus.
if (!open && choosingFile) return;
// A refusal does not outlive the menu it was shown in.
if (!open) setRefusalSeen(true);
setMenuOpen(open);
}}
onOpenChange={onOpenChange}
side="top"
trigger={
<Button
@@ -865,50 +340,28 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
/>
}
>
{/* Pinned where there is room: the devices and the effects scroll
beneath it, because it is what the choosing below is for. */}
{previewPinned && preview}
<div
ref={listRef}
// Transparent to assistive technology, so the menu still sees its
// items as its own children.
ref={trackFocusSource}
// Keeps the items the menu's own children for assistive tech.
role="none"
className={classNames(styles.deviceList, {
[styles.deviceListLeadsWithPreview]: hasPreview && !previewPinned,
})}
className={styles.deviceList}
data-testid="device-list"
style={
{
"--device-list-max-height":
listMaxHeight === undefined ? undefined : `${listMaxHeight}px`,
// The menu takes its width from its content, and this is the
// content that could otherwise set it. Only the camera menu.
"--device-list-inline-size":
iconsAndLabels === "video" ? `${cameraMenuWidth}px` : undefined,
"--device-list-scroll-padding-end":
meterHeight === undefined ? undefined : `${meterHeight}px`,
"--device-list-scroll-padding-start":
headingHeight === undefined ? undefined : `${headingHeight}px`,
"--device-list-stuck-heading-height": `${edges.stuck}px`,
} as CSSProperties
}
>
{iconsAndLabels === "video" && (
<div
aria-hidden
className={classNames(styles.scrollEdge, styles.scrollEdgeTop, {
[styles.scrollEdgeShown]: edges.above,
})}
/>
)}
{hasPreview && !previewPinned && preview}
{iconsAndLabels === "audio" && speakerOptions && (
<>
{/* A menu may only contain items, separators and groups, so each
heading belongs to a group rather than sitting beside the
items it names. */}
heading is a hidden part of a named group. */}
<div role="group" aria-label={t("settings.devices.speaker")}>
{/* The heading is decoration: the group carries the name, and
a menu may only contain items, separators and groups. */}
<div aria-hidden className={styles.sectionHeading}>
<MenuTitle title={t("settings.devices.speaker")} />
</div>
@@ -926,8 +379,7 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
<div ref={heading} aria-hidden className={styles.sectionHeading}>
<MenuTitle title={optionsButtonLabel} />
</div>
{/* The heading sits outside, so the meter can never ride up over it:
sticky only holds while this block is in view. */}
{/* Apart from the heading, so the sticky meter can't ride over it. */}
<div role="none">
{deviceItems(
"input",
@@ -937,68 +389,17 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
numberedLabel,
)}
{iconsAndLabels === "audio" && (
<MicrophoneLevelMeter
<LiveMicrophoneLevelMeter
ref={meter}
state={microphoneState}
deviceId={selectedOption}
// Only while open, so the microphone isn't held all call.
active={menuOpen}
className={styles.stickyMeter}
/>
)}
</div>
</div>
{iconsAndLabels === "video" &&
backgroundEffects !== undefined &&
backgroundEffects.length > 0 && (
<div
role="group"
aria-label={t("settings.background_effects_header")}
>
<div aria-hidden className={styles.sectionHeading}>
<MenuTitle title={t("settings.background_effects_header")} />
</div>
<div role="none" className={styles.effectGrid}>
{effectTiles()}
</div>
</div>
)}
{iconsAndLabels === "video" && (
<div
aria-hidden
className={classNames(
styles.scrollEdge,
styles.scrollEdgeBottom,
{
[styles.scrollEdgeShown]: edges.below,
},
)}
/>
)}
</div>
{backgroundEffectError !== undefined &&
!refusalSeen && (
// Outside the list, which scrolls and is bounded by the call: inside
// it, a message could be cut off by the menu's own height or left
// below the fold, which is no way to tell someone something failed.
<div role="none" className={styles.effectError}>
<Alert
type="critical"
title={backgroundEffectError}
onClose={(): void => setRefusalSeen(true)}
/>
</div>
)}
{backgroundEffectError === undefined &&
backgroundEffectNotice !== undefined && (
// Never both at once: a refusal is about what the user just did and
// wins the space, where this is about the browser and will still be
// true next time. Not dismissible for the same reason — there is
// nothing to dismiss, only something to know, and it is drawn to be
// read once rather than to interrupt: a subtle ground, an outline
// mark and text at its ordinary weight, as the design has it.
<div role="none" className={styles.effectNotice}>
<InfoIcon width={20} height={20} />
<span>{backgroundEffectNotice}</span>
</div>
)}
{toggles.length > 0 && <hr />}
{toggles.map((toggle) => (
<ToggleMenuItem
@@ -1015,3 +416,25 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
</div>
);
};
/** Follows an element's height. */
function useMeasuredHeight(): [
number | undefined,
(element: HTMLElement | null) => (() => void) | undefined,
] {
const [height, setHeight] = useState<number>();
const ref = useCallback(
(element: HTMLElement | null): (() => void) | undefined => {
if (element === null) return;
const subscription = observeElementSize$(element)
.pipe(
map((size) => size.height),
distinctUntilChanged(),
)
.subscribe(setHeight);
return (): void => subscription.unsubscribe();
},
[],
);
return [height, ref];
}
+18 -33
View File
@@ -6,20 +6,21 @@ Please see LICENSE in the repository root for full details.
*/
.meter {
/* A device row's icon column, so the bars start where the names do. */
--meter-icon-size: 24px;
/* Compound's radio size, whose centre line the icon shares. */
--device-control-size: 20px;
display: flex;
align-items: center;
gap: var(--cpd-space-3x);
padding-block: var(--cpd-space-2x);
/* The icon shares a centre with the radio controls in the device rows above,
which sit one 4x padding in from the menu's edge. Two things pull this row
out of line with them, and both come off the padding: the row is inset by a
border width so that the menu's frame stays visible behind it (see
`.stickyMeter`), and the icon is wider than a radio control, so it has to
start further left for the two to share a centre. Both are
measured in the MeterAlignsWithTheDeviceRows story rather than trusted:
jsdom lays nothing out, so only a real browser can hold this. */
/* Centres the icon on the radio column: less the row's inset border and half
the icon's extra width. Checked in the MeterAlignsWithTheDeviceRows story. */
padding-inline-start: calc(
var(--cpd-space-4x) - var(--cpd-border-width-1) - 2px
var(--cpd-space-4x) - var(--cpd-border-width-1) -
(var(--meter-icon-size) - var(--device-control-size)) / 2
);
padding-inline-end: calc(var(--cpd-space-4x) * 2 + var(--cpd-space-2x));
}
@@ -27,33 +28,20 @@ Please see LICENSE in the repository root for full details.
.icon {
color: var(--cpd-color-icon-secondary);
flex-shrink: 0;
/* Same width as a device row's icon column, so the bars start where the
device names start. */
inline-size: 24px;
inline-size: var(--meter-icon-size);
block-size: var(--meter-icon-size);
}
.segments {
/* A bar and the space beside it are always the same size; what changes with
the width available is how many bars there are. Spreading a fixed number of
bars instead would make the meter a different shape in every place it is
used, and close them up into one solid block wherever the space ran short —
and a meter whose bars touch stops reading as a count, which is what
carries the level for anyone who cannot rely on the colour.
Both sizes are set here and nowhere else: the component reads them back off
the rendered bars to work out how many fit. */
/* Bars keep one size and the width decides how many; the component reads the
sizes back from here. */
flex: 1;
/* The width decides the count, never the other way round. Containment is
what enforces that: without it the bars are both a floor under the width
and the widest thing in the menu, so the meter would set the menu's width
and the device names — which are what a person is reading — would have to
fit around it. */
/* So the bars never set the menu's width. */
contain: inline-size;
min-inline-size: 0;
display: flex;
align-items: center;
/* Half again as wide as a bar, as the design has it: bars set as close as
their own width read as a solid block long before they touch. */
/* Half again a bar's width, per the design. */
gap: var(--cpd-space-1-5x);
}
@@ -65,16 +53,13 @@ Please see LICENSE in the repository root for full details.
background: var(--cpd-color-bg-subtle-primary);
}
/* Filled segments carry the level by count as well as by colour, so the meter
stays readable without relying on hue. */
/* The count of lit bars carries the level as well as their colour. */
.segmentLit {
background: var(--cpd-color-bg-accent-rest);
}
.message {
color: var(--cpd-color-text-secondary);
/* A paragraph brings a margin below it, which in a centred row does not push
the text down but lifts it: the margin box is what gets centred, so the
words end up above the middle and the icon beside them looks low. */
/* A paragraph's margin would lift the text off centre. */
margin-block: 0;
}
+26 -54
View File
@@ -14,17 +14,10 @@ import {
type MicrophoneLevelMeterProps,
} from "./MicrophoneLevelMeter";
import styles from "./MicrophoneLevelMeter.module.css";
import { METER_SEGMENTS } from "../state/MicrophoneLevel";
import { LEVEL_SCALE } from "../state/MicrophoneLevel";
import { constant } from "../state/Behavior";
/**
* A width to show the meter at, close to the menu it lives in.
*
* Not a copy of the menu's width, and nothing depends on the two agreeing: a
* bar and the gap beside it are a fixed size now, so this only decides how many
* bars there is room for. Without a width at all the stories would shrink-wrap
* to almost nothing and show a meter two bars wide, which is no use to anyone
* looking at them.
*/
/** Roughly the menu's width. It only decides how many bars fit. */
const STORY_WIDTH = 256;
const meta = {
@@ -47,29 +40,22 @@ const meta = {
export default meta;
type Story = StoryObj<typeof meta>;
/**
* A quiet room. Nothing is lit: room hiss below the noise floor must not read
* as "it can hear me".
*/
/** A quiet room: hiss below the noise floor lights nothing. */
export const Silent: Story = {
args: { state: { type: "level", level: 0 } },
args: { state: { type: "level", level$: constant(0) } },
play: async ({ canvasElement }) => {
await expect(litSegments(canvasElement)).toBe(0);
},
};
export const QuietSpeech: Story = {
args: { state: { type: "level", level: 5 } },
args: { state: { type: "level", level$: constant(5) } },
};
export const NormalSpeech: Story = {
args: { state: { type: "level", level: 12 } },
args: { state: { type: "level", level$: constant(12) } },
play: async ({ canvasElement }) => {
// Shown at something like the width of the menu, so the meter in a story
// reads like the meter in a call rather than like a handful of bars. The
// design's own mock has sixteen of them at this width; a floor rather than
// a count, because the number follows from the bar and gap sizes and those
// are the design's to change.
// A floor, not a count: the count follows from the design's bar and gap sizes.
await expect(
canvasElement.getElementsByClassName(styles.segment).length,
).toBeGreaterThanOrEqual(15);
@@ -77,48 +63,44 @@ export const NormalSpeech: Story = {
};
export const LoudSpeech: Story = {
args: { state: { type: "level", level: METER_SEGMENTS } },
args: { state: { type: "level", level$: constant(LEVEL_SCALE) } },
};
/**
* The three volumes differ by how many bars are lit, so the level survives
* greyscale and a screen reader as well as it survives colour.
*/
/** The three volumes differ in how many bars are lit, not only in colour. */
export const VolumesAreDistinguishable: Story = {
args: { state: { type: "level", level: 5 } },
args: { state: { type: "level", level$: constant(5) } },
play: async ({ canvasElement, mount }) => {
const lit: number[] = [];
for (const level of [5, 12, METER_SEGMENTS]) {
await mount(<MicrophoneLevelMeter state={{ type: "level", level }} />);
for (const level of [5, 12, LEVEL_SCALE]) {
await mount(
<MicrophoneLevelMeter
state={{ type: "level", level$: constant(level) }}
/>,
);
lit.push(litSegments(canvasElement));
await expect(within(canvasElement).getByRole("meter")).toHaveAttribute(
"aria-valuenow",
String(level),
);
}
// Three different counts, rising: the level is carried by how many bars
// are lit, not by their colour alone.
await expect(new Set(lit).size).toBe(lit.length);
await expect(lit).toEqual([...lit].sort((a, b) => a - b));
},
};
/**
* Permission refused. A message with a next action, never a still meter that
* reads as silence.
*/
/** Permission refused: a message with a next action, not a still meter. */
export const PermissionDenied: Story = {
args: { state: { type: "permission-denied" } },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(canvas.queryByRole("meter")).toBeNull();
await expect(
canvas.getByText(/Allow access in your browser settings/),
canvas.getByText(/Microphone access is blocked/),
).toBeVisible();
},
};
/** No input device at all, told apart from a refusal. */
/** No input device, told apart from a refusal. */
export const NoDevice: Story = {
args: { state: { type: "no-device" } },
play: async ({ canvasElement }) => {
@@ -126,9 +108,7 @@ export const NoDevice: Story = {
await expect(canvas.queryByRole("meter")).toBeNull();
await expect(canvas.getByText(/No microphone found/)).toBeVisible();
// The icon sits on the middle of the words, however many lines they run to.
// A paragraph's own margin would centre its margin box instead, leaving the
// text high and the icon looking low beside it.
// The icon sits on the middle of the text, however many lines it runs to.
const middle = (element: Element): number => {
const box = element.getBoundingClientRect();
return box.top + box.height / 2;
@@ -139,17 +119,9 @@ export const NoDevice: Story = {
},
};
/**
* The same meter at two widths.
*
* A bar and the space beside it are always the same size; what changes is how
* many bars there are. Spreading a fixed number of bars instead would make the
* meter a different shape in every place it is used, and close the bars up into
* one block wherever the space ran short — and bars that touch cannot be
* counted, which is what carries the level without colour.
*/
/** The same meter at two widths: the bars keep their size and only their count changes. */
export const ShapeStaysTheSameAtAnyWidth: Story = {
args: { state: { type: "level", level: 12 } },
args: { state: { type: "level", level$: constant(12) } },
play: async ({ mount, args }) => {
const narrow = await measureAt(mount, args, 180);
const wide = await measureAt(mount, args, 400);
@@ -157,17 +129,17 @@ export const ShapeStaysTheSameAtAnyWidth: Story = {
await expect(narrow.bar).toBe(wide.bar);
await expect(narrow.gap).toBe(wide.gap);
await expect(narrow.count).toBeLessThan(wide.count);
// And the bars are still bars, not one run of colour.
// The bars are still separate, not one run of colour.
await expect(narrow.gap).toBeGreaterThan(0);
},
};
/** How many bars are painted as carrying level, rather than as empty. */
/** How many bars are lit. */
function litSegments(canvasElement: HTMLElement): number {
return canvasElement.getElementsByClassName(styles.segmentLit).length;
}
/** Renders the meter at one width and reports the shape of its bars. */
/** Renders the meter at one width and reports its bars' shape. */
async function measureAt(
mount: (ui: JSX.Element) => Promise<unknown>,
args: MicrophoneLevelMeterProps,
+8 -6
View File
@@ -9,25 +9,27 @@ import { describe, expect, test } from "vitest";
import { render, screen } from "@testing-library/react";
import { MicrophoneLevelMeter } from "./MicrophoneLevelMeter";
import { METER_SEGMENTS } from "../state/MicrophoneLevel";
import { LEVEL_SCALE } from "../state/MicrophoneLevel";
import { constant } from "../state/Behavior";
describe("MicrophoneLevelMeter", () => {
test("announces the level rather than relying on hue", () => {
render(<MicrophoneLevelMeter state={{ type: "level", level: 6 }} />);
render(
<MicrophoneLevelMeter state={{ type: "level", level$: constant(6) }} />,
);
const meter = screen.getByRole("meter", { name: "Microphone level" });
expect(meter).toHaveAttribute("aria-valuenow", "6");
expect(meter).toHaveAttribute("aria-valuemax", String(METER_SEGMENTS));
expect(meter).toHaveAttribute("aria-valuetext", `6 of ${METER_SEGMENTS}`);
expect(meter).toHaveAttribute("aria-valuemax", String(LEVEL_SCALE));
expect(meter).toHaveAttribute("aria-valuetext", `6 of ${LEVEL_SCALE}`);
});
test("shows distinct messages for denied permission and no input device", () => {
const denied = render(
<MicrophoneLevelMeter state={{ type: "permission-denied" }} />,
);
// A next action, not a flat meter that reads as silence.
expect(
denied.getByText(/Allow access in your browser settings/),
denied.getByText(/Microphone access is blocked/),
).toBeInTheDocument();
expect(denied.queryByRole("meter")).toBeNull();
+90 -64
View File
@@ -5,7 +5,14 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { useCallback, useState, type FC, type Ref } from "react";
import {
useCallback,
useLayoutEffect,
useRef,
useState,
type FC,
type Ref,
} from "react";
import { Text } from "@vector-im/compound-web";
import { MicOnIcon } from "@vector-im/compound-design-tokens/assets/web/icons";
import classNames from "classnames";
@@ -14,107 +21,126 @@ import { useTranslation } from "react-i18next";
import { distinctUntilChanged, map } from "rxjs";
import styles from "./MicrophoneLevelMeter.module.css";
import { METER_SEGMENTS, type MicrophoneState } from "../state/MicrophoneLevel";
import { LEVEL_SCALE, type MicrophoneState } from "../state/MicrophoneLevel";
import { observeElementSize$ } from "../utils/elementSize";
import { useMicrophoneLevel } from "./useMicrophoneLevel";
export interface MicrophoneLevelMeterProps {
state: MicrophoneState;
export interface LiveMicrophoneLevelMeterProps {
/** The microphone to listen to. */
deviceId: string | undefined;
/** Whether to hold a capture at all. */
active: boolean;
className?: string;
/**
* The meter's own element. Its height is what a scroll container has to keep
* clear to stop the meter covering the row it has just scrolled to.
*/
ref?: Ref<HTMLDivElement>;
}
/**
* The live input level of the selected microphone, shown beneath it.
*
* It says whether the microphone is picking anything up, which is not the same
* as whether the user is being heard: it keeps moving while muted, and the mute
* control is what says nothing is transmitted.
* The meter, wired to a live capture. Its own component so a changing level
* re-renders the meter, not its parent.
*/
export const LiveMicrophoneLevelMeter: FC<LiveMicrophoneLevelMeterProps> = ({
deviceId,
active,
className,
ref,
}) => {
const state = useMicrophoneLevel(deviceId, active);
return <MicrophoneLevelMeter state={state} className={className} ref={ref} />;
};
export interface MicrophoneLevelMeterProps {
state: MicrophoneState;
className?: string;
ref?: Ref<HTMLDivElement>;
}
/** The input level of a microphone, or why there is none. */
export const MicrophoneLevelMeter: FC<MicrophoneLevelMeterProps> = ({
state,
className,
ref,
}) => {
const { t } = useTranslation();
// How many bars there is room for. The bars never change size, so this is
// what absorbs a change of width. Starts at the full count so that the first
// paint is a meter rather than a single bar, and so that a renderer with no
// layout at all — jsdom — still draws the whole thing.
const [barCount, setBarCount] = useState(METER_SEGMENTS);
// Bars keep one size, so their count follows the width. Starts full so the
// first paint, and jsdom, draw a whole meter.
const [barCount, setBarCount] = useState(LEVEL_SCALE);
const segments = useRef<HTMLDivElement | null>(null);
const track = useCallback(
(element: HTMLDivElement | null): (() => void) | undefined => {
if (element === null) return;
segments.current = element;
const subscription = observeElementSize$(element)
.pipe(
map(({ width }) => barsThatFit(element, width)),
distinctUntilChanged(),
)
.subscribe(setBarCount);
return (): void => subscription.unsubscribe();
return (): void => {
subscription.unsubscribe();
segments.current = null;
};
},
[],
);
if (state.type !== "level")
return (
<div ref={ref} className={classNames(styles.meter, className)}>
<MicOnIcon width={24} height={24} className={styles.icon} aria-hidden />
// Drawn straight into the DOM: the level changes many times a second, and
// re-rendering for each change is what this avoids.
useLayoutEffect(() => {
const element = segments.current;
if (state.type !== "level" || element === null) return;
const subscription = state.level$.subscribe((level) => {
element.setAttribute("aria-valuenow", String(level));
element.setAttribute(
"aria-valuetext",
t("microphone_level.value", { level, max: LEVEL_SCALE }),
);
// The level is a share of the scale, not a bar count.
const lit = Math.round((level / LEVEL_SCALE) * barCount);
Array.from(element.children).forEach((bar, i) =>
bar.classList.toggle(styles.segmentLit, i < lit),
);
});
return (): void => subscription.unsubscribe();
}, [state, barCount, t]);
return (
<div ref={ref} className={classNames(styles.meter, className)}>
<MicOnIcon className={styles.icon} aria-hidden />
{state.type !== "level" ? (
<Text size="sm" className={styles.message}>
{state.type === "permission-denied"
? t("microphone_level.permission_denied")
: t("microphone_level.no_device")}
</Text>
</div>
);
return (
<div ref={ref} className={classNames(styles.meter, className)}>
<MicOnIcon width={24} height={24} className={styles.icon} aria-hidden />
<div
ref={track}
className={styles.segments}
role="meter"
aria-label={t("microphone_level.label")}
aria-valuemin={0}
aria-valuemax={METER_SEGMENTS}
aria-valuenow={state.level}
aria-valuetext={t("microphone_level.value", {
level: state.level,
max: METER_SEGMENTS,
})}
>
{Array.from({ length: barCount }, (_, i) => (
<span
key={i}
aria-hidden
className={classNames(styles.segment, {
// The level is a share of the scale, not a number of bars: how
// many bars stand for it depends on how many there are.
[styles.segmentLit]:
i < Math.round((state.level / METER_SEGMENTS) * barCount),
})}
/>
))}
</div>
) : (
<div
ref={track}
className={styles.segments}
role="meter"
aria-label={t("microphone_level.label")}
aria-valuemin={0}
aria-valuemax={LEVEL_SCALE}
// The first paint's value; the effect above keeps it current.
aria-valuenow={state.level$.value}
aria-valuetext={t("microphone_level.value", {
level: state.level$.value,
max: LEVEL_SCALE,
})}
>
{Array.from({ length: barCount }, (_, i) => (
<span key={i} aria-hidden className={styles.segment} />
))}
</div>
)}
</div>
);
};
/**
* How many bars fit across `width`, at the size the stylesheet draws them.
*
* Measured off a rendered bar rather than told: the size of a bar and of the
* space beside it are a design question, settled in the stylesheet, and reading
* them back is what keeps them from being settled twice.
*/
/** How many bars fit across `width`, measured so the stylesheet alone sets their size. */
function barsThatFit(track: HTMLElement, width: number): number {
const gap = Number.parseFloat(getComputedStyle(track).columnGap);
const bar = track.firstElementChild?.getBoundingClientRect().width ?? 0;
// A renderer that lays nothing out tells us nothing; keep the full count.
if (!(bar > 0) || !(gap >= 0)) return METER_SEGMENTS;
// No layout (jsdom): keep the full count.
if (!(bar > 0) || !(gap >= 0)) return LEVEL_SCALE;
return Math.max(1, Math.floor((width + gap) / (bar + gap)));
}
-54
View File
@@ -1,54 +0,0 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC, useEffect, useRef } from "react";
import { facingModeFromLocalTrack, type LocalVideoTrack } from "livekit-client";
import { VideoCallOffIcon } from "@vector-im/compound-design-tokens/assets/web/icons";
import styles from "./MediaMuteAndSwitchButton.module.css";
interface Props {
/** The camera as the pipeline has it, or none while the camera is off. */
track: LocalVideoTrack | null;
}
/**
* The user's own camera, for the camera menu's preview.
*
* Attached to the same track the pipeline is synced to, so it shows whatever
* effect is in force — the processed picture where one has been chosen, and
* the plain one where none ever has — without a capture or a pipeline of its
* own. A second consumer of one track, not a second track.
*/
export const SelfPreview: FC<Props> = ({ track }) => {
const video = useRef<HTMLVideoElement>(null);
useEffect(() => {
const element = video.current;
if (!track || !element) return;
track.attach(element);
return (): void => {
track.detach(element);
};
}, [track]);
// Nothing to show, and saying so: an effect chosen now still applies when
// the camera comes back, so the grid below stays live.
if (!track) return <VideoCallOffIcon width={32} height={32} />;
return (
<video
ref={video}
muted
playsInline
className={
facingModeFromLocalTrack(track).facingMode === "user"
? styles.mirrored
: undefined
}
/>
);
};
+4 -5
View File
@@ -11,9 +11,7 @@ import { renderHook } from "@testing-library/react";
import { useMicrophoneLevel } from "./useMicrophoneLevel";
import { restoreAudioCapture, stubAudioCapture } from "../utils/test";
// The capture itself, and releasing it, belong to observeMicrophoneState$ and
// are covered in src/state/MicrophoneLevel.test.ts. What is left here is the
// bridging: when the hook watches, and what it reports before it has an answer.
// Capture and release are covered in MicrophoneLevel.test.ts; this covers the bridge.
describe("useMicrophoneLevel", () => {
afterEach(restoreAudioCapture);
@@ -36,7 +34,8 @@ describe("useMicrophoneLevel", () => {
rerender({ id: "mic2" });
// No level carried over: the meter reads the new device or nothing at all.
expect(result.current).toEqual({ type: "level", level: 0 });
// No level carried over from the previous device.
const state = result.current;
expect(state.type === "level" ? state.level$.value : state.type).toBe(0);
});
});
+4 -11
View File
@@ -11,17 +11,11 @@ import {
type MicrophoneState,
observeMicrophoneState$,
} from "../state/MicrophoneLevel";
import { constant } from "../state/Behavior";
const IDLE: MicrophoneState = { type: "level", level: 0 };
const IDLE: MicrophoneState = { type: "level", level$: constant(0) };
/**
* Reads the live input level of a microphone, while `active`.
*
* A bridge and nothing else: the capture, its lifetime and the maths belong to
* {@link observeMicrophoneState$}. Scoped to `active` so the device is held
* only while whatever shows the meter is on screen, rather than for the length
* of a call.
*/
/** The live level of a microphone, captured only while `active`. */
export function useMicrophoneLevel(
deviceId: string | undefined,
active: boolean,
@@ -30,8 +24,7 @@ export function useMicrophoneLevel(
useEffect(() => {
if (!active) return;
// Idle first, so a new device starts from nothing rather than from the
// level the previous one was reading.
// Idle first, so a new device doesn't start from the previous level.
setState(IDLE);
const subscription = observeMicrophoneState$(deviceId).subscribe(setState);
return (): void => subscription.unsubscribe();
+12 -10
View File
@@ -97,11 +97,19 @@ export interface ConfigOptions {
};
};
// Describes the LiveKit configuration to be used.
/**
* Describes the LiveKit configuration to be used.
*/
livekit?: {
// The link to the service that returns a livekit url and token to use it.
// This is a fallback link in case the homeserver in use does not advertise
// a livekit service url over the transports endpoint.
/**
* The link to the service that returns a livekit url and token to use it.
* This is a fallback link in case the homeserver in use does not advertise
* a livekit service url over the transports endpoint.
*
* @deprecated Homeservers should advertise available transports over the
* /_matrix/client/unstable/org.matrix.msc4143/rtc/transports endpoint
* instead.
*/
livekit_service_url: string;
};
@@ -207,12 +215,6 @@ export interface ConfigOptions {
* Take care when changing these options.
*/
matrix_rtc_session?: {
/**
* How long (in milliseconds) to wait before rotating end-to-end media encryption keys
* when someone leaves a call.
*/
wait_for_key_rotation_ms?: number;
/**
* How long we wait before retrying after a network error on any of the requests.
*/
Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

+187
View File
@@ -0,0 +1,187 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { act, render, screen } from "@testing-library/react";
import { BehaviorSubject } from "rxjs";
import { describe, expect, it, vi } from "vitest";
import { arrangeTiles, type Bounds } from "./CallLayout";
import { makeGridLayout } from "./GridLayout";
import { makeSpotlightLandscapeLayout } from "./SpotlightLandscapeLayout";
import { makeSpotlightPortraitLayout } from "./SpotlightPortraitLayout";
import { type DragCallback, type LayoutProps } from "./Grid";
import { constant } from "../state/Behavior";
import {
type Alignment,
type GridLayout,
type SpotlightLandscapeLayout,
type SpotlightPortraitLayout,
} from "../state/layout-types";
import { type UserMediaViewModel } from "../state/media/UserMediaViewModel";
import {
GridTileViewModel,
SpotlightTileViewModel,
type TileViewModel,
} from "../state/TileViewModel";
// Grid supplies these hooks to the layouts it renders; on their own the
// layouts have no grid to report to
vi.mock("./Grid", async (importOriginal) => ({
...(await importOriginal<Record<string, unknown>>()),
useUpdateLayout: (): void => {},
useVisibleTiles: (): void => {},
}));
describe("grid layout", () => {
it("sizes the tiles from the minimum bounds", () => {
const minBounds$ = new BehaviorSubject<Bounds>({ width: 800, height: 600 });
const { scrolling: Scrolling } = makeGridLayout({ minBounds$ });
render(<Scrolling model={gridModel()} Slot={Slot} />);
const layer = screen.getByTestId("slot-0").parentElement!;
expect(layer.style.width).toBe("800px");
expect(layer.style.getPropertyValue("--width")).toBe(
`${Math.floor(arrangeTiles(800, 600, 3).tileWidth)}px`,
);
act(() => minBounds$.next({ width: 400, height: 300 }));
expect(layer.style.width).toBe("400px");
expect(layer.style.getPropertyValue("--width")).toBe(
`${Math.floor(arrangeTiles(400, 300, 3).tileWidth)}px`,
);
});
it("moves the spotlight only when a drag reaches another corner", () => {
const model = gridModel();
const { fixed: Fixed } = makeGridLayout({ minBounds$: bounds });
render(<Fixed model={model} Slot={Slot} />);
const alignments: Alignment[] = [];
model.spotlightAlignment$.subscribe((a) => alignments.push(a));
const drag = drags.get("spotlight")!;
// Still in the bottom right corner
act(() => drag({ x: 0, y: 0, xRatio: 0.9, yRatio: 0.9 }));
// Over to the bottom left
act(() => drag({ x: 0, y: 0, xRatio: 0.1, yRatio: 0.9 }));
// And a little further, still bottom left
act(() => drag({ x: 0, y: 0, xRatio: 0.2, yRatio: 0.8 }));
// Up to the top left
act(() => drag({ x: 0, y: 0, xRatio: 0.2, yRatio: 0.1 }));
expect(alignments).toEqual([
{ inline: "end", block: "end" },
{ inline: "start", block: "end" },
{ inline: "start", block: "start" },
]);
const slot = screen.getByTestId("slot-spotlight");
expect(slot.getAttribute("data-inline-alignment")).toBe("start");
expect(slot.getAttribute("data-block-alignment")).toBe("start");
});
});
describe("spotlight landscape layout", () => {
it("puts the spotlight in the fixed layer and the grid in the scrolling one", () => {
const { fixed: Fixed, scrolling: Scrolling } = makeSpotlightLandscapeLayout(
{ minBounds$: bounds },
);
const model = landscapeModel();
const { unmount } = render(<Fixed model={model} Slot={Slot} />);
expect(screen.getByTestId("slot-spotlight")).toBeInTheDocument();
unmount();
render(<Scrolling model={model} Slot={Slot} />);
expect(slotIds()).toEqual(model.grid.map((tile) => tile.id));
});
});
describe("spotlight portrait layout", () => {
it("sizes the grid tiles from the minimum width", () => {
const minBounds$ = new BehaviorSubject<Bounds>({ width: 300, height: 700 });
const { scrolling: Scrolling } = makeSpotlightPortraitLayout({
minBounds$,
});
const model = portraitModel();
render(<Scrolling model={model} Slot={Slot} />);
expect(slotIds()).toEqual(model.grid.map((tile) => tile.id));
const layer = screen.getByTestId(`slot-${model.grid[0].id}`).parentElement!
.parentElement!;
expect(layer.style.getPropertyValue("--grid-tile-width")).toBe(
`${Math.floor(arrangeTiles(300, 300, 3).tileWidth)}px`,
);
act(() => minBounds$.next({ width: 500, height: 700 }));
expect(layer.style.getPropertyValue("--grid-tile-width")).toBe(
`${Math.floor(arrangeTiles(500, 500, 3).tileWidth)}px`,
);
});
});
const bounds = constant<Bounds>({ width: 800, height: 600 });
// The layouts only pass tile view models through to their slots
const media = {} as UserMediaViewModel;
const spotlight = new SpotlightTileViewModel(
constant([]),
constant(false),
constant("solid"),
);
const tiles = (): GridTileViewModel[] =>
Array.from({ length: 3 }, () => new GridTileViewModel(constant(media)));
function gridModel(): GridLayout {
return {
type: "grid",
spotlight,
grid: tiles(),
spotlightAlignment$: new BehaviorSubject<Alignment>({
inline: "end",
block: "end",
}),
setVisibleTiles: () => {},
};
}
function landscapeModel(): SpotlightLandscapeLayout {
return {
type: "spotlight-landscape",
spotlight,
grid: tiles(),
setVisibleTiles: () => {},
};
}
function portraitModel(): SpotlightPortraitLayout {
return {
type: "spotlight-portrait",
spotlight,
grid: tiles(),
setVisibleTiles: () => {},
};
}
// Slots record their drag callback so a test can drag them, and show the id
// of the tile they hold
const drags = new Map<string, DragCallback | undefined>();
const Slot: LayoutProps<unknown, TileViewModel, HTMLDivElement>["Slot"] = ({
id,
model,
onDrag,
...props
}) => {
drags.set(id, onDrag);
return (
<div data-testid={`slot-${id}`} {...props}>
{model instanceof GridTileViewModel ? model.id : "spotlight"}
</div>
);
};
function slotIds(): string[] {
return screen
.getAllByTestId(/^slot-/)
.map((slot) => slot.textContent ?? "")
.filter((id) => id !== "spotlight");
}
+2 -2
View File
@@ -5,11 +5,11 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type Observable } from "rxjs";
import { type ComponentType } from "react";
import { type LayoutProps } from "./Grid";
import { type TileViewModel } from "../state/TileViewModel";
import { type Behavior } from "../state/Behavior";
export interface Bounds {
width: number;
@@ -20,7 +20,7 @@ export interface CallLayoutInputs {
/**
* The minimum bounds of the layout area.
*/
minBounds$: Observable<Bounds>;
minBounds$: Behavior<Bounds>;
}
export interface CallLayoutOutputs<Model> {
+12 -18
View File
@@ -11,14 +11,12 @@ import {
useCallback,
useMemo,
} from "react";
import { distinctUntilChanged } from "rxjs";
import { useObservableEagerState } from "observable-hooks";
import { type GridLayout as GridLayoutModel } from "../state/layout-types.ts";
import styles from "./GridLayout.module.css";
import { useInitial } from "../useInitial";
import { type CallLayout, arrangeTiles } from "./CallLayout";
import { type DragCallback, useUpdateLayout, useVisibleTiles } from "./Grid";
import { useBehavior } from "../useBehavior";
interface GridCSSProperties extends CSSProperties {
"--gap": string;
@@ -39,22 +37,18 @@ export const makeGridLayout: CallLayout<GridLayoutModel> = ({
// lives
fixed: function GridLayoutFixed({ ref, model, Slot }): ReactNode {
useUpdateLayout();
const alignment = useObservableEagerState(
useInitial(() =>
model.spotlightAlignment$.pipe(
distinctUntilChanged(
(a1, a2) => a1.block === a2.block && a1.inline === a2.inline,
),
),
),
);
const alignment = useBehavior(model.spotlightAlignment$);
const onDragSpotlight: DragCallback = useCallback(
({ xRatio, yRatio }) =>
model.spotlightAlignment$.next({
block: yRatio < 0.5 ? "start" : "end",
inline: xRatio < 0.5 ? "start" : "end",
}),
({ xRatio, yRatio }) => {
const block = yRatio < 0.5 ? "start" : "end";
const inline = xRatio < 0.5 ? "start" : "end";
// A drag reports the same alignment on every move; only a change is
// worth a re-render
const current = model.spotlightAlignment$.value;
if (current.block !== block || current.inline !== inline)
model.spotlightAlignment$.next({ block, inline });
},
[model.spotlightAlignment$],
);
@@ -78,7 +72,7 @@ export const makeGridLayout: CallLayout<GridLayoutModel> = ({
scrolling: function GridLayout({ ref, model, Slot }): ReactNode {
useUpdateLayout();
useVisibleTiles(model.setVisibleTiles);
const { width, height: minHeight } = useObservableEagerState(minBounds$);
const { width, height: minHeight } = useBehavior(minBounds$);
const { gap, tileWidth, tileHeight } = useMemo(
() => arrangeTiles(width, minHeight, model.grid.length),
[width, minHeight, model.grid.length],
+1 -2
View File
@@ -7,7 +7,6 @@ Please see LICENSE in the repository root for full details.
*/
import { type ReactNode, useCallback, useMemo } from "react";
import { useObservableEagerState } from "observable-hooks";
import classNames from "classnames";
import { type OneOnOneDesktopLayout as OneOnOneDesktopLayoutModel } from "../state/layout-types.ts";
@@ -37,7 +36,7 @@ export const makeOneOnOneDesktopLayout: CallLayout<
Slot,
}): ReactNode {
useUpdateLayout();
const { width, height } = useObservableEagerState(minBounds$);
const { width, height } = useBehavior(minBounds$);
const pipAlignment = useBehavior(model.pipAlignment$);
const { tileWidth, tileHeight } = useMemo(
() => arrangeTiles(width, height, 1),
+2 -3
View File
@@ -6,7 +6,6 @@ Please see LICENSE in the repository root for full details.
*/
import { type FC, type ReactNode } from "react";
import { useObservableEagerState } from "observable-hooks";
import classNames from "classnames";
import { type CallLayout } from "./CallLayout";
@@ -33,7 +32,7 @@ export const makeSpotlightLandscapeLayout: CallLayout<
Slot,
}): ReactNode {
useUpdateLayout();
useObservableEagerState(minBounds$);
useBehavior(minBounds$);
return (
<div ref={ref} className={styles.layer}>
@@ -56,7 +55,7 @@ export const makeSpotlightLandscapeLayout: CallLayout<
}): ReactNode {
useUpdateLayout();
useVisibleTiles(model.setVisibleTiles);
useObservableEagerState(minBounds$);
useBehavior(minBounds$);
return (
<div ref={ref} className={styles.layer}>
+1 -2
View File
@@ -6,7 +6,6 @@ Please see LICENSE in the repository root for full details.
*/
import { type ReactNode, type CSSProperties } from "react";
import { useObservableEagerState } from "observable-hooks";
import classNames from "classnames";
import { type CallLayout, arrangeTiles } from "./CallLayout";
@@ -58,7 +57,7 @@ export const makeSpotlightPortraitLayout: CallLayout<
}): ReactNode {
useUpdateLayout();
useVisibleTiles(model.setVisibleTiles);
const { width } = useObservableEagerState(minBounds$);
const { width } = useBehavior(minBounds$);
const { gap, tileWidth, tileHeight } = arrangeTiles(
width,
// TODO: We pretend that the minimum height is the width, because the
+15 -1
View File
@@ -7,10 +7,24 @@ Please see LICENSE in the repository root for full details.
/* Styles for Element Call as a page of its own. The parts that apply wherever
Element Call is shown live in base.css; these are about owning the document,
and are not loaded when a host embeds Element Call as a component. */
and are not loaded when a host embeds Element Call as a component.
That includes Compound: the page brings the design tokens, the component styles
and their fonts, where a host embedding the component supplies its own. The
`compound` layers are ordered by base.css. */
@import url("./base.css");
@import url("@fontsource/inter/400.css");
@import url("@fontsource/inter/500.css");
@import url("@fontsource/inter/600.css");
@import url("@fontsource/inter/700.css");
@import url("@fontsource/inconsolata/400.css");
@import url("@fontsource/inconsolata/700.css");
@import url("@vector-im/compound-design-tokens/assets/web/css/compound-design-tokens.css") layer(compound);
@import url("@vector-im/compound-web/dist/style.css") layer(compound.components);
body {
background-color: var(--cpd-color-bg-canvas-default);
color: var(--cpd-color-text-primary);
-401
View File
@@ -1,401 +0,0 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC, useCallback, useRef, useState } from "react";
import {
createLocalVideoTrack,
type LocalVideoTrack,
VideoPresets,
} from "livekit-client";
import {
BackgroundProcessorWrapper,
supportsBackgroundProcessors as supportsBackgroundProcessorsLivekitSdk,
supportsModernBackgroundProcessors,
type SwitchBackgroundProcessorOptions,
} from "@livekit/track-processors";
import { logger } from "matrix-js-sdk/lib/logger";
import { BackgroundEffectTransformer } from "./BackgroundEffectTransformer";
import { blurRadius, imagePathFor } from "./backgroundEffects";
import { supportsBackgroundProcessors } from "./backgroundProcessing";
import { platform } from "../Platform";
/**
* Whether background effects can hold their frame rate on this device, asked
* without a call in the way.
*
* The camera and the pipeline only: no SFU, no encoder, no second participant.
* If a device cannot hold a frame rate here it cannot hold one in a call
* either, so this is the cheap half of the question — and it is the half that
* has never been asked. The gate that keeps effects off phones was added from
* reports, not measurements, and nothing in the app will run the pipeline
* there, so there has been nothing to measure.
*
* Deliberately ignores that gate: measuring what it forbids is the point. It
* reports what the gate would have said, and runs anyway.
*
* Exploration only, and never ported: an unlisted route, its text not
* translated, its numbers meant for whoever is holding the phone.
*/
type Effect = "none" | "blur" | "image";
interface Bucket {
/** Seconds since the first frame. */
at: number;
fps: number;
}
interface Run {
effect: Effect;
device: string;
platform: string;
/** Whether the browser has the API that avoids the canvas fallback. */
modernApi: boolean;
/** What the app's own verdict would have said about this device. */
appWouldAllow: boolean;
sdkWouldAllow: boolean;
capture: { width?: number; height?: number; frameRate?: number };
processed: { width?: number; height?: number; frameRate?: number };
/** Milliseconds from asking for the effect until the pipeline reports ready. */
startupMs: number | null;
/**
* Where that time went: attaching builds the segmenter — fetching its
* assets, compiling the WASM, setting up the GPU — while switching only
* tells an already-built one what to draw. Split, because the two have
* different answers: one can be paid ahead of time, the other cannot.
*/
attachMs: number | null;
switchMs: number | null;
/**
* The segmenter's own assets, as the browser saw them. A transfer size of
* zero means the bytes came from cache and no network was involved, which is
* what separates "the first time on this device" from "every time".
*/
assets: {
name: string;
transferredBytes: number;
bytes: number;
ms: number;
}[];
meanFps: number;
/** The worst five-second stretch — where throttling shows up first. */
worstFps: number;
buckets: Bucket[];
/** Worst observed delay on a 100ms timer: the main thread's own stutter. */
worstTimerLagMs: number;
/** The longest the picture simply stopped, in milliseconds. */
worstFrameGapMs: number;
/** From asking for the run to the first frame on screen. */
firstFrameMs: number | null;
frames: number;
seconds: number;
}
const bucketSeconds = 5;
function optionsFor(effect: Effect): SwitchBackgroundProcessorOptions {
switch (effect) {
case "blur":
return { mode: "background-blur", blurRadius };
case "image": {
const imagePath = imagePathFor("indoor");
return imagePath
? { mode: "virtual-background", imagePath }
: { mode: "disabled" };
}
default:
return { mode: "disabled" };
}
}
export const BackgroundEffectsBench: FC = () => {
const video = useRef<HTMLVideoElement | null>(null);
const [seconds, setSeconds] = useState(60);
const [busy, setBusy] = useState(false);
const [status, setStatus] = useState("Ready.");
const [runs, setRuns] = useState<Run[]>([]);
const measure = useCallback(
async (effect: Effect): Promise<void> => {
setBusy(true);
setStatus(`Starting the camera for "${effect}"…`);
let track: LocalVideoTrack | undefined;
try {
// The resolution and frame rate a real call captures at, so the cost
// measured here is the cost a call would pay.
track = await createLocalVideoTrack({
resolution: VideoPresets.h720.resolution,
});
const capture = track.mediaStreamTrack.getSettings();
let startupMs: number | null = null;
let attachMs: number | null = null;
let switchMs: number | null = null;
if (effect !== "none") {
setStatus("Starting the pipeline…");
const askedAt = performance.now();
const pipeline = new BackgroundProcessorWrapper(
new BackgroundEffectTransformer({ backgroundDisabled: true }),
"background-effect-bench",
);
await track.setProcessor(pipeline);
const attachedAt = performance.now();
await pipeline.switchTo(optionsFor(effect));
const readyAt = performance.now();
attachMs = Math.round(attachedAt - askedAt);
switchMs = Math.round(readyAt - attachedAt);
startupMs = Math.round(readyAt - askedAt);
}
const element = video.current!;
track.attach(element);
await element.play().catch(() => undefined);
setStatus(`Measuring "${effect}" for ${seconds}s…`);
const result = await countFrames(element, seconds, (done) =>
setStatus(`Measuring "${effect}": ${done}s of ${seconds}s`),
);
const processed = track.mediaStreamTrack.getSettings();
setRuns((previous) => [
...previous,
{
effect,
device: navigator.userAgent,
platform,
modernApi: supportsModernBackgroundProcessors(),
appWouldAllow: supportsBackgroundProcessors(),
sdkWouldAllow: supportsBackgroundProcessorsLivekitSdk(),
capture: {
width: capture.width,
height: capture.height,
frameRate: capture.frameRate,
},
processed: {
width: processed.width,
height: processed.height,
frameRate: processed.frameRate,
},
startupMs,
attachMs,
switchMs,
assets: segmenterAssets(),
...result,
},
]);
setStatus(`Done: ${effect}.`);
} catch (e) {
logger.error("Background effects bench failed", e);
setStatus(`Failed: ${e instanceof Error ? e.message : String(e)}`);
} finally {
track?.stop();
setBusy(false);
}
},
[seconds],
);
return (
<div style={{ padding: 16, font: "14px system-ui", lineHeight: 1.5 }}>
<h1 style={{ fontSize: 20 }}>Background effects: can this device?</h1>
<p>
The camera and the pipeline, with no call in the way. Run{" "}
<strong>No effect</strong> first — every other number is only meaningful
beside it, on this device, in this light.
</p>
<p>
This device: <code>{platform}</code>, the app would{" "}
<strong>{supportsBackgroundProcessors() ? "allow" : "refuse"}</strong>{" "}
effects here, the browser has{" "}
<strong>
{supportsModernBackgroundProcessors()
? "the fast path"
: "only the canvas fallback"}
</strong>
.
</p>
<label>
Seconds per run:{" "}
<select
value={seconds}
onChange={(e) => setSeconds(Number(e.target.value))}
disabled={busy}
>
<option value={30}>30</option>
<option value={60}>60</option>
<option value={300}>300 (five minutes, for throttling)</option>
<option value={600}>600 (ten minutes)</option>
</select>
</label>
<div style={{ display: "flex", gap: 8, margin: "12px 0" }}>
{(["none", "blur", "image"] as const).map((effect) => (
<button
key={effect}
disabled={busy}
onClick={() => void measure(effect)}
style={{ padding: "8px 12px" }}
>
{effect === "none" ? "No effect" : effect}
</button>
))}
</div>
<p>
<strong>{status}</strong>
</p>
<video
ref={video}
muted
playsInline
style={{ inlineSize: "100%", maxInlineSize: 320, background: "#000" }}
/>
{runs.map((run, i) => (
<div key={i} style={{ marginBlockStart: 12 }}>
<strong>{run.effect}</strong>: mean {run.meanFps.toFixed(1)}fps, worst{" "}
{run.worstFps.toFixed(1)}fps
{run.startupMs !== null && (
<>
, ready in {run.startupMs}ms ({run.attachMs}ms attaching,{" "}
{run.switchMs}ms switching)
</>
)}
, first frame at {run.firstFrameMs ?? "?"}ms, picture stopped for up
to {run.worstFrameGapMs}ms, main thread stalled up to{" "}
{run.worstTimerLagMs}ms
</div>
))}
{runs.length > 0 && (
<textarea
readOnly
value={JSON.stringify(runs, null, 2)}
style={{ inlineSize: "100%", blockSize: 240, marginBlockStart: 12 }}
/>
)}
</div>
);
};
/**
* What the browser did to fetch the segmenter: bytes over the wire, bytes
* total, and how long each took. Same-origin, so the sizes are not hidden.
*/
function segmenterAssets(): Run["assets"] {
return performance
.getEntriesByType("resource")
.filter((e): e is PerformanceResourceTiming =>
/\.(wasm|tflite|task|binarypb)(\?|$)/.test(e.name),
)
.map((e) => ({
name: e.name.slice(e.name.lastIndexOf("/") + 1),
transferredBytes: e.transferSize,
bytes: e.encodedBodySize,
ms: Math.round(e.duration),
}));
}
/**
* Counts frames as the video actually presents them, which is the rate the
* pipeline is really delivering rather than the rate it was asked for.
*/
async function countFrames(
element: HTMLVideoElement,
seconds: number,
onProgress: (done: number) => void,
): Promise<
Pick<
Run,
| "meanFps"
| "worstFps"
| "buckets"
| "worstTimerLagMs"
| "worstFrameGapMs"
| "firstFrameMs"
| "frames"
| "seconds"
>
> {
return new Promise((resolve) => {
const buckets: Bucket[] = [];
const startedAt = performance.now();
let frames = 0;
let inBucket = 0;
let firstFrameMs: number | null = null;
let lastFrameAt = startedAt;
let worstFrameGapMs = 0;
// A phone that is struggling stalls its main thread, and a timer is the
// cheapest witness to that: Safari has no long-task observer.
let worstTimerLagMs = 0;
let lastTick = performance.now();
const ticker = window.setInterval(() => {
const now = performance.now();
worstTimerLagMs = Math.max(
worstTimerLagMs,
Math.round(now - lastTick - 100),
);
lastTick = now;
}, 100);
// On the clock, not on the frames. Closing a bucket only when a frame
// arrives cannot report a stretch with no frames in it at all — the first
// measurement on a phone spent fourteen seconds inside one bucket and
// reported it as a seventh of a frame per second rather than as a freeze.
const bucketer = window.setInterval(() => {
const at = Math.round((performance.now() - startedAt) / 1000);
buckets.push({ at, fps: inBucket / bucketSeconds });
inBucket = 0;
onProgress(at);
}, bucketSeconds * 1000);
const finish = (): void => {
window.clearInterval(ticker);
window.clearInterval(bucketer);
const elapsed = (performance.now() - startedAt) / 1000;
resolve({
meanFps: elapsed > 0 ? frames / elapsed : 0,
worstFps: buckets.length ? Math.min(...buckets.map((b) => b.fps)) : 0,
buckets,
worstTimerLagMs,
worstFrameGapMs: Math.round(worstFrameGapMs),
firstFrameMs: firstFrameMs === null ? null : Math.round(firstFrameMs),
frames,
seconds: Math.round(elapsed),
});
};
const onFrame = (): void => {
const now = performance.now();
firstFrameMs ??= now - startedAt;
worstFrameGapMs = Math.max(worstFrameGapMs, now - lastFrameAt);
lastFrameAt = now;
frames += 1;
inBucket += 1;
if (now - startedAt >= seconds * 1000) return finish();
element.requestVideoFrameCallback(onFrame);
};
if (!("requestVideoFrameCallback" in element)) {
window.clearInterval(ticker);
window.clearInterval(bucketer);
resolve({
meanFps: 0,
worstFps: 0,
buckets: [],
worstTimerLagMs: 0,
worstFrameGapMs: 0,
firstFrameMs: null,
frames: 0,
seconds: 0,
});
return;
}
element.requestVideoFrameCallback(onFrame);
// A freeze can outlast the run: without this, a device that never presents
// another frame never finishes and reports nothing at all.
window.setTimeout(finish, seconds * 1000 + 2000);
});
}
@@ -43,40 +43,13 @@ const wasmFileset: WasmFileset = {
};
/**
* Track processor that applies background effects — blur, or a replacement
* image — to a user's camera.
* Track processor that applies effects such as blurring to a user's background.
*
* This is just like LiveKit's prebuilt BackgroundTransformer except that it
* loads the segmentation models from our own bundle rather than as an external
* resource fetched from the public internet. The library's own `assetPaths`
* option cannot do this: it takes a directory and resolves the WASM through
* MediaPipe's FilesetResolver, which picks its own variants, whereas we want
* only the SIMD ones.
* resource fetched from the public internet.
*/
export class BackgroundEffectTransformer extends BackgroundTransformer {
/**
* Called once, when a frame has actually come out of the pipeline.
*
* The promises say nothing useful about when that happens. Measured on four
* devices, attaching resolves in about three seconds and switching in none
* at all, and then a browser on the slow path spends another twelve to
* fifteen seconds before the first frame appears. Only the frame itself
* marks the end of the wait.
*/
public onFirstFrame: (() => void) | undefined;
private produced = false;
public override async transform(
frame: VideoFrame,
controller: TransformStreamDefaultController<VideoFrame>,
): Promise<void> {
await super.transform(frame, controller);
if (!this.produced) {
this.produced = true;
this.onFirstFrame?.();
}
}
export class BlurBackgroundTransformer extends BackgroundTransformer {
public async init({
outputCanvas,
inputElement: inputVideo,
@@ -100,16 +73,8 @@ export class BackgroundEffectTransformer extends BackgroundTransformer {
outputConfidenceMasks: false,
});
// BackgroundTransformer applies these at the end of its own init. Because
// we replace init wholesale rather than extending it, we have to repeat
// them, or an effect that was already selected when the pipeline starts is
// silently ignored.
if (this.options.imagePath) {
await this.loadAndSetBackground(this.options.imagePath);
}
if (typeof this.options.blurRadius === "number") {
if (this.options.blurRadius) {
this.gl?.setBlurRadius(this.options.blurRadius);
}
this.gl?.setBackgroundDisabled(this.options.backgroundDisabled ?? false);
}
}
+4 -3
View File
@@ -24,6 +24,7 @@ import { testAudioContext } from "../useAudioContext.test";
import * as MediaDevicesContext from "../MediaDevicesContext";
import { LivekitRoomAudioRenderer } from "./MatrixAudioRenderer";
import {
exampleTransportLocator,
mockLocalParticipant,
mockMediaDevices,
mockRemoteParticipant,
@@ -118,7 +119,7 @@ function renderTestComponent(
<LivekitRoomAudioRenderer
validIdentities={participants.map((p) => p.identity)}
livekitRoom={livekitRoom}
url={""}
transport={exampleTransportLocator}
/>
</MediaDevicesProvider>,
);
@@ -150,7 +151,7 @@ it("should not render or warn for the local participant", () => {
<LivekitRoomAudioRenderer
validIdentities={[]}
livekitRoom={{ remoteParticipants: new Map() } as unknown as Room}
url={""}
transport={exampleTransportLocator}
/>
</MediaDevicesProvider>,
);
@@ -177,7 +178,7 @@ it("should warn only once per unexpected participant", () => {
<LivekitRoomAudioRenderer
validIdentities={[]}
livekitRoom={{ remoteParticipants: new Map() } as unknown as Room}
url={""}
transport={exampleTransportLocator}
/>
</MediaDevicesProvider>,
);
+4 -6
View File
@@ -19,12 +19,10 @@ import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { useEarpieceAudioConfig } from "../MediaDevicesContext";
import { useReactiveState } from "../useReactiveState";
import * as controls from "../controls";
import { type TransportLocator } from "./auth";
export interface MatrixAudioRendererProps {
/**
* The service URL of the LiveKit room.
*/
url: string;
transport: TransportLocator;
livekitRoom: LivekitRoom;
/**
* The list of participant identities to render audio for.
@@ -54,7 +52,7 @@ export interface MatrixAudioRendererProps {
* @public
*/
export function LivekitRoomAudioRenderer({
url,
transport,
livekitRoom,
validIdentities,
muted,
@@ -88,7 +86,7 @@ export function LivekitRoomAudioRenderer({
warnedIdentities.current.add(identity);
// Log that there is an invalid identity, that means that someone is publishing audio that is not expected to be in the call.
logger.warn(
`Audio track ${identity} from ${url} has no matching matrix call member`,
`Audio track ${identity} from ${JSON.stringify(transport)} has no matching matrix call member`,
`current members: ${validIdentities.join()}`,
`track will not get rendered`,
);
+22 -273
View File
@@ -6,122 +6,41 @@ Please see LICENSE in the repository root for full details.
*/
import {
BackgroundProcessorWrapper,
type ProcessorWrapper,
ProcessorWrapper,
supportsBackgroundProcessors as supportsBackgroundProcessorsLivekitSdk,
type BackgroundOptions,
type SwitchBackgroundProcessorOptions,
} from "@livekit/track-processors";
import {
createContext,
type FC,
type JSX,
use,
useCallback,
useEffect,
useMemo,
useRef,
useState,
} from "react";
import { type LocalVideoTrack } from "livekit-client";
import { logger } from "matrix-js-sdk/lib/logger";
import { BehaviorSubject, combineLatest, map, type Observable } from "rxjs";
import { useObservable, useObservableEagerState } from "observable-hooks";
import { combineLatest } from "rxjs";
import {
backgroundEffect as backgroundEffectSetting,
backgroundBlur as backgroundBlurSettings,
useSetting,
} from "../settings/settings";
import { BackgroundEffectTransformer } from "./BackgroundEffectTransformer";
import {
blurRadius,
imagePathFor,
parseEffect,
type BackgroundEffect,
} from "./backgroundEffects";
import { BackgroundImageStore } from "./backgroundImages";
import { supportsBackgroundProcessors } from "./backgroundProcessing";
import { BlurBackgroundTransformer } from "./BlurBackgroundTransformer";
import { type Behavior } from "../state/Behavior";
import { type ObservableScope } from "../state/ObservableScope";
import { platform } from "../Platform";
import { useValueBehavior } from "../useValueBehavior";
//TODO-MULTI-SFU: This is not yet fully there.
// it is a combination of exposing observable and react hooks.
// preferably we should not make this a context anymore and instead just a vm?
/** A background the user added, as the view needs it: an id and a picture. */
export interface AddedBackgroundImage {
id: string;
/** Lives as long as the provider does. */
url: string;
}
/**
* What the publishing side needs: whether effects can run at all, and the
* pipeline to attach. Deliberately not the backgrounds a user has added — the
* publisher has no use for those, and every test that stands in for this would
* have to carry them.
*/
export type ProcessorState = {
supported: boolean | undefined;
processor: undefined | ProcessorWrapper<BackgroundOptions>;
};
/** What the camera menu needs: the backgrounds this device keeps. */
export interface AddedBackgrounds {
/** Oldest first. */
added: AddedBackgroundImage[];
/**
* Keeps a file as a background. Rejects with `UnusableImage` for a file that
* cannot be used, and `RangeError` once the device keeps as many as it will.
*/
addBackground: (file: Blob) => Promise<string>;
removeBackground: (id: string) => Promise<void>;
}
/** What the camera menu needs to know about the pipeline itself. */
export interface BackgroundProcessing {
/**
* Whether the pipeline is still being built. Only ever true once a session,
* and only the first time an effect is chosen: building it fetches, compiles
* and sets up the segmenter, which on some browsers holds the page still for
* long enough that saying nothing looks like a crash.
*/
settling: boolean;
/**
* The camera track the pipeline is synced to: the pre-join preview's before
* joining, the published one after, and none while the camera is off. It
* draws with whatever effect is in force, and plainly when none has ever
* been chosen, so the menu can show someone how they look without building
* anything of its own.
*/
cameraTrack: LocalVideoTrack | null;
}
const BackgroundProcessingContext = createContext<
BackgroundProcessing | undefined
>(undefined);
export function useBackgroundProcessing(): BackgroundProcessing {
const value = use(BackgroundProcessingContext);
if (value === undefined)
throw new Error(
"useBackgroundProcessing must be used within a ProcessorProvider",
);
return value;
}
const AddedBackgroundsContext = createContext<AddedBackgrounds | undefined>(
undefined,
);
export function useAddedBackgrounds(): AddedBackgrounds {
const value = use(AddedBackgroundsContext);
if (value === undefined)
throw new Error(
"useAddedBackgrounds must be used within a ProcessorProvider",
);
return value;
}
const ProcessorContext = createContext<ProcessorState | undefined>(undefined);
export function useTrackProcessor(): ProcessorState {
@@ -133,18 +52,8 @@ export function useTrackProcessor(): ProcessorState {
return state;
}
export function useTrackProcessorObservable$(): Observable<ProcessorState> {
const state = use(ProcessorContext);
if (state === undefined)
throw new Error(
"useTrackProcessor must be used within a ProcessorProvider",
);
const state$ = useObservable(
(init$) => init$.pipe(map(([init]) => init)),
[state],
);
return state$;
export function useTrackProcessorState$(): Behavior<ProcessorState> {
return useValueBehavior(useTrackProcessor());
}
/**
@@ -178,27 +87,6 @@ export function applyProcessor(
/**
* Updates your video tracks to always use the given processor.
*/
/**
* The camera track the pipeline was last synced to.
*
* Module state rather than context, because one of the two paths that sync a
* track is not React at all: in a call the publisher does it, from an
* observable. There is one camera and one pipeline for the app's lifetime, so
* there is one track to know about; each path reports its own, and clears it
* only if it is still the one reported, so a lobby leaving cannot blank a call
* that has just started.
*/
const syncedCameraTrack$ = new BehaviorSubject<LocalVideoTrack | null>(null);
function reportCameraTrack(track: LocalVideoTrack | null): void {
if (syncedCameraTrack$.value !== track) syncedCameraTrack$.next(track);
}
function withdrawCameraTrack(track: LocalVideoTrack | null): void {
if (track !== null && syncedCameraTrack$.value === track)
syncedCameraTrack$.next(null);
}
export const trackProcessorSync = (
scope: ObservableScope,
videoTrack$: Behavior<LocalVideoTrack | null>,
@@ -211,15 +99,6 @@ export const trackProcessorSync = (
if (!videoTrack) return;
applyProcessor(videoTrack, processorState.processor);
});
let reported: LocalVideoTrack | null = null;
videoTrack$.pipe(scope.bind()).subscribe({
next: (videoTrack) => {
withdrawCameraTrack(reported);
reported = videoTrack;
reportCameraTrack(videoTrack);
},
complete: () => withdrawCameraTrack(reported),
});
};
export const useTrackProcessorSync = (
@@ -230,167 +109,37 @@ export const useTrackProcessorSync = (
if (!videoTrack) return;
applyProcessor(videoTrack, processor);
}, [processor, videoTrack]);
useEffect(() => {
reportCameraTrack(videoTrack);
return (): void => withdrawCameraTrack(videoTrack);
}, [videoTrack]);
};
interface Props {
children: JSX.Element;
}
/** Translates a chosen effect into the pipeline's own vocabulary. */
function switchOptionsFor(
effect: BackgroundEffect,
added: AddedBackgroundImage[],
): SwitchBackgroundProcessorOptions {
const withImage = (
imagePath: string | undefined,
): SwitchBackgroundProcessorOptions =>
// A background the device no longer has — removed, or storage cleared —
// leaves the user with no effect rather than a pipeline drawing nothing.
imagePath
? { mode: "virtual-background", imagePath }
: { mode: "disabled" };
switch (effect.kind) {
case "blur":
return { mode: "background-blur", blurRadius };
case "shipped":
return withImage(imagePathFor(effect.id));
case "added":
return withImage(added.find((a) => a.id === effect.id)?.url);
default:
return { mode: "disabled" };
}
function supportsBackgroundProcessors(): boolean {
return supportsBackgroundProcessorsLivekitSdk() && platform === "desktop";
}
export const ProcessorProvider: FC<Props> = ({ children }) => {
const [effectRaw] = useSetting(backgroundEffectSetting);
// The setting the user wants to have
const [blurActivated] = useSetting(backgroundBlurSettings);
const supported = useMemo(() => supportsBackgroundProcessors(), []);
// One pipeline for the lifetime of the app, so the pre-join preview and the
// call share it and its priming frame is spent before anything is published
// (D4).
const transformer = useMemo(
() => new BackgroundEffectTransformer({ backgroundDisabled: true }),
const blur = useMemo(
() =>
new ProcessorWrapper(
new BlurBackgroundTransformer({ blurRadius: 15 }),
"background-blur",
),
[],
);
const pipeline = useMemo(
() => new BackgroundProcessorWrapper(transformer, "background-effect"),
[transformer],
);
// The backgrounds this device keeps. Their URLs live as long as the provider
// does, and are replaced wholesale whenever the set changes: an object URL
// outlives the blob it names unless it is revoked, and the alternative is
// tracking one lifetime per image.
const store = useMemo(() => new BackgroundImageStore(), []);
const [added, setAdded] = useState<AddedBackgroundImage[]>([]);
const urls = useRef<string[]>([]);
const reread = useCallback(async (): Promise<void> => {
const kept = await store.list();
urls.current.forEach((url) => URL.revokeObjectURL(url));
urls.current = kept.map((background) =>
URL.createObjectURL(background.image),
);
setAdded(kept.map(({ id }, i) => ({ id, url: urls.current[i] })));
}, [store]);
useEffect(() => {
reread().catch((e) => logger.warn("Could not read added backgrounds", e));
const opened = urls;
return (): void => {
opened.current.forEach((url) => URL.revokeObjectURL(url));
opened.current = [];
};
}, [reread]);
const addBackground = useCallback(
async (file: Blob): Promise<string> => {
const kept = await store.add(file);
await reread();
return kept.id;
},
[store, reread],
);
const removeBackground = useCallback(
async (id: string): Promise<void> => {
await store.remove(id);
await reread();
},
[store, reread],
);
// D5: nothing is attached until an effect is first chosen, so a user who
// never chooses one pays neither the segmentation assets nor the time to
// initialise them. Once attached it stays attached, including at no effect,
// so a later change is a switch rather than a fresh attachment and spends no
// further priming frame.
const [attached, setAttached] = useState(
() => parseEffect(effectRaw).kind !== "none",
);
useEffect(() => {
if (parseEffect(effectRaw).kind !== "none") setAttached(true);
}, [effectRaw]);
// D2: switch the running pipeline in place rather than tearing it down, so
// the previous effect stays in force until the new one is live.
const options = useMemo(
() => switchOptionsFor(parseEffect(effectRaw), added),
[effectRaw, added],
);
useEffect(() => {
if (!supported || !attached) return;
pipeline
.switchTo(options)
.catch((e) => logger.warn("Failed to switch background effect", e));
}, [pipeline, supported, attached, options]);
// The wait runs from deciding to attach until a frame actually comes out,
// which is the only thing that marks the end of it: the promises resolve
// while the segmenter is still being built, and on the slow path the page
// then holds still for another twelve to fifteen seconds. Bounded by the
// frame rather than by a timer, so it cannot end early and claim to be ready.
const [producedAFrame, setProducedAFrame] = useState(false);
useEffect(() => {
transformer.onFirstFrame = (): void => setProducedAFrame(true);
return (): void => {
transformer.onFirstFrame = undefined;
};
}, [transformer]);
const settling = supported === true && attached && !producedAFrame;
// This is the actual state exposed through the context
const processorState = useMemo(
() => ({
supported,
processor: supported && attached ? pipeline : undefined,
processor: supported && blurActivated ? blur : undefined,
}),
[supported, attached, pipeline],
[supported, blurActivated, blur],
);
const addedBackgrounds = useMemo(
() => ({ added, addBackground, removeBackground }),
[added, addBackground, removeBackground],
);
const cameraTrack = useObservableEagerState(syncedCameraTrack$);
const backgroundProcessing = useMemo(
() => ({ settling, cameraTrack }),
[settling, cameraTrack],
);
return (
<ProcessorContext value={processorState}>
<AddedBackgroundsContext value={addedBackgrounds}>
<BackgroundProcessingContext value={backgroundProcessing}>
{children}
</BackgroundProcessingContext>
</AddedBackgroundsContext>
</ProcessorContext>
);
return <ProcessorContext value={processorState}>{children}</ProcessorContext>;
};
+137
View File
@@ -0,0 +1,137 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { ConnectionError, HTTPError, type MatrixClient } from "matrix-js-sdk";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { extractFullConfigFromToken, type SFUConfig } from "./types.ts";
import { doNetworkOperationWithRetry } from "../../utils/matrix.ts";
/**
* In widget mode the host may refuse an action before any request is made
* which surfaces as a plain error. Asking again cannot change its answer,
* so only errors from the network are worth retrying.
*/
function isNetworkError(e: unknown): boolean {
return e instanceof HTTPError || e instanceof ConnectionError;
}
export type ClientGetTokenParts = Pick<
MatrixClient,
"_unstable_getLivekitToken"
>;
export interface GetSFUConfigParams {
/**
* The Matrix client.
*/
client: ClientGetTokenParts;
/**
* Data identifying the local user's session membership.
*/
membership: CallMembershipIdentityParts;
/**
* The WebSocket URL of the SFU for which we wish to get an access token.
*/
url: string;
/**
* The name of the homeserver to which the SFU belongs.
*/
serverName: string;
/**
* The ID of the Matrix room in which the session takes place.
*/
roomId: string;
/**
* The ID of the MatrixRTC slot in which the session takes place.
*/
slotId: string;
}
/**
* Gets an {@link SFUConfig} appropriate for connecting to a given SFU, using
* the MSC4195 LiveKit client-server API endpoints.
*/
export async function getSFUConfig({
client,
membership,
url,
serverName,
roomId,
slotId,
}: GetSFUConfigParams): Promise<SFUConfig> {
const res = await doNetworkOperationWithRetry(
async () =>
client._unstable_getLivekitToken({
url,
server_name: serverName,
room_id: roomId,
slot_id: slotId,
member_id: membership.memberId,
}),
isNetworkError,
);
return extractFullConfigFromToken({ url, jwt: res.jwt });
}
export type ClientDelegationParts = Pick<
MatrixClient,
"_unstable_delegateDelayedLeave" | "baseUrl"
>;
export interface DelegateDelayedLeaveParams {
/**
* The Matrix client.
*/
client: ClientDelegationParts;
/**
* Data identifying the local user's session membership.
*/
membership: CallMembershipIdentityParts;
/**
* The WebSocket URL of the SFU to which we are publishing and wish to
* delegate the delayed leave event.
*/
url: string;
/**
* The ID of the Matrix room in which the session takes place.
*/
roomId: string;
/**
* The ID of the MatrixRTC slot in which the session takes place.
*/
slotId: string;
/**
* The delay ID of the leave event to be delegated.
*/
delayId: string;
}
/**
* Delegates a delayed leave event to a given SFU, so that the event will
* be sent on our behalf whenever we disconnect from the SFU.
*/
export async function delegateDelayedLeave({
client,
membership,
url,
roomId,
slotId,
delayId,
}: DelegateDelayedLeaveParams): Promise<void> {
await doNetworkOperationWithRetry(
async () =>
client._unstable_delegateDelayedLeave({
url,
room_id: roomId,
slot_id: slotId,
member_id: membership.memberId,
delay_id: delayId,
}),
isNetworkError,
);
}
+306
View File
@@ -0,0 +1,306 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, expect, type Mocked, vi, describe, it } from "vitest";
import { type UnstableLivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { logger } from "matrix-js-sdk/lib/logger";
import {
aliceDeviceId,
aliceId,
aliceUserId,
createJWTToken,
} from "../../utils/test-fixtures";
import type * as csApi from "./csApi";
import * as openID from "./openID";
import { delegateDelayedLeave, getSFUConfig } from ".";
import { mocked } from "storybook/test";
import { ConnectionError, MatrixError } from "matrix-js-sdk";
import { NoMatrix2AuthorizationService } from "../../utils/errors";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
vi.mock("./openID", () => ({ getSFUConfig: vi.fn() }));
vi.useFakeTimers();
const csApiJWTToken = createJWTToken(aliceId, "via CS API");
const openIDJWTToken = createJWTToken(aliceId, "via OpenID");
mocked(openID).getSFUConfig.mockResolvedValue({
url: "mock-openID-url",
jwt: openIDJWTToken,
livekitAlias: "via OpenID",
livekitIdentity: aliceId,
});
const client: Mocked<
csApi.ClientGetTokenParts &
csApi.ClientDelegationParts &
openID.ClientOpenIDParts
> = {
baseUrl: "https://matrix.example.org",
_unstable_getLivekitToken: vi.fn().mockResolvedValue({ jwt: csApiJWTToken }),
_unstable_delegateDelayedLeave: vi.fn().mockResolvedValue({}),
getOpenIdToken: vi
.fn()
.mockResolvedValue({ access_token: "mock-openID-token" }),
};
const membership: CallMembershipIdentityParts = {
userId: aliceUserId,
deviceId: aliceDeviceId,
memberId: "123",
};
const roomId = "!room:example.org";
const slotId = "m.call#ROOM";
const unsupportedError = new MatrixError({ errcode: "M_UNSUPPORTED" }, 404);
const urlTransport: UnstableLivekitTransport = {
type: "livekit",
url: "wss://livekit.example.org",
};
const serviceUrlTransport: UnstableLivekitTransport = {
type: "livekit",
livekit_service_url: "https://livekit-jwt.example.org",
};
const urlAndServiceUrlTransport: UnstableLivekitTransport = {
...urlTransport,
...serviceUrlTransport,
};
afterEach(() => vi.clearAllMocks());
describe("getSFUConfig", () => {
const defaultParams = {
client,
membership,
serverName: "example.org",
role: "subscriber" as const,
roomId,
slotId,
logger,
};
it("uses CS API with url transport", async () => {
expect(
await getSFUConfig({ ...defaultParams, transport: urlTransport }),
).toEqual(
expect.objectContaining({
jwt: csApiJWTToken,
livekitAlias: "via CS API",
}),
);
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
it("does not fall back to OpenID with url transport", async () => {
client._unstable_getLivekitToken.mockRejectedValueOnce(unsupportedError);
await expect(
getSFUConfig({ ...defaultParams, transport: urlTransport }),
).rejects.toThrow(expect.any(NoMatrix2AuthorizationService));
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
it("uses OpenID with livekit_service_url transport", async () => {
expect(
await getSFUConfig({ ...defaultParams, transport: serviceUrlTransport }),
).toEqual(
expect.objectContaining({
jwt: openIDJWTToken,
livekitAlias: "via OpenID",
}),
);
expect(client._unstable_getLivekitToken).not.toHaveBeenCalled();
expect(openID.getSFUConfig).toHaveBeenCalled();
});
it("uses CS API with url + livekit_service_url transport", async () => {
expect(
await getSFUConfig({
...defaultParams,
transport: urlAndServiceUrlTransport,
}),
).toEqual(
expect.objectContaining({
jwt: csApiJWTToken,
livekitAlias: "via CS API",
}),
);
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
it("falls back to OpenID with url + livekit_service_url transport", async () => {
client._unstable_getLivekitToken.mockRejectedValueOnce(unsupportedError);
expect(
await getSFUConfig({
...defaultParams,
transport: urlAndServiceUrlTransport,
}),
).toEqual(
expect.objectContaining({
jwt: openIDJWTToken,
livekitAlias: "via OpenID",
}),
);
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(openID.getSFUConfig).toHaveBeenCalled();
});
it("falls back to OpenID when the widget host refuses the CS API", async () => {
// In widget mode a host without MSC4533 rejects the request before it
// reaches the homeserver, with a plain error rather than a Matrix error
client._unstable_getLivekitToken.mockRejectedValueOnce(
new Error(
"The rtc_livekit_get_token action is not supported by the client.",
),
);
expect(
await getSFUConfig({
...defaultParams,
transport: urlAndServiceUrlTransport,
}),
).toEqual(expect.objectContaining({ jwt: openIDJWTToken }));
expect(openID.getSFUConfig).toHaveBeenCalled();
});
it("surfaces the widget host's refusal with url transport", async () => {
const error = new Error(
"The rtc_livekit_get_token action is not supported by the client.",
);
client._unstable_getLivekitToken.mockRejectedValueOnce(error);
await expect(
getSFUConfig({ ...defaultParams, transport: urlTransport }),
).rejects.toThrow(error);
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
it("uses CS API when publishing with url transport", async () => {
expect(
await getSFUConfig({
...defaultParams,
transport: urlTransport,
role: "publisher",
}),
).toEqual(expect.objectContaining({ jwt: csApiJWTToken }));
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
it("does not fall back to OpenID when publishing", async () => {
client._unstable_getLivekitToken.mockRejectedValueOnce(unsupportedError);
await expect(
getSFUConfig({
...defaultParams,
transport: urlTransport,
role: "publisher",
}),
).rejects.toThrow(expect.any(NoMatrix2AuthorizationService));
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
it("uses OpenID when publishing with url + livekit_service_url transport", async () => {
// Subscribers predating MSC4195 expect a legacy participant identity from
// anyone advertising a state event, which only the JWT service assigns
expect(
await getSFUConfig({
...defaultParams,
transport: urlAndServiceUrlTransport,
role: "publisher",
}),
).toEqual(
expect.objectContaining({
jwt: openIDJWTToken,
livekitAlias: "via OpenID",
}),
);
expect(client._unstable_getLivekitToken).not.toHaveBeenCalled();
expect(openID.getSFUConfig).toHaveBeenCalledWith(
expect.objectContaining({ role: "publisher" }),
);
});
it("does not fall back to OpenID on connection error", async () => {
const client_ = {
...client,
_unstable_getLivekitToken: vi
.fn()
.mockRejectedValue(new ConnectionError("No connection")),
};
await Promise.all([
expect(
getSFUConfig({
...defaultParams,
client: client_,
transport: urlAndServiceUrlTransport,
}),
).rejects.toThrow(expect.any(ConnectionError)),
vi.runAllTimersAsync(), // Allow request to be retried
]);
expect(client_._unstable_getLivekitToken).toHaveBeenCalled();
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
});
describe("delegateDelayedLeave", () => {
const defaultParams = {
client,
membership,
roomId,
slotId,
delayId: "delay123",
logger,
};
it("uses CS API with url transport", async () => {
await delegateDelayedLeave({ ...defaultParams, transport: urlTransport });
expect(client._unstable_delegateDelayedLeave).toHaveBeenCalled();
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
it("does not fall back to OpenID with url transport", async () => {
client._unstable_delegateDelayedLeave.mockRejectedValueOnce(
unsupportedError,
);
await expect(
delegateDelayedLeave({ ...defaultParams, transport: urlTransport }),
).rejects.toThrow();
expect(client._unstable_delegateDelayedLeave).toHaveBeenCalled();
expect(openID.getSFUConfig).not.toHaveBeenCalled();
});
it("uses OpenID with livekit_service_url transport", async () => {
await delegateDelayedLeave({
...defaultParams,
transport: serviceUrlTransport,
});
expect(client._unstable_delegateDelayedLeave).not.toHaveBeenCalled();
expect(openID.getSFUConfig).toHaveBeenCalled();
});
it("uses OpenID with url + livekit_service_url transport", async () => {
// Must match the flow that got the token, so the identities line up
await delegateDelayedLeave({
...defaultParams,
transport: urlAndServiceUrlTransport,
});
expect(client._unstable_delegateDelayedLeave).not.toHaveBeenCalled();
expect(openID.getSFUConfig).toHaveBeenCalledWith(
expect.objectContaining({
role: "publisher",
delayId: "delay123",
delayEndpointBaseUrl: client.baseUrl,
}),
);
});
});
+253
View File
@@ -0,0 +1,253 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { ConnectionError, MatrixError } from "matrix-js-sdk";
import { type UnstableLivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import * as openID from "./openID";
import * as csApi from "./csApi";
import { type SFUConfig } from "./types";
import { NoMatrix2AuthorizationService } from "../../utils/errors";
export type { ClientOpenIDParts } from "./openID";
export type { ClientGetTokenParts, ClientDelegationParts } from "./csApi";
export type * from "./types";
/**
* Checks whether an error means "this homeserver does not implement the
* (unstable) MSC4195 endpoints".
*
* MSC4195 documents `M_NOT_FOUND` for this, but a homeserver without support
* answers with whatever it uses for unknown endpoints (Synapse: 404 with
* `M_UNRECOGNIZED`), so we accept any 404 as "unsupported".
* @param e The error to check.
*/
function isEndpointUnsupported(e: unknown): boolean {
return (
e instanceof MatrixError &&
(e.httpStatus === 404 ||
e.errcode === "M_NOT_FOUND" ||
e.errcode === "M_UNRECOGNIZED")
);
}
/**
* Checks whether an error means the MSC4195 CS API cannot be used from here:
* the homeserver does not implement it, or the request never reached the
* homeserver because in widget mode the host does not expose the endpoint
* (MSC4533), which the widget API reports as a plain error rather than a
* Matrix error. A connection error is neither, as the endpoint may well exist.
* @param e The error to check.
*/
function isCsApiUnavailable(e: unknown): boolean {
if (isEndpointUnsupported(e)) return true;
return !(e instanceof MatrixError) && !(e instanceof ConnectionError);
}
export interface GetSFUConfigParams {
/**
* The Matrix client.
*/
client: csApi.ClientGetTokenParts & openID.ClientOpenIDParts;
/**
* Data identifying the local user's session membership.
*/
membership: CallMembershipIdentityParts;
/**
* The transport for which we wish to get an access token.
*/
transport: UnstableLivekitTransport;
/**
* The name of the homeserver to which the {@link transport} belongs.
*/
serverName: string;
/**
* The ID of the Matrix room in which the session takes place.
*/
roomId: string;
/**
* The ID of the MatrixRTC slot in which the session takes place.
*/
slotId: string;
/**
* Whether we want to publish or only subscribe on the {@link transport}.
*/
role: "publisher" | "subscriber";
logger: Logger;
}
/**
* Gets an {@link SFUConfig} appropriate for connecting to a given transport.
*/
export async function getSFUConfig({
client,
membership,
transport,
serverName,
roomId,
slotId,
role,
logger,
}: GetSFUConfigParams): Promise<SFUConfig> {
const viaOpenID = async (serviceUrl: string): Promise<SFUConfig> =>
await openID.getSFUConfig({
client,
membership,
serviceUrl,
roomId,
slotId,
role,
logger,
});
const flow = chooseAuthFlow(transport, role);
if (flow.kind === "openID") return await viaOpenID(flow.serviceUrl);
try {
return await csApi.getSFUConfig({
client,
membership,
url: flow.url,
serverName,
roomId,
slotId,
});
} catch (e) {
// Publishers should never fall back. If a homeserver advertises a `url`
// but doesn't support MSC4195 itself, that's a misconfiguration which
// the admin should know about so they can prepare for the future
// deprecation of `livekit_service_url`.
const mayFallBack =
role === "subscriber" && "livekit_service_url" in transport;
if (mayFallBack && isCsApiUnavailable(e)) {
logger.warn(
`Cannot use the MSC4195 get_token endpoint (${e}), falling back to the OpenID flow with service ${transport.livekit_service_url}`,
);
return await viaOpenID(transport.livekit_service_url);
}
if (isEndpointUnsupported(e)) {
logger.error(
`Homeserver does not support the MSC4195 get_token endpoint, cannot get token for transport ${flow.url}`,
);
throw new NoMatrix2AuthorizationService(e as Error);
}
throw e;
}
}
export interface DelegateDelayedLeaveParams {
/**
* The Matrix client.
*/
client: csApi.ClientDelegationParts & openID.ClientOpenIDParts;
/**
* Data identifying the local user's session membership.
*/
membership: CallMembershipIdentityParts;
/**
* The transport to which we are publishing and wish to delegate the delayed
* leave event.
*/
transport: UnstableLivekitTransport;
/**
* The ID of the Matrix room in which the session takes place.
*/
roomId: string;
/**
* The ID of the MatrixRTC slot in which the session takes place.
*/
slotId: string;
/**
* The delay ID of the leave event to be delegated.
*/
delayId: string;
logger: Logger;
}
/**
* Delegates a delayed leave event to a given transport, so that the event will
* be sent on our behalf whenever we disconnect from the transport's SFU.
*/
export async function delegateDelayedLeave({
client,
membership,
transport,
roomId,
slotId,
delayId,
logger,
}: DelegateDelayedLeaveParams): Promise<void> {
// It's important that we use the same flow (CS API or OpenID) as whatever we
// would've used to get a JWT token for this transport, since the participant
// identity computed by the service needs to match.
const flow = chooseAuthFlow(transport, "publisher");
if (flow.kind === "csApi") {
try {
await csApi.delegateDelayedLeave({
client,
membership,
url: flow.url,
roomId,
slotId,
delayId,
});
} catch (e) {
if (isEndpointUnsupported(e))
logger.error(
"Server misconfigured: transport contains a 'url' field but server does not support the MSC4195 delegate_delayed_leave endpoint",
);
throw e;
}
} else {
// This will technically cause the service to issue a new JWT token, but
// it's safe to discard. We're only interested in triggering delegation.
await openID.getSFUConfig({
client,
membership,
serviceUrl: flow.serviceUrl,
roomId,
slotId,
role: "publisher",
delayEndpointBaseUrl: client.baseUrl,
delayId,
logger,
});
}
}
/**
* The flow with which the local member authenticates against a transport.
*/
export type AuthFlow =
| { kind: "csApi"; url: string }
| { kind: "openID"; serviceUrl: string };
/**
* Chooses between the MSC4195 CS API and the legacy OpenID flow against the
* transport's JWT service.
*
* Only the JWT service's `/sfu/get` endpoint assigns the legacy participant
* identity that clients predating MSC4195 expect from anyone advertising a
* membership state event. So a publisher keeps using the JWT service wherever
* the transport still offers one, even if it also has a `url`. Subscribers may
* use the CS API regardless, as nobody matches their identity.
* @param transport The transport to authenticate against.
* @param role Whether we want to publish or only subscribe on the transport.
*/
export function chooseAuthFlow(
transport: UnstableLivekitTransport,
role: "publisher" | "subscriber",
): AuthFlow {
if ("url" in transport) {
const useCsApi =
!("livekit_service_url" in transport) || role === "subscriber";
if (useCsApi) return { kind: "csApi", url: transport.url };
}
return { kind: "openID", serviceUrl: transport.livekit_service_url };
}
+230
View File
@@ -0,0 +1,230 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
beforeEach,
afterEach,
describe,
expect,
it,
type MockedObject,
vitest,
} from "vitest";
import fetchMock from "fetch-mock";
import { MatrixError } from "matrix-js-sdk";
import {
getSFUConfig,
type GetSFUConfigParams,
type ClientOpenIDParts,
} from "./openID";
import { testJWTToken } from "../../utils/test-fixtures";
import { ownMemberMock } from "../../utils/test";
import { FailToGetOpenIdToken } from "../../utils/errors";
import { logger } from "matrix-js-sdk/lib/logger";
const serviceUrl = "https://sfu.example.org";
const delayParams = {
delayEndpointBaseUrl: "https://matrix.homeserverserver.org",
delayId: "mock_delay_id",
};
describe("getSFUConfig", () => {
let matrixClient: MockedObject<ClientOpenIDParts>;
let params: GetSFUConfigParams;
beforeEach(() => {
fetchMock.catch(404);
matrixClient = { getOpenIdToken: vitest.fn() };
params = {
client: matrixClient,
membership: ownMemberMock,
serviceUrl: "https://sfu.example.org",
roomId: "!example_room_id",
slotId: "m.call#ROOM",
role: "publisher",
logger,
};
});
afterEach(async () => {
void (await fetchMock.flush());
vitest.clearAllMocks();
fetchMock.reset();
});
it("should handle fetching a token", async () => {
fetchMock.post("https://sfu.example.org/sfu/get", () => {
return {
status: 200,
body: { url: serviceUrl, jwt: testJWTToken },
};
});
const config = await getSFUConfig(params);
expect(config).toEqual({
jwt: testJWTToken,
url: serviceUrl,
livekitIdentity: "@me:example.org:ABCDEF",
livekitAlias: "!example_room_id",
});
});
it("should fail if the SFU errors", async () => {
fetchMock.post("https://sfu.example.org/sfu/get", () => {
return {
status: 500,
body: {
errcode: "M_LOOKUP_FAILED",
error: "Failed to look up user info from homeserver",
},
};
});
try {
await getSFUConfig(params);
} catch (ex: unknown) {
expect(ex).toBeInstanceOf(FailToGetOpenIdToken);
expect((ex as FailToGetOpenIdToken).cause).toBeInstanceOf(MatrixError);
const mxError = (ex as Error).cause as MatrixError;
expect(mxError.message).toEqual(
"MatrixError: [500] Failed to look up user info from homeserver",
);
return;
}
expect.fail("Expected test to throw;");
});
it("should retry without delay params if the JWT service legacy endpoint returns M_BAD_JSON 400", async () => {
let callCount = 0;
fetchMock.post(
"https://sfu.example.org/sfu/get",
(url, opts) => {
callCount++;
const body = JSON.parse(opts.body as string);
// First call: check if it has delay parts and return 400
if (callCount === 1) {
expect(body).toHaveProperty("delay_id", "mock_delay_id");
return {
status: 400,
body: { errcode: "M_BAD_JSON", error: "Unsupported parameters" },
};
}
// Second call: check if delay parts were stripped and return success
expect(body).not.toHaveProperty("delay_id");
expect(body).not.toHaveProperty("delay_timeout");
expect(body).not.toHaveProperty("delay_cs_api_url");
return {
status: 200,
body: { url: serviceUrl, jwt: testJWTToken },
};
},
{ overwriteRoutes: true },
);
// Note: Assuming getSFUConfig eventually calls getLiveKitJWT
const config = await getSFUConfig({ ...params, ...delayParams });
expect(config.jwt).toBe(testJWTToken);
expect(callCount).toBe(2);
});
it("should successfully send delay parameters to the JWT service legacy endpoint", async () => {
fetchMock.post(
"https://sfu.example.org/sfu/get",
(url, opts) => {
const body = JSON.parse(opts.body as string);
// Verify, that the request contains the expected delay parameters
if (
body.delay_id === "mock_delay_id" &&
body.delay_timeout === 3600000 &&
body.delay_cs_api_url === "https://matrix.homeserverserver.org"
) {
return {
status: 200,
body: { url: serviceUrl, jwt: testJWTToken },
};
}
return {
status: 400,
body: { error: "Missing expected delay params" },
};
},
{ overwriteRoutes: true },
);
const config = await getSFUConfig({ ...params, ...delayParams });
// Prüfe das Ergebnis
expect(config).toMatchObject({
jwt: testJWTToken,
url: serviceUrl,
});
});
it("should use the default endpoint when subscribing", async () => {
fetchMock.post("https://sfu.example.org/get_token", (url, opts) => {
const body = JSON.parse(opts.body as string);
expect(body).toMatchObject({
room_id: "!example_room_id",
slot_id: "m.call#ROOM",
member: {
id: ownMemberMock.memberId,
claimed_user_id: ownMemberMock.userId,
claimed_device_id: ownMemberMock.deviceId,
},
});
return { status: 200, body: { url: serviceUrl, jwt: testJWTToken } };
});
const config = await getSFUConfig({ ...params, role: "subscriber" });
expect(config.jwt).toBe(testJWTToken);
expect(fetchMock.called("https://sfu.example.org/sfu/get")).toBe(false);
});
it("should fail when subscribing if the default endpoint is missing", async () => {
// Nothing but the catch-all 404 answers the default endpoint. Subscribers
// must not fall back to the legacy endpoint, which would hand them a
// legacy identity.
await expect(
getSFUConfig({ ...params, role: "subscriber" }),
).rejects.toThrow(FailToGetOpenIdToken);
expect(fetchMock.called("https://sfu.example.org/get_token")).toBe(true);
expect(fetchMock.called("https://sfu.example.org/sfu/get")).toBe(false);
});
it("should retry fetching the openid token", async () => {
let count = 0;
matrixClient.getOpenIdToken.mockImplementation(async () => {
count++;
if (count < 2) {
throw Error("Test failure");
}
return Promise.resolve({
token_type: "Bearer",
access_token: "foobar",
matrix_server_name: "example.org",
expires_in: 30,
});
});
fetchMock.post("https://sfu.example.org/sfu/get", () => {
return {
status: 200,
body: { url: serviceUrl, jwt: testJWTToken },
};
});
const config = await getSFUConfig({ ...params, ...delayParams });
expect(config).toEqual({
jwt: testJWTToken,
url: serviceUrl,
livekitIdentity: "@me:example.org:ABCDEF",
livekitAlias: "!example_room_id",
});
});
});

Some files were not shown because too many files have changed in this diff Show More