Compare commits

...

319 Commits

Author SHA1 Message Date
Valere
09092e97c8 fix error in test 2026-03-25 12:15:12 +01:00
Valere
cb6bef3849 try extend timeout? 2026-03-25 10:10:29 +01:00
Valere
4d491be4a9 post merge fix 2026-03-25 08:21:32 +01:00
Valere
aad160a715 Merge branch 'livekit' into valere/default_route 2026-03-24 18:28:54 +01:00
Valere
fc61a36d4a review: improve comments 2026-03-24 18:25:09 +01:00
Valere
1bc2abb84f rename ControlledAudioOutput to iOSControlledAudioOutput 2026-03-24 18:03:33 +01:00
Valere
4f518819d3 review: extract ControlledAudioOutput in its own file 2026-03-24 18:02:27 +01:00
Robin
b40feae7da Merge pull request #3810 from element-hq/robin/button-accessibility
Improve accessibility of microphone, camera, and screen share buttons
2026-03-24 17:36:36 +01:00
Robin
e15761d8ee Merge pull request #3703 from element-hq/renovate/compound
Update Compound
2026-03-24 17:20:01 +01:00
Robin
c60ed50a9d Attempt to fix end-to-end widget tests 2026-03-24 17:04:28 +01:00
Robin
6dcd883fc3 Update test snapshots 2026-03-24 16:10:19 +01:00
Robin
c0d60b2c29 Improve accessibility of microphone, camera, and screen share buttons
Taking Valere's suggestion of giving them the 'switch' role. Also, the aria-label attributes were redundant (having tooltips already gives the buttons aria-labelledby).
2026-03-21 02:53:48 +01:00
Valere
c8b8d350d5 Merge branch 'livekit' into valere/default_route 2026-03-19 18:41:36 +01:00
Valere
9c2b2d4780 fix: playwright, new web popup interfering with test to witch room 2026-03-19 10:44:38 +01:00
Valere
396225a900 fix playwright earpiece mode 2026-03-19 10:44:38 +01:00
Valere
c4ec52ae15 add some test for normal AudiOutput 2026-03-19 10:44:38 +01:00
Valere
4be2bc7560 android: Select default output device based on callIntent
Add comments on existing code
Extracted a specific android controller for isolation and better testing

lint fixes

Fix device update logic and more tests

better typescript
2026-03-19 10:44:38 +01:00
Valere Fedronic
385ab0a0ed Merge pull request #3805 from element-hq/robin/ringing
New ringing UI
2026-03-19 10:43:16 +01:00
Robin
4d69565312 Merge pull request #3809 from element-hq/robin/invert-buttons
Invert the colors of the camera and microphone buttons
2026-03-18 14:49:55 +01:00
Robin
fa844446b6 Invert the colors of the camera and microphone buttons
So that they use primary color tokens when unmuted, and secondary color tokens when muted. This makes them work like the screen sharing button.
2026-03-18 11:29:55 +01:00
Robin
9dfade68ee New ringing UI
This implements the new ringing UI by showing a placeholder tile for the participant being dialed, rather than an overlay.
2026-03-18 11:20:43 +01:00
renovate[bot]
32ea8f522c Update Compound 2026-03-15 01:47:49 +00:00
renovate[bot]
6d14f1d06f Update GitHub Actions (#3804)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-14 14:48:44 +00:00
Robin
5cf5227eaa Merge pull request #3802 from element-hq/toger5/fix-qs-glob-js-yaml-security
Update qs, js-yaml, glob for security patches
2026-03-13 17:31:40 +01:00
Timo K
78240c2ec8 update qs, js-yaml, glob for security patches 2026-03-13 08:01:52 +01:00
Timo K
bf8bf80417 Revert "update qs, js-yaml, glob for security patches"
This reverts commit 7da9bca08c.
2026-03-13 08:01:08 +01:00
Timo K
7da9bca08c update qs, js-yaml, glob for security patches 2026-03-13 07:59:49 +01:00
Timo
748c8e6d0d Merge pull request #3800 from element-hq/toger5/fix-tar-minimatch-security
Bump tar, minimatch
2026-03-12 23:54:10 +08:00
Timo
40ef16a55f Merge pull request #3799 from element-hq/toger5/fix-rollup-security-alert
Update vite, vitest and rollup
2026-03-12 23:51:53 +08:00
Timo
07ad8374a9 Merge pull request #3595 from element-hq/toger5/dont-trap-in-invalid-config
Reset overwrite url if it is invalid (does fail to reach sfu)
2026-03-12 22:06:26 +08:00
Timo K
a0d5c79999 also update coverage 2026-03-12 14:52:16 +01:00
Timo K
3e171d9639 bump tar, minimatch
security alert
2026-03-12 14:48:05 +01:00
fkwp
413329cd26 Fix: zizmor findings (#3797)
* zizmor auto fixes

* add github action for security analysis with zizmor

* add access token to iOS push action

* fix zizmor findings

* add exceptions for dangerous-triggers including comments for reasoning

* improve comments

* prettier
2026-03-12 13:30:45 +01:00
Timo K
6b8f6e9405 update vite vitest and rollup
(rollup needs updating to fix a security alert)
2026-03-12 12:10:17 +01:00
fkwp
af54b39698 fix: typo pushing element registry OCI images now to the correct target (#3796)
* Push docker images to oci.element.io

* prettier

* add id-token permission as its required by tailscale login

* pass secrets to reusable workflows

* change secret path team -> voip

* Update .github/workflows/build-and-publish-docker.yaml

Co-authored-by: Gaël Goinvic <97093369+gaelgatelement@users.noreply.github.com>

* typo

---------

Co-authored-by: Gaël Goinvic <97093369+gaelgatelement@users.noreply.github.com>
2026-03-11 16:09:02 +01:00
Timo
c05d223133 Merge pull request #3775 from element-hq/toger5/new-pip-layout
Implement new Pip Layout (with control buttons)
2026-03-11 23:07:47 +08:00
Timo K
c7f25feb66 use better test condition for mute buttons 2026-03-11 15:46:19 +01:00
Timo K
a20edca9a1 fix pip container query 2026-03-11 15:36:37 +01:00
Timo K
d00ff78d65 fix pip interaction test (button presses) 2026-03-11 15:21:36 +01:00
fkwp
839c4dd738 fix: OCI image push to element registry (#3795)
* Push docker images to oci.element.io

* prettier

* add id-token permission as its required by tailscale login

* pass secrets to reusable workflows

* change secret path team -> voip

* Update .github/workflows/build-and-publish-docker.yaml

Co-authored-by: Gaël Goinvic <97093369+gaelgatelement@users.noreply.github.com>

---------

Co-authored-by: Gaël Goinvic <97093369+gaelgatelement@users.noreply.github.com>
2026-03-11 15:17:12 +01:00
fkwp
41f7b643fb Add zizmor checks on CI (#3792)
* zizmor auto fixes

* add github action for security analysis with zizmor

* add access token to iOS push action
2026-03-11 14:20:05 +01:00
Timo K
3a9d394529 activate click tests 2026-03-11 14:05:17 +01:00
fkwp
c9557e91d5 fix: add id-token permission as its required by tailscale login (part 3) (#3793)
* Push docker images to oci.element.io

* prettier

* add id-token permission as its required by tailscale login

* pass secrets to reusable workflows
2026-03-11 13:06:20 +01:00
Timo K
1e400bc550 remove unsused import 2026-03-10 18:26:12 +01:00
Timo K
6485da8fff add playwright tests for new pip layout 2026-03-10 15:17:41 +01:00
Timo K
54bef07b3b linter 2026-03-10 13:57:06 +01:00
Timo K
273eedd256 keep pip as it was before on mobile 2026-03-10 13:57:06 +01:00
Timo K
38382539ad fix lints 2026-03-10 13:57:06 +01:00
Timo K
8db1c4c370 Implement new Pip Layout (with control buttons) 2026-03-10 13:57:06 +01:00
Valere Fedronic
c98d03e2e1 Merge pull request #3787 from element-hq/valere/fix_typo_width_height
fix: typo inverting with/height in PIP spotlight tile
2026-03-10 09:38:53 +01:00
Valere
143b560b20 Add a playwright test to ensure object fit is correct 2026-03-10 09:05:05 +01:00
Valere
3da762ab36 fix: typo inverting with/height in PIP spotlight tile 2026-03-09 17:49:04 +01:00
Valere Fedronic
efcde9edfe Merge pull request #3756 from element-hq/valere/auto_fit_based_on_video_ratio
Auto fit video in tile depending on orientation of the tile and the source stream
2026-03-09 15:31:15 +01:00
Valere
ca3837f44e fix merge issue that added back a deprecated test 2026-03-09 15:07:42 +01:00
Robin
467675ff40 Merge pull request #3785 from robintown/unpin-deps
Unpin Knip and LiveKit track processors
2026-03-09 14:35:52 +01:00
Valere
5a612fea91 Merge branch 'livekit' into valere/auto_fit_based_on_video_ratio 2026-03-09 14:30:54 +01:00
Valere Fedronic
17ec488f0b Merge pull request #3784 from element-hq/valere/pr_template
Devx: Add a PR template
2026-03-09 14:15:14 +01:00
Valere
8f418ce9fd fix cla link 2026-03-09 13:59:39 +01:00
Robin
4c7fad30c0 Silence new Knip errors 2026-03-09 13:11:48 +01:00
Robin
4922249f41 Unpin Knip and LiveKit track processors
These dependencies were held back on old versions because of issues that are now resolved (see 2fca7e3719 and 44eb8acaee).
2026-03-09 13:00:43 +01:00
Robin
06c15ded37 Merge pull request #3747 from JakeTripplJ/screenshare-volume
Add volume control to screen shares
2026-03-09 10:59:45 +01:00
Robin
885a523e91 Fix formatting 2026-03-09 10:44:22 +01:00
Robin
313b8285d9 Make the screen share volume button accessible on mobile
In landscape orientation the button would be buried underneath the footer, which would block interaction with it. This commit changes the footer to not show in cases where a button has been pressed.
2026-03-09 10:30:42 +01:00
Valere
d00e1ca983 Devx: Add a PR template 2026-03-09 09:55:56 +01:00
Valere
513477d280 review: Use targetWidth/Height instead of listening to element bounds 2026-03-09 09:45:25 +01:00
Valere
273fff20bd review: add comment 2026-03-09 09:12:03 +01:00
Robin
3bbbac23a0 Adjust dimensions of screen share volume menu 2026-03-06 23:15:59 +01:00
Robin
c7a16e9dfd Refactor screen share volume button into a component 2026-03-06 23:14:15 +01:00
Jake Janicke
5f2d1c8a7e Comment typo
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 17:50:31 -06:00
Jake Janicke
e99e8628d6 Clean up and streamline safety checking instead of using multiple separate checks
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 17:09:07 -06:00
Jake Janicke
65045c264b Fix formatting
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 16:11:39 -06:00
Jake Janicke
3216d68470 Make screenShareVolume safer
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 16:05:56 -06:00
Jake Janicke
99401a7285 Make screenShareLocallyMuted check safer
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 15:13:20 -06:00
Jake Janicke
c74d19ad33 Add more guards against undefined
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 13:56:30 -06:00
Jake Janicke
ea074ceb2d Sort translation keys
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 13:16:51 -06:00
Jake Janicke
b88daf0198 Add tests for screen share volume UI presence logic
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 05:07:41 -06:00
Jake Janicke
a34fe7817a Add tests for screen share volume controls
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 03:52:46 -06:00
Jake Janicke
72520dbb3f Add screen share volume slider UI
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-04 03:52:38 -06:00
Jake Janicke
2cf4a38c17 Add volume logic to RemoteScreenShareViewModel
Signed-off-by: Jake Janicke <jaketripplj@gmail.com>
2026-03-03 16:47:41 -06:00
fkwp
00f880108e fix: add id-token permission as its required by tailscale login (part 2) (#3770)
* Push docker images to oci.element.io

* prettier

* add id-token permission as its required by tailscale login
2026-03-02 17:03:01 +01:00
Valere Fedronic
a2ee0163b6 Merge pull request #3752 from element-hq/valere/remove_deprecated_param
remove deprecated `analyticsID` url param
2026-03-02 16:10:27 +01:00
Valere
5165e95d82 fix: default to cover is size are 0 2026-03-02 15:38:43 +01:00
fkwp
6e0f48e99b fix: add id-token permission as its required by tailscale login (#3768)
* Push docker images to oci.element.io

* prettier

* add id-token permission as its required by tailscale login
2026-03-02 15:04:18 +01:00
Valere
adc329a7e7 post merge fix 2026-03-02 14:41:47 +01:00
Valere
c199d00300 Merge branch 'livekit' into valere/auto_fit_based_on_video_ratio 2026-03-02 14:31:47 +01:00
Robin
38e79bb7db Merge pull request #3764 from element-hq/renovate/github-actions
Update docker/login-action digest to c94ce9f
2026-03-02 13:44:15 +01:00
Robin
23050075b7 Merge pull request #3765 from element-hq/renovate/embedded-package-dependencies
Update embedded package dependencies
2026-03-02 12:31:27 +01:00
renovate[bot]
b2475e4d1d Update embedded package dependencies 2026-03-01 01:11:41 +00:00
renovate[bot]
8e7e099294 Update docker/login-action digest to c94ce9f 2026-03-01 01:11:01 +00:00
Robin
dcb7143359 Merge pull request #3755 from element-hq/robin/ringing
Convert media view model classes to interfaces
2026-02-27 17:27:58 +01:00
Robin
0ce24929dc Add TODO to clean up encryption status code 2026-02-27 17:12:24 +01:00
Robin
9930288d1f Move observeInboundRtpStreamStats$ into the appropriate file 2026-02-27 17:08:04 +01:00
Valere
ae8b1f840f add missing mocking 2026-02-26 17:02:43 +01:00
Valere
15aa67ebb9 remove unused fit to frame translation 2026-02-26 16:46:08 +01:00
Valere
1de8d93b4b feat: video auto fit based on video stream size 2026-02-26 16:28:37 +01:00
Robin
6b51b7dc58 Split MediaViewModel into multiple files 2026-02-25 22:41:28 +01:00
Robin
d16fa4531c Merge pull request #3712 from element-hq/renovate/all-minor-patch
Update all non-major dependencies
2026-02-25 19:52:22 +01:00
Robin
44eb8acaee Pin Knip to 5.82.1 to work around Yarn plugin issue
We can upgrade beyond 5.82.1 once https://github.com/webpro-nl/knip/pull/1574 is merged.
2026-02-25 19:36:00 +01:00
Robin
2fca7e3719 Roll back @livekit/track-processors to fix type error (again!)
While we wait for https://github.com/livekit/track-processors-js/pull/118 to be released
2026-02-25 18:03:04 +01:00
renovate[bot]
62b408a2b3 Update all non-major dependencies 2026-02-25 16:47:27 +00:00
fkwp
465d28a1cd bumping LiveKit SFU and synapse versions for ci/cd (#3757) 2026-02-25 17:46:35 +01:00
fkwp
dcf3a722a7 Push docker images to oci.element.io (#3725)
* Push docker images to oci.element.io

* prettier
2026-02-25 17:45:56 +01:00
Robin
6995388a29 Convert media view model classes to interfaces
Timo and I agreed previously that we should ditch the class pattern for view models and instead have them be interfaces which are simply created by functions. They're more straightforward to write, mock, and instantiate this way.

The code for media view models and media items is pretty much the last remaining instance of the class pattern. Since I was about to introduce a new media view model for ringing, I wanted to get this refactor out of the way first rather than add to the technical debt.

This refactor also makes things a little easier for https://github.com/element-hq/element-call/pull/3747 by extracting volume controls into their own module.
2026-02-25 14:47:43 +01:00
Robin
4039272e75 Merge pull request #3745 from element-hq/robin/ringing
Tidy up media view models a little bit
2026-02-25 14:22:03 +01:00
Valere
c8f5be7e4f remove deprecated analyticsID url param 2026-02-24 11:56:54 +01:00
Robin
4a0e89730d Merge pull request #3728 from element-hq/renovate/matrix-widget-api
Update dependency matrix-widget-api to v1.17.0
2026-02-18 19:43:32 +01:00
Robin
bc238778ad Make the type of participant$ stricter
It is, in fact, required to be a behavior.
2026-02-18 14:01:55 +01:00
Robin
9d37125677 Only expose RTC backend identity debug info where we actually use it
We only ever inspect the RTC backend identity of user media tiles. So it only needs to be exposed on the user media view model.
2026-02-18 13:46:18 +01:00
Robin
d87c3293c9 Make one-on-one layout media types stricter 2026-02-18 13:44:20 +01:00
renovate[bot]
789de1963a Update GitHub Actions (#3711)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-02-17 16:01:46 +01:00
Timo
ba95780813 Merge pull request #3734 from element-hq/robin/local-transport
Logically separate the advertised transport from the active transport
2026-02-16 15:21:28 +01:00
Robin
4508002947 Test local transport code in oldest member mode 2026-02-13 14:06:44 +01:00
Robin
2a56830426 Fix existing LocalTransport tests 2026-02-13 14:06:44 +01:00
Robin
6cf859fd9e Fix resource leak due to unsafe scope reference 2026-02-13 12:40:12 +01:00
renovate[bot]
91da088da4 Update dependency matrix-widget-api to v1.17.0 2026-02-12 12:15:03 +00:00
Valere Fedronic
076cc035d3 Merge pull request #3733 from element-hq/valere/bug_fix_stringify_crash
Fix: crash on mobile (android) with `0.17.0-rc.2` due to a log causing stringify to crash with `TypedError: circular structure`
2026-02-11 14:46:37 +01:00
Valere
92992df7df test: ensure ragelogger resist to JSON.stringify throws 2026-02-11 13:56:42 +01:00
Robin
13d131c2e9 Logically separate the advertised transport from the active transport
To correctly implement the legacy "oldest membership" mode, we need the code to be more nuanced about the local transport. Specifically, it needs to allow for the transport we advertise in our membership to be different from the transport that we connect to and publish media on. Otherwise, if these two are yoked together, members will resend their memberships whenever an SFU hop occurs, which an attacker could use to cause an amplified wave of state changes.
2026-02-11 13:47:15 +01:00
Valere Fedronic
8ea773aa24 Merge pull request #3731 from element-hq/valere/fix_netlify_deploy
fix: Netlify preview broken
2026-02-11 13:19:50 +01:00
Valere
0d24995c3e add defensive coding against JSON.stringify 2026-02-10 17:46:58 +01:00
Valere
1e9f2e6282 fix: Simplify log causing stringify TypeError on widget mode 2026-02-10 17:24:01 +01:00
Valere
73245e875d fix: Netlify preview broken 2026-02-10 15:11:43 +01:00
Timo
a329ae0ad1 Merge pull request #3720 from element-hq/toger5/sdk-improvements-only
[MatrixRTC SDK] improvements - compatible with custom rtc application
2026-02-09 09:50:16 +01:00
Timo K
652ff0060c Back to develop branck (matrix-js-skd) 2026-02-09 09:07:54 +01:00
Robin
d3476c0446 Merge pull request #3726 from element-hq/robin/generate-items-logs
Add debug logs to generateItems
2026-02-06 13:45:13 +01:00
Robin
a0209eb433 Add debug logs to generateItems
It's always worth having logs for when state holders are created or destroyed (these are often the most interesting things happening in the application), so I thought it would be nice to have generateItems always log for you when it's doing that.
2026-02-06 13:26:02 +01:00
Timo K
e1ef28fdcc use js-sdk fixing room hack for any app 2026-02-05 09:04:03 +01:00
Timo K
2e26193c4a update leave 2026-02-05 06:33:51 +01:00
Timo K
42efc05bd0 Add more granular leave/join api 2026-02-05 06:06:15 +01:00
Valere
809186a2e7 test: test requesting send message permission in initialize widget 2026-02-04 21:16:50 +01:00
Timo K
927e8e195c add experimental label 2026-02-04 20:16:31 +01:00
Timo K
33f6271d13 sdk improvements - compatible with custom rtc application
- add local member
 - add optional make sticky
 - default to video not enabled
 - allow sending matrix events
2026-02-04 20:06:26 +01:00
Timo
2c1476f151 Merge pull request #3715 from element-hq/revert-3701-toger5/sdk-improvements
Revert "[WIP] SDK target improvements"
2026-02-04 18:41:54 +01:00
Timo
97ee4630b0 Revert "[WIP] SDK target improvements" 2026-02-04 08:51:01 +01:00
Timo
02b29f98b6 Merge pull request #3701 from element-hq/toger5/sdk-improvements
[WIP] SDK target improvements
2026-02-04 08:47:28 +01:00
Valere
fdc33a223f build against js-sdk rc 40.2.0-rc.0 2026-02-03 17:37:58 +01:00
Valere
7beafcac92 Merge branch 'livekit' into toger5/sdk-improvements 2026-02-03 16:32:36 +01:00
Valere Fedronic
5563f30bf2 Merge pull request #3705 from element-hq/toger5/fix-test-no-more-legacy-ring
[Matrix2.0 compatibility] stop advertising `livekit_alias`, use `"ROOM"` slot_id (backwards compat), stop sending deprecated notify
2026-02-03 16:26:52 +01:00
Valere
9d8ebf8ef3 fix import 2026-02-03 15:37:41 +01:00
Valere
a0ad238952 fix: DidSendCallNotification is not emitting array anymore 2026-02-03 15:29:19 +01:00
Timo K
18b771e620 Allow sending room events in rtc sdk 2026-02-03 11:10:57 +01:00
Timo K
461a1f1b70 add to sending room messages 2026-01-31 12:29:56 +01:00
Timo K
a066495c5f bump js-sdk fixup custom applications 2026-01-31 12:09:29 +01:00
Timo K
506b85770b Merge branch 'livekit' into toger5/sdk-improvements 2026-01-31 11:53:15 +01:00
Timo K
bd8c4188d0 bump js-sdk 2026-01-29 17:24:21 +01:00
Timo K
c78e8c54b8 js-sdk-bump 2026-01-29 17:22:56 +01:00
Valere Fedronic
c18cc964a8 Merge pull request #3706 from element-hq/valere/fix_remove_livekit_alias_signaling
The advertised livekit_alias in membership is deprecated
2026-01-29 16:13:53 +01:00
Valere
8ee098eef1 review: quick comment 2026-01-29 15:46:41 +01:00
Valere
a7bd2d3668 review: comment and better type 2026-01-29 14:58:54 +01:00
Valere
ca7c8a2f11 Send livekit_alias in multi sfu with state 2026-01-28 16:42:07 +01:00
Valere
146375e320 Only remove livekit_alias in sticky events 2026-01-28 16:33:52 +01:00
Valere
992fb9093c fix test and use anonymised alias 2026-01-28 15:25:19 +01:00
Valere
99ccfcfbd4 fix linter 2026-01-28 15:18:29 +01:00
Valere
420304fd9d fix test mocking 2026-01-28 15:16:02 +01:00
Valere
81cb89e599 fix test mocking 2026-01-28 15:11:10 +01:00
Valere
ee8b27837a fix lint issues 2026-01-28 14:38:23 +01:00
Valere
a645bbe0ba fix bad import 2026-01-28 14:29:16 +01:00
Valere
4c7db0147e The advertised livekit_alias in membership is deprecated 2026-01-28 14:22:21 +01:00
Timo K
c1b2346e22 Update tests based on new js-sdk changes
(no more legacy notify event)
2026-01-27 16:49:25 +01:00
Timo K
92a4150d6d more/better logging 2026-01-26 15:43:07 +01:00
Timo K
829619cca7 fix build:sdk:production 2026-01-26 12:44:25 +01:00
Timo K
408495b07a add back permissions to pr-deploy.yaml 2026-01-26 12:43:15 +01:00
Timo K
02f1a0da82 add local member and rtcBackendIdentity to sdk target 2026-01-26 12:43:01 +01:00
Timo K
6d8ade073a use sessionManager with custom slot instead of only session. 2026-01-25 19:27:07 +01:00
Timo K
1c946f7bb2 fix package full 2026-01-25 14:38:45 +01:00
Timo K
5724c3a09c add sdk:production 2026-01-25 14:37:37 +01:00
Timo K
d55464f3ab build sdk 2026-01-25 14:32:30 +01:00
Timo K
73a5a10c6d add workflow to publish sdk version to netlify 2026-01-25 14:25:15 +01:00
Timo K
112420cb5b fix widget api appliaction capabilities 2026-01-25 14:24:52 +01:00
Timo K
31823dbd53 improve widget tests
(this we might want to cherry pick in its own PR)
2026-01-25 14:24:31 +01:00
Timo
15c39372f4 Merge pull request #3697 from element-hq/renovate/npm-lodash-es-vulnerability
Update dependency lodash-es to v4.17.23 [SECURITY]
2026-01-23 10:51:41 +01:00
Timo
330ad3448d Merge pull request #3686 from element-hq/robin/sticky-widget
Make Element Call widgets compatible with Matrix 2.0 mode
2026-01-22 20:17:16 +01:00
Timo K
220fc51eb5 add initializeWidget to tests. 2026-01-22 19:38:39 +01:00
Timo K
b754106feb Add widget.ts tests 2026-01-22 19:23:08 +01:00
renovate[bot]
05107a3e5c Update dependency lodash-es to v4.17.23 [SECURITY] 2026-01-22 04:37:37 +00:00
Robin
d516f0f602 Make Element Call widgets compatible with Matrix 2.0 mode
This gives Element Call widgets the ability to send and receive sticky RTC membership events.
2026-01-21 19:59:34 +01:00
Timo
3eb88ec4a8 Merge pull request #3693 from element-hq/toger5/fix-sticky-rejoin-error
Fix rejoin EC crash
2026-01-20 16:30:09 +01:00
Timo K
745772f672 Fix rejoin EC crash
Due to a duplcaited key (the key not being specific enough)
2026-01-20 13:45:07 +01:00
Valere Fedronic
486c0b884b Merge pull request #3690 from element-hq/valere/playwright/improve_sticky_test
playwright: intercept send request to ensure sticky events are sent
2026-01-20 08:26:47 +01:00
Valere
9f323d21a8 review: quick comment 2026-01-20 08:10:50 +01:00
Timo
a72aae0b8d Merge pull request #3679 from element-hq/toger5/fix-unnecassary-error-logs
Fix some error log lines that lead to confusion
2026-01-19 17:57:34 +01:00
Timo
b1ceeda9ee Merge pull request #3687 from toger5/toger5/fix-reemitted-event-params
Fix delayId delegation. Use correct event reemitting types.
2026-01-19 16:00:16 +01:00
Valere
ed8cee9f58 review: fix/clean the route interception 2026-01-19 15:28:22 +01:00
Timo K
3a5b9dd519 remove overkill log 2026-01-19 14:58:45 +01:00
Valere
a985281b46 playwright: intercept send request to ensure sticky event are sent 2026-01-19 13:58:36 +01:00
Timo K
a7008d37c6 better log msg 2026-01-19 13:54:46 +01:00
Timo K
94bcb1bb27 remove leftover layout action code 2026-01-19 13:54:36 +01:00
Timo
36a1aa3c59 Merge pull request #3689 from toger5/toger5/remove-deprecated-layout-widget-actions
Remove deprecated widget layout actions
2026-01-19 13:20:04 +01:00
Valere Fedronic
b4ca55f7df Merge pull request #3688 from element-hq/valere/bug_deny_permissions
Do not end call with error in case of MediaDeviceError
2026-01-19 12:32:01 +01:00
Timo K
a102b3f4aa Fix some error log lines that lead to confusion.
In the past those log lines often were referenced for issues but they
are no real issues.
Either they are just deprecated code running or expected.
2026-01-19 12:20:37 +01:00
Timo K
d8802aa079 linter 2026-01-19 12:18:13 +01:00
Timo K
4833d4b867 Bump js-sdk to use version that properly emits delayId updates 2026-01-19 12:15:40 +01:00
Timo K
abb80e4e87 Fix event reemitting types
The callback args for reemitting will change to an array including the
originals emitter.
2026-01-19 12:15:15 +01:00
Timo K
2c3365a401 remove deprecated widget layout actions 2026-01-19 12:11:32 +01:00
Valere
2817362e15 Do not end call with error in case of MediaDeviceError 2026-01-19 11:46:17 +01:00
Timo
95b730e93f Merge pull request #3683 from element-hq/valere/playwright/spa_sticky_test
playwright: Test call in SPA mode with matrix 2.0
2026-01-19 09:34:49 +01:00
Valere
c2eea67b35 Merge branch 'livekit' into valere/playwright/spa_sticky_test 2026-01-16 17:41:39 +01:00
Valere
202e06a16d skip FF 2026-01-16 17:41:32 +01:00
Timo
e278752942 Merge pull request #3675 from element-hq/toger5/fixing-duble-publishing
Fix Publisher cleanup procedure
2026-01-16 17:37:46 +01:00
Valere
89c878cf3f playwright: Test call in SPA mode with matrix 2.0 2026-01-16 17:11:45 +01:00
Timo K
f41175444b revert changes unrelated to this PR 2026-01-16 16:23:05 +01:00
Timo K
102664482b Activate the hotswap test that was failing before. 2026-01-16 15:41:25 +01:00
Timo K
d5df9e65dd Merge branch 'livekit' into toger5/fixing-duble-publishing 2026-01-16 15:38:56 +01:00
Timo
f7255b3697 Merge pull request #3681 from element-hq/toger5/playwright-publisher-swap
Add playwright test for hotswatpping the transport (swap from `oldest_member` (singeSFU) to `multi_sfu`)
2026-01-16 15:34:15 +01:00
Timo K
3f8b3ba3f1 add test to check if publisher is properly destroyed 2026-01-16 14:41:11 +01:00
Timo K
22c1948a82 Add playwright test for hotswatpping transport 2026-01-16 14:27:52 +01:00
Timo K
a99a413b88 temp 2026-01-16 14:07:56 +01:00
Timo K
83df152608 review 2026-01-16 13:20:03 +01:00
Timo K
c376d9e5c1 fix tests 2026-01-16 13:12:31 +01:00
Timo K
f09e2c8b55 Merge branch 'livekit' into toger5/dont-trap-in-invalid-config 2026-01-16 13:11:06 +01:00
Valere Fedronic
5c366320cb Merge pull request #3676 from element-hq/valere/playwright/compat_bug_reproduce
playwright: Federated call between legacy and compat
2026-01-16 13:09:02 +01:00
Valere
16b08994cc Try fixup mixed conten error loading widget 2026-01-16 12:53:10 +01:00
Timo K
f5c31626a6 fix unit tests 2026-01-16 12:43:13 +01:00
Timo K
5a48820927 remove widget layout widget action for good. 2026-01-16 10:22:23 +01:00
Timo K
971d16d243 revert jwt to stable version 2026-01-16 10:21:26 +01:00
Timo K
9b40d7420a refactor to use destroy method 2026-01-15 19:04:32 +01:00
Timo K
f71e483080 logging updates 2026-01-15 18:13:34 +01:00
Timo K
f7590a33d7 Fix publisher clean up! This is the acutal bug we chased 2026-01-15 18:12:00 +01:00
Timo K
c74accb906 remove "fetching" state again (we have to be able to deal with those
kind of switches!) Optimize to not always fetch the oldest member jwt
(only do so in legacy mode)
2026-01-15 18:11:08 +01:00
Valere
a00c6501c2 fix discarding pop ups 2026-01-15 15:22:04 +01:00
Valere
9e933b74ca cleanup notifications toast on start 2026-01-15 14:31:33 +01:00
Valere
d25d6d3196 Test all pairs of rtc mode in federated calls 2026-01-15 12:06:41 +01:00
Valere
243f369a24 re-use test helper in other files 2026-01-15 11:50:37 +01:00
Valere
0d1488af4c playwright: Federated call between legacy and compat 2026-01-14 18:07:12 +01:00
Timo K
b49411abfa dont set localTransport while still fetching oldest member transport 2026-01-14 17:35:35 +01:00
Timo
27e351270c Merge pull request #3669 from element-hq/toger5/livekit-alias-simple-roomid
Change the livekit alias to just be the room id (for backwards compatibility)
2026-01-14 12:43:08 +01:00
Timo K
41eb45b3c4 fix comments 2026-01-14 12:29:22 +01:00
Valere Fedronic
f5a6a05570 Merge pull request #3671 from element-hq/valere/playwright/huddle_test
playwright: End to end test for huddle calls in widget mode
2026-01-14 12:27:57 +01:00
Timo
31cc12eda4 Merge pull request #3670 from element-hq/toger5/livekitAlias-debugging-info
Add livekitAlias debugging info
2026-01-14 12:26:50 +01:00
Robin
39c24eb0f5 Merge pull request #3665 from element-hq/renovate/livekit-components
Update dependency @livekit/components-react to v2.9.17
2026-01-14 12:15:48 +01:00
Valere
7338a55e76 review: improve doc 2026-01-14 12:15:00 +01:00
Valere
68f04d46a9 playwright: End to end test for huddle calls in widget mode 2026-01-13 19:11:12 +01:00
Timo K
dbdcde2489 fix lint 2026-01-13 18:17:37 +01:00
Timo K
636f737cd3 reduce diff 2026-01-13 18:05:05 +01:00
Timo K
93c9225e8a Merge branch 'livekit' into toger5/dont-trap-in-invalid-config 2026-01-13 18:01:30 +01:00
Valere Fedronic
16ad89df47 Merge pull request #3667 from element-hq/valere/playwright/discard_call_widget_test
playwright: Test that declining a call in DM works as expected
2026-01-13 18:00:36 +01:00
Timo
3ce0afcacc Merge pull request #3664 from element-hq/toger5/userMedia-alternative
Small refactor: User media alternative
2026-01-13 17:51:36 +01:00
Timo K
4d35b77077 bump jwt docker 2026-01-13 17:41:46 +01:00
Timo K
20e63c8dc7 fix tests 2026-01-13 17:39:55 +01:00
Timo K
ff8d71380d Add livekitAlias debugging info 2026-01-13 15:14:34 +01:00
Timo K
d185b52091 more docstrings to docuemnt the migration away from livekit_alias 2026-01-13 12:44:17 +01:00
Timo K
72ca3821ab Change the livekit alias to just be the room id for backwards
compatibility.
2026-01-13 12:37:46 +01:00
Valere Fedronic
4b7dc8a290 Merge pull request #3659 from element-hq/valere/initial_mute_states
Fix: Ensure voice and audio intent are respected
2026-01-13 11:50:32 +01:00
Gordnho
f391c68055 Add Haproxy configuration for element call (#3661)
* Add Haproxy configuration for livekit SFU

---------

Co-authored-by: fkwp <fkwp@users.noreply.github.com>
2026-01-13 10:39:15 +00:00
Valere
e82a048088 playwright: Test that declining a call in DM works as expected 2026-01-13 09:46:44 +01:00
Valere
7a8c2d6ca9 Merge branch 'livekit' into valere/initial_mute_states 2026-01-12 19:13:40 +01:00
Valere
f3df8bc709 review: Use is_widget directly instead of using the packageType 2026-01-12 17:28:18 +01:00
Valere
534f934378 review: just inline intent and skipLobby from urlParam 2026-01-12 16:37:28 +01:00
Valere
56beb59346 review: Update copyright properly 2026-01-12 15:35:21 +01:00
Valere
fbdcacb344 review: remove dead comment 2026-01-12 15:23:02 +01:00
Timo K
89e5e5b10a fix uuid 2026-01-12 12:34:50 +01:00
Timo K
8d8eb09922 Merge branch 'livekit' into toger5/userMedia-alternative 2026-01-12 12:34:03 +01:00
renovate[bot]
fbdcde3058 Update dependency @livekit/components-react to v2.9.17 2026-01-12 01:33:44 +00:00
Timo
f444c9bc11 Merge pull request #3638 from element-hq/toger5/delayed-event-delegation
Pseudonomous identity and use the new jwt service endpoint (with delayed event delegation)
2026-01-09 19:58:30 +01:00
Timo K
47e389678b add uuid in our own package.json 2026-01-09 19:48:11 +01:00
Timo K
7a8f5cc859 use crypto.randomUUID() 2026-01-09 19:40:21 +01:00
Timo K
45bb233673 Merge branch 'toger5/delayed-event-delegation' into toger5/userMedia-alternative 2026-01-09 19:38:21 +01:00
Timo K
341277932c fix tests 2026-01-09 19:34:41 +01:00
Timo K
735c17d3e6 fix tile flicker on video mute/unmute
(only pass rtcBackendIdenitity instead of full rtcMember)
2026-01-09 19:31:40 +01:00
Timo K
1a9e4a1a5c bump js-sdk 2026-01-09 19:12:54 +01:00
Timo K
7387a00406 lint 2026-01-09 19:04:42 +01:00
Timo K
32694fddf6 improve rtcBackendIdentity debugging 2026-01-09 19:04:02 +01:00
Timo K
444c50d8c4 Change code style of user media 2026-01-09 18:27:56 +01:00
Timo K
9a88e3dab6 review valere (more readable force new endpoint logic) 2026-01-09 18:05:26 +01:00
Timo K
39432310e3 use actual uuid for member.id
review with some renaming and removal of unnecassary if
2026-01-09 17:56:30 +01:00
Timo
0addc7c09b Merge pull request #3662 from element-hq/toger5/simplify-sdk-example
Update sdk example
2026-01-09 17:34:57 +01:00
Timo
0b766472ed Update sdk/README.md
Co-authored-by: Robin <robin@robin.town>
2026-01-09 17:30:02 +01:00
Timo K
ed3d168ac3 fix tests 2026-01-09 17:29:48 +01:00
Timo K
70b7ebff80 prettier 2026-01-09 16:32:14 +01:00
Timo K
b7838e4353 Update README.md 2026-01-09 16:25:06 +01:00
Timo K
0eb0169ad9 update example 2026-01-09 16:01:03 +01:00
Timo K
c2e3774749 prettier 2026-01-09 15:40:39 +01:00
Timo K
c7c6dc18bf fix js-sdk dependency 2026-01-09 15:28:11 +01:00
Timo K
7dbbd763b9 Refactor how we aquire the jwt token for the local user. (only fetch it
once)

The local jwt token needs to be aquired via the right endpoint. The
endpoint defines how our rtcBackendIdentity is computed. Based on us
using sticky events or state events we also need to use the right
endpoint. This cannot be done generically in the connection manager. The
jwt token now is computed in the localTransport and the resolved sfu
config is passed to the connection manager.

Add JWT endpoint version and SFU config support Pin matrix-js-sdk to a
specific commit and update dev auth image tag. Propagate SFU config and
JWT endpoint choice through local transport, ConnectionManager and
Connection; add JwtEndpointVersion enum and LocalTransportWithSFUConfig
type. Add NO_MATRIX_2 auth error and locale string, thread
rtcBackendIdentity through UI props, and include related test, CSS and
minor imports updates
2026-01-09 13:38:26 +01:00
Robin
a567505ef5 Merge pull request #3478 from element-hq/renovate/compound
Update Compound
2026-01-09 13:12:48 +01:00
Robin
f89a7a8211 Fix tests 2026-01-09 12:58:06 +01:00
Valere Fedronic
29854a77b4 Merge pull request #3660 from element-hq/valere/add_codecov_shield
Add codecov shields.io badge
2026-01-09 12:57:56 +01:00
Valere
bf19b388ea Add codecov shields.io badge 2026-01-09 12:43:33 +01:00
renovate[bot]
8fe49d681a Update Compound 2026-01-08 19:49:47 +00:00
Timo K
d4b06b0f9c fix connection recreation which breaks EC lk connection 2026-01-08 14:27:47 +01:00
Timo K
1909aef186 temp 2026-01-08 12:27:17 +01:00
Timo K
385f63e83e fix tests 2026-01-07 17:46:39 +01:00
Timo K
a5a4bb2b82 add retries inside the getLiveKitJWTWithDelayDelegation and
`getLiveKitJWT` functions.
2026-01-07 17:38:29 +01:00
Timo K
5556d363b9 add retries and be more specific when we fall back to legacy endpoint 2026-01-07 17:21:14 +01:00
Timo K
0eeed4e18e fix test snapshot 2026-01-07 17:05:00 +01:00
Timo K
d5ad2e38e2 fix up tests 2026-01-07 16:09:08 +01:00
Timo K
75fca3108a cleanup an rename compatibility mode 2026-01-07 16:08:20 +01:00
Timo K
d814f60f23 review (docstrings) and remove unused artifacts. 2026-01-07 15:47:44 +01:00
Timo K
dd562bdaf5 dont use throw for logic. 2026-01-07 15:36:32 +01:00
Timo K
d48042f522 fix lint 2026-01-07 15:18:37 +01:00
Timo K
6480df44e9 add tests for open id delay fallback 2026-01-07 15:10:32 +01:00
Timo K
83d04ac122 fix tests and remove duplicated mocks. 2026-01-07 13:26:37 +01:00
Timo K
556a87d141 fix js-doc 2026-01-05 22:20:19 +01:00
Timo K
69a4189517 self review 2026-01-05 22:00:02 +01:00
Timo
009c9e046c Merge branch 'livekit' into toger5/delayed-event-delegation 2026-01-05 21:26:15 +01:00
Timo K
1580087286 use latest js-sdk 2026-01-05 21:24:52 +01:00
Timo K
00fca6e3c7 simplify localTransport 2026-01-05 21:17:37 +01:00
Timo K
5d5d75ebdf fixup merge 2026-01-05 21:08:33 +01:00
Timo K
df2bd22005 Merge branch 'livekit' into toger5/delayed-event-delegation 2026-01-05 21:08:21 +01:00
Timo K
582d1d8495 Merge branch 'livekit' into toger5/dont-trap-in-invalid-config 2026-01-05 14:37:48 +01:00
Timo K
eb60473510 add tests 2026-01-05 14:20:16 +01:00
Timo K
c35b960627 fix lints 2026-01-05 10:42:08 +01:00
Timo K
0f5c5d8be5 cleanup based on new js-sdk impl 2025-12-29 17:38:54 +01:00
Timo K
7591e2bda1 Merge branch 'livekit' into toger5/delayed-event-delegation 2025-12-28 21:04:49 +01:00
Timo K
3ba2d13e27 use the js-sdk where the hashed rtcSessionIdeintity is already part of
the event (no need to compute it in the encryption manager)
2025-12-28 21:04:46 +01:00
Timo K
55d18f10fe temp refactored membership rtcidentity 2025-12-19 19:23:41 +01:00
Timo K
50f3bf00ae use correct js-sdk 2025-12-17 10:22:46 +01:00
Timo K
ab7e3486b3 Make use of the new jwt service endpoint (with delayed event delegation)
This also does all the compatibility work. When to use which endpoint to
authenticate agains a jwt service.
2025-12-17 09:53:49 +01:00
Timo K
9bd51fdfc4 Merge branch 'livekit' into toger5/pseudonomous-identities 2025-12-16 12:18:00 +01:00
fkwp
ff3d6bd088 enable sticky events 2025-12-16 11:22:25 +01:00
fkwp
6fe6daba31 switch synapse docker image to one with sticky event support 2025-12-16 11:20:38 +01:00
Timo
3d8d0ae9dd Merge branch 'livekit' into toger5/pseudonomous-identities 2025-12-15 21:20:55 +01:00
Timo K
5bc6ed5885 small refactor to make it testable. 2025-12-15 20:17:57 +01:00
Timo K
909d980dff still with broken tests... 2025-12-15 18:23:30 +01:00
Timo K
df2f503a05 review 2025-12-11 18:00:23 +01:00
Timo K
c20b206ab2 i18n 2025-12-01 19:53:27 +01:00
Timo K
5ceb140d13 remove check in local transport 2025-12-01 19:51:48 +01:00
Timo K
ed4517703f better UX (valiate on save) 2025-12-01 19:40:44 +01:00
Timo K
bd3e91738e Reset overwrite url if it is invalid (does fail to reach sfu) 2025-12-01 19:40:44 +01:00
166 changed files with 10134 additions and 4937 deletions

View File

@@ -99,6 +99,13 @@ module.exports = {
"jsdoc/require-param-description": "off",
},
},
{
files: ["playwright/**"],
rules: {
// Playwright as a `use` function that has nothing to do with React hooks.
"react-hooks/rules-of-hooks": "off",
},
},
],
settings: {
react: {

39
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,39 @@
<!-- Thanks for submitting a PR! Please ensure the following requirements are met in order for us to review your PR -->
## Content
<!-- Describe shortly what has been changed -->
## Motivation and context
<!-- Provide link to the corresponding issue if applicable or explain the context -->
## Screenshots / GIFs
<!--
You can use a table like this to show screenshots comparison.
Uncomment this markdown table below and edit the last line `|||`:
|copy screenshot of before here|copy screenshot of after here|
|Before|After|
|-|-|
|||
-->
## Tests
<!-- Explain how you tested your development -->
- Step 1
- Step 2
- Step ...
-
## Checklist
- [ ] I have read through [CONTRIBUTING.md](https://github.com/element-hq/element-call/blob/livekit/CONTRIBUTING.md).
- [ ] Pull request includes screenshots or videos if containing UI changes
- [ ] Tests written for new code (and old code if feasible).
- [ ] Linter and other CI checks pass.
- [ ] I have licensed the changes to Element by completing the [Contributor License Agreement (CLA)](https://cla-assistant.io/element-hq/element-call)

View File

@@ -1,7 +1,16 @@
name: Prevent blocked
on:
# zizmor: ignore[dangerous-triggers]
# Reason: This workflow does not checkout code or use secrets.
# It only reads labels to set a failure status on the PR.
pull_request_target:
types: [opened, labeled, unlabeled, synchronize]
permissions:
pull-requests: read
# Required to fail the check on the PR
statuses: write
jobs:
prevent-blocked:
name: Prevent blocked

View File

@@ -20,10 +20,13 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write # required to upload release asset
packages: write
packages: write # needed for publishing packages to GHCR
id-token: write # needed for login into tailscale with GitHub OIDC Token
steps:
- name: Check it out
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: 📥 Download artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -34,26 +37,64 @@ jobs:
path: dist
- name: Log in to container registry
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Connect to Tailscale
uses: tailscale/github-action@306e68a486fd2350f2bfc3b19fcd143891a4a2d8 # v4
if: github.event_name != 'pull_request'
with:
oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
audience: ${{ secrets.TS_AUDIENCE }}
tags: tag:github-actions
- name: Compute vault jwt role name
id: vault-jwt-role
if: github.event_name != 'pull_request'
run: |
echo "role_name=github_service_management_$( echo "${{ github.repository }}" | sed -r 's|[/-]|_|g')" | tee -a "$GITHUB_OUTPUT"
- name: Get team registry token
id: import-secrets
uses: hashicorp/vault-action@4c06c5ccf5c0761b6029f56cfb1dcf5565918a3b # v3
if: github.event_name != 'pull_request'
with:
url: https://vault.infra.ci.i.element.dev
role: ${{ steps.vault-jwt-role.outputs.role_name }}
path: service-management/github-actions
jwtGithubAudience: https://vault.infra.ci.i.element.dev
method: jwt
secrets: |
services/voip-repositories/secret/data/oci.element.io username | OCI_USERNAME ;
services/voip-repositories/secret/data/oci.element.io password | OCI_PASSWORD ;
- name: Login to oci.element.io Registry
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
if: github.event_name != 'pull_request'
with:
registry: oci-push.vpn.infra.element.io
username: ${{ steps.import-secrets.outputs.OCI_USERNAME }}
password: ${{ steps.import-secrets.outputs.OCI_PASSWORD }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@318604b99e75e41977312d83839a89be02ca4893 # v5.9.0
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: ${{ inputs.docker_tags}}
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
oci-push.vpn.infra.element.io/element-call
tags: ${{ inputs.docker_tags }}
labels: |
org.opencontainers.image.licenses=AGPL-3.0-only OR LicenseRef-Element-Commercial
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Build and push Docker image
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
with:
context: .
platforms: linux/amd64,linux/arm64

View File

@@ -7,7 +7,7 @@ on:
type: string
package:
type: string # This would ideally be a `choice` type, but that isn't supported yet
description: The package type to be built. Must be one of 'full' or 'embedded'
description: The package type to be built. Must be one of 'full', 'embedded', or 'sdk'
required: true
build_mode:
type: string # This would ideally be a `choice` type, but that isn't supported yet
@@ -33,6 +33,8 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Enable Corepack
run: corepack enable
- name: Yarn cache
@@ -43,7 +45,7 @@ jobs:
- name: Install dependencies
run: "yarn install --immutable"
- name: Build Element Call
run: ${{ format('yarn run build:{0}:{1}', inputs.package, inputs.build_mode) }}
run: yarn run build:"$PACKAGE":"$BUILD_MODE"
env:
SENTRY_ORG: ${{ secrets.SENTRY_ORG }}
SENTRY_PROJECT: ${{ secrets.SENTRY_PROJECT }}
@@ -52,6 +54,8 @@ jobs:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
VITE_APP_VERSION: ${{ inputs.vite_app_version }}
NODE_OPTIONS: "--max-old-space-size=4096"
PACKAGE: ${{ inputs.package }}
BUILD_MODE: ${{ inputs.build_mode }}
- name: Upload Artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:

View File

@@ -49,7 +49,9 @@ jobs:
permissions:
contents: write
packages: write
id-token: write
uses: ./.github/workflows/build-and-publish-docker.yaml
secrets: inherit
with:
artifact_run_id: ${{ github.run_id }}
docker_tags: |
@@ -69,3 +71,17 @@ jobs:
SENTRY_URL: ${{ secrets.SENTRY_URL }}
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
build_sdk_element_call:
# Use the embedded package vite build
uses: ./.github/workflows/build-element-call.yaml
with:
package: sdk
vite_app_version: ${{ github.event.release.tag_name || github.sha }}
build_mode: ${{ github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'development build') && 'development' || 'production' }}
secrets:
SENTRY_ORG: ${{ secrets.SENTRY_ORG }}
SENTRY_PROJECT: ${{ secrets.SENTRY_PROJECT }}
SENTRY_URL: ${{ secrets.SENTRY_URL }}
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

View File

@@ -1,8 +1,16 @@
name: PR changelog label
on:
# zizmor: ignore[dangerous-triggers]
# This is safe because we do not use actions/checkout or execute untrusted code.
# Using pull_request_target is necessary to allow status writes for PRs from forks.
pull_request_target:
types: [labeled, unlabeled, opened]
permissions:
pull-requests: read
statuses: write
jobs:
pr-changelog-label:
runs-on: ubuntu-latest

View File

@@ -14,6 +14,10 @@ on:
deployment_ref:
required: true
type: string
package:
required: true
type: string
description: Which package to deploy - 'full', 'embedded', or 'sdk'
artifact_run_id:
required: false
type: string
@@ -50,7 +54,7 @@ jobs:
with:
github-token: ${{ secrets.ELEMENT_BOT_TOKEN }}
run-id: ${{ inputs.artifact_run_id }}
name: build-output-full
name: build-output-${{ inputs.package }}
path: webapp
- name: Add redirects file
@@ -58,15 +62,23 @@ jobs:
run: curl -s https://raw.githubusercontent.com/element-hq/element-call/main/config/netlify_redirects > webapp/_redirects
- name: Add config file
run: curl -s "https://raw.githubusercontent.com/${{ inputs.pr_head_full_name }}/${{ inputs.pr_head_ref }}/config/config_netlify_preview.json" > webapp/config.json
run: |
if [ "${INPUTS_PACKAGE}" = "full" ]; then
curl -s "https://raw.githubusercontent.com/${INPUTS_PR_HEAD_FULL_NAME}/${INPUTS_PR_HEAD_REF}/config/config_netlify_preview.json" > webapp/config.json
else
curl -s "https://raw.githubusercontent.com/${INPUTS_PR_HEAD_FULL_NAME}/${INPUTS_PR_HEAD_REF}/config/config_netlify_preview_sdk.json" > webapp/config.json
fi
env:
INPUTS_PACKAGE: ${{ inputs.package }}
INPUTS_PR_HEAD_FULL_NAME: ${{ inputs.pr_head_full_name }}
INPUTS_PR_HEAD_REF: ${{ inputs.pr_head_ref }}
- name: ☁️ Deploy to Netlify
id: netlify
uses: nwtgck/actions-netlify@4cbaf4c08f1a7bfa537d6113472ef4424e4eb654 # v3.0
with:
publish-dir: webapp
deploy-message: "Deploy from GitHub Actions"
alias: pr${{ inputs.pr_number }}
alias: ${{ inputs.package == 'sdk' && format('pr{0}-sdk', inputs.pr_number) || format('pr{0}', inputs.pr_number) }}
env:
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}

View File

@@ -8,6 +8,8 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Enable Corepack
run: corepack enable
- name: Yarn cache

View File

@@ -1,5 +1,7 @@
name: Deploy previews for PRs
on:
# zizmor: ignore[dangerous-triggers]
# Reason: This is now restricted to internal PRs only using the 'if' condition below.
workflow_run:
workflows: ["Build"]
types:
@@ -7,7 +9,14 @@ on:
jobs:
prdetails:
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.event == 'pull_request' }}
# Logic:
# 1. Build must be successful
# 2. Event must be a pull_request
# 3. Head repository must be the SAME as the base repository (No Forks!)
if: >
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'pull_request' &&
github.event.workflow_run.head_repository.full_name == github.repository
runs-on: ubuntu-latest
outputs:
pr_number: ${{ steps.prdetails.outputs.pr_id }}
@@ -20,7 +29,7 @@ jobs:
owner: ${{ github.event.workflow_run.head_repository.owner.login }}
branch: ${{ github.event.workflow_run.head_branch }}
netlify:
netlify-full:
needs: prdetails
permissions:
deployments: write
@@ -31,6 +40,24 @@ jobs:
pr_head_full_name: ${{ github.event.workflow_run.head_repository.full_name }}
pr_head_ref: ${{ needs.prdetails.outputs.pr_data_json && fromJSON(needs.prdetails.outputs.pr_data_json).head.ref }}
deployment_ref: ${{ needs.prdetails.outputs.pr_data_json && fromJSON(needs.prdetails.outputs.pr_data_json).head.sha || github.ref || github.head_ref }}
package: full
secrets:
ELEMENT_BOT_TOKEN: ${{ secrets.ELEMENT_BOT_TOKEN }}
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
netlify-sdk:
needs: prdetails
permissions:
deployments: write
uses: ./.github/workflows/deploy-to-netlify.yaml
with:
artifact_run_id: ${{ github.event.workflow_run.id || github.run_id }}
pr_number: ${{ needs.prdetails.outputs.pr_number }}
pr_head_full_name: ${{ github.event.workflow_run.head_repository.full_name }}
pr_head_ref: ${{ needs.prdetails.outputs.pr_data_json && fromJSON(needs.prdetails.outputs.pr_data_json).head.ref }}
deployment_ref: ${{ needs.prdetails.outputs.pr_data_json && fromJSON(needs.prdetails.outputs.pr_data_json).head.sha || github.ref || github.head_ref }}
package: sdk
secrets:
ELEMENT_BOT_TOKEN: ${{ secrets.ELEMENT_BOT_TOKEN }}
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
@@ -42,7 +69,9 @@ jobs:
permissions:
contents: write
packages: write
id-token: write
uses: ./.github/workflows/build-and-publish-docker.yaml
secrets: inherit
with:
artifact_run_id: ${{ github.event.workflow_run.id || github.run_id }}
docker_tags: |

View File

@@ -22,8 +22,18 @@ jobs:
TAG: ${{ steps.tag.outputs.TAG }}
steps:
- name: Calculate VERSION
# We should only use the hard coded test value for a dry run
run: echo "VERSION=${{ github.event_name == 'release' && github.event.release.tag_name || 'v0.0.0-pre.0' }}" >> "$GITHUB_ENV"
# Safely store dynamic values in environment variables
# to prevent shell injection (template-injection)
run: |
# The logic is executed within the shell using the env variables
if [ "$EVENT_NAME" = "release" ]; then
echo "VERSION=$RELEASE_TAG" >> "$GITHUB_ENV"
else
echo "VERSION=v0.0.0-pre.0" >> "$GITHUB_ENV"
fi
env:
RELEASE_TAG: ${{ github.event.release.tag_name }}
EVENT_NAME: ${{ github.event_name }}
- id: dry_run
name: Set DRY_RUN
# We perform a dry run for all events except releases.
@@ -71,7 +81,9 @@ jobs:
contents: write # required to upload release asset
steps:
- name: Determine filename
run: echo "FILENAME_PREFIX=element-call-embedded-${{ needs.versioning.outputs.UNPREFIXED_VERSION }}" >> "$GITHUB_ENV"
run: echo "FILENAME_PREFIX=element-call-embedded-${NEEDS_VERSIONING_OUTPUTS_UNPREFIXED_VERSION}" >> "$GITHUB_ENV"
env:
NEEDS_VERSIONING_OUTPUTS_UNPREFIXED_VERSION: ${{ needs.versioning.outputs.UNPREFIXED_VERSION }}
- name: 📥 Download built element-call artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
@@ -80,9 +92,9 @@ jobs:
name: build-output-embedded
path: ${{ env.FILENAME_PREFIX}}
- name: Create Tarball
run: tar --numeric-owner -cvzf ${{ env.FILENAME_PREFIX }}.tar.gz ${{ env.FILENAME_PREFIX }}
run: tar --numeric-owner -cvzf ${FILENAME_PREFIX}.tar.gz ${FILENAME_PREFIX}
- name: Create Checksum
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
run: find ${FILENAME_PREFIX} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${FILENAME_PREFIX}.sha256
- name: Upload
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
@@ -104,6 +116,8 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: 📥 Download built element-call artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -123,13 +137,16 @@ jobs:
- name: Publish npm
working-directory: embedded/web
run: |
npm version ${{ needs.versioning.outputs.PREFIXED_VERSION }} --no-git-tag-version
npm version ${NEEDS_VERSIONING_OUTPUTS_PREFIXED_VERSION} --no-git-tag-version
echo "ARTIFACT_VERSION=$(jq '.version' --raw-output package.json)" >> "$GITHUB_ENV"
npm publish --provenance --access public --tag ${{ needs.versioning.outputs.TAG }} ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
npm publish --provenance --access public --tag ${NEEDS_VERSIONING_OUTPUTS_TAG} ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
env:
NEEDS_VERSIONING_OUTPUTS_PREFIXED_VERSION: ${{ needs.versioning.outputs.PREFIXED_VERSION }}
NEEDS_VERSIONING_OUTPUTS_TAG: ${{ needs.versioning.outputs.TAG }}
- id: artifact_version
name: Output artifact version
run: echo "ARTIFACT_VERSION=${{env.ARTIFACT_VERSION}}" >> "$GITHUB_OUTPUT"
run: echo "ARTIFACT_VERSION=${ARTIFACT_VERSION}" >> "$GITHUB_OUTPUT"
publish_android:
needs: [build_element_call, versioning]
@@ -143,6 +160,8 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: 📥 Download built element-call artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -153,7 +172,7 @@ jobs:
path: embedded/android/lib/src/main/assets/element-call
- name: ☕️ Setup Java
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4
uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4
with:
distribution: "temurin"
java-version: "17"
@@ -161,16 +180,19 @@ jobs:
- name: Get artifact version
# Anything that is not a final release will be tagged as a snapshot
run: |
if [[ "${{ needs.versioning.outputs.TAG }}" == "latest" ]]; then
echo "ARTIFACT_VERSION=${{ needs.versioning.outputs.UNPREFIXED_VERSION }}" >> "$GITHUB_ENV"
elif [[ "${{ needs.versioning.outputs.TAG }}" == "rc" ]]; then
echo "ARTIFACT_VERSION=${{ needs.versioning.outputs.UNPREFIXED_VERSION }}" >> "$GITHUB_ENV"
if [[ "${NEEDS_VERSIONING_OUTPUTS_TAG}" == "latest" ]]; then
echo "ARTIFACT_VERSION=${NEEDS_VERSIONING_OUTPUTS_UNPREFIXED_VERSION}" >> "$GITHUB_ENV"
elif [[ "${NEEDS_VERSIONING_OUTPUTS_TAG}" == "rc" ]]; then
echo "ARTIFACT_VERSION=${NEEDS_VERSIONING_OUTPUTS_UNPREFIXED_VERSION}" >> "$GITHUB_ENV"
else
echo "ARTIFACT_VERSION=${{ needs.versioning.outputs.UNPREFIXED_VERSION }}-SNAPSHOT" >> "$GITHUB_ENV"
echo "ARTIFACT_VERSION=${NEEDS_VERSIONING_OUTPUTS_UNPREFIXED_VERSION}-SNAPSHOT" >> "$GITHUB_ENV"
fi
env:
NEEDS_VERSIONING_OUTPUTS_TAG: ${{ needs.versioning.outputs.TAG }}
NEEDS_VERSIONING_OUTPUTS_UNPREFIXED_VERSION: ${{ needs.versioning.outputs.UNPREFIXED_VERSION }}
- name: Set version string
run: sed -i "s/0.0.0/${{ env.ARTIFACT_VERSION }}/g" embedded/android/lib/src/main/kotlin/io/element/android/call/embedded/Version.kt
run: sed -i "s/0.0.0/${ARTIFACT_VERSION}/g" embedded/android/lib/src/main/kotlin/io/element/android/call/embedded/Version.kt
- name: Publish AAR
working-directory: embedded/android
@@ -184,7 +206,7 @@ jobs:
- id: artifact_version
name: Output artifact version
run: echo "ARTIFACT_VERSION=${{env.ARTIFACT_VERSION}}" >> "$GITHUB_OUTPUT"
run: echo "ARTIFACT_VERSION=${ARTIFACT_VERSION}" >> "$GITHUB_OUTPUT"
publish_ios:
needs: [build_element_call, versioning]
@@ -200,6 +222,7 @@ jobs:
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
path: element-call
persist-credentials: false
- name: 📥 Download built element-call artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -215,15 +238,18 @@ jobs:
repository: element-hq/element-call-swift
path: element-call-swift
token: ${{ secrets.SWIFT_RELEASE_TOKEN }}
persist-credentials: false
- name: Copy files
run: rsync -a --delete --exclude .git element-call/embedded/ios/ element-call-swift
- name: Get artifact version
run: echo "ARTIFACT_VERSION=${{ needs.versioning.outputs.UNPREFIXED_VERSION }}" >> "$GITHUB_ENV"
run: echo "ARTIFACT_VERSION=${NEEDS_VERSIONING_OUTPUTS_UNPREFIXED_VERSION}" >> "$GITHUB_ENV"
env:
NEEDS_VERSIONING_OUTPUTS_UNPREFIXED_VERSION: ${{ needs.versioning.outputs.UNPREFIXED_VERSION }}
- name: Set version string
run: sed -i "s/0.0.0/${{ env.ARTIFACT_VERSION }}/g" element-call-swift/Sources/EmbeddedElementCall/EmbeddedElementCall.swift
run: sed -i "s/0.0.0/${ARTIFACT_VERSION}/g" element-call-swift/Sources/EmbeddedElementCall/EmbeddedElementCall.swift
- name: Test build
working-directory: element-call-swift
@@ -235,17 +261,22 @@ jobs:
git config --global user.email "ci@element.io"
git config --global user.name "Element CI"
git add -A
git commit -am "Release ${{ needs.versioning.outputs.PREFIXED_VERSION }}"
git tag -a ${{ env.ARTIFACT_VERSION }} -m "${{ github.event.release.html_url }}"
git commit -am "Release ${NEEDS_VERSIONING_OUTPUTS_PREFIXED_VERSION}"
git tag -a ${ARTIFACT_VERSION} -m "${GITHUB_EVENT_RELEASE_HTML_URL}"
env:
NEEDS_VERSIONING_OUTPUTS_PREFIXED_VERSION: ${{ needs.versioning.outputs.PREFIXED_VERSION }}
GITHUB_EVENT_RELEASE_HTML_URL: ${{ github.event.release.html_url }}
- name: Push
working-directory: element-call-swift
run: |
git push --tags ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
git push "https://x-access-token:${SWIFT_RELEASE_TOKEN}@github.com/element-hq/element-call-swift.git" --tags ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
env:
SWIFT_RELEASE_TOKEN: ${{ secrets.SWIFT_RELEASE_TOKEN }}
- id: artifact_version
name: Output artifact version
run: echo "ARTIFACT_VERSION=${{env.ARTIFACT_VERSION}}" >> "$GITHUB_OUTPUT"
run: echo "ARTIFACT_VERSION=${ARTIFACT_VERSION}" >> "$GITHUB_OUTPUT"
release_notes:
needs: [versioning, publish_npm, publish_android, publish_ios]
@@ -257,9 +288,13 @@ jobs:
steps:
- name: Log versions
run: |
echo "NPM: ${{ needs.publish_npm.outputs.ARTIFACT_VERSION }}"
echo "Android: ${{ needs.publish_android.outputs.ARTIFACT_VERSION }}"
echo "iOS: ${{ needs.publish_ios.outputs.ARTIFACT_VERSION }}"
echo "NPM: ${NEEDS_PUBLISH_NPM_OUTPUTS_ARTIFACT_VERSION}"
echo "Android: ${NEEDS_PUBLISH_ANDROID_OUTPUTS_ARTIFACT_VERSION}"
echo "iOS: ${NEEDS_PUBLISH_IOS_OUTPUTS_ARTIFACT_VERSION}"
env:
NEEDS_PUBLISH_NPM_OUTPUTS_ARTIFACT_VERSION: ${{ needs.publish_npm.outputs.ARTIFACT_VERSION }}
NEEDS_PUBLISH_ANDROID_OUTPUTS_ARTIFACT_VERSION: ${{ needs.publish_android.outputs.ARTIFACT_VERSION }}
NEEDS_PUBLISH_IOS_OUTPUTS_ARTIFACT_VERSION: ${{ needs.publish_ios.outputs.ARTIFACT_VERSION }}
- name: Add release notes
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2

View File

@@ -38,9 +38,9 @@ jobs:
name: build-output-full
path: ${{ env.FILENAME_PREFIX }}
- name: Create Tarball
run: tar --numeric-owner --transform "s/dist/${{ env.FILENAME_PREFIX }}/" -cvzf ${{ env.FILENAME_PREFIX }}.tar.gz ${{ env.FILENAME_PREFIX }}
run: tar --numeric-owner --transform "s/dist/${FILENAME_PREFIX}/" -cvzf ${FILENAME_PREFIX}.tar.gz ${FILENAME_PREFIX}
- name: Create Checksum
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
run: find ${FILENAME_PREFIX} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${FILENAME_PREFIX}.sha256
- name: Upload
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with:
@@ -55,7 +55,9 @@ jobs:
permissions:
contents: write
packages: write
id-token: write
uses: ./.github/workflows/build-and-publish-docker.yaml
secrets: inherit
with:
artifact_run_id: ${{ github.event.workflow_run.id || github.run_id }}
docker_tags: |

View File

@@ -10,6 +10,8 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Enable Corepack
run: corepack enable
- name: Yarn cache
@@ -22,7 +24,7 @@ jobs:
- name: Vitest
run: "yarn run test:coverage"
- name: Upload to codecov
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
@@ -34,6 +36,8 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Enable Corepack
run: corepack enable
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4

View File

@@ -14,6 +14,8 @@ jobs:
steps:
- name: Checkout the code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Enable Corepack
run: corepack enable
@@ -42,7 +44,7 @@ jobs:
- name: Create Pull Request
id: cpr
uses: peter-evans/create-pull-request@84ae59a2cdc2258d6fa0732dd66352dddae2a412 # v7.0.9
uses: peter-evans/create-pull-request@22a9089034f40e5a961c8808d113e2c98fb63676 # v7.0.11
with:
token: ${{ secrets.ELEMENT_BOT_TOKEN }}
branch: actions/localazy-download

View File

@@ -15,6 +15,8 @@ jobs:
steps:
- name: Checkout the code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Upload
uses: localazy/upload@27e6b5c0fddf4551596b42226b1c24124335d24a # v1

23
.github/workflows/zizmor.yml vendored Normal file
View File

@@ -0,0 +1,23 @@
name: GitHub Actions Security Analysis with zizmor 🌈
on:
push:
branches: ["livekit", "full-mesh"]
pull_request: {}
permissions: {}
jobs:
zizmor:
name: Run zizmor 🌈
runs-on: ubuntu-latest
permissions:
security-events: write # Required for upload-sarif (used by zizmor-action) to upload SARIF files.
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Run zizmor 🌈
uses: zizmorcore/zizmor-action@71321a20a9ded102f6e9ce5718a2fcec2c4f70d8 # v0.5.2

View File

@@ -3,6 +3,7 @@
[![Chat](https://img.shields.io/matrix/webrtc:matrix.org)](https://matrix.to/#/#webrtc:matrix.org)
[![Localazy](https://img.shields.io/endpoint?url=https%3A%2F%2Fconnect.localazy.com%2Fstatus%2Felement-call%2Fdata%3Fcontent%3Dall%26title%3Dlocalazy%26logo%3Dtrue)](https://localazy.com/p/element-call)
[![License](https://img.shields.io/github/license/element-hq/element-call)](LICENSE-AGPL-3.0)
[![Codecov](https://img.shields.io/codecov/c/github/element-hq/element-call)](https://app.codecov.io/gh/element-hq/element-call)
[🎬 Live Demo 🎬](https://call.element.io)

View File

@@ -38,6 +38,8 @@ experimental_features:
# MSC4222 needed for syncv2 state_after. This allow clients to
# correctly track the state of the room.
msc4222_enabled: true
# sticky events for MatrixRTC user state
msc4354_enabled: true
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no

View File

@@ -38,7 +38,7 @@ experimental_features:
# MSC4222 needed for syncv2 state_after. This allow clients to
# correctly track the state of the room.
msc4222_enabled: true
# sticky events for matrixRTC user state
# sticky events for MatrixRTC user state
msc4354_enabled: true
# The maximum allowed duration by which sent events can be delayed, as

View File

@@ -181,7 +181,10 @@ server {
}
# Convenience reverse proxy for the call.m.localhost domain to yarn dev --host
# Convenience reverse proxy for the call.m.localhost domain to element call
# running on the host either via
# - yarn dev --host or
# - falling back to http (the element call docker container)
server {
listen 80;
listen [::]:80;
@@ -197,7 +200,7 @@ server {
ssl_certificate /root/ssl/cert.pem;
ssl_certificate_key /root/ssl/key.pem;
# 1. Attempt HTTPS first
location ^~ / {
proxy_set_header Host $host;
@@ -208,8 +211,23 @@ server {
proxy_pass https://host.docker.internal:3000;
proxy_ssl_verify off;
# 2. Redirect specific errors (e.g., 502 Bad Gateway or 504 Timeout)
# to the named fallback location
error_page 502 503 504 = @http_fallback;
}
# 3. Fallback location using HTTP
location @http_fallback {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://host.docker.internal:8080;
}
error_page 500 502 503 504 /50x.html;
}

View File

@@ -38,6 +38,8 @@ experimental_features:
# MSC4222 needed for syncv2 state_after. This allow clients to
# correctly track the state of the room.
msc4222_enabled: true
# sticky events for MatrixRTC user state
msc4354_enabled: true
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no

View File

@@ -38,6 +38,8 @@ experimental_features:
# MSC4222 needed for syncv2 state_after. This allow clients to
# correctly track the state of the room.
msc4222_enabled: true
# sticky events for MatrixRTC user state
msc4354_enabled: true
# The maximum allowed duration by which sent events can be delayed, as
# per MSC4140. Must be a positive value if set. Defaults to no

View File

@@ -15,6 +15,5 @@
"delayed_leave_event_delay_ms": 18000,
"delayed_leave_event_restart_ms": 4000,
"network_error_retry_ms": 100
},
"trust_localhost_for_mute_state": true
}
}

View File

@@ -0,0 +1,16 @@
{
"default_server_config": {
"m.homeserver": {
"base_url": "https://call-unstable.ems.host",
"server_name": "call-unstable.ems.host"
}
},
"ssla": "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
"matrix_rtc_session": {
"wait_for_key_rotation_ms": 3000,
"membership_event_expiry_ms": 180000000,
"delayed_leave_event_delay_ms": 18000,
"delayed_leave_event_restart_ms": 4000,
"network_error_retry_ms": 100
}
}

View File

@@ -3,7 +3,7 @@ networks:
services:
auth-service:
image: ghcr.io/element-hq/lk-jwt-service:latest-ci
image: ghcr.io/element-hq/lk-jwt-service:sha-f8ddd00
pull_policy: always
hostname: auth-server
environment:
@@ -25,7 +25,7 @@ services:
- ecbackend
auth-service-1:
image: ghcr.io/element-hq/lk-jwt-service:latest-ci
image: ghcr.io/element-hq/lk-jwt-service:sha-f8ddd00
pull_policy: always
hostname: auth-server-1
environment:
@@ -47,7 +47,7 @@ services:
- ecbackend
livekit:
image: livekit/livekit-server:v1.9.4
image: livekit/livekit-server:v1.9.11
pull_policy: always
hostname: livekit-sfu
command: --dev --config /etc/livekit.yaml
@@ -67,7 +67,7 @@ services:
- ecbackend
livekit-1:
image: livekit/livekit-server:v1.9.4
image: livekit/livekit-server:v1.9.11
pull_policy: always
hostname: livekit-sfu-1
command: --dev --config /etc/livekit.yaml
@@ -88,7 +88,7 @@ services:
synapse:
hostname: homeserver
image: docker.io/matrixdotorg/synapse:latest
image: ghcr.io/element-hq/synapse:latest
pull_policy: always
environment:
- SYNAPSE_CONFIG_PATH=/data/cfg/homeserver.yaml
@@ -106,7 +106,7 @@ services:
synapse-1:
hostname: homeserver-1
image: docker.io/matrixdotorg/synapse:latest
image: ghcr.io/element-hq/synapse:latest
pull_policy: always
environment:
- SYNAPSE_CONFIG_PATH=/data/cfg/homeserver.yaml

View File

@@ -152,6 +152,40 @@ handle {
}
```
Using Haproxy, you can achieve this by:
```
# Frontend
# Match /livekit/sfu/ path
acl is_sfu path_beg -i /livekit/sfu/
use_backend sfu_backend if is_sfu matrixrtc_domain
acl is_mxrtc_auth path_beg -i /sfu/get
use_backend mxrtc_auth_backend if is_mxrtc_auth matrixrtc_domain
# Backend
## MatrixRTC backend
backend sfu_backend
server livekit 127.0.0.1:7880
http-request set-path %[path,regsub(^/livekit/sfu/,/)]
http-request set-header Host %[req.hdr(host)]
timeout server 120s
# WebSocket support
option forwardfor
option http-server-close
option http-buffer-request
backend mxrtc_auth_backend
server sfu 127.0.0.1:8070
http-request set-header Host %[req.hdr(host)]
timeout server 120s
# WebSocket support
option forwardfor
option http-server-close
option http-buffer-request
```
#### MatrixRTC backend announcement
> [!IMPORTANT]

View File

@@ -46,32 +46,32 @@ possible to support encryption.
These parameters are relevant to both [widget](./embedded-standalone.md) and [standalone](./embedded-standalone.md) modes:
| Name | Values | Required for widget | Required for SPA | Description |
| ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `intent` | `start_call`, `join_existing`, `start_call_dm`, `join_existing_dm. | No, defaults to `start_call` | No, defaults to `start_call` | The intent is a special url parameter that defines the defaults for all the other parameters. In most cases it should be enough to only set the intent to setup element-call. |
| `allowIceFallback` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Allows use of fallback STUN servers for ICE if the user's homeserver doesnt provide any. |
| `analyticsID` (deprecated: use `posthogUserId` instead) | Posthog analytics ID | No | No | Available only with user's consent for sharing telemetry in Element Web. |
| `appPrompt` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Prompts the user to launch the native mobile app upon entering a room, applicable only on Android and iOS, and must be enabled in config. |
| `confineToRoom` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Keeps the user confined to the current call/room. |
| `displayName` | | No | No | Display name used for auto-registration. |
| `enableE2EE` (deprecated) | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Legacy flag to enable end-to-end encryption, not used in the `livekit` branch. |
| `fontScale` | A decimal number such as `0.9` | No, defaults to `1.0` | No, defaults to `1.0` | Factor by which to scale the interface's font size. |
| `fonts` | | No | No | Defines the font(s) used by the interface. Multiple font parameters can be specified: `?font=font-one&font=font-two...`. |
| `header` | `none`, `standard` or `app_bar` | No, defaults to `standard` | No, defaults to `standard` | The style of headers to show. `standard` is the default arrangement, `none` hides the header entirely, and `app_bar` produces a header with a back button like you might see in mobile apps. The callback for the back button is `window.controls.onBackButtonPressed`. |
| `hideScreensharing` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Hides the screen-sharing button. |
| `homeserver` | | Not applicable | No | Homeserver for registering a new (guest) user, configures non-default guest user server when creating a spa link. |
| `lang` | [BCP 47](https://www.rfc-editor.org/info/bcp47) code | No | No | The language the app should use. |
| `password` | | No | No | E2EE password when using a shared secret. (For individual sender keys in embedded mode this is not required.) |
| `perParticipantE2EE` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Enables per participant encryption with Keys exchanged over encrypted matrix room messages. |
| `controlledAudioDevices` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the [global JS controls for audio devices](./controls.md#audio-devices) should be enabled, allowing the list of audio devices to be controlled by the app hosting Element Call. |
| `roomId` | [Matrix Room ID](https://spec.matrix.org/v1.12/appendices/#room-ids) | Yes | No | Anything about what room we're pointed to should be from useRoomIdentifier which parses the path and resolves alias with respect to the default server name, however roomId is an exception as we need the room ID in embedded widget mode, and not the room alias (or even the via params because we are not trying to join it). This is also not validated, where it is in `useRoomIdentifier()`. |
| `showControls` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Displays controls like mute, screen-share, invite, and hangup buttons during a call. |
| `skipLobby` (deprecated: use `intent` instead) | `true` or `false` | No. If `intent` is explicitly `start_call` then defaults to `true`. Otherwise defaults to `false` | No, defaults to `false` | Skips the lobby to join a call directly, can be combined with preload in widget. When `true` the audio and video inputs will be muted by default. (This means there currently is no way to start without muted video if one wants to skip the lobby. Also not in widget mode.) |
| `theme` | One of: `light`, `dark`, `light-high-contrast`, `dark-high-contrast` | No, defaults to `dark` | No, defaults to `dark` | UI theme to use. |
| `viaServers` | Comma separated list of [Matrix Server Names](https://spec.matrix.org/v1.12/appendices/#server-name) | Not applicable | No | Homeserver for joining a room, non-empty value required for rooms not on the users default homeserver. |
| `sendNotificationType` | `ring` or `notification` | No | No | Will send a "ring" or "notification" `m.rtc.notification` event if the user is the first one in the call. |
| `autoLeaveWhenOthersLeft` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the app should automatically leave the call when there is no one left in the call. |
| `waitForCallPickup` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | When sending a notification, show UI that the app is awaiting an answer, play a dial tone, and (in widget mode) auto-close the widget once the notification expires. |
| Name | Values | Required for widget | Required for SPA | Description |
| ---------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `intent` | `start_call`, `join_existing`, `start_call_dm`, `join_existing_dm. | No, defaults to `start_call` | No, defaults to `start_call` | The intent is a special url parameter that defines the defaults for all the other parameters. In most cases it should be enough to only set the intent to setup element-call. |
| `allowIceFallback` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Allows use of fallback STUN servers for ICE if the user's homeserver doesnt provide any. |
| `posthogUserId` | Posthog analytics ID | No | No | Available only with user's consent for sharing telemetry in Element Web. |
| `appPrompt` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Prompts the user to launch the native mobile app upon entering a room, applicable only on Android and iOS, and must be enabled in config. |
| `confineToRoom` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Keeps the user confined to the current call/room. |
| `displayName` | | No | No | Display name used for auto-registration. |
| `enableE2EE` (deprecated) | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Legacy flag to enable end-to-end encryption, not used in the `livekit` branch. |
| `fontScale` | A decimal number such as `0.9` | No, defaults to `1.0` | No, defaults to `1.0` | Factor by which to scale the interface's font size. |
| `fonts` | | No | No | Defines the font(s) used by the interface. Multiple font parameters can be specified: `?font=font-one&font=font-two...`. |
| `header` | `none`, `standard` or `app_bar` | No, defaults to `standard` | No, defaults to `standard` | The style of headers to show. `standard` is the default arrangement, `none` hides the header entirely, and `app_bar` produces a header with a back button like you might see in mobile apps. The callback for the back button is `window.controls.onBackButtonPressed`. |
| `hideScreensharing` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Hides the screen-sharing button. |
| `homeserver` | | Not applicable | No | Homeserver for registering a new (guest) user, configures non-default guest user server when creating a spa link. |
| `lang` | [BCP 47](https://www.rfc-editor.org/info/bcp47) code | No | No | The language the app should use. |
| `password` | | No | No | E2EE password when using a shared secret. (For individual sender keys in embedded mode this is not required.) |
| `perParticipantE2EE` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Enables per participant encryption with Keys exchanged over encrypted matrix room messages. |
| `controlledAudioDevices` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the [global JS controls for audio devices](./controls.md#audio-devices) should be enabled, allowing the list of audio devices to be controlled by the app hosting Element Call. |
| `roomId` | [Matrix Room ID](https://spec.matrix.org/v1.12/appendices/#room-ids) | Yes | No | Anything about what room we're pointed to should be from useRoomIdentifier which parses the path and resolves alias with respect to the default server name, however roomId is an exception as we need the room ID in embedded widget mode, and not the room alias (or even the via params because we are not trying to join it). This is also not validated, where it is in `useRoomIdentifier()`. |
| `showControls` | `true` or `false` | No, defaults to `true` | No, defaults to `true` | Displays controls like mute, screen-share, invite, and hangup buttons during a call. |
| `skipLobby` (deprecated: use `intent` instead) | `true` or `false` | No. If `intent` is explicitly `start_call` then defaults to `true`. Otherwise defaults to `false` | No, defaults to `false` | Skips the lobby to join a call directly, can be combined with preload in widget. When `true` the audio and video inputs will be muted by default. (This means there currently is no way to start without muted video if one wants to skip the lobby. Also not in widget mode.) |
| `theme` | One of: `light`, `dark`, `light-high-contrast`, `dark-high-contrast` | No, defaults to `dark` | No, defaults to `dark` | UI theme to use. |
| `viaServers` | Comma separated list of [Matrix Server Names](https://spec.matrix.org/v1.12/appendices/#server-name) | Not applicable | No | Homeserver for joining a room, non-empty value required for rooms not on the users default homeserver. |
| `sendNotificationType` | `ring` or `notification` | No | No | Will send a "ring" or "notification" `m.rtc.notification` event if the user is the first one in the call. |
| `autoLeaveWhenOthersLeft` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | Whether the app should automatically leave the call when there is no one left in the call. |
| `waitForCallPickup` | `true` or `false` | No, defaults to `false` | No, defaults to `false` | When sending a notification, show UI that the app is awaiting an answer, play a dial tone, and (in widget mode) auto-close the widget once the notification expires. |
### Widget-only parameters

View File

@@ -2,11 +2,11 @@
# https://docs.gradle.org/current/userguide/platforms.html#sub::toml-dependencies-format
[versions]
android_gradle_plugin = "8.13.1"
android_gradle_plugin = "8.13.2"
[libraries]
android_gradle_plugin = { module = "com.android.tools.build:gradle", version.ref = "android_gradle_plugin" }
[plugins]
android_library = { id = "com.android.library", version.ref = "android_gradle_plugin" }
maven_publish = { id = "com.vanniktech.maven.publish", version = "0.35.0" }
maven_publish = { id = "com.vanniktech.maven.publish", version = "0.36.0" }

View File

@@ -1,6 +1,6 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.14.3-all.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.14.4-all.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME

View File

@@ -34,6 +34,12 @@ export default {
// then Knip will flag it as a false positive
// https://github.com/webpro-nl/knip/issues/766
"@vector-im/compound-web",
// Yarn plugins are allowed to depend on packages provided by the Yarn
// runtime. These shouldn't be listed in package.json, because plugins
// should work before Yarn even installs dependencies for the first time.
// https://yarnpkg.com/advanced/plugin-tutorial#what-does-a-plugin-look-like
"@yarnpkg/core",
"@yarnpkg/parsers",
"matrix-widget-api",
],
ignoreExportsUsedInFile: true,

View File

@@ -116,6 +116,7 @@
"matrix_rtc_transport_missing": "The server is not configured to work with {{brand}}. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",
"membership_manager": "Membership Manager Error",
"membership_manager_description": "The Membership Manager had to shut down. This is caused by many consequtive failed network requests.",
"no_matrix_2_authorization_service": "The authorization service for your media server (SFU) is out of date.",
"open_elsewhere": "Opened in another tab",
"open_elsewhere_description": "{{brand}} has been opened in another tab. If that doesn't sound right, try reloading the page.",
"room_creation_restricted": "Failed to create call",
@@ -248,12 +249,14 @@
"version": "{{productName}} version: {{version}}",
"video_tile": {
"always_show": "Always show",
"call_ended": "Call ended",
"calling": "Calling…",
"camera_starting": "Video loading...",
"change_fit_contain": "Fit to frame",
"collapse": "Collapse",
"expand": "Expand",
"mute_for_me": "Mute for me",
"muted_for_me": "Muted for me",
"screen_share_volume": "Screen share volume",
"volume": "Volume",
"waiting_for_media": "Waiting for media..."
}

View File

@@ -13,8 +13,9 @@
"build:embedded": "yarn build:full --config vite-embedded.config.js",
"build:embedded:production": "yarn build:embedded",
"build:embedded:development": "yarn build:embedded --mode development",
"build:sdk": "yarn build:full --config vite-sdk.config.js",
"build:sdk:development": "yarn build:sdk --mode development",
"build:sdk": "yarn build:full --config vite-sdk.config.js",
"build:sdk:production": "yarn build:sdk",
"serve": "vite preview",
"prettier:check": "prettier -c .",
"prettier:format": "prettier -w .",
@@ -42,12 +43,12 @@
"@codecov/vite-plugin": "^1.3.0",
"@fontsource/inconsolata": "^5.1.0",
"@fontsource/inter": "^5.1.0",
"@formatjs/intl-durationformat": "^0.9.0",
"@formatjs/intl-durationformat": "^0.10.0",
"@formatjs/intl-segmenter": "^11.7.3",
"@livekit/components-core": "^0.12.0",
"@livekit/components-react": "^2.0.0",
"@livekit/protocol": "^1.42.2",
"@livekit/track-processors": "^0.6.0 || ^0.7.1",
"@livekit/track-processors": "^0.7.1",
"@mediapipe/tasks-vision": "^0.10.18",
"@playwright/test": "^1.57.0",
"@radix-ui/react-dialog": "^1.0.4",
@@ -78,7 +79,7 @@
"@vector-im/compound-design-tokens": "^6.0.0",
"@vector-im/compound-web": "^8.0.0",
"@vitejs/plugin-react": "^4.0.1",
"@vitest/coverage-v8": "^3.0.0",
"@vitest/coverage-v8": "^4.0.18",
"babel-plugin-transform-vite-meta-env": "^1.0.3",
"classnames": "^2.3.1",
"copy-to-clipboard": "^3.3.3",
@@ -100,12 +101,12 @@
"i18next-browser-languagedetector": "^8.0.0",
"i18next-parser": "^9.1.0",
"jsdom": "^26.0.0",
"knip": "^5.27.2",
"knip": "^5.86.0",
"livekit-client": "^2.13.0",
"lodash-es": "^4.17.21",
"loglevel": "^1.9.1",
"matrix-js-sdk": "matrix-org/matrix-js-sdk#develop",
"matrix-widget-api": "^1.14.0",
"matrix-js-sdk": "matrix-org/matrix-js-sdk#6e3efef0c5f660df47cf00874927dec1c75cc3cf",
"matrix-widget-api": "^1.16.1",
"node-stdlib-browser": "^1.3.1",
"normalize.css": "^8.0.1",
"observable-hooks": "^4.2.3",
@@ -117,7 +118,7 @@
"qrcode": "^1.5.4",
"react": "19",
"react-dom": "19",
"react-i18next": "^16.0.0 <16.1.0",
"react-i18next": "^16.0.0 <16.6.0",
"react-router-dom": "^7.0.0",
"react-use-measure": "^2.1.1",
"rxjs": "^7.8.1",
@@ -125,18 +126,24 @@
"typescript": "^5.8.3",
"typescript-eslint-language-service": "^5.0.5",
"unique-names-generator": "^4.6.0",
"uuid": "^13.0.0",
"vaul": "^1.0.0",
"vite": "^7.0.0",
"vite": "^7.3.0",
"vite-plugin-generate-file": "^0.3.0",
"vite-plugin-html": "^3.2.2",
"vite-plugin-node-stdlib-browser": "^0.2.1",
"vite-plugin-svgr": "^4.0.0",
"vitest": "^3.0.0",
"vitest": "^4.0.18",
"vitest-axe": "^1.0.0-pre.3"
},
"resolutions": {
"@livekit/components-core/rxjs": "^7.8.1",
"@livekit/track-processors/@mediapipe/tasks-vision": "^0.10.18"
"@livekit/track-processors/@mediapipe/tasks-vision": "^0.10.18",
"minimatch": "^10.2.3",
"tar": "^7.5.11",
"glob": "^10.5.0",
"qs": "^6.14.1",
"js-yaml": "^4.1.1"
},
"packageManager": "yarn@4.7.0"
}

View File

@@ -1,5 +1,6 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -67,8 +68,6 @@ export default defineConfig({
firefoxUserPrefs: {
"permissions.default.microphone": 1,
"permissions.default.camera": 1,
// "media.navigator.streams.fake": true,
// "media.navigator.permission.disabled": true,
},
},
},

View File

@@ -7,6 +7,8 @@ Please see LICENSE in the repository root for full details.
import { expect, test } from "@playwright/test";
import { SpaHelpers } from "./spa-helpers.ts";
test("Sign up a new account, then login, then logout", async ({ browser }) => {
const userId = `test_user-id_${Date.now()}`;
@@ -69,12 +71,7 @@ test("As a guest, create a call, share link and other join", async ({
// ========
// ARRANGE: The first user creates a call as guest, join it, then click the invite button to copy the invite link
// ========
await creatorPage.getByTestId("home_callName").click();
await creatorPage.getByTestId("home_callName").fill("Welcome");
await creatorPage.getByTestId("home_displayName").click();
await creatorPage.getByTestId("home_displayName").fill("Inviter");
await creatorPage.getByTestId("home_go").click();
await expect(creatorPage.locator("video")).toBeVisible();
await SpaHelpers.createCall(creatorPage, "Inviter", "Welcome");
// join
await creatorPage.getByTestId("lobby_joinCall").click();
@@ -82,19 +79,7 @@ test("As a guest, create a call, share link and other join", async ({
await creatorPage.getByRole("radio", { name: "Spotlight" }).check();
// Get the invite link
await creatorPage.getByRole("button", { name: "Invite" }).click();
await expect(
creatorPage.getByRole("heading", { name: "Invite to this call" }),
).toBeVisible();
await expect(creatorPage.getByRole("img", { name: "QR Code" })).toBeVisible();
await expect(creatorPage.getByTestId("modal_inviteLink")).toBeVisible();
await expect(creatorPage.getByTestId("modal_inviteLink")).toBeVisible();
await creatorPage.getByTestId("modal_inviteLink").click();
const inviteLink = (await creatorPage.evaluate(
"navigator.clipboard.readText()",
)) as string;
expect(inviteLink).toContain("room/#/");
const inviteLink = await SpaHelpers.getCallInviteLink(creatorPage);
// ========
// ACT: The other user use the invite link to join the call as a guest
@@ -103,13 +88,7 @@ test("As a guest, create a call, share link and other join", async ({
reducedMotion: "reduce",
});
const guestPage = await guestInviteeContext.newPage();
await guestPage.goto(inviteLink);
await guestPage.getByTestId("joincall_displayName").fill("Invitee");
await expect(guestPage.getByTestId("joincall_joincall")).toBeVisible();
await guestPage.getByTestId("joincall_joincall").click();
await guestPage.getByTestId("lobby_joinCall").click();
await guestPage.getByRole("radio", { name: "Spotlight" }).check();
await SpaHelpers.joinCallFromInviteLink(guestPage, inviteLink);
// ========
// ASSERT: check that there are two members in the call

View File

@@ -22,8 +22,8 @@ test("Start a new call then leave and show the feedback screen", async ({
await expect(page.getByTestId("lobby_joinCall")).toBeVisible();
// Check the button toolbar
// await expect(page.getByRole('button', { name: 'Mute microphone' })).toBeVisible();
// await expect(page.getByRole('button', { name: 'Stop video' })).toBeVisible();
// await expect(page.getByRole('switch', { name: 'Mute microphone' })).toBeVisible();
// await expect(page.getByRole('switch', { name: 'Stop video' })).toBeVisible();
await expect(page.getByRole("button", { name: "Settings" })).toBeVisible();
await expect(page.getByRole("button", { name: "End call" })).toBeVisible();

View File

@@ -1,19 +1,15 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type Browser,
type Page,
test,
expect,
type JSHandle,
} from "@playwright/test";
import { type Page, test, expect, type JSHandle } from "@playwright/test";
import type { MatrixClient } from "matrix-js-sdk";
import { HOST1, TestHelpers } from "../widget/test-helpers.ts";
export type UserBaseFixture = {
mxId: string;
@@ -30,10 +26,9 @@ export type BaseWidgetSetup = {
export interface MyFixtures {
asWidget: BaseWidgetSetup;
callType: "room" | "dm";
addUser: (username: string, host: string) => Promise<UserBaseFixture>;
}
const PASSWORD = "foobarbaz1!";
// Minimal config.json for the local element-web instance
const CONFIG_JSON = {
default_server_config: {
@@ -67,85 +62,6 @@ const CONFIG_JSON = {
},
};
/**
* Set the Element Call URL in the dev tool settings using `window.mxSettingsStore` via `page.evaluate`.
*/
const setDevToolElementCallDevUrl = process.env.USE_DOCKER
? async (page: Page): Promise<void> => {
await page.evaluate(() => {
window.mxSettingsStore.setValue(
"Developer.elementCallUrl",
null,
"device",
"http://localhost:8080/room",
);
});
}
: async (page: Page): Promise<void> => {
await page.evaluate(() => {
window.mxSettingsStore.setValue(
"Developer.elementCallUrl",
null,
"device",
"https://localhost:3000/room",
);
});
};
/**
* Registers a new user and returns page, clientHandle and mxId.
*/
async function registerUser(
browser: Browser,
username: string,
): Promise<{ page: Page; clientHandle: JSHandle<MatrixClient>; mxId: string }> {
const userContext = await browser.newContext({
reducedMotion: "reduce",
});
const page = await userContext.newPage();
await page.goto("http://localhost:8081/#/welcome");
await page.getByRole("link", { name: "Create Account" }).click();
await page.getByRole("textbox", { name: "Username" }).fill(username);
await page
.getByRole("textbox", { name: "Password", exact: true })
.fill(PASSWORD);
await page.getByRole("textbox", { name: "Confirm password" }).click();
await page.getByRole("textbox", { name: "Confirm password" }).fill(PASSWORD);
await page.getByRole("button", { name: "Register" }).click();
await expect(
page.getByRole("heading", { name: `Welcome ${username}` }),
).toBeVisible();
const browserUnsupportedToast = page
.getByText("Element does not support this browser")
.locator("..")
.locator("..");
// Dismiss incompatible browser toast
const dismissButton = browserUnsupportedToast.getByRole("button", {
name: "Dismiss",
});
try {
await expect(dismissButton).toBeVisible({ timeout: 700 });
await dismissButton.click();
} catch {
// dismissButton not visible, continue as normal
}
await setDevToolElementCallDevUrl(page);
const clientHandle = await page.evaluateHandle(() =>
window.mxMatrixClientPeg.get(),
);
const mxId = (await clientHandle.evaluate(
(cli: MatrixClient) => cli.getUserId(),
clientHandle,
))!;
return { page, clientHandle, mxId };
}
export const widgetTest = test.extend<MyFixtures>({
// allow per-test override: `widgetTest.use({ callType: "dm" })`
callType: ["room", { option: true }],
@@ -162,25 +78,16 @@ export const widgetTest = test.extend<MyFixtures>({
page: ewPage1,
clientHandle: brooksClientHandle,
mxId: brooksMxId,
} = await registerUser(browser, brooksDisplayName);
} = await TestHelpers.registerUser(browser, brooksDisplayName);
const {
page: ewPage2,
clientHandle: whistlerClientHandle,
mxId: whistlerMxId,
} = await registerUser(browser, whistlerDisplayName);
} = await TestHelpers.registerUser(browser, whistlerDisplayName);
// Invite the second user
await ewPage1
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.click();
if (callType === "room") {
await ewPage1.getByRole("menuitem", { name: "New Room" }).click();
await ewPage1.getByRole("textbox", { name: "Name" }).fill("Welcome Room");
await ewPage1.getByRole("button", { name: "Create room" }).click();
await expect(ewPage1.getByText("You created this room.")).toBeVisible();
await expect(ewPage1.getByText("Encryption enabled")).toBeVisible();
await TestHelpers.createRoom("Welcome Room", ewPage1);
await ewPage1
.getByRole("button", { name: "Invite to this room", exact: true })
@@ -210,6 +117,11 @@ export const widgetTest = test.extend<MyFixtures>({
.getByRole("heading", { name: "Welcome Room" }),
).toBeVisible();
} else if (callType === "dm") {
await ewPage1
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.click();
await ewPage1.getByRole("menuitem", { name: "Start chat" }).click();
await ewPage1.getByRole("textbox", { name: "Search" }).click();
await ewPage1.getByRole("textbox", { name: "Search" }).fill(whistlerMxId);
@@ -252,4 +164,30 @@ export const widgetTest = test.extend<MyFixtures>({
},
});
},
/**
* Provide a way to add additional users within a test.
* The returned user will be registered on the default homeserver, the name will be made unique by appending a timestamp.
*/
addUser: async ({ browser }, use) => {
await use(
async (
username: string,
host: string = HOST1,
): Promise<UserBaseFixture> => {
const uniqueSuffix = Date.now();
const { page, clientHandle, mxId } = await TestHelpers.registerUser(
browser,
`${username.toLowerCase()}_${uniqueSuffix}`,
host,
);
return {
mxId,
displayName: username,
page,
clientHandle,
};
},
);
},
});

View File

@@ -100,8 +100,16 @@ mobileTest(
{ id: "earpiece", name: "Handset", isEarpiece: true },
{ id: "headphones", name: "Headphones" },
]);
window.controls.setAudioDevice("earpiece");
});
// Open settings to select earpiece
await guestPage.getByRole("button", { name: "Settings" }).click();
await guestPage.getByText("Handset", { exact: true }).click();
// dismiss settings
await guestPage.locator("#root").getByLabel("Settings").press("Escape");
await guestPage.pause();
await expect(
guestPage.getByRole("heading", { name: "Handset Mode" }),
).toBeVisible();

View File

@@ -49,12 +49,12 @@ test("can only interact with header and footer while reconnecting", async ({
).toBeVisible();
// Tab order should jump directly from header to footer, skipping media tiles
await page.getByRole("button", { name: "Mute microphone" }).focus();
await page.getByRole("switch", { name: "Mute microphone" }).focus();
await expect(
page.getByRole("button", { name: "Mute microphone" }),
page.getByRole("switch", { name: "Mute microphone" }),
).toBeFocused();
await page.keyboard.press("Tab");
await expect(page.getByRole("button", { name: "Stop video" })).toBeFocused();
await expect(page.getByRole("switch", { name: "Stop video" })).toBeFocused();
// Most critically, we should be able to press the hangup button
await page.getByRole("button", { name: "End call" }).click();
});

View File

@@ -0,0 +1,135 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
expect,
type Page,
test,
type Request,
type Browser,
} from "@playwright/test";
import { SpaHelpers } from "./spa-helpers";
async function setupTwoUserSpaCall(
browser: Browser,
page: Page,
browserName: string,
): Promise<{ guestPage: Page }> {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
await page.goto("/");
let androlHasSentStickyEvent = false;
await interceptEventSend(
page,
// This room is not encrypted, so the event is sent in clear
"org.matrix.msc4143.rtc.member",
(req) => {
androlHasSentStickyEvent =
androlHasSentStickyEvent || isStickySend(req.url());
},
);
await SpaHelpers.createCall(page, "Androl", "HelloCall", true, "2_0");
const inviteLink = await SpaHelpers.getCallInviteLink(page);
// Other
const guestInviteeContext = await browser.newContext({
reducedMotion: "reduce",
});
const guestPage = await guestInviteeContext.newPage();
await guestPage.goto("/");
let pevaraHasSentStickyEvent = false;
await interceptEventSend(
guestPage,
// This room is not encrypted, so the event is sent in clear
"org.matrix.msc4143.rtc.member",
(req) => {
pevaraHasSentStickyEvent =
pevaraHasSentStickyEvent || isStickySend(req.url());
},
);
await SpaHelpers.joinCallFromInviteLink(
guestPage,
inviteLink,
"Pevara",
"2_0",
);
// Assert both sides have sent sticky membership events
expect(androlHasSentStickyEvent).toEqual(true);
expect(pevaraHasSentStickyEvent).toEqual(true);
return { guestPage };
}
test("One to One call using matrix rtc 2.0 aka sticky events", async ({
browser,
page,
browserName,
}) => {
const { guestPage } = await setupTwoUserSpaCall(browser, page, browserName);
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
});
// This issue occurs when a member leave but does not clean up their sticky event.
// If they rejoin they will use a new stickye key (stickyKey = member.id = UUID())
// We end up with two memberships with the same user and device id. This previously
// was a impossible case since that would be the same state event. Now its possible.
// We need to ALWAYS key by userId, deviceId and member.id. This test checks that.
test("One to One rejoin after improper leave does not crash EC", async ({
browser,
page,
browserName,
}) => {
const { guestPage } = await setupTwoUserSpaCall(browser, page, browserName);
await SpaHelpers.expectVideoTilesCount(page, 2);
await SpaHelpers.expectVideoTilesCount(guestPage, 2);
await guestPage.reload();
await expect(guestPage.getByTestId("lobby_joinCall")).toBeVisible();
// Check if rejoining with the same browser context (device) breaks EC.
// This has happened on versions that do not consider the member.id as part of the key for a media tile.
await guestPage.getByTestId("lobby_joinCall").click();
// We cannot use the `expectVideoTilesCount` helper here since one of them is expected to show waiting for media
await expect(page.getByTestId("videoTile")).toHaveCount(3);
await expect(guestPage.getByTestId("videoTile")).toHaveCount(2);
});
function isStickySend(url: string): boolean {
return !!new URL(url).searchParams.get(
"org.matrix.msc4354.sticky_duration_ms",
);
}
async function interceptEventSend(
page: Page,
eventType: string,
callback: (request: Request) => void,
): Promise<void> {
await page.route(
`**/_matrix/client/v3/rooms/**/send/${eventType}/**`,
async (route, req) => {
callback(req);
return route.continue();
},
);
}

148
playwright/spa-helpers.ts Normal file
View File

@@ -0,0 +1,148 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, type Page } from "@playwright/test";
import { type RtcMode } from "./widget/test-helpers.ts";
/**
* Create and join a call from the SPA home page.
*
* @param page - The Playwright page object
* @param userName - The display name to use for the call
* @param callName - The name of the call to create
* @param autoJoin - Whether to automatically join the call after creating it
* @param mode - The RTC mode to use for the call
*/
async function createCall(
page: Page,
userName: string,
callName: string,
autoJoin: boolean = false,
mode: RtcMode | undefined = undefined,
): Promise<void> {
await page.getByTestId("home_callName").click();
await page.getByTestId("home_callName").fill(callName);
await page.getByTestId("home_displayName").click();
await page.getByTestId("home_displayName").fill(userName);
await page.getByTestId("home_go").click();
await expect(page.locator("video")).toBeVisible();
await expect(page.getByTestId("lobby_joinCall")).toBeVisible();
if (mode) {
await setRtcModeFromSettings(page, mode);
}
if (autoJoin) {
// Join the call
await page.getByTestId("lobby_joinCall").click();
}
}
/**
* Get the invite link for the current call.
*/
async function getCallInviteLink(page: Page): Promise<string> {
await page.getByRole("button", { name: "Invite" }).click();
await expect(
page.getByRole("heading", { name: "Invite to this call" }),
).toBeVisible();
await expect(page.getByRole("img", { name: "QR Code" })).toBeVisible();
await expect(page.getByTestId("modal_inviteLink")).toBeVisible();
await expect(page.getByTestId("modal_inviteLink")).toBeVisible();
await page.getByTestId("modal_inviteLink").click();
const inviteLink = (await page.evaluate(
"navigator.clipboard.readText()",
)) as string;
expect(inviteLink).toContain("room/#/");
return inviteLink;
}
/**
* Join a call from an invitation link.
* @param page - The Playwright page object
* @param inviteLink - The invite link to join
* @param displayName - The display name to use when joining the call
* @param mode - The RTC mode to use for the call
*/
async function joinCallFromInviteLink(
page: Page,
inviteLink: string,
displayName: string = "Invitee",
mode: RtcMode | undefined = undefined,
): Promise<void> {
await page.goto(inviteLink);
await page.getByTestId("joincall_displayName").fill(displayName);
await expect(page.getByTestId("joincall_joincall")).toBeVisible();
await page.getByTestId("joincall_joincall").click();
if (mode) {
await setRtcModeFromSettings(page, mode);
}
await page.getByTestId("lobby_joinCall").click();
await page.getByRole("radio", { name: "Spotlight" }).check();
}
async function setRtcModeFromSettings(
page: Page,
mode: RtcMode,
): Promise<void> {
await page.getByRole("button", { name: "Settings" }).click();
await page.getByRole("tab", { name: "Preferences" }).click();
await page.getByText("Developer mode", { exact: true }).check(); // Idempotent: won't uncheck if already checked
// Move to Developer tab now
await page.getByRole("tab", { name: "Developer" }).click();
if (mode == "legacy") {
await page.getByText("Legacy: state events").click();
} else if (mode == "2_0") {
await page.getByText("Matrix 2.0").click();
} else {
// compat
await page.getByText("Compatibility: state events").click();
}
await page.getByTestId("modal_close").click();
}
/**
* Expect a certain number of video tiles to be present and visible.
*/
async function expectVideoTilesCount(page: Page, count: number): Promise<void> {
await expect(page.getByTestId("videoTile")).toHaveCount(2);
// There are no other options than to wait for all media to be ready?
// Or it is too flaky :/
await page.waitForTimeout(3000);
// No one should be waiting for media
await expect(page.getByText("Waiting for media...")).not.toBeVisible();
// There should be 5 video elements, visible and autoplaying
const videoElements = await page.locator("video").all();
expect(videoElements.length).toBe(count);
const blockDisplayCount = await page
.locator("video")
.evaluateAll(
(videos: Element[]) =>
videos.filter(
(v: Element) => window.getComputedStyle(v).display === "block",
).length,
);
expect(blockDisplayCount).toBe(count);
}
export const SpaHelpers = {
createCall,
getCallInviteLink,
joinCallFromInviteLink,
expectVideoTilesCount,
};

View File

@@ -0,0 +1,86 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user";
import { HOST1, HOST2, type RtcMode, TestHelpers } from "./test-helpers";
const modePairs: [RtcMode, RtcMode][] = [
["compat", "compat"],
["legacy", "legacy"],
["legacy", "compat"],
["compat", "legacy"],
];
modePairs.forEach(([rtcMode1, rtcMode2]) => {
widgetTest(
`Test federated call with rtc modes ${rtcMode1} and ${rtcMode2}`,
async ({ addUser, browserName }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
const florian = await addUser("floriant", HOST1);
const timo = await addUser("timo", HOST2);
const roomName = "Call Room";
await TestHelpers.createRoom(roomName, florian.page, [timo.mxId]);
await TestHelpers.acceptRoomInvite(roomName, timo.page);
await florian.page.pause();
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
florian.page,
rtcMode1,
);
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
timo.page,
rtcMode2,
);
await TestHelpers.startCallInCurrentRoom(florian.page, false);
await TestHelpers.joinCallFromLobby(florian.page);
// timo joins
await TestHelpers.joinCallInCurrentRoom(timo.page);
// We should see 2 video tiles everywhere now
for (const user of [timo, florian]) {
const frame = user.page
.locator('iframe[title="Element Call"]')
.contentFrame();
await expect(frame.getByTestId("videoTile")).toHaveCount(2);
// There are no other options than to wait for all media to be ready?
// Or it is too flaky :/
await user.page.waitForTimeout(3000);
// No one should be waiting for media
await expect(frame.getByText("Waiting for media...")).not.toBeVisible();
// There should be 2 video elements, visible and autoplaying
const videoElements = await frame.locator("video").all();
expect(videoElements.length).toBe(2);
const blockDisplayCount = await frame
.locator("video")
.evaluateAll(
(videos: Element[]) =>
videos.filter(
(v: Element) => window.getComputedStyle(v).display === "block",
).length,
);
expect(blockDisplayCount).toBe(2);
}
// await florian.page.pause();
},
);
});

View File

@@ -0,0 +1,104 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user";
import { HOST1, HOST2, TestHelpers } from "./test-helpers";
// ## Issue
// This test reproduces an issue with the publisher.
// When switching local focus, we need to recreate the publisher.
// This failed because of a dead lock in the old publishers destruction.
//
// There are numerus ways to enforece this situation:
// - oldest member swap (manually set the oldest member focus and leave with the prev oldest member)
// This almost never happens in the real worls since clients will set their preferredFoci list to what the oldest member is.
// - switch from oldest member to multi sfu as the NOT the first joiner + the first joiner is on a different sfu than your preferred sfu.
//
// This test uses the "switch from oldest member to multi sfu" approach.
//
// It is a copy of federated-call.test.ts in the `["legacy", "legacy"]` setup,
// which once connected will make the second user switch to multi sfu.
widgetTest(
`Test swapping publisher from ${HOST1} to ${HOST2}`,
async ({ addUser, browserName }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
const florian = await addUser("floriant", HOST1);
const timo = await addUser("timo", HOST2);
const roomName = "Call Room";
await TestHelpers.createRoom(roomName, florian.page, [timo.mxId]);
await TestHelpers.acceptRoomInvite(roomName, timo.page);
await florian.page.pause();
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
florian.page,
"legacy",
);
await TestHelpers.openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
timo.page,
"legacy",
);
await TestHelpers.startCallInCurrentRoom(florian.page, false);
await TestHelpers.joinCallFromLobby(florian.page);
// timo joins
await TestHelpers.joinCallInCurrentRoom(timo.page);
// We should see 2 video tiles everywhere now
for (const user of [timo, florian]) {
const frame = user.page
.locator('iframe[title="Element Call"]')
.contentFrame();
await expect(frame.getByTestId("videoTile")).toHaveCount(2);
// There are no other options than to wait for all media to be ready?
// Or it is too flaky :/
await user.page.waitForTimeout(3000);
// No one should be waiting for media
await expect(frame.getByText("Waiting for media...")).not.toBeVisible();
// There should be 2 video elements, visible and autoplaying
const videoElements = await frame.locator("video").all();
expect(videoElements.length).toBe(2);
const blockDisplayCount = await frame
.locator("video")
.evaluateAll(
(videos: Element[]) =>
videos.filter(
(v: Element) => window.getComputedStyle(v).display === "block",
).length,
);
expect(blockDisplayCount).toBe(2);
}
// now we switch the mode for timo (second joiner on multi-sfu HOST2 but currently HOST1)
await TestHelpers.setEmbeddedElementCallRtcMode(timo.page, "compat");
await timo.page.waitForTimeout(3000);
const blockDisplayCount = await timo.page
.locator('iframe[title="Element Call"]')
.contentFrame()
.locator("video")
.evaluateAll(
(videos: Element[]) =>
videos.filter(
(v: Element) => window.getComputedStyle(v).display === "block",
).length,
);
expect(blockDisplayCount).toBe(2);
},
);

View File

@@ -0,0 +1,135 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user.ts";
import { HOST1, TestHelpers } from "./test-helpers.ts";
widgetTest("Create and join a group call", async ({ addUser, browserName }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
test.slow(); // We are registering multiple users here, give it more time
const valere = await addUser("Valere", HOST1);
const timo = await addUser("Timo", HOST1);
const robin = await addUser("Robin", HOST1);
const halfshot = await addUser("Halfshot", HOST1);
const florian = await addUser("florian", HOST1);
const roomName = "Group Call Room";
await TestHelpers.createRoom(roomName, valere.page, [
timo.mxId,
robin.mxId,
halfshot.mxId,
florian.mxId,
]);
for (const user of [timo, robin, halfshot, florian]) {
// Accept the invite
// This isn't super stable to get this as this super generic locator,
// but it works for now.
await TestHelpers.acceptRoomInvite(roomName, user.page);
}
// Start the call as Valere
await TestHelpers.startCallInCurrentRoom(valere.page, false);
await expect(
valere.page.locator('iframe[title="Element Call"]'),
).toBeVisible();
await TestHelpers.joinCallFromLobby(valere.page);
for (const user of [timo, robin, halfshot, florian]) {
await TestHelpers.joinCallInCurrentRoom(user.page);
}
for (const user of [timo, robin, halfshot, florian]) {
const frame = user.page
.locator('iframe[title="Element Call"]')
.contentFrame();
// No lobby, should start with video on
await expect(
frame.getByRole("switch", { name: "Stop video", checked: true }),
).toBeVisible();
}
// We should see 5 video tiles everywhere now
for (const user of [valere, timo, robin, halfshot, florian]) {
const frame = user.page
.locator('iframe[title="Element Call"]')
.contentFrame();
await expect(frame.getByTestId("videoTile")).toHaveCount(5);
for (const participant of [valere, timo, robin, halfshot, florian]) {
// Check the names are correct
await expect(frame.getByText(participant.displayName)).toBeVisible();
}
// There is no other options than to wait for all media to be ready?
// Or it is too flaky :/
await user.page.waitForTimeout(5000);
// No one should be waiting for media
await expect(frame.getByText("Waiting for media...")).not.toBeVisible();
// There should be 5 video elements, visible and autoplaying
const videoElements = await frame.locator("video").all();
expect(videoElements.length).toBe(5);
await expect(frame.locator("video[autoplay]")).toHaveCount(5);
const blockDisplayCount = await frame
.locator("video")
.evaluateAll(
(videos: Element[]) =>
videos.filter(
(v: Element) => window.getComputedStyle(v).display === "block",
).length,
);
expect(blockDisplayCount).toBe(5);
}
// Quickly test muting one participant to see it reflects and that our asserts works
const florianFrame = florian.page
.locator('iframe[title="Element Call"]')
.contentFrame();
const florianVideoButton = florianFrame.getByRole("switch", {
name: /video/,
});
await expect(florianVideoButton).toHaveAccessibleName("Stop video");
await expect(florianVideoButton).toBeChecked();
await florianVideoButton.click();
// Now the button should indicate we can start video
await expect(florianVideoButton).toHaveAccessibleName("Start video");
await expect(florianVideoButton).not.toBeChecked();
// wait a bit for the state to propagate
await valere.page.waitForTimeout(3000);
{
const frame = valere.page
.locator('iframe[title="Element Call"]')
.contentFrame();
const videoElements = await frame.locator("video").all();
expect(videoElements.length).toBe(5);
const blockDisplayCount = await frame
.locator("video")
.evaluateAll(
(videos: Element[]) =>
videos.filter(
(v: Element) => window.getComputedStyle(v).display === "block",
).length,
);
// out of 5 ONLY 4 are visible (display:block) !!
// XXX we need to be better at our HTML markup and accessibility, it would make
// this kind of stuff way easier to test if we could look out for aria attributes.
expect(blockDisplayCount).toBe(4);
}
});

View File

@@ -0,0 +1,73 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user.ts";
import { HOST1, TestHelpers } from "./test-helpers.ts";
widgetTest("Footer interaction in PiP", async ({ addUser, browserName }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
test.slow();
const valere = await addUser("Valere", HOST1);
const callRoom = "CallRoom";
await TestHelpers.createRoom("CallRoom", valere.page);
await TestHelpers.createRoom("OtherRoom", valere.page);
await TestHelpers.switchToRoomNamed(valere.page, callRoom);
// Start the call as Valere
await TestHelpers.startCallInCurrentRoom(valere.page, false);
await expect(
valere.page.locator('iframe[title="Element Call"]'),
).toBeVisible();
await TestHelpers.joinCallFromLobby(valere.page);
// wait a bit so that the PIP has rendered
await valere.page.waitForTimeout(600);
// Switch to the other room, the call should go to PIP
await TestHelpers.switchToRoomNamed(valere.page, "OtherRoom");
// We should see the PIP overlay
const iFrame = valere.page
.locator('iframe[title="Element Call"]')
.contentFrame();
{
// Check for a bug where the video had the wrong fit in PIP
const audioBtn = iFrame.getByRole("switch", { name: /microphone/ });
const videoBtn = iFrame.getByRole("switch", { name: /video/ });
await expect(
iFrame.getByRole("button", { name: "End call" }),
).toBeVisible();
await expect(audioBtn).toBeVisible();
await expect(videoBtn).toBeVisible();
await expect(audioBtn).toHaveAccessibleName("Mute microphone");
await expect(audioBtn).toBeChecked();
await expect(videoBtn).toHaveAccessibleName("Stop video");
await expect(videoBtn).toBeChecked();
await videoBtn.click();
await audioBtn.click();
// stop hovering on any of the buttons
await iFrame.getByTestId("videoTile").hover();
await expect(audioBtn).toHaveAccessibleName("Unmute microphone");
await expect(audioBtn).not.toBeChecked();
await expect(videoBtn).toHaveAccessibleName("Start video");
await expect(videoBtn).not.toBeChecked();
}
});

View File

@@ -0,0 +1,72 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user.ts";
import { HOST1, TestHelpers } from "./test-helpers.ts";
widgetTest("Put call in PIP", async ({ addUser, browserName }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
test.slow();
const valere = await addUser("Valere", HOST1);
const timo = await addUser("Timo", HOST1);
const callRoom = "TeamRoom";
await TestHelpers.createRoom(callRoom, valere.page, [timo.mxId]);
await TestHelpers.createRoom("DoubleTask", valere.page);
await TestHelpers.acceptRoomInvite(callRoom, timo.page);
await TestHelpers.switchToRoomNamed(valere.page, callRoom);
// Start the call as Valere
await TestHelpers.startCallInCurrentRoom(valere.page, false);
await expect(
valere.page.locator('iframe[title="Element Call"]'),
).toBeVisible();
await TestHelpers.joinCallFromLobby(valere.page);
await TestHelpers.joinCallInCurrentRoom(timo.page);
const frame = timo.page
.locator('iframe[title="Element Call"]')
.contentFrame();
// check that the video is on
await expect(
frame.getByRole("switch", { name: "Stop video", checked: true }),
).toBeVisible();
// Switch to the other room, the call should go to PIP
await TestHelpers.switchToRoomNamed(valere.page, "DoubleTask");
// We should see the PIP overlay
await expect(valere.page.getByTestId("widget-pip-container")).toBeVisible();
{
// wait a bit so that the PIP has rendered the video
await valere.page.waitForTimeout(600);
// Check for a bug where the video had the wrong fit in PIP
const frame = valere.page
.locator('iframe[title="Element Call"]')
.contentFrame();
const videoElements = await frame.locator("video").all();
expect(videoElements.length).toBe(1);
const pipVideo = videoElements[0];
await expect(pipVideo).toHaveCSS("object-fit", "cover");
}
});

View File

@@ -0,0 +1,309 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type Browser,
expect,
type JSHandle,
type Page,
} from "@playwright/test";
import { type MatrixClient } from "matrix-js-sdk";
const PASSWORD = "foobarbaz1!";
export const HOST1 = "https://app.m.localhost/#/welcome";
export const HOST2 = "https://app.othersite.m.localhost/#/welcome";
export type RtcMode = "legacy" | "compat" | "2_0";
export class TestHelpers {
public static async startCallInCurrentRoom(
page: Page,
voice: boolean = false,
): Promise<void> {
const buttonName = voice ? "Voice call" : "Video call";
await expect(page.getByRole("button", { name: buttonName })).toBeVisible();
await page.getByRole("button", { name: buttonName }).click();
await expect(
page.getByRole("menuitem", { name: "Element Call" }),
).toBeVisible();
await page.getByRole("menuitem", { name: "Element Call" }).click();
}
public static async joinCallFromLobby(page: Page): Promise<void> {
await expect(
page
.locator('iframe[title="Element Call"]')
.contentFrame()
.getByTestId("lobby_joinCall"),
).toBeVisible();
await page
.locator('iframe[title="Element Call"]')
.contentFrame()
.getByTestId("lobby_joinCall")
.click();
}
public static async joinCallInCurrentRoom(
page: Page,
audioOnly: boolean = false,
): Promise<void> {
// This is the header button that notifies about an ongoing call
const label = audioOnly ? "Voice call started" : "Video call started";
await expect(page.getByText(label)).toBeVisible();
await expect(page.getByRole("button", { name: "Join" })).toBeVisible();
await page.getByRole("button", { name: "Join" }).click();
}
/**
* Registers a new user and returns page, clientHandle and mxId.
*/
public static async registerUser(
browser: Browser,
username: string,
host: string = HOST1,
): Promise<{
page: Page;
clientHandle: JSHandle<MatrixClient>;
mxId: string;
}> {
const userContext = await browser.newContext({
reducedMotion: "reduce",
});
const page = await userContext.newPage();
await page.goto(host);
await page.getByRole("link", { name: "Create Account" }).click();
await page.getByRole("textbox", { name: "Username" }).fill(username);
await page.getByRole("textbox", { name: "Password", exact: true }).click();
await page
.getByRole("textbox", { name: "Password", exact: true })
.fill(PASSWORD);
await page.getByRole("textbox", { name: "Confirm password" }).click();
await page
.getByRole("textbox", { name: "Confirm password" })
.fill(PASSWORD);
await page.getByRole("button", { name: "Register" }).click();
await expect(
page.getByRole("heading", { name: `Welcome ${username}` }),
).toBeVisible({
// Increase timeout as registration can be slow :/
timeout: 15_000,
});
await this.maybeDismissBrowserNotSupportedToast(page);
await this.maybeDismissServiceWorkerWarningToast(page);
await TestHelpers.setDevToolElementCallDevUrl(page);
const clientHandle = await page.evaluateHandle(() =>
window.mxMatrixClientPeg.get(),
);
const mxId = (await clientHandle.evaluate(
(cli: MatrixClient) => cli.getUserId(),
clientHandle,
))!;
return { page, clientHandle, mxId };
}
private static async maybeDismissBrowserNotSupportedToast(
page: Page,
): Promise<void> {
const browserUnsupportedToast = page
.getByText("Element does not support this browser")
.locator("..")
.locator("..");
// Dismiss incompatible browser toast
const dismissButton = browserUnsupportedToast.getByRole("button", {
name: "Dismiss",
});
try {
await expect(dismissButton).toBeVisible({ timeout: 700 });
await dismissButton.click();
} catch {
// dismissButton not visible, continue as normal
}
}
private static async maybeDismissServiceWorkerWarningToast(
page: Page,
): Promise<void> {
const toast = page
.locator(".mx_Toast_toast")
.getByText("Failed to load service worker");
try {
await expect(toast).toBeVisible({ timeout: 700 });
await page
.locator(".mx_Toast_toast")
.getByRole("button", { name: "OK" })
.click();
} catch {
// toast not visible, continue as normal
}
}
public static async maybeDismissKeyBackupToast(page: Page): Promise<void> {
const toast = page
.locator(".mx_Toast_toast")
.getByText("Back up your chats");
try {
await expect(toast).toBeVisible({ timeout: 700 });
await page
.locator(".mx_Toast_toast")
.getByRole("button", { name: "Dismiss" })
.click();
} catch {
// toast not visible, continue as normal
}
}
public static async createRoom(
name: string,
page: Page,
andInvite: string[] = [],
): Promise<void> {
await page
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.click();
await page.getByRole("menuitem", { name: "New Room" }).click();
await page.getByRole("textbox", { name: "Name" }).fill(name);
await page.getByRole("button", { name: "Create room" }).click();
await expect(page.getByText("You created this room.")).toBeVisible();
await expect(page.getByText("Encryption enabled")).toBeVisible();
await TestHelpers.maybeDismissKeyBackupToast(page);
// Invite users if any
if (andInvite.length > 0) {
await page
.getByRole("button", { name: "Invite to this room", exact: true })
.click();
const inviteInput = page.getByRole("dialog").getByRole("textbox");
for (const mxId of andInvite) {
await inviteInput.focus();
await inviteInput.fill(mxId);
await inviteInput.press("Enter");
}
await page.getByRole("button", { name: "Invite" }).click();
}
}
/**
* Accepts a room invite using the room name.
* Locatest the invite in the room list.
*
*/
public static async acceptRoomInvite(
roomName: string,
page: Page,
): Promise<void> {
await expect(page.getByRole("option", { name: roomName })).toBeVisible();
await page.getByRole("option", { name: roomName }).click();
await page.getByRole("button", { name: "Accept" }).click();
await expect(
page.getByRole("main").getByRole("heading", { name: roomName }),
).toBeVisible();
await TestHelpers.maybeDismissKeyBackupToast(page);
}
/**
* Opens the widget and then goes to the settings to set the RTC mode.
* then closes the widget lobby.
*
* intended to be used before joining!
*
* WORKS IF A ROOM IS CURRENTLY OPENED IN THE PAGE
*/
public static async openWidgetSetEmbeddedElementCallRtcModeCloseWidget(
page: Page,
mode: RtcMode,
): Promise<void> {
await page.getByRole("button", { name: "Video call" }).click();
await page.getByRole("menuitem", { name: "Element Call" }).click();
await TestHelpers.setEmbeddedElementCallRtcMode(page, mode);
await page.getByRole("button", { name: "Close lobby" }).click();
}
/**
* Goes to the settings to set the RTC mode.
* then closes the settings modal.
*
* WORKS IF A ROOM IS CURRENTLY SHOWING THE EC WIDGET
*/
public static async setEmbeddedElementCallRtcMode(
page: Page,
mode: RtcMode,
): Promise<void> {
const iframe = page.locator('iframe[title="Element Call"]').contentFrame();
await iframe.getByRole("button", { name: "Settings" }).click();
await iframe.getByRole("tab", { name: "Preferences" }).click();
// await iframe.getByText("Developer mode", { exact: true }).click();
await iframe.getByText("Developer mode", { exact: true }).check(); // Idempotent: won't uncheck if already checked
// Move to Developer tab now
await iframe.getByRole("tab", { name: "Developer" }).click();
if (mode == "legacy") {
await iframe.getByText("Legacy: state events").click();
} else if (mode == "2_0") {
await iframe.getByText("Matrix 2.0").click();
} else {
// compat
await iframe.getByText("Compatibility: state events").click();
}
await iframe.getByTestId("modal_close").click();
}
/**
* Sets the current Element Web app to use the dev Element Call URL.
* @param page - The EW page
*/
public static async setDevToolElementCallDevUrl(page: Page): Promise<void> {
if (process.env.USE_DOCKER) {
await page.evaluate(() => {
window.mxSettingsStore.setValue(
"Developer.elementCallUrl",
null,
"device",
"https://call.m.localhost/room",
);
});
} else {
await page.evaluate(() => {
window.mxSettingsStore.setValue(
"Developer.elementCallUrl",
null,
"device",
"https://localhost:3000/room",
);
});
}
}
/**
* Switches to a room in the room list by its name.
* @param page - The EW page
* @param roomName - The name of the room to switch to
*/
public static async switchToRoomNamed(
page: Page,
roomName: string,
): Promise<void> {
await page.getByRole("option", { name: `Open room ${roomName}` }).click();
}
}

View File

@@ -8,6 +8,7 @@ Please see LICENSE in the repository root for full details.
import { expect, test } from "@playwright/test";
import { widgetTest } from "../fixtures/widget-user.ts";
import { TestHelpers } from "./test-helpers.ts";
widgetTest.use({ callType: "dm" });
@@ -23,16 +24,7 @@ widgetTest(
const { brooks, whistler } = asWidget;
await expect(
brooks.page.getByRole("button", { name: "Voice call" }),
).toBeVisible();
await brooks.page.getByRole("button", { name: "Voice call" }).click();
await expect(
brooks.page.getByRole("menuitem", { name: "Element Call" }),
).toBeVisible();
await brooks.page.getByRole("menuitem", { name: "Element Call" }).click();
await TestHelpers.startCallInCurrentRoom(brooks.page, true);
await expect(
brooks.page.locator('iframe[title="Element Call"]'),
@@ -42,9 +34,12 @@ widgetTest(
.locator('iframe[title="Element Call"]')
.contentFrame();
// We should show a ringing overlay, let's check for that
// We should show a ringing tile, let's check for that
await expect(
brooksFrame.getByText(`Waiting for ${whistler.displayName} to join…`),
brooksFrame
.getByTestId("videoTile")
.filter({ has: brooksFrame.getByText(whistler.displayName) })
.filter({ has: brooksFrame.getByText("Calling…") }),
).toBeVisible();
await expect(whistler.page.getByText("Incoming voice call")).toBeVisible();
@@ -59,34 +54,36 @@ widgetTest(
.contentFrame();
// ASSERT the button states for whistler (the callee)
{
// The only way to know if it is muted or not is to look at the data-kind attribute..
const videoButton = whistlerFrame.getByTestId("incall_videomute");
// video should be off by default in a voice call
await expect(videoButton).toHaveAttribute("aria-label", /^Start video$/);
const audioButton = whistlerFrame.getByTestId("incall_mute");
// audio should be on for the voice call
await expect(audioButton).toHaveAttribute(
"aria-label",
/^Mute microphone$/,
);
}
// video should be off by default in a voice call
await expect(
whistlerFrame.getByRole("switch", {
name: "Start video",
checked: false,
}),
).toBeVisible();
// audio should be on for the voice call
await expect(
whistlerFrame.getByRole("switch", {
name: "Mute microphone",
checked: true,
}),
).toBeVisible();
// ASSERT the button states for brools (the caller)
{
// The only way to know if it is muted or not is to look at the data-kind attribute..
const videoButton = brooksFrame.getByTestId("incall_videomute");
// video should be off by default in a voice call
await expect(videoButton).toHaveAttribute("aria-label", /^Start video$/);
const audioButton = brooksFrame.getByTestId("incall_mute");
// audio should be on for the voice call
await expect(audioButton).toHaveAttribute(
"aria-label",
/^Mute microphone$/,
);
}
// video should be off by default in a voice call
await expect(
whistlerFrame.getByRole("switch", {
name: "Start video",
checked: false,
}),
).toBeVisible();
// audio should be on for the voice call
await expect(
whistlerFrame.getByRole("switch", {
name: "Mute microphone",
checked: true,
}),
).toBeVisible();
// In order to confirm that the call is disconnected we will check that the message composer is shown again.
// So first we need to confirm that it is hidden when in the call.
@@ -98,10 +95,7 @@ widgetTest(
).not.toBeVisible();
// ASSERT hanging up on one side ends the call for both
{
const hangupButton = brooksFrame.getByTestId("incall_leave");
await hangupButton.click();
}
await brooksFrame.getByRole("button", { name: "End call" }).click();
// The widget should be closed on both sides and the timeline should be back on screen
await expect(
@@ -123,16 +117,7 @@ widgetTest(
const { brooks, whistler } = asWidget;
await expect(
brooks.page.getByRole("button", { name: "Video call" }),
).toBeVisible();
await brooks.page.getByRole("button", { name: "Video call" }).click();
await expect(
brooks.page.getByRole("menuitem", { name: "Element Call" }),
).toBeVisible();
await brooks.page.getByRole("menuitem", { name: "Element Call" }).click();
await TestHelpers.startCallInCurrentRoom(brooks.page, false);
await expect(
brooks.page.locator('iframe[title="Element Call"]'),
@@ -142,9 +127,12 @@ widgetTest(
.locator('iframe[title="Element Call"]')
.contentFrame();
// We should show a ringing overlay, let's check for that
// We should show a ringing tile, let's check for that
await expect(
brooksFrame.getByText(`Waiting for ${whistler.displayName} to join…`),
brooksFrame
.getByTestId("videoTile")
.filter({ has: brooksFrame.getByText(whistler.displayName) })
.filter({ has: brooksFrame.getByText("Calling…") }),
).toBeVisible();
await expect(whistler.page.getByText("Incoming video call")).toBeVisible();
@@ -159,34 +147,30 @@ widgetTest(
.contentFrame();
// ASSERT the button states for whistler (the callee)
{
// The only way to know if it is muted or not is to look at the data-kind attribute..
const videoButton = whistlerFrame.getByTestId("incall_videomute");
// video should be on by default in a voice call
await expect(videoButton).toHaveAttribute("aria-label", /^Stop video$/);
const audioButton = whistlerFrame.getByTestId("incall_mute");
// audio should be on for the voice call
await expect(audioButton).toHaveAttribute(
"aria-label",
/^Mute microphone$/,
);
}
// video should be off by default in a video call
await expect(
whistlerFrame.getByRole("switch", { name: "Stop video", checked: true }),
).toBeVisible();
// audio should be on too
await expect(
whistlerFrame.getByRole("switch", {
name: "Mute microphone",
checked: true,
}),
).toBeVisible();
// ASSERT the button states for brools (the caller)
{
// The only way to know if it is muted or not is to look at the data-kind attribute..
const videoButton = brooksFrame.getByTestId("incall_videomute");
// video should be on by default in a voice call
await expect(videoButton).toHaveAttribute("aria-label", /^Stop video$/);
const audioButton = brooksFrame.getByTestId("incall_mute");
// audio should be on for the voice call
await expect(audioButton).toHaveAttribute(
"aria-label",
/^Mute microphone$/,
);
}
// video should be off by default in a video call
await expect(
whistlerFrame.getByRole("switch", { name: "Stop video", checked: true }),
).toBeVisible();
// audio should be on too
await expect(
whistlerFrame.getByRole("switch", {
name: "Mute microphone",
checked: true,
}),
).toBeVisible();
// In order to confirm that the call is disconnected we will check that the message composer is shown again.
// So first we need to confirm that it is hidden when in the call.
@@ -198,10 +182,7 @@ widgetTest(
).not.toBeVisible();
// ASSERT hanging up on one side ends the call for both
{
const hangupButton = brooksFrame.getByTestId("incall_leave");
await hangupButton.click();
}
await brooksFrame.getByRole("button", { name: "End call" }).click();
// The widget should be closed on both sides and the timeline should be back on screen
await expect(
@@ -210,3 +191,51 @@ widgetTest(
await expect(brooks.page.locator(".mx_BasicMessageComposer")).toBeVisible();
},
);
widgetTest(
"Decline a new video call in DM as widget",
async ({ asWidget, browserName }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment. No mic/audio device inputs so cam/mic are disabled",
);
test.slow(); // Triples the timeout
const { brooks, whistler } = asWidget;
await TestHelpers.startCallInCurrentRoom(brooks.page, false);
await expect(
brooks.page.locator('iframe[title="Element Call"]'),
).toBeVisible();
const brooksFrame = brooks.page
.locator('iframe[title="Element Call"]')
.contentFrame();
// We should show a ringing tile, let's check for that
await expect(
brooksFrame
.getByTestId("videoTile")
.filter({ has: brooksFrame.getByText(whistler.displayName) })
.filter({ has: brooksFrame.getByText("Calling…") }),
).toBeVisible();
await expect(whistler.page.getByText("Incoming video call")).toBeVisible();
await whistler.page.getByRole("button", { name: "Decline" }).click();
await expect(
whistler.page.locator('iframe[title="Element Call"]'),
).not.toBeVisible();
// The widget should be closed and the timeline should be back on screen
await expect(
brooks.page.locator('iframe[title="Element Call"]'),
).not.toBeVisible();
await expect(
brooks.page.getByText("This is the beginning of your"),
).toBeVisible();
},
);

View File

@@ -1,4 +1,4 @@
# SDK mode
# SDK mode (EXPERIMENTAL)
EC can be build in sdk mode. This will result in a compiled js file that can be imported in very simple webapps.
@@ -6,16 +6,40 @@ It allows to use matrixRTC in combination with livekit without relying on elemen
This is done by instantiating the call view model and exposing some useful behaviors (observables) and methods.
This folder contains an example index.html file that showcases the sdk in use (hosted on localhost:8123 with a webserver ellowing cors (for example `npx serve -l 81234 --cors`)) as a godot engine HTML export template.
This folder contains an example index.html file that showcases the sdk in use (hosted on localhost:8123 with a webserver allowing cors (for example `npx serve -l 81234 --cors`)) as a godot engine HTML export template.
## Getting started
To get started run
```
yarn
yarn build:sdk
```
in the repository root.
It will create a `dist` folder containing the compiled js file.
This file needs to be hosted. Locally (via `npx serve -l 81234 --cors`) or on a remote server.
Now you just need to add the widget to element web via:
```
/addwidget http://localhost:3000?widgetId=$matrix_widget_id&perParticipantE2EE=true&userId=$matrix_user_id&deviceId=$org.matrix.msc3819.matrix_device_id&baseUrl=$org.matrix.msc4039.matrix_base_url&roomId=$matrix_room_id
```
## Widgets
The sdk mode is particularly interesting to be used in widgets where you do not need to pay attention to matrix login/cs api ...
To create a widget see the example index.html file in this folder. And add it to EW via:
The sdk mode is particularly interesting to be used in widgets. In widgets you do not need to pay attention to matrix login/cs api ...
To create a widget see the example `index.html` file in this folder. And add it to EW via:
`/addwidget <widgetUrl>` (see **url parameters** for more details on `<widgetUrl>`)
### url parameters
The url parameters are needed to pass initial data to the widget. They will automatically be used
by the matrixRTCSdk to start the postmessage widget api (communication between the client (e.g. Element Web) and the widget)
```
widgetId = $matrix_widget_id
perParticipantE2EE = true

View File

@@ -12,15 +12,12 @@ Please see LICENSE in the repository root for full details.
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { scan } from "rxjs";
import { widget as _widget } from "../src/widget";
import { type WidgetHelpers } from "../src/widget";
import { type LivekitRoomItem } from "../src/state/CallViewModel/CallViewModel";
export const logger = rootLogger.getChild("[MatrixRTCSdk]");
if (!_widget) throw Error("No widget. This webapp can only start as a widget");
export const widget = _widget;
export const tryMakeSticky = (): void => {
export const tryMakeSticky = (widget: WidgetHelpers): void => {
logger.info("try making sticky MatrixRTCSdk");
void widget.api
.setAlwaysOnScreen(true)

View File

@@ -1,7 +1,7 @@
<!doctype html>
<html>
<head>
<title>Godot MatrixRTC Widget</title>
<title>MatrixRTC Widget</title>
<meta charset="utf-8" />
<script type="module">
// TODO use the url where the matrixrtc-sdk.js file from dist is hosted
@@ -9,7 +9,7 @@
try {
window.matrixRTCSdk = await createMatrixRTCSdk(
"com.github.toger5.godot-game",
"com.github.toger5.rtc-application-type", // rtc application type
);
console.info("createMatrixRTCSdk was created!");
} catch (e) {
@@ -17,16 +17,6 @@
}
const sdk = window.matrixRTCSdk;
// This is the main bridging interface to godot
window.matrixRTCSdkGodot = {
dataObs: sdk.data$,
memberObs: sdk.members$,
// join: sdk.join, // lets stick with autojoin for now
sendData: sdk.sendData,
leave: sdk.leave,
connectedObs: sdk.connected$,
};
console.info("matrixRTCSdk join ", sdk);
const connectionState = sdk.join();
console.info("matrixRTCSdk joined");
@@ -38,15 +28,13 @@
const child = document.createElement("p");
child.innerHTML = JSON.stringify(data);
div.appendChild(child);
// TODO forward to godot
});
sdk.members$.subscribe((memberObjects) => {
// reset div
const div = document.getElementById("members");
div.innerHTML = "<h3>Members:</h3>";
// create member list
// Create member list
const members = memberObjects.map((member) => member.membership.sender);
console.info("members changed", members);
for (const m of members) {
@@ -62,17 +50,10 @@
const div = document.getElementById("connect_status");
div.innerHTML = connected ? "Connected" : "Disconnected";
});
let engine = new Engine($GODOT_CONFIG);
engine.startGame();
</script>
<!--// TODO use it as godot HTML template-->
<script src="$GODOT_URL"></script>
</head>
<body>
<canvas id="canvas"></canvas>
<div
id="overlay"
style="position: absolute; top: 0; right: 0; background-color: #ffffff10"
>
<div id="connect_status"></div>

View File

@@ -1,11 +1,13 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025-2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* EXPERIMENTAL
*
* This file is the entrypoint for the sdk build of element call: `yarn build:sdk`
* use in widgets.
* It exposes the `createMatrixRTCSdk` which creates the `MatrixRTCSdk` interface (see below) that
@@ -30,8 +32,8 @@ import {
} from "rxjs";
import {
type CallMembership,
MatrixRTCSession,
MatrixRTCSessionEvent,
MatrixRTCSessionManager,
} from "matrix-js-sdk/lib/matrixrtc";
import {
type Room as LivekitRoom,
@@ -50,14 +52,12 @@ import { getUrlParams } from "../src/UrlParams";
import { MuteStates } from "../src/state/MuteStates";
import { MediaDevices } from "../src/state/MediaDevices";
import { E2eeType } from "../src/e2ee/e2eeType";
import { currentAndPrev, logger, TEXT_LK_TOPIC, tryMakeSticky } from "./helper";
import {
currentAndPrev,
logger,
TEXT_LK_TOPIC,
tryMakeSticky,
widget,
} from "./helper";
import { ElementWidgetActions } from "../src/widget";
ElementWidgetActions,
widget as _widget,
initializeWidget,
} from "../src/widget";
import { type Connection } from "../src/state/CallViewModel/remoteMembers/Connection";
interface MatrixRTCSdk {
@@ -68,7 +68,13 @@ interface MatrixRTCSdk {
join: () => void;
/** @throws on leave errors */
leave: () => void;
data$: Observable<{ sender: string; data: string }>;
/**
* Ends the rtc sdk. This will unsubscribe any event listeners. And end the associated scope.
* No updates can be received from the rtc sdk. The sdk cannot be restarted after.
* A new sdk needs to be created via createMatrixRTCSdk.
*/
stop: () => void;
data$: Observable<{ rtcBackendIdentity: string; data: string }>;
/**
* flattened list of members
*/
@@ -79,37 +85,54 @@ interface MatrixRTCSdk {
participant: LocalParticipant | RemoteParticipant | null;
}[]
>;
/**
* flattened local members
*/
localMember$: Behavior<{
connection: Connection | null;
membership: CallMembership;
participant: LocalParticipant | null;
} | null>;
/** Use the LocalMemberConnectionState returned from `join` for a more detailed connection state */
connected$: Behavior<boolean>;
sendData?: (data: unknown) => Promise<void>;
sendRoomMessage?: (message: string) => Promise<void>;
}
export async function createMatrixRTCSdk(
application: string = "m.call",
id: string = "",
sticky: boolean = false,
): Promise<MatrixRTCSdk> {
logger.info("Hello");
const scope = new ObservableScope();
// widget client
initializeWidget(application, true);
const widget = _widget;
if (!widget) throw Error("No widget. This webapp can only start as a widget");
const client = await widget.client;
logger.info("client created");
const scope = new ObservableScope();
// url params
const { roomId } = getUrlParams();
if (roomId === null) throw Error("could not get roomId from url params");
const room = client.getRoom(roomId);
if (room === null) throw Error("could not get room from client");
// rtc session
const slot = { application, id };
const rtcSessionManager = new MatrixRTCSessionManager(logger, client, slot);
rtcSessionManager.start();
const rtcSession = rtcSessionManager.getRoomSession(room);
// media devices
const mediaDevices = new MediaDevices(scope);
const muteStates = new MuteStates(scope, mediaDevices, {
audioEnabled: true,
videoEnabled: true,
audioEnabled: false,
videoEnabled: false,
});
const slot = { application, id };
const rtcSession = new MatrixRTCSession(
client,
room,
MatrixRTCSession.sessionMembershipsForSlot(room, slot),
slot,
);
// call view model
const callViewModel = createCallViewModel$(
scope,
rtcSession,
@@ -122,8 +145,9 @@ export async function createMatrixRTCSdk(
constant({ supported: false, processor: undefined }),
);
logger.info("CallViewModelCreated");
// create data listener
const data$ = new Subject<{ sender: string; data: string }>();
const data$ = new Subject<{ rtcBackendIdentity: string; data: string }>();
const lkTextStreamHandlerFunction = async (
reader: TextStreamReader,
@@ -145,7 +169,7 @@ export async function createMatrixRTCSdk(
if (participants && participants.includes(participantInfo.identity)) {
const text = await reader.readAll();
logger.info(`Received text: ${text}`);
data$.next({ sender: participantInfo.identity, data: text });
data$.next({ rtcBackendIdentity: participantInfo.identity, data: text });
} else {
logger.warn(
"Received text from unknown participant",
@@ -235,6 +259,16 @@ export async function createMatrixRTCSdk(
}
};
const sendRoomMessage = async (message: string): Promise<void> => {
const messageString = JSON.stringify(message);
logger.info("try sending to room: ", messageString);
try {
await client.sendTextMessage(room.roomId, message);
} catch (e) {
logger.error("failed sending to room: ", messageString, e);
}
};
// after hangup gets called
const leaveSubs = callViewModel.leave$.subscribe(() => {
const scheduleWidgetCloseOnLeave = async (): Promise<void> => {
@@ -262,9 +296,6 @@ export async function createMatrixRTCSdk(
// schedule close first and then leave (scope.end)
void scheduleWidgetCloseOnLeave();
// actual hangup (ending scope will send the leave event.. its kinda odd. since you might end up closing the widget too fast)
scope.end();
});
logger.info("createMatrixRTCSdk done");
@@ -272,15 +303,40 @@ export async function createMatrixRTCSdk(
return {
join: (): void => {
// first lets try making the widget sticky
tryMakeSticky();
if (sticky) tryMakeSticky(widget);
callViewModel.join();
},
leave: (): void => {
callViewModel.hangup();
callViewModel.leave();
},
stop: (): void => {
leaveSubs.unsubscribe();
livekitRoomItemsSub.unsubscribe();
scope.end();
},
data$,
localMember$: scope.behavior(
callViewModel.localMatrixLivekitMember$.pipe(
tap((member) =>
logger.info("localMatrixLivekitMember$ next: ", member),
),
switchMap((member) => {
if (member === null) return of(null);
return combineLatest([
member.connection$,
member.membership$,
member.participant.value$,
]).pipe(
map(([connection, membership, participant]) => ({
connection,
membership,
participant,
})),
);
}),
tap((member) => logger.info("localMember$ next: ", member)),
),
),
connected$: callViewModel.connected$,
members$: scope.behavior(
callViewModel.matrixLivekitMembers$.pipe(
@@ -307,5 +363,6 @@ export async function createMatrixRTCSdk(
[],
),
sendData,
sendRoomMessage,
};
}

View File

@@ -20,6 +20,7 @@ interface Props {
audio?: RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats;
video?: RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats;
focusUrl?: string;
rtcBackendIdentity?: string;
}
const extractDomain = (url: string): string => {
@@ -37,6 +38,7 @@ export const RTCConnectionStats: FC<Props> = ({
audio,
video,
focusUrl,
rtcBackendIdentity,
...rest
}) => {
const [showModal, setShowModal] = useState(false);
@@ -71,6 +73,9 @@ export const RTCConnectionStats: FC<Props> = ({
</pre>
</div>
</Modal>
<Text as="span" size="xs" title="rtcBackendIdentity">
rtcBackendIdentity:{rtcBackendIdentity}
</Text>
{focusUrl && (
<div>
<Text as="span" size="xs" title="focusURL">

View File

@@ -1,5 +1,6 @@
/*
Copyright 2023, 2024 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.

View File

@@ -1,5 +1,6 @@
/*
Copyright 2022-2024 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -472,8 +473,7 @@ export const computeUrlParams = (search = "", hash = ""): UrlParams => {
homeserver: !isWidget ? parser.getParam("homeserver") : null,
posthogApiHost: parser.getParam("posthogApiHost"),
posthogApiKey: parser.getParam("posthogApiKey"),
posthogUserId:
parser.getParam("posthogUserId") ?? parser.getParam("analyticsID"),
posthogUserId: parser.getParam("posthogUserId"),
rageshakeSubmitUrl: parser.getParam("rageshakeSubmitUrl"),
sentryDsn: parser.getParam("sentryDsn"),
sentryEnvironment: parser.getParam("sentryEnvironment"),

View File

@@ -14,14 +14,14 @@ exports[`AppBar > renders 1`] = `
>
<button
aria-labelledby="_r_0_"
class="_icon-button_1pz9o_8"
class="_icon-button_1215g_8"
data-kind="primary"
role="button"
style="--cpd-icon-button-size: 32px;"
tabindex="0"
>
<div
class="_indicator-icon_zr2a0_17"
class="_indicator-icon_147l5_17"
style="--cpd-icon-button-size: 100%;"
>
<svg

View File

@@ -1,5 +1,6 @@
/*
Copyright 2022-2024 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -21,23 +22,25 @@ import {
import styles from "./Button.module.css";
interface MicButtonProps extends ComponentPropsWithoutRef<"button"> {
muted: boolean;
enabled: boolean;
size?: "sm" | "lg";
}
export const MicButton: FC<MicButtonProps> = ({ muted, ...props }) => {
export const MicButton: FC<MicButtonProps> = ({ enabled, ...props }) => {
const { t } = useTranslation();
const Icon = muted ? MicOffSolidIcon : MicOnSolidIcon;
const label = muted
? t("unmute_microphone_button_label")
: t("mute_microphone_button_label");
const Icon = enabled ? MicOnSolidIcon : MicOffSolidIcon;
const label = enabled
? t("mute_microphone_button_label")
: t("unmute_microphone_button_label");
return (
<Tooltip label={label}>
<CpdButton
iconOnly
aria-label={label}
Icon={Icon}
kind={muted ? "primary" : "secondary"}
kind={enabled ? "primary" : "secondary"}
role="switch"
aria-checked={enabled}
{...props}
/>
</Tooltip>
@@ -45,23 +48,25 @@ export const MicButton: FC<MicButtonProps> = ({ muted, ...props }) => {
};
interface VideoButtonProps extends ComponentPropsWithoutRef<"button"> {
muted: boolean;
enabled: boolean;
size?: "sm" | "lg";
}
export const VideoButton: FC<VideoButtonProps> = ({ muted, ...props }) => {
export const VideoButton: FC<VideoButtonProps> = ({ enabled, ...props }) => {
const { t } = useTranslation();
const Icon = muted ? VideoCallOffSolidIcon : VideoCallSolidIcon;
const label = muted
? t("start_video_button_label")
: t("stop_video_button_label");
const Icon = enabled ? VideoCallSolidIcon : VideoCallOffSolidIcon;
const label = enabled
? t("stop_video_button_label")
: t("start_video_button_label");
return (
<Tooltip label={label}>
<CpdButton
iconOnly
aria-label={label}
Icon={Icon}
kind={muted ? "primary" : "secondary"}
kind={enabled ? "primary" : "secondary"}
role="switch"
aria-checked={enabled}
{...props}
/>
</Tooltip>
@@ -70,6 +75,7 @@ export const VideoButton: FC<VideoButtonProps> = ({ muted, ...props }) => {
interface ShareScreenButtonProps extends ComponentPropsWithoutRef<"button"> {
enabled: boolean;
size: "sm" | "lg";
}
export const ShareScreenButton: FC<ShareScreenButtonProps> = ({
@@ -87,13 +93,19 @@ export const ShareScreenButton: FC<ShareScreenButtonProps> = ({
iconOnly
Icon={ShareScreenSolidIcon}
kind={enabled ? "primary" : "secondary"}
role="switch"
aria-checked={enabled}
{...props}
/>
</Tooltip>
);
};
export const EndCallButton: FC<ComponentPropsWithoutRef<"button">> = ({
interface EndCallButtonProps extends ComponentPropsWithoutRef<"button"> {
size?: "sm" | "lg";
}
export const EndCallButton: FC<EndCallButtonProps> = ({
className,
...props
}) => {
@@ -104,7 +116,6 @@ export const EndCallButton: FC<ComponentPropsWithoutRef<"button">> = ({
<CpdButton
className={classNames(className, styles.endCall)}
iconOnly
aria-label={t("hangup_button_label")}
Icon={EndCallIcon}
destructive
{...props}
@@ -113,9 +124,10 @@ export const EndCallButton: FC<ComponentPropsWithoutRef<"button">> = ({
);
};
export const SettingsButton: FC<ComponentPropsWithoutRef<"button">> = (
props,
) => {
interface SettingsButtonProps extends ComponentPropsWithoutRef<"button"> {
size?: "sm" | "lg";
}
export const SettingsButton: FC<SettingsButtonProps> = (props) => {
const { t } = useTranslation();
return (

View File

@@ -18,7 +18,8 @@ import { getBasicCallViewModelEnvironment } from "../utils/test-viewmodel";
import { alice, local, localRtcMember } from "../utils/test-fixtures";
import { type MockRTCSession } from "../utils/test";
import { ReactionsSenderProvider } from "../reactions/useReactionsSender";
import { initializeWidget } from "../widget";
initializeWidget();
vi.mock("livekit-client/e2ee-worker?worker");
const localIdent = `${localRtcMember.userId}:${localRtcMember.deviceId}`;

View File

@@ -166,6 +166,7 @@ export function ReactionPopupMenu({
interface ReactionToggleButtonProps extends ComponentPropsWithoutRef<"button"> {
identifier: string;
vm: CallViewModel;
size?: "sm" | "lg";
}
export function ReactionToggleButton({

View File

@@ -10,7 +10,7 @@ exports[`Can close reaction dialog 1`] = `
aria-expanded="true"
aria-haspopup="true"
aria-labelledby="_r_bb_"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
class="_button_13vu4_8 _has-icon_13vu4_60 _icon-only_13vu4_53"
data-kind="primary"
data-size="lg"
role="button"
@@ -44,7 +44,7 @@ exports[`Can fully expand emoji picker 1`] = `
aria-expanded="true"
aria-haspopup="true"
aria-labelledby="_r_7m_"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
class="_button_13vu4_8 _has-icon_13vu4_60 _icon-only_13vu4_53"
data-kind="primary"
data-size="lg"
role="button"
@@ -75,7 +75,7 @@ exports[`Can lower hand 1`] = `
aria-expanded="false"
aria-haspopup="true"
aria-labelledby="_r_36_"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
class="_button_13vu4_8 _has-icon_13vu4_60 _icon-only_13vu4_53"
data-kind="secondary"
data-size="lg"
role="button"
@@ -109,7 +109,7 @@ exports[`Can open menu 1`] = `
aria-expanded="true"
aria-haspopup="true"
aria-labelledby="_r_0_"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
class="_button_13vu4_8 _has-icon_13vu4_60 _icon-only_13vu4_53"
data-kind="primary"
data-size="lg"
role="button"
@@ -140,7 +140,7 @@ exports[`Can raise hand 1`] = `
aria-expanded="false"
aria-haspopup="true"
aria-labelledby="_r_1j_"
class="_button_vczzf_8 raisedButton _has-icon_vczzf_57 _icon-only_vczzf_50"
class="_button_13vu4_8 raisedButton _has-icon_13vu4_60 _icon-only_13vu4_53"
data-kind="primary"
data-size="lg"
role="button"

View File

@@ -1,5 +1,6 @@
/*
Copyright 2022-2024 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -164,9 +165,6 @@ export interface ResolvedConfigOptions extends ConfigOptions {
};
ssla: string;
app_prompt: boolean;
// For privacy reasons when lobby is skipped we do not unmute by default on full builds
// but for development builds we want to unmute by default to speed up testing.
trust_localhost_for_mute_state: boolean;
}
export const DEFAULT_CONFIG: ResolvedConfigOptions = {
@@ -181,5 +179,4 @@ export const DEFAULT_CONFIG: ResolvedConfigOptions = {
},
ssla: "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf",
app_prompt: true,
trust_localhost_for_mute_state: false,
};

View File

@@ -33,12 +33,38 @@ export interface Controls {
showNativeOutputDevicePicker?: () => void;
}
/**
* Output Audio device when using the controlled audio output mode (mobile).
*/
export interface OutputDevice {
id: string;
name: string;
/**
* `forEarpiece` in an iOS only flag, that will be set on the default speaker device.
* The default speaker device will be used for the earpiece mode by
* using a stereo pan and reducing the volume significantly. (in combination this is similar to a dedicated earpiece mode)
* - on iOS this is true if output is routed to speaker.
* In that case then ElementCalls manually appends an earpiece device with id `EARPIECE_CONFIG_ID` and `{ type: "earpiece" }`
* - on Android this is unused.
*/
forEarpiece?: boolean;
/**
* Is the device the OS earpiece audio configuration?
* - on iOS always undefined
* - on Android true for the `TYPE_BUILTIN_EARPIECE`
*/
isEarpiece?: boolean;
/**
* Is the device the OS default speaker:
* - on iOS always true if output is routed to speaker. In other case iOS on declare a `dummy` id device.
* - on Android true for the `TYPE_BUILTIN_SPEAKER`
*/
isSpeaker?: boolean;
/**
* Is the device the OS default external headset (bluetooth):
* - on iOS always undefined.
* - on Android true for the `TYPE_BLUETOOTH_SCO`
*/
isExternalHeadset?: boolean;
}
@@ -47,8 +73,16 @@ export interface OutputDevice {
*/
export const setPipEnabled$ = new Subject<boolean>();
/**
* Stores the list of available controlled audio output devices.
* This is set when the native code calls `setAvailableAudioDevices` with the list of available audio output devices.
*/
export const availableOutputDevices$ = new Subject<OutputDevice[]>();
/**
* Stores the current audio output device id.
* This is set when the native code calls `setAudioDevice`
*/
export const outputDevice$ = new Subject<string>();
/**
@@ -80,16 +114,41 @@ window.controls = {
setPipEnabled$.next(false);
},
/**
* Reverse engineered:
*
* - on iOS:
* This always a list of one thing. If current route output is speaker it returns
* the single `{"id":"Speaker","name":"Speaker","forEarpiece":true,"isSpeaker":true}` Notice that EC will
* also manually add a virtual earpiece device with id `EARPIECE_CONFIG_ID` and `{ type: "earpiece" }`.
* If the route output is not speaker then it will be `{id: 'dummy', name: 'dummy'}`
*
*
* - on Android:
* This is a list of all available output audio devices. The `id` is the Android AudioDeviceInfo.getId()
* and the `name` is based the Android AudioDeviceInfo.productName (mapped to static strings for known types)
* The `isEarpiece`, `isSpeaker` and `isExternalHeadset` are set based on the Android AudioDeviceInfo.type
* matching the corresponding types for earpiece, speaker and bluetooth headset.
*/
setAvailableAudioDevices(devices: OutputDevice[]): void {
logger.info("setAvailableAudioDevices called from native:", devices);
logger.info(
"[MediaDevices controls] setAvailableAudioDevices called from native:",
devices,
);
availableOutputDevices$.next(devices);
},
setAudioDevice(id: string): void {
logger.info("setAudioDevice called from native", id);
logger.info(
"[MediaDevices controls] setAudioDevice called from native",
id,
);
outputDevice$.next(id);
},
setAudioEnabled(enabled: boolean): void {
logger.info("setAudioEnabled called from native:", enabled);
logger.info(
"[MediaDevices controls] setAudioEnabled called from native:",
enabled,
);
if (!setAudioEnabled$.observed)
throw new Error(
"Output controls are disabled. No setAudioEnabled$ observer",

View File

@@ -6,11 +6,13 @@ Please see LICENSE in the repository root for full details.
*/
import { BaseKeyProvider } from "livekit-client";
import { logger } from "matrix-js-sdk/lib/logger";
import {
type MatrixRTCSession,
MatrixRTCSessionEvent,
} from "matrix-js-sdk/lib/matrixrtc";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
const logger = rootLogger.getChild("[MatrixKeyProvider]");
export class MatrixKeyProvider extends BaseKeyProvider {
private rtcSession?: MatrixRTCSession;
@@ -42,7 +44,8 @@ export class MatrixKeyProvider extends BaseKeyProvider {
private onEncryptionKeyChanged = (
encryptionKey: Uint8Array<ArrayBuffer>,
encryptionKeyIndex: number,
participantId: string,
membershipParts: CallMembershipIdentityParts,
rtcBackendIdentity: string,
): void => {
crypto.subtle
.importKey("raw", encryptionKey, "HKDF", false, [
@@ -53,17 +56,17 @@ export class MatrixKeyProvider extends BaseKeyProvider {
(keyMaterial) => {
this.onSetEncryptionKey(
keyMaterial,
participantId,
rtcBackendIdentity,
encryptionKeyIndex,
);
logger.debug(
`Sent new key to livekit room=${this.rtcSession?.room.roomId} participantId=${participantId} encryptionKeyIndex=${encryptionKeyIndex}`,
`Sent new key to livekit room=${this.rtcSession?.room.roomId} participantId=${rtcBackendIdentity} (before hash: ${membershipParts.userId}:${membershipParts.deviceId}) encryptionKeyIndex=${encryptionKeyIndex}`,
);
},
(e) => {
logger.error(
`Failed to create key material from buffer for livekit room=${this.rtcSession?.room.roomId} participantId=${participantId} encryptionKeyIndex=${encryptionKeyIndex}`,
`Failed to create key material from buffer for livekit room=${this.rtcSession?.room.roomId} participantId before hash=${membershipParts.userId}:${membershipParts.deviceId} encryptionKeyIndex=${encryptionKeyIndex}`,
e,
);
},

View File

@@ -51,15 +51,15 @@ export const makeOneOnOneLayout: CallLayout<OneOnOneLayoutModel> = ({
return (
<div ref={ref} className={styles.layer}>
<Slot
id={model.remote.id}
model={model.remote}
id={model.spotlight.id}
model={model.spotlight}
className={styles.container}
style={{ width: tileWidth, height: tileHeight }}
>
<Slot
className={classNames(styles.slot, styles.local)}
id={model.local.id}
model={model.local}
id={model.pip.id}
model={model.pip}
onDrag={onDragLocalTile}
data-block-alignment={pipAlignmentValue.block}
data-inline-alignment={pipAlignmentValue.inline}

View File

@@ -27,7 +27,13 @@ interface Props<M, R extends HTMLElement> {
state: Parameters<Handler<"drag", EventTypes["drag"]>>[0],
) => void
> | null;
/**
* The width this tile will have once its animations have settled.
*/
targetWidth: number;
/**
* The width this tile will have once its animations have settled.
*/
targetHeight: number;
model: M;
Tile: ComponentType<TileProps<M, R>>;

View File

@@ -28,6 +28,7 @@ import { getUrlParams } from "./UrlParams";
import { Config } from "./config/Config";
import { platform } from "./Platform";
import { isFailure } from "./utils/fetch";
import { initializeWidget } from "./widget";
// This generates a map of locale names to their URL (based on import.meta.url), which looks like this:
// {
@@ -115,6 +116,8 @@ export class Initializer {
}
public static async initBeforeReact(): Promise<void> {
initializeWidget();
const polyfills: Promise<unknown>[] = [];
if (shouldPolyfillSegmenter()) {
polyfills.push(import("@formatjs/intl-segmenter/polyfill-force"));

View File

@@ -28,8 +28,15 @@ import {
mockRemoteParticipant,
mockTrack,
} from "../utils/test";
export const TestAudioContextConstructor = vi.fn(() => testAudioContext);
import { initializeWidget } from "../widget";
initializeWidget();
export const TestAudioContextConstructor = vi.fn(
class {
public constructor() {
return testAudioContext;
}
},
);
const MediaDevicesProvider = MediaDevicesContext.MediaDevicesContext.Provider;

View File

@@ -15,7 +15,6 @@ import {
type AudioTrackProps,
} from "@livekit/components-react";
import { logger } from "matrix-js-sdk/lib/logger";
import { type ParticipantId } from "matrix-js-sdk/lib/matrixrtc";
import { useEarpieceAudioConfig } from "../MediaDevicesContext";
import { useReactiveState } from "../useReactiveState";
@@ -32,7 +31,7 @@ export interface MatrixAudioRendererProps {
* This list needs to be composed based on the matrixRTC members so that we do not play audio from users
* that are not expected to be in the rtc session (local user is excluded).
*/
validIdentities: ParticipantId[];
validIdentities: string[];
/**
* If set to `true`, mutes all audio tracks rendered by the component.
* @remarks
@@ -79,6 +78,7 @@ export function LivekitRoomAudioRenderer({
.filter((ref) => {
const isValid = validIdentities.includes(ref.participant.identity);
if (!isValid) {
// TODO make sure to also skip the warn logging for the local identity
// Log that there is an invalid identity, that means that someone is publishing audio that is not expected to be in the call.
prefixedLogger.warn(
`Audio track ${ref.participant.identity} from ${url} has no matching matrix call member`,

View File

@@ -18,6 +18,7 @@ import fetchMock from "fetch-mock";
import { getSFUConfigWithOpenID, type OpenIDClientParts } from "./openIDSFU";
import { testJWTToken } from "../utils/test-fixtures";
import { ownMemberMock } from "../utils/test";
const sfuUrl = "https://sfu.example.org";
@@ -33,6 +34,7 @@ describe("getSFUConfigWithOpenID", () => {
vitest.clearAllMocks();
fetchMock.reset();
});
it("should handle fetching a token", async () => {
fetchMock.post("https://sfu.example.org/sfu/get", () => {
return {
@@ -42,6 +44,7 @@ describe("getSFUConfigWithOpenID", () => {
});
const config = await getSFUConfigWithOpenID(
matrixClient,
ownMemberMock,
"https://sfu.example.org",
"!example_room_id",
);
@@ -53,6 +56,7 @@ describe("getSFUConfigWithOpenID", () => {
});
void (await fetchMock.flush());
});
it("should fail if the SFU errors", async () => {
fetchMock.post("https://sfu.example.org/sfu/get", () => {
return {
@@ -63,11 +67,12 @@ describe("getSFUConfigWithOpenID", () => {
try {
await getSFUConfigWithOpenID(
matrixClient,
ownMemberMock,
"https://sfu.example.org",
"!example_room_id",
);
} catch (ex) {
expect(((ex as Error).cause as Error).message).toEqual(
expect((ex as Error).message).toEqual(
"SFU Config fetch failed with status code 500",
);
void (await fetchMock.flush());
@@ -76,6 +81,104 @@ describe("getSFUConfigWithOpenID", () => {
expect.fail("Expected test to throw;");
});
it("should try legacy and then new endpoint with delay delegation", async () => {
fetchMock.post("https://sfu.example.org/get_token", () => {
return {
status: 500,
body: { error: "Test failure" },
};
});
fetchMock.post("https://sfu.example.org/sfu/get", () => {
return {
status: 500,
body: { error: "Test failure" },
};
});
try {
await getSFUConfigWithOpenID(
matrixClient,
ownMemberMock,
"https://sfu.example.org",
"!example_room_id",
{
delayEndpointBaseUrl: "https://matrix.homeserverserver.org",
delayId: "mock_delay_id",
},
);
} catch (ex) {
expect((ex as Error).message).toEqual(
"SFU Config fetch failed with status code 500",
);
void (await fetchMock.flush());
}
const calls = fetchMock.calls();
expect(calls.length).toBe(2);
expect(calls[0][0]).toStrictEqual("https://sfu.example.org/get_token");
expect(calls[0][1]).toStrictEqual({
// check if it uses correct delayID!
body: '{"room_id":"!example_room_id","slot_id":"m.call#ROOM","member":{"id":"@alice:example.org:DEVICE","claimed_user_id":"@alice:example.org","claimed_device_id":"DEVICE"},"delay_id":"mock_delay_id","delay_timeout":1000,"delay_cs_api_url":"https://matrix.homeserverserver.org"}',
method: "POST",
headers: {
"Content-Type": "application/json",
},
});
expect(calls[1][0]).toStrictEqual("https://sfu.example.org/sfu/get");
expect(calls[1][1]).toStrictEqual({
body: '{"room":"!example_room_id","device_id":"DEVICE"}',
headers: {
"Content-Type": "application/json",
},
method: "POST",
});
});
it("dont try legacy if endpoint with delay delegation is sucessful", async () => {
fetchMock.post("https://sfu.example.org/get_token", () => {
return {
status: 200,
body: { url: sfuUrl, jwt: testJWTToken },
};
});
fetchMock.post("https://sfu.example.org/sfu/get", () => {
return {
status: 500,
body: { error: "Test failure" },
};
});
try {
await getSFUConfigWithOpenID(
matrixClient,
ownMemberMock,
"https://sfu.example.org",
"!example_room_id",
{
delayEndpointBaseUrl: "https://matrix.homeserverserver.org",
delayId: "mock_delay_id",
},
);
} catch (ex) {
expect((ex as Error).message).toEqual(
"SFU Config fetch failed with status code 500",
);
void (await fetchMock.flush());
}
const calls = fetchMock.calls();
expect(calls.length).toBe(1);
expect(calls[0][0]).toStrictEqual("https://sfu.example.org/get_token");
expect(calls[0][1]).toStrictEqual({
// check if it uses correct delayID!
body: '{"room_id":"!example_room_id","slot_id":"m.call#ROOM","member":{"id":"@alice:example.org:DEVICE","claimed_user_id":"@alice:example.org","claimed_device_id":"DEVICE"},"delay_id":"mock_delay_id","delay_timeout":1000,"delay_cs_api_url":"https://matrix.homeserverserver.org"}',
method: "POST",
headers: {
"Content-Type": "application/json",
},
});
});
it("should retry fetching the openid token", async () => {
let count = 0;
matrixClient.getOpenIdToken.mockImplementation(async () => {
@@ -98,6 +201,7 @@ describe("getSFUConfigWithOpenID", () => {
});
const config = await getSFUConfigWithOpenID(
matrixClient,
ownMemberMock,
"https://sfu.example.org",
"!example_room_id",
);

View File

@@ -5,11 +5,21 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type IOpenIDToken, type MatrixClient } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import {
retryNetworkOperation,
type IOpenIDToken,
type MatrixClient,
} from "matrix-js-sdk";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { FailToGetOpenIdToken } from "../utils/errors";
import {
FailToGetOpenIdToken,
NoMatrix2AuthorizationService,
} from "../utils/errors";
import { doNetworkOperationWithRetry } from "../utils/matrix";
import { Config } from "../config/Config";
import { JwtEndpointVersion } from "../state/CallViewModel/localMember/LocalTransport";
/**
* Configuration and access tokens provided by the SFU on successful authentication.
@@ -18,6 +28,7 @@ export interface SFUConfig {
url: string;
jwt: string;
livekitAlias: string;
// NOTE: Currently unused.
livekitIdentity: string;
}
@@ -64,15 +75,32 @@ export type OpenIDClientParts = Pick<
* to the matrix RTC backend in order to get acces to the SFU.
* It has built-in retry for calls to the homeserver with a backoff policy.
* @param client The Matrix client
* @param membership Our own membership identity parts used to send to jwt service.
* @param serviceUrl The URL of the livekit SFU service
* @param matrixRoomId The Matrix room ID for which to get the SFU config
* @param roomId The room id used in the jwt request. This is NOT the livekit_alias. The jwt service will provide the alias. It maps matrix room ids <-> Livekit aliases.
* @param opts Additional options to modify which endpoint with which data will be used to acquire the jwt token.
* @param opts.forceJwtEndpoint This will use the old jwt endpoint which will create the rtc backend identity based on string concatenation
* instead of a hash.
* This function by default uses whatever is possible with the current jwt service installed next to the SFU.
* For remote connections this does not matter, since we will not publish there we can rely on the newest option.
* For our own connection we can only use the hashed version if we also send the new matrix2.0 sticky events.
* @param opts.delayEndpointBaseUrl The URL of the matrix homeserver.
* @param opts.delayId The delay id used for the jwt service to manage.
* @param logger optional logger.
* @returns Object containing the token information
* @throws FailToGetOpenIdToken
*/
export async function getSFUConfigWithOpenID(
client: OpenIDClientParts,
membership: CallMembershipIdentityParts,
serviceUrl: string,
matrixRoomId: string,
roomId: string,
opts?: {
forceJwtEndpoint?: JwtEndpointVersion;
delayEndpointBaseUrl?: string;
delayId?: string;
},
logger?: Logger,
): Promise<SFUConfig> {
let openIdToken: IOpenIDToken;
try {
@@ -84,16 +112,67 @@ export async function getSFUConfigWithOpenID(
error instanceof Error ? error : new Error("Unknown error"),
);
}
logger.debug("Got openID token", openIdToken);
logger?.debug("Got openID token", openIdToken);
logger.info(`Trying to get JWT for focus ${serviceUrl}...`);
const sfuConfig = await getLiveKitJWT(
client,
serviceUrl,
matrixRoomId,
openIdToken,
);
logger.info(`Got JWT from call's active focus URL.`);
logger?.info(`Trying to get JWT for focus ${serviceUrl}...`);
let sfuConfig: { url: string; jwt: string } | undefined;
const tryBothJwtEndpoints = opts?.forceJwtEndpoint === undefined; // This is for SFUs where we do not publish.
const forceMatrix2Jwt =
opts?.forceJwtEndpoint === JwtEndpointVersion.Matrix_2_0;
// We want to start using the new endpoint (with optional delay delegation)
// if we can use both or if we are forced to use the new one.
if (tryBothJwtEndpoints || forceMatrix2Jwt) {
try {
sfuConfig = await getLiveKitJWTWithDelayDelegation(
membership,
serviceUrl,
roomId,
openIdToken,
opts?.delayEndpointBaseUrl,
opts?.delayId,
);
logger?.info(`Got JWT from call's active focus URL.`);
} catch (e) {
if (e instanceof NotSupportedError) {
logger?.warn(
`Failed fetching jwt with matrix 2.0 endpoint (retry with legacy) Not supported`,
e,
);
sfuConfig = undefined;
} else {
logger?.warn(
`Failed fetching jwt with matrix 2.0 endpoint other issues ->`,
`(not going to try with legacy endpoint: forceOldJwtEndpoint is set to false, we did not get a not supported error from the sfu)`,
e,
);
// Make this throw a hard error in case we force the matrix2.0 endpoint.
if (forceMatrix2Jwt)
throw new NoMatrix2AuthorizationService(e as Error);
// NEVER get bejond this point if we forceMatrix2 and it failed!
}
}
}
// DEPRECATED
// here we either have a sfuConfig or we alredy exited because of `if (forceMatrix2) throw ...`
// The only case we can get into this condition is, if `forceMatrix2` is `false`
if (sfuConfig === undefined) {
sfuConfig = await getLiveKitJWT(
membership.deviceId,
serviceUrl,
roomId,
openIdToken,
);
logger?.info(`Got JWT from call's active focus URL.`);
}
if (!sfuConfig) {
throw new Error("No `sfuConfig` after trying with old and new endpoints");
}
// Pull the details from the JWT
const [, payloadStr] = sfuConfig.jwt.split(".");
@@ -104,33 +183,108 @@ export async function getSFUConfigWithOpenID(
url: sfuConfig.url,
livekitAlias: payload.video.room,
// NOTE: Currently unused.
// Probably also not helpful since we now compute the backendIdentity on joining the call so we can use it for the encryption manager.
// The only reason for us to know it locally is to connect the right users with the lk world. (and to set our own keys)
livekitIdentity: payload.sub,
};
}
const RETRIES = 4;
async function getLiveKitJWT(
client: OpenIDClientParts,
deviceId: string,
livekitServiceURL: string,
roomName: string,
matrixRoomId: string,
openIDToken: IOpenIDToken,
): Promise<{ url: string; jwt: string }> {
try {
const res = await fetch(livekitServiceURL + "/sfu/get", {
let res: Response | undefined;
await retryNetworkOperation(RETRIES, async () => {
res = await fetch(livekitServiceURL + "/sfu/get", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
room: roomName,
// This is the actual livekit room alias. For the legacy jwt endpoint simply the room id was used.
room: matrixRoomId,
openid_token: openIDToken,
device_id: client.getDeviceId(),
device_id: deviceId,
}),
});
if (!res.ok) {
throw new Error("SFU Config fetch failed with status code " + res.status);
}
return await res.json();
} catch (e) {
throw new Error("SFU Config fetch failed with exception", { cause: e });
});
if (!res) {
throw new Error(
`Network error while connecting to jwt service after ${RETRIES} retries`,
);
}
if (!res.ok) {
throw new Error("SFU Config fetch failed with status code " + res.status);
}
return await res.json();
}
class NotSupportedError extends Error {
public constructor(message: string) {
super(message);
this.name = "NotSupported";
}
}
export async function getLiveKitJWTWithDelayDelegation(
membership: CallMembershipIdentityParts,
livekitServiceURL: string,
matrixRoomId: string,
openIDToken: IOpenIDToken,
delayEndpointBaseUrl?: string,
delayId?: string,
): Promise<{ url: string; jwt: string }> {
const { userId, deviceId, memberId } = membership;
const body = {
room_id: matrixRoomId,
slot_id: "m.call#ROOM",
openid_token: openIDToken,
member: {
id: memberId,
claimed_user_id: userId,
claimed_device_id: deviceId,
},
};
let bodyDalayParts = {};
// Also check for empty string
if (delayId && delayEndpointBaseUrl) {
const delayTimeoutMs =
Config.get().matrix_rtc_session?.delayed_leave_event_delay_ms ?? 1000;
bodyDalayParts = {
delay_id: delayId,
delay_timeout: delayTimeoutMs,
delay_cs_api_url: delayEndpointBaseUrl,
};
}
let res: Response | undefined;
await retryNetworkOperation(RETRIES, async () => {
res = await fetch(livekitServiceURL + "/get_token", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({ ...body, ...bodyDalayParts }),
});
});
if (!res) {
throw new Error(
`Network error while connecting to jwt service after ${RETRIES} retries`,
);
}
if (!res.ok) {
const msg = "SFU Config fetch failed with status code " + res.status;
if (res.status === 404) {
throw new NotSupportedError(msg);
} else {
throw new Error(msg);
}
}
return await res.json();
}

View File

@@ -67,6 +67,6 @@ Initializer.initBeforeReact()
);
})
.catch((e) => {
logger.error("Failed to initialize app", e);
logger.error(`Failed to initialize app ${e.message}`, e);
root.render(e.message);
});

View File

@@ -39,7 +39,8 @@ import {
localRtcMember,
} from "../utils/test-fixtures";
import { MAX_PARTICIPANT_COUNT_FOR_SOUND } from "../state/CallViewModel/CallViewModel";
import { initializeWidget } from "../widget";
initializeWidget();
vitest.mock("livekit-client/e2ee-worker?worker");
vitest.mock("../useAudioContext");
vitest.mock("../soundUtils");

View File

@@ -65,6 +65,7 @@ Please see LICENSE in the repository root for full details.
.footer.overlay.hidden {
display: grid;
opacity: 0;
pointer-events: none;
}
.footer.overlay:has(:focus-visible) {
@@ -107,22 +108,9 @@ Please see LICENSE in the repository root for full details.
}
}
@media (max-width: 370px) {
.shareScreen {
display: none;
}
@media (max-height: 400px) {
.footer {
display: none;
}
}
}
@media (max-width: 320px) {
.invite,
.raiseHand {
display: none;
@media (max-height: 800px) {
.footer {
padding-block: var(--cpd-space-8x);
}
}
@@ -132,9 +120,27 @@ Please see LICENSE in the repository root for full details.
}
}
@media (max-height: 800px) {
.footer {
padding-block: var(--cpd-space-8x);
@media (max-width: 370px) {
.shareScreen {
display: none;
}
/* PIP custom css */
@media (max-height: 400px) {
.shareScreen {
display: flex;
}
.footer {
padding-block-start: var(--cpd-space-3x);
padding-block-end: var(--cpd-space-2x);
}
}
}
@media (max-width: 320px) {
.invite,
.raiseHand {
display: none;
}
}

View File

@@ -1,5 +1,6 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -42,7 +43,8 @@ import { useRoomEncryptionSystem } from "../e2ee/sharedKeyManagement";
import { LivekitRoomAudioRenderer } from "../livekit/MatrixAudioRenderer";
import { MediaDevicesContext } from "../MediaDevicesContext";
import { HeaderStyle } from "../UrlParams";
import { initializeWidget } from "../widget";
initializeWidget();
vi.hoisted(
() =>
(global.ImageData = class MockImageData {

View File

@@ -5,12 +5,12 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { IconButton, Text, Tooltip } from "@vector-im/compound-web";
import { IconButton, Tooltip } from "@vector-im/compound-web";
import { type MatrixClient, type Room as MatrixRoom } from "matrix-js-sdk";
import {
type FC,
type PointerEvent,
type TouchEvent,
type MouseEvent as ReactMouseEvent,
type PointerEvent as ReactPointerEvent,
useCallback,
useEffect,
useMemo,
@@ -33,7 +33,6 @@ import { useTranslation } from "react-i18next";
import LogoMark from "../icons/LogoMark.svg?react";
import LogoType from "../icons/LogoType.svg?react";
import type { IWidgetApiRequest } from "matrix-widget-api";
import {
EndCallButton,
MicButton,
@@ -45,7 +44,7 @@ import {
import { Header, LeftNav, RightNav, RoomHeaderInfo } from "../Header";
import { type HeaderStyle, useUrlParams } from "../UrlParams";
import { useCallViewKeyboardShortcuts } from "../useCallViewKeyboardShortcuts";
import { ElementWidgetActions, widget } from "../widget";
import { widget } from "../widget";
import styles from "./InCallView.module.css";
import { GridTile } from "../tile/GridTile";
import { SettingsModal, defaultSettingsTab } from "../settings/SettingsModal";
@@ -99,8 +98,6 @@ import { useAppBarHidden, useAppBarSecondaryButton } from "../AppBar.tsx";
import { useBehavior } from "../useBehavior.ts";
import { Toast } from "../Toast.tsx";
import overlayStyles from "../Overlay.module.css";
import { Avatar, Size as AvatarSize } from "../Avatar";
import waitingStyles from "./WaitingForJoin.module.css";
import { prefetchSounds } from "../soundUtils";
import { useAudioContext } from "../useAudioContext";
import ringtoneMp3 from "../sound/ringtone.mp3?url";
@@ -108,11 +105,10 @@ import ringtoneOgg from "../sound/ringtone.ogg?url";
import { useTrackProcessorObservable$ } from "../livekit/TrackProcessorContext.tsx";
import { type Layout } from "../state/layout-types.ts";
import { ObservableScope } from "../state/ObservableScope.ts";
import { useLatest } from "../useLatest.ts";
const logger = rootLogger.getChild("[InCallView]");
const maxTapDurationMs = 400;
export interface ActiveCallProps extends Omit<
InCallViewProps,
"vm" | "livekitRoom" | "connState"
@@ -227,8 +223,6 @@ export const InCallView: FC<InCallViewProps> = ({
const { showControls } = useUrlParams();
const muteAllAudio = useBehavior(muteAllAudio$);
// Call pickup state and display names are needed for waiting overlay/sounds
const callPickupState = useBehavior(vm.callPickupState$);
// Preload a waiting and decline sounds
const pickupPhaseSoundCache = useInitial(async () => {
@@ -242,6 +236,7 @@ export const InCallView: FC<InCallViewProps> = ({
latencyHint: "interactive",
muted: muteAllAudio,
});
const latestPickupPhaseAudio = useLatest(pickupPhaseAudio);
const audioEnabled = useBehavior(muteStates.audio.enabled$);
const videoEnabled = useBehavior(muteStates.video.enabled$);
@@ -260,6 +255,7 @@ export const InCallView: FC<InCallViewProps> = ({
() => void toggleRaisedHand(),
);
const ringing = useBehavior(vm.ringing$);
const audioParticipants = useBehavior(vm.livekitRoomItems$);
const participantCount = useBehavior(vm.participantCount$);
const reconnecting = useBehavior(vm.reconnecting$);
@@ -274,7 +270,6 @@ export const InCallView: FC<InCallViewProps> = ({
const audioOutputSwitcher = useBehavior(vm.audioOutputSwitcher$);
const sharingScreen = useBehavior(vm.sharingScreen$);
const ringOverlay = useBehavior(vm.ringOverlay$);
const fatalCallError = useBehavior(vm.fatalError$);
// Stop the rendering and throw for the error boundary
if (fatalCallError) {
@@ -282,93 +277,36 @@ export const InCallView: FC<InCallViewProps> = ({
throw fatalCallError;
}
// We need to set the proper timings on the animation based upon the sound length.
const ringDuration = pickupPhaseAudio?.soundDuration["waiting"] ?? 1;
useEffect((): (() => void) => {
// The CSS animation includes the delay, so we must double the length of the sound.
window.document.body.style.setProperty(
"--call-ring-duration-s",
`${ringDuration * 2}s`,
);
window.document.body.style.setProperty(
"--call-ring-delay-s",
`${ringDuration}s`,
);
// Remove properties when we unload.
return () => {
window.document.body.style.removeProperty("--call-ring-duration-s");
window.document.body.style.removeProperty("--call-ring-delay-s");
};
}, [pickupPhaseAudio?.soundDuration, ringDuration]);
// When waiting for pickup, loop a waiting sound
// While ringing, loop the ringtone
useEffect((): void | (() => void) => {
if (callPickupState !== "ringing" || !pickupPhaseAudio) return;
const endSound = pickupPhaseAudio.playSoundLooping("waiting", ringDuration);
return () => {
void endSound().catch((e) => {
logger.error("Failed to stop ringing sound", e);
});
};
}, [callPickupState, pickupPhaseAudio, ringDuration]);
const audio = latestPickupPhaseAudio.current;
if (ringing && audio) {
const endSound = audio.playSoundLooping(
"waiting",
audio.soundDuration["waiting"] ?? 1,
);
return () => {
void endSound().catch((e) => {
logger.error("Failed to stop ringing sound", e);
});
};
}
}, [ringing, latestPickupPhaseAudio]);
// Waiting UI overlay
const waitingOverlay: JSX.Element | null = useMemo(() => {
return ringOverlay ? (
<div className={classNames(overlayStyles.bg, waitingStyles.overlay)}>
<div
className={classNames(overlayStyles.content, waitingStyles.content)}
>
<div className={waitingStyles.pulse}>
<Avatar
id={ringOverlay.idForAvatar}
name={ringOverlay.name}
src={ringOverlay.avatarMxc}
size={AvatarSize.XL}
/>
</div>
<Text size="md" className={waitingStyles.text}>
{ringOverlay.text}
</Text>
</div>
</div>
) : null;
}, [ringOverlay]);
// Ideally we could detect taps by listening for click events and checking
// that the pointerType of the event is "touch", but this isn't yet supported
// in Safari: https://developer.mozilla.org/en-US/docs/Web/API/Element/click_event#browser_compatibility
// Instead we have to watch for sufficiently fast touch events.
const touchStart = useRef<number | null>(null);
const onTouchStart = useCallback(() => (touchStart.current = Date.now()), []);
const onTouchEnd = useCallback(() => {
const start = touchStart.current;
if (start !== null && Date.now() - start <= maxTapDurationMs)
vm.tapScreen();
touchStart.current = null;
}, [vm]);
const onTouchCancel = useCallback(() => (touchStart.current = null), []);
// We also need to tell the footer controls to prevent touch events from
// bubbling up, or else the footer will be dismissed before a click/change
// event can be registered on the control
const onControlsTouchEnd = useCallback(
(e: TouchEvent) => {
// Somehow applying pointer-events: none to the controls when the footer
// is hidden is not enough to stop clicks from happening as the footer
// becomes visible, so we check manually whether the footer is shown
if (showFooter) {
e.stopPropagation();
vm.tapControls();
} else {
e.preventDefault();
}
const onViewClick = useCallback(
(e: ReactMouseEvent) => {
if (
(e.nativeEvent as PointerEvent).pointerType === "touch" &&
// If an interactive element was tapped, don't count this as a tap on the screen
(e.target as Element).closest?.("button, input") === null
)
vm.tapScreen();
},
[vm, showFooter],
[vm],
);
const onPointerMove = useCallback(
(e: PointerEvent) => {
(e: ReactPointerEvent) => {
if (e.pointerType === "mouse") vm.hoverScreen();
},
[vm],
@@ -435,46 +373,6 @@ export const InCallView: FC<InCallViewProps> = ({
[vm],
);
useEffect(() => {
widget?.api.transport
.send(
gridMode === "grid"
? ElementWidgetActions.TileLayout
: ElementWidgetActions.SpotlightLayout,
{},
)
.catch((e) => {
logger.error("Failed to send layout change to widget API", e);
});
}, [gridMode]);
useEffect(() => {
if (widget) {
const onTileLayout = (ev: CustomEvent<IWidgetApiRequest>): void => {
setGridMode("grid");
widget!.api.transport.reply(ev.detail, {});
};
const onSpotlightLayout = (ev: CustomEvent<IWidgetApiRequest>): void => {
setGridMode("spotlight");
widget!.api.transport.reply(ev.detail, {});
};
widget.lazyActions.on(ElementWidgetActions.TileLayout, onTileLayout);
widget.lazyActions.on(
ElementWidgetActions.SpotlightLayout,
onSpotlightLayout,
);
return (): void => {
widget!.lazyActions.off(ElementWidgetActions.TileLayout, onTileLayout);
widget!.lazyActions.off(
ElementWidgetActions.SpotlightLayout,
onSpotlightLayout,
);
};
}
}, [setGridMode]);
useAppBarSecondaryButton(
useMemo(() => {
if (audioOutputSwitcher === null) return null;
@@ -647,8 +545,8 @@ export const InCallView: FC<InCallViewProps> = ({
vm={layout.spotlight}
expanded
onToggleExpanded={null}
targetWidth={gridBounds.height}
targetHeight={gridBounds.width}
targetWidth={gridBounds.width}
targetHeight={gridBounds.height}
showIndicators={false}
focusable={!contentObscured}
aria-hidden={contentObscured}
@@ -703,20 +601,21 @@ export const InCallView: FC<InCallViewProps> = ({
const buttons: JSX.Element[] = [];
const buttonSize = layout.type === "pip" ? "sm" : "lg";
buttons.push(
<MicButton
size={buttonSize}
key="audio"
muted={!audioEnabled}
enabled={audioEnabled}
onClick={toggleAudio ?? undefined}
onTouchEnd={onControlsTouchEnd}
disabled={toggleAudio === null}
data-testid="incall_mute"
/>,
<VideoButton
size={buttonSize}
key="video"
muted={!videoEnabled}
enabled={videoEnabled}
onClick={toggleVideo ?? undefined}
onTouchEnd={onControlsTouchEnd}
disabled={toggleVideo === null}
data-testid="incall_videomute"
/>,
@@ -724,11 +623,11 @@ export const InCallView: FC<InCallViewProps> = ({
if (vm.toggleScreenSharing !== null) {
buttons.push(
<ShareScreenButton
size={buttonSize}
key="share_screen"
className={styles.shareScreen}
enabled={sharingScreen}
onClick={vm.toggleScreenSharing}
onTouchEnd={onControlsTouchEnd}
data-testid="incall_screenshare"
/>,
);
@@ -736,30 +635,30 @@ export const InCallView: FC<InCallViewProps> = ({
if (supportsReactions) {
buttons.push(
<ReactionToggleButton
size={buttonSize}
vm={vm}
key="raise_hand"
className={styles.raiseHand}
identifier={`${client.getUserId()}:${client.getDeviceId()}`}
onTouchEnd={onControlsTouchEnd}
/>,
);
}
if (layout.type !== "pip")
buttons.push(
<SettingsButton
size={buttonSize}
key="settings"
onClick={openSettings}
onTouchEnd={onControlsTouchEnd}
/>,
);
buttons.push(
<EndCallButton
size={buttonSize}
key="end_call"
onClick={function (): void {
vm.hangup();
}}
onTouchEnd={onControlsTouchEnd}
data-testid="incall_leave"
/>,
);
@@ -792,19 +691,21 @@ export const InCallView: FC<InCallViewProps> = ({
className={styles.layout}
layout={gridMode}
setLayout={setGridMode}
onTouchEnd={onControlsTouchEnd}
/>
)}
</div>
);
const allConnections = useBehavior(vm.allConnections$);
return (
// The onClick handler here exists to control the visibility of the footer,
// and the footer is also viewable by moving focus into it, so this is fine.
// eslint-disable-next-line jsx-a11y/no-static-element-interactions, jsx-a11y/click-events-have-key-events
<div
className={styles.inRoom}
ref={containerRef}
onTouchStart={onTouchStart}
onTouchEnd={onTouchEnd}
onTouchCancel={onTouchCancel}
onClick={onViewClick}
onPointerMove={onPointerMove}
onPointerOut={onPointerOut}
>
@@ -824,7 +725,6 @@ export const InCallView: FC<InCallViewProps> = ({
{reconnectingToast}
{earpieceOverlay}
<ReactionsOverlay vm={vm} />
{waitingOverlay}
{footer}
{layout.type !== "pip" && (
<>
@@ -836,8 +736,15 @@ export const InCallView: FC<InCallViewProps> = ({
onDismiss={closeSettings}
tab={settingsTab}
onTabChange={setSettingsTab}
// TODO expose correct data to setttings modal
livekitRooms={[]}
livekitRooms={allConnections
.getConnections()
.map((connectionItem) => ({
room: connectionItem.livekitRoom,
livekitAlias: connectionItem.livekitAlias,
// TODO compute is local or tag it in the livekit room items already
isLocal: undefined,
url: connectionItem.transport.livekit_service_url,
}))}
/>
</>
)}

View File

@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type ChangeEvent, type FC, type TouchEvent, useCallback } from "react";
import { type ChangeEvent, type FC, useCallback } from "react";
import { useTranslation } from "react-i18next";
import { Tooltip } from "@vector-im/compound-web";
import {
@@ -22,15 +22,9 @@ interface Props {
layout: Layout;
setLayout: (layout: Layout) => void;
className?: string;
onTouchEnd?: (e: TouchEvent) => void;
}
export const LayoutToggle: FC<Props> = ({
layout,
setLayout,
className,
onTouchEnd,
}) => {
export const LayoutToggle: FC<Props> = ({ layout, setLayout, className }) => {
const { t } = useTranslation();
const onChange = useCallback(
@@ -47,7 +41,6 @@ export const LayoutToggle: FC<Props> = ({
value="spotlight"
checked={layout === "spotlight"}
onChange={onChange}
onTouchEnd={onTouchEnd}
/>
</Tooltip>
<SpotlightIcon aria-hidden width={24} height={24} />
@@ -58,7 +51,6 @@ export const LayoutToggle: FC<Props> = ({
value="grid"
checked={layout === "grid"}
onChange={onChange}
onTouchEnd={onTouchEnd}
/>
</Tooltip>
<GridIcon aria-hidden width={24} height={24} />

View File

@@ -230,12 +230,12 @@ export const LobbyView: FC<Props> = ({
{recentsButtonInFooter && recentsButton}
<div className={inCallStyles.buttons}>
<MicButton
muted={!audioEnabled}
enabled={audioEnabled}
onClick={toggleAudio ?? undefined}
disabled={toggleAudio === null}
/>
<VideoButton
muted={!videoEnabled}
enabled={videoEnabled}
onClick={toggleVideo ?? undefined}
disabled={toggleVideo === null}
/>

View File

@@ -37,6 +37,8 @@ import {
local,
localRtcMember,
} from "../utils/test-fixtures";
import { initializeWidget } from "../widget";
initializeWidget();
function TestComponent({ vm }: { vm: CallViewModel }): ReactNode {
return (

View File

@@ -19,6 +19,8 @@ import {
bobRtcMember,
} from "../utils/test-fixtures";
import { getBasicCallViewModelEnvironment } from "../utils/test-viewmodel";
import { initializeWidget } from "../widget";
initializeWidget();
vi.mock("livekit-client/e2ee-worker?worker");

View File

@@ -1,5 +1,6 @@
/*
Copyright 2021-2024 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -76,10 +77,9 @@ export const RoomPage: FC = () => {
scope,
devices,
calculateInitialMuteState(
urlParams,
import.meta.env.VITE_PACKAGE,
window.location.hostname,
Config.get().trust_localhost_for_mute_state,
urlParams.skipLobby,
urlParams.callIntent,
widget !== null,
),
),
);

View File

@@ -1,61 +0,0 @@
.overlay {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
pointer-events: none;
}
.content {
display: flex;
flex-direction: column;
align-items: center;
gap: 16px;
}
.pulse {
position: relative;
height: 90px;
}
.pulse::before {
content: "";
position: absolute;
inset: -12px;
border-radius: 9999px;
border: 12px solid rgba(255, 255, 255, 0.6);
animation: pulse var(--call-ring-duration-s) ease-out infinite;
animation-delay: 1s;
opacity: 0;
}
.text {
color: var(--cpd-color-text-on-solid-primary);
}
@keyframes pulse {
0% {
transform: scale(0.95);
opacity: 0.7;
transform: scale(0);
opacity: 1;
}
35% {
transform: scale(1.15);
opacity: 0.15;
}
50% {
transform: scale(1.2);
opacity: 0;
}
50.01% {
transform: scale(0);
}
85% {
transform: scale(0);
}
100% {
transform: scale(0);
}
}

View File

@@ -108,7 +108,7 @@ exports[`ConnectionLostError: Action handling should reset error state 1`] = `
class="error"
>
<div
class="_content_o77nw_8 icon"
class="_content_1r8kr_8 icon"
data-size="large"
>
<svg
@@ -133,7 +133,7 @@ exports[`ConnectionLostError: Action handling should reset error state 1`] = `
You were disconnected from the call.
</p>
<button
class="_button_vczzf_8"
class="_button_13vu4_8"
data-kind="secondary"
data-size="lg"
role="button"
@@ -142,7 +142,7 @@ exports[`ConnectionLostError: Action handling should reset error state 1`] = `
Reconnect
</button>
<button
class="_button_vczzf_8 homeLink"
class="_button_13vu4_8 homeLink"
data-kind="tertiary"
data-size="lg"
role="button"
@@ -265,7 +265,7 @@ exports[`should have a close button in widget mode 1`] = `
class="error"
>
<div
class="_content_o77nw_8 icon"
class="_content_1r8kr_8 icon"
data-size="large"
>
<svg
@@ -295,7 +295,7 @@ exports[`should have a close button in widget mode 1`] = `
The server is not configured to work with Element Call. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
</p>
<button
class="_button_vczzf_8"
class="_button_13vu4_8"
data-kind="primary"
data-size="lg"
role="button"
@@ -418,7 +418,7 @@ exports[`should render the error page with link back to home 1`] = `
class="error"
>
<div
class="_content_o77nw_8 icon"
class="_content_1r8kr_8 icon"
data-size="large"
>
<svg
@@ -448,7 +448,7 @@ exports[`should render the error page with link back to home 1`] = `
The server is not configured to work with Element Call. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
</p>
<button
class="_button_vczzf_8 homeLink"
class="_button_13vu4_8 homeLink"
data-kind="tertiary"
data-size="lg"
role="button"
@@ -571,7 +571,7 @@ exports[`should report correct error for 'Call is not supported' 1`] = `
class="error"
>
<div
class="_content_o77nw_8 icon"
class="_content_1r8kr_8 icon"
data-size="large"
>
<svg
@@ -601,7 +601,7 @@ exports[`should report correct error for 'Call is not supported' 1`] = `
The server is not configured to work with Element Call. Please contact your server admin (Domain: example.com, Error Code: MISSING_MATRIX_RTC_TRANSPORT).
</p>
<button
class="_button_vczzf_8 homeLink"
class="_button_13vu4_8 homeLink"
data-kind="tertiary"
data-size="lg"
role="button"
@@ -724,7 +724,7 @@ exports[`should report correct error for 'Connection lost' 1`] = `
class="error"
>
<div
class="_content_o77nw_8 icon"
class="_content_1r8kr_8 icon"
data-size="large"
>
<svg
@@ -749,7 +749,7 @@ exports[`should report correct error for 'Connection lost' 1`] = `
You were disconnected from the call.
</p>
<button
class="_button_vczzf_8"
class="_button_13vu4_8"
data-kind="secondary"
data-size="lg"
role="button"
@@ -758,7 +758,7 @@ exports[`should report correct error for 'Connection lost' 1`] = `
Reconnect
</button>
<button
class="_button_vczzf_8 homeLink"
class="_button_13vu4_8 homeLink"
data-kind="tertiary"
data-size="lg"
role="button"
@@ -881,7 +881,7 @@ exports[`should report correct error for 'Incompatible browser' 1`] = `
class="error"
>
<div
class="_content_o77nw_8 icon"
class="_content_1r8kr_8 icon"
data-size="large"
>
<svg
@@ -906,7 +906,7 @@ exports[`should report correct error for 'Incompatible browser' 1`] = `
Your web browser does not support encrypted calls. Supported browsers include Chrome, Safari, and Firefox 117+.
</p>
<button
class="_button_vczzf_8 homeLink"
class="_button_13vu4_8 homeLink"
data-kind="tertiary"
data-size="lg"
role="button"
@@ -1029,7 +1029,7 @@ exports[`should report correct error for 'Insufficient capacity' 1`] = `
class="error"
>
<div
class="_content_o77nw_8 icon"
class="_content_1r8kr_8 icon"
data-size="large"
>
<svg
@@ -1054,7 +1054,7 @@ exports[`should report correct error for 'Insufficient capacity' 1`] = `
The server has reached its maximum capacity and you cannot join the call at this time. Try again later, or contact your server admin if the problem persists.
</p>
<button
class="_button_vczzf_8 homeLink"
class="_button_13vu4_8 homeLink"
data-kind="tertiary"
data-size="lg"
role="button"

View File

@@ -17,7 +17,7 @@ exports[`InCallView > rendering > renders 1`] = `
>
<span
aria-label=""
class="_avatar_1qbcf_8 roomAvatar _avatar-imageless_1qbcf_52"
class="_avatar_zysgz_8 roomAvatar _avatar-imageless_zysgz_55"
data-color="1"
data-type="round"
role="img"
@@ -117,7 +117,7 @@ exports[`InCallView > rendering > renders 1`] = `
data-show="false"
>
<div
class="_content_o77nw_8 icon"
class="_content_1r8kr_8 icon"
data-size="large"
>
<svg
@@ -128,24 +128,11 @@ exports[`InCallView > rendering > renders 1`] = `
width="1em"
xmlns="http://www.w3.org/2000/svg"
>
<g
clip-path="url(#a)"
>
<path
clip-rule="evenodd"
d="M8.929 15.1a13.6 13.6 0 0 0 4.654 3.066q2.62 1.036 5.492.923h.008l.003-.004.003-.002-.034-3.124-3.52-.483-1.791 1.792-.645-.322a13.5 13.5 0 0 1-3.496-2.52 13.4 13.4 0 0 1-2.52-3.496l-.322-.644 1.792-1.792-.483-3.519-3.123-.034-.003.002-.003.004v.002a13.65 13.65 0 0 0 .932 5.492A13.4 13.4 0 0 0 8.93 15.1m3.92 4.926a15.6 15.6 0 0 1-5.334-3.511 15.4 15.4 0 0 1-3.505-5.346 15.6 15.6 0 0 1-1.069-6.274 1.93 1.93 0 0 1 .589-1.366c.366-.366.84-.589 1.386-.589h.01l3.163.035a1.96 1.96 0 0 1 1.958 1.694v.005l.487 3.545v.003c.043.297.025.605-.076.907a2 2 0 0 1-.485.773l-.762.762a11.4 11.4 0 0 0 3.206 3.54q.457.33.948.614l.762-.761a2 2 0 0 1 .774-.486c.302-.1.61-.118.907-.076l3.553.487a1.96 1.96 0 0 1 1.694 1.958l.034 3.174c0 .546-.223 1.02-.588 1.386-.361.36-.827.582-1.363.588a15.3 15.3 0 0 1-6.29-1.062"
fill-rule="evenodd"
/>
</g>
<defs>
<clippath
id="a"
>
<path
d="M0 0h24v24H0z"
/>
</clippath>
</defs>
<path
clip-rule="evenodd"
d="M8.929 15.1a13.6 13.6 0 0 0 4.654 3.066q2.62 1.036 5.492.923h.008l.003-.004.003-.002-.034-3.124-3.52-.483-1.791 1.792-.645-.322a13.5 13.5 0 0 1-3.496-2.52 13.4 13.4 0 0 1-2.52-3.496l-.322-.645 1.792-1.791-.483-3.52-3.123-.033-.003.002-.003.004v.002a13.65 13.65 0 0 0 .932 5.492A13.4 13.4 0 0 0 8.93 15.1m3.92 4.926a15.6 15.6 0 0 1-5.334-3.511 15.4 15.4 0 0 1-3.505-5.346 15.6 15.6 0 0 1-1.069-6.274 1.93 1.93 0 0 1 .589-1.366c.366-.366.84-.589 1.386-.589h.01l3.163.035a1.96 1.96 0 0 1 1.958 1.694v.005l.487 3.545v.003c.043.297.025.605-.076.907a2 2 0 0 1-.485.773l-.762.762a11.3 11.3 0 0 0 1.806 2.348 11.4 11.4 0 0 0 2.348 1.806l.762-.762a2 2 0 0 1 .774-.485c.302-.1.61-.118.907-.076l3.553.487a1.96 1.96 0 0 1 1.694 1.958l.034 3.174c0 .546-.223 1.02-.588 1.386-.36.36-.827.582-1.363.588a15.3 15.3 0 0 1-6.29-1.062"
fill-rule="evenodd"
/>
</svg>
</div>
<h2
@@ -159,7 +146,7 @@ exports[`InCallView > rendering > renders 1`] = `
Only works while using app
</p>
<button
class="_button_vczzf_8"
class="_button_13vu4_8"
data-kind="primary"
data-size="sm"
role="button"
@@ -285,14 +272,14 @@ exports[`InCallView > rendering > renders 1`] = `
class="buttons"
>
<button
aria-checked="false"
aria-disabled="true"
aria-label="Unmute microphone"
aria-labelledby="_r_8_"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
data-kind="primary"
class="_button_13vu4_8 _has-icon_13vu4_60 _icon-only_13vu4_53"
data-kind="secondary"
data-size="lg"
data-testid="incall_mute"
role="button"
role="switch"
tabindex="0"
>
<svg
@@ -309,14 +296,14 @@ exports[`InCallView > rendering > renders 1`] = `
</svg>
</button>
<button
aria-checked="false"
aria-disabled="true"
aria-label="Start video"
aria-labelledby="_r_d_"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
data-kind="primary"
class="_button_13vu4_8 _has-icon_13vu4_60 _icon-only_13vu4_53"
data-kind="secondary"
data-size="lg"
data-testid="incall_videomute"
role="button"
role="switch"
tabindex="0"
>
<svg
@@ -334,7 +321,7 @@ exports[`InCallView > rendering > renders 1`] = `
</button>
<button
aria-labelledby="_r_i_"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
class="_button_13vu4_8 _has-icon_13vu4_60 _icon-only_13vu4_53"
data-kind="secondary"
data-size="lg"
role="button"
@@ -354,9 +341,8 @@ exports[`InCallView > rendering > renders 1`] = `
</svg>
</button>
<button
aria-label="End call"
aria-labelledby="_r_n_"
class="_button_vczzf_8 endCall _has-icon_vczzf_57 _icon-only_vczzf_50 _destructive_vczzf_107"
class="_button_13vu4_8 endCall _has-icon_13vu4_60 _icon-only_13vu4_53 _destructive_13vu4_110"
data-kind="primary"
data-size="lg"
data-testid="incall_leave"

View File

@@ -8,3 +8,14 @@ Please see LICENSE in the repository root for full details.
pre {
font-size: var(--font-size-micro);
}
.livekit_room_box {
border: 3px solid var(--cpd-color-bg-subtle-secondary);
border-radius: var(--cpd-space-8x);
padding: var(--cpd-space-4x);
margin-bottom: var(--cpd-space-4x);
margin-top: var(--cpd-space-4x);
li {
font-size: var(--font-size-micro);
}
}

View File

@@ -5,13 +5,16 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, vi } from "vitest";
import { render, waitFor } from "@testing-library/react";
import { afterEach, describe, expect, it, type Mock, vi } from "vitest";
import { render, waitFor, screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { TooltipProvider } from "@vector-im/compound-web";
import type { MatrixClient } from "matrix-js-sdk";
import type { Room as LivekitRoom } from "livekit-client";
import { DeveloperSettingsTab } from "./DeveloperSettingsTab";
import { getSFUConfigWithOpenID } from "../livekit/openIDSFU";
import { customLivekitUrl as customLivekitUrlSetting } from "./settings";
// Mock url params hook to avoid environment-dependent snapshot churn.
vi.mock("../UrlParams", () => ({
useUrlParams: (): { mocked: boolean; answer: number } => ({
@@ -20,22 +23,33 @@ vi.mock("../UrlParams", () => ({
}),
}));
// IMPORTANT: mock the same specifier used by DeveloperSettingsTab
vi.mock("../livekit/openIDSFU", () => ({
getSFUConfigWithOpenID: vi.fn().mockResolvedValue({
url: "mock-url",
jwt: "mock-jwt",
}),
}));
// Provide a minimal mock of a Livekit Room structure used by the component.
function createMockLivekitRoom(
wsUrl: string,
serverInfo: object,
metadata: string,
): { isLocal: boolean; url: string; room: LivekitRoom } {
): { isLocal: boolean; url: string; room: LivekitRoom; livekitAlias: string } {
const mockRoom = {
serverInfo,
metadata,
engine: { client: { ws: { url: wsUrl } } },
localParticipant: { identity: "localParticipantIdentity" },
remoteParticipants: new Map(),
} as unknown as LivekitRoom;
return {
isLocal: true,
url: wsUrl,
room: mockRoom,
livekitAlias: "TestAlias",
};
}
@@ -59,6 +73,7 @@ describe("DeveloperSettingsTab", () => {
room: LivekitRoom;
url: string;
isLocal?: boolean;
livekitAlias: string;
}[] = [
createMockLivekitRoom(
"wss://local-sfu.example.org",
@@ -67,8 +82,11 @@ describe("DeveloperSettingsTab", () => {
),
{
isLocal: false,
livekitAlias: "TestAlias2",
url: "wss://remote-sfu.example.org",
room: {
localParticipant: { identity: "localParticipantIdentity" },
remoteParticipants: new Map(),
serverInfo: { region: "remote", version: "4.5.6" },
metadata: "remote-metadata",
engine: { client: { ws: { url: "wss://remote-sfu.example.org" } } },
@@ -79,6 +97,7 @@ describe("DeveloperSettingsTab", () => {
const { container } = render(
<DeveloperSettingsTab
client={client}
roomId={"#room:example.org"}
livekitRooms={livekitRooms}
env={{ MY_MOCK_ENV: 10, ENV: "test" } as unknown as ImportMetaEnv}
/>,
@@ -92,4 +111,141 @@ describe("DeveloperSettingsTab", () => {
expect(container).toMatchSnapshot();
});
describe("custom livekit url", () => {
afterEach(() => {
customLivekitUrlSetting.setValue(null);
});
const client = {
doesServerSupportUnstableFeature: vi.fn().mockResolvedValue(true),
getCrypto: () => ({ getVersion: (): string => "x" }),
getUserId: () => "@u:hs",
getDeviceId: () => "DEVICE",
} as unknown as MatrixClient;
it("will not update custom livekit url without roomId", async () => {
const user = userEvent.setup();
render(
<TooltipProvider>
<DeveloperSettingsTab
client={client}
env={{} as unknown as ImportMetaEnv}
/>
</TooltipProvider>,
);
const input = screen.getByLabelText("Custom Livekit-url");
await user.clear(input);
await user.type(input, "wss://example.livekit.invalid");
const saveButton = screen.getByRole("button", { name: "Save" });
await user.click(saveButton);
expect(getSFUConfigWithOpenID).not.toHaveBeenCalled();
expect(customLivekitUrlSetting.getValue()).toBe(null);
});
it("will not update custom livekit url without text in input", async () => {
const user = userEvent.setup();
render(
<TooltipProvider>
<DeveloperSettingsTab
client={client}
roomId="#testRoom"
env={{} as unknown as ImportMetaEnv}
/>
</TooltipProvider>,
);
const input = screen.getByLabelText("Custom Livekit-url");
await user.clear(input);
const saveButton = screen.getByRole("button", { name: "Save" });
await user.click(saveButton);
expect(getSFUConfigWithOpenID).not.toHaveBeenCalled();
expect(customLivekitUrlSetting.getValue()).toBe(null);
});
it("will not update custom livekit url when pressing cancel", async () => {
const user = userEvent.setup();
render(
<TooltipProvider>
<DeveloperSettingsTab
client={client}
roomId="#testRoom"
env={{} as unknown as ImportMetaEnv}
/>
</TooltipProvider>,
);
const input = screen.getByLabelText("Custom Livekit-url");
await user.clear(input);
await user.type(input, "wss://example.livekit.invalid");
const cancelButton = screen.getByRole("button", {
name: "Reset overwrite",
});
await user.click(cancelButton);
expect(getSFUConfigWithOpenID).not.toHaveBeenCalled();
expect(customLivekitUrlSetting.getValue()).toBe(null);
});
it("will update custom livekit url", async () => {
const user = userEvent.setup();
render(
<TooltipProvider>
<DeveloperSettingsTab
client={client}
roomId="#testRoom"
env={{} as unknown as ImportMetaEnv}
/>
</TooltipProvider>,
);
const input = screen.getByLabelText("Custom Livekit-url");
await user.clear(input);
await user.type(input, "wss://example.livekit.valid");
const saveButton = screen.getByRole("button", { name: "Save" });
await user.click(saveButton);
expect(getSFUConfigWithOpenID).toHaveBeenCalledWith(
expect.anything(),
expect.anything(),
"wss://example.livekit.valid",
"#testRoom",
);
expect(customLivekitUrlSetting.getValue()).toBe(
"wss://example.livekit.valid",
);
});
it("will show error on invalid url", async () => {
const user = userEvent.setup();
render(
<TooltipProvider>
<DeveloperSettingsTab
client={client}
roomId="#testRoom"
env={{} as unknown as ImportMetaEnv}
/>
</TooltipProvider>,
);
const input = screen.getByLabelText("Custom Livekit-url");
await user.clear(input);
await user.type(input, "wss://example.livekit.valid");
const saveButton = screen.getByRole("button", { name: "Save" });
(getSFUConfigWithOpenID as Mock).mockImplementation(() => {
throw new Error("Invalid URL");
});
await user.click(saveButton);
expect(
screen.getByText("invalid URL (did not update)"),
).toBeInTheDocument();
expect(customLivekitUrlSetting.getValue()).toBe(null);
});
});
});

View File

@@ -22,6 +22,7 @@ import {
import { logger } from "matrix-js-sdk/lib/logger";
import {
EditInPlace,
ErrorMessage,
Root as Form,
Heading,
HelpMessage,
@@ -29,6 +30,7 @@ import {
Label,
RadioControl,
} from "@vector-im/compound-web";
import { type Room as LivekitRoom } from "livekit-client";
import { FieldRow, InputField } from "../input/Input";
import {
@@ -42,19 +44,26 @@ import {
customLivekitUrl as customLivekitUrlSetting,
MatrixRTCMode,
} from "./settings";
import type { Room as LivekitRoom } from "livekit-client";
import styles from "./DeveloperSettingsTab.module.css";
import { useUrlParams } from "../UrlParams";
import { getSFUConfigWithOpenID } from "../livekit/openIDSFU";
interface Props {
client: MatrixClient;
livekitRooms?: { room: LivekitRoom; url: string; isLocal?: boolean }[];
roomId?: string;
livekitRooms?: {
room: LivekitRoom;
url: string;
isLocal?: boolean;
livekitAlias?: string;
}[];
env: ImportMetaEnv;
}
export const DeveloperSettingsTab: FC<Props> = ({
client,
livekitRooms,
roomId,
env,
}) => {
const { t } = useTranslation();
@@ -92,6 +101,8 @@ export const DeveloperSettingsTab: FC<Props> = ({
alwaysShowIphoneEarpieceSetting,
);
const [customLivekitUrlUpdateError, setCustomLivekitUrlUpdateError] =
useState<string | null>(null);
const [customLivekitUrl, setCustomLivekitUrl] = useSetting(
customLivekitUrlSetting,
);
@@ -229,14 +240,36 @@ export const DeveloperSettingsTab: FC<Props> = ({
savingLabel={t("developer_mode.custom_livekit_url.saving")}
cancelButtonLabel={t("developer_mode.custom_livekit_url.reset")}
onSave={useCallback(
(e: React.FormEvent<HTMLFormElement>) => {
setCustomLivekitUrl(
customLivekitUrlTextBuffer === ""
? null
: customLivekitUrlTextBuffer,
);
async (e: React.FormEvent<HTMLFormElement>): Promise<void> => {
if (
roomId === undefined ||
customLivekitUrlTextBuffer === "" ||
customLivekitUrlTextBuffer === null
) {
setCustomLivekitUrl(null);
return;
}
try {
const userId = client.getUserId();
const deviceId = client.getDeviceId();
if (userId === null || deviceId === null) {
throw new Error("Invalid user or device ID");
}
await getSFUConfigWithOpenID(
client,
{ userId, deviceId, memberId: "" },
customLivekitUrlTextBuffer,
roomId,
);
setCustomLivekitUrlUpdateError(null);
setCustomLivekitUrl(customLivekitUrlTextBuffer);
} catch {
setCustomLivekitUrlUpdateError("invalid URL (did not update)");
}
},
[setCustomLivekitUrl, customLivekitUrlTextBuffer],
[customLivekitUrlTextBuffer, setCustomLivekitUrl, client, roomId],
)}
value={customLivekitUrlTextBuffer ?? ""}
onChange={useCallback(
@@ -251,7 +284,12 @@ export const DeveloperSettingsTab: FC<Props> = ({
},
[setCustomLivekitUrl],
)}
/>
serverInvalid={customLivekitUrlUpdateError !== null}
>
{customLivekitUrlUpdateError !== null && (
<ErrorMessage>{customLivekitUrlUpdateError}</ErrorMessage>
)}
</EditInPlace>
<Heading as="h3" type="body" weight="semibold" size="lg">
{t("developer_mode.matrixRTCMode.title")}
</Heading>
@@ -275,8 +313,8 @@ export const DeveloperSettingsTab: FC<Props> = ({
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Compatibil}
value={MatrixRTCMode.Compatibil}
checked={matrixRTCMode === MatrixRTCMode.Compatibility}
value={MatrixRTCMode.Compatibility}
onChange={onMatrixRTCModeChange}
/>
}
@@ -304,12 +342,14 @@ export const DeveloperSettingsTab: FC<Props> = ({
</InlineField>
</Form>
{livekitRooms?.map((livekitRoom) => (
<>
<h3>
<div className={styles.livekit_room_box}>
<h4>
{t("developer_mode.livekit_sfu", {
url: livekitRoom.url || "unknown",
})}
</h3>
</h4>
<p>LivekitAlias: {livekitRoom.livekitAlias}</p>
<p>connectionState (wont hot reload): {livekitRoom.room.state}</p>
{livekitRoom.isLocal && <p>ws-url: {localSfuUrl?.href}</p>}
<p>
{t("developer_mode.livekit_server_info")}(
@@ -321,7 +361,19 @@ export const DeveloperSettingsTab: FC<Props> = ({
: "undefined"}
{livekitRoom.room.metadata}
</pre>
</>
<p>Local Participant</p>
<pre className={styles.pre}>
{livekitRoom.room.localParticipant.identity}
</pre>
<p>Remote Participants</p>
<ul>
{Array.from(livekitRoom.room.remoteParticipants.keys()).map(
(id) => (
<li key={id}>{id}</li>
),
)}
</ul>
</div>
))}
<p>{t("developer_mode.environment_variables")}</p>
<pre>{JSON.stringify(env, null, 2)}</pre>

View File

@@ -213,6 +213,7 @@ export const SettingsModal: FC<Props> = ({
env={import.meta.env}
client={client}
livekitRooms={livekitRooms}
roomId={roomId}
/>
),
};

View File

@@ -234,12 +234,12 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
class="_container_1ug7n_10"
>
<input
aria-describedby="radix-_r_9_ radix-_r_b_ radix-_r_d_"
checked=""
class="_input_1e0uz_18"
class="_input_1ug7n_18"
id="radix-_r_8_"
name="_r_0_"
title=""
@@ -247,7 +247,7 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
value="legacy"
/>
<div
class="_ui_1e0uz_19"
class="_ui_1ug7n_19"
/>
</div>
</div>
@@ -275,19 +275,19 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
class="_container_1ug7n_10"
>
<input
aria-describedby="radix-_r_9_ radix-_r_b_ radix-_r_d_"
class="_input_1e0uz_18"
class="_input_1ug7n_18"
id="radix-_r_a_"
name="_r_0_"
title=""
type="radio"
value="compatibil"
value="compatibility"
/>
<div
class="_ui_1e0uz_19"
class="_ui_1ug7n_19"
/>
</div>
</div>
@@ -315,11 +315,11 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
class="_container_1ug7n_10"
>
<input
aria-describedby="radix-_r_9_ radix-_r_b_ radix-_r_d_"
class="_input_1e0uz_18"
class="_input_1ug7n_18"
id="radix-_r_c_"
name="_r_0_"
title=""
@@ -327,7 +327,7 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
value="matrix_2_0"
/>
<div
class="_ui_1e0uz_19"
class="_ui_1ug7n_19"
/>
</div>
</div>
@@ -349,46 +349,92 @@ exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
</div>
</div>
</form>
<h3>
LiveKit SFU: wss://local-sfu.example.org
</h3>
<p>
ws-url:
wss://local-sfu.example.org/
</p>
<p>
LiveKit Server Info
(
local
)
</p>
<pre
class="pre"
<div
class="livekit_room_box"
>
{
<h4>
LiveKit SFU: wss://local-sfu.example.org
</h4>
<p>
LivekitAlias:
TestAlias
</p>
<p>
connectionState (wont hot reload):
</p>
<p>
ws-url:
wss://local-sfu.example.org/
</p>
<p>
LiveKit Server Info
(
local
)
</p>
<pre
class="pre"
>
{
"region": "local",
"version": "1.2.3"
}
local-metadata
</pre>
<h3>
LiveKit SFU: wss://remote-sfu.example.org
</h3>
<p>
LiveKit Server Info
(
remote
)
</p>
<pre
class="pre"
local-metadata
</pre>
<p>
Local Participant
</p>
<pre
class="pre"
>
localParticipantIdentity
</pre>
<p>
Remote Participants
</p>
<ul />
</div>
<div
class="livekit_room_box"
>
{
<h4>
LiveKit SFU: wss://remote-sfu.example.org
</h4>
<p>
LivekitAlias:
TestAlias2
</p>
<p>
connectionState (wont hot reload):
</p>
<p>
LiveKit Server Info
(
remote
)
</p>
<pre
class="pre"
>
{
"region": "remote",
"version": "4.5.6"
}
remote-metadata
</pre>
remote-metadata
</pre>
<p>
Local Participant
</p>
<pre
class="pre"
>
localParticipantIdentity
</pre>
<p>
Remote Participants
</p>
<ul />
</div>
<p>
Environment variables
</p>

View File

@@ -0,0 +1,34 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, it } from "vitest";
import { init as initRageshake } from "./rageshake";
it("Logger should not crash if JSON.stringify fails", async () => {
// JSON.stringify can throw. We want to make sure that the logger can handle this gracefully.
await initRageshake();
const bigIntObj = { n: 1n };
const notStringifiable = {
bigIntObj,
};
// @ts-expect-error - we want to create an object that cannot be stringified
notStringifiable.foo = notStringifiable; // circular reference
// ensure this cannot be stringified
expect(() => JSON.stringify(notStringifiable)).toThrow();
expect(() =>
global.mx_rage_logger.log(
1,
"test",
"This is a test message",
notStringifiable,
),
).not.toThrow();
});

View File

@@ -75,7 +75,14 @@ class ConsoleLogger extends EventEmitter {
} else if (arg instanceof Error) {
return arg.message + (arg.stack ? `\n${arg.stack}` : "");
} else if (typeof arg === "object") {
return JSON.stringify(arg, getCircularReplacer());
try {
return JSON.stringify(arg, getCircularReplacer());
} catch {
// Stringify can fail if the object has circular references or if
// there is a bigInt.
// Did happen even with our `getCircularReplacer`. In this case, just log
return "<$ failed to serialize object $>";
}
} else {
return arg;
}

View File

@@ -34,15 +34,20 @@ export class Setting<T> {
this._value$ = new BehaviorSubject(initialValue);
this.value$ = this._value$;
this._lastUpdateReason$ = new BehaviorSubject<string | null>(null);
this.lastUpdateReason$ = this._lastUpdateReason$;
}
private readonly key: string;
private readonly _value$: BehaviorSubject<T>;
private readonly _lastUpdateReason$: BehaviorSubject<string | null>;
public readonly value$: Behavior<T>;
public readonly lastUpdateReason$: Behavior<string | null>;
public readonly setValue = (value: T): void => {
public readonly setValue = (value: T, reason?: string): void => {
this._value$.next(value);
this._lastUpdateReason$.next(reason ?? null);
localStorage.setItem(this.key, JSON.stringify(value));
};
public readonly getValue = (): T => {
@@ -126,7 +131,13 @@ export const alwaysShowIphoneEarpiece = new Setting<boolean>(
export enum MatrixRTCMode {
Legacy = "legacy",
Compatibil = "compatibil",
Compatibility = "compatibility",
/** This implies using
* - sticky events
* - hashed RTC backend identity
* - the new endpoint for the jwt token on the local membership (remote memberships will always try the new jwt endpoint first -> then the legacy one)
* - use the hashed identity for the local membership
*/
Matrix_2_0 = "matrix_2_0",
}

View File

@@ -0,0 +1,563 @@
/*
Copyright 2026 Element Corp.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { it, vi, expect, beforeEach, afterEach, describe } from "vitest";
import { firstValueFrom, of, Subject, take, toArray } from "rxjs";
import { type RTCCallIntent } from "matrix-js-sdk/lib/matrixrtc";
import { AndroidControlledAudioOutput } from "./AndroidControlledAudioOutput.ts";
import type { Controls, OutputDevice } from "../controls";
import { ObservableScope } from "./ObservableScope";
import { withTestScheduler } from "../utils/test";
// All the following device types are real device types that have been observed in the wild on Android devices,
// gathered from logs.
// There are no BT Speakers because they are currently filtered out by EXA (native layer)
// A device type describing the speaker system (i.e. a mono speaker or stereo speakers) built in a device.
const SPEAKER_DEVICE: OutputDevice = {
id: "3",
name: "Built-in speaker",
isEarpiece: false,
isSpeaker: true,
isExternalHeadset: false,
};
// A device type describing the attached earphone speaker.
const EARPIECE_DEVICE: OutputDevice = {
id: "2",
name: "Built-in earpiece",
isEarpiece: true,
isSpeaker: false,
isExternalHeadset: false,
};
// A device type describing a Bluetooth device typically used for telephony
const BT_HEADSET_DEVICE: OutputDevice = {
id: "2226",
name: "Bluetooth - OpenMove by Shokz",
isEarpiece: false,
isSpeaker: false,
isExternalHeadset: true,
};
// A device type describing a USB audio headset.
const USB_HEADSET_DEVICE: OutputDevice = {
id: "29440",
name: "USB headset - USB-Audio - AB13X USB Audio",
isEarpiece: false,
isSpeaker: false,
isExternalHeadset: false,
};
// A device type describing a headset, which is the combination of a headphones and microphone
const WIRED_HEADSET_DEVICE: OutputDevice = {
id: "54509",
name: "Wired headset - 23117RA68G",
isEarpiece: false,
isSpeaker: false,
isExternalHeadset: false,
};
// A device type describing a pair of wired headphones
const WIRED_HEADPHONE_DEVICE: OutputDevice = {
id: "679",
name: "Wired headphones - TB02",
isEarpiece: false,
isSpeaker: false,
isExternalHeadset: false,
};
/**
* The base device list that is always present on Android devices.
* This list is ordered by the OS, the speaker is listed before the earpiece.
*/
const BASE_DEVICE_LIST = [SPEAKER_DEVICE, EARPIECE_DEVICE];
const BT_HEADSET_BASE_DEVICE_LIST = [BT_HEADSET_DEVICE, ...BASE_DEVICE_LIST];
const WIRED_HEADSET_BASE_DEVICE_LIST = [
WIRED_HEADSET_DEVICE,
...BASE_DEVICE_LIST,
];
/**
* A full device list containing all the observed device types in the wild on Android devices.
* Ordered as they would be ordered by the OS.
*/
const FULL_DEVICE_LIST = [
BT_HEADSET_DEVICE,
USB_HEADSET_DEVICE,
WIRED_HEADSET_DEVICE,
WIRED_HEADPHONE_DEVICE,
...BASE_DEVICE_LIST,
];
let testScope: ObservableScope;
let mockControls: Controls;
beforeEach(() => {
testScope = new ObservableScope();
mockControls = {
onAudioDeviceSelect: vi.fn(),
onOutputDeviceSelect: vi.fn(),
} as unknown as Controls;
});
afterEach(() => {
testScope.end();
});
describe("Default selection", () => {
it("Default to speaker for video calls", async () => {
const controlledAudioOutput = new AndroidControlledAudioOutput(
of(BASE_DEVICE_LIST),
testScope,
"video",
mockControls,
);
const emissions = await firstValueFrom(
controlledAudioOutput.selected$.pipe(take(1), toArray()),
);
expect(emissions).toEqual([
{ id: SPEAKER_DEVICE.id, virtualEarpiece: false },
]);
[
mockControls.onAudioDeviceSelect,
mockControls.onOutputDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(SPEAKER_DEVICE.id);
});
});
it("Default to earpiece for audio calls for base config", async () => {
const controlledAudioOutput = new AndroidControlledAudioOutput(
of(BASE_DEVICE_LIST),
testScope,
"audio",
mockControls,
);
const emissions = await firstValueFrom(
controlledAudioOutput.selected$.pipe(take(1), toArray()),
);
expect(emissions).toEqual([
{ id: EARPIECE_DEVICE.id, virtualEarpiece: false },
]);
[
mockControls.onAudioDeviceSelect,
mockControls.onOutputDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(EARPIECE_DEVICE.id);
});
});
["audio", "video"].forEach((callIntent) => {
it(`Default to BT headset for ${callIntent} calls if present`, async () => {
const controlledAudioOutput = new AndroidControlledAudioOutput(
of(BT_HEADSET_BASE_DEVICE_LIST),
testScope,
callIntent,
mockControls,
);
const emissions = await firstValueFrom(
controlledAudioOutput.selected$.pipe(take(1), toArray()),
);
expect(emissions).toEqual([
{ id: BT_HEADSET_DEVICE.id, virtualEarpiece: false },
]);
[
mockControls.onAudioDeviceSelect,
mockControls.onOutputDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(BT_HEADSET_DEVICE.id);
});
});
});
["audio", "video"].forEach((callIntent) => {
it(`Default to wired headset for ${callIntent} calls if present`, async () => {
const controlledAudioOutput = new AndroidControlledAudioOutput(
of(WIRED_HEADSET_BASE_DEVICE_LIST),
testScope,
callIntent,
mockControls,
);
const emissions = await firstValueFrom(
controlledAudioOutput.selected$.pipe(take(1), toArray()),
);
expect(emissions).toEqual([
{ id: WIRED_HEADSET_DEVICE.id, virtualEarpiece: false },
]);
expect(mockControls.onAudioDeviceSelect).toHaveBeenCalledExactlyOnceWith(
WIRED_HEADSET_DEVICE.id,
);
expect(mockControls.onOutputDeviceSelect).toHaveBeenCalledExactlyOnceWith(
WIRED_HEADSET_DEVICE.id,
);
});
});
});
describe("Test mappings", () => {
it("Should map output device to correct AudioDeviceLabel", async () => {
const controlledAudioOutput = new AndroidControlledAudioOutput(
of(FULL_DEVICE_LIST),
testScope,
undefined,
mockControls,
);
const availableDevices = await firstValueFrom(
controlledAudioOutput.available$.pipe(take(1)),
);
expect(availableDevices).toEqual(
new Map([
[BT_HEADSET_DEVICE.id, { type: "name", name: BT_HEADSET_DEVICE.name }],
[
USB_HEADSET_DEVICE.id,
{ type: "name", name: USB_HEADSET_DEVICE.name },
],
[
WIRED_HEADSET_DEVICE.id,
{ type: "name", name: WIRED_HEADSET_DEVICE.name },
],
[
WIRED_HEADPHONE_DEVICE.id,
{ type: "name", name: WIRED_HEADPHONE_DEVICE.name },
],
[SPEAKER_DEVICE.id, { type: "speaker" }],
[EARPIECE_DEVICE.id, { type: "earpiece" }],
]),
);
});
});
describe("Test select a device", () => {
it(`Switch to correct device `, () => {
withTestScheduler(({ cold, schedule, expectObservable, flush }) => {
const controlledAudioOutput = new AndroidControlledAudioOutput(
cold("a", { a: FULL_DEVICE_LIST }),
testScope,
undefined,
mockControls,
);
schedule("-abc", {
a: () => controlledAudioOutput.select(EARPIECE_DEVICE.id),
b: () => controlledAudioOutput.select(USB_HEADSET_DEVICE.id),
c: () => controlledAudioOutput.select(SPEAKER_DEVICE.id),
});
expectObservable(controlledAudioOutput.selected$).toBe("abcd", {
// virtualEarpiece is always false on android.
// Initially the BT_HEADSET is selected.
a: { id: BT_HEADSET_DEVICE.id, virtualEarpiece: false },
b: { id: EARPIECE_DEVICE.id, virtualEarpiece: false },
c: { id: USB_HEADSET_DEVICE.id, virtualEarpiece: false },
d: { id: SPEAKER_DEVICE.id, virtualEarpiece: false },
});
flush();
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(4);
expect(mockFn).toHaveBeenNthCalledWith(1, BT_HEADSET_DEVICE.id);
expect(mockFn).toHaveBeenNthCalledWith(2, EARPIECE_DEVICE.id);
expect(mockFn).toHaveBeenNthCalledWith(3, USB_HEADSET_DEVICE.id);
expect(mockFn).toHaveBeenNthCalledWith(4, SPEAKER_DEVICE.id);
});
});
});
it(`manually switch then a bt headset is added`, () => {
withTestScheduler(({ cold, schedule, expectObservable, flush }) => {
const controlledAudioOutput = new AndroidControlledAudioOutput(
cold("a--b", {
a: BASE_DEVICE_LIST,
b: BT_HEADSET_BASE_DEVICE_LIST,
}),
testScope,
"audio",
mockControls,
);
// Default was earpiece (audio call), let's switch to speaker
schedule("-a--", {
a: () => controlledAudioOutput.select(SPEAKER_DEVICE.id),
});
expectObservable(controlledAudioOutput.selected$).toBe("ab-c", {
// virtualEarpiece is always false on android.
// Initially the BT_HEADSET is selected.
a: { id: EARPIECE_DEVICE.id, virtualEarpiece: false },
b: { id: SPEAKER_DEVICE.id, virtualEarpiece: false },
c: { id: BT_HEADSET_DEVICE.id, virtualEarpiece: false },
});
flush();
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(3);
expect(mockFn).toHaveBeenNthCalledWith(1, EARPIECE_DEVICE.id);
expect(mockFn).toHaveBeenNthCalledWith(2, SPEAKER_DEVICE.id);
expect(mockFn).toHaveBeenNthCalledWith(3, BT_HEADSET_DEVICE.id);
});
});
});
it(`Go back to the previously selected after the auto-switch device goes away`, () => {
withTestScheduler(({ cold, schedule, expectObservable, flush }) => {
const controlledAudioOutput = new AndroidControlledAudioOutput(
cold("a--b-c", {
a: BASE_DEVICE_LIST,
b: BT_HEADSET_BASE_DEVICE_LIST,
c: BASE_DEVICE_LIST,
}),
testScope,
"audio",
mockControls,
);
// Default was earpiece (audio call), let's switch to speaker
schedule("-a---", {
a: () => controlledAudioOutput.select(SPEAKER_DEVICE.id),
});
expectObservable(controlledAudioOutput.selected$).toBe("ab-c-d", {
// virtualEarpiece is always false on android.
// Initially the BT_HEADSET is selected.
a: { id: EARPIECE_DEVICE.id, virtualEarpiece: false },
b: { id: SPEAKER_DEVICE.id, virtualEarpiece: false },
c: { id: BT_HEADSET_DEVICE.id, virtualEarpiece: false },
d: { id: SPEAKER_DEVICE.id, virtualEarpiece: false },
});
flush();
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(4);
expect(mockFn).toHaveBeenNthCalledWith(1, EARPIECE_DEVICE.id);
expect(mockFn).toHaveBeenNthCalledWith(2, SPEAKER_DEVICE.id);
expect(mockFn).toHaveBeenNthCalledWith(3, BT_HEADSET_DEVICE.id);
expect(mockFn).toHaveBeenNthCalledWith(4, SPEAKER_DEVICE.id);
});
});
});
});
describe("Available device changes", () => {
let availableSource$: Subject<OutputDevice[]>;
const createAudioControlledOutput = (
intent: RTCCallIntent,
): AndroidControlledAudioOutput => {
return new AndroidControlledAudioOutput(
availableSource$,
testScope,
intent,
mockControls,
);
};
beforeEach(() => {
availableSource$ = new Subject<OutputDevice[]>();
});
it("When a BT headset is added, control should switch to use it", () => {
createAudioControlledOutput("video");
// Emit the base device list, the speaker should be selected
availableSource$.next(BASE_DEVICE_LIST);
// Initially speaker would be selected
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(SPEAKER_DEVICE.id);
});
// Emit a new device list with a BT device, the control should switch to it
availableSource$.next([BT_HEADSET_DEVICE, ...BASE_DEVICE_LIST]);
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(2);
expect(mockFn).toHaveBeenLastCalledWith(BT_HEADSET_DEVICE.id);
});
});
// Android does not set `isExternalHeadset` to true for wired headphones, so we can't test this case.'
it.skip("When a wired headset is added, control should switch to use it", async () => {
const controlledAudioOutput = createAudioControlledOutput("video");
// Emit the base device list, the speaker should be selected
availableSource$.next(BASE_DEVICE_LIST);
await firstValueFrom(controlledAudioOutput.selected$.pipe(take(1)));
// Initially speaker would be selected
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(SPEAKER_DEVICE.id);
});
// Emit a new device list with a wired headset, the control should switch to it
availableSource$.next([WIRED_HEADPHONE_DEVICE, ...BASE_DEVICE_LIST]);
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(2);
expect(mockFn).toHaveBeenLastCalledWith(WIRED_HEADPHONE_DEVICE.id);
});
});
it("When the active bt headset is removed on audio call, control should switch to earpiece", () => {
createAudioControlledOutput("audio");
// Emit the BT headset device list, the BT headset should be selected
availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
// Initially speaker would be selected
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(BT_HEADSET_DEVICE.id);
});
// Emit a new device list without the BT headset, the control should switch to the earpiece for
// audio calls
availableSource$.next(BASE_DEVICE_LIST);
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(2);
expect(mockFn).toHaveBeenLastCalledWith(EARPIECE_DEVICE.id);
});
});
it("When the active bt headset is removed on video call, control should switch to speaker", () => {
createAudioControlledOutput("video");
availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
// Initially bt headset would be selected
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(BT_HEADSET_DEVICE.id);
});
// Emit a new device list without the BT headset, the control should switch to speaker for video call
availableSource$.next(BASE_DEVICE_LIST);
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(2);
expect(mockFn).toHaveBeenLastCalledWith(SPEAKER_DEVICE.id);
});
});
it("Do not repeatidly set the same device", () => {
createAudioControlledOutput("video");
availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
// Initially bt headset would be selected
[
mockControls.onOutputDeviceSelect,
mockControls.onAudioDeviceSelect,
].forEach((mockFn) => {
expect(mockFn).toHaveBeenCalledTimes(1);
expect(mockFn).toHaveBeenCalledWith(BT_HEADSET_DEVICE.id);
});
});
});
describe("Scope management", () => {
it("Should stop emitting when scope ends", () => {
const aScope = new ObservableScope();
const controlledAudioOutput = new AndroidControlledAudioOutput(
of(BASE_DEVICE_LIST),
aScope,
undefined,
mockControls,
);
expect(mockControls.onAudioDeviceSelect).toHaveBeenCalledOnce();
aScope.end();
controlledAudioOutput.select(EARPIECE_DEVICE.id);
expect(mockControls.onAudioDeviceSelect).not.toHaveBeenCalledTimes(2);
expect(mockControls.onAudioDeviceSelect).toHaveBeenCalledOnce();
});
it("Should stop updating when scope ends", () => {
const aScope = new ObservableScope();
const availableSource$ = new Subject<OutputDevice[]>();
new AndroidControlledAudioOutput(
availableSource$,
aScope,
undefined,
mockControls,
);
availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
expect(mockControls.onAudioDeviceSelect).toHaveBeenCalledOnce();
expect(mockControls.onAudioDeviceSelect).toHaveBeenCalledWith(
BT_HEADSET_DEVICE.id,
);
aScope.end();
availableSource$.next(BASE_DEVICE_LIST);
expect(mockControls.onAudioDeviceSelect).not.toHaveBeenCalledTimes(2);
// Should have been called only once with the initial BT_HEADSET_DEVICE.id
expect(mockControls.onAudioDeviceSelect).toHaveBeenCalledOnce();
});
});

View File

@@ -0,0 +1,360 @@
/*
Copyright 2026 Element Corp.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import {
distinctUntilChanged,
map,
merge,
type Observable,
scan,
startWith,
Subject,
tap,
} from "rxjs";
import {
type AudioOutputDeviceLabel,
type MediaDevice,
type SelectedAudioOutputDevice,
} from "./MediaDevices.ts";
import type { ObservableScope } from "./ObservableScope.ts";
import type { RTCCallIntent } from "matrix-js-sdk/lib/matrixrtc";
import { type Controls, type OutputDevice } from "../controls.ts";
import { type Behavior } from "./Behavior.ts";
type ControllerState = {
/**
* The list of available output devices, ordered by preference order (most preferred first).
*/
devices: OutputDevice[];
/**
* Explicit user preference for the selected device.
*/
preferredDeviceId: string | undefined;
/**
* The effective selected device, always valid against available devices.
*/
selectedDeviceId: string | undefined;
};
/**
* The possible actions that can be performed on the controller,
* either by the user or by the system.
*/
type ControllerAction =
| { type: "selectDevice"; deviceId: string | undefined }
| { type: "deviceUpdated"; devices: OutputDevice[] };
/**
* The implementation of the audio output media device for Android when using the controlled audio output mode.
*
* In this mode, the hosting application (e.g. Element Mobile) is responsible for providing the list of available audio output devices.
* There are some android specific logic compared to others:
* - AndroidControlledAudioOutput is the only one responsible for selecting the best output device.
* - On android, we don't listen to the selected device from native code (control.setAudioDevice).
* - If a new device is added or removed, this controller will determine the new selected device based
* on the available devices (that is ordered by preference order) and the user's selection (if any).
*
* Given the differences in how the native code is handling the audio routing on Android compared to iOS,
* we have this separate implementation. It allows us to have proper testing and avoid side effects
* from platform specific logic breaking the other platform's implementation.
*/
export class AndroidControlledAudioOutput implements MediaDevice<
AudioOutputDeviceLabel,
SelectedAudioOutputDevice
> {
private logger = rootLogger.getChild(
"[MediaDevices AndroidControlledAudioOutput]",
);
/**
* STATE stream: the current state of the controller, including the list of available devices and the selected device.
*/
private readonly controllerState$: Behavior<ControllerState>;
/**
* @inheritdoc
*/
public readonly available$: Behavior<Map<string, AudioOutputDeviceLabel>>;
/**
* Effective selected device, always valid against available devices.
*
* On android, we don't listen to the selected device from native code (control.setAudioDevice).
* Instead, we determine the selected device ourselves based on the available devices and the user's selection (if any).
*/
public readonly selected$: Behavior<SelectedAudioOutputDevice | undefined>;
// COMMAND stream: user asks to select a device
private readonly selectDeviceCommand$ = new Subject<string | undefined>();
public select(id: string): void {
this.logger.info(`select device: ${id}`);
this.selectDeviceCommand$.next(id);
}
/**
* Creates an instance of AndroidControlledAudioOutput.
*
* @constructor
* @param controlledDevices$ - The list of available output devices coming from the hosting application, ordered by preference order (most preferred first).
* @param scope - The ObservableScope to create the Behaviors in.
* @param initialIntent - The initial call intent (e.g. "audio" or "video") that can be used to determine the default audio routing (e.g. default to earpiece for audio calls and speaker for video calls).
* @param controls - The controls provided by the hosting application to control the audio routing and notify of user actions.
*/
public constructor(
private readonly controlledDevices$: Observable<OutputDevice[]>,
private readonly scope: ObservableScope,
private initialIntent: RTCCallIntent | undefined = undefined,
controls: Controls,
) {
this.controllerState$ = this.startObservingState$();
this.selected$ = this.effectiveSelectionFromState$(this.controllerState$);
this.available$ = scope.behavior(
this.controllerState$.pipe(
map((state) => {
this.logger.info("available devices updated:", state.devices);
return new Map<string, AudioOutputDeviceLabel>(
state.devices.map((outputDevice) => {
return [outputDevice.id, mapDeviceToLabel(outputDevice)];
}),
);
}),
),
);
// Effect 1: notify host when effective selection changes
this.selected$
// It is a behavior so it has built-in distinct until change
.pipe(scope.bind())
.subscribe((device) => {
// Let the hosting application know which output device has been selected.
if (device !== undefined) {
this.logger.info("onAudioDeviceSelect called:", device);
controls.onAudioDeviceSelect?.(device.id);
// Also invoke the deprecated callback for backward compatibility
// TODO: it appears that on Android the hosting application is only using the deprecated callback (onOutputDeviceSelect)
// and not the new one (onAudioDeviceSelect), we should clean this up and only have one callback for audio device selection.
controls.onOutputDeviceSelect?.(device.id);
}
});
}
private startObservingState$(): Behavior<ControllerState> {
const initialState: ControllerState = {
devices: [],
preferredDeviceId: undefined,
selectedDeviceId: undefined,
};
// Merge the two possible inputs observable as a single
// stream of actions that will update the state of the controller.
const actions$: Observable<ControllerAction> = merge(
this.controlledDevices$.pipe(
map(
(devices) =>
({ type: "deviceUpdated", devices }) satisfies ControllerAction,
),
),
this.selectDeviceCommand$.pipe(
map(
(deviceId) =>
({ type: "selectDevice", deviceId }) satisfies ControllerAction,
),
),
);
const initialAction: ControllerAction = {
type: "deviceUpdated",
devices: [],
};
return this.scope.behavior(
actions$.pipe(
startWith(initialAction),
scan((state, action): ControllerState => {
switch (action.type) {
case "deviceUpdated": {
const chosenDevice = this.chooseEffectiveSelection({
previousDevices: state.devices,
availableDevices: action.devices,
currentSelectedId: state.selectedDeviceId,
preferredDeviceId: state.preferredDeviceId,
});
return {
...state,
devices: action.devices,
selectedDeviceId: chosenDevice,
};
}
case "selectDevice": {
const chosenDevice = this.chooseEffectiveSelection({
previousDevices: state.devices,
availableDevices: state.devices,
currentSelectedId: state.selectedDeviceId,
preferredDeviceId: action.deviceId,
});
return {
...state,
preferredDeviceId: action.deviceId,
selectedDeviceId: chosenDevice,
};
}
}
}, initialState),
),
);
}
private effectiveSelectionFromState$(
state$: Observable<ControllerState>,
): Behavior<SelectedAudioOutputDevice | undefined> {
return this.scope.behavior(
state$
.pipe(
map((state) => {
if (state.selectedDeviceId) {
return {
id: state.selectedDeviceId,
/** This is an iOS thing, always false for android*/
virtualEarpiece: false,
};
}
return undefined;
}),
distinctUntilChanged((a, b) => a?.id === b?.id),
)
.pipe(
tap((selected) => {
this.logger.debug(`selected device: ${selected?.id}`);
}),
),
);
}
private chooseEffectiveSelection(args: {
previousDevices: OutputDevice[];
availableDevices: OutputDevice[];
currentSelectedId: string | undefined;
preferredDeviceId: string | undefined;
}): string | undefined {
const {
previousDevices,
availableDevices,
currentSelectedId,
preferredDeviceId,
} = args;
this.logger.debug(`chooseEffectiveSelection with args:`, args);
// Take preferredDeviceId in priority or default to the last effective selection.
const activeSelectedDeviceId = preferredDeviceId || currentSelectedId;
const isAvailable = availableDevices.some(
(device) => device.id === activeSelectedDeviceId,
);
// If there is no current device, or it is not available anymore,
// choose the default device selection logic.
if (activeSelectedDeviceId === undefined || !isAvailable) {
this.logger.debug(
`No current device or it is not available, using default selection logic.`,
);
// use the default selection logic
return this.chooseDefaultDeviceId(availableDevices);
}
// Is there a new added device?
// If a device is added, we might want to switch to it if it's more preferred than the currently selected device.
const newDeviceWasAdded = availableDevices.some(
(device) => !previousDevices.some((d) => d.id === device.id),
);
if (newDeviceWasAdded) {
// TODO only want to check from the added device, not all devices.?
// check if the currently selected device is the most preferred one, if not switch to the most preferred one.
const mostPreferredDevice = availableDevices[0];
this.logger.debug(
`A new device was added, checking if we should switch to it.`,
mostPreferredDevice,
);
if (mostPreferredDevice.id !== activeSelectedDeviceId) {
// Given this is automatic switching, we want to be careful and only switch to a more private device
// (e.g. from speaker to a BT headset) but not switch from a more private device to a less private one
// (e.g. from a BT headset to the speaker), as that can be disruptive for the user if it happens unexpectedly.
if (mostPreferredDevice.isExternalHeadset == true) {
this.logger.info(
`The currently selected device ${mostPreferredDevice.id} is not the most preferred one, switching to the most preferred one ${activeSelectedDeviceId} instead.`,
);
// Let's switch as it is a more private device.
return mostPreferredDevice.id;
}
}
}
// no changes
return activeSelectedDeviceId;
}
/**
* The logic for the default is different based on the call type.
* For example for a voice call we want to default to the earpiece if it's available,
* but for a video call we want to default to the speaker.
* If the user is using a BT headset we want to default to that, as it's likely what they want to use for both video and voice calls.
*
* @param available the available audio output devices to choose from, keyed by their id, sorted by likelihood of it being used for communication.
*
*/
private chooseDefaultDeviceId(available: OutputDevice[]): string | undefined {
this.logger.debug(
`Android routing logic intent: ${this.initialIntent} finding best default...`,
);
if (this.initialIntent === "audio") {
const systemProposed = available[0];
// If no headset is connected, android will route to the speaker by default,
// but for a voice call we want to route to the earpiece instead,
// so override the system proposed routing in that case.
if (systemProposed?.isSpeaker == true) {
// search for the earpiece
const earpieceDevice = available.find(
(device) => device.isEarpiece == true,
);
if (earpieceDevice) {
this.logger.debug(
`Android routing: Switch to earpiece instead of speaker for voice call`,
);
return earpieceDevice.id;
} else {
this.logger.debug(
`Android routing: no earpiece found, cannot switch, use system proposed routing`,
);
return systemProposed.id;
}
} else {
this.logger.debug(
`Android routing: Use system proposed routing `,
systemProposed,
);
return systemProposed?.id;
}
} else {
// Use the system best proposed best routing.
return available[0]?.id;
}
}
}
// Utilities
function mapDeviceToLabel(device: OutputDevice): AudioOutputDeviceLabel {
const { name, isEarpiece, isSpeaker } = device;
if (isEarpiece) return { type: "earpiece" };
else if (isSpeaker) return { type: "speaker" };
else return { type: "name", name };
}

View File

@@ -0,0 +1,193 @@
/*
Copyright 2026 Element Corp.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, vi, it } from "vitest";
import * as ComponentsCore from "@livekit/components-core";
import { ObservableScope } from "./ObservableScope";
import { AudioOutput } from "./MediaDevices";
import { withTestScheduler } from "../utils/test";
const BT_SPEAKER = {
deviceId: "f9fc8f5f94578fe3abd89e086c1e78c08477aa564dd9e917950f0e7ebb37a6a2",
kind: "audiooutput",
label: "JBL (Bluetooth)",
groupId: "309a5c086cd8eb885a164046db6ec834c349be01d86448d02c1a5279456ff9e4",
} as unknown as MediaDeviceInfo;
const BUILT_IN_SPEAKER = {
deviceId: "acdbb8546ea6fa85ba2d861e9bcc0e71810d03bbaf6d1712c69e8d9c0c6c2e0a",
kind: "audiooutput",
label: "MacBook Speakers (Built-in)",
groupId: "08a5a3a486473aaa898eb81cda3113f3e21053fb8b84155f4e612fe3f8db5d17",
} as unknown as MediaDeviceInfo;
const BT_HEADSET = {
deviceId: "ff8e6edb4ebb512b2b421335bfd14994a5b4c7192b3e84a8696863d83cf46d12",
kind: "audiooutput",
label: "OpenMove (Bluetooth)",
groupId: "c2893c2438c44248368e0533300245c402764991506f42cd73818dc8c3ee9c88",
} as unknown as MediaDeviceInfo;
const AMAC_DEVICE_LIST = [BT_SPEAKER, BUILT_IN_SPEAKER];
const AMAC_DEVICE_LIST_WITH_DEFAULT = [
asDefault(BUILT_IN_SPEAKER),
...AMAC_DEVICE_LIST,
];
const AMAC_HS_DEVICE_LIST = [
asDefault(BT_HEADSET),
BT_SPEAKER,
BT_HEADSET,
BUILT_IN_SPEAKER,
];
const LAPTOP_SPEAKER = {
deviceId: "EcUxTMu8He2wz+3Y8m/u0fy6M92pUk=",
kind: "audiooutput",
label: "Raptor AVS Speaker",
groupId: "kSrdanhpEDLg3vN8z6Z9MJ1EdanB8zI+Q1dxA=",
} as unknown as MediaDeviceInfo;
const MONITOR_SPEAKER = {
deviceId: "gBryZdAdC8I/rrJpr9r6R+rZzKkoIK5cpU=",
kind: "audiooutput",
label: "Raptor AVS HDMI / DisplayPort 1 Output",
groupId: "kSrdanhpEDLg3vN8z6Z9MJ1EdanB8zI+Q1dxA=",
} as unknown as MediaDeviceInfo;
const DEVICE_LIST_B = [LAPTOP_SPEAKER, MONITOR_SPEAKER];
// On chrome, there is an additional synthetic device called "Default - <device name>",
// it represents what the OS default is now.
function asDefault(device: MediaDeviceInfo): MediaDeviceInfo {
return {
...device,
deviceId: "default",
label: `Default - ${device.label}`,
};
}
// When the authorization is not yet granted, every device is still listed
// but only with empty/blank labels and ids.
// This is a transition state.
function toBlankDevice(device: MediaDeviceInfo): MediaDeviceInfo {
return {
...device,
deviceId: "",
label: "",
groupId: "",
};
}
vi.mock("@livekit/components-core", () => ({
createMediaDeviceObserver: vi.fn(),
}));
describe("AudioOutput Tests", () => {
let testScope: ObservableScope;
beforeEach(() => {
testScope = new ObservableScope();
});
afterEach(() => {
testScope.end();
});
it("should select the default audio output device", () => {
// In a real life setup there would be first a blanked list
// then the real one.
withTestScheduler(({ behavior, cold, expectObservable }) => {
vi.mocked(ComponentsCore.createMediaDeviceObserver).mockReturnValue(
cold("ab", {
// In a real life setup there would be first a blanked list
// then the real one.
a: AMAC_DEVICE_LIST_WITH_DEFAULT.map(toBlankDevice),
b: AMAC_DEVICE_LIST_WITH_DEFAULT,
}),
);
const audioOutput = new AudioOutput(
behavior("a", { a: true }),
testScope,
);
expectObservable(audioOutput.selected$).toBe("ab", {
a: undefined,
b: { id: "default", virtualEarpiece: false },
});
});
});
it("Select the correct device when requested", () => {
// In a real life setup there would be first a blanked list
// then the real one.
withTestScheduler(({ behavior, cold, schedule, expectObservable }) => {
vi.mocked(ComponentsCore.createMediaDeviceObserver).mockReturnValue(
cold("ab", {
// In a real life setup there would be first a blanked list
// then the real one.
a: DEVICE_LIST_B.map(toBlankDevice),
b: DEVICE_LIST_B,
}),
);
const audioOutput = new AudioOutput(
behavior("a", { a: true }),
testScope,
);
schedule("--abc", {
a: () => audioOutput.select(MONITOR_SPEAKER.deviceId),
b: () => audioOutput.select(LAPTOP_SPEAKER.deviceId),
c: () => audioOutput.select(MONITOR_SPEAKER.deviceId),
});
expectObservable(audioOutput.selected$).toBe("abcde", {
a: undefined,
b: { id: LAPTOP_SPEAKER.deviceId, virtualEarpiece: false },
c: { id: MONITOR_SPEAKER.deviceId, virtualEarpiece: false },
d: { id: LAPTOP_SPEAKER.deviceId, virtualEarpiece: false },
e: { id: MONITOR_SPEAKER.deviceId, virtualEarpiece: false },
});
});
});
it("Test mappings", () => {
// In a real life setup there would be first a blanked list
// then the real one.
withTestScheduler(({ behavior, cold, schedule, expectObservable }) => {
vi.mocked(ComponentsCore.createMediaDeviceObserver).mockReturnValue(
cold("a", {
// In a real life setup there would be first a blanked list
// then the real one.
a: AMAC_HS_DEVICE_LIST,
}),
);
const audioOutput = new AudioOutput(
behavior("a", { a: true }),
testScope,
);
const expectedMappings = new Map([
[`default`, { type: "name", name: asDefault(BT_HEADSET).label }],
[BT_SPEAKER.deviceId, { type: "name", name: BT_SPEAKER.label }],
[BT_HEADSET.deviceId, { type: "name", name: BT_HEADSET.label }],
[
BUILT_IN_SPEAKER.deviceId,
{ type: "name", name: BUILT_IN_SPEAKER.label },
],
]);
expectObservable(audioOutput.available$).toBe("a", {
a: expectedMappings,
});
});
});
});

View File

@@ -5,10 +5,6 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type ICallNotifyContent,
type IRTCNotificationContent,
} from "matrix-js-sdk/lib/matrixrtc";
import { describe, it } from "vitest";
import {
EventType,
@@ -25,23 +21,23 @@ import {
localRtcMember,
} from "../../utils/test-fixtures";
import {
type CallNotificationWrapper,
createCallNotificationLifecycle$,
type Props as CallNotificationLifecycleProps,
} from "./CallNotificationLifecycle";
import { trackEpoch } from "../ObservableScope";
const mockLegacyRingEvent = {} as { event_id: string } & ICallNotifyContent;
function mockRingEvent(
eventId: string,
lifetimeMs: number | undefined,
sender = local.userId,
): { event_id: string } & IRTCNotificationContent {
): CallNotificationWrapper {
return {
event_id: eventId,
...(lifetimeMs === undefined ? {} : { lifetime: lifetimeMs }),
notification_type: "ring",
sender,
} as unknown as { event_id: string } & IRTCNotificationContent;
} as unknown as CallNotificationWrapper;
}
describe("waitForCallPickup$", () => {
@@ -54,7 +50,7 @@ describe("waitForCallPickup$", () => {
behavior("a", { a: [] }).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$notif1", 30), mockLegacyRingEvent],
a: mockRingEvent("$notif1", 30),
}),
receivedDecline$: hot(""),
options: {
@@ -86,7 +82,7 @@ describe("waitForCallPickup$", () => {
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("5ms a", {
a: [mockRingEvent("$notif2", 100), mockLegacyRingEvent],
a: mockRingEvent("$notif2", 100),
}),
receivedDecline$: hot(""),
options: {
@@ -115,7 +111,7 @@ describe("waitForCallPickup$", () => {
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("20ms a", {
a: [mockRingEvent("$notif2", 50), mockLegacyRingEvent],
a: mockRingEvent("$notif2", 50),
}),
receivedDecline$: hot(""),
options: {
@@ -142,7 +138,7 @@ describe("waitForCallPickup$", () => {
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$notif2", undefined), mockLegacyRingEvent],
a: mockRingEvent("$notif2", undefined),
}),
receivedDecline$: hot(""),
options: {
@@ -171,7 +167,7 @@ describe("waitForCallPickup$", () => {
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$notif5", 30), mockLegacyRingEvent],
a: mockRingEvent("$notif5", 30),
}),
receivedDecline$: hot(""),
options: {
@@ -210,7 +206,7 @@ describe("waitForCallPickup$", () => {
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$decl1", 50), mockLegacyRingEvent],
a: mockRingEvent("$decl1", 50),
}),
receivedDecline$: hot("40ms d", {
d: [
@@ -254,7 +250,7 @@ describe("waitForCallPickup$", () => {
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$decl", 20), mockLegacyRingEvent],
a: mockRingEvent("$decl", 20),
}),
receivedDecline$: hot("40ms d", {
d: [
@@ -305,7 +301,7 @@ describe("waitForCallPickup$", () => {
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$right", 50), mockLegacyRingEvent],
a: mockRingEvent("$right", 50),
}),
receivedDecline$: hot("20ms d", {
d: [

View File

@@ -7,9 +7,9 @@ Please see LICENSE in the repository root for full details.
import {
type CallMembership,
type IRTCNotificationContent,
type MatrixRTCSession,
MatrixRTCSessionEvent,
type MatrixRTCSessionEventHandlerMap,
} from "matrix-js-sdk/lib/matrixrtc";
import {
combineLatest,
@@ -38,6 +38,7 @@ import {
import { type Behavior } from "../Behavior";
import { type Epoch, mapEpoch, type ObservableScope } from "../ObservableScope";
export type AutoLeaveReason = "allOthersLeft" | "timeout" | "decline";
export type CallPickupState =
| "unknown"
@@ -46,9 +47,11 @@ export type CallPickupState =
| "decline"
| "success"
| null;
export type CallNotificationWrapper = Parameters<
MatrixRTCSessionEventHandlerMap[MatrixRTCSessionEvent.DidSendCallNotification]
>;
export type CallNotificationWrapper = {
event_id: string;
} & IRTCNotificationContent;
export function createSentCallNotification$(
scope: ObservableScope,
matrixRTCSession: MatrixRTCSession,
@@ -80,12 +83,12 @@ export interface Props {
options: { waitForCallPickup?: boolean; autoLeaveWhenOthersLeft?: boolean };
localUser: { deviceId: string; userId: string };
}
/**
* @returns two observables:
* `callPickupState$` The current call pickup state of the call.
* - "unknown": The client has not yet sent the notification event. We don't know if it will because it first needs to send its own membership.
* Then we can conclude if we were the first one to join or not.
* This may also be set if we are disconnected.
* - "ringing": The call is ringing on other devices in this room (This client should give audiovisual feedback that this is happening).
* - "timeout": No-one picked up in the defined time this call should be ringing on others devices.
* The call failed. If desired this can be used as a trigger to exit the call.
@@ -127,25 +130,19 @@ export function createCallNotificationLifecycle$({
) as Behavior<Epoch<boolean>>;
/**
* Whenever the RTC session tells us that it intends to ring the remote
* participant's devices, this emits an Observable tracking the current state of
* that ringing process.
* The state of the current ringing attempt, if the RTC session is indeed
* ringing the remote participant's devices. Otherwise `null`.
*/
// This is a behavior since we need to store the latest state for when we subscribe to this after `didSendCallNotification$`
// has already emitted but we still need the latest observable with a timeout timer that only gets created on after receiving `notificationEvent`.
// A behavior will emit the latest observable with the running timer to new subscribers.
// see also: callPickupState$ and in particular the line: `return this.ring$.pipe(mergeAll());` here we otherwise might get an EMPTY observable if
// `ring$` would not be a behavior.
const remoteRingState$: Behavior<"ringing" | "timeout" | "decline" | null> =
scope.behavior(
sentCallNotification$.pipe(
filter(
(newAndLegacyEvents) =>
(notificationEventArgs: CallNotificationWrapper | null) =>
// only care about new events (legacy do not have decline pattern)
newAndLegacyEvents?.[0].notification_type === "ring",
notificationEventArgs?.notification_type === "ring",
),
map((e) => e as CallNotificationWrapper),
switchMap(([notificationEvent]) => {
switchMap((notificationEvent) => {
const lifetimeMs = notificationEvent?.lifetime ?? 0;
return concat(
lifetimeMs === 0

View File

@@ -29,7 +29,6 @@ import {
Status,
type CallMembership,
type IRTCNotificationContent,
type ICallNotifyContent,
MatrixRTCSessionEvent,
type LivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
@@ -47,9 +46,11 @@ import {
} from "../../utils/test.ts";
import { E2eeType } from "../../e2ee/e2eeType.ts";
import {
alice,
aliceId,
aliceParticipant,
aliceRtcMember,
aliceUserId,
bobId,
bobRtcMember,
local,
@@ -62,6 +63,9 @@ import { getValue } from "../../utils/observable.ts";
import { type Behavior, constant } from "../Behavior.ts";
import { withCallViewModel as withCallViewModelInMode } from "./CallViewModelTestUtils.ts";
import { MatrixRTCMode } from "../../settings/settings.ts";
import { initializeWidget } from "../../widget.ts";
initializeWidget();
vi.mock("rxjs", async (importOriginal) => ({
...(await importOriginal()),
@@ -78,11 +82,14 @@ vi.mock("../e2ee/matrixKeyProvider");
const getUrlParams = vi.hoisted(() => vi.fn(() => ({})));
vi.mock("../UrlParams", () => ({ getUrlParams }));
vi.mock("../rtcSessionHelpers", async (importOriginal) => ({
...(await importOriginal()),
makeTransport: async (): Promise<LivekitTransport> =>
Promise.resolve(exampleTransport),
}));
vi.mock(
"../state/CallViewModel/localMember/localTransport",
async (importOriginal) => ({
...(await importOriginal()),
makeTransport: async (): Promise<LivekitTransport> =>
Promise.resolve(exampleTransport),
}),
);
const yesNo = {
y: true,
@@ -135,8 +142,8 @@ export interface SpotlightExpandedLayoutSummary {
export interface OneOnOneLayoutSummary {
type: "one-on-one";
local: string;
remote: string;
spotlight: string;
pip: string;
}
export interface PipLayoutSummary {
@@ -189,11 +196,11 @@ function summarizeLayout$(l$: Observable<Layout>): Observable<LayoutSummary> {
);
case "one-on-one":
return combineLatest(
[l.local.media$, l.remote.media$],
(local, remote) => ({
[l.spotlight.media$, l.pip.media$],
(spotlight, pip) => ({
type: l.type,
local: local.id,
remote: remote.id,
spotlight: spotlight.id,
pip: pip.id,
}),
);
case "pip":
@@ -226,13 +233,9 @@ function mockRingEvent(
} as unknown as { event_id: string } & IRTCNotificationContent;
}
// The app doesn't really care about the content of these legacy events, we just
// need a value to fill in for them when emitting notifications
const mockLegacyRingEvent = {} as { event_id: string } & ICallNotifyContent;
describe.each([
[MatrixRTCMode.Legacy],
[MatrixRTCMode.Compatibil],
[MatrixRTCMode.Compatibility],
[MatrixRTCMode.Matrix_2_0],
])("CallViewModel (%s mode)", (mode) => {
const withCallViewModel = withCallViewModelInMode(mode);
@@ -536,8 +539,8 @@ describe.each([
b: {
// In a larger window, expect the normal one-on-one layout
type: "one-on-one",
local: `${localId}:0`,
remote: `${aliceId}:0`,
pip: `${localId}:0`,
spotlight: `${aliceId}:0`,
},
c: {
// In a PiP-sized window, we of course expect a PiP layout
@@ -839,8 +842,8 @@ describe.each([
},
b: {
type: "one-on-one",
local: `${localId}:0`,
remote: `${aliceId}:0`,
pip: `${localId}:0`,
spotlight: `${aliceId}:0`,
},
c: {
type: "grid",
@@ -882,8 +885,8 @@ describe.each([
},
b: {
type: "one-on-one",
local: `${localId}:0`,
remote: `${aliceId}:0`,
pip: `${localId}:0`,
spotlight: `${aliceId}:0`,
},
c: {
type: "grid",
@@ -892,8 +895,8 @@ describe.each([
},
d: {
type: "one-on-one",
local: `${localId}:0`,
remote: `${daveId}:0`,
pip: `${localId}:0`,
spotlight: `${daveId}:0`,
},
},
);
@@ -1086,85 +1089,81 @@ describe.each([
});
});
describe("waitForCallPickup$", () => {
it.skip("regression test: does stop ringing in case livekitConnectionState$ emits after didSendCallNotification$ has already emitted", () => {
withTestScheduler(({ schedule, expectObservable, behavior }) => {
withCallViewModel(
{
livekitConnectionState$: behavior("d 9ms c", {
d: ConnectionState.Disconnected,
c: ConnectionState.Connected,
}),
},
(vm, rtcSession) => {
// Fire a call notification IMMEDIATELY (its important for this test, that this happens before the livekitConnectionState$ emits)
schedule("n", {
n: () => {
rtcSession.emit(
MatrixRTCSessionEvent.DidSendCallNotification,
mockRingEvent("$notif1", 30),
mockLegacyRingEvent,
);
},
});
test("recipient has placeholder tile while ringing or timed out", () => {
withTestScheduler(({ schedule, expectObservable }) => {
withCallViewModel(
{
roomMembers: [alice, local], // Simulate a DM
},
(vm, rtcSession) => {
// Fire a ringing notification
schedule("n", {
n: () => {
rtcSession.emit(
MatrixRTCSessionEvent.DidSendCallNotification,
mockRingEvent("$notif1", 30),
);
},
});
expectObservable(vm.callPickupState$).toBe("a 9ms b 19ms c", {
a: "unknown",
b: "ringing",
c: "timeout",
});
},
{
waitForCallPickup: true,
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
},
);
});
// Should ring for 30ms and then time out
expectObservable(vm.ringing$).toBe("(ny) 26ms n", yesNo);
// Layout should show placeholder media for the participant we're
// ringing the entire time (even once timed out)
expectObservable(summarizeLayout$(vm.layout$)).toBe("a", {
a: {
type: "one-on-one",
spotlight: `${localId}:0`,
pip: `ringing:${aliceUserId}`,
},
});
},
{ waitForCallPickup: true },
);
});
});
it.skip("ringing -> unknown if we get disconnected", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => {
const connectionState$ = new BehaviorSubject(ConnectionState.Connected);
// Someone joins at 20ms (both LiveKit participant and MatrixRTC member)
withCallViewModel(
{
remoteParticipants$: behavior("a 19ms b", {
a: [],
b: [aliceParticipant],
}),
rtcMembers$: behavior("a 19ms b", {
a: [localRtcMember],
b: [localRtcMember, aliceRtcMember],
}),
livekitConnectionState$: connectionState$,
},
(vm, rtcSession) => {
// Notify at 5ms so we enter ringing, then get disconnected 5ms later
schedule(" 5ms r 5ms d", {
r: () => {
rtcSession.emit(
MatrixRTCSessionEvent.DidSendCallNotification,
mockRingEvent("$notif2", 100),
mockLegacyRingEvent,
);
},
d: () => {
connectionState$.next(ConnectionState.Disconnected);
},
});
test("recipient's placeholder tile is replaced by their real tile once they answer", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => {
withCallViewModel(
{
// Alice answers after 20ms
rtcMembers$: behavior("a 20ms b", {
a: [localRtcMember],
b: [localRtcMember, aliceRtcMember],
}),
roomMembers: [alice, local], // Simulate a DM
},
(vm, rtcSession) => {
// Fire a ringing notification
schedule("n", {
n: () => {
rtcSession.emit(
MatrixRTCSessionEvent.DidSendCallNotification,
mockRingEvent("$notif1", 30),
);
},
});
expectObservable(vm.callPickupState$).toBe("a 4ms b 5ms c", {
a: "unknown",
b: "ringing",
c: "unknown",
});
},
{
waitForCallPickup: true,
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
},
);
});
// Should ring until Alice joins
expectObservable(vm.ringing$).toBe("(ny) 17ms n", yesNo);
// Layout should show placeholder media for the participant we're
// ringing the entire time
expectObservable(summarizeLayout$(vm.layout$)).toBe("a 20ms b", {
a: {
type: "one-on-one",
spotlight: `${localId}:0`,
pip: `ringing:${aliceUserId}`,
},
b: {
type: "one-on-one",
spotlight: `${aliceId}:0`,
pip: `${localId}:0`,
},
});
},
{ waitForCallPickup: true },
);
});
});
@@ -1255,11 +1254,6 @@ describe.each([
y: () => {
rtcSession.membershipStatus = Status.Connected;
},
n: () => {
// NOTE: This was removed in https://github.com/matrix-org/matrix-js-sdk/pull/5103 accidentally.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
rtcSession.membershipStatus = "Reconnecting" as any;
},
});
schedule(probablyLeftMarbles, {
y: () => {

View File

@@ -41,21 +41,19 @@ import {
} from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import {
type LivekitTransport,
MembershipManagerEvent,
type LivekitTransportConfig,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { type IWidgetApiRequest } from "matrix-widget-api";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { v4 as uuidv4 } from "uuid";
import { type IMembershipManager } from "matrix-js-sdk/lib/matrixrtc/IMembershipManager";
import {
LocalUserMediaViewModel,
type MediaViewModel,
type RemoteUserMediaViewModel,
ScreenShareViewModel,
type UserMediaViewModel,
} from "../MediaViewModel";
import {
accumulate,
createToggle$,
filterBehavior,
generateItem,
generateItems,
pauseWhen,
} from "../../utils/observable";
@@ -65,7 +63,7 @@ import {
playReactionsSound,
showReactions,
} from "../../settings/settings";
import { isFirefox } from "../../Platform";
import { isFirefox, platform } from "../../Platform";
import { setPipEnabled$ } from "../../controls";
import { TileStore } from "../TileStore";
import { gridLikeLayout } from "../GridLikeLayout";
@@ -87,8 +85,6 @@ import { type MuteStates } from "../MuteStates";
import { getUrlParams } from "../../UrlParams";
import { type ProcessorState } from "../../livekit/TrackProcessorContext";
import { ElementWidgetActions, widget } from "../../widget";
import { UserMedia } from "../UserMedia.ts";
import { ScreenShare } from "../ScreenShare.ts";
import {
type GridLayoutMedia,
type Layout,
@@ -98,26 +94,32 @@ import {
type SpotlightLandscapeLayoutMedia,
type SpotlightPortraitLayoutMedia,
} from "../layout-types.ts";
import { ElementCallError } from "../../utils/errors.ts";
import { type ObservableScope } from "../ObservableScope.ts";
import { ElementCallError, UnknownCallError } from "../../utils/errors.ts";
import { type Epoch, type ObservableScope } from "../ObservableScope.ts";
import { createHomeserverConnected$ } from "./localMember/HomeserverConnected.ts";
import {
createLocalMembership$,
enterRTCSession,
TransportState,
} from "./localMember/LocalMember.ts";
import { createLocalTransport$ } from "./localMember/LocalTransport.ts";
import {
createLocalTransport$,
JwtEndpointVersion,
} from "./localMember/LocalTransport.ts";
import {
createMemberships$,
membershipsAndTransports$,
} from "../SessionBehaviors.ts";
import { ECConnectionFactory } from "./remoteMembers/ConnectionFactory.ts";
import { createConnectionManager$ } from "./remoteMembers/ConnectionManager.ts";
import {
type ConnectionManagerData,
createConnectionManager$,
} from "./remoteMembers/ConnectionManager.ts";
import {
createMatrixLivekitMembers$,
type TaggedParticipant,
type LocalMatrixLivekitMember,
type RemoteMatrixLivekitMember,
type MatrixLivekitMember,
} from "./remoteMembers/MatrixLivekitMembers.ts";
import {
type AutoLeaveReason,
@@ -126,13 +128,25 @@ import {
createSentCallNotification$,
} from "./CallNotificationLifecycle.ts";
import {
createDMMember$,
createMatrixMemberMetadata$,
createRoomMembers$,
} from "./remoteMembers/MatrixMemberMetadata.ts";
import { Publisher } from "./localMember/Publisher.ts";
import { type Connection } from "./remoteMembers/Connection.ts";
import { createLayoutModeSwitch } from "./LayoutSwitch.ts";
import {
createWrappedUserMedia,
type WrappedUserMediaViewModel,
} from "../media/WrappedUserMediaViewModel.ts";
import { type ScreenShareViewModel } from "../media/ScreenShareViewModel.ts";
import { type UserMediaViewModel } from "../media/UserMediaViewModel.ts";
import { type MediaViewModel } from "../media/MediaViewModel.ts";
import { type LocalUserMediaViewModel } from "../media/LocalUserMediaViewModel.ts";
import { type RemoteUserMediaViewModel } from "../media/RemoteUserMediaViewModel.ts";
import {
createRingingMedia,
type RingingMediaViewModel,
} from "../media/RingingMediaViewModel.ts";
const logger = rootLogger.getChild("[CallViewModel]");
//TODO
@@ -182,7 +196,6 @@ interface LayoutScanState {
tiles: TileStore;
}
type MediaItem = UserMedia | ScreenShare;
export type LivekitRoomItem = {
livekitRoom: LivekitRoom;
participants: string[];
@@ -201,21 +214,28 @@ export type LivekitRoomItem = {
export interface CallViewModel {
// lifecycle
autoLeave$: Observable<AutoLeaveReason>;
// TODO if we are in "unknown" state we need a loading rendering (or empty screen)
// Otherwise it looks like we already connected and only than the ringing starts which is weird.
callPickupState$: Behavior<
"unknown" | "ringing" | "timeout" | "decline" | "success" | null
>;
/**
* Whether we are ringing a call recipient.
*/
ringing$: Behavior<boolean>;
/** Observable that emits when the user should leave the call (hangup pressed, widget action, error).
* THIS DOES NOT LEAVE THE CALL YET. The only way to leave the call (send the hangup event) is by ending the scope.
* THIS DOES NOT LEAVE THE CALL YET. The only way to leave the call (send the hangup event) is
* - by ending the scope
* - or calling requestDisconnect
*
* TODO: it seems more reasonable to add a leave() method (that calls requestDisconnect) that will then update leave$ and remove the hangup pattern
*/
leave$: Observable<"user" | AutoLeaveReason>;
/** Call to initiate hangup. Use in conbination with reconnectino state track the async hangup process. */
/** Call to initiate hangup. Use in conbination with reconnection state track the async hangup process. */
hangup: () => void;
// joining
join: () => void;
/**
* calls requestDisconnect. The async leave state can than be observed via connected$
*/
leave: () => void;
// screen sharing
/**
* Callback to toggle screen sharing. If null, screen sharing is not possible.
@@ -261,9 +281,9 @@ export interface CallViewModel {
* multiple devices.
*/
participantCount$: Behavior<number>;
allConnections$: Behavior<ConnectionManagerData>;
/** Participants sorted by livekit room so they can be used in the audio rendering */
livekitRoomItems$: Behavior<LivekitRoomItem[]>;
userMedia$: Behavior<UserMedia[]>;
/** use the layout instead, this is just for the sdk export. */
matrixLivekitMembers$: Behavior<RemoteMatrixLivekitMember[]>;
localMatrixLivekitMember$: Behavior<LocalMatrixLivekitMember | null>;
@@ -272,13 +292,6 @@ export interface CallViewModel {
/** List of reactions. Keys are: membership.membershipId (currently predefined as: `${membershipEvent.userId}:${membershipEvent.deviceId}`)*/
reactions$: Behavior<Record<string, ReactionOption>>;
ringOverlay$: Behavior<null | {
name: string;
/** roomId or userId for the avatar generation. */
idForAvatar: string;
text: string;
avatarMxc?: string;
}>;
// sounds and events
joinSoundEffect$: Observable<void>;
leaveSoundEffect$: Observable<void>;
@@ -314,10 +327,6 @@ export interface CallViewModel {
gridMode$: Behavior<GridMode>;
setGridMode: (value: GridMode) => void;
// media view models and layout
grid$: Behavior<UserMediaViewModel[]>;
spotlight$: Behavior<MediaViewModel[]>;
pip$: Behavior<UserMediaViewModel | null>;
/**
* The layout of tiles in the call interface.
*/
@@ -381,8 +390,11 @@ export function createCallViewModel$(
trackProcessorState$: Behavior<ProcessorState>,
): CallViewModel {
const client = matrixRoom.client;
const userId = client.getUserId()!;
const deviceId = client.getDeviceId()!;
const userId = client.getUserId();
const deviceId = client.getDeviceId();
if (!(userId && deviceId))
throw new UnknownCallError(new Error("userId and deviceId are required"));
const livekitKeyProvider = getE2eeKeyProvider(
options.encryptionSystem,
matrixRTCSession,
@@ -415,18 +427,49 @@ export function createCallViewModel$(
memberships$,
);
const localTransport$ = createLocalTransport$({
scope: scope,
memberships$: memberships$,
client,
roomId: matrixRoom.roomId,
useOldestMember$: scope.behavior(
matrixRTCMode$.pipe(map((v) => v === MatrixRTCMode.Legacy)),
const ownMembershipIdentity: CallMembershipIdentityParts = {
userId,
deviceId,
// This will only be consumed by the sticky membership manager. So it has no impact on legacy calls.
memberId: uuidv4(),
};
const localTransport$ = scope.behavior(
matrixRTCMode$.pipe(
generateItem(
"CallViewModel localTransport$",
// Re-create LocalTransport whenever the mode changes
(mode) => ({ keys: [mode], data: undefined }),
(scope, _data$, mode) =>
createLocalTransport$({
scope: scope,
memberships$: memberships$,
ownMembershipIdentity,
client,
delayId$: scope.behavior(
(
fromEvent(
matrixRTCSession,
MembershipManagerEvent.DelayIdChanged,
// The type of reemitted event includes the original emitted as the second arg.
) as Observable<[string | undefined, IMembershipManager]>
).pipe(map(([delayId]) => delayId ?? null)),
matrixRTCSession.delayId ?? null,
),
roomId: matrixRoom.roomId,
forceJwtEndpoint:
mode === MatrixRTCMode.Matrix_2_0
? JwtEndpointVersion.Matrix_2_0
: JwtEndpointVersion.Legacy,
useOldestMember: mode === MatrixRTCMode.Legacy,
}),
),
),
});
);
const connectionFactory = new ECConnectionFactory(
client,
matrixRoom.roomId,
mediaDevices,
trackProcessorState$,
livekitKeyProvider,
@@ -439,38 +482,30 @@ export function createCallViewModel$(
const connectionManager = createConnectionManager$({
scope: scope,
connectionFactory: connectionFactory,
inputTransports$: scope.behavior(
combineLatest(
[
localTransport$.pipe(
catchError((e: unknown) => {
logger.info(
"dont pass local transport to createConnectionManager$. localTransport$ threw an error",
e,
);
return of(null);
}),
),
membershipsAndTransports.transports$,
],
(localTransport, transports) => {
const localTransportAsArray = localTransport ? [localTransport] : [];
return transports.mapInner((transports) => [
...localTransportAsArray,
...transports,
]);
},
localTransport$: scope.behavior(
localTransport$.pipe(
switchMap((t) => t.active$),
catchError((e: unknown) => {
logger.info(
"could not pass local transport to createConnectionManager$. localTransport$ threw an error",
e,
);
return of(null);
}),
),
),
logger,
remoteTransports$: membershipsAndTransports.transports$,
logger: logger,
ownMembershipIdentity,
});
const matrixLivekitMembers$ = createMatrixLivekitMembers$({
scope: scope,
membershipsWithTransport$:
membershipsAndTransports.membershipsWithTransport$,
connectionManager: connectionManager,
});
const matrixLivekitMembers$: Behavior<Epoch<RemoteMatrixLivekitMember[]>> =
createMatrixLivekitMembers$({
scope: scope,
membershipsWithTransport$:
membershipsAndTransports.membershipsWithTransport$,
connectionManager: connectionManager,
});
const connectOptions$ = scope.behavior(
matrixRTCMode$.pipe(
@@ -483,23 +518,23 @@ export function createCallViewModel$(
);
const localMembership = createLocalMembership$({
scope: scope,
scope,
homeserverConnected: createHomeserverConnected$(
scope,
client,
matrixRTCSession,
),
muteStates: muteStates,
joinMatrixRTC: (transport: LivekitTransport) => {
muteStates,
joinMatrixRTC: (transport: LivekitTransportConfig) => {
return enterRTCSession(
matrixRTCSession,
ownMembershipIdentity,
transport,
connectOptions$.value,
);
},
createPublisherFactory: (connection: Connection) => {
return new Publisher(
scope,
connection,
mediaDevices,
muteStates,
@@ -509,9 +544,11 @@ export function createCallViewModel$(
),
);
},
connectionManager: connectionManager,
matrixRTCSession: matrixRTCSession,
localTransport$: localTransport$,
connectionManager,
matrixRTCSession,
localTransport$: scope.behavior(
localTransport$.pipe(switchMap((t) => t.advertised$)),
),
logger: logger.getChild(`[${Date.now()}]`),
});
@@ -570,49 +607,14 @@ export function createCallViewModel$(
matrixRoomMembers$,
);
const dmMember$ = createDMMember$(scope, matrixRoomMembers$, matrixRoom);
const noUserToCallInRoom$ = scope.behavior(
matrixRoomMembers$.pipe(
map(
(roomMembersMap) =>
roomMembersMap.size === 1 && roomMembersMap.get(userId) !== undefined,
),
),
const allConnections$ = scope.behavior(
connectionManager.connectionManagerData$.pipe(map((d) => d.value)),
);
const ringOverlay$ = scope.behavior(
combineLatest([noUserToCallInRoom$, dmMember$, callPickupState$]).pipe(
map(([noUserToCallInRoom, dmMember, callPickupState]) => {
// No overlay if not in ringing state
if (callPickupState !== "ringing" || noUserToCallInRoom) return null;
const name = dmMember ? dmMember.rawDisplayName : matrixRoom.name;
const id = dmMember ? dmMember.userId : matrixRoom.roomId;
const text = dmMember
? `Waiting for ${name} to join…`
: "Waiting for other participants…";
const avatarMxc = dmMember
? (dmMember.getMxcAvatarUrl?.() ?? undefined)
: (matrixRoom.getMxcAvatarUrl() ?? undefined);
return {
name: name ?? id,
idForAvatar: id,
text,
avatarMxc,
};
}),
),
);
const livekitRoomItems$ = scope.behavior(
matrixLivekitMembers$.pipe(
tap((val) => {
logger.debug("matrixLivekitMembers$ updated", val.value);
}),
switchMap((membersWithEpoch) => {
const members = membersWithEpoch.value;
switchMap((members) => {
const a$ = combineLatest(
members.map((member) =>
members.value.map((member) =>
combineLatest([member.connection$, member.participant.value$]).pipe(
map(([connection, participant]) => {
// do not render audio for local participant
@@ -674,129 +676,146 @@ export function createCallViewModel$(
/**
* List of user media (camera feeds) that we want tiles for.
*/
const userMedia$ = scope.behavior<UserMedia[]>(
const userMedia$ = scope.behavior<WrappedUserMediaViewModel[]>(
combineLatest([
localMatrixLivekitMember$,
matrixLivekitMembers$,
duplicateTiles.value$,
]).pipe(
// Generate a collection of MediaItems from the list of expected (whether
// Generate a collection of user media from the list of expected (whether
// present or missing) LiveKit participants.
generateItems(
"CallViewModel userMedia$",
function* ([
localMatrixLivekitMember,
{ value: matrixLivekitMembers },
matrixLivekitMembers,
duplicateTiles,
]) {
let localParticipantId: string | undefined = undefined;
// add local member if available
if (localMatrixLivekitMember) {
const computeMediaId = (m: MatrixLivekitMember): string =>
`${m.userId}:${m.membership$.value.deviceId}`;
const localUserMediaId = localMatrixLivekitMember
? computeMediaId(localMatrixLivekitMember)
: undefined;
const localAsArray = localMatrixLivekitMember
? [localMatrixLivekitMember]
: [];
const remoteWithoutLocal = matrixLivekitMembers.value.filter(
(m) => computeMediaId(m) !== localUserMediaId,
);
const allMatrixLivekitMembers = [
...localAsArray,
...remoteWithoutLocal,
];
for (const matrixLivekitMember of allMatrixLivekitMembers) {
const { userId, participant, connection$, membership$ } =
localMatrixLivekitMember;
localParticipantId = `${userId}:${membership$.value.deviceId}`; // should be membership$.value.membershipID which is not optional
// const participantId = membership$.value.membershipID;
if (localParticipantId) {
for (let dup = 0; dup < 1 + duplicateTiles; dup++) {
yield {
keys: [
dup,
localParticipantId,
userId,
participant satisfies TaggedParticipant as TaggedParticipant, // Widen the type safely
connection$,
],
data: undefined,
};
}
}
}
// add remote members that are available
for (const {
userId,
participant,
connection$,
membership$,
} of matrixLivekitMembers) {
const participantId = `${userId}:${membership$.value.deviceId}`;
if (participantId === localParticipantId) continue;
// const participantId = membership$.value?.identity;
matrixLivekitMember;
const rtcId = membership$.value.rtcBackendIdentity; // rtcBackendIdentity
const mediaId = computeMediaId(matrixLivekitMember);
for (let dup = 0; dup < 1 + duplicateTiles; dup++) {
yield {
keys: [dup, participantId, userId, participant, connection$],
keys: [dup, mediaId, userId, participant, connection$, rtcId],
data: undefined,
};
}
}
},
(
scope,
_data$,
dup,
participantId,
userId,
participant,
connection$,
) => {
const livekitRoom$ = scope.behavior(
connection$.pipe(map((c) => c?.livekitRoom)),
);
const focusUrl$ = scope.behavior(
connection$.pipe(map((c) => c?.transport.livekit_service_url)),
);
const displayName$ = scope.behavior(
matrixMemberMetadataStore
.createDisplayNameBehavior$(userId)
.pipe(map((name) => name ?? userId)),
);
return new UserMedia(
scope,
`${participantId}:${dup}`,
(scope, _, dup, mediaId, userId, participant, connection$, rtcId) =>
createWrappedUserMedia(scope, {
id: `${mediaId}:${dup}`,
userId,
rtcBackendIdentity: rtcId,
participant,
options.encryptionSystem,
livekitRoom$,
focusUrl$,
encryptionSystem: options.encryptionSystem,
livekitRoom$: scope.behavior(
connection$.pipe(map((c) => c?.livekitRoom)),
),
focusUrl$: scope.behavior(
connection$.pipe(map((c) => c?.transport.livekit_service_url)),
),
mediaDevices,
localMembership.reconnecting$,
displayName$,
matrixMemberMetadataStore.createAvatarUrlBehavior$(userId),
handsRaised$.pipe(map((v) => v[participantId]?.time ?? null)),
reactions$.pipe(map((v) => v[participantId] ?? undefined)),
);
},
pretendToBeDisconnected$: localMembership.reconnecting$,
displayName$: scope.behavior(
matrixMemberMetadataStore
.createDisplayNameBehavior$(userId)
.pipe(map((name) => name ?? userId)),
),
mxcAvatarUrl$:
matrixMemberMetadataStore.createAvatarUrlBehavior$(userId),
handRaised$: scope.behavior(
handsRaised$.pipe(map((v) => v[mediaId]?.time ?? null)),
),
reaction$: scope.behavior(
reactions$.pipe(map((v) => v[mediaId] ?? undefined)),
),
}),
),
),
);
const ringingMedia$ = scope.behavior<RingingMediaViewModel[]>(
combineLatest([userMedia$, matrixRoomMembers$, callPickupState$]).pipe(
generateItems(
"CallViewModel ringingMedia$",
function* ([userMedia, roomMembers, callPickupState]) {
if (
callPickupState === "ringing" ||
callPickupState === "timeout" ||
callPickupState === "decline"
) {
for (const member of roomMembers.values()) {
if (!userMedia.some((vm) => vm.userId === member.userId))
yield {
keys: [member.userId],
data: callPickupState,
};
}
}
},
(scope, pickupState$, userId) =>
createRingingMedia({
id: `ringing:${userId}`,
userId,
displayName$: scope.behavior(
matrixRoomMembers$.pipe(
map((members) => members.get(userId)?.rawDisplayName || userId),
),
),
mxcAvatarUrl$:
matrixMemberMetadataStore.createAvatarUrlBehavior$(userId),
pickupState$,
muteStates,
}),
),
distinctUntilChanged(shallowEquals),
tap((ringingMedia) => {
if (ringingMedia.length > 1)
// Warn that UI may do something unexpected in this case
logger.warn(
`Ringing more than one participant is not supported (ringing ${ringingMedia.map((vm) => vm.userId).join(", ")})`,
);
}),
),
);
/**
* List of all media items (user media and screen share media) that we want
* tiles for.
* All screen share media that we want to display.
*/
const mediaItems$ = scope.behavior<MediaItem[]>(
const screenShares$ = scope.behavior<ScreenShareViewModel[]>(
userMedia$.pipe(
switchMap((userMedia) =>
userMedia.length === 0
? of([])
: combineLatest(
userMedia.map((m) => m.screenShares$),
(...screenShares) => [...userMedia, ...screenShares.flat(1)],
(...screenShares) => screenShares.flat(1),
),
),
),
);
/**
* List of MediaItems that we want to display, that are of type ScreenShare
*/
const screenShares$ = scope.behavior<ScreenShare[]>(
mediaItems$.pipe(
map((mediaItems) =>
mediaItems.filter((m): m is ScreenShare => m instanceof ScreenShare),
),
),
);
const joinSoundEffect$ = userMedia$.pipe(
pairwise(),
filter(
@@ -856,39 +875,39 @@ export function createCallViewModel$(
merge(userHangup$, widgetHangup$).pipe(map(() => "user" as const)),
).pipe(scope.share);
const spotlightSpeaker$ = scope.behavior<UserMediaViewModel | null>(
const spotlightSpeaker$ = scope.behavior<UserMediaViewModel | undefined>(
userMedia$.pipe(
switchMap((mediaItems) =>
mediaItems.length === 0
? of([])
: combineLatest(
mediaItems.map((m) =>
m.vm.speaking$.pipe(map((s) => [m, s] as const)),
m.speaking$.pipe(map((s) => [m, s] as const)),
),
),
),
scan<(readonly [UserMedia, boolean])[], UserMedia | undefined, null>(
(prev, mediaItems) => {
// Only remote users that are still in the call should be sticky
const [stickyMedia, stickySpeaking] =
(!prev?.vm.local && mediaItems.find(([m]) => m === prev)) || [];
// Decide who to spotlight:
// If the previous speaker is still speaking, stick with them rather
// than switching eagerly to someone else
return stickySpeaking
? stickyMedia!
: // Otherwise, select any remote user who is speaking
(mediaItems.find(([m, s]) => !m.vm.local && s)?.[0] ??
// Otherwise, stick with the person who was last speaking
stickyMedia ??
// Otherwise, spotlight an arbitrary remote user
mediaItems.find(([m]) => !m.vm.local)?.[0] ??
// Otherwise, spotlight the local user
mediaItems.find(([m]) => m.vm.local)?.[0]);
},
null,
),
map((speaker) => speaker?.vm ?? null),
scan<
(readonly [UserMediaViewModel, boolean])[],
UserMediaViewModel | undefined,
undefined
>((prev, mediaItems) => {
// Only remote users that are still in the call should be sticky
const [stickyMedia, stickySpeaking] =
(!prev?.local && mediaItems.find(([m]) => m === prev)) || [];
// Decide who to spotlight:
// If the previous speaker is still speaking, stick with them rather
// than switching eagerly to someone else
return stickySpeaking
? stickyMedia!
: // Otherwise, select any remote user who is speaking
(mediaItems.find(([m, s]) => !m.local && s)?.[0] ??
// Otherwise, stick with the person who was last speaking
stickyMedia ??
// Otherwise, spotlight an arbitrary remote user
mediaItems.find(([m]) => !m.local)?.[0] ??
// Otherwise, spotlight the local user
mediaItems.find(([m]) => m.local)?.[0]);
}, undefined),
),
);
@@ -902,71 +921,71 @@ export function createCallViewModel$(
return bins.length === 0
? of([])
: combineLatest(bins, (...bins) =>
bins.sort(([, bin1], [, bin2]) => bin1 - bin2).map(([m]) => m.vm),
bins.sort(([, bin1], [, bin2]) => bin1 - bin2).map(([m]) => m),
);
}),
distinctUntilChanged(shallowEquals),
),
);
const spotlight$ = scope.behavior<MediaViewModel[]>(
screenShares$.pipe(
switchMap((screenShares) => {
if (screenShares.length > 0) {
return of(screenShares.map((m) => m.vm));
}
return spotlightSpeaker$.pipe(
map((speaker) => (speaker ? [speaker] : [])),
/**
* Local user media suitable for displaying in a PiP (undefined if not found
* or if user prefers to not see themselves).
*/
const localUserMediaForPip$ = scope.behavior<
LocalUserMediaViewModel | undefined
>(
userMedia$.pipe(
switchMap((userMedia) => {
const localUserMedia = userMedia.find(
(m): m is WrappedUserMediaViewModel & LocalUserMediaViewModel =>
m.type === "user" && m.local,
);
if (!localUserMedia) return of(undefined);
return localUserMedia.alwaysShow$.pipe(
map((alwaysShow) => (alwaysShow ? localUserMedia : undefined)),
);
}),
distinctUntilChanged<MediaViewModel[]>(shallowEquals),
),
);
const pip$ = scope.behavior<UserMediaViewModel | null>(
combineLatest([
// TODO This also needs epoch logic to dedupe the screenshares and mediaItems emits
screenShares$,
spotlightSpeaker$,
mediaItems$,
]).pipe(
switchMap(([screenShares, spotlight, mediaItems]) => {
if (screenShares.length > 0) {
return spotlightSpeaker$;
}
if (!spotlight || spotlight.local) {
return of(null);
}
const spotlightAndPip$ = scope.behavior<{
spotlight: MediaViewModel[];
pip$: Behavior<UserMediaViewModel | undefined>;
}>(
ringingMedia$.pipe(
switchMap((ringingMedia) => {
if (ringingMedia.length > 0)
return of({ spotlight: ringingMedia, pip$: localUserMediaForPip$ });
const localUserMedia = mediaItems.find(
(m) => m.vm instanceof LocalUserMediaViewModel,
) as UserMedia | undefined;
return screenShares$.pipe(
switchMap((screenShares) => {
if (screenShares.length > 0)
return of({ spotlight: screenShares, pip$: spotlightSpeaker$ });
const localUserMediaViewModel = localUserMedia?.vm as
| LocalUserMediaViewModel
| undefined;
if (!localUserMediaViewModel) {
return of(null);
}
return localUserMediaViewModel.alwaysShow$.pipe(
map((alwaysShow) => {
if (alwaysShow) {
return localUserMediaViewModel;
}
return null;
return spotlightSpeaker$.pipe(
map((speaker) => ({
spotlight: speaker ? [speaker] : [],
pip$: localUserMediaForPip$,
})),
);
}),
);
}),
),
);
const spotlight$ = scope.behavior<MediaViewModel[]>(
spotlightAndPip$.pipe(
map(({ spotlight }) => spotlight),
distinctUntilChanged<MediaViewModel[]>(shallowEquals),
),
);
const hasRemoteScreenShares$ = scope.behavior<boolean>(
spotlight$.pipe(
map((spotlight) =>
spotlight.some((vm) => !vm.local && vm instanceof ScreenShareViewModel),
spotlight.some((vm) => vm.type === "screen share" && !vm.local),
),
),
);
@@ -1007,8 +1026,10 @@ export function createCallViewModel$(
);
const spotlightExpandedToggle$ = new Subject<void>();
const spotlightExpanded$ = scope.behavior<boolean>(
spotlightExpandedToggle$.pipe(accumulate(false, (expanded) => !expanded)),
const spotlightExpanded$ = createToggle$(
scope,
false,
spotlightExpandedToggle$,
);
const { setGridMode, gridMode$ } = createLayoutModeSwitch(
@@ -1021,7 +1042,7 @@ export function createCallViewModel$(
[grid$, spotlight$],
(grid, spotlight) => ({
type: "grid",
spotlight: spotlight.some((vm) => vm instanceof ScreenShareViewModel)
spotlight: spotlight.some((vm) => vm.type === "screen share")
? spotlight
: undefined,
grid,
@@ -1043,28 +1064,61 @@ export function createCallViewModel$(
}));
const spotlightExpandedLayoutMedia$: Observable<SpotlightExpandedLayoutMedia> =
combineLatest([spotlight$, pip$], (spotlight, pip) => ({
type: "spotlight-expanded",
spotlight,
pip: pip ?? undefined,
}));
spotlightAndPip$.pipe(
switchMap(({ spotlight, pip$ }) =>
pip$.pipe(
map((pip) => ({
type: "spotlight-expanded" as const,
spotlight,
pip: pip ?? undefined,
})),
),
),
);
const oneOnOneLayoutMedia$: Observable<OneOnOneLayoutMedia | null> =
mediaItems$.pipe(
map((mediaItems) => {
if (mediaItems.length !== 2) return null;
const local = mediaItems.find((vm) => vm.vm.local)?.vm as
| LocalUserMediaViewModel
| undefined;
const remote = mediaItems.find((vm) => !vm.vm.local)?.vm as
| RemoteUserMediaViewModel
| undefined;
// There might not be a remote tile if there are screen shares, or if
// only the local user is in the call and they're using the duplicate
// tiles option
if (!remote || !local) return null;
userMedia$.pipe(
switchMap((userMedia) => {
if (userMedia.length <= 2) {
const local = userMedia.find(
(vm): vm is WrappedUserMediaViewModel & LocalUserMediaViewModel =>
vm.type === "user" && vm.local,
);
return { type: "one-on-one", local, remote };
if (local !== undefined) {
const remote = userMedia.find(
(
vm,
): vm is WrappedUserMediaViewModel & RemoteUserMediaViewModel =>
vm.type === "user" && !vm.local,
);
if (remote !== undefined)
return of({
type: "one-on-one" as const,
spotlight: remote,
pip: local,
});
// If there's no other user media in the call (could still happen in
// this branch due to the duplicate tiles option), we could possibly
// show ringing media instead
if (userMedia.length === 1)
return ringingMedia$.pipe(
map((ringingMedia) => {
return ringingMedia.length === 1
? {
type: "one-on-one" as const,
spotlight: local,
pip: ringingMedia[0],
}
: null;
}),
);
}
}
return of(null);
}),
);
@@ -1106,7 +1160,7 @@ export function createCallViewModel$(
oneOnOne === null
? combineLatest([grid$, spotlight$], (grid, spotlight) =>
grid.length > smallMobileCallThreshold ||
spotlight.some((vm) => vm instanceof ScreenShareViewModel)
spotlight.some((vm) => vm.type === "screen share")
? spotlightPortraitLayoutMedia$
: gridLayoutMedia$,
).pipe(switchAll())
@@ -1213,7 +1267,7 @@ export function createCallViewModel$(
// screen sharing feeds are in the spotlight we still need them.
return l.spotlight.media$.pipe(
map((models: MediaViewModel[]) =>
models.some((m) => m instanceof ScreenShareViewModel),
models.some((m) => m.type === "screen share"),
),
);
// In expanded spotlight layout, the active speaker is always shown in
@@ -1264,7 +1318,7 @@ export function createCallViewModel$(
switchMap((mode) => {
switch (mode) {
case "pip":
return of(false);
return of(platform === "desktop" ? true : false);
case "normal":
case "narrow":
return of(true);
@@ -1475,11 +1529,13 @@ export function createCallViewModel$(
return {
autoLeave$: autoLeave$,
callPickupState$: callPickupState$,
ringOverlay$: ringOverlay$,
ringing$: scope.behavior(
callPickupState$.pipe(map((state) => state === "ringing")),
),
leave$: leave$,
hangup: (): void => userHangup$.next(),
join: localMembership.requestJoinAndPublish,
leave: localMembership.requestDisconnect,
toggleScreenSharing: toggleScreenSharing,
sharingScreen$: sharingScreen$,
@@ -1503,6 +1559,7 @@ export function createCallViewModel$(
),
null,
),
allConnections$,
participantCount$: participantCount$,
handsRaised$: handsRaised$,
reactions$: reactions$,
@@ -1518,17 +1575,21 @@ export function createCallViewModel$(
toggleSpotlightExpanded$: toggleSpotlightExpanded$,
gridMode$: gridMode$,
setGridMode: setGridMode,
grid$: grid$,
spotlight$: spotlight$,
pip$: pip$,
layout$: layout$,
userMedia$,
localMatrixLivekitMember$,
matrixLivekitMembers$: scope.behavior(
matrixLivekitMembers$.pipe(
map((members) => members.value),
tap((v) => {
logger.debug("matrixLivekitMembers$ updated (exported)", v);
const listForLogs = v
.map(
(m) =>
m.membership$.value.userId + "|" + m.membership$.value.deviceId,
)
.join(",");
logger.debug(
`matrixLivekitMembers$ updated (exported) [${listForLogs}]`,
);
}),
),
),

View File

@@ -17,7 +17,7 @@ import {
import { SyncState } from "matrix-js-sdk/lib/sync";
import { BehaviorSubject, type Observable, map, of } from "rxjs";
import { onTestFinished, vi } from "vitest";
import { ClientEvent, type MatrixClient } from "matrix-js-sdk";
import { ClientEvent, type RoomMember, type MatrixClient } from "matrix-js-sdk";
import EventEmitter from "events";
import * as ComponentsCore from "@livekit/components-core";
@@ -63,15 +63,10 @@ const carol = local;
const dave = mockMatrixRoomMember(daveRTLRtcMember, { rawDisplayName: "Dave" });
const roomMembers = new Map(
[alice, aliceDoppelganger, bob, bobZeroWidthSpace, carol, dave, daveRTL].map(
(p) => [p.userId, p],
),
);
export interface CallViewModelInputs {
remoteParticipants$: Behavior<RemoteParticipant[]>;
rtcMembers$: Behavior<Partial<CallMembership>[]>;
roomMembers: RoomMember[];
livekitConnectionState$: Behavior<ConnectionState>;
speaking: Map<Participant, Observable<boolean>>;
mediaDevices: MediaDevices;
@@ -86,6 +81,15 @@ export function withCallViewModel(mode: MatrixRTCMode) {
{
remoteParticipants$ = constant([]),
rtcMembers$ = constant([localRtcMember]),
roomMembers = [
alice,
aliceDoppelganger,
bob,
bobZeroWidthSpace,
carol,
dave,
daveRTL,
],
livekitConnectionState$: connectionState$ = constant(
ConnectionState.Connected,
),
@@ -128,8 +132,8 @@ export function withCallViewModel(mode: MatrixRTCMode) {
return syncState;
}
})() as Partial<MatrixClient> as MatrixClient,
getMembers: () => Array.from(roomMembers.values()),
getMembersWithMembership: () => Array.from(roomMembers.values()),
getMembers: () => roomMembers,
getMembersWithMembership: () => roomMembers,
});
const rtcSession = new MockRTCSession(room, []).withMemberships(
rtcMembers$,

View File

@@ -8,7 +8,7 @@ Please see LICENSE in the repository root for full details.
import {
Status as RTCMemberStatus,
type LivekitTransport,
type LivekitTransportConfig,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { describe, expect, it, vi } from "vitest";
@@ -24,6 +24,7 @@ import {
mockLivekitRoom,
mockMuteStates,
withTestScheduler,
ownMemberMock,
} from "../../../utils/test";
import {
TransportState,
@@ -38,6 +39,9 @@ import { constant } from "../../Behavior";
import { ConnectionManagerData } from "../remoteMembers/ConnectionManager";
import { ConnectionState, type Connection } from "../remoteMembers/Connection";
import { type Publisher } from "./Publisher";
import { initializeWidget } from "../../../widget";
initializeWidget();
const MATRIX_RTC_MODE = MatrixRTCMode.Legacy;
const getUrlParams = vi.hoisted(() => vi.fn(() => ({})));
@@ -103,11 +107,12 @@ describe("LocalMembership", () => {
getOldestMembership: vi.fn().mockReturnValue({
getPreferredFoci: vi.fn().mockReturnValue([focusFromOlderMembership]),
}),
joinRoomSession: vi.fn(),
joinRTCSession: vi.fn(),
}) as unknown as MatrixRTCSession;
enterRTCSession(
mockedSession,
ownMemberMock,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
@@ -119,7 +124,12 @@ describe("LocalMembership", () => {
},
);
expect(mockedSession.joinRoomSession).toHaveBeenLastCalledWith(
expect(mockedSession.joinRTCSession).toHaveBeenLastCalledWith(
{
deviceId: "DEVICE",
memberId: "@alice:example.org:DEVICE",
userId: "@alice:example.org",
},
[
{
livekit_alias: "roomId",
@@ -161,11 +171,12 @@ describe("LocalMembership", () => {
},
memberships: [],
getFocusInUse: vi.fn(),
joinRoomSession: vi.fn(),
joinRTCSession: vi.fn(),
}) as unknown as MatrixRTCSession;
enterRTCSession(
mockedSession,
ownMemberMock,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
@@ -185,8 +196,8 @@ describe("LocalMembership", () => {
matrixRTCMode: MatrixRTCMode.Matrix_2_0,
}),
matrixRTCSession: {
updateCallIntent: () => {},
leaveRoomSession: () => {},
updateCallIntent: vi.fn().mockReturnValue(Promise.resolve()),
leaveRoomSession: vi.fn(),
} as unknown as MatrixRTCSession,
muteStates: mockMuteStates(),
trackProcessorState$: constant({
@@ -204,7 +215,7 @@ describe("LocalMembership", () => {
it("throws error on missing RTC config error", () => {
withTestScheduler(({ scope, hot, expectObservable }) => {
const localTransport$ = scope.behavior<null | LivekitTransport>(
const localTransport$ = scope.behavior<null | LivekitTransportConfig>(
hot("1ms #", {}, new MatrixRTCTransportMissingError("domain.com")),
null,
);
@@ -234,12 +245,43 @@ describe("LocalMembership", () => {
});
});
it("logs if callIntent cannot be updated", async () => {
const scope = new ObservableScope();
const localTransport$ = new BehaviorSubject(aTransport);
const mockConnectionManager = {
transports$: constant(new Epoch([])),
connectionManagerData$: constant(new Epoch(new ConnectionManagerData())),
};
async function reject(): Promise<void> {
return Promise.reject(new Error("Not connected yet"));
}
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
matrixRTCSession: {
updateCallIntent: vi.fn().mockImplementation(reject),
leaveRoomSession: vi.fn(),
},
connectionManager: mockConnectionManager,
localTransport$,
});
const expextedLog =
"'not connected yet' while updating the call intent (this is expected on startup)";
const internalLogger = vi.spyOn(localMembership.internalLoggerRef, "debug");
await flushPromises();
defaultCreateLocalMemberValues.muteStates.video.setEnabled$.value?.(true);
expect(internalLogger).toHaveBeenCalledWith(expextedLog);
scope.end();
});
const aTransport = {
livekit_service_url: "a",
} as LivekitTransport;
} as LivekitTransportConfig;
const bTransport = {
livekit_service_url: "b",
} as LivekitTransport;
} as LivekitTransportConfig;
const connectionTransportAConnected = {
livekitRoom: mockLivekitRoom({
@@ -262,7 +304,7 @@ describe("LocalMembership", () => {
livekitRoom: mockLivekitRoom({}),
} as unknown as Connection;
it("recreates publisher if new connection is used and ENDS always unpublish and end tracks", async () => {
it("recreates publisher if new connection is used, always unpublish and end tracks", async () => {
const scope = new ObservableScope();
const localTransport$ = new BehaviorSubject(aTransport);
@@ -275,6 +317,9 @@ describe("LocalMembership", () => {
seed += 1;
logger.info(`creating [${a}]`);
const p = {
// It is enought to check if destroy is called. Destroy itself is tested in the publisher to make sure it does
// all the cleanup we need.
destroy: vi.fn(),
stopPublishing: vi.fn().mockImplementation(() => {
logger.info(`stopPublishing [${a}]`);
}),
@@ -303,19 +348,18 @@ describe("LocalMembership", () => {
await flushPromises();
localTransport$.next(bTransport);
await flushPromises();
expect(publisherFactory).toHaveBeenCalledTimes(2);
expect(publishers.length).toBe(2);
// stop the first Publisher and let the second one life.
expect(publishers[0].stopTracks).toHaveBeenCalled();
expect(publishers[1].stopTracks).not.toHaveBeenCalled();
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[1].stopPublishing).not.toHaveBeenCalled();
expect(publishers[0].destroy).toHaveBeenCalled();
expect(publishers[1].destroy).not.toHaveBeenCalled();
expect(publisherFactory.mock.calls[0][0].transport).toBe(aTransport);
expect(publisherFactory.mock.calls[1][0].transport).toBe(bTransport);
scope.end();
await flushPromises();
// stop all tracks after ending scopes
expect(publishers[1].stopPublishing).toHaveBeenCalled();
expect(publishers[1].destroy).toHaveBeenCalled();
// expect(publishers[1].stopTracks).toHaveBeenCalled();
defaultCreateLocalMemberValues.createPublisherFactory.mockReset();
@@ -333,8 +377,9 @@ describe("LocalMembership", () => {
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const p = {
stopPublishing: vi.fn(),
stopTracks: vi.fn(),
// It is enought to check if destroy is called. Destroy itself is tested in the publisher to make sure it does
// all the cleanup we need.
destroy: vi.fn(),
createAndSetupTracks: vi.fn().mockImplementation(async () => {
tracks$.next([{}, {}] as LocalTrack[]);
return Promise.resolve();
@@ -369,11 +414,11 @@ describe("LocalMembership", () => {
localMembership.startTracks();
await flushPromises();
expect(publishers[0].createAndSetupTracks).toHaveBeenCalled();
// expect(localMembership.tracks$.value.length).toBe(2);
scope.end();
await flushPromises();
// stop all tracks after ending scopes
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[0].destroy).toHaveBeenCalled();
// expect(publishers[0].stopTracks).toHaveBeenCalled();
publisherFactory.mockClear();
});
@@ -383,33 +428,30 @@ describe("LocalMembership", () => {
const scope = new ObservableScope();
const connectionManagerData = new ConnectionManagerData();
const localTransport$ = new BehaviorSubject<null | LivekitTransport>(null);
const localTransport$ = new BehaviorSubject<null | LivekitTransportConfig>(
null,
);
const connectionManagerData$ = new BehaviorSubject(
new Epoch(connectionManagerData),
);
const publishers: Publisher[] = [];
const tracks$ = new BehaviorSubject<LocalTrack[]>([]);
const publishing$ = new BehaviorSubject<boolean>(false);
const createTrackResolver = Promise.withResolvers<void>();
const publishResolver = Promise.withResolvers<void>();
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const p = {
stopPublishing: vi.fn(),
stopTracks: vi.fn().mockImplementation(() => {
logger.info("stopTracks");
tracks$.next([]);
}),
// It is enought to check if destroy is called. Destroy itself is tested in the publisher to make sure it does
// all the cleanup we need.
destroy: vi.fn(),
createAndSetupTracks: vi.fn().mockImplementation(async () => {
await createTrackResolver.promise;
tracks$.next([{}, {}] as LocalTrack[]);
}),
startPublishing: vi.fn().mockImplementation(async () => {
await publishResolver.promise;
publishing$.next(true);
}),
tracks$,
publishing$,
};
publishers.push(p as unknown as Publisher);
@@ -509,7 +551,7 @@ describe("LocalMembership", () => {
(localMembership.localMemberState$.value as any).media,
).toStrictEqual(PublishState.Publishing);
expect(publishers[0].stopPublishing).not.toHaveBeenCalled();
expect(publishers[0].destroy).not.toHaveBeenCalled();
expect(localMembership.localMemberState$.isStopped).toBe(false);
scope.end();
@@ -520,7 +562,7 @@ describe("LocalMembership", () => {
(localMembership.localMemberState$.value as any).media,
).toStrictEqual(PublishState.Publishing);
// stop all tracks after ending scopes
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[0].destroy).toHaveBeenCalled();
// expect(publishers[0].stopTracks).toHaveBeenCalled();
});
// TODO add tests for matrix local matrix participation.

View File

@@ -17,6 +17,7 @@ import { observeParticipantEvents } from "@livekit/components-core";
import {
Status as RTCSessionStatus,
type LivekitTransport,
type LivekitTransportConfig,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import {
@@ -36,6 +37,7 @@ import {
} from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { deepCompare } from "matrix-js-sdk/lib/utils";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { type Behavior } from "../../Behavior.ts";
import { type IConnectionManager } from "../remoteMembers/ConnectionManager.ts";
@@ -123,9 +125,9 @@ interface Props {
muteStates: MuteStates;
connectionManager: IConnectionManager;
createPublisherFactory: (connection: Connection) => Publisher;
joinMatrixRTC: (transport: LivekitTransport) => void;
joinMatrixRTC: (transport: LivekitTransportConfig) => void;
homeserverConnected: HomeserverConnected;
localTransport$: Behavior<LivekitTransport | null>;
localTransport$: Behavior<LivekitTransportConfig | null>;
matrixRTCSession: Pick<
MatrixRTCSession,
"updateCallIntent" | "leaveRoomSession"
@@ -144,7 +146,7 @@ interface Props {
* @param props.createPublisherFactory Factory to create a publisher once we have a connection.
* @param props.joinMatrixRTC Callback to join the matrix RTC session once we have a transport.
* @param props.homeserverConnected The homeserver connected state.
* @param props.localTransport$ The local transport to use for publishing.
* @param props.localTransport$ The transport to advertise in our membership.
* @param props.logger The logger to use.
* @param props.muteStates The mute states for video and audio.
* @param props.matrixRTCSession The matrix RTC session to join.
@@ -198,6 +200,7 @@ export const createLocalMembership$ = ({
* Fully connected
*/
connected$: Behavior<boolean>;
internalLoggerRef: Logger;
} => {
const logger = parentLogger.getChild("[LocalMembership]");
logger.debug(`Creating local membership..`);
@@ -262,11 +265,11 @@ export const createLocalMembership$ = ({
mediaErrors$.pipe(scope.bind()).subscribe((error) => {
if (error) {
// This is a MediaDevice error, can be PermissionDenied, NotFound, DeviceInUse, Other.
// Will also occurs if you cancel screen sharing browser prompt.
// This is not necessarily fatal, since the user might be able to join without media.
// XXX We might want to give some user feedback here to let them know their media is not working.
logger.error(`Failed to create local tracks:`, error);
setMatrixError(
// TODO is it fatal? Do we need to create a new Specialized Error?
new UnknownCallError(new Error(`Media device error: ${error}`)),
);
}
});
// MATRIX RELATED
@@ -306,13 +309,17 @@ export const createLocalMembership$ = ({
// - destruct all current streams
// - overwrite current publisher
scope.reconcile(localConnection$, async (connection) => {
logger.info(
"reconcile based on new localConnection:",
connection?.transport.livekit_service_url,
);
if (connection !== null) {
const publisher = createPublisherFactory(connection);
publisher$.next(publisher);
// Clean-up callback
return Promise.resolve(async (): Promise<void> => {
await publisher.stopPublishing();
await publisher.stopTracks();
await publisher.destroy();
});
}
});
@@ -516,12 +523,19 @@ export const createLocalMembership$ = ({
}
});
combineLatest([muteStates.video.enabled$, homeserverConnected.combined$])
.pipe(scope.bind())
.subscribe(([videoEnabled, connected]) => {
if (!connected) return;
void matrixRTCSession.updateCallIntent(videoEnabled ? "video" : "audio");
});
muteStates.video.enabled$.pipe(scope.bind()).subscribe((videoEnabled) => {
void matrixRTCSession
.updateCallIntent(videoEnabled ? "video" : "audio")
.catch((e) => {
if (e instanceof Error && e.message === "Not connected yet") {
logger.debug(
"'not connected yet' while updating the call intent (this is expected on startup)",
);
} else {
throw e;
}
});
});
// Keep matrix rtc session in sync with localTransport$, connectRequested$
scope.reconcile(
@@ -551,7 +565,12 @@ export const createLocalMembership$ = ({
);
const participant$ = scope.behavior(
localConnection$.pipe(map((c) => c?.livekitRoom?.localParticipant ?? null)),
localConnection$.pipe(
map((c) => c?.livekitRoom?.localParticipant ?? null),
tap((p) => {
logger.debug("participant$ updated:", p?.identity);
}),
),
);
// Pause upstream of all local media tracks when we're disconnected from
@@ -660,6 +679,7 @@ export const createLocalMembership$ = ({
sharingScreen$,
toggleScreenSharing,
connection$: localConnection$,
internalLoggerRef: logger,
};
};
@@ -686,18 +706,19 @@ interface EnterRTCSessionOptions {
* - Handles retries (fails only after several attempts)
*
* @param rtcSession - The MatrixRTCSession to join.
* @param ownMembershipIdentity - Options for entering the RTC session.
* @param transport - The LivekitTransport to use for this session.
* @param options - Options for entering the RTC session.
* @param options.encryptMedia - Whether to encrypt media.
* @param options.matrixRTCMode - The Matrix RTC mode to use.
* @param options - `encryptMedia`: Whether to encrypt media `matrixRTCMode`: The Matrix RTC mode to use.
* @throws If the widget could not send ElementWidgetActions.JoinCall action.
*/
// Exported for unit testing
export function enterRTCSession(
rtcSession: MatrixRTCSession,
transport: LivekitTransport,
{ encryptMedia, matrixRTCMode }: EnterRTCSessionOptions,
ownMembershipIdentity: CallMembershipIdentityParts,
transport: LivekitTransportConfig,
options: EnterRTCSessionOptions,
): void {
const { encryptMedia, matrixRTCMode } = options;
PosthogAnalytics.instance.eventCallEnded.cacheStartCall(new Date());
PosthogAnalytics.instance.eventCallStarted.track(rtcSession.room.roomId);
@@ -709,12 +730,29 @@ export function enterRTCSession(
const useDeviceSessionMemberEvents =
features?.feature_use_device_session_member_events;
const { sendNotificationType: notificationType, callIntent } = getUrlParams();
const multiSFU = matrixRTCMode !== MatrixRTCMode.Legacy;
const multiSFU =
matrixRTCMode === MatrixRTCMode.Compatibility ||
matrixRTCMode === MatrixRTCMode.Matrix_2_0;
// For backwards compatibility with Element Call versions that do not do Matrix 2.0,
// we add the livekit alias to the transport.
let backwardCompatibleTransport: LivekitTransport | LivekitTransportConfig;
if (matrixRTCMode === MatrixRTCMode.Matrix_2_0) {
backwardCompatibleTransport = transport;
} else {
backwardCompatibleTransport = {
livekit_alias: rtcSession.room.roomId,
...transport,
};
}
// Multi-sfu does not need a preferred foci list. just the focus that is actually used.
// TODO where/how do we track errors originating from the ongoing rtcSession?
rtcSession.joinRoomSession(
multiSFU ? [] : [transport],
multiSFU ? transport : undefined,
rtcSession.joinRTCSession(
ownMembershipIdentity,
multiSFU ? [] : [backwardCompatibleTransport],
multiSFU ? backwardCompatibleTransport : undefined,
{
notificationType,
callIntent,

Some files were not shown because too many files have changed in this diff Show More