62 Commits

Author SHA1 Message Date
Robin
d516f0f602 Make Element Call widgets compatible with Matrix 2.0 mode
This gives Element Call widgets the ability to send and receive sticky RTC membership events.
2026-01-21 19:59:34 +01:00
Timo K
7dbbd763b9 Refactor how we aquire the jwt token for the local user. (only fetch it
once)

The local jwt token needs to be aquired via the right endpoint. The
endpoint defines how our rtcBackendIdentity is computed. Based on us
using sticky events or state events we also need to use the right
endpoint. This cannot be done generically in the connection manager. The
jwt token now is computed in the localTransport and the resolved sfu
config is passed to the connection manager.

Add JWT endpoint version and SFU config support Pin matrix-js-sdk to a
specific commit and update dev auth image tag. Propagate SFU config and
JWT endpoint choice through local transport, ConnectionManager and
Connection; add JwtEndpointVersion enum and LocalTransportWithSFUConfig
type. Add NO_MATRIX_2 auth error and locale string, thread
rtcBackendIdentity through UI props, and include related test, CSS and
minor imports updates
2026-01-09 13:38:26 +01:00
Timo K
c0913b6546 fix playwright test 2025-11-27 18:02:46 +01:00
Timo K
d22d7460fe Another larger refactor to fix sfu switches and in general proper
cleanup.
2025-11-25 20:18:34 +01:00
Timo K
ddd015d696 fix dev EditInPlace 2025-11-24 10:04:01 +01:00
Timo K
7532f72c90 use edit in place (WIP) This is WIP since it will submite -> reload the
page onSave
2025-11-21 18:49:30 +01:00
Timo K
a731981388 Cleanup transport creation and local store hack. 2025-11-21 14:02:15 +01:00
Robin
cf8298cd69 Fix lints 2025-11-14 18:57:39 -05:00
Robin
494ed7c126 Fix the MatrixRTC mode developer controls
They weren't responding properly to clicks, and make more sense as radio buttons.
2025-11-14 14:15:58 -05:00
Timo K
9f4d954cfa The source of the local participant is the createLocalMembership$ and
not the MatrixLivekitMembers!

Co-authored-by: Valere <bill.carson@valrsoft.com>
2025-11-12 12:09:31 +01:00
Timo K
4d0de2fb71 Refactor Matrix/LiveKit session merging
- Replace MatrixLivekitItem with MatrixLivekitMember, add displayName$
  and participantId, and use explicit LiveKit participant types
- Make sessionBehaviors$ accept a props object and return a typed
  RxRtcSession
- Update CallViewModel to use the new session behaviors, rebuild media
  items from matrixLivekitMembers, handle missing connections and use
  participantId-based keys
- Change localMembership/localTransport to accept Behavior-based
  options, read options.value for enterRTCSession, and fix advertised
  transport selection order
- Update tests and minor UI adjustments (settings modal livekitRooms
  stubbed) and fix JSON formatting in locales
2025-11-05 17:55:36 +01:00
Timo K
107ef16d94 Add MatrixRTCMode and refactor local membership
Remove preferStickyEvents and multiSfu in favor of a MatrixRTCMode
enum/setting (Legacy, Compatibil, Matrix_2_0). Move session join/leave,
track pause/resume, and config error handling out of CallViewModel into
the localMembership module. Update developer settings UI, i18n strings,
and related RTC session helpers and wiring accordingly.
2025-11-05 12:56:58 +01:00
Robin
db2004fb90 Remove unused string 2025-10-22 23:29:36 -04:00
Timo
5526cd74cb Add sticky event support (#3513)
* add sticky event support
 - use new js-sdk
 - use custom synapse
 - don't filter rooms by existing call state events

Signed-off-by: Timo K <toger5@hotmail.de>

* enable sticky events in the joinSessionConfig

Signed-off-by: Timo K <toger5@hotmail.de>

* Remove unused useNewMembershipmanager setting

* Add prefer sticky setting]

* Fixup call detection logic to allow sticky events

* lint

* update docker image

* More tidy

* update checksum

* bump js-sdk fix sticky events type

Signed-off-by: Timo K <toger5@hotmail.de>

* fix demo

Signed-off-by: Timo K <toger5@hotmail.de>

* always use multi sfu if we are using sticky events.

Signed-off-by: Timo K <toger5@hotmail.de>

* review

Signed-off-by: Timo K <toger5@hotmail.de>

* lint

Signed-off-by: Timo K <toger5@hotmail.de>

* Always consider multi-SFU mode enabled when using sticky events

CallViewModel would pass the wrong transport to enterRtcSession when the user enabled sticky events but didn't manually enable multi-SFU mode as well. This likely would've added some confusion to our attempts to test these modes.

* Fix test type errors

* add todo comment

Signed-off-by: Timo K <toger5@hotmail.de>

---------

Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Half-Shot <will@half-shot.uk>
Co-authored-by: Robin <robin@robin.town>
2025-10-22 12:53:22 +02:00
Robin
86fb026be8 Turn multi-SFU media transport into a developer option 2025-10-03 14:43:22 -04:00
Robin
a4a0a58a72 Remove the option to show non-member ("ghost") participants
As we'd like to get the multi-SFU feature branch shipped, this is not the most important debugging tool to expend effort on at the moment.
2025-09-26 13:27:18 -04:00
Robin
f08ae36f9e Pause media tracks and show a message when reconnecting to MatrixRTC 2025-08-15 18:38:52 +02:00
Valere
d294be8bd4 custom error for restricted SFU config error 2025-07-31 17:18:02 +02:00
Timo
5ea0759427 Earpiece follow up: Change labels and icons (#3401)
* fix collapse icon on android

* update wording `earpiece` -> `headset`

* update icon `earpiece` -> `phone`

* i18n

* update icons to solid (top right) and non solid (overlay)

* update snapshots

* add config.json to gitignore

* add offset for earpice overlay

* update snapshots to include offset spacer
2025-07-18 15:19:53 +02:00
Timo
31355ba7af Update earpiece overlay button text for clarity (#3370)
* Update earpiece overlay button text for clarity

* update test snapshots
2025-06-27 15:24:46 +02:00
Robin
30eea63aae Merge pull request #3337 from element-hq/robin/error-fixes
Touch up some error screens
2025-06-26 09:52:35 -04:00
Robin
f509c06cc6 Earpiece switcher and overlay (#3347)
* Add a global control for toggling earpiece mode

This will be used by Element X to show an earpiece toggle button in the header.

* Add an earpiece overlay


* Fix header
The header needs to be passed forward as a string to some components and as a bool (hideHeader) to others.
Also use a enum instead of string options.

* fix top clipping with header


* hide app bar in pip

* revert android overlay app_bar

* Modernize AppBarContext

* Style header icon color as desired and switch earpice/speaker icon

* fix initial selection when using controlled media

* Add "Back to video" button

* fix tests

* remove dead code

* add snapshot test

* fix back to video button

* Request capability to learn the room name

We now need the room name in order to implement the mobile (widget-based) designs with the app bar.

* Test the CallViewModel output switcher directly

---------

Co-authored-by: Timo <toger5@hotmail.de>
2025-06-26 11:08:57 +02:00
Robin
764c6fce24 Fix missing text on "Call not found" screen 2025-06-12 23:54:40 -04:00
Timo
acaf69ca1b add change audio button with callback on ios 2025-05-16 12:28:49 +02:00
Timo
56328108ca Add custom audio renderer to only render joined participants & add ios earpice workaround
fix left right to match chromium + safari
(firefox is swapped)

earpice as setting

Simpler code and documentation
The doc explains, what this class actually does and why it is so complicated.

Signed-off-by: Timo K <toger5@hotmail.de>

use only one audioContext, remove (non working) standby fallback
2025-05-14 10:41:03 +02:00
Timo
04bc6c77a3 developer option to mute all audio (#3239) 2025-05-13 21:11:12 +02:00
Robin
9f5eae0ac1 Clean up a string 2025-04-29 13:02:02 +02:00
Robin
3243afd061 Merge branch 'livekit' into toger5/track-processor-blur 2025-04-18 12:32:03 +02:00
Timo
d5407a4fde Update developer key transport setting description (#3194) 2025-04-14 09:56:53 +02:00
Timo
3c0d81844f Add option to enable to-device-encryption (#3167)
* enable to-device-encryption

* add logging for key provider

* make rooms encrypted

* add dev setting to choose to-device or room encryption

* add indicator when to-device is used.
2025-04-11 10:07:50 +02:00
Timo
75933a508c Merge branch 'livekit' into toger5/track-processor-blur 2025-04-05 00:00:00 +02:00
Timo
2073cd3ac1 Rename EULA to SSLA (#3135)
* Change EULA to SSLA
 - rename i18n fields
 - update the config property to `ssla` and deprecate `eula`
 - use `eula` instead of ssla in case it is provided in the config.

* fix default config

* completely remove eula fallback
2025-03-31 14:18:45 +02:00
Hugh Nimmo-Smith
bfd4bb2ec1 Show URL params and environment variables in developer tab (#3134) 2025-03-27 16:35:31 -04:00
Timo
750db09156 Use the New MatrixRTCSession MembershipManager (#3015)
* provide option to use the New MembershipManager

* fix cryptoApi import change

* add error screen

* bump js-sdk

* rename to `setUnrecoverableError` and remove onLeave call because that will be handled by an effect.

* this was doing nothing (it is a fragment back when there was no deprecated `rtcSession.room`)

* rename to error

* Update src/utils/errors.ts

Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>

* Update src/utils/errors.ts

Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>

* review

* bump js-sdk

* expose lk log level changing in `window`

* bump js-sdk
 - always log "Missing own membership: force re-join"
 - also check insertions queue

* change lk log level to warn

* Bump js-sdk

* Bump js-sdk

* .

* Bump js-sdk

* show user count based on meberships not users.

Signed-off-by: Timo K <toger5@hotmail.de>

* bump js-sdk

* rename setting name

* remove unused import

* js sdk bump

* remove `window.setLKLogLevel`

* bump js sdk with reverted incompatible change

* bump js-sdk with one less merge

---------

Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
2025-03-07 16:27:04 +00:00
Valere
2b35573672 review: Upate error.unexpected_ec_error for consistency 2025-02-28 12:06:31 +01:00
Valere
4701e413fd fixup: MissingRTCFocus error, fix confusion between domain/brand 2025-02-27 10:09:26 +01:00
Valere
b43defedfd fixup: i18n incorrect ordering of keys 2025-02-27 09:32:43 +01:00
Valere
2ba803fcef review: Improve error structure + better RTCFocus error message 2025-02-27 09:26:38 +01:00
Valere
3c23e5a4e6 Error Handling: gracefully handle missing livekit service url in config 2025-02-26 15:19:08 +01:00
Robin
31577d7263 Show an error screen when the SFU is at capacity (#3022)
Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
Co-authored-by: fkwp <fkwp@users.noreply.github.com>
2025-02-26 12:00:56 +00:00
Hugh Nimmo-Smith
294e67a5ba Show current SFU and Server Info in developer tab (#3000)
* Show current SFU and Server Info in developer tab

* Lint
2025-02-10 18:22:19 +01:00
Robin
cda802a2e9 Make error screens more visually consistent (#2951) 2025-01-17 09:35:39 +00:00
Hugh Nimmo-Smith
77ff3583b0 Merge branch 'livekit' into toger5/track-processor-blur 2025-01-06 14:48:54 +00:00
Hugh Nimmo-Smith
2c33d65824 Add developer mode option to show RTC connection statistics (#2904)
* Add developer mode option to show RTC connection statistics

* Add note about localization

* Add titles to help explain what the numbers are

* Workaround horizontal scrolling

* Use modal to show detailed stats instead of alert

* Changed styling and fixed fps = 0 (#2916)

(React rendered 0 instead of <Text /> for fps && <Text>{fps}</text>)

---------

Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
2025-01-06 09:47:39 +00:00
Hugh Nimmo-Smith
ba5da7e9af Inform user that their camera is starting in Lobby (#2869)
* Inform user that their camera is starting

Instead of just showing a grey box.

* Review feedback

* Show spinner from design suggestion

* useMemo

* Lint

* Lint

* Feedback from review

* Use colour that actually exists

* Refactor into Avatar superclass

* .

* Remove size limit behaviour

* Add VideoPreview tests
2024-12-18 15:31:45 +00:00
Hugh Nimmo-Smith
6accf088bf Merge branch 'livekit' into toger5/track-processor-blur 2024-12-18 09:41:38 +00:00
Robin
de276b1fc3 Annotate the default device with a label 2024-12-13 15:22:44 -05:00
Robin
9c57720852 Merge branch 'livekit' into firefox-audio-output 2024-12-13 14:55:27 -05:00
Hugh Nimmo-Smith
a723f10d2c Developer setting to show LiveKit participants that do not have MatrixRTC sessions a.k.a. non-member tiles (#2771)
* make tiles based on rtc member

* display missing lk participant + fix tile multiplier

* add show_non_member_participants config option

* per member tiles

* merge fixes

* linter

* linter and tests

* tests

* adapt tests (wip)

* Remove unused keys

* Fix optionality of nonMemberItemCount

* video is optional

* Mock RTC members

* Lint

* Merge fixes

* Fix user id

* Add explicit types for public fields

* isRTCParticipantAvailable => isLiveKitParticipantAvailable

* isLiveKitParticipantAvailable

* Readonly

* More keys removal

* Make local field based on view model class not observable

* Wording

* Fix RTC members in tes

* Tests again

* Lint

* Disable showing non-member tiles by default

* Duplicate screen sharing tiles like we used to

* Lint

* Revert function reordering

* Remove throttleTime from bad merge

* Cleanup

* Tidy config of show non-member settings

* tidy up handling of local rtc member in tests

* tidy up test init

* Fix mocks

* Cleanup

* Apply local override where participant not yet known

* Handle no visible media id

* Assertions for one-on-one view

* Remove isLiveKitParticipantAvailable and show via encryption status

* Handle no local media (yet)

* Remove unused effect for setting

* Tidy settings

* Avoid case of one-to-one layout with missing local or remote

* Iterate

* Remove option to show non-member tiles to simplify code review

* Remove unused code

* Remove more remnants of show-non-member-tiles

* iterate

* back

* Fix unit test

* Refactor

* Expose TestScheduler as global

* Fix incorrect type assertion

* Simplify speaking observer

* Fix

* Whitespace

* Make it clear that we are mocking MatrixRTC memberships

* Test case for only showing tiles for MatrixRTC session members

* Simplify diff

* Simplify diff

These changes are in https://github.com/element-hq/element-call/pull/2809

* .

* Whitespaces

* Use asObservable when exposing subject

* Show "waiting for media..." when no participant

* Additional test case

* Don't show "waiting for media..." in case of local participant

* Make the loading state more subtle
 - instead of a label we show a animated gradient

* Use correct key for matrix rtc foci in code comment. (#2838)

* Update src/tile/SpotlightTile.tsx

Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>

* Update src/state/CallViewModel.ts

Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>

* Make the purpose of BaseMediaViewModel.local explicit

* Use named object instead of unnamed array for spotlightAndPip

* Refactor spotlightAndPip into spotlight and pip

* Use if statement instead of ternary for readability in spotlight and pip logic

* Review feedback

* Fix tests for CallEventAudioRenderer

* Lint

* Developer setting to show non-member tiles

This is based on top of https://github.com/element-hq/element-call/pull/2701

* Lint

* Remove unused code

* Remove changes that should be in https://github.com/element-hq/element-call/pull/2858

* Update src/state/CallViewModel.test.ts

Co-authored-by: Robin <robin@robin.town>

* Merge branch 'livekit' into toger5/show-non-member-tiles

* Remove unused nonMemberItemCount

* Restore default showNonMemberTiles value after test

* Update comments

---------

Co-authored-by: Timo <toger5@hotmail.de>
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
Co-authored-by: Robin <robin@robin.town>
2024-12-13 14:53:08 +00:00
Hugh Nimmo-Smith
e84ecc68b3 Refactor preferences tab strings for consistency (#2888)
* Refactor preferences tab strings for consistency

* Lint
2024-12-13 13:23:32 +00:00