once)
The local jwt token needs to be aquired via the right endpoint. The
endpoint defines how our rtcBackendIdentity is computed. Based on us
using sticky events or state events we also need to use the right
endpoint. This cannot be done generically in the connection manager. The
jwt token now is computed in the localTransport and the resolved sfu
config is passed to the connection manager.
Add JWT endpoint version and SFU config support Pin matrix-js-sdk to a
specific commit and update dev auth image tag. Propagate SFU config and
JWT endpoint choice through local transport, ConnectionManager and
Connection; add JwtEndpointVersion enum and LocalTransportWithSFUConfig
type. Add NO_MATRIX_2 auth error and locale string, thread
rtcBackendIdentity through UI props, and include related test, CSS and
minor imports updates
On second glance, the way that we determined a media tile to be 'waiting for media' was too implicit for my taste. It would appear on a surface reading to depend on whether a participant was currently publishing any video. But in reality, the 'video' object was always defined as long as a LiveKit participant existed, so in reality it depended on just the participant. We should show this relationship more explicitly by moving the computation into the view model, where it can depend on the participant directly.
Previously we had a ViewModel class which was responsible for little more than creating an ObservableScope. However, since this ObservableScope would be created implicitly upon view model construction, it became a tad bit harder for callers to remember to eventually end the scope (as you wouldn't just have to remember to end ObservableScopes, but also to destroy ViewModels). Requiring the scope to be specified explicitly by the caller also makes it possible for the caller to reuse the scope for other purposes, reducing the number of scopes mentally in flight that need tending to, and for all state holders (not just view models) to be handled uniformly by helper functions such as generateKeyed$.
This hook is simpler in its implementation (therefore hopefully more correct & performant) and enforces a type-level distinction between raw Observables and Behaviors.
* Refactor media devices to live outside React as Observables
This moves the media devices state out of React to further our transition to a MVVM architecture in which we can more easily model and store complex application state. I have created an AppViewModel to act as the overarching state holder for any future non-React state we end up creating, and the MediaDevices reside within this. We should move more application logic (including the CallViewModel itself) there in the future.
* Address review feedback
* Fixes from ios debugging session: (#3342)
- dont use preferred vs selected concept in controlled media. Its not needed since we dont use the id for actual browser media devices (the id's are not even actual browser media devices)
- add more logging
- add more conditions to not accidently set a deviceId that is not a browser deviceId but one provided via controlled.
---------
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Disambigute displaynames
* Add test
* fixup test functions
* prettier
* lint
* Split displayname utils into own file and add tests.
* Split out fixtures
* Add more testcases for displayname calculation.
* lint
* Also listen for displayname changes. (I stand corrected!)
* fix missing media tiles on missing member
* Add developer mode option to show RTC connection statistics
* Add note about localization
* Add titles to help explain what the numbers are
* Workaround horizontal scrolling
* Use modal to show detailed stats instead of alert
* Changed styling and fixed fps = 0 (#2916)
(React rendered 0 instead of <Text /> for fps && <Text>{fps}</text>)
---------
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Add support for using CallViewModel for reactions sounds.
* Drop setting
* Convert reaction sounds to call view model / rxjs
* Use call view model for hand raised reactions
* Support raising reactions for matrix rtc members.
* Tie up last bits of useReactions
* linting
* Update calleventaudiorenderer
* Update reaction audio renderer
* more test bits
* All the test bits and pieces
* More refactors
* Refactor reactions into a sender and receiver.
* Fixup reaction toggle button
* Adapt reactions test
* Tests all pass.
* lint
* fix a couple of bugs
* remove unused helper file
* lint
* finnish notation
* Add tests for useReactionsReader
* remove mistaken vitest file
* fix
* filter
* invert
* fixup tests with fake timers
* Port useReactionsReader hook to ReactionsReader class.
* lint
* exclude some files from coverage
* Add screen share sound effect.
* cancel sub on destroy
* tidy tidy
* Enable @typescript-eslint/consistent-type-imports lint rule
This is to help ensure that we get proper vite/rollup lazy loading by not `import`ing more than we need to.
Revert "Enable @typescript-eslint/consistent-type-imports lint rule"
This reverts commit ba385fa00b7e410cc508fd5fb9fe972233ae114f.
Enable @typescript-eslint/consistent-type-imports lint rule
This is to help ensure that we get proper vite/rollup lazy loading by not `import`ing more than we need to.
.
* Format
* make tiles based on rtc member
* display missing lk participant + fix tile multiplier
* add show_non_member_participants config option
* per member tiles
* merge fixes
* linter
* linter and tests
* tests
* adapt tests (wip)
* Remove unused keys
* Fix optionality of nonMemberItemCount
* video is optional
* Mock RTC members
* Lint
* Merge fixes
* Fix user id
* Add explicit types for public fields
* isRTCParticipantAvailable => isLiveKitParticipantAvailable
* isLiveKitParticipantAvailable
* Readonly
* More keys removal
* Make local field based on view model class not observable
* Wording
* Fix RTC members in tes
* Tests again
* Lint
* Disable showing non-member tiles by default
* Duplicate screen sharing tiles like we used to
* Lint
* Revert function reordering
* Remove throttleTime from bad merge
* Cleanup
* Tidy config of show non-member settings
* tidy up handling of local rtc member in tests
* tidy up test init
* Fix mocks
* Cleanup
* Apply local override where participant not yet known
* Handle no visible media id
* Assertions for one-on-one view
* Remove isLiveKitParticipantAvailable and show via encryption status
* Handle no local media (yet)
* Remove unused effect for setting
* Tidy settings
* Avoid case of one-to-one layout with missing local or remote
* Iterate
* Remove option to show non-member tiles to simplify code review
* Remove unused code
* Remove more remnants of show-non-member-tiles
* iterate
* back
* Fix unit test
* Refactor
* Expose TestScheduler as global
* Fix incorrect type assertion
* Simplify speaking observer
* Fix
* Whitespace
* Make it clear that we are mocking MatrixRTC memberships
* Test case for only showing tiles for MatrixRTC session members
* Simplify diff
* Simplify diff
These changes are in https://github.com/element-hq/element-call/pull/2809
* .
* Whitespaces
* Use asObservable when exposing subject
* Show "waiting for media..." when no participant
* Additional test case
* Don't show "waiting for media..." in case of local participant
* Make the loading state more subtle
- instead of a label we show a animated gradient
* Use correct key for matrix rtc foci in code comment. (#2838)
* Update src/tile/SpotlightTile.tsx
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Update src/state/CallViewModel.ts
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Make the purpose of BaseMediaViewModel.local explicit
* Use named object instead of unnamed array for spotlightAndPip
* Refactor spotlightAndPip into spotlight and pip
* Use if statement instead of ternary for readability in spotlight and pip logic
* Review feedback
* Fix tests for CallEventAudioRenderer
* Lint
* Revert "Make the loading state more subtle"
This reverts commit 765f7b4f31.
* Update src/state/CallViewModel.ts
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Fix spelling
* Remove a non-null assertion that failed at runtime
---------
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
* Refactor to make encryption system available in view models
* WIP show encryption errors from LiveKit
* Missing CSS
* Show encryption status based on LK and RTC
* Lint
* Lint
* Fix tests
* Update wording
* Refactor
* Lint
Previously, dragging it all the way to the left would *not* mute the participant but rather bottom out at 10% volume, and people have found this unintuitive. Let's make it less silly by giving the slider a range of 0% to 100%, and making the mute toggle button have the same effect as dragging the slider to zero. When unmuting, it will reset to the last non-zero "committed" volume, similar to how the volume sliders in desktop environments work.
* Stop sharing state observables when the view model is destroyed
By default, observables running with shareReplay will continue running forever even if there are no subscribers. We need to stop them when the view model is destroyed to avoid memory leaks and other unintuitive behavior.
* Hydrate the call view model in a less hacky way
This ensures that only a single view model is created per call, unlike the previous solution which would create extra view models in strict mode which it was unable to dispose of. The other way was invalid because React gives us no way to reliably dispose of a resource created in the render phase. This is essentially a memory leak fix.
* Add simple global controls to put the call in picture-in-picture mode
Our web and mobile apps (will) all support putting calls into a picture-in-picture mode. However, it'd be nice to have a way of doing this that's more explicit than a breakpoint, because PiP views could in theory get fairly large. Specifically, on mobile, we want a way to do this that can tell you whether the call is ongoing, and that works even without the widget API (because we support SPA calls in the Element X apps…)
To this end, I've created a simple global "controls" API on the window. Right now it only has methods for controlling the picture-in-picture state, but in theory we can expand it to also control mute states, which is current possible via the widget API only.
* Fix footer appearing in large PiP views
* Add a method for whether you can enter picture-in-picture mode
* Have the controls emit booleans directly
react-rxjs is the library we've been using to connect our React components to view models and consume observables. However, after spending some time with react-rxjs, I feel that it's a very heavy-handed solution. It requires us to sprinkle <Subscribe /> and <RemoveSubscribe /> components all throughout the code, and makes React go through an extra render cycle whenever we mount a component that binds to a view model. What I really want is a lightweight React hook that just gets the current value out of a plain observable, without any extra setup. Luckily the observable-hooks library with its useObservableEagerState hook seems to do just that—and it's more actively maintained, too!
Here I've implemented an MVP for the new unified grid layout, which scales smoothly up to arbitrarily many participants. It doesn't yet have a special 1:1 layout, so in spotlight mode and 1:1s, we will still fall back to the legacy grid systems.
Things that happened along the way:
- The part of VideoTile that is common to both spotlight and grid tiles, I refactored into MediaView
- VideoTile renamed to GridTile
- Added SpotlightTile for the new, glassy spotlight designs
- NewVideoGrid renamed to Grid, and refactored to be even more generic
- I extracted the media name logic into a custom React hook
- Deleted the BigGrid experiment
This is a start at implementing the call layouts from the new designs. I've added data types to model the contents of each possible layout, and begun implementing the business logic to produce these layouts in the call view model.
