This hook is simpler in its implementation (therefore hopefully more correct & performant) and enforces a type-level distinction between raw Observables and Behaviors.
* Add a global control for toggling earpiece mode
This will be used by Element X to show an earpiece toggle button in the header.
* Add an earpiece overlay
* Fix header
The header needs to be passed forward as a string to some components and as a bool (hideHeader) to others.
Also use a enum instead of string options.
* fix top clipping with header
* hide app bar in pip
* revert android overlay app_bar
* Modernize AppBarContext
* Style header icon color as desired and switch earpice/speaker icon
* fix initial selection when using controlled media
* Add "Back to video" button
* fix tests
* remove dead code
* add snapshot test
* fix back to video button
* Request capability to learn the room name
We now need the room name in order to implement the mobile (widget-based) designs with the app bar.
* Test the CallViewModel output switcher directly
---------
Co-authored-by: Timo <toger5@hotmail.de>
* Refactor media devices to live outside React as Observables
This moves the media devices state out of React to further our transition to a MVVM architecture in which we can more easily model and store complex application state. I have created an AppViewModel to act as the overarching state holder for any future non-React state we end up creating, and the MediaDevices reside within this. We should move more application logic (including the CallViewModel itself) there in the future.
* Address review feedback
* Fixes from ios debugging session: (#3342)
- dont use preferred vs selected concept in controlled media. Its not needed since we dont use the id for actual browser media devices (the id's are not even actual browser media devices)
- add more logging
- add more conditions to not accidently set a deviceId that is not a browser deviceId but one provided via controlled.
---------
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Use share() on fromEvent() so that we multiplex subscribers onto the event emitter
* .
* .
* Comment
* Comment
---------
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
* Disambigute displaynames
* Add test
* fixup test functions
* prettier
* lint
* Split displayname utils into own file and add tests.
* Split out fixtures
* Add more testcases for displayname calculation.
* lint
* Also listen for displayname changes. (I stand corrected!)
* fix missing media tiles on missing member
* Add developer mode option to show RTC connection statistics
* Add note about localization
* Add titles to help explain what the numbers are
* Workaround horizontal scrolling
* Use modal to show detailed stats instead of alert
* Changed styling and fixed fps = 0 (#2916)
(React rendered 0 instead of <Text /> for fps && <Text>{fps}</text>)
---------
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Add support for using CallViewModel for reactions sounds.
* Drop setting
* Convert reaction sounds to call view model / rxjs
* Use call view model for hand raised reactions
* Support raising reactions for matrix rtc members.
* Tie up last bits of useReactions
* linting
* Update calleventaudiorenderer
* Update reaction audio renderer
* more test bits
* All the test bits and pieces
* More refactors
* Refactor reactions into a sender and receiver.
* Fixup reaction toggle button
* Adapt reactions test
* Tests all pass.
* lint
* fix a couple of bugs
* remove unused helper file
* lint
* finnish notation
* Add tests for useReactionsReader
* remove mistaken vitest file
* fix
* filter
* invert
* fixup tests with fake timers
* Port useReactionsReader hook to ReactionsReader class.
* lint
* exclude some files from coverage
* Add screen share sound effect.
* cancel sub on destroy
* tidy tidy
While debugging our layout shift issue I learned that a single change to the sort order of the participants can cause 3 or 4 redundant emissions of the same items in the same order. Since each of these would cause React to re-render the grid, skipping these spurious emissions seems like an easy performance win.
* make tiles based on rtc member
* display missing lk participant + fix tile multiplier
* add show_non_member_participants config option
* per member tiles
* merge fixes
* linter
* linter and tests
* tests
* adapt tests (wip)
* Remove unused keys
* Fix optionality of nonMemberItemCount
* video is optional
* Mock RTC members
* Lint
* Merge fixes
* Fix user id
* Add explicit types for public fields
* isRTCParticipantAvailable => isLiveKitParticipantAvailable
* isLiveKitParticipantAvailable
* Readonly
* More keys removal
* Make local field based on view model class not observable
* Wording
* Fix RTC members in tes
* Tests again
* Lint
* Disable showing non-member tiles by default
* Duplicate screen sharing tiles like we used to
* Lint
* Revert function reordering
* Remove throttleTime from bad merge
* Cleanup
* Tidy config of show non-member settings
* tidy up handling of local rtc member in tests
* tidy up test init
* Fix mocks
* Cleanup
* Apply local override where participant not yet known
* Handle no visible media id
* Assertions for one-on-one view
* Remove isLiveKitParticipantAvailable and show via encryption status
* Handle no local media (yet)
* Remove unused effect for setting
* Tidy settings
* Avoid case of one-to-one layout with missing local or remote
* Iterate
* Remove option to show non-member tiles to simplify code review
* Remove unused code
* Remove more remnants of show-non-member-tiles
* iterate
* back
* Fix unit test
* Refactor
* Expose TestScheduler as global
* Fix incorrect type assertion
* Simplify speaking observer
* Fix
* Whitespace
* Make it clear that we are mocking MatrixRTC memberships
* Test case for only showing tiles for MatrixRTC session members
* Simplify diff
* Simplify diff
These changes are in https://github.com/element-hq/element-call/pull/2809
* .
* Whitespaces
* Use asObservable when exposing subject
* Show "waiting for media..." when no participant
* Additional test case
* Don't show "waiting for media..." in case of local participant
* Make the loading state more subtle
- instead of a label we show a animated gradient
* Use correct key for matrix rtc foci in code comment. (#2838)
* Update src/tile/SpotlightTile.tsx
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Update src/state/CallViewModel.ts
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Make the purpose of BaseMediaViewModel.local explicit
* Use named object instead of unnamed array for spotlightAndPip
* Refactor spotlightAndPip into spotlight and pip
* Use if statement instead of ternary for readability in spotlight and pip logic
* Review feedback
* Fix tests for CallEventAudioRenderer
* Lint
* Developer setting to show non-member tiles
This is based on top of https://github.com/element-hq/element-call/pull/2701
* Lint
* Remove unused code
* Remove changes that should be in https://github.com/element-hq/element-call/pull/2858
* Update src/state/CallViewModel.test.ts
Co-authored-by: Robin <robin@robin.town>
* Merge branch 'livekit' into toger5/show-non-member-tiles
* Remove unused nonMemberItemCount
* Restore default showNonMemberTiles value after test
* Update comments
---------
Co-authored-by: Timo <toger5@hotmail.de>
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
Co-authored-by: Robin <robin@robin.town>
Instead of tracking for each individual tile whether it's visible, just track the total number of tiles that appear on screen. This ought to make the whole thing a lot less dynamic, which is crucial given that our UI renders asynchronously and RxJS doesn't really support cyclic dependencies in any rigorous way.
In particular this ought to make the following kind of situation impossible:
1. There 3 tiles, ABC. A and B are on screen.
2. Now C becomes important. The requested order is now CAB.
3. To reduce the size of the layout shift, the algorithm selects to swap just B and C in the original order, giving ACB. However, the UI is blocked and doesn't render this order yet.
4. For whatever reason, a spurious update of the importance algorithm occurs. It once again requests CAB.
5. Now because the UI was blocked, the layout still thinks that A and B are on screen (rather than A and C). It thinks that C is some weird island of "off-screen territory" in the middle of the tile order. This confuses it into swapping A and C rather than keeping the layout stable.
The reality is that whenever we think N tiles are visible on screen, we're always referring to the first N tiles in the grid. It's best if the code reflects this assumption.
* Add some quick-and-dirty debug info for TileStore
I'm still in need of more detailed data in order to understand why big layout shifts happen in large calls. This adds a developer option to enable logging and a visual indicator for the state of the TileStore. The indicator should be useful for matching up the behavior I'm seeing in my recordings with the right timestamps.
* Reduce performance impact of checking for whether debug mode is enabled
---------
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
* Enable @typescript-eslint/consistent-type-imports lint rule
This is to help ensure that we get proper vite/rollup lazy loading by not `import`ing more than we need to.
Revert "Enable @typescript-eslint/consistent-type-imports lint rule"
This reverts commit ba385fa00b7e410cc508fd5fb9fe972233ae114f.
Enable @typescript-eslint/consistent-type-imports lint rule
This is to help ensure that we get proper vite/rollup lazy loading by not `import`ing more than we need to.
.
* Format
* Refactor to use AudioContext
* Remove unused audio format.
* Reduce update frequency for volume
* Port to useAudioContext
* Port reactionaudiorenderer to useAudioContext
* Integrate raise hand sound into call event renderer.
* Simplify reaction sounds
* only play one sound per reaction type
* Start to build out tests
* fixup tests / comments
* Fix reaction sound
* remove console line
* Remove another debug line.
* fix lint
* Use testing library click
* lint
* fix a few things
* Change the way we as unknown the mock RTC session.
* Lint
* Fix types for MockRTCSession
* value change should always be set
* Update volume slider description.
* Only load reaction sound effects if enabled.
* cache improvements
* lowercase soundMap
* lint
* move prefetch sounds to fix hot reload
* correct docs
* add a header
* Wording change
---------
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
* make tiles based on rtc member
* display missing lk participant + fix tile multiplier
* add show_non_member_participants config option
* per member tiles
* merge fixes
* linter
* linter and tests
* tests
* adapt tests (wip)
* Remove unused keys
* Fix optionality of nonMemberItemCount
* video is optional
* Mock RTC members
* Lint
* Merge fixes
* Fix user id
* Add explicit types for public fields
* isRTCParticipantAvailable => isLiveKitParticipantAvailable
* isLiveKitParticipantAvailable
* Readonly
* More keys removal
* Make local field based on view model class not observable
* Wording
* Fix RTC members in tes
* Tests again
* Lint
* Disable showing non-member tiles by default
* Duplicate screen sharing tiles like we used to
* Lint
* Revert function reordering
* Remove throttleTime from bad merge
* Cleanup
* Tidy config of show non-member settings
* tidy up handling of local rtc member in tests
* tidy up test init
* Fix mocks
* Cleanup
* Apply local override where participant not yet known
* Handle no visible media id
* Assertions for one-on-one view
* Remove isLiveKitParticipantAvailable and show via encryption status
* Handle no local media (yet)
* Remove unused effect for setting
* Tidy settings
* Avoid case of one-to-one layout with missing local or remote
* Iterate
* Remove option to show non-member tiles to simplify code review
* Remove unused code
* Remove more remnants of show-non-member-tiles
* iterate
* back
* Fix unit test
* Refactor
* Expose TestScheduler as global
* Fix incorrect type assertion
* Simplify speaking observer
* Fix
* Whitespace
* Make it clear that we are mocking MatrixRTC memberships
* Test case for only showing tiles for MatrixRTC session members
* Simplify diff
* Simplify diff
These changes are in https://github.com/element-hq/element-call/pull/2809
* .
* Whitespaces
* Use asObservable when exposing subject
* Show "waiting for media..." when no participant
* Additional test case
* Don't show "waiting for media..." in case of local participant
* Make the loading state more subtle
- instead of a label we show a animated gradient
* Use correct key for matrix rtc foci in code comment. (#2838)
* Update src/tile/SpotlightTile.tsx
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Update src/state/CallViewModel.ts
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Make the purpose of BaseMediaViewModel.local explicit
* Use named object instead of unnamed array for spotlightAndPip
* Refactor spotlightAndPip into spotlight and pip
* Use if statement instead of ternary for readability in spotlight and pip logic
* Review feedback
* Fix tests for CallEventAudioRenderer
* Lint
* Revert "Make the loading state more subtle"
This reverts commit 765f7b4f31.
* Update src/state/CallViewModel.ts
Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
* Fix spelling
* Remove a non-null assertion that failed at runtime
---------
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
* Add renderer for call joined / left
* lint
* Add new sounds
* Updates sounds in renderer
* lint
* move import
* pad sounds with silence
* lint
* tidy
* Drop autoplay since we now subscribe correctly.
* Comitting test files I am going to be going to lunch so will tidy up in a little while.
* finish up tests
* Add support for multiple channels per sound.
* lint
* Refactor the speaker detection logic into observeSpeaker and add tests
@robintown the tests pass, but some of the values were off by 1ms from what I was expecting. Please can you sanity check them?
* Extra test cases and clean up
* Make distinctUntilChanged part of the observable itself
* More suggestions from code review
* Use hot marbles for speaker tests
This was originally part of https://github.com/element-hq/element-call/pull/2810
* Only feed speaking mocks to observables that ask for IsSpeakingChanged
* Refactor the speaker detection logic into observeSpeaker and add tests
@robintown the tests pass, but some of the values were off by 1ms from what I was expecting. Please can you sanity check them?
* Extra test cases and clean up
* Make distinctUntilChanged part of the observable itself
* More suggestions from code review
We have Matrix room members and MatrixRTC session memberships. Livekit also has rooms.
So, this attempts to make it more obvious as to what type you are referring to.
