Compare commits

..
Author SHA1 Message Date
Valere be7407ea3d review: quick renaming 2025-12-04 09:37:07 +01:00
Valere fdc66a1d62 fix: existing screenshare switching twice 2025-12-03 18:36:51 +01:00
Valere 6e7bedbaec Merge branch 'livekit' into valere/layout_switch 2025-12-03 09:17:31 +01:00
Valere a93ceeae4b review: Keep previous behavior for now to always auto switch 2025-12-03 09:01:26 +01:00
Valere b85f36598c fix: mistake in file name 2025-12-03 08:54:52 +01:00
Valere Fedronic e0bf51b63b Merge pull request #3590 from element-hq/valere/noise_cancellation
Config: UrlParams to control noiseSuppression and echoCancellation
2025-12-02 18:36:32 +01:00
Valere f6a3a371cb fix lint 2025-12-02 17:52:27 +01:00
Valere ac9acc0158 review: refactor convert params to object for generateRoomOption 2025-12-02 17:47:00 +01:00
Valere 91f3b8b114 Merge branch 'livekit' into valere/noise_cancellation 2025-12-02 17:44:06 +01:00
Valere be0c7eb365 review: fix mock import module 2025-12-02 17:43:58 +01:00
Valere b5d3f3c72a fix: Allow switching back to grid when there is a screenshare 2025-12-02 11:18:50 +01:00
Valere 60bc6f1e93 refactor: Extract layout mode switch + test 2025-12-02 11:00:32 +01:00
Timo f05d4b158e Merge pull request #3565 from element-hq/renovate/matrix-widget-api
Update dependency matrix-widget-api to v1.15.0
2025-12-01 19:14:52 +01:00
Robin 0240087afe Unpin matrix-js-sdk so it can be upgraded 2025-12-01 13:09:10 -05:00
Timo K 4ec1b19fe2 fix js-sdk tests 2025-12-01 18:54:57 +01:00
Timo K 5f6a45f0e9 also bump js-sdk 2025-12-01 18:45:47 +01:00
renovate[bot] e2916d9093 Update dependency matrix-widget-api to v1.14.0 2025-12-01 18:23:35 +01:00
Timo 90d4b6a6ce Merge pull request #3598 from element-hq/renovate/github-actions
Update GitHub Actions
2025-12-01 17:42:05 +01:00
Timo e3eb2ea12b Merge pull request #3599 from element-hq/renovate/embedded-package-dependencies
Update embedded package dependencies
2025-12-01 17:40:29 +01:00
Timo 4f45825cf2 Merge pull request #3591 from element-hq/toger5/fix-local-transport-update
Fix reactivity based on localTransport updates (Cleanup for LocalMembership.ts)
2025-12-01 17:39:55 +01:00
Timo K 8cb8357398 make logs just debug. (info -> debug) 2025-12-01 17:34:37 +01:00
Timo K 63cd4f79dd fix playwright test! (It caught an actual bug!!! so the right wording
would be: fix implementation thanks to the playwright test!
2025-12-01 17:29:35 +01:00
Valere Fedronic 7fd4424e7b Merge pull request #3601 from element-hq/valere/fix_switch_spotlight_screenshare
fix: Automatically switch to spotlight if remote screenshare
2025-12-01 15:30:24 +01:00
Valere b7fc86ee06 review: move setGridMode close to the value it sets 2025-12-01 15:25:17 +01:00
Timo K f26aa8f970 fix tests 2025-12-01 15:10:38 +01:00
Valere 1ff479a315 fix: Automatically switch to spotlight if remote screenshare 2025-12-01 15:06:17 +01:00
Timo K 47c6a17d1e review 2025-12-01 14:58:06 +01:00
Timo K 24ed43ce13 Merge branch 'livekit' into toger5/fix-local-transport-update 2025-12-01 14:23:11 +01:00
renovate[bot] 04bbf83ac5 Update GitHub Actions 2025-12-01 11:00:52 +00:00
renovate[bot] 5aa82295fd Update embedded package dependencies 2025-12-01 01:28:55 +00:00
Timo 3331af1108 Merge pull request #3592 from element-hq/valere/fix_flaky_test
test: use `setSystemTime` for better test stability
2025-11-28 22:00:24 +01:00
Timo K 66dece98a5 add more test for publisher 2025-11-28 21:50:22 +01:00
Timo K 2011aef116 skip "Should show error screen if call creation is restricted" on ff 2025-11-28 18:00:04 +01:00
Valere Fedronic f5c422c559 Merge pull request #3597 from element-hq/valere/test_reconcile
test: Add simple tests for scope.reconcile
2025-11-28 10:00:06 +01:00
Valere c6bfda94cf test: additional reconcile test 2025-11-28 09:53:18 +01:00
Valere cdd391b3a2 test: Add simple tests for scope.reconcile 2025-11-28 09:12:05 +01:00
Valere Fedronic 5bc7b31dcd Merge pull request #3593 from element-hq/valere/quality_gate
test: Enable quality gate and touch a file
2025-11-27 18:28:30 +01:00
Timo K c0913b6546 fix playwright test 2025-11-27 18:02:46 +01:00
Timo K 46f8fe4ec7 fix test errors 2025-11-27 16:43:28 +01:00
Valere c00e332dd8 Revert "vitest: Add threshold on vitest to check locally"
This reverts commit e5509ba49b.
2025-11-27 16:43:12 +01:00
Valere e5509ba49b vitest: Add threshold on vitest to check locally 2025-11-27 16:39:45 +01:00
Valere 4c2c7d51cf revert test changes on ObservableScope 2025-11-27 16:38:51 +01:00
Timo K 4b0f6e76c4 revert complete behavior check 2025-11-27 15:38:31 +01:00
Timo K e5117f705d More testing and cleaning up 2025-11-27 14:42:23 +01:00
Valere 794585514a Revert "add untested function"
This reverts commit c078dd87fa.
2025-11-27 10:04:09 +01:00
Valere c078dd87fa add untested function 2025-11-26 19:28:44 +01:00
Valere 2b3e2f6188 test: Enable quality gate and touch a file 2025-11-26 19:13:07 +01:00
Valere acd5dde6d8 test: use setSystemTime for better test stability 2025-11-26 16:27:54 +01:00
Timo K d22d7460fe Another larger refactor to fix sfu switches and in general proper
cleanup.
2025-11-25 20:18:34 +01:00
Valere 7f3596845c fix formatting 2025-11-25 11:40:38 +01:00
Valere e2607d6399 Config: UrlParams to control noiseSuppression and echoCancellation 2025-11-25 11:10:53 +01:00
Timo 3491a68362 Merge pull request #3576 from element-hq/toger5/fix-skipped-tests
Refactor local transport testing and local membership initialization
2025-11-24 11:36:27 +01:00
Valere 294b90b6dc fix fixture following web change 2025-11-24 11:29:03 +01:00
Timo K ddd015d696 fix dev EditInPlace 2025-11-24 10:04:01 +01:00
Timo K 7532f72c90 use edit in place (WIP) This is WIP since it will submite -> reload the
page onSave
2025-11-21 18:49:30 +01:00
Timo K e60bc9e98f add developer tab snapshot test 2025-11-21 17:44:35 +01:00
Timo 1aa1af2ade Merge pull request #3585 from element-hq/toger5/add-log-screenshare
Add log to screenshare
2025-11-21 16:58:04 +01:00
Timo K ccadc17406 Add log to screenshare. 2025-11-21 16:19:19 +01:00
Timo K a731981388 Cleanup transport creation and local store hack. 2025-11-21 14:02:15 +01:00
Timo K 4099c4383d move HomeserverConnected 2025-11-21 13:04:28 +01:00
Timo K d34775fc47 fix linter 2025-11-20 16:40:06 +01:00
Timo K a29f0162ad remove widget log 2025-11-20 16:34:53 +01:00
Timo K a69a50d30c add log to error boudnary 2025-11-20 16:32:40 +01:00
Timo K 89a1bfac2d remove unused test helper 2025-11-20 15:25:53 +01:00
Timo K 1fd9ac93c9 refactor local transport testing and local memberhsip initialization 2025-11-20 15:19:00 +01:00
Timo f6ef872242 Merge pull request #3575 from element-hq/hs/lint-src-imports
Restrict importing src/lib in most cases for matrix- libs
2025-11-20 14:55:41 +01:00
Half-Shot 07de8483d5 Restrict importing src/lib in most cases for matrix- libs 2025-11-20 13:32:09 +00:00
Valere Fedronic 42b4c8301f Merge pull request #3547 from element-hq/valere/fix_blank_widget_auto_leave
fix: Send close widget action on auto-leave
2025-11-20 11:41:43 +01:00
Valere c2f541ff8c split test in 2 one testing sound and one to just leave 2025-11-20 11:28:24 +01:00
Valere c1049b6088 fix import causing lint errors in node_modules 2025-11-20 10:46:24 +01:00
Valere 6577db16ce Merge branch 'livekit' into valere/fix_blank_widget_auto_leave 2025-11-20 10:41:31 +01:00
renovate[bot] cdc3c2b216 Update LiveKit components (#3573)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-11-19 20:18:18 +01:00
renovate[bot] 47324b52f8 Update dependency livekit-client to v2.16.0 (#3574)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-11-19 20:14:37 +01:00
alutarius 82706cca4f Correct Caddy reverse proxy config example (#3572) 2025-11-18 18:46:00 +01:00
Timo 665a6be484 Merge pull request #3560 from element-hq/voip-team/call-viewmodel-refactor
`CallViewmodel` refactor
2025-11-18 13:48:03 +01:00
Valere 0076b54e52 minor comments/TODO cleaning 2025-11-18 12:14:17 +01:00
Valere 614762b5f8 fixup headers 2025-11-18 10:13:10 +01:00
Timo 222982c52c Merge pull request #3571 from element-hq/toger5/createCallViewModelFunction$
`createCallViewModel$` function refactor
2025-11-17 19:46:06 +01:00
Timo K 2a19f06e4a reword docstring 2025-11-17 19:45:14 +01:00
Robin 171c289f2d Merge pull request #3564 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.15.15
2025-11-17 13:41:44 -05:00
Timo K 2e2c799f72 convert CallViewModel into create function pattern. (with much more
minimal changes thanks to the intermediate class refactor)
2025-11-17 18:24:47 +01:00
Valere FedronicandTimo K 16e1c59e11 playwright: Fix error boundary mgmt or openId errors (#3570)
* playwright: Fix error boundary mgmt or openId errors

* do not use tap for important logic

* fix lint

---------

Co-authored-by: Timo K <toger5@hotmail.de>
2025-11-17 18:22:37 +01:00
Timo b51df36a06 Move one-on-one layout into CallViewModel (#3567)
* move ononone layout into CallViewModel

* move even more into the vm.

* tests
2025-11-17 17:42:37 +01:00
Timo 49ec4b4298 Merge pull request #3568 from element-hq/toger5/createCallViewModel$
refactor to createCallViewModel$
2025-11-17 15:04:18 +01:00
renovate[bot] 9ad029d4ff Update dependency livekit-client to v2.15.15 2025-11-17 02:42:33 +00:00
Ben Banfield-Zanin a3950869fb Use the dedicated bug report sub-domain (#3557)
* Update docs for 'rageshake.submit_url' to use the dedicated subdomain

* Update Netlify preview deployment to use the dedicated subdomain for 'rageshake.submit_url'
2025-11-13 15:33:59 +00:00
Robin 5afc3c2598 Merge pull request #3556 from element-hq/renovate/node-24.x
Update Node.js to v24
2025-11-05 13:55:24 -05:00
Robin 8ad0cb1955 Merge pull request #3551 from element-hq/renovate/embedded-package-dependencies
Update embedded package dependencies to v8.13.0
2025-11-05 13:53:27 -05:00
Robin 9bd464a7f1 Merge pull request #3550 from element-hq/renovate/github-actions
Update softprops/action-gh-release digest to 6da8fa9
2025-11-05 13:53:08 -05:00
Robin f0adae41ef Merge pull request #3552 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.15.14
2025-11-05 13:49:44 -05:00
renovate[bot] f9cc930155 Update Node.js to v24 2025-11-05 18:49:21 +00:00
Robin eebcd133f3 Merge pull request #3553 from element-hq/robin/trusted-publishing
Remove use of npm access token now that trusted publishing is set up
2025-11-05 13:45:52 -05:00
Robin 79bd458dc2 Remove use of npm access token now that trusted publishing is set up
npm has recently limited the lifetime of all access tokens to 90 days (https://gh.io/npm-token-changes), so it would be a bit inconvenient to stick to our current access token-based method of publishing releases. Meanwhile npm has implemented a more secure publishing method based on OIDC in which you tell the registry that a particular GitHub Actions workflow should be a "trusted publisher" for a given package, and then the CLI will authenticate automatically. (https://docs.npmjs.com/trusted-publishers)

I've already set trusted publishing up on the registry side, and since we're already granting the job permission to generate ID tokens for provenance, there should be no additional lines of config needed to make it work. Let's take away the access token and see how this goes next time we release.
2025-11-03 17:13:03 -05:00
renovate[bot] a4b1ce52cd Update dependency livekit-client to v2.15.14 2025-11-03 01:02:47 +00:00
renovate[bot] b22a937e7c Update softprops/action-gh-release digest to 6da8fa9 2025-11-01 02:02:21 +00:00
Valere 0309ceb6aa fix lint 2025-10-27 15:59:01 +01:00
Valere 3ecffccd9e fix prettier 2025-10-27 15:41:16 +01:00
Valere 3dd0b156cf fix: Send close widget action on auto-leave 2025-10-27 14:31:23 +01:00
renovate[bot] 214bb75d27 Update embedded package dependencies to v8.13.0 2025-10-01 02:29:37 +00:00
66 changed files with 4467 additions and 2006 deletions
+23
View File
@@ -43,6 +43,29 @@ module.exports = {
// To encourage good usage of RxJS: // To encourage good usage of RxJS:
"rxjs/no-exposed-subjects": "error", "rxjs/no-exposed-subjects": "error",
"rxjs/finnish": ["error", { names: { "^this$": false } }], "rxjs/finnish": ["error", { names: { "^this$": false } }],
"no-restricted-imports": [
"error",
{
paths: ["matrix-widget-api", "matrix-js-sdk"].flatMap((lib) =>
["src", "src/", "src/index", "lib", "lib/", "lib/index"]
.map((path) => `${lib}/${path}`)
.map((name) => ({ name, message: `Please use ${lib} instead` })),
),
patterns: [
...["matrix-widget-api"].map((lib) => ({
group: ["src", "src/", "src/**", "lib", "lib/", "lib/**"].map(
(path) => `${lib}/${path}`,
),
message: `Please use ${lib} instead`,
})),
// XXX: We use /lib in lots of places, so allow for now.
...["matrix-js-sdk"].map((lib) => ({
group: ["src", "src/", "src/**"].map((path) => `${lib}/${path}`),
message: `Please use ${lib} instead`,
})),
],
},
],
}, },
overrides: [ overrides: [
{ {
@@ -23,7 +23,7 @@ jobs:
packages: write packages: write
steps: steps:
- name: Check it out - name: Check it out
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: 📥 Download artifact - name: 📥 Download artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -42,7 +42,7 @@ jobs:
- name: Extract metadata (tags, labels) for Docker - name: Extract metadata (tags, labels) for Docker
id: meta id: meta
uses: docker/metadata-action@c1e51972afc2121e065aed6d45c65596fe445f3f # v5.8.0 uses: docker/metadata-action@318604b99e75e41977312d83839a89be02ca4893 # v5.9.0
with: with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: ${{ inputs.docker_tags}} tags: ${{ inputs.docker_tags}}
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack - name: Enable Corepack
run: corepack enable run: corepack enable
- name: Yarn cache - name: Yarn cache
+1 -1
View File
@@ -7,7 +7,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack - name: Enable Corepack
run: corepack enable run: corepack enable
- name: Yarn cache - name: Yarn cache
@@ -85,7 +85,7 @@ jobs:
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256 run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
- name: Upload - name: Upload
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }} if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2 uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with: with:
files: | files: |
${{ env.FILENAME_PREFIX }}.tar.gz ${{ env.FILENAME_PREFIX }}.tar.gz
@@ -100,10 +100,10 @@ jobs:
ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }} ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
permissions: permissions:
contents: read contents: read
id-token: write # required for the provenance flag on npm publish id-token: write # Allow npm to authenticate as a trusted publisher
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: 📥 Download built element-call artifact - name: 📥 Download built element-call artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -126,8 +126,6 @@ jobs:
npm version ${{ needs.versioning.outputs.PREFIXED_VERSION }} --no-git-tag-version npm version ${{ needs.versioning.outputs.PREFIXED_VERSION }} --no-git-tag-version
echo "ARTIFACT_VERSION=$(jq '.version' --raw-output package.json)" >> "$GITHUB_ENV" echo "ARTIFACT_VERSION=$(jq '.version' --raw-output package.json)" >> "$GITHUB_ENV"
npm publish --provenance --access public --tag ${{ needs.versioning.outputs.TAG }} ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }} npm publish --provenance --access public --tag ${{ needs.versioning.outputs.TAG }} ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_RELEASE_TOKEN }}
- id: artifact_version - id: artifact_version
name: Output artifact version name: Output artifact version
@@ -144,7 +142,7 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: 📥 Download built element-call artifact - name: 📥 Download built element-call artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -199,7 +197,7 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with: with:
path: element-call path: element-call
@@ -212,7 +210,7 @@ jobs:
path: element-call/embedded/ios/Sources/dist path: element-call/embedded/ios/Sources/dist
- name: Checkout element-call-swift - name: Checkout element-call-swift
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with: with:
repository: element-hq/element-call-swift repository: element-hq/element-call-swift
path: element-call-swift path: element-call-swift
@@ -264,7 +262,7 @@ jobs:
echo "iOS: ${{ needs.publish_ios.outputs.ARTIFACT_VERSION }}" echo "iOS: ${{ needs.publish_ios.outputs.ARTIFACT_VERSION }}"
- name: Add release notes - name: Add release notes
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }} if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2 uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with: with:
append_body: true append_body: true
body: | body: |
+2 -2
View File
@@ -42,7 +42,7 @@ jobs:
- name: Create Checksum - name: Create Checksum
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256 run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
- name: Upload - name: Upload
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2 uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with: with:
files: | files: |
${{ env.FILENAME_PREFIX }}.tar.gz ${{ env.FILENAME_PREFIX }}.tar.gz
@@ -68,7 +68,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Add release note - name: Add release note
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2 uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with: with:
append_body: true append_body: true
body: | body: |
+2 -2
View File
@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack - name: Enable Corepack
run: corepack enable run: corepack enable
- name: Yarn cache - name: Yarn cache
@@ -33,7 +33,7 @@ jobs:
timeout-minutes: 60 timeout-minutes: 60
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack - name: Enable Corepack
run: corepack enable run: corepack enable
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
+2 -2
View File
@@ -13,7 +13,7 @@ jobs:
steps: steps:
- name: Checkout the code - name: Checkout the code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack - name: Enable Corepack
run: corepack enable run: corepack enable
@@ -42,7 +42,7 @@ jobs:
- name: Create Pull Request - name: Create Pull Request
id: cpr id: cpr
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8 uses: peter-evans/create-pull-request@84ae59a2cdc2258d6fa0732dd66352dddae2a412 # v7.0.9
with: with:
token: ${{ secrets.ELEMENT_BOT_TOKEN }} token: ${{ secrets.ELEMENT_BOT_TOKEN }}
branch: actions/localazy-download branch: actions/localazy-download
+1 -1
View File
@@ -14,7 +14,7 @@ jobs:
steps: steps:
- name: Checkout the code - name: Checkout the code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Upload - name: Upload
uses: localazy/upload@27e6b5c0fddf4551596b42226b1c24124335d24a # v1 uses: localazy/upload@27e6b5c0fddf4551596b42226b1c24124335d24a # v1
+1 -1
View File
@@ -1 +1 @@
22 24
+1 -2
View File
@@ -13,7 +13,6 @@ coverage:
informational: true informational: true
patch: patch:
default: default:
# Encourage (but don't enforce) 80% coverage on all lines that a PR # Enforce 80% coverage on all lines that a PR
# touches # touches
target: 80% target: 80%
informational: true
+1 -1
View File
@@ -18,7 +18,7 @@
"api_host": "https://posthog-element-call.element.io" "api_host": "https://posthog-element-call.element.io"
}, },
"rageshake": { "rageshake": {
"submit_url": "https://element.io/bugreports/submit" "submit_url": "https://rageshakes.element.io/api/submit"
}, },
"sentry": { "sentry": {
"environment": "netlify-pr-preview", "environment": "netlify-pr-preview",
+4 -4
View File
@@ -136,8 +136,8 @@ handle @jwt_service {
reverse_proxy http://[::1]:8080 { reverse_proxy http://[::1]:8080 {
header_up Host {host} header_up Host {host}
header_up X-Forwarded-Server {host} header_up X-Forwarded-Server {host}
header_up X-Real-IP {remote_addr} header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_addr} header_up X-Forwarded-For {remote_host}
} }
} }
@@ -146,8 +146,8 @@ handle {
reverse_proxy http://localhost:7880 { reverse_proxy http://localhost:7880 {
header_up Host {host} header_up Host {host}
header_up X-Forwarded-Server {host} header_up X-Forwarded-Server {host}
header_up X-Real-IP {remote_addr} header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_addr} header_up X-Forwarded-For {remote_host}
} }
} }
``` ```
+1 -1
View File
@@ -96,6 +96,6 @@ These parameters are only supported in the [embedded](./embedded-standalone.md)
| -------------------- | -------------------------------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------- | | -------------------- | -------------------------------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `posthogApiHost` | Posthog server URL | No | e.g. `https://posthog-element-call.element.io`. Only supported in embedded package. In full package the value from config is used. | | `posthogApiHost` | Posthog server URL | No | e.g. `https://posthog-element-call.element.io`. Only supported in embedded package. In full package the value from config is used. |
| `posthogApiKey` | Posthog project API key | No | Only supported in embedded package. In full package the value from config is used. | | `posthogApiKey` | Posthog project API key | No | Only supported in embedded package. In full package the value from config is used. |
| `rageshakeSubmitUrl` | Rageshake server URL endpoint | No | e.g. `https://element.io/bugreports/submit`. In full package the value from config is used. | | `rageshakeSubmitUrl` | Rageshake server URL endpoint | No | e.g. `https://rageshakes.element.io/api/submit`. In full package the value from config is used. |
| `sentryDsn` | Sentry [DSN](https://docs.sentry.io/concepts/key-terms/dsn-explainer/) | No | In full package the value from config is used. | | `sentryDsn` | Sentry [DSN](https://docs.sentry.io/concepts/key-terms/dsn-explainer/) | No | In full package the value from config is used. |
| `sentryEnvironment` | Sentry [environment](https://docs.sentry.io/concepts/key-terms/key-terms/) | No | In full package the value from config is used. | | `sentryEnvironment` | Sentry [environment](https://docs.sentry.io/concepts/key-terms/key-terms/) | No | In full package the value from config is used. |
+2 -2
View File
@@ -2,11 +2,11 @@
# https://docs.gradle.org/current/userguide/platforms.html#sub::toml-dependencies-format # https://docs.gradle.org/current/userguide/platforms.html#sub::toml-dependencies-format
[versions] [versions]
android_gradle_plugin = "8.11.1" android_gradle_plugin = "8.13.1"
[libraries] [libraries]
android_gradle_plugin = { module = "com.android.tools.build:gradle", version.ref = "android_gradle_plugin" } android_gradle_plugin = { module = "com.android.tools.build:gradle", version.ref = "android_gradle_plugin" }
[plugins] [plugins]
android_library = { id = "com.android.library", version.ref = "android_gradle_plugin" } android_library = { id = "com.android.library", version.ref = "android_gradle_plugin" }
maven_publish = { id = "com.vanniktech.maven.publish", version = "0.34.0" } maven_publish = { id = "com.vanniktech.maven.publish", version = "0.35.0" }
+11
View File
@@ -64,6 +64,14 @@
"developer_mode": { "developer_mode": {
"always_show_iphone_earpiece": "Show iPhone earpiece option on all platforms", "always_show_iphone_earpiece": "Show iPhone earpiece option on all platforms",
"crypto_version": "Crypto version: {{version}}", "crypto_version": "Crypto version: {{version}}",
"custom_livekit_url": {
"current_url": "Currently set to: ",
"from_config": "Currently, no overwrite is set. Url from well-known or config is used.",
"label": "Custom Livekit-url",
"reset": "Reset overwrite",
"save": "Save",
"saving": "Saving..."
},
"debug_tile_layout_label": "Debug tile layout", "debug_tile_layout_label": "Debug tile layout",
"device_id": "Device ID: {{id}}", "device_id": "Device ID: {{id}}",
"duplicate_tiles_label": "Number of additional tile copies per participant", "duplicate_tiles_label": "Number of additional tile copies per participant",
@@ -100,11 +108,14 @@
"connection_lost_description": "You were disconnected from the call.", "connection_lost_description": "You were disconnected from the call.",
"e2ee_unsupported": "Incompatible browser", "e2ee_unsupported": "Incompatible browser",
"e2ee_unsupported_description": "Your web browser does not support encrypted calls. Supported browsers include Chrome, Safari, and Firefox 117+.", "e2ee_unsupported_description": "Your web browser does not support encrypted calls. Supported browsers include Chrome, Safari, and Firefox 117+.",
"failed_to_start_livekit": "Failed to start Livekit connection",
"generic": "Something went wrong", "generic": "Something went wrong",
"generic_description": "Submitting debug logs will help us track down the problem.", "generic_description": "Submitting debug logs will help us track down the problem.",
"insufficient_capacity": "Insufficient capacity", "insufficient_capacity": "Insufficient capacity",
"insufficient_capacity_description": "The server has reached its maximum capacity and you cannot join the call at this time. Try again later, or contact your server admin if the problem persists.", "insufficient_capacity_description": "The server has reached its maximum capacity and you cannot join the call at this time. Try again later, or contact your server admin if the problem persists.",
"matrix_rtc_transport_missing": "The server is not configured to work with {{brand}}. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).", "matrix_rtc_transport_missing": "The server is not configured to work with {{brand}}. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",
"membership_manager": "Membership Manager Error",
"membership_manager_description": "The Membership Manager had to shut down. This is caused by many consequtive failed network requests.",
"open_elsewhere": "Opened in another tab", "open_elsewhere": "Opened in another tab",
"open_elsewhere_description": "{{brand}} has been opened in another tab. If that doesn't sound right, try reloading the page.", "open_elsewhere_description": "{{brand}} has been opened in another tab. If that doesn't sound right, try reloading the page.",
"room_creation_restricted": "Failed to create call", "room_creation_restricted": "Failed to create call",
+3 -3
View File
@@ -71,7 +71,7 @@
"@types/grecaptcha": "^3.0.9", "@types/grecaptcha": "^3.0.9",
"@types/jsdom": "^21.1.7", "@types/jsdom": "^21.1.7",
"@types/lodash-es": "^4.17.12", "@types/lodash-es": "^4.17.12",
"@types/node": "^22.0.0", "@types/node": "^24.0.0",
"@types/pako": "^2.0.3", "@types/pako": "^2.0.3",
"@types/qrcode": "^1.5.5", "@types/qrcode": "^1.5.5",
"@types/react": "^19.0.0", "@types/react": "^19.0.0",
@@ -109,8 +109,8 @@
"livekit-client": "^2.13.0", "livekit-client": "^2.13.0",
"lodash-es": "^4.17.21", "lodash-es": "^4.17.21",
"loglevel": "^1.9.1", "loglevel": "^1.9.1",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#head=toger5/sticky-events&commit=e7f5bec51b6f70501a025b79fe5021c933385b21", "matrix-js-sdk": "^39.2.0",
"matrix-widget-api": "^1.13.0", "matrix-widget-api": "^1.14.0",
"normalize.css": "^8.0.1", "normalize.css": "^8.0.1",
"observable-hooks": "^4.2.3", "observable-hooks": "^4.2.3",
"pako": "^2.0.4", "pako": "^2.0.4",
+5
View File
@@ -75,7 +75,12 @@ test("Should automatically retry non fatal JWT errors", async ({
test("Should show error screen if call creation is restricted", async ({ test("Should show error screen if call creation is restricted", async ({
page, page,
browserName,
}) => { }) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment.",
);
await page.goto("/"); await page.goto("/");
// We need the socket connection to fail, but this cannot be done by using the websocket route. // We need the socket connection to fail, but this cannot be done by using the websocket route.
+5 -1
View File
@@ -159,7 +159,11 @@ export const widgetTest = test.extend<MyFixtures>({
} = await registerUser(browser, userB); } = await registerUser(browser, userB);
// Invite the second user // Invite the second user
await ewPage1.getByRole("button", { name: "Add", exact: true }).click(); await ewPage1
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.click();
await ewPage1.getByRole("menuitem", { name: "New Room" }).click(); await ewPage1.getByRole("menuitem", { name: "New Room" }).click();
await ewPage1.getByRole("textbox", { name: "Name" }).fill("Welcome Room"); await ewPage1.getByRole("textbox", { name: "Name" }).fill("Welcome Room");
await ewPage1.getByRole("button", { name: "Create room" }).click(); await ewPage1.getByRole("button", { name: "Create room" }).click();
+36
View File
@@ -332,6 +332,42 @@ describe("UrlParams", () => {
expect(computeUrlParams("?intent=join_existing").skipLobby).toBe(false); expect(computeUrlParams("?intent=join_existing").skipLobby).toBe(false);
}); });
}); });
describe("noiseSuppression", () => {
it("defaults to true", () => {
expect(computeUrlParams().noiseSuppression).toBe(true);
});
it("is parsed", () => {
expect(
computeUrlParams("?intent=start_call&noiseSuppression=true")
.noiseSuppression,
).toBe(true);
expect(
computeUrlParams("?intent=start_call&noiseSuppression&bar=foo")
.noiseSuppression,
).toBe(true);
expect(computeUrlParams("?noiseSuppression=false").noiseSuppression).toBe(
false,
);
});
});
describe("echoCancellation", () => {
it("defaults to true", () => {
expect(computeUrlParams().echoCancellation).toBe(true);
});
it("is parsed", () => {
expect(computeUrlParams("?echoCancellation=true").echoCancellation).toBe(
true,
);
expect(computeUrlParams("?echoCancellation=false").echoCancellation).toBe(
false,
);
});
});
describe("header", () => { describe("header", () => {
it("uses header if provided", () => { it("uses header if provided", () => {
expect(computeUrlParams("?header=app_bar&hideHeader=true").header).toBe( expect(computeUrlParams("?header=app_bar&hideHeader=true").header).toBe(
+13
View File
@@ -233,6 +233,17 @@ export interface UrlConfiguration {
*/ */
waitForCallPickup: boolean; waitForCallPickup: boolean;
/**
* Whether to enable echo cancellation for audio capture.
* Defaults to true.
*/
echoCancellation?: boolean;
/**
* Whether to enable noise suppression for audio capture.
* Defaults to true.
*/
noiseSuppression?: boolean;
callIntent?: RTCCallIntent; callIntent?: RTCCallIntent;
} }
interface IntentAndPlatformDerivedConfiguration { interface IntentAndPlatformDerivedConfiguration {
@@ -525,6 +536,8 @@ export const computeUrlParams = (search = "", hash = ""): UrlParams => {
]), ]),
waitForCallPickup: parser.getFlag("waitForCallPickup"), waitForCallPickup: parser.getFlag("waitForCallPickup"),
autoLeaveWhenOthersLeft: parser.getFlag("autoLeave"), autoLeaveWhenOthersLeft: parser.getFlag("autoLeave"),
noiseSuppression: parser.getFlagParam("noiseSuppression", true),
echoCancellation: parser.getFlagParam("echoCancellation", true),
}; };
// Log the final configuration for debugging purposes. // Log the final configuration for debugging purposes.
+1 -1
View File
@@ -122,7 +122,7 @@ export interface ConfigOptions {
delayed_leave_event_delay_ms?: number; delayed_leave_event_delay_ms?: number;
/** /**
* The time (in milliseconds) after which a we consider a delayed event restart http request to have failed. * The time (in milliseconds) after which we consider a delayed event restart http request to have failed.
* Setting this to a lower value will result in more frequent retries but also a higher chance of failiour. * Setting this to a lower value will result in more frequent retries but also a higher chance of failiour.
* *
* In the presence of network packet loss (hurting TCP connections), the custom delayedEventRestartLocalTimeoutMs * In the presence of network packet loss (hurting TCP connections), the custom delayedEventRestartLocalTimeoutMs
+10 -1
View File
@@ -21,7 +21,16 @@ export type OpenIDClientParts = Pick<
MatrixClient, MatrixClient,
"getOpenIdToken" | "getDeviceId" "getOpenIdToken" | "getDeviceId"
>; >;
/**
* Gets a bearer token from the homeserver and then use it to authenticate
* to the matrix RTC backend in order to get acces to the SFU.
* It has built-in retry for calls to the homeserver with a backoff policy.
* @param client
* @param serviceUrl
* @param matrixRoomId
* @returns Object containing the token information
* @throws FailToGetOpenIdToken
*/
export async function getSFUConfigWithOpenID( export async function getSFUConfigWithOpenID(
client: OpenIDClientParts, client: OpenIDClientParts,
serviceUrl: string, serviceUrl: string,
+8 -1
View File
@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
*/ */
import { describe, expect, test } from "vitest"; import { beforeEach, describe, expect, test, vi } from "vitest";
import { render, configure } from "@testing-library/react"; import { render, configure } from "@testing-library/react";
import { RaisedHandIndicator } from "./RaisedHandIndicator"; import { RaisedHandIndicator } from "./RaisedHandIndicator";
@@ -15,6 +15,13 @@ configure({
}); });
describe("RaisedHandIndicator", () => { describe("RaisedHandIndicator", () => {
const fixedTime = new Date("2025-01-01T12:00:00.000Z");
beforeEach(() => {
vi.useFakeTimers();
vi.setSystemTime(fixedTime);
});
test("renders nothing when no hand has been raised", () => { test("renders nothing when no hand has been raised", () => {
const { container } = render(<RaisedHandIndicator />); const { container } = render(<RaisedHandIndicator />);
expect(container.firstChild).toBeNull(); expect(container.firstChild).toBeNull();
@@ -15,7 +15,7 @@ exports[`RaisedHandIndicator > renders a smaller indicator when miniature is spe
</span> </span>
</div> </div>
<p> <p>
00:01 00:00
</p> </p>
</div> </div>
`; `;
@@ -35,7 +35,7 @@ exports[`RaisedHandIndicator > renders an indicator when a hand has been raised
</span> </span>
</div> </div>
<p> <p>
00:01 00:00
</p> </p>
</div> </div>
`; `;
@@ -55,7 +55,7 @@ exports[`RaisedHandIndicator > renders an indicator when a hand has been raised
</span> </span>
</div> </div>
<p> <p>
01:01 01:00
</p> </p>
</div> </div>
`; `;
+2 -1
View File
@@ -22,6 +22,7 @@ import {
WebBrowserIcon, WebBrowserIcon,
} from "@vector-im/compound-design-tokens/assets/web/icons"; } from "@vector-im/compound-design-tokens/assets/web/icons";
import { Button } from "@vector-im/compound-web"; import { Button } from "@vector-im/compound-web";
import { logger } from "matrix-js-sdk/lib/logger";
import { import {
ConnectionLostError, ConnectionLostError,
@@ -53,7 +54,7 @@ const ErrorPage: FC<ErrorPageProps> = ({
widget, widget,
}: ErrorPageProps): ReactElement => { }: ErrorPageProps): ReactElement => {
const { t } = useTranslation(); const { t } = useTranslation();
logger.error("Error boundary caught:", error);
let icon: ComponentType<SVGAttributes<SVGElement>>; let icon: ComponentType<SVGAttributes<SVGElement>>;
switch (error.category) { switch (error.category) {
case ErrorCategory.CONFIGURATION_ISSUE: case ErrorCategory.CONFIGURATION_ISSUE:
+112 -1
View File
@@ -29,6 +29,7 @@ import userEvent from "@testing-library/user-event";
import { type RelationsContainer } from "matrix-js-sdk/lib/models/relations-container"; import { type RelationsContainer } from "matrix-js-sdk/lib/models/relations-container";
import { useState } from "react"; import { useState } from "react";
import { TooltipProvider } from "@vector-im/compound-web"; import { TooltipProvider } from "@vector-im/compound-web";
import { type ITransport } from "matrix-widget-api";
import { prefetchSounds } from "../soundUtils"; import { prefetchSounds } from "../soundUtils";
import { useAudioContext } from "../useAudioContext"; import { useAudioContext } from "../useAudioContext";
@@ -43,7 +44,7 @@ import {
MockRTCSession, MockRTCSession,
} from "../utils/test"; } from "../utils/test";
import { GroupCallView } from "./GroupCallView"; import { GroupCallView } from "./GroupCallView";
import { type WidgetHelpers } from "../widget"; import { ElementWidgetActions, type WidgetHelpers } from "../widget";
import { LazyEventEmitter } from "../LazyEventEmitter"; import { LazyEventEmitter } from "../LazyEventEmitter";
import { MatrixRTCTransportMissingError } from "../utils/errors"; import { MatrixRTCTransportMissingError } from "../utils/errors";
import { ProcessorProvider } from "../livekit/TrackProcessorContext"; import { ProcessorProvider } from "../livekit/TrackProcessorContext";
@@ -112,6 +113,10 @@ beforeEach(() => {
return ( return (
<div> <div>
<button onClick={() => onLeave("user")}>Leave</button> <button onClick={() => onLeave("user")}>Leave</button>
<button onClick={() => onLeave("allOthersLeft")}>
SimulateOtherLeft
</button>
<button onClick={() => onLeave("error")}>SimulateErrorLeft</button>
</div> </div>
); );
}, },
@@ -243,6 +248,112 @@ test.skip("GroupCallView plays a leave sound synchronously in widget mode", asyn
expect(leaveRTCSession).toHaveBeenCalledOnce(); expect(leaveRTCSession).toHaveBeenCalledOnce();
}); });
test.skip("Should close widget when all other left and have time to play a sound", async () => {
const user = userEvent.setup();
const widgetClosedCalled = Promise.withResolvers<void>();
const widgetSendMock = vi.fn().mockImplementation((action: string) => {
if (action === ElementWidgetActions.Close) {
widgetClosedCalled.resolve();
}
});
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const resolvePlaySound = Promise.withResolvers<void>();
playSound = vi.fn().mockReturnValue(resolvePlaySound);
(useAudioContext as MockedFunction<typeof useAudioContext>).mockReturnValue({
playSound,
playSoundLooping: vitest.fn(),
soundDuration: {},
});
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateOtherLeft");
await user.click(leaveButton);
await flushPromises();
expect(widgetSendMock).not.toHaveBeenCalled();
resolvePlaySound.resolve();
await flushPromises();
expect(playSound).toHaveBeenCalledWith("left");
await widgetClosedCalled.promise;
await flushPromises();
expect(widgetStopMock).toHaveBeenCalledOnce();
});
test("Should close widget when all other left", async () => {
const user = userEvent.setup();
const widgetClosedCalled = Promise.withResolvers<void>();
const widgetSendMock = vi.fn().mockImplementation((action: string) => {
if (action === ElementWidgetActions.Close) {
widgetClosedCalled.resolve();
}
});
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateOtherLeft");
await user.click(leaveButton);
await flushPromises();
await widgetClosedCalled.promise;
await flushPromises();
expect(widgetStopMock).toHaveBeenCalledOnce();
});
test("Should not close widget when auto leave due to error", async () => {
const user = userEvent.setup();
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widgetSendMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const alwaysOnScreenSpy = vi.spyOn(widget.api, "setAlwaysOnScreen");
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateErrorLeft");
await user.click(leaveButton);
await flushPromises();
// When onLeft is called, we first set always on screen to false
await waitFor(() => expect(alwaysOnScreenSpy).toHaveBeenCalledWith(false));
await flushPromises();
// But then we do not close the widget automatically
expect(widgetStopMock).not.toHaveBeenCalledOnce();
expect(widgetSendMock).not.toHaveBeenCalledOnce();
});
test.skip("GroupCallView leaves the session when an error occurs", async () => { test.skip("GroupCallView leaves the session when an error occurs", async () => {
(ActiveCall as MockedFunction<typeof ActiveCall>).mockImplementation(() => { (ActiveCall as MockedFunction<typeof ActiveCall>).mockImplementation(() => {
const [error, setError] = useState<Error | null>(null); const [error, setError] = useState<Error | null>(null);
+6 -4
View File
@@ -159,6 +159,7 @@ export const GroupCallView: FC<Props> = ({
}; };
}, [rtcSession]); }, [rtcSession]);
// TODO move this into the callViewModel LocalMembership.ts
useTypedEventEmitter( useTypedEventEmitter(
rtcSession, rtcSession,
MatrixRTCSessionEvent.MembershipManagerError, MatrixRTCSessionEvent.MembershipManagerError,
@@ -313,7 +314,9 @@ export const GroupCallView: FC<Props> = ({
const navigate = useNavigate(); const navigate = useNavigate();
const onLeft = useCallback( const onLeft = useCallback(
(reason: "timeout" | "user" | "allOthersLeft" | "decline"): void => { (
reason: "timeout" | "user" | "allOthersLeft" | "decline" | "error",
): void => {
let playSound: CallEventSounds = "left"; let playSound: CallEventSounds = "left";
if (reason === "timeout" || reason === "decline") playSound = reason; if (reason === "timeout" || reason === "decline") playSound = reason;
@@ -366,7 +369,7 @@ export const GroupCallView: FC<Props> = ({
} }
// On a normal user hangup we can shut down and close the widget. But if an // On a normal user hangup we can shut down and close the widget. But if an
// error occurs we should keep the widget open until the user reads it. // error occurs we should keep the widget open until the user reads it.
if (reason === "user" && !getUrlParams().returnToLobby) { if (reason != "error" && !getUrlParams().returnToLobby) {
try { try {
await widget.api.transport.send(ElementWidgetActions.Close, {}); await widget.api.transport.send(ElementWidgetActions.Close, {});
} catch (e) { } catch (e) {
@@ -518,8 +521,7 @@ export const GroupCallView: FC<Props> = ({
}} }}
onError={ onError={
(/**error*/) => { (/**error*/) => {
// TODO this should not be "user". It needs a new case if (rtcSession.isJoined()) onLeft("error");
if (rtcSession.isJoined()) onLeft("user");
} }
} }
> >
+17 -33
View File
@@ -59,7 +59,8 @@ import { type MatrixInfo } from "./VideoPreview";
import { InviteButton } from "../button/InviteButton"; import { InviteButton } from "../button/InviteButton";
import { LayoutToggle } from "./LayoutToggle"; import { LayoutToggle } from "./LayoutToggle";
import { import {
CallViewModel, type CallViewModel,
createCallViewModel$,
type GridMode, type GridMode,
} from "../state/CallViewModel/CallViewModel.ts"; } from "../state/CallViewModel/CallViewModel.ts";
import { Grid, type TileProps } from "../grid/Grid"; import { Grid, type TileProps } from "../grid/Grid";
@@ -114,7 +115,9 @@ export interface ActiveCallProps
extends Omit<InCallViewProps, "vm" | "livekitRoom" | "connState"> { extends Omit<InCallViewProps, "vm" | "livekitRoom" | "connState"> {
e2eeSystem: EncryptionSystem; e2eeSystem: EncryptionSystem;
// TODO refactor those reasons into an enum // TODO refactor those reasons into an enum
onLeft: (reason: "user" | "timeout" | "decline" | "allOthersLeft") => void; onLeft: (
reason: "user" | "timeout" | "decline" | "allOthersLeft" | "error",
) => void;
} }
export const ActiveCall: FC<ActiveCallProps> = (props) => { export const ActiveCall: FC<ActiveCallProps> = (props) => {
@@ -128,7 +131,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
const reactionsReader = new ReactionsReader(scope, props.rtcSession); const reactionsReader = new ReactionsReader(scope, props.rtcSession);
const { autoLeaveWhenOthersLeft, waitForCallPickup, sendNotificationType } = const { autoLeaveWhenOthersLeft, waitForCallPickup, sendNotificationType } =
urlParams; urlParams;
const vm = new CallViewModel( const vm = createCallViewModel$(
scope, scope,
props.rtcSession, props.rtcSession,
props.matrixRoom, props.matrixRoom,
@@ -143,6 +146,8 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
reactionsReader.reactions$, reactionsReader.reactions$,
scope.behavior(trackProcessorState$), scope.behavior(trackProcessorState$),
); );
// TODO move this somewhere else once we use the callViewModel in the lobby as well!
vm.join();
setVm(vm); setVm(vm);
vm.leave$.pipe(scope.bind()).subscribe(props.onLeft); vm.leave$.pipe(scope.bind()).subscribe(props.onLeft);
@@ -248,7 +253,6 @@ export const InCallView: FC<InCallViewProps> = ({
() => void toggleRaisedHand(), () => void toggleRaisedHand(),
); );
// const allLivekitRooms = useBehavior(vm.allLivekitRooms$);
const audioParticipants = useBehavior(vm.audioParticipants$); const audioParticipants = useBehavior(vm.audioParticipants$);
const participantCount = useBehavior(vm.participantCount$); const participantCount = useBehavior(vm.participantCount$);
const reconnecting = useBehavior(vm.reconnecting$); const reconnecting = useBehavior(vm.reconnecting$);
@@ -263,7 +267,8 @@ export const InCallView: FC<InCallViewProps> = ({
const audioOutputSwitcher = useBehavior(vm.audioOutputSwitcher$); const audioOutputSwitcher = useBehavior(vm.audioOutputSwitcher$);
const sharingScreen = useBehavior(vm.sharingScreen$); const sharingScreen = useBehavior(vm.sharingScreen$);
const fatalCallError = useBehavior(vm.configError$); const ringOverlay = useBehavior(vm.ringOverlay$);
const fatalCallError = useBehavior(vm.fatalError$);
// Stop the rendering and throw for the error boundary // Stop the rendering and throw for the error boundary
if (fatalCallError) throw fatalCallError; if (fatalCallError) throw fatalCallError;
@@ -299,47 +304,26 @@ export const InCallView: FC<InCallViewProps> = ({
// Waiting UI overlay // Waiting UI overlay
const waitingOverlay: JSX.Element | null = useMemo(() => { const waitingOverlay: JSX.Element | null = useMemo(() => {
// No overlay if not in ringing state return ringOverlay ? (
if (callPickupState !== "ringing") return null;
// Use room state for other participants data (the one that we likely want to reach)
// TODO: this screams it wants to be a behavior in the vm.
const roomOthers = [
...matrixRoom.getMembersWithMembership("join"),
...matrixRoom.getMembersWithMembership("invite"),
].filter((m) => m.userId !== client.getUserId());
// Yield if there are not other members in the room.
if (roomOthers.length === 0) return null;
const otherMember = roomOthers.length > 0 ? roomOthers[0] : undefined;
const isOneOnOne = roomOthers.length === 1 && otherMember;
const text = isOneOnOne
? `Waiting for ${otherMember.name ?? otherMember.userId} to join…`
: "Waiting for other participants…";
const avatarMxc = isOneOnOne
? (otherMember.getMxcAvatarUrl?.() ?? undefined)
: (matrixRoom.getMxcAvatarUrl() ?? undefined);
return (
<div className={classNames(overlayStyles.bg, waitingStyles.overlay)}> <div className={classNames(overlayStyles.bg, waitingStyles.overlay)}>
<div <div
className={classNames(overlayStyles.content, waitingStyles.content)} className={classNames(overlayStyles.content, waitingStyles.content)}
> >
<div className={waitingStyles.pulse}> <div className={waitingStyles.pulse}>
<Avatar <Avatar
id={isOneOnOne ? otherMember.userId : matrixRoom.roomId} id={ringOverlay.idForAvatar}
name={isOneOnOne ? otherMember.name : matrixRoom.name} name={ringOverlay.name}
src={avatarMxc} src={ringOverlay.avatarMxc}
size={AvatarSize.XL} size={AvatarSize.XL}
/> />
</div> </div>
<Text size="md" className={waitingStyles.text}> <Text size="md" className={waitingStyles.text}>
{text} {ringOverlay.text}
</Text> </Text>
</div> </div>
</div> </div>
); ) : null;
}, [callPickupState, client, matrixRoom]); }, [ringOverlay]);
// Ideally we could detect taps by listening for click events and checking // Ideally we could detect taps by listening for click events and checking
// that the pointerType of the event is "touch", but this isn't yet supported // that the pointerType of the event is "touch", but this isn't yet supported
@@ -0,0 +1,95 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, vi } from "vitest";
import { render, waitFor } from "@testing-library/react";
import type { MatrixClient } from "matrix-js-sdk";
import type { Room as LivekitRoom } from "livekit-client";
import { DeveloperSettingsTab } from "./DeveloperSettingsTab";
// Mock url params hook to avoid environment-dependent snapshot churn.
vi.mock("../UrlParams", () => ({
useUrlParams: (): { mocked: boolean; answer: number } => ({
mocked: true,
answer: 42,
}),
}));
// Provide a minimal mock of a Livekit Room structure used by the component.
function createMockLivekitRoom(
wsUrl: string,
serverInfo: object,
metadata: string,
): { isLocal: boolean; url: string; room: LivekitRoom } {
const mockRoom = {
serverInfo,
metadata,
engine: { client: { ws: { url: wsUrl } } },
} as unknown as LivekitRoom;
return {
isLocal: true,
url: wsUrl,
room: mockRoom,
};
}
// Minimal MatrixClient mock with only the methods used by the component.
function createMockMatrixClient(): MatrixClient {
return {
doesServerSupportUnstableFeature: vi.fn().mockResolvedValue(true), // ensure stickyEventsSupported eventually becomes true
getCrypto: (): { getVersion: () => string } | undefined => ({
getVersion: () => "crypto-1.0.0",
}),
getUserId: () => "@alice:example.org",
getDeviceId: () => "DEVICE123",
} as unknown as MatrixClient;
}
describe("DeveloperSettingsTab", () => {
it("renders and matches snapshot", async () => {
const client = createMockMatrixClient();
const livekitRooms: {
room: LivekitRoom;
url: string;
isLocal?: boolean;
}[] = [
createMockLivekitRoom(
"wss://local-sfu.example.org",
{ region: "local", version: "1.2.3" },
"local-metadata",
),
{
isLocal: false,
url: "wss://remote-sfu.example.org",
room: {
serverInfo: { region: "remote", version: "4.5.6" },
metadata: "remote-metadata",
engine: { client: { ws: { url: "wss://remote-sfu.example.org" } } },
} as unknown as LivekitRoom,
},
];
const { container } = render(
<DeveloperSettingsTab
client={client}
livekitRooms={livekitRooms}
env={{ MY_MOCK_ENV: 10, ENV: "test" } as unknown as ImportMetaEnv}
/>,
);
// Wait for the async sticky events feature check to resolve so the final UI
// (e.g. enabled Matrix_2_0 radio button) appears deterministically.
await waitFor(() =>
expect(client.doesServerSupportUnstableFeature).toHaveBeenCalled(),
);
expect(container).toMatchSnapshot();
});
});
+59 -5
View File
@@ -11,8 +11,8 @@ import {
useCallback, useCallback,
useEffect, useEffect,
useMemo, useMemo,
useState,
useId, useId,
useState,
} from "react"; } from "react";
import { useTranslation } from "react-i18next"; import { useTranslation } from "react-i18next";
import { import {
@@ -21,6 +21,7 @@ import {
} from "matrix-js-sdk"; } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger"; import { logger } from "matrix-js-sdk/lib/logger";
import { import {
EditInPlace,
Root as Form, Root as Form,
Heading, Heading,
HelpMessage, HelpMessage,
@@ -38,6 +39,7 @@ import {
muteAllAudio as muteAllAudioSetting, muteAllAudio as muteAllAudioSetting,
alwaysShowIphoneEarpiece as alwaysShowIphoneEarpieceSetting, alwaysShowIphoneEarpiece as alwaysShowIphoneEarpieceSetting,
matrixRTCMode as matrixRTCModeSetting, matrixRTCMode as matrixRTCModeSetting,
customLivekitUrl as customLivekitUrlSetting,
MatrixRTCMode, MatrixRTCMode,
} from "./settings"; } from "./settings";
import type { Room as LivekitRoom } from "livekit-client"; import type { Room as LivekitRoom } from "livekit-client";
@@ -47,9 +49,14 @@ import { useUrlParams } from "../UrlParams";
interface Props { interface Props {
client: MatrixClient; client: MatrixClient;
livekitRooms?: { room: LivekitRoom; url: string; isLocal?: boolean }[]; livekitRooms?: { room: LivekitRoom; url: string; isLocal?: boolean }[];
env: ImportMetaEnv;
} }
export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => { export const DeveloperSettingsTab: FC<Props> = ({
client,
livekitRooms,
env,
}) => {
const { t } = useTranslation(); const { t } = useTranslation();
const [duplicateTiles, setDuplicateTiles] = useSetting(duplicateTilesSetting); const [duplicateTiles, setDuplicateTiles] = useSetting(duplicateTilesSetting);
const [debugTileLayout, setDebugTileLayout] = useSetting( const [debugTileLayout, setDebugTileLayout] = useSetting(
@@ -85,6 +92,15 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
alwaysShowIphoneEarpieceSetting, alwaysShowIphoneEarpieceSetting,
); );
const [customLivekitUrl, setCustomLivekitUrl] = useSetting(
customLivekitUrlSetting,
);
const [customLivekitUrlTextBuffer, setCustomLivekitUrlTextBuffer] =
useState(customLivekitUrl);
useEffect(() => {
setCustomLivekitUrlTextBuffer(customLivekitUrl);
}, [customLivekitUrl]);
const [muteAllAudio, setMuteAllAudio] = useSetting(muteAllAudioSetting); const [muteAllAudio, setMuteAllAudio] = useSetting(muteAllAudioSetting);
const urlParams = useUrlParams(); const urlParams = useUrlParams();
@@ -101,7 +117,7 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
}, [livekitRooms]); }, [livekitRooms]);
return ( return (
<Form> <>
<p> <p>
{t("developer_mode.hostname", { {t("developer_mode.hostname", {
hostname: window.location.hostname || "unknown", hostname: window.location.hostname || "unknown",
@@ -200,9 +216,46 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
)} )}
/>{" "} />{" "}
</FieldRow> </FieldRow>
<EditInPlace
onSubmit={(e) => e.preventDefault()}
helpLabel={
customLivekitUrl === null
? t("developer_mode.custom_livekit_url.from_config")
: t("developer_mode.custom_livekit_url.current_url") +
customLivekitUrl
}
label={t("developer_mode.custom_livekit_url.label")}
saveButtonLabel={t("developer_mode.custom_livekit_url.save")}
savingLabel={t("developer_mode.custom_livekit_url.saving")}
cancelButtonLabel={t("developer_mode.custom_livekit_url.reset")}
onSave={useCallback(
(e: React.FormEvent<HTMLFormElement>) => {
setCustomLivekitUrl(
customLivekitUrlTextBuffer === ""
? null
: customLivekitUrlTextBuffer,
);
},
[setCustomLivekitUrl, customLivekitUrlTextBuffer],
)}
value={customLivekitUrlTextBuffer ?? ""}
onChange={useCallback(
(event: ChangeEvent<HTMLInputElement>): void => {
setCustomLivekitUrlTextBuffer(event.target.value);
},
[setCustomLivekitUrlTextBuffer],
)}
onCancel={useCallback(
(e: React.FormEvent<HTMLFormElement>) => {
setCustomLivekitUrl(null);
},
[setCustomLivekitUrl],
)}
/>
<Heading as="h3" type="body" weight="semibold" size="lg"> <Heading as="h3" type="body" weight="semibold" size="lg">
{t("developer_mode.matrixRTCMode.title")} {t("developer_mode.matrixRTCMode.title")}
</Heading> </Heading>
<Form>
<InlineField <InlineField
name={matrixRTCModeRadioGroup} name={matrixRTCModeRadioGroup}
control={ control={
@@ -249,6 +302,7 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
{t("developer_mode.matrixRTCMode.Matrix_2_0.description")} {t("developer_mode.matrixRTCMode.Matrix_2_0.description")}
</HelpMessage> </HelpMessage>
</InlineField> </InlineField>
</Form>
{livekitRooms?.map((livekitRoom) => ( {livekitRooms?.map((livekitRoom) => (
<> <>
<h3> <h3>
@@ -270,9 +324,9 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
</> </>
))} ))}
<p>{t("developer_mode.environment_variables")}</p> <p>{t("developer_mode.environment_variables")}</p>
<pre>{JSON.stringify(import.meta.env, null, 2)}</pre> <pre>{JSON.stringify(env, null, 2)}</pre>
<p>{t("developer_mode.url_params")}</p> <p>{t("developer_mode.url_params")}</p>
<pre>{JSON.stringify(urlParams, null, 2)}</pre> <pre>{JSON.stringify(urlParams, null, 2)}</pre>
</Form> </>
); );
}; };
+5 -1
View File
@@ -209,7 +209,11 @@ export const SettingsModal: FC<Props> = ({
key: "developer", key: "developer",
name: t("settings.developer_tab_title"), name: t("settings.developer_tab_title"),
content: ( content: (
<DeveloperSettingsTab client={client} livekitRooms={livekitRooms} /> <DeveloperSettingsTab
env={import.meta.env}
client={client}
livekitRooms={livekitRooms}
/>
), ),
}; };
@@ -0,0 +1,411 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
<div>
<p>
Hostname: localhost
</p>
<p>
Element Call version: dev
</p>
<p>
Crypto version: crypto-1.0.0
</p>
<p>
Matrix ID: @alice:example.org
</p>
<p>
Device ID: DEVICE123
</p>
<div
class="fieldRow"
>
<div
class="field inputField"
>
<input
aria-describedby="«r1»"
id="duplicateTiles"
min="0"
type="number"
value="0"
/>
<label
for="duplicateTiles"
>
Number of additional tile copies per participant
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r2»"
id="debugTileLayout"
type="checkbox"
/>
<label
for="debugTileLayout"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Debug tile layout
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r3»"
id="showConnectionStats"
type="checkbox"
/>
<label
for="showConnectionStats"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Show connection statistics
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r4»"
id="muteAllAudio"
type="checkbox"
/>
<label
for="muteAllAudio"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Mute all audio (participants, reactions, join sounds)
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r5»"
id="alwaysShowIphoneEarpiece"
type="checkbox"
/>
<label
for="alwaysShowIphoneEarpiece"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Show iPhone earpiece option on all platforms
</label>
</div>
</div>
<form
class="_root_19upo_16"
>
<div
class="_field_19upo_26"
>
<label
class="_label_19upo_59"
for="radix-«r6»"
>
Custom Livekit-url
</label>
<div
class="_controls_17lij_8"
>
<input
aria-describedby="radix-«r7»"
class="_control_sqdq4_10"
id="radix-«r6»"
name="input"
title=""
value=""
/>
</div>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«r7»"
>
Currently, no overwrite is set. Url from well-known or config is used.
</span>
</div>
</form>
<h3
class="_typography_6v6n8_153 _font-body-lg-semibold_6v6n8_74"
>
MatrixRTC mode
</h3>
<form
class="_root_19upo_16"
>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
checked=""
class="_input_1e0uz_18"
id="radix-«r8»"
name="«r0»"
title=""
type="radio"
value="legacy"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«r8»"
>
Legacy: state events & oldest membership SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«r9»"
>
Compatible with old versions of EC that do not support multi SFU
</span>
</div>
</div>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
class="_input_1e0uz_18"
id="radix-«ra»"
name="«r0»"
title=""
type="radio"
value="compatibil"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«ra»"
>
Compatibility: state events & multi SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«rb»"
>
Compatible with homeservers that do not support sticky events (but all other EC clients are v0.17.0 or later)
</span>
</div>
</div>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
class="_input_1e0uz_18"
id="radix-«rc»"
name="«r0»"
title=""
type="radio"
value="matrix_2_0"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«rc»"
>
Matrix 2.0: sticky events & multi SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«rd»"
>
Compatible only with homservers supporting sticky events and all EC clients v0.17.0 or later
</span>
</div>
</div>
</form>
<h3>
LiveKit SFU: wss://local-sfu.example.org
</h3>
<p>
ws-url:
wss://local-sfu.example.org/
</p>
<p>
LiveKit Server Info
(
local
)
</p>
<pre
class="pre"
>
{
"region": "local",
"version": "1.2.3"
}
local-metadata
</pre>
<h3>
LiveKit SFU: wss://remote-sfu.example.org
</h3>
<p>
LiveKit Server Info
(
remote
)
</p>
<pre
class="pre"
>
{
"region": "remote",
"version": "4.5.6"
}
remote-metadata
</pre>
<p>
Environment variables
</p>
<pre>
{
"MY_MOCK_ENV": 10,
"ENV": "test"
}
</pre>
<p>
URL parameters
</p>
<pre>
{
"mocked": true,
"answer": 42
}
</pre>
</div>
`;
+5
View File
@@ -134,3 +134,8 @@ export const matrixRTCMode = new Setting<MatrixRTCMode>(
"matrix-rtc-mode", "matrix-rtc-mode",
MatrixRTCMode.Legacy, MatrixRTCMode.Legacy,
); );
export const customLivekitUrl = new Setting<string | null>(
"custom-livekit-url",
null,
);
+4 -1
View File
@@ -16,7 +16,10 @@ import { BehaviorSubject } from "rxjs";
* distinction between Behaviors and Observables, see * distinction between Behaviors and Observables, see
* https://monoid.dk/post/behaviors-and-streams-why-both/. * https://monoid.dk/post/behaviors-and-streams-why-both/.
*/ */
export type Behavior<T> = Omit<BehaviorSubject<T>, "next" | "observers">; export type Behavior<T> = Omit<
BehaviorSubject<T>,
"next" | "observers" | "error"
>;
/** /**
* Creates a Behavior which never changes in value. * Creates a Behavior which never changes in value.
@@ -1,5 +1,5 @@
/* /*
Copyright 2025 New Vector Ltd. Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
@@ -1,5 +1,5 @@
/* /*
Copyright 2025 New Vector Ltd. Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
+4 -83
View File
@@ -1,12 +1,12 @@
/* /*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd. Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
*/ */
import { test, vi, onTestFinished, it, describe, expect } from "vitest"; import { test, vi, onTestFinished, it, describe } from "vitest";
import EventEmitter from "events";
import { import {
BehaviorSubject, BehaviorSubject,
combineLatest, combineLatest,
@@ -18,12 +18,11 @@ import {
of, of,
switchMap, switchMap,
} from "rxjs"; } from "rxjs";
import { SyncState, type MatrixClient } from "matrix-js-sdk"; import { SyncState } from "matrix-js-sdk";
import { import {
ConnectionState, ConnectionState,
type LocalTrackPublication, type LocalTrackPublication,
type RemoteParticipant, type RemoteParticipant,
type Room as LivekitRoom,
} from "livekit-client"; } from "livekit-client";
import * as ComponentsCore from "@livekit/components-core"; import * as ComponentsCore from "@livekit/components-core";
import { import {
@@ -35,27 +34,18 @@ import {
type LivekitTransport, type LivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc"; } from "matrix-js-sdk/lib/matrixrtc";
import { deepCompare } from "matrix-js-sdk/lib/utils"; import { deepCompare } from "matrix-js-sdk/lib/utils";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { CallViewModel } from "./CallViewModel";
import { type Layout } from "../layout-types.ts"; import { type Layout } from "../layout-types.ts";
import { import {
mockLocalParticipant, mockLocalParticipant,
mockMatrixRoom,
mockMatrixRoomMember, mockMatrixRoomMember,
mockRemoteParticipant, mockRemoteParticipant,
withTestScheduler, withTestScheduler,
mockRtcMembership, mockRtcMembership,
MockRTCSession,
mockMediaDevices,
mockMuteStates,
mockConfig,
testScope, testScope,
mockLivekitRoom,
exampleTransport, exampleTransport,
} from "../../utils/test.ts"; } from "../../utils/test.ts";
import { E2eeType } from "../../e2ee/e2eeType.ts"; import { E2eeType } from "../../e2ee/e2eeType.ts";
import type { RaisedHandInfo, ReactionInfo } from "../../reactions/index.ts";
import { import {
aliceId, aliceId,
aliceParticipant, aliceParticipant,
@@ -70,10 +60,6 @@ import {
import { MediaDevices } from "../MediaDevices.ts"; import { MediaDevices } from "../MediaDevices.ts";
import { getValue } from "../../utils/observable.ts"; import { getValue } from "../../utils/observable.ts";
import { type Behavior, constant } from "../Behavior.ts"; import { type Behavior, constant } from "../Behavior.ts";
import {
type ElementCallError,
MatrixRTCTransportMissingError,
} from "../../utils/errors.ts";
import { withCallViewModel } from "./CallViewModelTestUtils.ts"; import { withCallViewModel } from "./CallViewModelTestUtils.ts";
vi.mock("rxjs", async (importOriginal) => ({ vi.mock("rxjs", async (importOriginal) => ({
@@ -244,71 +230,6 @@ function mockRingEvent(
const mockLegacyRingEvent = {} as { event_id: string } & ICallNotifyContent; const mockLegacyRingEvent = {} as { event_id: string } & ICallNotifyContent;
describe("CallViewModel", () => { describe("CallViewModel", () => {
// TODO: Restore this test. It requires makeTransport to not be mocked, unlike
// the rest of the tests in this file… what do we do?
it.skip("test missing RTC config error", async () => {
const rtcMemberships$ = new BehaviorSubject<CallMembership[]>([]);
const emitter = new EventEmitter();
const client = vi.mocked<MatrixClient>({
on: emitter.on.bind(emitter),
off: emitter.off.bind(emitter),
getSyncState: vi.fn().mockReturnValue(SyncState.Syncing),
getUserId: vi.fn().mockReturnValue("@user:localhost"),
getUser: vi.fn().mockReturnValue(null),
getDeviceId: vi.fn().mockReturnValue("DEVICE"),
credentials: {
userId: "@user:localhost",
},
getCrypto: vi.fn().mockReturnValue(undefined),
getDomain: vi.fn().mockReturnValue("example.org"),
} as unknown as MatrixClient);
const matrixRoom = mockMatrixRoom({
roomId: "!myRoomId:example.com",
client,
getMember: vi.fn().mockReturnValue(undefined),
});
const fakeRtcSession = new MockRTCSession(matrixRoom).withMemberships(
rtcMemberships$,
);
mockConfig({});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({});
const callVM = new CallViewModel(
testScope(),
fakeRtcSession.asMockedSession(),
matrixRoom,
mockMediaDevices({}),
mockMuteStates(),
{
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
autoLeaveWhenOthersLeft: false,
livekitRoomFactory: (): LivekitRoom =>
mockLivekitRoom({
localParticipant,
disconnect: async () => Promise.resolve(),
setE2EEEnabled: async () => Promise.resolve(),
}),
},
new BehaviorSubject({} as Record<string, RaisedHandInfo>),
new BehaviorSubject({} as Record<string, ReactionInfo>),
constant({ processor: undefined, supported: false }),
);
const failPromise = Promise.withResolvers<ElementCallError>();
callVM.configError$.subscribe((error) => {
if (error) {
failPromise.resolve(error);
}
});
const error = await failPromise.promise;
expect(error).toBeInstanceOf(MatrixRTCTransportMissingError);
});
test("participants are retained during a focus switch", () => { test("participants are retained during a focus switch", () => {
withTestScheduler(({ behavior, expectObservable }) => { withTestScheduler(({ behavior, expectObservable }) => {
// Participants disappear on frame 2 and come back on frame 3 // Participants disappear on frame 2 and come back on frame 3
@@ -346,7 +267,7 @@ describe("CallViewModel", () => {
}); });
}); });
it.skip("screen sharing activates spotlight layout", () => { test("screen sharing activates spotlight layout", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => { withTestScheduler(({ behavior, schedule, expectObservable }) => {
// Start with no screen shares, then have Alice and Bob share their screens, // Start with no screen shares, then have Alice and Bob share their screens,
// then return to no screen shares, then have just Alice share for a bit // then return to no screen shares, then have just Alice share for a bit
+210 -221
View File
@@ -1,4 +1,5 @@
/* /*
Copyright 2025 Element Creations Ltd.
Copyright 2023, 2024, 2025 New Vector Ltd. Copyright 2023, 2024, 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -12,11 +13,10 @@ import {
type Room as LivekitRoom, type Room as LivekitRoom,
type RoomOptions, type RoomOptions,
} from "livekit-client"; } from "livekit-client";
import { type RoomMember, type Room as MatrixRoom } from "matrix-js-sdk"; import { type Room as MatrixRoom } from "matrix-js-sdk";
import { import {
combineLatest, combineLatest,
distinctUntilChanged, distinctUntilChanged,
EMPTY,
filter, filter,
fromEvent, fromEvent,
map, map,
@@ -27,7 +27,6 @@ import {
pairwise, pairwise,
race, race,
scan, scan,
skip,
skipWhile, skipWhile,
startWith, startWith,
Subject, Subject,
@@ -40,7 +39,10 @@ import {
timer, timer,
} from "rxjs"; } from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger"; import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type MatrixRTCSession } from "matrix-js-sdk/lib/matrixrtc"; import {
type LivekitTransport,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { type IWidgetApiRequest } from "matrix-widget-api"; import { type IWidgetApiRequest } from "matrix-widget-api";
import { import {
@@ -93,9 +95,11 @@ import {
} from "../layout-types.ts"; } from "../layout-types.ts";
import { type ElementCallError } from "../../utils/errors.ts"; import { type ElementCallError } from "../../utils/errors.ts";
import { type ObservableScope } from "../ObservableScope.ts"; import { type ObservableScope } from "../ObservableScope.ts";
import { createHomeserverConnected$ } from "./localMember/HomeserverConnected.ts";
import { import {
createLocalMembership$, createLocalMembership$,
type LocalMemberConnectionState, enterRTCSession,
RTCBackendState,
} from "./localMember/LocalMembership.ts"; } from "./localMember/LocalMembership.ts";
import { createLocalTransport$ } from "./localMember/LocalTransport.ts"; import { createLocalTransport$ } from "./localMember/LocalTransport.ts";
import { import {
@@ -115,9 +119,13 @@ import {
createSentCallNotification$, createSentCallNotification$,
} from "./CallNotificationLifecycle.ts"; } from "./CallNotificationLifecycle.ts";
import { import {
createDMMember$,
createMatrixMemberMetadata$, createMatrixMemberMetadata$,
createRoomMembers$, createRoomMembers$,
} from "./remoteMembers/MatrixMemberMetadata.ts"; } from "./remoteMembers/MatrixMemberMetadata.ts";
import { Publisher } from "./localMember/Publisher.ts";
import { type Connection } from "./remoteMembers/Connection.ts";
import { createLayoutModeSwitch } from "./LayoutSwitch.ts";
const logger = rootLogger.getChild("[CallViewModel]"); const logger = rootLogger.getChild("[CallViewModel]");
//TODO //TODO
@@ -171,55 +179,56 @@ type AudioLivekitItem = {
}; };
/** /**
* A view model providing all the application logic needed to show the in-call * The return of createCallViewModel$
* UI (may eventually be expanded to cover the lobby and feedback screens in the * this interface represents the root source of data for the call view.
* future). * They are a list of observables and objects containing observables to allow for a very granular update mechanism.
*
* This allows to have one huge call view model that represents the entire view without a unnecessary amount of updates.
*
* (Mocking this interface should allow building a full view in all states.)
*/ */
// Throughout this class and related code we must distinguish between MatrixRTC export interface CallViewModel {
// state and LiveKit state. We use the common terminology of room "members", RTC
// "memberships", and LiveKit "participants".
export class CallViewModel {
// lifecycle // lifecycle
public autoLeave$: Observable<AutoLeaveReason>; autoLeave$: Observable<AutoLeaveReason>;
// TODO if we are in "unknown" state we need a loading rendering (or empty screen) // TODO if we are in "unknown" state we need a loading rendering (or empty screen)
// Otherwise it looks like we already connected and only than the ringing starts which is weird. // Otherwise it looks like we already connected and only than the ringing starts which is weird.
public callPickupState$: Behavior< callPickupState$: Behavior<
"unknown" | "ringing" | "timeout" | "decline" | "success" | null "unknown" | "ringing" | "timeout" | "decline" | "success" | null
>; >;
public leave$: Observable<"user" | AutoLeaveReason>; leave$: Observable<"user" | AutoLeaveReason>;
/** Call to initiate hangup. Use in conbination with connectino state track the async hangup process. */ /** Call to initiate hangup. Use in conbination with connectino state track the async hangup process. */
public hangup: () => void; hangup: () => void;
// joining // joining
public join: () => LocalMemberConnectionState; join: () => void;
// screen sharing // screen sharing
/** /**
* Callback to toggle screen sharing. If null, screen sharing is not possible. * Callback to toggle screen sharing. If null, screen sharing is not possible.
*/ */
public toggleScreenSharing: (() => void) | null; toggleScreenSharing: (() => void) | null;
/** /**
* Whether we are sharing our screen. * Whether we are sharing our screen.
*/ */
public sharingScreen$: Behavior<boolean>; sharingScreen$: Behavior<boolean>;
// UI interactions // UI interactions
/** /**
* Callback for when the user taps the call view. * Callback for when the user taps the call view.
*/ */
public tapScreen: () => void; tapScreen: () => void;
/** /**
* Callback for when the user taps the call's controls. * Callback for when the user taps the call's controls.
*/ */
public tapControls: () => void; tapControls: () => void;
/** /**
* Callback for when the user hovers over the call view. * Callback for when the user hovers over the call view.
*/ */
public hoverScreen: () => void; hoverScreen: () => void;
/** /**
* Callback for when the user stops hovering over the call view. * Callback for when the user stops hovering over the call view.
*/ */
public unhoverScreen: () => void; unhoverScreen: () => void;
// errors // errors
/** /**
@@ -227,7 +236,7 @@ export class CallViewModel {
* This is a fatal error that prevents the call from being created/joined. * This is a fatal error that prevents the call from being created/joined.
* Should render a blocking error screen. * Should render a blocking error screen.
*/ */
public configError$: Behavior<ElementCallError | null>; fatalError$: Behavior<ElementCallError | null>;
// participants and counts // participants and counts
/** /**
@@ -237,40 +246,43 @@ export class CallViewModel {
* - There can be multiple participants for one Matrix user if they join from * - There can be multiple participants for one Matrix user if they join from
* multiple devices. * multiple devices.
*/ */
public participantCount$: Behavior<number>; participantCount$: Behavior<number>;
/** Participants sorted by livekit room so they can be used in the audio rendering */ /** Participants sorted by livekit room so they can be used in the audio rendering */
public audioParticipants$: Behavior<AudioLivekitItem[]>; audioParticipants$: Behavior<AudioLivekitItem[]>;
/** List of participants raising their hand */ /** List of participants raising their hand */
public handsRaised$: Behavior<Record<string, RaisedHandInfo>>; handsRaised$: Behavior<Record<string, RaisedHandInfo>>;
/** List of reactions. Keys are: membership.membershipId (currently predefined as: `${membershipEvent.userId}:${membershipEvent.deviceId}`)*/ /** List of reactions. Keys are: membership.membershipId (currently predefined as: `${membershipEvent.userId}:${membershipEvent.deviceId}`)*/
public reactions$: Behavior<Record<string, ReactionOption>>; reactions$: Behavior<Record<string, ReactionOption>>;
public isOneOnOneWith$: Behavior<Pick<
RoomMember, ringOverlay$: Behavior<null | {
"userId" | "getMxcAvatarUrl" | "rawDisplayName" name: string;
> | null>; /** roomId or userId for the avatar generation. */
public localUserIsAlone$: Behavior<boolean>; idForAvatar: string;
text: string;
avatarMxc?: string;
}>;
// sounds and events // sounds and events
public joinSoundEffect$: Observable<void>; joinSoundEffect$: Observable<void>;
public leaveSoundEffect$: Observable<void>; leaveSoundEffect$: Observable<void>;
/** /**
* Emits an event every time a new hand is raised in * Emits an event every time a new hand is raised in
* the call. * the call.
*/ */
public newHandRaised$: Observable<{ value: number; playSounds: boolean }>; newHandRaised$: Observable<{ value: number; playSounds: boolean }>;
/** /**
* Emits an event every time a new screenshare is started in * Emits an event every time a new screenshare is started in
* the call. * the call.
*/ */
public newScreenShare$: Observable<{ value: number; playSounds: boolean }>; newScreenShare$: Observable<{ value: number; playSounds: boolean }>;
/** /**
* Emits an array of reactions that should be played. * Emits an array of reactions that should be played.
*/ */
public audibleReactions$: Observable<string[]>; audibleReactions$: Observable<string[]>;
/** /**
* Emits an array of reactions that should be visible on the screen. * Emits an array of reactions that should be visible on the screen.
*/ */
// DISCUSSION move this into a reaction file // DISCUSSION move this into a reaction file
public visibleReactions$: Behavior< visibleReactions$: Behavior<
{ sender: string; emoji: string; startX: number }[] { sender: string; emoji: string; startX: number }[]
>; >;
@@ -278,43 +290,43 @@ export class CallViewModel {
/** /**
* The general shape of the window. * The general shape of the window.
*/ */
public windowMode$: Behavior<WindowMode>; windowMode$: Behavior<WindowMode>;
public spotlightExpanded$: Behavior<boolean>; spotlightExpanded$: Behavior<boolean>;
public toggleSpotlightExpanded$: Behavior<(() => void) | null>; toggleSpotlightExpanded$: Behavior<(() => void) | null>;
public gridMode$: Behavior<GridMode>; gridMode$: Behavior<GridMode>;
public setGridMode: (value: GridMode) => void; setGridMode: (value: GridMode) => void;
// media view models and layout // media view models and layout
public grid$: Behavior<UserMediaViewModel[]>; grid$: Behavior<UserMediaViewModel[]>;
public spotlight$: Behavior<MediaViewModel[]>; spotlight$: Behavior<MediaViewModel[]>;
public pip$: Behavior<UserMediaViewModel | null>; pip$: Behavior<UserMediaViewModel | null>;
/** /**
* The layout of tiles in the call interface. * The layout of tiles in the call interface.
*/ */
public layout$: Behavior<Layout>; layout$: Behavior<Layout>;
/** /**
* The current generation of the tile store, exposed for debugging purposes. * The current generation of the tile store, exposed for debugging purposes.
*/ */
public tileStoreGeneration$: Behavior<number>; tileStoreGeneration$: Behavior<number>;
public showSpotlightIndicators$: Behavior<boolean>; showSpotlightIndicators$: Behavior<boolean>;
public showSpeakingIndicators$: Behavior<boolean>; showSpeakingIndicators$: Behavior<boolean>;
// header/footer visibility // header/footer visibility
public showHeader$: Behavior<boolean>; showHeader$: Behavior<boolean>;
public showFooter$: Behavior<boolean>; showFooter$: Behavior<boolean>;
// audio routing // audio routing
/** /**
* Whether audio is currently being output through the earpiece. * Whether audio is currently being output through the earpiece.
*/ */
public earpieceMode$: Behavior<boolean>; earpieceMode$: Behavior<boolean>;
/** /**
* Callback to toggle between the earpiece and the loudspeaker. * Callback to toggle between the earpiece and the loudspeaker.
* *
* This will be `null` in case the target does not exist in the list * This will be `null` in case the target does not exist in the list
* of available audio outputs. * of available audio outputs.
*/ */
public audioOutputSwitcher$: Behavior<{ audioOutputSwitcher$: Behavior<{
targetOutput: "earpiece" | "speaker"; targetOutput: "earpiece" | "speaker";
switch: () => void; switch: () => void;
} | null>; } | null>;
@@ -329,10 +341,18 @@ export class CallViewModel {
// down, for example, and we want to avoid making people worry that the app is // down, for example, and we want to avoid making people worry that the app is
// in a split-brained state. // in a split-brained state.
// DISCUSSION own membership manager ALSO this probably can be simplifis // DISCUSSION own membership manager ALSO this probably can be simplifis
public reconnecting$: Behavior<boolean>; reconnecting$: Behavior<boolean>;
}
// THIS has to be the last public field declaration /**
public constructor( * A view model providing all the application logic needed to show the in-call
* UI (may eventually be expanded to cover the lobby and feedback screens in the
* future).
*/
// Throughout this class and related code we must distinguish between MatrixRTC
// state and LiveKit state. We use the common terminology of room "members", RTC
// "memberships", and LiveKit "participants".
export function createCallViewModel$(
scope: ObservableScope, scope: ObservableScope,
// A call is permanently tied to a single Matrix room // A call is permanently tied to a single Matrix room
matrixRTCSession: MatrixRTCSession, matrixRTCSession: MatrixRTCSession,
@@ -343,10 +363,10 @@ export class CallViewModel {
handsRaisedSubject$: Observable<Record<string, RaisedHandInfo>>, handsRaisedSubject$: Observable<Record<string, RaisedHandInfo>>,
reactionsSubject$: Observable<Record<string, ReactionInfo>>, reactionsSubject$: Observable<Record<string, ReactionInfo>>,
trackProcessorState$: Behavior<ProcessorState>, trackProcessorState$: Behavior<ProcessorState>,
) { ): CallViewModel {
const userId = matrixRoom.client.getUserId()!; const client = matrixRoom.client;
const deviceId = matrixRoom.client.getDeviceId()!; const userId = client.getUserId()!;
const deviceId = client.getDeviceId()!;
const livekitKeyProvider = getE2eeKeyProvider( const livekitKeyProvider = getE2eeKeyProvider(
options.encryptionSystem, options.encryptionSystem,
matrixRTCSession, matrixRTCSession,
@@ -358,8 +378,8 @@ export class CallViewModel {
// For mocking purposes it is recommended to only mock the functions creating those outputs. // For mocking purposes it is recommended to only mock the functions creating those outputs.
// All other fields are just temp computations for the mentioned output. // All other fields are just temp computations for the mentioned output.
// The class does not need anything except the values underneath the bar. // The class does not need anything except the values underneath the bar.
// The creation of the values under the bar are all tested independently and testing the callViewModel Should // The creations of the values under the bar are all tested independently and testing the callViewModel Should
// not test their cretation. Call view model only needs: // not test their creation. Call view model only needs:
// - memberships$ via createMemberships$ // - memberships$ via createMemberships$
// - localMembership via createLocalMembership$ // - localMembership via createLocalMembership$
// - callLifecycle via createCallNotificationLifecycle$ // - callLifecycle via createCallNotificationLifecycle$
@@ -380,7 +400,7 @@ export class CallViewModel {
const localTransport$ = createLocalTransport$({ const localTransport$ = createLocalTransport$({
scope: scope, scope: scope,
memberships$: memberships$, memberships$: memberships$,
client: matrixRoom.client, client,
roomId: matrixRoom.roomId, roomId: matrixRoom.roomId,
useOldestMember$: scope.behavior( useOldestMember$: scope.behavior(
matrixRTCMode.value$.pipe(map((v) => v === MatrixRTCMode.Legacy)), matrixRTCMode.value$.pipe(map((v) => v === MatrixRTCMode.Legacy)),
@@ -388,12 +408,14 @@ export class CallViewModel {
}); });
const connectionFactory = new ECConnectionFactory( const connectionFactory = new ECConnectionFactory(
matrixRoom.client, client,
mediaDevices, mediaDevices,
trackProcessorState$, trackProcessorState$,
livekitKeyProvider, livekitKeyProvider,
getUrlParams().controlledAudioDevices, getUrlParams().controlledAudioDevices,
options.livekitRoomFactory, options.livekitRoomFactory,
getUrlParams().echoCancellation,
getUrlParams().noiseSuppression,
); );
const connectionManager = createConnectionManager$({ const connectionManager = createConnectionManager$({
@@ -403,9 +425,7 @@ export class CallViewModel {
combineLatest( combineLatest(
[localTransport$, membershipsAndTransports.transports$], [localTransport$, membershipsAndTransports.transports$],
(localTransport, transports) => { (localTransport, transports) => {
const localTransportAsArray = localTransport const localTransportAsArray = localTransport ? [localTransport] : [];
? [localTransport]
: [];
return transports.mapInner((transports) => [ return transports.mapInner((transports) => [
...localTransportAsArray, ...localTransportAsArray,
...transports, ...transports,
@@ -427,7 +447,7 @@ export class CallViewModel {
matrixRTCMode.value$.pipe( matrixRTCMode.value$.pipe(
map((mode) => ({ map((mode) => ({
encryptMedia: livekitKeyProvider !== undefined, encryptMedia: livekitKeyProvider !== undefined,
// TODO. This might need to get called again on each cahnge of matrixRTCMode... // TODO. This might need to get called again on each change of matrixRTCMode...
matrixRTCMode: mode, matrixRTCMode: mode,
})), })),
), ),
@@ -435,15 +455,34 @@ export class CallViewModel {
const localMembership = createLocalMembership$({ const localMembership = createLocalMembership$({
scope: scope, scope: scope,
homeserverConnected$: createHomeserverConnected$(
scope,
client,
matrixRTCSession,
),
muteStates: muteStates, muteStates: muteStates,
mediaDevices: mediaDevices, joinMatrixRTC: async (transport: LivekitTransport) => {
return enterRTCSession(
matrixRTCSession,
transport,
connectOptions$.value,
);
},
createPublisherFactory: (connection: Connection) => {
return new Publisher(
scope,
connection,
mediaDevices,
muteStates,
trackProcessorState$,
logger.getChild(
"[Publisher" + connection.transport.livekit_service_url + "]",
),
);
},
connectionManager: connectionManager, connectionManager: connectionManager,
matrixRTCSession: matrixRTCSession, matrixRTCSession: matrixRTCSession,
matrixRoom: matrixRoom,
localTransport$: localTransport$, localTransport$: localTransport$,
trackProcessorState$: trackProcessorState$,
widget,
options: connectOptions$,
logger: logger.getChild(`[${Date.now()}]`), logger: logger.getChild(`[${Date.now()}]`),
}); });
@@ -453,8 +492,7 @@ export class CallViewModel {
(memberships) => (memberships) =>
memberships.value.find( memberships.value.find(
(membership) => (membership) =>
membership.userId === userId && membership.userId === userId && membership.deviceId === deviceId,
membership.deviceId === deviceId,
) ?? null, ) ?? null,
), ),
), ),
@@ -483,13 +521,12 @@ export class CallViewModel {
// ------------------------------------------------------------------------ // ------------------------------------------------------------------------
// callLifecycle // callLifecycle
const callLifecycle = createCallNotificationLifecycle$({ // TODO if we are in "unknown" state we need a loading rendering (or empty screen)
// Otherwise it looks like we already connected and only than the ringing starts which is weird.
const { callPickupState$, autoLeave$ } = createCallNotificationLifecycle$({
scope: scope, scope: scope,
memberships$: memberships$, memberships$: memberships$,
sentCallNotification$: createSentCallNotification$( sentCallNotification$: createSentCallNotification$(scope, matrixRTCSession),
scope,
matrixRTCSession,
),
receivedDecline$: createReceivedDecline$(matrixRoom), receivedDecline$: createReceivedDecline$(matrixRoom),
options: options, options: options,
localUser: { userId: userId, deviceId: deviceId }, localUser: { userId: userId, deviceId: deviceId },
@@ -505,38 +542,39 @@ export class CallViewModel {
matrixRoomMembers$, matrixRoomMembers$,
); );
/** const dmMember$ = createDMMember$(scope, matrixRoomMembers$, matrixRoom);
* Returns the Member {userId, getMxcAvatarUrl, rawDisplayName} of the other user in the call, if it's a one-on-one call. const noUserToCallInRoom$ = scope.behavior(
*/
const isOneOnOneWith$ = scope.behavior(
matrixRoomMembers$.pipe(
map((roomMembersMap) => {
const otherMembers = Array.from(roomMembersMap.values()).filter(
(member) => member.userId !== userId,
);
return otherMembers.length === 1 ? otherMembers[0] : null;
}),
),
);
const localUserIsAlone$ = scope.behavior(
matrixRoomMembers$.pipe( matrixRoomMembers$.pipe(
map( map(
(roomMembersMap) => (roomMembersMap) =>
roomMembersMap.size === 1 && roomMembersMap.size === 1 && roomMembersMap.get(userId) !== undefined,
roomMembersMap.get(userId) !== undefined,
), ),
), ),
); );
// CODESMELL? const ringOverlay$ = scope.behavior(
// This is functionally the same Observable as leave$, except here it's combineLatest([noUserToCallInRoom$, dmMember$, callPickupState$]).pipe(
// hoisted to the top of the class. This enables the cyclic dependency between map(([noUserToCallInRoom, dmMember, callPickupState]) => {
// leave$ -> autoLeave$ -> callPickupState$ -> livekitConnectionState$ -> // No overlay if not in ringing state
// localConnection$ -> transports$ -> joined$ -> leave$. if (callPickupState !== "ringing" || noUserToCallInRoom) return null;
const leaveHoisted$ = new Subject<
"user" | "timeout" | "decline" | "allOthersLeft" const name = dmMember ? dmMember.rawDisplayName : matrixRoom.name;
>(); const id = dmMember ? dmMember.userId : matrixRoom.roomId;
const text = dmMember
? `Waiting for ${name} to join…`
: "Waiting for other participants…";
const avatarMxc = dmMember
? (dmMember.getMxcAvatarUrl?.() ?? undefined)
: (matrixRoom.getMxcAvatarUrl() ?? undefined);
return {
name: name ?? id,
idForAvatar: id,
text,
avatarMxc,
};
}),
),
);
/** /**
* Whether various media/event sources should pretend to be disconnected from * Whether various media/event sources should pretend to be disconnected from
@@ -548,7 +586,6 @@ export class CallViewModel {
// in a split-brained state. // in a split-brained state.
// DISCUSSION own membership manager ALSO this probably can be simplifis // DISCUSSION own membership manager ALSO this probably can be simplifis
const reconnecting$ = localMembership.reconnecting$; const reconnecting$ = localMembership.reconnecting$;
const pretendToBeDisconnected$ = reconnecting$;
const audioParticipants$ = scope.behavior( const audioParticipants$ = scope.behavior(
matrixLivekitMembers$.pipe( matrixLivekitMembers$.pipe(
@@ -597,7 +634,7 @@ export class CallViewModel {
); );
const handsRaised$ = scope.behavior( const handsRaised$ = scope.behavior(
handsRaisedSubject$.pipe(pauseWhen(pretendToBeDisconnected$)), handsRaisedSubject$.pipe(pauseWhen(reconnecting$)),
); );
const reactions$ = scope.behavior( const reactions$ = scope.behavior(
@@ -610,14 +647,13 @@ export class CallViewModel {
]), ]),
), ),
), ),
pauseWhen(pretendToBeDisconnected$), pauseWhen(reconnecting$),
), ),
); );
/** /**
* List of user media (camera feeds) that we want tiles for. * List of user media (camera feeds) that we want tiles for.
*/ */
// TODO this also needs the local participant to be added.
const userMedia$ = scope.behavior<UserMedia[]>( const userMedia$ = scope.behavior<UserMedia[]>(
combineLatest([ combineLatest([
localMatrixLivekitMember$, localMatrixLivekitMember$,
@@ -632,7 +668,7 @@ export class CallViewModel {
{ value: matrixLivekitMembers }, { value: matrixLivekitMembers },
duplicateTiles, duplicateTiles,
]) { ]) {
let localParticipantId = undefined; let localParticipantId: string | undefined = undefined;
// add local member if available // add local member if available
if (localMatrixLivekitMember) { if (localMatrixLivekitMember) {
const { userId, participant$, connection$, membership$ } = const { userId, participant$, connection$, membership$ } =
@@ -702,7 +738,7 @@ export class CallViewModel {
livekitRoom$, livekitRoom$,
focusUrl$, focusUrl$,
mediaDevices, mediaDevices,
pretendToBeDisconnected$, reconnecting$,
displayName$, displayName$,
matrixMemberMetadataStore.createAvatarUrlBehavior$(userId), matrixMemberMetadataStore.createAvatarUrlBehavior$(userId),
handsRaised$.pipe(map((v) => v[participantId]?.time ?? null)), handsRaised$.pipe(map((v) => v[participantId]?.time ?? null)),
@@ -763,15 +799,7 @@ export class CallViewModel {
matrixLivekitMembers$.pipe(map((ms) => ms.value.length)), matrixLivekitMembers$.pipe(map((ms) => ms.value.length)),
); );
// only public to expose to the view. const leaveSoundEffect$ = combineLatest([callPickupState$, userMedia$]).pipe(
// TODO if we are in "unknown" state we need a loading rendering (or empty screen)
// Otherwise it looks like we already connected and only than the ringing starts which is weird.
const callPickupState$ = callLifecycle.callPickupState$;
const leaveSoundEffect$ = combineLatest([
callLifecycle.callPickupState$,
userMedia$,
]).pipe(
// Until the call is successful, do not play a leave sound. // Until the call is successful, do not play a leave sound.
// If callPickupState$ is null, then we always play the sound as it will not conflict with a decline sound. // If callPickupState$ is null, then we always play the sound as it will not conflict with a decline sound.
skipWhile(([c]) => c !== null && c !== "success"), skipWhile(([c]) => c !== null && c !== "success"),
@@ -804,12 +832,9 @@ export class CallViewModel {
const leave$: Observable<"user" | "timeout" | "decline" | "allOthersLeft"> = const leave$: Observable<"user" | "timeout" | "decline" | "allOthersLeft"> =
merge( merge(
callLifecycle.autoLeave$, autoLeave$,
merge(userHangup$, widgetHangup$).pipe(map(() => "user" as const)), merge(userHangup$, widgetHangup$).pipe(map(() => "user" as const)),
).pipe( ).pipe(scope.share);
scope.share,
tap((reason) => leaveHoisted$.next(reason)),
);
const spotlightSpeaker$ = scope.behavior<UserMediaViewModel | null>( const spotlightSpeaker$ = scope.behavior<UserMediaViewModel | null>(
userMedia$.pipe( userMedia$.pipe(
@@ -857,9 +882,7 @@ export class CallViewModel {
return bins.length === 0 return bins.length === 0
? of([]) ? of([])
: combineLatest(bins, (...bins) => : combineLatest(bins, (...bins) =>
bins bins.sort(([, bin1], [, bin2]) => bin1 - bin2).map(([m]) => m.vm),
.sort(([, bin1], [, bin2]) => bin1 - bin2)
.map(([m]) => m.vm),
); );
}), }),
distinctUntilChanged(shallowEquals), distinctUntilChanged(shallowEquals),
@@ -960,37 +983,12 @@ export class CallViewModel {
spotlightExpandedToggle$.pipe(accumulate(false, (expanded) => !expanded)), spotlightExpandedToggle$.pipe(accumulate(false, (expanded) => !expanded)),
); );
const gridModeUserSelection$ = new Subject<GridMode>(); const { setGridMode, gridMode$ } = createLayoutModeSwitch(
/** scope,
* The layout mode of the media tile grid. windowMode$,
*/ hasRemoteScreenShares$,
const gridMode$ =
// If the user hasn't selected spotlight and somebody starts screen sharing,
// automatically switch to spotlight mode and reset when screen sharing ends
scope.behavior<GridMode>(
gridModeUserSelection$.pipe(
switchMap((userSelection) =>
(userSelection === "spotlight"
? EMPTY
: combineLatest([hasRemoteScreenShares$, windowMode$]).pipe(
skip(userSelection === null ? 0 : 1),
map(
([hasScreenShares, windowMode]): GridMode =>
hasScreenShares || windowMode === "flat"
? "spotlight"
: "grid",
),
)
).pipe(startWith(userSelection ?? "grid")),
),
),
"grid",
); );
const setGridMode = (value: GridMode): void => {
gridModeUserSelection$.next(value);
};
const gridLayoutMedia$: Observable<GridLayoutMedia> = combineLatest( const gridLayoutMedia$: Observable<GridLayoutMedia> = combineLatest(
[grid$, spotlight$], [grid$, spotlight$],
(grid, spotlight) => ({ (grid, spotlight) => ({
@@ -1080,9 +1078,7 @@ export class CallViewModel {
oneOnOne === null oneOnOne === null
? combineLatest([grid$, spotlight$], (grid, spotlight) => ? combineLatest([grid$, spotlight$], (grid, spotlight) =>
grid.length > smallMobileCallThreshold || grid.length > smallMobileCallThreshold ||
spotlight.some( spotlight.some((vm) => vm instanceof ScreenShareViewModel)
(vm) => vm instanceof ScreenShareViewModel,
)
? spotlightPortraitLayoutMedia$ ? spotlightPortraitLayoutMedia$
: gridLayoutMedia$, : gridLayoutMedia$,
).pipe(switchAll()) ).pipe(switchAll())
@@ -1118,9 +1114,7 @@ export class CallViewModel {
const visibleTiles$ = new Subject<number>(); const visibleTiles$ = new Subject<number>();
const setVisibleTiles = (value: number): void => visibleTiles$.next(value); const setVisibleTiles = (value: number): void => visibleTiles$.next(value);
const layoutInternals$ = scope.behavior< const layoutInternals$ = scope.behavior<LayoutScanState & { layout: Layout }>(
LayoutScanState & { layout: Layout }
>(
combineLatest([ combineLatest([
layoutMedia$, layoutMedia$,
visibleTiles$.pipe(startWith(0), distinctUntilChanged()), visibleTiles$.pipe(startWith(0), distinctUntilChanged()),
@@ -1297,10 +1291,7 @@ export class CallViewModel {
*/ */
const earpieceMode$ = scope.behavior<boolean>( const earpieceMode$ = scope.behavior<boolean>(
combineLatest( combineLatest(
[ [mediaDevices.audioOutput.available$, mediaDevices.audioOutput.selected$],
mediaDevices.audioOutput.available$,
mediaDevices.audioOutput.selected$,
],
(available, selected) => (available, selected) =>
selected !== undefined && selected !== undefined &&
available.get(selected.id)?.type === "earpiece", available.get(selected.id)?.type === "earpiece",
@@ -1318,10 +1309,7 @@ export class CallViewModel {
switch: () => void; switch: () => void;
} | null>( } | null>(
combineLatest( combineLatest(
[ [mediaDevices.audioOutput.available$, mediaDevices.audioOutput.selected$],
mediaDevices.audioOutput.available$,
mediaDevices.audioOutput.selected$,
],
(available, selected) => { (available, selected) => {
const selectionType = selected && available.get(selected.id)?.type; const selectionType = selected && available.get(selected.id)?.type;
@@ -1354,8 +1342,7 @@ export class CallViewModel {
Record<string, ReactionOption>, Record<string, ReactionOption>,
{ sender: string; emoji: string; startX: number }[] { sender: string; emoji: string; startX: number }[]
>((acc, latest) => { >((acc, latest) => {
const newSet: { sender: string; emoji: string; startX: number }[] = const newSet: { sender: string; emoji: string; startX: number }[] = [];
[];
for (const [sender, reaction] of Object.entries(latest)) { for (const [sender, reaction] of Object.entries(latest)) {
const startX = const startX =
acc.find((v) => v.sender === sender && v.emoji)?.startX ?? acc.find((v) => v.sender === sender && v.emoji)?.startX ??
@@ -1427,58 +1414,60 @@ export class CallViewModel {
// reassigned here to make it publicly accessible // reassigned here to make it publicly accessible
const toggleScreenSharing = localMembership.toggleScreenSharing; const toggleScreenSharing = localMembership.toggleScreenSharing;
const join = localMembership.requestConnect; return {
join(); // TODO-MULTI-SFU: Use this view model for the lobby as well, and only call this once 'join' is clicked? autoLeave$: autoLeave$,
callPickupState$: callPickupState$,
ringOverlay$: ringOverlay$,
leave$: leave$,
hangup: (): void => userHangup$.next(),
join: localMembership.requestConnect,
toggleScreenSharing: toggleScreenSharing,
sharingScreen$: sharingScreen$,
this.autoLeave$ = callLifecycle.autoLeave$; tapScreen: (): void => screenTap$.next(),
this.callPickupState$ = callPickupState$; tapControls: (): void => controlsTap$.next(),
this.leave$ = leave$; hoverScreen: (): void => screenHover$.next(),
this.hangup = (): void => userHangup$.next(); unhoverScreen: (): void => screenUnhover$.next(),
this.join = join;
this.toggleScreenSharing = toggleScreenSharing;
this.sharingScreen$ = sharingScreen$;
this.tapScreen = (): void => screenTap$.next(); fatalError$: scope.behavior(
this.tapControls = (): void => controlsTap$.next(); localMembership.connectionState.livekit$.pipe(
this.hoverScreen = (): void => screenHover$.next(); filter((v) => v.state === RTCBackendState.Error),
this.unhoverScreen = (): void => screenUnhover$.next(); map((s) => s.error),
),
null,
),
this.configError$ = localMembership.configError$; participantCount$: participantCount$,
this.participantCount$ = participantCount$; audioParticipants$: audioParticipants$,
this.audioParticipants$ = audioParticipants$;
this.isOneOnOneWith$ = isOneOnOneWith$;
this.localUserIsAlone$ = localUserIsAlone$;
this.handsRaised$ = handsRaised$; handsRaised$: handsRaised$,
this.reactions$ = reactions$; reactions$: reactions$,
this.joinSoundEffect$ = joinSoundEffect$; joinSoundEffect$: joinSoundEffect$,
this.leaveSoundEffect$ = leaveSoundEffect$; leaveSoundEffect$: leaveSoundEffect$,
this.newHandRaised$ = newHandRaised$; newHandRaised$: newHandRaised$,
this.newScreenShare$ = newScreenShare$; newScreenShare$: newScreenShare$,
this.audibleReactions$ = audibleReactions$; audibleReactions$: audibleReactions$,
this.visibleReactions$ = visibleReactions$; visibleReactions$: visibleReactions$,
this.windowMode$ = windowMode$; windowMode$: windowMode$,
this.spotlightExpanded$ = spotlightExpanded$; spotlightExpanded$: spotlightExpanded$,
this.toggleSpotlightExpanded$ = toggleSpotlightExpanded$; toggleSpotlightExpanded$: toggleSpotlightExpanded$,
this.gridMode$ = gridMode$; gridMode$: gridMode$,
this.setGridMode = setGridMode; setGridMode: setGridMode,
this.grid$ = grid$; grid$: grid$,
this.spotlight$ = spotlight$; spotlight$: spotlight$,
this.pip$ = pip$; pip$: pip$,
this.layout$ = layout$; layout$: layout$,
this.tileStoreGeneration$ = tileStoreGeneration$; tileStoreGeneration$: tileStoreGeneration$,
this.showSpotlightIndicators$ = showSpotlightIndicators$; showSpotlightIndicators$: showSpotlightIndicators$,
this.showSpeakingIndicators$ = showSpeakingIndicators$; showSpeakingIndicators$: showSpeakingIndicators$,
this.showHeader$ = showHeader$; showHeader$: showHeader$,
this.showFooter$ = showFooter$; showFooter$: showFooter$,
this.earpieceMode$ = earpieceMode$; earpieceMode$: earpieceMode$,
this.audioOutputSwitcher$ = audioOutputSwitcher$; audioOutputSwitcher$: audioOutputSwitcher$,
this.reconnecting$ = reconnecting$; reconnecting$: reconnecting$,
} };
} }
// TODO-MULTI-SFU // Setup and update the keyProvider which was create by `createRoom` was a thing before. Now we never update if the E2EEsystem changes
// do we need this?
function getE2eeKeyProvider( function getE2eeKeyProvider(
e2eeSystem: EncryptionSystem, e2eeSystem: EncryptionSystem,
@@ -24,7 +24,11 @@ import * as ComponentsCore from "@livekit/components-core";
import type { CallMembership } from "matrix-js-sdk/lib/matrixrtc"; import type { CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { E2eeType } from "../../e2ee/e2eeType"; import { E2eeType } from "../../e2ee/e2eeType";
import { type RaisedHandInfo, type ReactionInfo } from "../../reactions"; import { type RaisedHandInfo, type ReactionInfo } from "../../reactions";
import { CallViewModel, type CallViewModelOptions } from "./CallViewModel"; import {
type CallViewModel,
createCallViewModel$,
type CallViewModelOptions,
} from "./CallViewModel";
import { import {
mockConfig, mockConfig,
mockLivekitRoom, mockLivekitRoom,
@@ -122,6 +126,7 @@ export function withCallViewModel(
} }
})() as Partial<MatrixClient> as MatrixClient, })() as Partial<MatrixClient> as MatrixClient,
getMembers: () => Array.from(roomMembers.values()), getMembers: () => Array.from(roomMembers.values()),
getMembersWithMembership: () => Array.from(roomMembers.values()),
}); });
const rtcSession = new MockRTCSession(room, []).withMemberships(rtcMembers$); const rtcSession = new MockRTCSession(room, []).withMemberships(rtcMembers$);
const participantsSpy = vi const participantsSpy = vi
@@ -153,7 +158,7 @@ export function withCallViewModel(
const raisedHands$ = new BehaviorSubject<Record<string, RaisedHandInfo>>({}); const raisedHands$ = new BehaviorSubject<Record<string, RaisedHandInfo>>({});
const reactions$ = new BehaviorSubject<Record<string, ReactionInfo>>({}); const reactions$ = new BehaviorSubject<Record<string, ReactionInfo>>({});
const vm = new CallViewModel( const vm = createCallViewModel$(
testScope(), testScope(),
rtcSession.asMockedSession(), rtcSession.asMockedSession(),
room, room,
@@ -0,0 +1,202 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, test } from "vitest";
import { firstValueFrom, of } from "rxjs";
import { createLayoutModeSwitch } from "./LayoutSwitch";
import { ObservableScope } from "../ObservableScope";
import { constant } from "../Behavior";
import { withTestScheduler } from "../../utils/test";
let scope: ObservableScope;
beforeEach(() => {
scope = new ObservableScope();
});
afterEach(() => {
scope.end();
});
describe("Default mode", () => {
test("Should be in grid layout by default", async () => {
const { gridMode$ } = createLayoutModeSwitch(
scope,
constant("normal"),
of(false),
);
const mode = await firstValueFrom(gridMode$);
expect(mode).toBe("grid");
});
test("Should switch to spotlight mode when window mode is flat", async () => {
const { gridMode$ } = createLayoutModeSwitch(
scope,
constant("flat"),
of(false),
);
const mode = await firstValueFrom(gridMode$);
expect(mode).toBe("spotlight");
});
});
test("Should allow switching modes manually", () => {
withTestScheduler(({ cold, behavior, expectObservable, schedule }): void => {
const { gridMode$, setGridMode } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold("f", { f: false, t: true }),
);
schedule("--sgs", {
s: () => setGridMode("spotlight"),
g: () => setGridMode("grid"),
});
expectObservable(gridMode$).toBe("g-sgs", {
g: "grid",
s: "spotlight",
});
});
});
test("Should switch to spotlight mode when there is a remote screen share", () => {
withTestScheduler(({ cold, behavior, expectObservable }): void => {
const shareMarble = "f--t";
const gridsMarble = "g--s";
const { gridMode$ } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold(shareMarble, { f: false, t: true }),
);
expectObservable(gridMode$).toBe(gridsMarble, {
g: "grid",
s: "spotlight",
});
});
});
test("Can manually force grid when there is a screenshare", () => {
withTestScheduler(({ cold, behavior, expectObservable, schedule }): void => {
const { gridMode$, setGridMode } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold("-ft", { f: false, t: true }),
);
schedule("---g", {
g: () => setGridMode("grid"),
});
expectObservable(gridMode$).toBe("ggsg", {
g: "grid",
s: "spotlight",
});
});
});
test("Should auto-switch after manually selected grid", () => {
withTestScheduler(({ cold, behavior, expectObservable, schedule }): void => {
const { gridMode$, setGridMode } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
// Two screenshares will happen in sequence
cold("-ft-ft", { f: false, t: true }),
);
// There was a screen-share that forced spotlight, then
// the user manually switch back to grid
schedule("---g", {
g: () => setGridMode("grid"),
});
// If we did want to respect manual selection, the expectation would be:
// const expectation = "ggsg";
const expectation = "ggsg-s";
expectObservable(gridMode$).toBe(expectation, {
g: "grid",
s: "spotlight",
});
});
});
test("Should switch back to grid mode when the remote screen share ends", () => {
withTestScheduler(({ cold, behavior, expectObservable }): void => {
const shareMarble = "f--t--f-";
const gridsMarble = "g--s--g-";
const { gridMode$ } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold(shareMarble, { f: false, t: true }),
);
expectObservable(gridMode$).toBe(gridsMarble, {
g: "grid",
s: "spotlight",
});
});
});
test("can auto-switch to spotlight again after first screen share ends", () => {
withTestScheduler(({ cold, behavior, expectObservable }): void => {
const shareMarble = "ftft";
const gridsMarble = "gsgs";
const { gridMode$ } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold(shareMarble, { f: false, t: true }),
);
expectObservable(gridMode$).toBe(gridsMarble, {
g: "grid",
s: "spotlight",
});
});
});
test("can switch manually to grid after screen share while manually in spotlight", () => {
withTestScheduler(({ cold, behavior, schedule, expectObservable }): void => {
// Initially, no one is sharing. Then the user manually switches to
// spotlight. After a screen share starts, the user manually switches to
// grid.
const shareMarbles = " f-t-";
const setModeMarbles = "-s-g";
const expectation = " gs-g";
const { gridMode$, setGridMode } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold(shareMarbles, { f: false, t: true }),
);
schedule(setModeMarbles, {
g: () => setGridMode("grid"),
s: () => setGridMode("spotlight"),
});
expectObservable(gridMode$).toBe(expectation, {
g: "grid",
s: "spotlight",
});
});
});
test("Should auto-switch to spotlight when in flat window mode", () => {
withTestScheduler(({ cold, behavior, expectObservable }): void => {
const { gridMode$ } = createLayoutModeSwitch(
scope,
behavior("naf", { n: "normal", a: "narrow", f: "flat" }),
cold("f", { f: false, t: true }),
);
expectObservable(gridMode$).toBe("g-s-", {
g: "grid",
s: "spotlight",
});
});
});
+130
View File
@@ -0,0 +1,130 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
BehaviorSubject,
combineLatest,
map,
type Observable,
scan,
} from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { type GridMode, type WindowMode } from "./CallViewModel.ts";
import { type Behavior } from "../Behavior.ts";
import { type ObservableScope } from "../ObservableScope.ts";
/**
* Creates a layout mode switch that allows switching between grid and spotlight modes.
* The actual layout mode can be overridden to spotlight mode if there is a remote screen share active
* or if the window mode is flat.
*
* @param scope - The observable scope to manage subscriptions.
* @param windowMode$ - The current window mode observable.
* @param hasRemoteScreenShares$ - An observable indicating if there are remote screen shares active.
*/
export function createLayoutModeSwitch(
scope: ObservableScope,
windowMode$: Behavior<WindowMode>,
hasRemoteScreenShares$: Observable<boolean>,
): {
gridMode$: Behavior<GridMode>;
setGridMode: (value: GridMode) => void;
} {
const gridModeUserSelection$ = new BehaviorSubject<GridMode>("grid");
// Callback to set the grid mode desired by the user.
// Notice that this is only a preference, the actual grid mode can be overridden
// if there is a remote screen share active.
const setGridMode = (value: GridMode): void => {
gridModeUserSelection$.next(value);
};
/**
* The layout mode of the media tile grid.
*/
const gridMode$ =
// If the user hasn't selected spotlight and somebody starts screen sharing,
// automatically switch to spotlight mode and reset when screen sharing ends
scope.behavior<GridMode>(
combineLatest([
gridModeUserSelection$,
hasRemoteScreenShares$,
windowMode$,
]).pipe(
// Scan to keep track if we have auto-switched already or not.
// To allow the user to override the auto-switch by selecting grid mode again.
scan<
[GridMode, boolean, WindowMode],
{
mode: GridMode;
/** Remember if the change was user driven or not */
hasAutoSwitched: boolean;
/** To know if it is new screen share or an already handled */
hasScreenShares: boolean;
}
>(
(prev, [userSelection, hasScreenShares, windowMode]) => {
const isFlatMode = windowMode === "flat";
// Always force spotlight in flat mode, grid layout is not supported
// in that mode.
// TODO: strange that we do that for flat mode but not for other modes?
// TODO: Why is this not handled in layoutMedia$ like other window modes?
if (isFlatMode) {
logger.debug(`Forcing spotlight mode, windowMode=${windowMode}`);
return {
mode: "spotlight",
hasAutoSwitched: prev.hasAutoSwitched,
hasScreenShares,
};
}
// User explicitly chose spotlight.
// Respect that choice.
if (userSelection === "spotlight") {
return {
mode: "spotlight",
hasAutoSwitched: prev.hasAutoSwitched,
hasScreenShares,
};
}
// User has chosen grid mode. If a screen share starts, we will
// auto-switch to spotlight mode for better experience.
// But we only do it once, if the user switches back to grid mode,
// we respect that choice until they explicitly change it again.
const isNewShare = hasScreenShares && !prev.hasScreenShares;
if (isNewShare && !prev.hasAutoSwitched) {
return {
mode: "spotlight",
hasAutoSwitched: true,
hasScreenShares: true,
};
}
// Respect user's grid choice
// XXX If we want to forbid switching automatically again after we can
// return hasAutoSwitched: acc.hasAutoSwitched here instead of setting to false.
return {
mode: "grid",
hasAutoSwitched: false,
hasScreenShares,
};
},
// initial value
{ mode: "grid", hasAutoSwitched: false, hasScreenShares: false },
),
map(({ mode }) => mode),
),
"grid",
);
return {
gridMode$,
setGridMode,
};
}
@@ -0,0 +1,202 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { EventEmitter } from "events";
import { ClientEvent, SyncState } from "matrix-js-sdk";
import { MembershipManagerEvent, Status } from "matrix-js-sdk/lib/matrixrtc";
import { ObservableScope } from "../../ObservableScope";
import { createHomeserverConnected$ } from "./HomeserverConnected";
/**
* Minimal stub of a Matrix client sufficient for our tests:
```
createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
)
```
*/
class MockMatrixClient extends EventEmitter {
private syncState: SyncState;
public constructor(initial: SyncState) {
super();
this.syncState = initial;
}
public setSyncState(state: SyncState): void {
this.syncState = state;
// Matrix's Sync event in createHomeserverConnected$ expects [SyncState]
this.emit(ClientEvent.Sync, [state]);
}
public getSyncState(): SyncState {
return this.syncState;
}
}
/**
* Minimal stub of MatrixRTCSession (membership manager):
```
createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
)
```
*/
class MockMatrixRTCSession extends EventEmitter {
public membershipStatus: Status;
public probablyLeft: boolean;
public constructor(props: {
membershipStatus: Status;
probablyLeft: boolean;
}) {
super();
this.membershipStatus = props.membershipStatus;
this.probablyLeft = props.probablyLeft;
}
public setMembershipStatus(status: Status): void {
this.membershipStatus = status;
this.emit(MembershipManagerEvent.StatusChanged);
}
public setProbablyLeft(flag: boolean): void {
this.probablyLeft = flag;
this.emit(MembershipManagerEvent.ProbablyLeft);
}
}
describe("createHomeserverConnected$", () => {
let scope: ObservableScope;
let client: MockMatrixClient;
let session: MockMatrixRTCSession;
beforeEach(() => {
scope = new ObservableScope();
client = new MockMatrixClient(SyncState.Error); // start disconnected
session = new MockMatrixRTCSession({
membershipStatus: Status.Disconnected,
probablyLeft: false,
});
});
afterEach(() => {
scope.end();
});
// LLM generated test cases. They are a bit overkill but I improved the mocking so it is
// easy enough to read them so I think they can stay.
it("is false when sync state is not Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
expect(hsConnected$.value).toBe(false);
});
it("remains false while membership status is not Connected even if sync is Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false); // membership still disconnected
});
it("is false when membership status transitions to Connected but ProbablyLeft is true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Make sync loop OK
client.setSyncState(SyncState.Syncing);
// Indicate probable leave before connection
session.setProbablyLeft(true);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(false);
});
it("becomes true only when all three conditions are satisfied", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// 1. Sync loop connected
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false); // not yet membership connected
// 2. Membership connected
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true); // probablyLeft is false
});
it("drops back to false when sync loop leaves Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Reach connected state
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
// Sync loop error => should flip false
client.setSyncState(SyncState.Error);
expect(hsConnected$.value).toBe(false);
});
it("drops back to false when membership status becomes disconnected", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setMembershipStatus(Status.Disconnected);
expect(hsConnected$.value).toBe(false);
});
it("drops to false when ProbablyLeft is emitted after being true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
});
it("recovers to true if ProbablyLeft becomes false again while other conditions remain true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
// Simulate clearing the flag (in realistic scenario membership manager would update)
session.setProbablyLeft(false);
expect(hsConnected$.value).toBe(true);
});
it("composite sequence reflects each individual failure reason", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Initially false (sync error + disconnected + not probably left)
expect(hsConnected$.value).toBe(false);
// Fix sync only
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false);
// Fix membership
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
// Introduce probablyLeft -> false
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
// Restore notProbablyLeft -> true again
session.setProbablyLeft(false);
expect(hsConnected$.value).toBe(true);
// Drop sync -> false
client.setSyncState(SyncState.Error);
expect(hsConnected$.value).toBe(false);
});
});
@@ -0,0 +1,85 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
MembershipManagerEvent,
Status,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { ClientEvent, type MatrixClient, SyncState } from "matrix-js-sdk";
import { fromEvent, startWith, map, tap, type Observable } from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type ObservableScope } from "../../ObservableScope";
import { type Behavior } from "../../Behavior";
import { and$ } from "../../../utils/observable";
import { type NodeStyleEventEmitter } from "../../../utils/test";
/**
* Logger instance (scoped child) for homeserver connection updates.
*/
const logger = rootLogger.getChild("[HomeserverConnected]");
/**
* Behavior representing whether we consider ourselves connected to the Matrix homeserver
* for the purposes of a MatrixRTC session.
*
* Becomes FALSE if ANY sub-condition is fulfilled:
* 1. Sync loop is not in SyncState.Syncing
* 2. membershipStatus !== Status.Connected
* 3. probablyLeft === true
*/
export function createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
): Behavior<boolean> {
const syncing$ = (
fromEvent(client, ClientEvent.Sync) as Observable<[SyncState]>
).pipe(
startWith([client.getSyncState()]),
map(([state]) => state === SyncState.Syncing),
);
const membershipConnected$ = fromEvent(
matrixRTCSession,
MembershipManagerEvent.StatusChanged,
).pipe(
startWith(null),
map(() => matrixRTCSession.membershipStatus === Status.Connected),
);
// This is basically notProbablyLeft$
//
// probablyLeft is computed by a local timer that mimics the server delayed event.
// If we locally predict our server event timed out. We consider ourselves as probablyLeft
// even though we might not yet have received the delayed event leave.
//
// If that is not the case we certainly still have a valid membership on the matrix network
// independet if the sync currently works.
const certainlyConnected$ = fromEvent(
matrixRTCSession,
MembershipManagerEvent.ProbablyLeft,
).pipe(
startWith(null),
map(() => matrixRTCSession.probablyLeft !== true),
);
const connectedCombined$ = and$(
syncing$,
membershipConnected$,
certainlyConnected$,
).pipe(
tap((connected) => {
logger.info(`Homeserver connected update: ${connected}`);
}),
);
return scope.behavior(connectedCombined$);
}
@@ -1,35 +1,57 @@
/* /*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd. Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
*/ */
import { type MatrixRTCSession } from "matrix-js-sdk/lib/matrixrtc"; import {
import { expect, test, vi } from "vitest"; type LivekitTransport,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { describe, expect, it, vi } from "vitest";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery"; import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import EventEmitter from "events"; import { BehaviorSubject, map, of } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import {
ConnectionState as LivekitConnectionState,
type LocalParticipant,
type LocalTrack,
} from "livekit-client";
import { MatrixRTCMode } from "../../../settings/settings"; import { MatrixRTCMode } from "../../../settings/settings";
import { mockConfig } from "../../../utils/test"; import {
import { enterRTCSession } from "./LocalMembership"; flushPromises,
mockConfig,
mockLivekitRoom,
mockMuteStates,
withTestScheduler,
} from "../../../utils/test";
import {
createLocalMembership$,
enterRTCSession,
RTCBackendState,
} from "./LocalMembership";
import { MatrixRTCTransportMissingError } from "../../../utils/errors";
import { Epoch, ObservableScope } from "../../ObservableScope";
import { constant } from "../../Behavior";
import { ConnectionManagerData } from "../remoteMembers/ConnectionManager";
import { type Connection } from "../remoteMembers/Connection";
import { type Publisher } from "./Publisher";
const MATRIX_RTC_MODE = MatrixRTCMode.Legacy; const MATRIX_RTC_MODE = MatrixRTCMode.Legacy;
const getUrlParams = vi.hoisted(() => vi.fn(() => ({}))); const getUrlParams = vi.hoisted(() => vi.fn(() => ({})));
vi.mock("../../../UrlParams", () => ({ getUrlParams })); vi.mock("../../../UrlParams", () => ({ getUrlParams }));
vi.mock("@livekit/components-core", () => ({
vi.mock("../../../widget", async (importOriginal) => ({ observeParticipantEvents: vi
...(await importOriginal()), .fn()
widget: { .mockReturnValue(of({ isScreenShareEnabled: false })),
api: {
setAlwaysOnScreen: (): void => {},
transport: { send: vi.fn(), reply: vi.fn(), stop: vi.fn() },
},
lazyActions: new EventEmitter(),
},
})); }));
test("It joins the correct Session", async () => { describe("LocalMembership", () => {
describe("enterRTCSession", () => {
it("It joins the correct Session", async () => {
const focusFromOlderMembership = { const focusFromOlderMembership = {
type: "livekit", type: "livekit",
livekit_service_url: "http://my-oldest-member-service-url.com", livekit_service_url: "http://my-oldest-member-service-url.com",
@@ -112,9 +134,9 @@ test("It joins the correct Session", async () => {
useLegacyMemberEvents: false, useLegacyMemberEvents: false,
}), }),
); );
}); });
test("It should not fail with configuration error if homeserver config has livekit url but not fallback", async () => { it("It should not fail with configuration error if homeserver config has livekit url but not fallback", async () => {
mockConfig({}); mockConfig({});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({ vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({
"org.matrix.msc4143.rtc_foci": [ "org.matrix.msc4143.rtc_foci": [
@@ -155,4 +177,316 @@ test("It should not fail with configuration error if homeserver config has livek
matrixRTCMode: MATRIX_RTC_MODE, matrixRTCMode: MATRIX_RTC_MODE,
}, },
); );
});
});
const defaultCreateLocalMemberValues = {
options: constant({
encryptMedia: false,
matrixRTCMode: MatrixRTCMode.Matrix_2_0,
}),
matrixRTCSession: {
updateCallIntent: () => {},
leaveRoomSession: () => {},
} as unknown as MatrixRTCSession,
muteStates: mockMuteStates(),
isHomeserverConnected: constant(true),
trackProcessorState$: constant({
supported: false,
processor: undefined,
}),
logger: logger,
createPublisherFactory: vi.fn(),
joinMatrixRTC: async (): Promise<void> => {},
homeserverConnected$: constant(true),
};
it("throws error on missing RTC config error", () => {
withTestScheduler(({ scope, hot, expectObservable }) => {
const goodTransport = {
livekit_service_url: "other",
} as LivekitTransport;
const localTransport$ = scope.behavior<LivekitTransport>(
hot("1ms #", {}, new MatrixRTCTransportMissingError("domain.com")),
goodTransport,
);
const mockConnectionManager = {
transports$: scope.behavior(
localTransport$.pipe(map((t) => new Epoch([t]))),
),
connectionManagerData$: constant(
new Epoch(new ConnectionManagerData()),
),
};
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: mockConnectionManager,
localTransport$,
});
expectObservable(localMembership.connectionState.livekit$).toBe("ne", {
n: { state: RTCBackendState.WaitingForConnection },
e: {
state: RTCBackendState.Error,
error: expect.toSatisfy(
(e) => e instanceof MatrixRTCTransportMissingError,
),
},
});
});
});
const aTransport = {
livekit_service_url: "a",
} as LivekitTransport;
const bTransport = {
livekit_service_url: "b",
} as LivekitTransport;
const connectionManagerData = new ConnectionManagerData();
connectionManagerData.add(
{
livekitRoom: mockLivekitRoom({
localParticipant: {
isScreenShareEnabled: false,
trackPublications: [],
} as unknown as LocalParticipant,
}),
state$: constant({
state: "ConnectedToLkRoom",
livekitConnectionState$: constant(LivekitConnectionState.Connected),
}),
transport: aTransport,
} as unknown as Connection,
[],
);
connectionManagerData.add(
{
state$: constant({
state: "ConnectedToLkRoom",
}),
transport: bTransport,
} as unknown as Connection,
[],
);
it("recreates publisher if new connection is used and ENDS always unpublish and end tracks", async () => {
const scope = new ObservableScope();
const localTransport$ = new BehaviorSubject(aTransport);
const publishers: Publisher[] = [];
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const p = {
stopPublishing: vi.fn(),
stopTracks: vi.fn(),
publishing$: constant(false),
};
publishers.push(p as unknown as Publisher);
return p;
},
);
const publisherFactory =
defaultCreateLocalMemberValues.createPublisherFactory as ReturnType<
typeof vi.fn
>;
createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport$,
});
await flushPromises();
localTransport$.next(bTransport);
await flushPromises();
expect(publisherFactory).toHaveBeenCalledTimes(2);
expect(publishers.length).toBe(2);
// stop the first Publisher and let the second one life.
expect(publishers[0].stopTracks).toHaveBeenCalled();
expect(publishers[1].stopTracks).not.toHaveBeenCalled();
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[1].stopPublishing).not.toHaveBeenCalled();
expect(publisherFactory.mock.calls[0][0].transport).toBe(aTransport);
expect(publisherFactory.mock.calls[1][0].transport).toBe(bTransport);
scope.end();
await flushPromises();
// stop all tracks after ending scopes
expect(publishers[1].stopPublishing).toHaveBeenCalled();
expect(publishers[1].stopTracks).toHaveBeenCalled();
defaultCreateLocalMemberValues.createPublisherFactory.mockReset();
});
it("only start tracks if requested", async () => {
const scope = new ObservableScope();
const localTransport$ = new BehaviorSubject(aTransport);
const publishers: Publisher[] = [];
const tracks$ = new BehaviorSubject<LocalTrack[]>([]);
const publishing$ = new BehaviorSubject<boolean>(false);
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const p = {
stopPublishing: vi.fn(),
stopTracks: vi.fn(),
createAndSetupTracks: vi.fn().mockImplementation(async () => {
tracks$.next([{}, {}] as LocalTrack[]);
return Promise.resolve();
}),
tracks$,
publishing$,
};
publishers.push(p as unknown as Publisher);
return p;
},
);
const publisherFactory =
defaultCreateLocalMemberValues.createPublisherFactory as ReturnType<
typeof vi.fn
>;
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport$,
});
await flushPromises();
expect(publisherFactory).toHaveBeenCalledOnce();
expect(localMembership.tracks$.value.length).toBe(0);
localMembership.startTracks();
await flushPromises();
expect(localMembership.tracks$.value.length).toBe(2);
scope.end();
await flushPromises();
// stop all tracks after ending scopes
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[0].stopTracks).toHaveBeenCalled();
publisherFactory.mockClear();
});
// TODO add an integration test combining publisher and localMembership
//
it("tracks livekit state correctly", async () => {
const scope = new ObservableScope();
const localTransport$ = new BehaviorSubject<null | LivekitTransport>(null);
const connectionManagerData$ = new BehaviorSubject<
Epoch<ConnectionManagerData>
>(new Epoch(new ConnectionManagerData()));
const publishers: Publisher[] = [];
const tracks$ = new BehaviorSubject<LocalTrack[]>([]);
const publishing$ = new BehaviorSubject<boolean>(false);
const createTrackResolver = Promise.withResolvers<void>();
const publishResolver = Promise.withResolvers<void>();
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const p = {
stopPublishing: vi.fn(),
stopTracks: vi.fn().mockImplementation(() => {
logger.info("stopTracks");
tracks$.next([]);
}),
createAndSetupTracks: vi.fn().mockImplementation(async () => {
await createTrackResolver.promise;
tracks$.next([{}, {}] as LocalTrack[]);
}),
startPublishing: vi.fn().mockImplementation(async () => {
await publishResolver.promise;
publishing$.next(true);
}),
tracks$,
publishing$,
};
publishers.push(p as unknown as Publisher);
return p;
},
);
const publisherFactory =
defaultCreateLocalMemberValues.createPublisherFactory as ReturnType<
typeof vi.fn
>;
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: {
connectionManagerData$,
},
localTransport$,
});
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.WaitingForTransport,
});
localTransport$.next(aTransport);
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.WaitingForConnection,
});
connectionManagerData$.next(new Epoch(connectionManagerData));
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.Initialized,
});
expect(publisherFactory).toHaveBeenCalledOnce();
expect(localMembership.tracks$.value.length).toBe(0);
// -------
localMembership.startTracks();
// -------
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.CreatingTracks,
});
createTrackResolver.resolve();
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.ReadyToPublish,
});
// -------
localMembership.requestConnect();
// -------
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.WaitingToPublish,
});
publishResolver.resolve();
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.Connected,
});
expect(publishers[0].stopPublishing).not.toHaveBeenCalled();
expect(localMembership.connectionState.livekit$.isStopped).toBe(false);
scope.end();
await flushPromises();
// stays in connected state because it is stopped before the update to tracks update the state.
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.Connected,
});
// stop all tracks after ending scopes
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[0].stopTracks).toHaveBeenCalled();
});
// TODO add tests for matrix local matrix participation.
}); });
@@ -1,5 +1,5 @@
/* /*
Copyright 2025 New Vector Ltd. Copyright 2025 Element Creations Ltd.
SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
@@ -10,19 +10,20 @@ import {
type Participant, type Participant,
ParticipantEvent, ParticipantEvent,
type LocalParticipant, type LocalParticipant,
type ScreenShareCaptureOptions,
ConnectionState,
} from "livekit-client"; } from "livekit-client";
import { observeParticipantEvents } from "@livekit/components-core"; import { observeParticipantEvents } from "@livekit/components-core";
import { import {
type LivekitTransport, type LivekitTransport,
type MatrixRTCSession, type MatrixRTCSession,
MembershipManagerEvent,
Status,
} from "matrix-js-sdk/lib/matrixrtc"; } from "matrix-js-sdk/lib/matrixrtc";
import { ClientEvent, SyncState, type Room as MatrixRoom } from "matrix-js-sdk";
import { import {
BehaviorSubject, BehaviorSubject,
catchError,
combineLatest, combineLatest,
fromEvent, distinctUntilChanged,
from,
map, map,
type Observable, type Observable,
of, of,
@@ -32,58 +33,73 @@ import {
tap, tap,
} from "rxjs"; } from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger"; import { type Logger } from "matrix-js-sdk/lib/logger";
import { deepCompare } from "matrix-js-sdk/lib/utils";
import { type Behavior } from "../../Behavior"; import { constant, type Behavior } from "../../Behavior";
import { type IConnectionManager } from "../remoteMembers/ConnectionManager"; import { type IConnectionManager } from "../remoteMembers/ConnectionManager";
import { ObservableScope } from "../../ObservableScope"; import { ObservableScope } from "../../ObservableScope";
import { Publisher } from "./Publisher"; import { type Publisher } from "./Publisher";
import { type MuteStates } from "../../MuteStates"; import { type MuteStates } from "../../MuteStates";
import { type ProcessorState } from "../../../livekit/TrackProcessorContext";
import { type MediaDevices } from "../../MediaDevices";
import { and$ } from "../../../utils/observable"; import { and$ } from "../../../utils/observable";
import { type ElementCallError } from "../../../utils/errors";
import { import {
ElementWidgetActions, ElementCallError,
widget, MembershipManagerError,
type WidgetHelpers, UnknownCallError,
} from "../../../widget"; } from "../../../utils/errors";
import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers"; import { ElementWidgetActions, widget } from "../../../widget";
import { getUrlParams } from "../../../UrlParams.ts"; import { getUrlParams } from "../../../UrlParams.ts";
import { PosthogAnalytics } from "../../../analytics/PosthogAnalytics.ts"; import { PosthogAnalytics } from "../../../analytics/PosthogAnalytics.ts";
import { MatrixRTCMode } from "../../../settings/settings.ts"; import { MatrixRTCMode } from "../../../settings/settings.ts";
import { Config } from "../../../config/Config.ts"; import { Config } from "../../../config/Config.ts";
import { type Connection } from "../remoteMembers/Connection.ts"; import { type Connection } from "../remoteMembers/Connection.ts";
export enum LivekitState { export enum RTCBackendState {
Uninitialized = "uninitialized",
Connecting = "connecting",
Connected = "connected",
Error = "error", Error = "error",
/** Not even a transport is available to the LocalMembership */
WaitingForTransport = "waiting_for_transport",
/** A connection appeared so we can initialise the publisher */
WaitingForConnection = "waiting_for_connection",
/** Connection and transport arrived, publisher Initialized */
Initialized = "Initialized",
CreatingTracks = "creating_tracks",
ReadyToPublish = "ready_to_publish",
WaitingToPublish = "waiting_to_publish",
Connected = "connected",
Disconnected = "disconnected", Disconnected = "disconnected",
Disconnecting = "disconnecting", Disconnecting = "disconnecting",
} }
type LocalMemberLivekitState = type LocalMemberRtcBackendState =
| { state: LivekitState.Error; error: string } | { state: RTCBackendState.Error; error: ElementCallError }
| { state: LivekitState.Connected } | { state: RTCBackendState.WaitingForTransport }
| { state: LivekitState.Connecting } | { state: RTCBackendState.WaitingForConnection }
| { state: LivekitState.Uninitialized } | { state: RTCBackendState.Initialized }
| { state: LivekitState.Disconnected } | { state: RTCBackendState.CreatingTracks }
| { state: LivekitState.Disconnecting }; | { state: RTCBackendState.ReadyToPublish }
| { state: RTCBackendState.WaitingToPublish }
| { state: RTCBackendState.Connected }
| { state: RTCBackendState.Disconnected }
| { state: RTCBackendState.Disconnecting };
export enum MatrixState { export enum MatrixState {
WaitingForTransport = "waiting_for_transport",
Ready = "ready",
Connecting = "connecting",
Connected = "connected", Connected = "connected",
Disconnected = "disconnected", Disconnected = "disconnected",
Connecting = "connecting", Error = "Error",
} }
type LocalMemberMatrixState = type LocalMemberMatrixState =
| { state: MatrixState.Connected } | { state: MatrixState.Connected }
| { state: MatrixState.WaitingForTransport }
| { state: MatrixState.Ready }
| { state: MatrixState.Connecting } | { state: MatrixState.Connecting }
| { state: MatrixState.Disconnected }; | { state: MatrixState.Disconnected }
| { state: MatrixState.Error; error: Error };
export interface LocalMemberConnectionState { export interface LocalMemberConnectionState {
livekit$: Behavior<LocalMemberLivekitState>; livekit$: Behavior<LocalMemberRtcBackendState>;
matrix$: Behavior<LocalMemberMatrixState>; matrix$: Behavior<LocalMemberMatrixState>;
} }
@@ -98,17 +114,21 @@ export interface LocalMemberConnectionState {
* - Publisher.publishTracks() * - Publisher.publishTracks()
* - send join state/sticky event * - send join state/sticky event
*/ */
interface Props { interface Props {
options: Behavior<EnterRTCSessionOptions>; // TODO add a comment into some code style readme or file header callviewmodel
// that the inputs for those createSomething$() functions should NOT contain any js-sdk objectes
scope: ObservableScope; scope: ObservableScope;
mediaDevices: MediaDevices;
muteStates: MuteStates; muteStates: MuteStates;
connectionManager: IConnectionManager; connectionManager: IConnectionManager;
matrixRTCSession: MatrixRTCSession; createPublisherFactory: (connection: Connection) => Publisher;
matrixRoom: MatrixRoom; joinMatrixRTC: (transport: LivekitTransport) => Promise<void>;
homeserverConnected$: Behavior<boolean>;
localTransport$: Behavior<LivekitTransport | null>; localTransport$: Behavior<LivekitTransport | null>;
trackProcessorState$: Behavior<ProcessorState>; matrixRTCSession: Pick<
widget: WidgetHelpers | null; MatrixRTCSession,
"updateCallIntent" | "leaveRoomSession"
>;
logger: Logger; logger: Logger;
} }
@@ -127,75 +147,74 @@ interface Props {
*/ */
export const createLocalMembership$ = ({ export const createLocalMembership$ = ({
scope, scope,
options,
muteStates,
mediaDevices,
connectionManager, connectionManager,
matrixRTCSession, localTransport$: localTransportCanThrow$,
localTransport$, homeserverConnected$,
matrixRoom, createPublisherFactory,
trackProcessorState$, joinMatrixRTC,
widget,
logger: parentLogger, logger: parentLogger,
muteStates,
matrixRTCSession,
}: Props): { }: Props): {
// publisher: Publisher /**
requestConnect: () => LocalMemberConnectionState; * This starts audio and video tracks. They will be reused when calling `requestConnect`.
*/
startTracks: () => Behavior<LocalTrack[]>; startTracks: () => Behavior<LocalTrack[]>;
requestDisconnect: () => Observable<LocalMemberLivekitState> | null; /**
* This sets a inner state (shouldConnect) to true and instructs the js-sdk and livekit to keep the user
* connected to matrix and livekit.
*/
requestConnect: () => void;
requestDisconnect: () => void;
connectionState: LocalMemberConnectionState; connectionState: LocalMemberConnectionState;
sharingScreen$: Behavior<boolean>; sharingScreen$: Behavior<boolean>;
/** /**
* Callback to toggle screen sharing. If null, screen sharing is not possible. * Callback to toggle screen sharing. If null, screen sharing is not possible.
*/ */
toggleScreenSharing: (() => void) | null; toggleScreenSharing: (() => void) | null;
tracks$: Behavior<LocalTrack[]>;
participant$: Behavior<LocalParticipant | null>; participant$: Behavior<LocalParticipant | null>;
connection$: Behavior<Connection | null>; connection$: Behavior<Connection | null>;
// deprecated fields
/** @deprecated use state instead*/
homeserverConnected$: Behavior<boolean>; homeserverConnected$: Behavior<boolean>;
/** @deprecated use state instead*/
connected$: Behavior<boolean>;
// this needs to be discussed // this needs to be discussed
/** @deprecated use state instead*/ /** @deprecated use state instead*/
reconnecting$: Behavior<boolean>; reconnecting$: Behavior<boolean>;
// also needs to be disccues
/** @deprecated use state instead*/
configError$: Behavior<ElementCallError | null>;
} => { } => {
const logger = parentLogger.getChild("[LocalMembership]"); const logger = parentLogger.getChild("[LocalMembership]");
logger.debug(`Creating local membership..`); logger.debug(`Creating local membership..`);
const state = {
livekit$: new BehaviorSubject<LocalMemberLivekitState>({ // Unwrap the local transport and set the state of the LocalMembership to error in case the transport is an error.
state: LivekitState.Uninitialized, const localTransport$ = scope.behavior(
localTransportCanThrow$.pipe(
catchError((e: unknown) => {
let error: ElementCallError;
if (e instanceof ElementCallError) {
error = e;
} else {
error = new UnknownCallError(
e instanceof Error
? e
: new Error("Unknown error from localTransport"),
);
}
setLivekitError(error);
return of(null);
}), }),
matrix$: new BehaviorSubject<LocalMemberMatrixState>({ ),
state: MatrixState.Disconnected, );
}),
};
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const trackStartRequested$ = new BehaviorSubject(false);
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const connectRequested$ = new BehaviorSubject(false);
// This should be used in a combineLatest with publisher$ to connect.
const tracks$ = new BehaviorSubject<LocalTrack[]>([]);
// Drop Epoch data here since we will not combine this anymore // Drop Epoch data here since we will not combine this anymore
const localConnection$ = scope.behavior( const localConnection$ = scope.behavior(
combineLatest([connectionManager.connections$, localTransport$]).pipe( combineLatest([
map(([connections, localTransport]) => { connectionManager.connectionManagerData$,
localTransport$,
]).pipe(
map(([{ value: connectionData }, localTransport]) => {
if (localTransport === null) { if (localTransport === null) {
return null; return null;
} }
return (
connections.value.find((connection) => return connectionData.getConnectionForTransport(localTransport);
areLivekitTransportsEqual(connection.transport, localTransport),
) ?? null
);
}), }),
tap((connection) => { tap((connection) => {
logger.info( logger.info(
@@ -204,88 +223,37 @@ export const createLocalMembership$ = ({
}), }),
), ),
); );
/**
* Whether we are connected to the MatrixRTC session. const localConnectionState$ = localConnection$.pipe(
*/ switchMap((connection) => (connection ? connection.state$ : of(null))),
const homeserverConnected$ = scope.behavior(
// To consider ourselves connected to MatrixRTC, we check the following:
and$(
// The client is connected to the sync loop
(
fromEvent(matrixRoom.client, ClientEvent.Sync) as Observable<
[SyncState]
>
).pipe(
startWith([matrixRoom.client.getSyncState()]),
map(([state]) => state === SyncState.Syncing),
),
// Room state observed by session says we're connected
fromEvent(matrixRTCSession, MembershipManagerEvent.StatusChanged).pipe(
startWith(null),
map(() => matrixRTCSession.membershipStatus === Status.Connected),
),
// Also watch out for warnings that we've likely hit a timeout and our
// delayed leave event is being sent (this condition is here because it
// provides an earlier warning than the sync loop timeout, and we wouldn't
// see the actual leave event until we reconnect to the sync loop)
fromEvent(matrixRTCSession, MembershipManagerEvent.ProbablyLeft).pipe(
startWith(null),
map(() => matrixRTCSession.probablyLeft !== true),
),
).pipe(
tap((connected) => {
logger.info(`Homeserver connected update: ${connected}`);
}),
),
); );
// /** // /**
// * Whether we are "fully" connected to the call. Accounts for both the // * Whether we are "fully" connected to the call. Accounts for both the
// * connection to the MatrixRTC session and the LiveKit publish connection. // * connection to the MatrixRTC session and the LiveKit publish connection.
// */ // */
// // TODO use this in combination with the MemberState.
const connected$ = scope.behavior( const connected$ = scope.behavior(
and$( and$(
homeserverConnected$, homeserverConnected$.pipe(
localConnection$.pipe( tap((v) => logger.debug("matrix: Connected state changed", v)),
switchMap((c) =>
c
? c.state$.pipe(map((state) => state.state === "ConnectedToLkRoom"))
: of(false),
),
),
), ),
localConnectionState$.pipe(
switchMap((state) => {
logger.debug("livekit: Connected state changed", state);
if (!state) return of(false);
if (state.state === "ConnectedToLkRoom") {
logger.debug(
"livekit: Connected state changed (inner livekitConnectionState$)",
state.livekitConnectionState$.value,
); );
return state.livekitConnectionState$.pipe(
const publisher$ = new BehaviorSubject<Publisher | null>(null); map((lkState) => lkState === ConnectionState.Connected),
localConnection$.subscribe((connection) => {
if (connection !== null && publisher$.value === null) {
// TODO looks strange to not change publisher if connection changes.
publisher$.next(
new Publisher(
scope,
connection,
mediaDevices,
muteStates,
trackProcessorState$,
),
); );
} }
}); return of(false);
}),
combineLatest([publisher$, trackStartRequested$]).subscribe( ),
([publisher, shouldStartTracks]) => { ).pipe(tap((v) => logger.debug("combined: Connected state changed", v))),
if (publisher && shouldStartTracks) {
publisher
.createAndSetupTracks()
.then((tracks) => {
tracks$.next(tracks);
})
.catch((error) => {
logger.error("Error creating tracks:", error);
});
}
},
); );
// MATRIX RELATED // MATRIX RELATED
@@ -310,83 +278,217 @@ export const createLocalMembership$ = ({
), ),
); );
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const trackStartRequested = Promise.withResolvers<void>();
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const connectRequested$ = new BehaviorSubject(false);
/**
* The publisher is stored in here an abstracts creating and publishing tracks.
*/
const publisher$ = new BehaviorSubject<Publisher | null>(null);
/**
* Extract the tracks from the published. Also reacts to changing publishers.
*/
const tracks$ = scope.behavior(
publisher$.pipe(switchMap((p) => (p?.tracks$ ? p.tracks$ : constant([])))),
);
const publishing$ = scope.behavior(
publisher$.pipe(switchMap((p) => p?.publishing$ ?? constant(false))),
);
const startTracks = (): Behavior<LocalTrack[]> => { const startTracks = (): Behavior<LocalTrack[]> => {
trackStartRequested$.next(true); trackStartRequested.resolve();
return tracks$; return tracks$;
}; };
combineLatest([publisher$, tracks$]).subscribe(([publisher, tracks]) => { const requestConnect = (): void => {
if ( trackStartRequested.resolve();
tracks.length === 0 ||
// change this to !== Publishing
state.livekit$.value.state !== LivekitState.Uninitialized
) {
return;
}
state.livekit$.next({ state: LivekitState.Connecting });
publisher
?.startPublishing()
.then(() => {
state.livekit$.next({ state: LivekitState.Connected });
})
.catch((error) => {
state.livekit$.next({ state: LivekitState.Error, error });
});
});
combineLatest([localTransport$, connectRequested$]).subscribe(
// TODO reconnect on options change.
([transport, connectRequested]) => {
if (
transport === null ||
!connectRequested ||
state.matrix$.value.state !== MatrixState.Disconnected
) {
logger.info(
"Not yet connecting because: ",
"transport === null:",
transport === null,
"!connectRequested:",
!connectRequested,
"state.matrix$.value.state !== MatrixState.Disconnected:",
state.matrix$.value.state !== MatrixState.Disconnected,
);
return;
}
state.matrix$.next({ state: MatrixState.Connecting });
logger.info("Matrix State connecting");
enterRTCSession(matrixRTCSession, transport, options.value).catch(
(error) => {
logger.error(error);
},
);
},
);
const requestConnect = (): LocalMemberConnectionState => {
trackStartRequested$.next(true);
connectRequested$.next(true); connectRequested$.next(true);
return state;
}; };
const requestDisconnect = (): Behavior<LocalMemberLivekitState> | null => { const requestDisconnect = (): void => {
if (state.livekit$.value.state !== LivekitState.Connected) return null; connectRequested$.next(false);
state.livekit$.next({ state: LivekitState.Disconnecting }); };
combineLatest([publisher$, tracks$], (publisher, tracks) => {
publisher // Take care of the publisher$
?.stopPublishing() // create a new one as soon as a local Connection is available
.then(() => { //
tracks.forEach((track) => track.stop()); // Recreate a new one once the local connection changes
state.livekit$.next({ state: LivekitState.Disconnected }); // - stop publishing
}) // - destruct all current streams
.catch((error) => { // - overwrite current publisher
state.livekit$.next({ state: LivekitState.Error, error }); scope.reconcile(localConnection$, async (connection) => {
if (connection !== null) {
publisher$.next(createPublisherFactory(connection));
}
return Promise.resolve(async (): Promise<void> => {
await publisher$?.value?.stopPublishing();
publisher$?.value?.stopTracks();
}); });
}); });
return state.livekit$; // Use reconcile here to not run concurrent createAndSetupTracks calls
// `tracks$` will update once they are ready.
scope.reconcile(
scope.behavior(
combineLatest([publisher$, tracks$, from(trackStartRequested.promise)]),
null,
),
async (valueIfReady) => {
if (!valueIfReady) return;
const [publisher, tracks] = valueIfReady;
if (publisher && tracks.length === 0) {
await publisher.createAndSetupTracks().catch((e) => logger.error(e));
}
},
);
// Based on `connectRequested$` we start publishing tracks. (once they are there!)
scope.reconcile(
scope.behavior(combineLatest([publisher$, tracks$, connectRequested$])),
async ([publisher, tracks, shouldConnect]) => {
if (shouldConnect === publisher?.publishing$.value) return;
if (tracks.length !== 0 && shouldConnect) {
try {
await publisher?.startPublishing();
} catch (error) {
setLivekitError(error as ElementCallError);
}
} else if (tracks.length !== 0 && !shouldConnect) {
try {
await publisher?.stopPublishing();
} catch (error) {
setLivekitError(new UnknownCallError(error as Error));
}
}
},
);
const fatalLivekitError$ = new BehaviorSubject<ElementCallError | null>(null);
const setLivekitError = (e: ElementCallError): void => {
if (fatalLivekitError$.value !== null)
logger.error("Multiple Livkit Errors:", e);
else fatalLivekitError$.next(e);
}; };
const livekitState$: Behavior<LocalMemberRtcBackendState> = scope.behavior(
combineLatest([
publisher$,
localTransport$,
tracks$.pipe(
tap((t) => {
logger.info("tracks$: ", t);
}),
),
publishing$,
connectRequested$,
from(trackStartRequested.promise).pipe(
map(() => true),
startWith(false),
),
fatalLivekitError$,
]).pipe(
map(
([
publisher,
localTransport,
tracks,
publishing,
shouldConnect,
shouldStartTracks,
error,
]) => {
// read this:
// if(!<A>) return {state: ...}
// if(!<B>) return {state: <MyState>}
//
// as:
// We do have <A> but not yet <B> so we are in <MyState>
if (error !== null) return { state: RTCBackendState.Error, error };
const hasTracks = tracks.length > 0;
if (!localTransport)
return { state: RTCBackendState.WaitingForTransport };
if (!publisher)
return { state: RTCBackendState.WaitingForConnection };
if (!shouldStartTracks) return { state: RTCBackendState.Initialized };
if (!hasTracks) return { state: RTCBackendState.CreatingTracks };
if (!shouldConnect) return { state: RTCBackendState.ReadyToPublish };
if (!publishing) return { state: RTCBackendState.WaitingToPublish };
return { state: RTCBackendState.Connected };
},
),
distinctUntilChanged(deepCompare),
),
);
const fatalMatrixError$ = new BehaviorSubject<ElementCallError | null>(null);
const setMatrixError = (e: ElementCallError): void => {
if (fatalMatrixError$.value !== null)
logger.error("Multiple Matrix Errors:", e);
else fatalMatrixError$.next(e);
};
const matrixState$: Behavior<LocalMemberMatrixState> = scope.behavior(
combineLatest([
localTransport$,
connectRequested$,
homeserverConnected$,
]).pipe(
map(([localTransport, connectRequested, homeserverConnected]) => {
if (!localTransport) return { state: MatrixState.WaitingForTransport };
if (!connectRequested) return { state: MatrixState.Ready };
if (!homeserverConnected) return { state: MatrixState.Connecting };
return { state: MatrixState.Connected };
}),
),
);
// Keep matrix rtc session in sync with localTransport$, connectRequested$ and muteStates.video.enabled$
scope.reconcile(
scope.behavior(combineLatest([localTransport$, connectRequested$])),
async ([transport, shouldConnect]) => {
if (!shouldConnect) return;
if (!transport) return;
try {
await joinMatrixRTC(transport);
} catch (error) {
logger.error("Error entering RTC session", error);
if (error instanceof Error)
setMatrixError(new MembershipManagerError(error));
}
// Update our member event when our mute state changes.
const callIntentScope = new ObservableScope();
// because this uses its own scope, we can start another reconciliation for the duration of one connection.
callIntentScope.reconcile(
muteStates.video.enabled$,
async (videoEnabled) =>
matrixRTCSession.updateCallIntent(videoEnabled ? "video" : "audio"),
);
return async (): Promise<void> => {
callIntentScope.end();
try {
// Update matrixRTCSession to allow udpating the transport without leaving the session!
await matrixRTCSession.leaveRoomSession();
} catch (e) {
logger.error("Error leaving RTC session", e);
}
try {
await widget?.api.transport.send(ElementWidgetActions.HangupCall, {});
} catch (e) {
logger.error("Failed to send hangup action", e);
}
};
},
);
const participant$ = scope.behavior(
localConnection$.pipe(map((c) => c?.livekitRoom?.localParticipant ?? null)),
);
// Pause upstream of all local media tracks when we're disconnected from // Pause upstream of all local media tracks when we're disconnected from
// MatrixRTC, because it can be an unpleasant surprise for the app to say // MatrixRTC, because it can be an unpleasant surprise for the app to say
@@ -394,12 +496,12 @@ export const createLocalMembership$ = ({
// We use matrixConnected$ rather than reconnecting$ because we want to // We use matrixConnected$ rather than reconnecting$ because we want to
// pause tracks during the initial joining sequence too until we're sure // pause tracks during the initial joining sequence too until we're sure
// that our own media is displayed on screen. // that our own media is displayed on screen.
combineLatest([localConnection$, homeserverConnected$]) // TODO refactor this based no livekitState$
combineLatest([participant$, homeserverConnected$])
.pipe(scope.bind()) .pipe(scope.bind())
.subscribe(([connection, connected]) => { .subscribe(([participant, connected]) => {
if (connection?.state$.value.state !== "ConnectedToLkRoom") return; if (!participant) return;
const publications = const publications = participant.trackPublications.values();
connection.livekitRoom.localParticipant.trackPublications.values();
if (connected) { if (connected) {
for (const p of publications) { for (const p of publications) {
if (p.track?.isUpstreamPaused === true) { if (p.track?.isUpstreamPaused === true) {
@@ -437,63 +539,33 @@ export const createLocalMembership$ = ({
} }
}); });
const configError$ = new BehaviorSubject<ElementCallError | null>(null);
// TODO I do not fully understand what this does.
// Is it needed?
// Is this at the right place?
// Can this be simplified?
// Start and stop session membership as needed
scope.reconcile(localTransport$, async (advertised) => {
if (advertised !== null && advertised !== undefined) {
try {
configError$.next(null);
await enterRTCSession(matrixRTCSession, advertised, options.value);
} catch (e) {
logger.error("Error entering RTC session", e);
}
// Update our member event when our mute state changes.
const intentScope = new ObservableScope();
intentScope.reconcile(muteStates.video.enabled$, async (videoEnabled) =>
matrixRTCSession.updateCallIntent(videoEnabled ? "video" : "audio"),
);
return async (): Promise<void> => {
intentScope.end();
// Only sends Matrix leave event. The LiveKit session will disconnect
// as soon as either the stopConnection$ handler above gets to it or
// the view model is destroyed.
try {
await matrixRTCSession.leaveRoomSession();
} catch (e) {
logger.error("Error leaving RTC session", e);
}
try {
await widget?.api.transport.send(ElementWidgetActions.HangupCall, {});
} catch (e) {
logger.error("Failed to send hangup action", e);
}
};
}
});
/** /**
* Whether the user is currently sharing their screen. * Whether the user is currently sharing their screen.
*/ */
const sharingScreen$ = scope.behavior( const sharingScreen$ = scope.behavior(
localConnection$.pipe( participant$.pipe(
switchMap((c) => switchMap((p) => (p !== null ? observeSharingScreen$(p) : of(false))),
c === null
? of(false)
: observeSharingScreen$(c.livekitRoom.localParticipant),
),
), ),
); );
const toggleScreenSharing = let toggleScreenSharing: (() => void) | null = null;
if (
"getDisplayMedia" in (navigator.mediaDevices ?? {}) && "getDisplayMedia" in (navigator.mediaDevices ?? {}) &&
!getUrlParams().hideScreensharing !getUrlParams().hideScreensharing
? (): void => ) {
toggleScreenSharing = (): void => {
const screenshareSettings: ScreenShareCaptureOptions = {
audio: true,
selfBrowserSurface: "include",
surfaceSwitching: "include",
systemAudio: "include",
};
const targetScreenshareState = !sharingScreen$.value;
logger.info(
`toggleScreenSharing called. Switching ${
targetScreenshareState ? "On" : "Off"
}`,
);
// If a connection is ready, toggle screen sharing. // If a connection is ready, toggle screen sharing.
// We deliberately do nothing in the case of a null connection because // We deliberately do nothing in the case of a null connection because
// it looks nice for the call control buttons to all become available // it looks nice for the call control buttons to all become available
@@ -502,31 +574,26 @@ export const createLocalMembership$ = ({
// We also allow screen sharing to be toggled even if the connection // We also allow screen sharing to be toggled even if the connection
// is still initializing or publishing tracks, because there's no // is still initializing or publishing tracks, because there's no
// technical reason to disallow this. LiveKit will publish if it can. // technical reason to disallow this. LiveKit will publish if it can.
void localConnection$.value?.livekitRoom.localParticipant participant$.value
.setScreenShareEnabled(!sharingScreen$.value, { ?.setScreenShareEnabled(targetScreenshareState, screenshareSettings)
audio: true, .catch(logger.error);
selfBrowserSurface: "include", };
surfaceSwitching: "include", }
systemAudio: "include",
})
.catch(logger.error)
: null;
const participant$ = scope.behavior(
localConnection$.pipe(map((c) => c?.livekitRoom.localParticipant ?? null)),
);
return { return {
startTracks, startTracks,
requestConnect, requestConnect,
requestDisconnect, requestDisconnect,
connectionState: state, connectionState: {
livekit$: livekitState$,
matrix$: matrixState$,
},
tracks$,
participant$,
homeserverConnected$, homeserverConnected$,
connected$,
reconnecting$, reconnecting$,
configError$,
sharingScreen$, sharingScreen$,
toggleScreenSharing, toggleScreenSharing,
participant$,
connection$: localConnection$, connection$: localConnection$,
}; };
}; };
@@ -547,7 +614,12 @@ interface EnterRTCSessionOptions {
} }
/** /**
* TODO! document this function properly * Does the necessary steps to enter the RTC session on the matrix side:
* - Preparing the membership info (FOCUS to use, options)
* - Sends the matrix event to join the call, and starts the membership manager:
* - Delay events management
* - Handles retries (fails only after several attempts)
*
* @param rtcSession * @param rtcSession
* @param transport * @param transport
* @param options * @param options
@@ -572,6 +644,7 @@ export async function enterRTCSession(
const { sendNotificationType: notificationType, callIntent } = getUrlParams(); const { sendNotificationType: notificationType, callIntent } = getUrlParams();
const multiSFU = matrixRTCMode !== MatrixRTCMode.Legacy; const multiSFU = matrixRTCMode !== MatrixRTCMode.Legacy;
// Multi-sfu does not need a preferred foci list. just the focus that is actually used. // Multi-sfu does not need a preferred foci list. just the focus that is actually used.
// TODO where/how do we track errors originating from the ongoing rtcSession?
rtcSession.joinRoomSession( rtcSession.joinRoomSession(
multiSFU ? [] : [transport], multiSFU ? [] : [transport],
multiSFU ? transport : undefined, multiSFU ? transport : undefined,
@@ -0,0 +1,156 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject } from "rxjs";
import { mockConfig, flushPromises } from "../../../utils/test";
import { createLocalTransport$ } from "./LocalTransport";
import { constant } from "../../Behavior";
import { Epoch, ObservableScope } from "../../ObservableScope";
import {
MatrixRTCTransportMissingError,
FailToGetOpenIdToken,
} from "../../../utils/errors";
import * as openIDSFU from "../../../livekit/openIDSFU";
describe("LocalTransport", () => {
let scope: ObservableScope;
beforeEach(() => (scope = new ObservableScope()));
afterEach(() => scope.end());
it("throws if config is missing", async () => {
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "",
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
await flushPromises();
expect(() => localTransport$.value).toThrow(
new MatrixRTCTransportMissingError(""),
);
});
it("throws FailToGetOpenIdToken when OpenID fetch fails", async () => {
// Provide a valid config so makeTransportInternal resolves a transport
const scope = new ObservableScope();
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const resolver = Promise.withResolvers<void>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockImplementation(
async () => {
await resolver.promise;
throw new FailToGetOpenIdToken(new Error("no openid"));
},
);
const observations: unknown[] = [];
const errors: Error[] = [];
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
// Use empty domain to skip .well-known and use config directly
getDomain: () => "",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
localTransport$.subscribe(
(o) => observations.push(o),
(e) => errors.push(e),
);
resolver.resolve();
await flushPromises();
const expectedError = new FailToGetOpenIdToken(new Error("no openid"));
expect(observations).toStrictEqual([null]);
expect(errors).toStrictEqual([expectedError]);
expect(() => localTransport$.value).toThrow(expectedError);
});
it("emits preferred transport after OpenID resolves", async () => {
// Use config so transport discovery succeeds, but delay OpenID JWT fetch
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const openIdResolver = Promise.withResolvers<openIDSFU.SFUConfig>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockReturnValue(
openIdResolver.promise,
);
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
openIdResolver.resolve?.({ url: "https://lk.example.org", jwt: "jwt" });
expect(localTransport$.value).toBe(null);
await flushPromises();
// final
expect(localTransport$.value).toStrictEqual({
livekit_alias: "!room:example.org",
livekit_service_url: "https://lk.example.org",
type: "livekit",
});
});
it("updates local transport when oldest member changes", async () => {
// Use config so transport discovery succeeds, but delay OpenID JWT fetch
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const memberships$ = new BehaviorSubject(new Epoch([]));
const openIdResolver = Promise.withResolvers<openIDSFU.SFUConfig>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockReturnValue(
openIdResolver.promise,
);
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(true),
memberships$,
client: {
getDomain: () => "",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
openIdResolver.resolve?.({ url: "https://lk.example.org", jwt: "jwt" });
expect(localTransport$.value).toBe(null);
await flushPromises();
// final
expect(localTransport$.value).toStrictEqual({
livekit_alias: "!room:example.org",
livekit_service_url: "https://lk.example.org",
type: "livekit",
});
});
});
@@ -1,5 +1,5 @@
/* /*
Copyright 2025 New Vector Ltd. Copyright 2025 Element Creations Ltd.
SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
@@ -13,32 +13,39 @@ import {
isLivekitTransportConfig, isLivekitTransportConfig,
} from "matrix-js-sdk/lib/matrixrtc"; } from "matrix-js-sdk/lib/matrixrtc";
import { type MatrixClient } from "matrix-js-sdk"; import { type MatrixClient } from "matrix-js-sdk";
import { combineLatest, distinctUntilChanged, first, from, map } from "rxjs"; import {
import { logger } from "matrix-js-sdk/lib/logger"; combineLatest,
distinctUntilChanged,
first,
from,
map,
switchMap,
} from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery"; import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { type Behavior } from "../../Behavior.ts"; import { type Behavior } from "../../Behavior.ts";
import { type Epoch, type ObservableScope } from "../../ObservableScope.ts"; import { type Epoch, type ObservableScope } from "../../ObservableScope.ts";
import { Config } from "../../../config/Config.ts"; import { Config } from "../../../config/Config.ts";
import { MatrixRTCTransportMissingError } from "../../../utils/errors.ts"; import { MatrixRTCTransportMissingError } from "../../../utils/errors.ts";
import { getSFUConfigWithOpenID } from "../../../livekit/openIDSFU.ts"; import {
getSFUConfigWithOpenID,
type OpenIDClientParts,
} from "../../../livekit/openIDSFU.ts";
import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers.ts"; import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers.ts";
import { customLivekitUrl } from "../../../settings/settings.ts";
const logger = rootLogger.getChild("[LocalTransport]");
/* /*
* - get well known * It figures out “which LiveKit focus URL/alias the local user should use,”
* - get oldest membership * optionally aligning with the oldest member, and ensures the SFU path is primed
* - get transport to use * before advertising that choice.
* - get openId + jwt token
* - wait for createTrack() call
* - create tracks
* - wait for join() call
* - Publisher.publishTracks()
* - send join state/sticky event
*/ */
interface Props { interface Props {
scope: ObservableScope; scope: ObservableScope;
memberships$: Behavior<Epoch<CallMembership[]>>; memberships$: Behavior<Epoch<CallMembership[]>>;
client: MatrixClient; client: Pick<MatrixClient, "getDomain"> & OpenIDClientParts;
roomId: string; roomId: string;
useOldestMember$: Behavior<boolean>; useOldestMember$: Behavior<boolean>;
} }
@@ -49,6 +56,8 @@ interface Props {
* *
* @prop useOldestMember Whether to use the same transport as the oldest member. * @prop useOldestMember Whether to use the same transport as the oldest member.
* This will only update once the first oldest member appears. Will not recompute if the oldest member leaves. * This will only update once the first oldest member appears. Will not recompute if the oldest member leaves.
*
* @throws MatrixRTCTransportMissingError | FailToGetOpenIdToken
*/ */
export const createLocalTransport$ = ({ export const createLocalTransport$ = ({
scope, scope,
@@ -75,16 +84,20 @@ export const createLocalTransport$ = ({
/** /**
* The transport that we would personally prefer to publish on (if not for the * The transport that we would personally prefer to publish on (if not for the
* transport preferences of others, perhaps). * transport preferences of others, perhaps).
*
* @throws
*/ */
const preferredTransport$: Behavior<LivekitTransport | null> = scope.behavior( const preferredTransport$: Behavior<LivekitTransport | null> = scope.behavior(
from(makeTransport(client, roomId)), customLivekitUrl.value$.pipe(
switchMap((customUrl) => from(makeTransport(client, roomId, customUrl))),
),
null, null,
); );
/** /**
* The transport we should advertise in our MatrixRTC membership. * The chosen transport we should advertise in our MatrixRTC membership.
*/ */
const advertisedTransport$ = scope.behavior( return scope.behavior(
combineLatest([ combineLatest([
useOldestMember$, useOldestMember$,
oldestMemberTransport$, oldestMemberTransport$,
@@ -98,78 +111,78 @@ export const createLocalTransport$ = ({
distinctUntilChanged(areLivekitTransportsEqual), distinctUntilChanged(areLivekitTransportsEqual),
), ),
); );
return advertisedTransport$;
}; };
const FOCI_WK_KEY = "org.matrix.msc4143.rtc_foci"; const FOCI_WK_KEY = "org.matrix.msc4143.rtc_foci";
async function makeTransportInternal( /**
client: MatrixClient, *
* @param client
* @param roomId
* @returns
* @throws MatrixRTCTransportMissingError | FailToGetOpenIdToken
*/
async function makeTransport(
client: Pick<MatrixClient, "getDomain"> & OpenIDClientParts,
roomId: string, roomId: string,
urlFromDevSettings: string | null,
): Promise<LivekitTransport> { ): Promise<LivekitTransport> {
logger.log("Searching for a preferred transport"); let transport: LivekitTransport | undefined;
logger.trace("Searching for a preferred transport");
//TODO refactor this to use the jwt service returned alias. //TODO refactor this to use the jwt service returned alias.
const livekitAlias = roomId; const livekitAlias = roomId;
// TODO-MULTI-SFU: Either remove this dev tool or make it more official
const urlFromStorage = // DEVTOOL: Highest priority: Load from devtool setting
localStorage.getItem("robin-matrixrtc-auth") ?? if (urlFromDevSettings !== null) {
localStorage.getItem("timo-focus-url");
if (urlFromStorage !== null) {
const transportFromStorage: LivekitTransport = { const transportFromStorage: LivekitTransport = {
type: "livekit", type: "livekit",
livekit_service_url: urlFromStorage, livekit_service_url: urlFromDevSettings,
livekit_alias: livekitAlias, livekit_alias: livekitAlias,
}; };
logger.log( logger.info(
"Using LiveKit transport from local storage: ", "Using LiveKit transport from dev tools: ",
transportFromStorage, transportFromStorage,
); );
return transportFromStorage; transport = transportFromStorage;
} }
// Prioritize the .well-known/matrix/client, if available, over the configured SFU // WELL_KNOWN: Prioritize the .well-known/matrix/client, if available, over the configured SFU
const domain = client.getDomain(); const domain = client.getDomain();
if (domain) { if (domain && transport === undefined) {
// we use AutoDiscovery instead of relying on the MatrixClient having already // we use AutoDiscovery instead of relying on the MatrixClient having already
// been fully configured and started // been fully configured and started
const wellKnownFoci = (await AutoDiscovery.getRawClientConfig(domain))?.[ const wellKnownFoci = (await AutoDiscovery.getRawClientConfig(domain))?.[
FOCI_WK_KEY FOCI_WK_KEY
]; ];
if (Array.isArray(wellKnownFoci)) { if (Array.isArray(wellKnownFoci)) {
const transport: LivekitTransportConfig | undefined = wellKnownFoci.find( const wellKnownTransport: LivekitTransportConfig | undefined =
(f) => f && isLivekitTransportConfig(f), wellKnownFoci.find((f) => f && isLivekitTransportConfig(f));
); if (wellKnownTransport !== undefined) {
if (transport !== undefined) { logger.info("Using LiveKit transport from .well-known: ", transport);
logger.log("Using LiveKit transport from .well-known: ", transport); transport = { ...wellKnownTransport, livekit_alias: livekitAlias };
return { ...transport, livekit_alias: livekitAlias };
} }
} }
} }
// CONFIG: Least prioritized; Load from config file
const urlFromConf = Config.get().livekit?.livekit_service_url; const urlFromConf = Config.get().livekit?.livekit_service_url;
if (urlFromConf) { if (urlFromConf && transport === undefined) {
const transportFromConf: LivekitTransport = { const transportFromConf: LivekitTransport = {
type: "livekit", type: "livekit",
livekit_service_url: urlFromConf, livekit_service_url: urlFromConf,
livekit_alias: livekitAlias, livekit_alias: livekitAlias,
}; };
logger.log("Using LiveKit transport from config: ", transportFromConf); logger.info("Using LiveKit transport from config: ", transportFromConf);
return transportFromConf; transport = transportFromConf;
} }
throw new MatrixRTCTransportMissingError(domain ?? ""); if (!transport) throw new MatrixRTCTransportMissingError(domain ?? ""); // this will call the jwt/sfu/get endpoint to pre create the livekit room.
}
async function makeTransport(
client: MatrixClient,
roomId: string,
): Promise<LivekitTransport> {
const transport = await makeTransportInternal(client, roomId);
// this will call the jwt/sfu/get endpoint to pre create the livekit room.
await getSFUConfigWithOpenID( await getSFUConfigWithOpenID(
client, client,
transport.livekit_service_url, transport.livekit_service_url,
transport.livekit_alias, transport.livekit_alias,
); );
return transport; return transport;
} }
@@ -0,0 +1,140 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
afterEach,
beforeEach,
describe,
expect,
it,
type Mock,
vi,
} from "vitest";
import { ConnectionState as LivekitConenctionState } from "livekit-client";
import { type BehaviorSubject } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { ObservableScope } from "../../ObservableScope";
import { constant } from "../../Behavior";
import {
mockLivekitRoom,
mockLocalParticipant,
mockMediaDevices,
} from "../../../utils/test";
import { Publisher } from "./Publisher";
import {
type Connection,
type ConnectionState,
} from "../remoteMembers/Connection";
import { type MuteStates } from "../../MuteStates";
import { FailToStartLivekitConnection } from "../../../utils/errors";
describe("Publisher", () => {
let scope: ObservableScope;
let connection: Connection;
let muteStates: MuteStates;
beforeEach(() => {
muteStates = {
audio: {
enabled$: constant(false),
unsetHandler: vi.fn(),
setHandler: vi.fn(),
},
video: {
enabled$: constant(false),
unsetHandler: vi.fn(),
setHandler: vi.fn(),
},
} as unknown as MuteStates;
scope = new ObservableScope();
connection = {
state$: constant({
state: "ConnectedToLkRoom",
livekitConnectionState$: constant(LivekitConenctionState.Connected),
}),
livekitRoom: mockLivekitRoom({
localParticipant: mockLocalParticipant({}),
}),
} as unknown as Connection;
});
afterEach(() => scope.end());
it("throws if livekit room could not publish", async () => {
const publisher = new Publisher(
scope,
connection,
mockMediaDevices({}),
muteStates,
constant({ supported: false, processor: undefined }),
logger,
);
// should do nothing if no tracks have been created yet.
await publisher.startPublishing();
expect(
connection.livekitRoom.localParticipant.publishTrack,
).not.toHaveBeenCalled();
await expect(publisher.createAndSetupTracks()).rejects.toThrow(
Error("audio and video is false"),
);
(muteStates.audio.enabled$ as BehaviorSubject<boolean>).next(true);
(
connection.livekitRoom.localParticipant.createTracks as Mock
).mockResolvedValue([{}, {}]);
await expect(publisher.createAndSetupTracks()).resolves.not.toThrow();
expect(
connection.livekitRoom.localParticipant.createTracks,
).toHaveBeenCalledOnce();
// failiour due to localParticipant.publishTrack
(
connection.livekitRoom.localParticipant.publishTrack as Mock
).mockRejectedValue(Error("testError"));
await expect(publisher.startPublishing()).rejects.toThrow(
new FailToStartLivekitConnection("testError"),
);
// does not try other conenction after the first one failed
expect(
connection.livekitRoom.localParticipant.publishTrack,
).toHaveBeenCalledTimes(1);
// failiour due to connection.state$
const beforeState = connection.state$.value;
(connection.state$ as BehaviorSubject<ConnectionState>).next({
state: "FailedToStart",
error: Error("testStartError"),
});
await expect(publisher.startPublishing()).rejects.toThrow(
new FailToStartLivekitConnection("testStartError"),
);
(connection.state$ as BehaviorSubject<ConnectionState>).next(beforeState);
// does not try other conenction after the first one failed
expect(
connection.livekitRoom.localParticipant.publishTrack,
).toHaveBeenCalledTimes(1);
// success case
(
connection.livekitRoom.localParticipant.publishTrack as Mock
).mockResolvedValue({});
await expect(publisher.startPublishing()).resolves.not.toThrow();
expect(
connection.livekitRoom.localParticipant.publishTrack,
).toHaveBeenCalledTimes(3);
});
});
@@ -1,4 +1,5 @@
/* /*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd. Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -13,6 +14,7 @@ import {
ConnectionState as LivekitConnectionState, ConnectionState as LivekitConnectionState,
} from "livekit-client"; } from "livekit-client";
import { import {
BehaviorSubject,
map, map,
NEVER, NEVER,
type Observable, type Observable,
@@ -32,6 +34,10 @@ import { getUrlParams } from "../../../UrlParams.ts";
import { observeTrackReference$ } from "../../MediaViewModel.ts"; import { observeTrackReference$ } from "../../MediaViewModel.ts";
import { type Connection } from "../remoteMembers/Connection.ts"; import { type Connection } from "../remoteMembers/Connection.ts";
import { type ObservableScope } from "../../ObservableScope.ts"; import { type ObservableScope } from "../../ObservableScope.ts";
import {
ElementCallError,
FailToStartLivekitConnection,
} from "../../../utils/errors.ts";
/** /**
* A wrapper for a Connection object. * A wrapper for a Connection object.
@@ -39,7 +45,6 @@ import { type ObservableScope } from "../../ObservableScope.ts";
* The Publisher is also responsible for creating the media tracks. * The Publisher is also responsible for creating the media tracks.
*/ */
export class Publisher { export class Publisher {
public tracks: LocalTrack<Track.Kind>[] = [];
/** /**
* Creates a new Publisher. * Creates a new Publisher.
* @param scope - The observable scope to use for managing the publisher. * @param scope - The observable scope to use for managing the publisher.
@@ -51,19 +56,19 @@ export class Publisher {
*/ */
public constructor( public constructor(
private scope: ObservableScope, private scope: ObservableScope,
private connection: Connection, private connection: Pick<Connection, "livekitRoom" | "state$">, //setE2EEEnabled,
devices: MediaDevices, devices: MediaDevices,
private readonly muteStates: MuteStates, private readonly muteStates: MuteStates,
trackerProcessorState$: Behavior<ProcessorState>, trackerProcessorState$: Behavior<ProcessorState>,
private logger?: Logger, private logger: Logger,
) { ) {
this.logger?.info("[PublishConnection] Create LiveKit room"); this.logger.info("Create LiveKit room");
const { controlledAudioDevices } = getUrlParams(); const { controlledAudioDevices } = getUrlParams();
const room = connection.livekitRoom; const room = connection.livekitRoom;
room.setE2EEEnabled(room.options.e2ee !== undefined)?.catch((e: Error) => { room.setE2EEEnabled(room.options.e2ee !== undefined)?.catch((e: Error) => {
this.logger?.error("Failed to set E2EE enabled on room", e); this.logger.error("Failed to set E2EE enabled on room", e);
}); });
// Setup track processor syncing (blur) // Setup track processor syncing (blur)
@@ -73,13 +78,31 @@ export class Publisher {
this.workaroundRestartAudioInputTrackChrome(devices, scope); this.workaroundRestartAudioInputTrackChrome(devices, scope);
this.scope.onEnd(() => { this.scope.onEnd(() => {
this.logger?.info( this.logger.info("Scope ended -> stop publishing all tracks");
"[PublishConnection] Scope ended -> stop publishing all tracks",
);
void this.stopPublishing(); void this.stopPublishing();
}); });
// TODO move mute state handling here using reconcile (instead of inside the mute state class)
// this.scope.reconcile(
// this.scope.behavior(
// combineLatest([this.muteStates.video.enabled$, this.tracks$]),
// ),
// async ([videoEnabled, tracks]) => {
// const track = tracks.find((t) => t.kind == Track.Kind.Video);
// if (!track) return;
// if (videoEnabled) {
// await track.unmute();
// } else {
// await track.mute();
// }
// },
// );
} }
private _tracks$ = new BehaviorSubject<LocalTrack<Track.Kind>[]>([]);
public tracks$ = this._tracks$ as Behavior<LocalTrack<Track.Kind>[]>;
/** /**
* Start the connection to LiveKit and publish local tracks. * Start the connection to LiveKit and publish local tracks.
* *
@@ -93,51 +116,46 @@ export class Publisher {
* @throws {InsufficientCapacityError} if the LiveKit server indicates that it has insufficient capacity to accept the connection. * @throws {InsufficientCapacityError} if the LiveKit server indicates that it has insufficient capacity to accept the connection.
* @throws {SFURoomCreationRestrictedError} if the LiveKit server indicates that the room does not exist and cannot be created. * @throws {SFURoomCreationRestrictedError} if the LiveKit server indicates that the room does not exist and cannot be created.
*/ */
public async createAndSetupTracks(): Promise<LocalTrack[]> { public async createAndSetupTracks(): Promise<void> {
this.logger.debug("createAndSetupTracks called");
const lkRoom = this.connection.livekitRoom; const lkRoom = this.connection.livekitRoom;
// Observe mute state changes and update LiveKit microphone/camera states accordingly // Observe mute state changes and update LiveKit microphone/camera states accordingly
this.observeMuteStates(this.scope); this.observeMuteStates(this.scope);
// TODO: This should be an autostarted connection no need to start here. just check the connection state.
// TODO: This will fetch the JWT token. Perhaps we could keep it preloaded
// instead? This optimization would only be safe for a publish connection,
// because we don't want to leak the user's intent to perhaps join a call to
// remote servers before they actually commit to it.
// const { promise, resolve, reject } = Promise.withResolvers<void>();
// const sub = this.connection.state$.subscribe((s) => {
// if (s.state === "FailedToStart") {
// reject(new Error("Disconnected from LiveKit server"));
// } else if (s.state === "ConnectedToLkRoom") {
// resolve();
// }
// });
// try {
// await promise;
// } catch (e) {
// throw e;
// } finally {
// sub.unsubscribe();
// }
// TODO-MULTI-SFU: Prepublish a microphone track // TODO-MULTI-SFU: Prepublish a microphone track
const audio = this.muteStates.audio.enabled$.value; const audio = this.muteStates.audio.enabled$.value;
const video = this.muteStates.video.enabled$.value; const video = this.muteStates.video.enabled$.value;
// createTracks throws if called with audio=false and video=false // createTracks throws if called with audio=false and video=false
if (audio || video) { if (audio || video) {
// TODO this can still throw errors? It will also prompt for permissions if not already granted // TODO this can still throw errors? It will also prompt for permissions if not already granted
this.tracks = return lkRoom.localParticipant
(await lkRoom.localParticipant
.createTracks({ .createTracks({
audio, audio,
video, video,
}) })
.then((tracks) => {
this.logger.info(
"created track",
tracks.map((t) => t.kind + ", " + t.id),
);
this._tracks$.next(tracks);
})
.catch((error) => { .catch((error) => {
this.logger?.error("Failed to create tracks", error); this.logger.error("Failed to create tracks", error);
})) ?? []; });
} }
return this.tracks; throw Error("audio and video is false");
} }
private _publishing$ = new BehaviorSubject<boolean>(false);
public publishing$ = this.scope.behavior(this._publishing$);
/**
*
* @returns
* @throws ElementCallError
*/
public async startPublishing(): Promise<LocalTrack[]> { public async startPublishing(): Promise<LocalTrack[]> {
this.logger.debug("startPublishing called");
const lkRoom = this.connection.livekitRoom; const lkRoom = this.connection.livekitRoom;
const { promise, resolve, reject } = Promise.withResolvers<void>(); const { promise, resolve, reject } = Promise.withResolvers<void>();
const sub = this.connection.state$.subscribe((s) => { const sub = this.connection.state$.subscribe((s) => {
@@ -146,10 +164,14 @@ export class Publisher {
resolve(); resolve();
break; break;
case "FailedToStart": case "FailedToStart":
reject(new Error("Failed to connect to LiveKit server")); reject(
s.error instanceof ElementCallError
? s.error
: new FailToStartLivekitConnection(s.error.message),
);
break; break;
default: default:
this.logger?.info("waiting for connection: ", s.state); this.logger.info("waiting for connection: ", s.state);
} }
}); });
try { try {
@@ -159,19 +181,27 @@ export class Publisher {
} finally { } finally {
sub.unsubscribe(); sub.unsubscribe();
} }
for (const track of this.tracks) {
for (const track of this.tracks$.value) {
this.logger.info("publish ", this.tracks$.value.length, "tracks");
// TODO: handle errors? Needs the signaling connection to be up, but it has some retries internally // TODO: handle errors? Needs the signaling connection to be up, but it has some retries internally
// with a timeout. // with a timeout.
await lkRoom.localParticipant.publishTrack(track).catch((error) => { await lkRoom.localParticipant.publishTrack(track).catch((error) => {
this.logger?.error("Failed to publish track", error); this.logger.error("Failed to publish track", error);
throw new FailToStartLivekitConnection(
error instanceof Error ? error.message : error,
);
}); });
this.logger.info("published track ", track.kind, track.id);
// TODO: check if the connection is still active? and break the loop if not? // TODO: check if the connection is still active? and break the loop if not?
} }
return this.tracks; this._publishing$.next(true);
return this.tracks$.value;
} }
public async stopPublishing(): Promise<void> { public async stopPublishing(): Promise<void> {
this.logger.debug("stopPublishing called");
// TODO-MULTI-SFU: Move these calls back to ObservableScope.onEnd once scope // TODO-MULTI-SFU: Move these calls back to ObservableScope.onEnd once scope
// actually has the right lifetime // actually has the right lifetime
this.muteStates.audio.unsetHandler(); this.muteStates.audio.unsetHandler();
@@ -183,7 +213,28 @@ export class Publisher {
if (p.track !== undefined) tracks.push(p.track); if (p.track !== undefined) tracks.push(p.track);
}; };
localParticipant.trackPublications.forEach(addToTracksIfDefined); localParticipant.trackPublications.forEach(addToTracksIfDefined);
await localParticipant.unpublishTracks(tracks); this.logger.debug(
"list of tracks to unpublish:",
tracks.map((t) => t.kind + ", " + t.id),
"start unpublishing now",
);
await localParticipant.unpublishTracks(tracks).catch((error) => {
this.logger.error("Failed to unpublish tracks", error);
throw error;
});
this.logger.debug(
"unpublished tracks",
tracks.map((t) => t.kind + ", " + t.id),
);
this._publishing$.next(false);
}
/**
* Stops all tracks that are currently running
*/
public stopTracks(): void {
this.tracks$.value.forEach((t) => t.stop());
this._tracks$.next([]);
} }
/// Private methods /// Private methods
@@ -220,6 +271,9 @@ export class Publisher {
// the process of being restarted. // the process of being restarted.
activeMicTrack.mediaStreamTrack.readyState !== "ended" activeMicTrack.mediaStreamTrack.readyState !== "ended"
) { ) {
this.logger?.info(
"Restarting audio device track due to active media device changed (workaroundRestartAudioInputTrackChrome)",
);
// Restart the track, which will cause Livekit to do another // Restart the track, which will cause Livekit to do another
// getUserMedia() call with deviceId: default to get the *new* default device. // getUserMedia() call with deviceId: default to get the *new* default device.
// Note that room.switchActiveDevice() won't work: Livekit will ignore it because // Note that room.switchActiveDevice() won't work: Livekit will ignore it because
@@ -228,7 +282,7 @@ export class Publisher {
.getTrackPublication(Track.Source.Microphone) .getTrackPublication(Track.Source.Microphone)
?.audioTrack?.restartTrack() ?.audioTrack?.restartTrack()
.catch((e) => { .catch((e) => {
this.logger?.error(`Failed to restart audio device track`, e); this.logger.error(`Failed to restart audio device track`, e);
}); });
} }
}); });
@@ -248,7 +302,7 @@ export class Publisher {
selected$.pipe(scope.bind()).subscribe((device) => { selected$.pipe(scope.bind()).subscribe((device) => {
if (lkRoom.state != LivekitConnectionState.Connected) return; if (lkRoom.state != LivekitConnectionState.Connected) return;
// if (this.connectionState$.value !== ConnectionState.Connected) return; // if (this.connectionState$.value !== ConnectionState.Connected) return;
this.logger?.info( this.logger.info(
"[LivekitRoom] syncDevice room.getActiveDevice(kind) !== d.id :", "[LivekitRoom] syncDevice room.getActiveDevice(kind) !== d.id :",
lkRoom.getActiveDevice(kind), lkRoom.getActiveDevice(kind),
" !== ", " !== ",
@@ -261,7 +315,7 @@ export class Publisher {
lkRoom lkRoom
.switchActiveDevice(kind, device.id) .switchActiveDevice(kind, device.id)
.catch((e: Error) => .catch((e: Error) =>
this.logger?.error( this.logger.error(
`Failed to sync ${kind} device with LiveKit`, `Failed to sync ${kind} device with LiveKit`,
e, e,
), ),
@@ -286,10 +340,7 @@ export class Publisher {
try { try {
await lkRoom.localParticipant.setMicrophoneEnabled(desired); await lkRoom.localParticipant.setMicrophoneEnabled(desired);
} catch (e) { } catch (e) {
this.logger?.error( this.logger.error("Failed to update LiveKit audio input mute state", e);
"Failed to update LiveKit audio input mute state",
e,
);
} }
return lkRoom.localParticipant.isMicrophoneEnabled; return lkRoom.localParticipant.isMicrophoneEnabled;
}); });
@@ -297,10 +348,7 @@ export class Publisher {
try { try {
await lkRoom.localParticipant.setCameraEnabled(desired); await lkRoom.localParticipant.setCameraEnabled(desired);
} catch (e) { } catch (e) {
this.logger?.error( this.logger.error("Failed to update LiveKit video input mute state", e);
"Failed to update LiveKit video input mute state",
e,
);
} }
return lkRoom.localParticipant.isCameraEnabled; return lkRoom.localParticipant.isCameraEnabled;
}); });
@@ -1,4 +1,5 @@
/* /*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd. Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -192,7 +193,7 @@ describe("Start connection states", () => {
capturedState = capturedStates.pop(); capturedState = capturedStates.pop();
if (capturedState!.state === "FailedToStart") { if (capturedState!.state === "FailedToStart") {
expect(capturedState!.error.message).toEqual("Something went wrong"); expect(capturedState!.error.message).toEqual("Something went wrong");
expect(capturedState!.transport.livekit_alias).toEqual( expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias, livekitFocus.livekit_alias,
); );
} else { } else {
@@ -248,7 +249,7 @@ describe("Start connection states", () => {
expect(capturedState?.error.message).toContain( expect(capturedState?.error.message).toContain(
"SFU Config fetch failed with exception Error", "SFU Config fetch failed with exception Error",
); );
expect(capturedState?.transport.livekit_alias).toEqual( expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias, livekitFocus.livekit_alias,
); );
} else { } else {
@@ -312,7 +313,7 @@ describe("Start connection states", () => {
expect(capturedState.error.message).toContain( expect(capturedState.error.message).toContain(
"Failed to connect to livekit", "Failed to connect to livekit",
); );
expect(capturedState.transport.livekit_alias).toEqual( expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias, livekitFocus.livekit_alias,
); );
} else { } else {
@@ -1,4 +1,5 @@
/* /*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd. Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -18,7 +19,7 @@ import {
RoomEvent, RoomEvent,
} from "livekit-client"; } from "livekit-client";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc"; import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, map, type Observable } from "rxjs"; import { BehaviorSubject, map } from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger"; import { type Logger } from "matrix-js-sdk/lib/logger";
import { import {
@@ -49,16 +50,14 @@ export interface ConnectionOpts {
export type ConnectionState = export type ConnectionState =
| { state: "Initialized" } | { state: "Initialized" }
| { state: "FetchingConfig"; transport: LivekitTransport } | { state: "FetchingConfig" }
| { state: "ConnectingToLkRoom"; transport: LivekitTransport } | { state: "ConnectingToLkRoom" }
| { state: "PublishingTracks"; transport: LivekitTransport }
| { state: "FailedToStart"; error: Error; transport: LivekitTransport }
| { | {
state: "ConnectedToLkRoom"; state: "ConnectedToLkRoom";
livekitConnectionState$: Observable<LivekitConenctionState>; livekitConnectionState$: Behavior<LivekitConenctionState>;
transport: LivekitTransport;
} }
| { state: "Stopped"; transport: LivekitTransport }; | { state: "FailedToStart"; error: Error }
| { state: "Stopped" };
/** /**
* A connection to a Matrix RTC LiveKit backend. * A connection to a Matrix RTC LiveKit backend.
@@ -76,6 +75,24 @@ export class Connection {
*/ */
public readonly state$: Behavior<ConnectionState> = this._state$; public readonly state$: Behavior<ConnectionState> = this._state$;
/**
* The media transport to connect to.
*/
public readonly transport: LivekitTransport;
public readonly livekitRoom: LivekitRoom;
private scope: ObservableScope;
/**
* An observable of the participants that are publishing on this connection. (Excluding our local participant)
* This is derived from `participantsIncludingSubscribers$` and `remoteTransports$`.
* It filters the participants to only those that are associated with a membership that claims to publish on this connection.
*/
public readonly remoteParticipantsWithTracks$: Behavior<
PublishingParticipant[]
>;
/** /**
* Whether the connection has been stopped. * Whether the connection has been stopped.
* @see Connection.stop * @see Connection.stop
@@ -95,7 +112,6 @@ export class Connection {
* @throws {InsufficientCapacityError} if the LiveKit server indicates that it has insufficient capacity to accept the connection. * @throws {InsufficientCapacityError} if the LiveKit server indicates that it has insufficient capacity to accept the connection.
* @throws {SFURoomCreationRestrictedError} if the LiveKit server indicates that the room does not exist and cannot be created. * @throws {SFURoomCreationRestrictedError} if the LiveKit server indicates that the room does not exist and cannot be created.
*/ */
// TODO dont make this throw and instead store a connection error state in this class?
// TODO consider an autostart pattern... // TODO consider an autostart pattern...
public async start(): Promise<void> { public async start(): Promise<void> {
this.logger.debug("Starting Connection"); this.logger.debug("Starting Connection");
@@ -103,7 +119,6 @@ export class Connection {
try { try {
this._state$.next({ this._state$.next({
state: "FetchingConfig", state: "FetchingConfig",
transport: this.transport,
}); });
const { url, jwt } = await this.getSFUConfigWithOpenID(); const { url, jwt } = await this.getSFUConfigWithOpenID();
// If we were stopped while fetching the config, don't proceed to connect // If we were stopped while fetching the config, don't proceed to connect
@@ -111,7 +126,6 @@ export class Connection {
this._state$.next({ this._state$.next({
state: "ConnectingToLkRoom", state: "ConnectingToLkRoom",
transport: this.transport,
}); });
try { try {
await this.livekitRoom.connect(url, jwt); await this.livekitRoom.connect(url, jwt);
@@ -142,15 +156,15 @@ export class Connection {
this._state$.next({ this._state$.next({
state: "ConnectedToLkRoom", state: "ConnectedToLkRoom",
transport: this.transport, livekitConnectionState$: this.scope.behavior(
livekitConnectionState$: connectionStateObserver(this.livekitRoom), connectionStateObserver(this.livekitRoom),
),
}); });
} catch (error) { } catch (error) {
this.logger.debug(`Failed to connect to LiveKit room: ${error}`); this.logger.debug(`Failed to connect to LiveKit room: ${error}`);
this._state$.next({ this._state$.next({
state: "FailedToStart", state: "FailedToStart",
error: error instanceof Error ? error : new Error(`${error}`), error: error instanceof Error ? error : new Error(`${error}`),
transport: this.transport,
}); });
throw error; throw error;
} }
@@ -178,28 +192,11 @@ export class Connection {
await this.livekitRoom.disconnect(); await this.livekitRoom.disconnect();
this._state$.next({ this._state$.next({
state: "Stopped", state: "Stopped",
transport: this.transport,
}); });
this.stopped = true; this.stopped = true;
} }
/**
* An observable of the participants that are publishing on this connection. (Excluding our local participant)
* This is derived from `participantsIncludingSubscribers$` and `remoteTransports$`.
* It filters the participants to only those that are associated with a membership that claims to publish on this connection.
*/
public readonly remoteParticipantsWithTracks$: Behavior<
PublishingParticipant[]
>;
/**
* The media transport to connect to.
*/
public readonly transport: LivekitTransport;
private readonly client: OpenIDClientParts; private readonly client: OpenIDClientParts;
public readonly livekitRoom: LivekitRoom;
private readonly logger: Logger; private readonly logger: Logger;
/** /**
@@ -216,6 +213,7 @@ export class Connection {
); );
const { transport, client, scope } = opts; const { transport, client, scope } = opts;
this.scope = scope;
this.livekitRoom = opts.livekitRoomFactory(); this.livekitRoom = opts.livekitRoomFactory();
this.transport = transport; this.transport = transport;
this.client = client; this.client = client;
@@ -7,10 +7,11 @@ Please see LICENSE in the repository root for full details.
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc"; import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { import {
type E2EEOptions,
Room as LivekitRoom, Room as LivekitRoom,
type RoomOptions, type RoomOptions,
type BaseKeyProvider, type BaseKeyProvider,
type E2EEManagerOptions,
type BaseE2EEManager,
} from "livekit-client"; } from "livekit-client";
import { type Logger } from "matrix-js-sdk/lib/logger"; import { type Logger } from "matrix-js-sdk/lib/logger";
import E2EEWorker from "livekit-client/e2ee-worker?worker"; import E2EEWorker from "livekit-client/e2ee-worker?worker";
@@ -23,6 +24,7 @@ import type { Behavior } from "../../Behavior.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx"; import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { defaultLiveKitOptions } from "../../../livekit/options.ts"; import { defaultLiveKitOptions } from "../../../livekit/options.ts";
// TODO evaluate if this should be done like the Publisher Factory
export interface ConnectionFactory { export interface ConnectionFactory {
createConnection( createConnection(
transport: LivekitTransport, transport: LivekitTransport,
@@ -40,8 +42,10 @@ export class ECConnectionFactory implements ConnectionFactory {
* @param client - The OpenID client parts for authentication, needed to get openID and JWT tokens. * @param client - The OpenID client parts for authentication, needed to get openID and JWT tokens.
* @param devices - Used for video/audio out/in capture options. * @param devices - Used for video/audio out/in capture options.
* @param processorState$ - Effects like background blur (only for publishing connection?) * @param processorState$ - Effects like background blur (only for publishing connection?)
* @param e2eeLivekitOptions - The E2EE options to use for the LiveKit Room. * @param livekitKeyProvider
* @param controlledAudioDevices - Option to indicate whether audio output device is controlled externally (native mobile app). * @param controlledAudioDevices - Option to indicate whether audio output device is controlled externally (native mobile app).
* @param echoCancellation - Whether to enable echo cancellation for audio capture.
* @param noiseSuppression - Whether to enable noise suppression for audio capture.
* @param livekitRoomFactory - Optional factory function (for testing) to create LivekitRoom instances. If not provided, a default factory is used. * @param livekitRoomFactory - Optional factory function (for testing) to create LivekitRoom instances. If not provided, a default factory is used.
*/ */
public constructor( public constructor(
@@ -51,20 +55,24 @@ export class ECConnectionFactory implements ConnectionFactory {
livekitKeyProvider: BaseKeyProvider | undefined, livekitKeyProvider: BaseKeyProvider | undefined,
private controlledAudioDevices: boolean, private controlledAudioDevices: boolean,
livekitRoomFactory?: () => LivekitRoom, livekitRoomFactory?: () => LivekitRoom,
echoCancellation: boolean = true,
noiseSuppression: boolean = true,
) { ) {
const defaultFactory = (): LivekitRoom => const defaultFactory = (): LivekitRoom =>
new LivekitRoom( new LivekitRoom(
generateRoomOption( generateRoomOption({
this.devices, devices: this.devices,
this.processorState$.value, processorState: this.processorState$.value,
livekitKeyProvider && { e2eeLivekitOptions: livekitKeyProvider && {
keyProvider: livekitKeyProvider, keyProvider: livekitKeyProvider,
// It's important that every room use a separate E2EE worker. // It's important that every room use a separate E2EE worker.
// They get confused if given streams from multiple rooms. // They get confused if given streams from multiple rooms.
worker: new E2EEWorker(), worker: new E2EEWorker(),
}, },
this.controlledAudioDevices, controlledAudioDevices: this.controlledAudioDevices,
), echoCancellation,
noiseSuppression,
}),
); );
this.livekitRoomFactory = livekitRoomFactory ?? defaultFactory; this.livekitRoomFactory = livekitRoomFactory ?? defaultFactory;
} }
@@ -89,12 +97,24 @@ export class ECConnectionFactory implements ConnectionFactory {
/** /**
* Generate the initial LiveKit RoomOptions based on the current media devices and processor state. * Generate the initial LiveKit RoomOptions based on the current media devices and processor state.
*/ */
function generateRoomOption( function generateRoomOption({
devices: MediaDevices, devices,
processorState: ProcessorState, processorState,
e2eeLivekitOptions: E2EEOptions | undefined, e2eeLivekitOptions,
controlledAudioDevices: boolean, controlledAudioDevices,
): RoomOptions { echoCancellation,
noiseSuppression,
}: {
devices: MediaDevices;
processorState: ProcessorState;
e2eeLivekitOptions:
| E2EEManagerOptions
| { e2eeManager: BaseE2EEManager }
| undefined;
controlledAudioDevices: boolean;
echoCancellation: boolean;
noiseSuppression: boolean;
}): RoomOptions {
return { return {
...defaultLiveKitOptions, ...defaultLiveKitOptions,
videoCaptureDefaults: { videoCaptureDefaults: {
@@ -105,6 +125,8 @@ function generateRoomOption(
audioCaptureDefaults: { audioCaptureDefaults: {
...defaultLiveKitOptions.audioCaptureDefaults, ...defaultLiveKitOptions.audioCaptureDefaults,
deviceId: devices.audioInput.selected$.value?.id, deviceId: devices.audioInput.selected$.value?.id,
echoCancellation,
noiseSuppression,
}, },
audioOutput: { audioOutput: {
// When using controlled audio devices, we don't want to set the // When using controlled audio devices, we don't want to set the
@@ -11,7 +11,7 @@ import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { type Participant as LivekitParticipant } from "livekit-client"; import { type Participant as LivekitParticipant } from "livekit-client";
import { logger } from "matrix-js-sdk/lib/logger"; import { logger } from "matrix-js-sdk/lib/logger";
import { Epoch, ObservableScope } from "../../ObservableScope.ts"; import { Epoch, mapEpoch, ObservableScope } from "../../ObservableScope.ts";
import { import {
createConnectionManager$, createConnectionManager$,
type ConnectionManagerData, type ConnectionManagerData,
@@ -73,7 +73,7 @@ afterEach(() => {
describe("connections$ stream", () => { describe("connections$ stream", () => {
test("Should create and start new connections for each transports", () => { test("Should create and start new connections for each transports", () => {
withTestScheduler(({ behavior, expectObservable }) => { withTestScheduler(({ behavior, expectObservable }) => {
const { connections$ } = createConnectionManager$({ const { connectionManagerData$ } = createConnectionManager$({
scope: testScope, scope: testScope,
connectionFactory: fakeConnectionFactory, connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("a", { inputTransports$: behavior("a", {
@@ -82,7 +82,9 @@ describe("connections$ stream", () => {
logger: logger, logger: logger,
}); });
expectObservable(connections$).toBe("a", { expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("a", {
a: expect.toSatisfy((e: Epoch<Connection[]>) => { a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value; const connections = e.value;
expect(connections.length).toBe(2); expect(connections.length).toBe(2);
@@ -110,7 +112,7 @@ describe("connections$ stream", () => {
test("Should start connection only once", () => { test("Should start connection only once", () => {
withTestScheduler(({ behavior, expectObservable }) => { withTestScheduler(({ behavior, expectObservable }) => {
const { connections$ } = createConnectionManager$({ const { connectionManagerData$ } = createConnectionManager$({
scope: testScope, scope: testScope,
connectionFactory: fakeConnectionFactory, connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("abcdef", { inputTransports$: behavior("abcdef", {
@@ -124,7 +126,9 @@ describe("connections$ stream", () => {
logger: logger, logger: logger,
}); });
expectObservable(connections$).toBe("xxxxxa", { expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("xxxxxa", {
x: expect.anything(), x: expect.anything(),
a: expect.toSatisfy((e: Epoch<Connection[]>) => { a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value; const connections = e.value;
@@ -153,7 +157,7 @@ describe("connections$ stream", () => {
test("Should cleanup connections when not needed anymore", () => { test("Should cleanup connections when not needed anymore", () => {
withTestScheduler(({ behavior, expectObservable }) => { withTestScheduler(({ behavior, expectObservable }) => {
const { connections$ } = createConnectionManager$({ const { connectionManagerData$ } = createConnectionManager$({
scope: testScope, scope: testScope,
connectionFactory: fakeConnectionFactory, connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("abc", { inputTransports$: behavior("abc", {
@@ -164,7 +168,9 @@ describe("connections$ stream", () => {
logger: logger, logger: logger,
}); });
expectObservable(connections$).toBe("xab", { expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("xab", {
x: expect.anything(), x: expect.anything(),
a: expect.toSatisfy((e: Epoch<Connection[]>) => { a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value; const connections = e.value;
@@ -1,6 +1,3 @@
// TODOs:
// - make ConnectionManager its own actual class
/* /*
Copyright 2025 Element Creations Ltd. Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd. Copyright 2025 New Vector Ltd.
@@ -95,9 +92,7 @@ interface Props {
} }
// TODO - write test for scopes (do we really need to bind scope) // TODO - write test for scopes (do we really need to bind scope)
export interface IConnectionManager { export interface IConnectionManager {
transports$: Behavior<Epoch<LivekitTransport[]>>;
connectionManagerData$: Behavior<Epoch<ConnectionManagerData>>; connectionManagerData$: Behavior<Epoch<ConnectionManagerData>>;
connections$: Behavior<Epoch<Connection[]>>;
} }
/** /**
* Crete a `ConnectionManager` * Crete a `ConnectionManager`
@@ -220,7 +215,7 @@ export function createConnectionManager$({
new Epoch(new ConnectionManagerData()), new Epoch(new ConnectionManagerData()),
); );
return { transports$, connectionManagerData$, connections$ }; return { connectionManagerData$ };
} }
function removeDuplicateTransports( function removeDuplicateTransports(
@@ -0,0 +1,133 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { Room as LivekitRoom } from "livekit-client";
import { BehaviorSubject } from "rxjs";
import fetchMock from "fetch-mock";
import { logger } from "matrix-js-sdk/lib/logger";
import EventEmitter from "events";
import { ObservableScope } from "../../ObservableScope.ts";
import { ECConnectionFactory } from "./ConnectionFactory.ts";
import type { OpenIDClientParts } from "../../../livekit/openIDSFU.ts";
import { exampleTransport, mockMediaDevices } from "../../../utils/test.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { constant } from "../../Behavior";
// At the top of your test file, after imports
vi.mock("livekit-client", async (importOriginal) => {
return {
...(await importOriginal()),
Room: vi.fn().mockImplementation(function (this: LivekitRoom, options) {
const emitter = new EventEmitter();
return {
on: emitter.on.bind(emitter),
off: emitter.off.bind(emitter),
emit: emitter.emit.bind(emitter),
disconnect: vi.fn(),
remoteParticipants: new Map(),
} as unknown as LivekitRoom;
}),
};
});
let testScope: ObservableScope;
let mockClient: OpenIDClientParts;
beforeEach(() => {
testScope = new ObservableScope();
mockClient = {
getOpenIdToken: vi.fn().mockReturnValue(""),
getDeviceId: vi.fn().mockReturnValue("DEV000"),
};
});
describe("ECConnectionFactory - Audio inputs options", () => {
test.each([
{ echo: true, noise: true },
{ echo: true, noise: false },
{ echo: false, noise: true },
{ echo: false, noise: false },
])(
"it sets echoCancellation=$echo and noiseSuppression=$noise based on constructor parameters",
({ echo, noise }) => {
// test("it sets echoCancellation and noiseSuppression based on constructor parameters", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
undefined,
echo,
noise,
);
ecConnectionFactory.createConnection(exampleTransport, testScope, logger);
// Check if Room was constructed with expected options
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
audioCaptureDefaults: expect.objectContaining({
echoCancellation: echo,
noiseSuppression: noise,
}),
}),
);
},
);
});
describe("ECConnectionFactory - ControlledAudioDevice", () => {
test.each([{ controlled: true }, { controlled: false }])(
"it sets controlledAudioDevice=$controlled then uses deviceId accordingly",
({ controlled }) => {
// test("it sets echoCancellation and noiseSuppression based on constructor parameters", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
mockMediaDevices({
audioOutput: {
available$: constant(new Map<never, never>()),
selected$: constant({ id: "DEV00", virtualEarpiece: false }),
select: () => {},
},
}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
controlled,
undefined,
false,
false,
);
ecConnectionFactory.createConnection(exampleTransport, testScope, logger);
// Check if Room was constructed with expected options
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
audioOutput: expect.objectContaining({
deviceId: controlled ? undefined : "DEV00",
}),
}),
);
},
);
});
afterEach(() => {
testScope.end();
fetchMock.reset();
});
@@ -58,6 +58,10 @@ describe("MatrixMemberMetadata", () => {
const members = Array.from(fakeMembersMap.values()); const members = Array.from(fakeMembersMap.values());
return members; return members;
}), }),
getMembersWithMembership: vi.fn().mockImplementation(() => {
const members = Array.from(fakeMembersMap.values());
return members;
}),
} as unknown as MatrixRoom; } as unknown as MatrixRoom;
}); });
@@ -100,7 +104,6 @@ describe("MatrixMemberMetadata", () => {
} }
} }
// TODO this is a regression, now there the own user is not always in the map. Ask Timo if fine
it("should show our own user if present in rtc session and room", () => { it("should show our own user if present in rtc session and room", () => {
withTestScheduler(({ behavior, expectObservable }) => { withTestScheduler(({ behavior, expectObservable }) => {
fakeMemberWith({ fakeMemberWith({
@@ -513,7 +516,6 @@ describe("MatrixMemberMetadata", () => {
} }
} }
// TODO this is a regression, now there the own user is not always in the map. Ask Timo if fine
it("should use avatar url from room members", () => { it("should use avatar url from room members", () => {
withTestScheduler(({ behavior, expectObservable }) => { withTestScheduler(({ behavior, expectObservable }) => {
fakeMemberWith({ fakeMemberWith({
@@ -1,5 +1,5 @@
/* /*
Copyright 2025 New Vector Ltd. Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
@@ -9,7 +9,10 @@ import { type RoomMember, RoomStateEvent } from "matrix-js-sdk";
import { combineLatest, fromEvent, map } from "rxjs"; import { combineLatest, fromEvent, map } from "rxjs";
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc"; import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger"; import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type Room as MatrixRoom } from "matrix-js-sdk/lib/matrix"; import {
KnownMembership,
type Room as MatrixRoom,
} from "matrix-js-sdk/lib/matrix";
// eslint-disable-next-line rxjs/no-internal // eslint-disable-next-line rxjs/no-internal
import { type ObservableScope } from "../../ObservableScope"; import { type ObservableScope } from "../../ObservableScope";
@@ -26,7 +29,10 @@ export type RoomMemberMap = Map<
Pick<RoomMember, "userId" | "getMxcAvatarUrl" | "rawDisplayName"> Pick<RoomMember, "userId" | "getMxcAvatarUrl" | "rawDisplayName">
>; >;
export function roomToMembersMap(matrixRoom: MatrixRoom): RoomMemberMap { export function roomToMembersMap(matrixRoom: MatrixRoom): RoomMemberMap {
return matrixRoom.getMembers().reduce((acc, member) => { const members = matrixRoom
.getMembersWithMembership(KnownMembership.Join)
.concat(matrixRoom.getMembersWithMembership(KnownMembership.Invite));
return members.reduce((acc, member) => {
acc.set(member.userId, { acc.set(member.userId, {
userId: member.userId, userId: member.userId,
getMxcAvatarUrl: member.getMxcAvatarUrl.bind(member), getMxcAvatarUrl: member.getMxcAvatarUrl.bind(member),
@@ -47,6 +53,32 @@ export function createRoomMembers$(
roomToMembersMap(matrixRoom), roomToMembersMap(matrixRoom),
); );
} }
/**
* creates the member that this DM is with in case it is a DM (two members) otherwise null
*/
export function createDMMember$(
scope: ObservableScope,
roomMembers$: Behavior<RoomMemberMap>,
matrixRoom: MatrixRoom,
): Behavior<Pick<
RoomMember,
"userId" | "getMxcAvatarUrl" | "rawDisplayName"
> | null> {
// We cannot use the normal direct check from matrix since we do not have access to the account data.
// use primitive member count === 2 check instead.
return scope.behavior(
roomMembers$.pipe(
map((membersMap) => {
// primitive appraoch do to no access to account data.
const isDM = membersMap.size === 2;
if (!isDM) return null;
return matrixRoom.getMember(matrixRoom.guessDMUserId());
}),
),
);
}
/** /**
* Displayname for each member of the call. This will disambiguate * Displayname for each member of the call. This will disambiguate
* any displayname that clashes with another member. Only members * any displayname that clashes with another member. Only members
+137 -2
View File
@@ -1,13 +1,14 @@
/* /*
Copyright 2025 New Vector Ltd. Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
*/ */
import { describe, expect, it } from "vitest"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { BehaviorSubject, combineLatest, Subject } from "rxjs"; import { BehaviorSubject, combineLatest, Subject } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger"; import { logger } from "matrix-js-sdk/lib/logger";
import { sleep } from "matrix-js-sdk/lib/utils";
import { import {
Epoch, Epoch,
@@ -102,3 +103,137 @@ describe("Epoch", () => {
s$.complete(); s$.complete();
}); });
}); });
describe("Reconcile", () => {
beforeEach(() => {
vi.useFakeTimers();
});
afterEach(() => {
vi.useRealTimers();
});
it("should wait clean up before processing next", async () => {
vi.useFakeTimers();
const scope = new ObservableScope();
const behavior$ = new BehaviorSubject<number>(0);
const setup = vi.fn().mockImplementation(async () => await sleep(100));
const cleanup = vi
.fn()
.mockImplementation(async (n: number) => await sleep(100));
scope.reconcile(behavior$, async (value) => {
await setup();
return async (): Promise<void> => {
await cleanup(value);
};
});
// Let the initial setup process
await vi.advanceTimersByTimeAsync(120);
expect(setup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledTimes(0);
// Send next value
behavior$.next(1);
await vi.advanceTimersByTimeAsync(50);
// Should not have started setup for 1 yet
expect(setup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledWith(0);
// Let cleanup finish
await vi.advanceTimersByTimeAsync(50);
// Now setup for 1 should have started
expect(setup).toHaveBeenCalledTimes(2);
});
it("should skip intermediates values that are not setup", async () => {
vi.useFakeTimers();
const scope = new ObservableScope();
const behavior$ = new BehaviorSubject<number>(0);
const setup = vi
.fn()
.mockImplementation(async (n: number) => await sleep(100));
const cleanupLock = Promise.withResolvers();
const cleanup = vi
.fn()
.mockImplementation(async (n: number) => await cleanupLock.promise);
scope.reconcile(behavior$, async (value) => {
await setup(value);
return async (): Promise<void> => {
await cleanup(value);
};
});
// Let the initial setup process (0)
await vi.advanceTimersByTimeAsync(120);
// Send 4 next values quickly
behavior$.next(1);
behavior$.next(2);
behavior$.next(3);
behavior$.next(4);
await vi.advanceTimersByTimeAsync(3000);
// should have only called cleanup for 0
expect(cleanup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledWith(0);
// Let cleanup finish
cleanupLock.resolve(undefined);
await vi.advanceTimersByTimeAsync(120);
// Now setup for 4 should have started, skipping 1,2,3
expect(setup).toHaveBeenCalledTimes(2);
expect(setup).toHaveBeenCalledWith(4);
expect(setup).not.toHaveBeenCalledWith(1);
expect(setup).not.toHaveBeenCalledWith(2);
expect(setup).not.toHaveBeenCalledWith(3);
});
it("should wait for setup to complete before starting cleanup", async () => {
vi.useFakeTimers();
const scope = new ObservableScope();
const behavior$ = new BehaviorSubject<number>(0);
const setup = vi
.fn()
.mockImplementation(async (n: number) => await sleep(3000));
const cleanupLock = Promise.withResolvers();
const cleanup = vi
.fn()
.mockImplementation(async (n: number) => await cleanupLock.promise);
scope.reconcile(behavior$, async (value) => {
await setup(value);
return async (): Promise<void> => {
await cleanup(value);
};
});
await vi.advanceTimersByTimeAsync(500);
// Setup for 0 should be in progress
expect(setup).toHaveBeenCalledTimes(1);
behavior$.next(1);
await vi.advanceTimersByTimeAsync(500);
// Should not have started setup for 1 yet
expect(setup).not.toHaveBeenCalledWith(1);
// Should not have called cleanup yet, because the setup for 0 is not done
expect(cleanup).toHaveBeenCalledTimes(0);
// Let setup for 0 finish
await vi.advanceTimersByTimeAsync(2500 + 100);
// Now cleanup for 0 should have started
expect(cleanup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledWith(0);
cleanupLock.resolve(undefined);
await vi.advanceTimersByTimeAsync(100);
// Now setup for 1 should have started
expect(setup).toHaveBeenCalledWith(1);
});
});
+23 -17
View File
@@ -123,8 +123,22 @@ export class ObservableScope {
callback: (value: T) => Promise<(() => Promise<void>) | void>, callback: (value: T) => Promise<(() => Promise<void>) | void>,
): void { ): void {
let latestValue: T | typeof nothing = nothing; let latestValue: T | typeof nothing = nothing;
let reconciledValue: T | typeof nothing = nothing; let reconcilePromise: Promise<void> | undefined = undefined;
let cleanUp: (() => Promise<void>) | void = undefined; let cleanUp: (() => Promise<void>) | void = undefined;
let prevVal: T | typeof nothing = nothing;
// While this loop runs it will process the latest from `value$` until it caught up with the updates.
// It might skip updates from `value$` and only process the newest value after callback has resolved.
const reconcileLoop = async (): Promise<void> => {
while (latestValue !== prevVal) {
await cleanUp?.(); // Call the previous value's clean-up handler
prevVal = latestValue;
if (latestValue !== nothing) cleanUp = await callback(latestValue); // Sync current value...
// `latestValue` might have gotten updated during the `await callback`. That is why we loop here
}
};
value$ value$
.pipe( .pipe(
catchError(() => EMPTY), // Ignore errors catchError(() => EMPTY), // Ignore errors
@@ -132,23 +146,15 @@ export class ObservableScope {
endWith(nothing), // Clean up when the scope ends endWith(nothing), // Clean up when the scope ends
) )
.subscribe((value) => { .subscribe((value) => {
void (async (): Promise<void> => { // Always track the latest value! The `reconcileLoop` will run until it "processed" the "last" `latestValue`.
if (latestValue === nothing) {
latestValue = value; latestValue = value;
while (latestValue !== reconciledValue) { // There's already an instance of the below 'reconcileLoop' loop running
await cleanUp?.(); // Call the previous value's clean-up handler // concurrently. So lets let the loop handle it. NEVER instanciate two `reconcileLoop`s.
reconciledValue = latestValue; if (reconcilePromise) return;
if (latestValue !== nothing)
cleanUp = await callback(latestValue); // Sync current value reconcilePromise = reconcileLoop().finally(() => {
} reconcilePromise = undefined;
// Reset to signal that reconciliation is done for now });
latestValue = nothing;
} else {
// There's already an instance of the above 'while' loop running
// concurrently. Just update the latest value and let it be handled.
latestValue = value;
}
})();
}); });
} }
+1 -1
View File
@@ -1,5 +1,5 @@
/* /*
Copyright 2025 New Vector Ltd. Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details. Please see LICENSE in the repository root for full details.
+6 -6
View File
@@ -25,7 +25,7 @@ import { roomToMembersMap } from "../state/CallViewModel/remoteMembers/MatrixMem
describe("shouldDisambiguate", () => { describe("shouldDisambiguate", () => {
test("should not disambiguate a solo member", () => { test("should not disambiguate a solo member", () => {
const room = mockMatrixRoom({ const room = mockMatrixRoom({
getMembers: () => [], getMembersWithMembership: () => [],
}); });
expect(shouldDisambiguate(alice, [], roomToMembersMap(room))).toEqual( expect(shouldDisambiguate(alice, [], roomToMembersMap(room))).toEqual(
false, false,
@@ -33,7 +33,7 @@ describe("shouldDisambiguate", () => {
}); });
test("should not disambiguate a member with an empty displayname", () => { test("should not disambiguate a member with an empty displayname", () => {
const room = mockMatrixRoom({ const room = mockMatrixRoom({
getMembers: () => [alice, aliceDoppelganger], getMembersWithMembership: () => [alice, aliceDoppelganger],
}); });
expect( expect(
shouldDisambiguate( shouldDisambiguate(
@@ -44,14 +44,14 @@ describe("shouldDisambiguate", () => {
).toEqual(false); ).toEqual(false);
}); });
test("should disambiguate a member with RTL characters", () => { test("should disambiguate a member with RTL characters", () => {
const room = mockMatrixRoom({ getMembers: () => [] }); const room = mockMatrixRoom({ getMembersWithMembership: () => [] });
expect(shouldDisambiguate(daveRTL, [], roomToMembersMap(room))).toEqual( expect(shouldDisambiguate(daveRTL, [], roomToMembersMap(room))).toEqual(
true, true,
); );
}); });
test("should disambiguate a member with a matching displayname", () => { test("should disambiguate a member with a matching displayname", () => {
const room = mockMatrixRoom({ const room = mockMatrixRoom({
getMembers: () => [alice, aliceDoppelganger], getMembersWithMembership: () => [alice, aliceDoppelganger],
}); });
expect( expect(
shouldDisambiguate( shouldDisambiguate(
@@ -70,7 +70,7 @@ describe("shouldDisambiguate", () => {
}); });
test("should disambiguate a member with a matching displayname with hidden spaces", () => { test("should disambiguate a member with a matching displayname with hidden spaces", () => {
const room = mockMatrixRoom({ const room = mockMatrixRoom({
getMembers: () => [bob, bobZeroWidthSpace], getMembersWithMembership: () => [bob, bobZeroWidthSpace],
}); });
expect( expect(
shouldDisambiguate( shouldDisambiguate(
@@ -91,7 +91,7 @@ describe("shouldDisambiguate", () => {
"should disambiguate a member with a displayname containing a mxid-like string '%s'", "should disambiguate a member with a displayname containing a mxid-like string '%s'",
(rawDisplayName) => { (rawDisplayName) => {
const room = mockMatrixRoom({ const room = mockMatrixRoom({
getMembers: () => [alice, aliceDoppelganger], getMembersWithMembership: () => [alice, aliceDoppelganger],
}); });
expect( expect(
shouldDisambiguate( shouldDisambiguate(
+26
View File
@@ -13,6 +13,8 @@ export enum ErrorCode {
*/ */
MISSING_MATRIX_RTC_TRANSPORT = "MISSING_MATRIX_RTC_TRANSPORT", MISSING_MATRIX_RTC_TRANSPORT = "MISSING_MATRIX_RTC_TRANSPORT",
CONNECTION_LOST_ERROR = "CONNECTION_LOST_ERROR", CONNECTION_LOST_ERROR = "CONNECTION_LOST_ERROR",
INTERNAL_MEMBERSHIP_MANAGER = "INTERNAL_MEMBERSHIP_MANAGER",
FAILED_TO_START_LIVEKIT = "FAILED_TO_START_LIVEKIT",
/** LiveKit indicates that the server has hit its track limits */ /** LiveKit indicates that the server has hit its track limits */
INSUFFICIENT_CAPACITY_ERROR = "INSUFFICIENT_CAPACITY_ERROR", INSUFFICIENT_CAPACITY_ERROR = "INSUFFICIENT_CAPACITY_ERROR",
E2EE_NOT_SUPPORTED = "E2EE_NOT_SUPPORTED", E2EE_NOT_SUPPORTED = "E2EE_NOT_SUPPORTED",
@@ -27,6 +29,7 @@ export enum ErrorCategory {
NETWORK_CONNECTIVITY = "NETWORK_CONNECTIVITY", NETWORK_CONNECTIVITY = "NETWORK_CONNECTIVITY",
CLIENT_CONFIGURATION = "CLIENT_CONFIGURATION", CLIENT_CONFIGURATION = "CLIENT_CONFIGURATION",
UNKNOWN = "UNKNOWN", UNKNOWN = "UNKNOWN",
SYSTEM_FAILURE = "SYSTEM_FAILURE",
// SYSTEM_FAILURE / FEDERATION_FAILURE .. // SYSTEM_FAILURE / FEDERATION_FAILURE ..
} }
@@ -83,6 +86,18 @@ export class ConnectionLostError extends ElementCallError {
} }
} }
export class MembershipManagerError extends ElementCallError {
public constructor(error: Error) {
super(
t("error.membership_manager"),
ErrorCode.INTERNAL_MEMBERSHIP_MANAGER,
ErrorCategory.SYSTEM_FAILURE,
t("error.membership_manager_description"),
error,
);
}
}
export class E2EENotSupportedError extends ElementCallError { export class E2EENotSupportedError extends ElementCallError {
public constructor() { public constructor() {
super( super(
@@ -120,6 +135,17 @@ export class FailToGetOpenIdToken extends ElementCallError {
} }
} }
export class FailToStartLivekitConnection extends ElementCallError {
public constructor(e?: string) {
super(
t("error.failed_to_start_livekit"),
ErrorCode.FAILED_TO_START_LIVEKIT,
ErrorCategory.NETWORK_CONNECTIVITY,
e,
);
}
}
export class InsufficientCapacityError extends ElementCallError { export class InsufficientCapacityError extends ElementCallError {
public constructor() { public constructor() {
super( super(
+5 -2
View File
@@ -20,7 +20,8 @@ import { ConnectionState, type Room as LivekitRoom } from "livekit-client";
import { E2eeType } from "../e2ee/e2eeType"; import { E2eeType } from "../e2ee/e2eeType";
import { import {
CallViewModel, type CallViewModel,
createCallViewModel$,
type CallViewModelOptions, type CallViewModelOptions,
} from "../state/CallViewModel/CallViewModel"; } from "../state/CallViewModel/CallViewModel";
import { import {
@@ -83,6 +84,8 @@ export function getBasicRTCSession(
} as Partial<MatrixClient> as MatrixClient, } as Partial<MatrixClient> as MatrixClient,
getMember: (userId) => matrixRoomMembers.get(userId) ?? null, getMember: (userId) => matrixRoomMembers.get(userId) ?? null,
getMembers: () => Array.from(matrixRoomMembers.values()), getMembers: () => Array.from(matrixRoomMembers.values()),
getMembersWithMembership: () => Array.from(matrixRoomMembers.values()),
guessDMUserId: vitest.fn(),
roomId: matrixRoomId, roomId: matrixRoomId,
on: vitest on: vitest
.fn() .fn()
@@ -143,7 +146,7 @@ export function getBasicCallViewModelEnvironment(
// const remoteParticipants$ = of([aliceParticipant]); // const remoteParticipants$ = of([aliceParticipant]);
const vm = new CallViewModel( const vm = createCallViewModel$(
testScope(), testScope(),
rtcSession.asMockedSession(), rtcSession.asMockedSession(),
matrixRoom, matrixRoom,
+18 -7
View File
@@ -44,12 +44,12 @@ import {
Track, Track,
} from "livekit-client"; } from "livekit-client";
import { randomUUID } from "crypto"; import { randomUUID } from "crypto";
import {
type RoomAndToDeviceEvents,
type RoomAndToDeviceEventsHandlerMap,
} from "matrix-js-sdk/lib/matrixrtc/RoomAndToDeviceKeyTransport";
import { type TrackReference } from "@livekit/components-core"; import { type TrackReference } from "@livekit/components-core";
import EventEmitter from "events"; import EventEmitter from "events";
import {
type KeyTransportEvents,
type KeyTransportEventsHandlerMap,
} from "matrix-js-sdk/lib/matrixrtc/IKeyTransport";
import { import {
LocalUserMediaViewModel, LocalUserMediaViewModel,
@@ -79,6 +79,13 @@ export async function flushPromises(): Promise<void> {
await new Promise<void>((resolve) => window.setTimeout(resolve)); await new Promise<void>((resolve) => window.setTimeout(resolve));
} }
export type NodeEventHandler = (...args: unknown[]) => void;
export interface NodeStyleEventEmitter {
addListener(eventName: string | symbol, handler: NodeEventHandler): this;
removeListener(eventName: string | symbol, handler: NodeEventHandler): this;
}
export interface OurRunHelpers extends RunHelpers { export interface OurRunHelpers extends RunHelpers {
/** /**
* Schedules a sequence of actions to happen, as described by a marble * Schedules a sequence of actions to happen, as described by a marble
@@ -277,6 +284,8 @@ export function mockLivekitRoom(
): LivekitRoom { ): LivekitRoom {
const livekitRoom = { const livekitRoom = {
options: {}, options: {},
setE2EEEnabled: vi.fn(),
...mockEmitter(), ...mockEmitter(),
...room, ...room,
} as Partial<LivekitRoom> as LivekitRoom; } as Partial<LivekitRoom> as LivekitRoom;
@@ -299,7 +308,9 @@ export function mockLocalParticipant(
return { return {
isLocal: true, isLocal: true,
trackPublications: new Map(), trackPublications: new Map(),
unpublishTracks: async () => Promise.resolve(), publishTrack: vi.fn(),
unpublishTracks: vi.fn().mockResolvedValue([]),
createTracks: vi.fn(),
getTrackPublication: () => getTrackPublication: () =>
({}) as Partial<LocalTrackPublication> as LocalTrackPublication, ({}) as Partial<LocalTrackPublication> as LocalTrackPublication,
...mockEmitter(), ...mockEmitter(),
@@ -387,9 +398,9 @@ export function mockConfig(
} }
export class MockRTCSession extends TypedEventEmitter< export class MockRTCSession extends TypedEventEmitter<
MatrixRTCSessionEvent | RoomAndToDeviceEvents | MembershipManagerEvent, MatrixRTCSessionEvent | MembershipManagerEvent | KeyTransportEvents,
KeyTransportEventsHandlerMap &
MatrixRTCSessionEventHandlerMap & MatrixRTCSessionEventHandlerMap &
RoomAndToDeviceEventsHandlerMap &
MembershipManagerEventHandlerMap MembershipManagerEventHandlerMap
> { > {
public asMockedSession(): MockedObject<MatrixRTCSession> { public asMockedSession(): MockedObject<MatrixRTCSession> {
+54 -36
View File
@@ -2731,37 +2731,39 @@ __metadata:
languageName: node languageName: node
linkType: hard linkType: hard
"@livekit/components-core@npm:0.12.10, @livekit/components-core@npm:^0.12.0": "@livekit/components-core@npm:0.12.11, @livekit/components-core@npm:^0.12.0":
version: 0.12.10 version: 0.12.11
resolution: "@livekit/components-core@npm:0.12.10" resolution: "@livekit/components-core@npm:0.12.11"
dependencies: dependencies:
"@floating-ui/dom": "npm:1.6.13" "@floating-ui/dom": "npm:1.6.13"
loglevel: "npm:1.9.1" loglevel: "npm:1.9.1"
rxjs: "npm:7.8.2" rxjs: "npm:7.8.2"
peerDependencies: peerDependencies:
livekit-client: ^2.13.3 livekit-client: ^2.15.14
tslib: ^2.6.2 tslib: ^2.6.2
checksum: 10c0/bfd84fb950f72dd037bd5329658c1e750a8ac6b8f2953ea673e3e944b8ea9d412ef9c98eb8b690052323e03c675964b162aacb00e60530cdc5187f77d21979bd checksum: 10c0/9c2ac3d30bb8cc9067ae0b2049784f81e90e57df9eabf7edbaf3c8ceb65a63f644a4e6abeb6cc38d3ebe52663d8dbb88535e01a965011f365d5ae1f3daf86052
languageName: node languageName: node
linkType: hard linkType: hard
"@livekit/components-react@npm:^2.0.0": "@livekit/components-react@npm:^2.0.0":
version: 2.9.15 version: 2.9.16
resolution: "@livekit/components-react@npm:2.9.15" resolution: "@livekit/components-react@npm:2.9.16"
dependencies: dependencies:
"@livekit/components-core": "npm:0.12.10" "@livekit/components-core": "npm:0.12.11"
clsx: "npm:2.1.1" clsx: "npm:2.1.1"
events: "npm:^3.3.0"
jose: "npm:^6.0.12"
usehooks-ts: "npm:3.1.1" usehooks-ts: "npm:3.1.1"
peerDependencies: peerDependencies:
"@livekit/krisp-noise-filter": ^0.2.12 || ^0.3.0 "@livekit/krisp-noise-filter": ^0.2.12 || ^0.3.0
livekit-client: ^2.13.3 livekit-client: ^2.15.14
react: ">=18" react: ">=18"
react-dom: ">=18" react-dom: ">=18"
tslib: ^2.6.2 tslib: ^2.6.2
peerDependenciesMeta: peerDependenciesMeta:
"@livekit/krisp-noise-filter": "@livekit/krisp-noise-filter":
optional: true optional: true
checksum: 10c0/58a93d85c3b8267d0afd00eceb4f34992ce66124f93450e828a78dd825ecc20e254c3123ed22ec33061a3728f50f4f020ff896769fb3a0ff79656ed8cf452a2b checksum: 10c0/4ba4ff473c5a29d3107412733a6676a3b708d70684ed463e9b34cda26abb3d2f317c2828a52e730837b756de9df3fc248260d6f390aedebfb6ec96ef63c7b151
languageName: node languageName: node
linkType: hard linkType: hard
@@ -5247,12 +5249,12 @@ __metadata:
languageName: node languageName: node
linkType: hard linkType: hard
"@types/node@npm:^22.0.0": "@types/node@npm:^24.0.0":
version: 22.17.0 version: 24.10.0
resolution: "@types/node@npm:22.17.0" resolution: "@types/node@npm:24.10.0"
dependencies: dependencies:
undici-types: "npm:~6.21.0" undici-types: "npm:~7.16.0"
checksum: 10c0/e1c603b660d3de3243dfc02ded5d40623ff3f36315ffbdd8cdc81bc2c5a8da172035879d437b72e9fa61ca01827f28e9c2b0c32898f411a8e9ba0a5efac0b4ca checksum: 10c0/f82ed7194e16f5590ef7afdc20c6d09068c76d50278b485ede8f0c5749683536e3064ffa8def8db76915196afb3724b854aa5723c64d6571b890b14492943b46
languageName: node languageName: node
linkType: hard linkType: hard
@@ -7507,7 +7509,7 @@ __metadata:
"@types/grecaptcha": "npm:^3.0.9" "@types/grecaptcha": "npm:^3.0.9"
"@types/jsdom": "npm:^21.1.7" "@types/jsdom": "npm:^21.1.7"
"@types/lodash-es": "npm:^4.17.12" "@types/lodash-es": "npm:^4.17.12"
"@types/node": "npm:^22.0.0" "@types/node": "npm:^24.0.0"
"@types/pako": "npm:^2.0.3" "@types/pako": "npm:^2.0.3"
"@types/qrcode": "npm:^1.5.5" "@types/qrcode": "npm:^1.5.5"
"@types/react": "npm:^19.0.0" "@types/react": "npm:^19.0.0"
@@ -7545,8 +7547,8 @@ __metadata:
livekit-client: "npm:^2.13.0" livekit-client: "npm:^2.13.0"
lodash-es: "npm:^4.17.21" lodash-es: "npm:^4.17.21"
loglevel: "npm:^1.9.1" loglevel: "npm:^1.9.1"
matrix-js-sdk: "github:matrix-org/matrix-js-sdk#head=toger5/sticky-events&commit=e7f5bec51b6f70501a025b79fe5021c933385b21" matrix-js-sdk: "npm:^39.2.0"
matrix-widget-api: "npm:^1.13.0" matrix-widget-api: "npm:^1.14.0"
normalize.css: "npm:^8.0.1" normalize.css: "npm:^8.0.1"
observable-hooks: "npm:^4.2.3" observable-hooks: "npm:^4.2.3"
pako: "npm:^2.0.4" pako: "npm:^2.0.4"
@@ -9828,6 +9830,13 @@ __metadata:
languageName: node languageName: node
linkType: hard linkType: hard
"jose@npm:^6.0.12":
version: 6.1.2
resolution: "jose@npm:6.1.2"
checksum: 10c0/55f79426f43e652ed6d5de938d50f66bb0a10dcae078db81a23f8d3303e889ce226f000e815f3211f9956bb84badce10da892d130d40fe2eca658045a6f1778e
languageName: node
linkType: hard
"jose@npm:^6.1.0": "jose@npm:^6.1.0":
version: 6.1.0 version: 6.1.0
resolution: "jose@npm:6.1.0" resolution: "jose@npm:6.1.0"
@@ -10110,8 +10119,8 @@ __metadata:
linkType: hard linkType: hard
"livekit-client@npm:^2.13.0": "livekit-client@npm:^2.13.0":
version: 2.15.13 version: 2.16.0
resolution: "livekit-client@npm:2.15.13" resolution: "livekit-client@npm:2.16.0"
dependencies: dependencies:
"@livekit/mutex": "npm:1.1.1" "@livekit/mutex": "npm:1.1.1"
"@livekit/protocol": "npm:1.42.2" "@livekit/protocol": "npm:1.42.2"
@@ -10125,7 +10134,7 @@ __metadata:
webrtc-adapter: "npm:^9.0.1" webrtc-adapter: "npm:^9.0.1"
peerDependencies: peerDependencies:
"@types/dom-mediacapture-record": ^1 "@types/dom-mediacapture-record": ^1
checksum: 10c0/5a061df9000461a6d40ef8aa1e72e8aedc640181cc57fe6f2c48c5c7f90ce96a735b125aede377fc43f4692a685e098f17eeae0f42c5b2fed473305867bf2789 checksum: 10c0/5d03adc5d09efde343ab894db397529dff26117598e773b23a5df90a4fb166bde12c6bb1f2cfd1d28dbaf93fe9f275026d7abb75f2ffd2ba816393a2d58e6c7e
languageName: node languageName: node
linkType: hard linkType: hard
@@ -10343,9 +10352,9 @@ __metadata:
languageName: node languageName: node
linkType: hard linkType: hard
"matrix-js-sdk@github:matrix-org/matrix-js-sdk#head=toger5/sticky-events&commit=e7f5bec51b6f70501a025b79fe5021c933385b21": "matrix-js-sdk@npm:^39.2.0":
version: 38.4.0 version: 39.2.0
resolution: "matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk.git#commit=e7f5bec51b6f70501a025b79fe5021c933385b21" resolution: "matrix-js-sdk@npm:39.2.0"
dependencies: dependencies:
"@babel/runtime": "npm:^7.12.5" "@babel/runtime": "npm:^7.12.5"
"@matrix-org/matrix-sdk-crypto-wasm": "npm:^15.3.0" "@matrix-org/matrix-sdk-crypto-wasm": "npm:^15.3.0"
@@ -10355,23 +10364,23 @@ __metadata:
jwt-decode: "npm:^4.0.0" jwt-decode: "npm:^4.0.0"
loglevel: "npm:^1.9.2" loglevel: "npm:^1.9.2"
matrix-events-sdk: "npm:0.0.1" matrix-events-sdk: "npm:0.0.1"
matrix-widget-api: "npm:^1.10.0" matrix-widget-api: "npm:^1.14.0"
oidc-client-ts: "npm:^3.0.1" oidc-client-ts: "npm:^3.0.1"
p-retry: "npm:7" p-retry: "npm:7"
sdp-transform: "npm:^2.14.1" sdp-transform: "npm:^3.0.0"
unhomoglyph: "npm:^1.0.6" unhomoglyph: "npm:^1.0.6"
uuid: "npm:13" uuid: "npm:13"
checksum: 10c0/7adffdc183affd2d3ee1e8497cad6ca7904a37f98328ff7bc15aa6c1829dc9f9a92f8e1bd6260432a33626ff2a839644de938270163e73438b7294675cd954e4 checksum: 10c0/f8b5261de2744305330ba3952821ca9303698170bfd3a0ff8a767b9286d4e8d4ed5aaf6fbaf8a1e8ff9dbd859102a2a47d882787e2da3b3078965bec00157959
languageName: node languageName: node
linkType: hard linkType: hard
"matrix-widget-api@npm:^1.10.0, matrix-widget-api@npm:^1.13.0": "matrix-widget-api@npm:^1.14.0":
version: 1.13.1 version: 1.15.0
resolution: "matrix-widget-api@npm:1.13.1" resolution: "matrix-widget-api@npm:1.15.0"
dependencies: dependencies:
"@types/events": "npm:^3.0.0" "@types/events": "npm:^3.0.0"
events: "npm:^3.2.0" events: "npm:^3.2.0"
checksum: 10c0/25ded744922755b3eb65f4e171cf6cff1a2e0fe43fc3fecbb13e565e41d8af066daa817dd2c3c7d921b996af399eec3b23df70ab7b682cf422d9cee7ca202512 checksum: 10c0/1c08b5284cd98aed312d95594335e1391d937dfad70ef862a1f90fdbaaa27709e1c44dcda37f8045e4814779d8d5816d240aee396d52cfd9b37fbf243a6baf6a
languageName: node languageName: node
linkType: hard linkType: hard
@@ -12544,7 +12553,7 @@ __metadata:
languageName: node languageName: node
linkType: hard linkType: hard
"sdp-transform@npm:^2.14.1, sdp-transform@npm:^2.15.0": "sdp-transform@npm:^2.15.0":
version: 2.15.0 version: 2.15.0
resolution: "sdp-transform@npm:2.15.0" resolution: "sdp-transform@npm:2.15.0"
bin: bin:
@@ -12553,6 +12562,15 @@ __metadata:
languageName: node languageName: node
linkType: hard linkType: hard
"sdp-transform@npm:^3.0.0":
version: 3.0.0
resolution: "sdp-transform@npm:3.0.0"
bin:
sdp-verify: checker.js
checksum: 10c0/828a4595041ba64c86b29075aa4007ab384519b1fa29882db59ccb83b54b2b2a33b60848293f8da537fe151c52f5844fc17c8325396cac309fb19e2e81ec5bf4
languageName: node
linkType: hard
"sdp@npm:^3.2.0": "sdp@npm:^3.2.0":
version: 3.2.0 version: 3.2.0
resolution: "sdp@npm:3.2.0" resolution: "sdp@npm:3.2.0"
@@ -13603,10 +13621,10 @@ __metadata:
languageName: node languageName: node
linkType: hard linkType: hard
"undici-types@npm:~6.21.0": "undici-types@npm:~7.16.0":
version: 6.21.0 version: 7.16.0
resolution: "undici-types@npm:6.21.0" resolution: "undici-types@npm:7.16.0"
checksum: 10c0/c01ed51829b10aa72fc3ce64b747f8e74ae9b60eafa19a7b46ef624403508a54c526ffab06a14a26b3120d055e1104d7abe7c9017e83ced038ea5cf52f8d5e04 checksum: 10c0/3033e2f2b5c9f1504bdc5934646cb54e37ecaca0f9249c983f7b1fc2e87c6d18399ebb05dc7fd5419e02b2e915f734d872a65da2e3eeed1813951c427d33cc9a
languageName: node languageName: node
linkType: hard linkType: hard