Compare commits

...

94 Commits

Author SHA1 Message Date
Valere
be7407ea3d review: quick renaming 2025-12-04 09:37:07 +01:00
Valere
fdc66a1d62 fix: existing screenshare switching twice 2025-12-03 18:36:51 +01:00
Valere
6e7bedbaec Merge branch 'livekit' into valere/layout_switch 2025-12-03 09:17:31 +01:00
Valere
a93ceeae4b review: Keep previous behavior for now to always auto switch 2025-12-03 09:01:26 +01:00
Valere
b85f36598c fix: mistake in file name 2025-12-03 08:54:52 +01:00
Valere Fedronic
e0bf51b63b Merge pull request #3590 from element-hq/valere/noise_cancellation
Config: UrlParams to control noiseSuppression and echoCancellation
2025-12-02 18:36:32 +01:00
Valere
f6a3a371cb fix lint 2025-12-02 17:52:27 +01:00
Valere
ac9acc0158 review: refactor convert params to object for generateRoomOption 2025-12-02 17:47:00 +01:00
Valere
91f3b8b114 Merge branch 'livekit' into valere/noise_cancellation 2025-12-02 17:44:06 +01:00
Valere
be0c7eb365 review: fix mock import module 2025-12-02 17:43:58 +01:00
Valere
b5d3f3c72a fix: Allow switching back to grid when there is a screenshare 2025-12-02 11:18:50 +01:00
Valere
60bc6f1e93 refactor: Extract layout mode switch + test 2025-12-02 11:00:32 +01:00
Timo
f05d4b158e Merge pull request #3565 from element-hq/renovate/matrix-widget-api
Update dependency matrix-widget-api to v1.15.0
2025-12-01 19:14:52 +01:00
Robin
0240087afe Unpin matrix-js-sdk so it can be upgraded 2025-12-01 13:09:10 -05:00
Timo K
4ec1b19fe2 fix js-sdk tests 2025-12-01 18:54:57 +01:00
Timo K
5f6a45f0e9 also bump js-sdk 2025-12-01 18:45:47 +01:00
renovate[bot]
e2916d9093 Update dependency matrix-widget-api to v1.14.0 2025-12-01 18:23:35 +01:00
Timo
90d4b6a6ce Merge pull request #3598 from element-hq/renovate/github-actions
Update GitHub Actions
2025-12-01 17:42:05 +01:00
Timo
e3eb2ea12b Merge pull request #3599 from element-hq/renovate/embedded-package-dependencies
Update embedded package dependencies
2025-12-01 17:40:29 +01:00
Timo
4f45825cf2 Merge pull request #3591 from element-hq/toger5/fix-local-transport-update
Fix reactivity based on localTransport updates (Cleanup for LocalMembership.ts)
2025-12-01 17:39:55 +01:00
Timo K
8cb8357398 make logs just debug. (info -> debug) 2025-12-01 17:34:37 +01:00
Timo K
63cd4f79dd fix playwright test! (It caught an actual bug!!! so the right wording
would be: fix implementation thanks to the playwright test!
2025-12-01 17:29:35 +01:00
Valere Fedronic
7fd4424e7b Merge pull request #3601 from element-hq/valere/fix_switch_spotlight_screenshare
fix: Automatically switch to spotlight if remote screenshare
2025-12-01 15:30:24 +01:00
Valere
b7fc86ee06 review: move setGridMode close to the value it sets 2025-12-01 15:25:17 +01:00
Timo K
f26aa8f970 fix tests 2025-12-01 15:10:38 +01:00
Valere
1ff479a315 fix: Automatically switch to spotlight if remote screenshare 2025-12-01 15:06:17 +01:00
Timo K
47c6a17d1e review 2025-12-01 14:58:06 +01:00
Timo K
24ed43ce13 Merge branch 'livekit' into toger5/fix-local-transport-update 2025-12-01 14:23:11 +01:00
renovate[bot]
04bbf83ac5 Update GitHub Actions 2025-12-01 11:00:52 +00:00
renovate[bot]
5aa82295fd Update embedded package dependencies 2025-12-01 01:28:55 +00:00
Timo
3331af1108 Merge pull request #3592 from element-hq/valere/fix_flaky_test
test: use `setSystemTime` for better test stability
2025-11-28 22:00:24 +01:00
Timo K
66dece98a5 add more test for publisher 2025-11-28 21:50:22 +01:00
Timo K
2011aef116 skip "Should show error screen if call creation is restricted" on ff 2025-11-28 18:00:04 +01:00
Valere Fedronic
f5c422c559 Merge pull request #3597 from element-hq/valere/test_reconcile
test: Add simple tests for scope.reconcile
2025-11-28 10:00:06 +01:00
Valere
c6bfda94cf test: additional reconcile test 2025-11-28 09:53:18 +01:00
Valere
cdd391b3a2 test: Add simple tests for scope.reconcile 2025-11-28 09:12:05 +01:00
Valere Fedronic
5bc7b31dcd Merge pull request #3593 from element-hq/valere/quality_gate
test: Enable quality gate and touch a file
2025-11-27 18:28:30 +01:00
Timo K
c0913b6546 fix playwright test 2025-11-27 18:02:46 +01:00
Timo K
46f8fe4ec7 fix test errors 2025-11-27 16:43:28 +01:00
Valere
c00e332dd8 Revert "vitest: Add threshold on vitest to check locally"
This reverts commit e5509ba49b.
2025-11-27 16:43:12 +01:00
Valere
e5509ba49b vitest: Add threshold on vitest to check locally 2025-11-27 16:39:45 +01:00
Valere
4c2c7d51cf revert test changes on ObservableScope 2025-11-27 16:38:51 +01:00
Timo K
4b0f6e76c4 revert complete behavior check 2025-11-27 15:38:31 +01:00
Timo K
e5117f705d More testing and cleaning up 2025-11-27 14:42:23 +01:00
Valere
794585514a Revert "add untested function"
This reverts commit c078dd87fa.
2025-11-27 10:04:09 +01:00
Valere
c078dd87fa add untested function 2025-11-26 19:28:44 +01:00
Valere
2b3e2f6188 test: Enable quality gate and touch a file 2025-11-26 19:13:07 +01:00
Valere
acd5dde6d8 test: use setSystemTime for better test stability 2025-11-26 16:27:54 +01:00
Timo K
d22d7460fe Another larger refactor to fix sfu switches and in general proper
cleanup.
2025-11-25 20:18:34 +01:00
Valere
7f3596845c fix formatting 2025-11-25 11:40:38 +01:00
Valere
e2607d6399 Config: UrlParams to control noiseSuppression and echoCancellation 2025-11-25 11:10:53 +01:00
Timo
3491a68362 Merge pull request #3576 from element-hq/toger5/fix-skipped-tests
Refactor local transport testing and local membership initialization
2025-11-24 11:36:27 +01:00
Valere
294b90b6dc fix fixture following web change 2025-11-24 11:29:03 +01:00
Timo K
ddd015d696 fix dev EditInPlace 2025-11-24 10:04:01 +01:00
Timo K
7532f72c90 use edit in place (WIP) This is WIP since it will submite -> reload the
page onSave
2025-11-21 18:49:30 +01:00
Timo K
e60bc9e98f add developer tab snapshot test 2025-11-21 17:44:35 +01:00
Timo
1aa1af2ade Merge pull request #3585 from element-hq/toger5/add-log-screenshare
Add log to screenshare
2025-11-21 16:58:04 +01:00
Timo K
ccadc17406 Add log to screenshare. 2025-11-21 16:19:19 +01:00
Timo K
a731981388 Cleanup transport creation and local store hack. 2025-11-21 14:02:15 +01:00
Timo K
4099c4383d move HomeserverConnected 2025-11-21 13:04:28 +01:00
Timo K
d34775fc47 fix linter 2025-11-20 16:40:06 +01:00
Timo K
a29f0162ad remove widget log 2025-11-20 16:34:53 +01:00
Timo K
a69a50d30c add log to error boudnary 2025-11-20 16:32:40 +01:00
Timo K
89a1bfac2d remove unused test helper 2025-11-20 15:25:53 +01:00
Timo K
1fd9ac93c9 refactor local transport testing and local memberhsip initialization 2025-11-20 15:19:00 +01:00
Timo
f6ef872242 Merge pull request #3575 from element-hq/hs/lint-src-imports
Restrict importing src/lib in most cases for matrix- libs
2025-11-20 14:55:41 +01:00
Half-Shot
07de8483d5 Restrict importing src/lib in most cases for matrix- libs 2025-11-20 13:32:09 +00:00
Valere Fedronic
42b4c8301f Merge pull request #3547 from element-hq/valere/fix_blank_widget_auto_leave
fix: Send close widget action on auto-leave
2025-11-20 11:41:43 +01:00
Valere
c2f541ff8c split test in 2 one testing sound and one to just leave 2025-11-20 11:28:24 +01:00
Valere
c1049b6088 fix import causing lint errors in node_modules 2025-11-20 10:46:24 +01:00
Valere
6577db16ce Merge branch 'livekit' into valere/fix_blank_widget_auto_leave 2025-11-20 10:41:31 +01:00
renovate[bot]
cdc3c2b216 Update LiveKit components (#3573)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-11-19 20:18:18 +01:00
renovate[bot]
47324b52f8 Update dependency livekit-client to v2.16.0 (#3574)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-11-19 20:14:37 +01:00
alutarius
82706cca4f Correct Caddy reverse proxy config example (#3572) 2025-11-18 18:46:00 +01:00
Timo
665a6be484 Merge pull request #3560 from element-hq/voip-team/call-viewmodel-refactor
`CallViewmodel` refactor
2025-11-18 13:48:03 +01:00
Valere
0076b54e52 minor comments/TODO cleaning 2025-11-18 12:14:17 +01:00
Valere
614762b5f8 fixup headers 2025-11-18 10:13:10 +01:00
Timo
222982c52c Merge pull request #3571 from element-hq/toger5/createCallViewModelFunction$
`createCallViewModel$` function refactor
2025-11-17 19:46:06 +01:00
Robin
171c289f2d Merge pull request #3564 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.15.15
2025-11-17 13:41:44 -05:00
renovate[bot]
9ad029d4ff Update dependency livekit-client to v2.15.15 2025-11-17 02:42:33 +00:00
Ben Banfield-Zanin
a3950869fb Use the dedicated bug report sub-domain (#3557)
* Update docs for 'rageshake.submit_url' to use the dedicated subdomain

* Update Netlify preview deployment to use the dedicated subdomain for 'rageshake.submit_url'
2025-11-13 15:33:59 +00:00
Robin
5afc3c2598 Merge pull request #3556 from element-hq/renovate/node-24.x
Update Node.js to v24
2025-11-05 13:55:24 -05:00
Robin
8ad0cb1955 Merge pull request #3551 from element-hq/renovate/embedded-package-dependencies
Update embedded package dependencies to v8.13.0
2025-11-05 13:53:27 -05:00
Robin
9bd464a7f1 Merge pull request #3550 from element-hq/renovate/github-actions
Update softprops/action-gh-release digest to 6da8fa9
2025-11-05 13:53:08 -05:00
Robin
f0adae41ef Merge pull request #3552 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.15.14
2025-11-05 13:49:44 -05:00
renovate[bot]
f9cc930155 Update Node.js to v24 2025-11-05 18:49:21 +00:00
Robin
eebcd133f3 Merge pull request #3553 from element-hq/robin/trusted-publishing
Remove use of npm access token now that trusted publishing is set up
2025-11-05 13:45:52 -05:00
Robin
79bd458dc2 Remove use of npm access token now that trusted publishing is set up
npm has recently limited the lifetime of all access tokens to 90 days (https://gh.io/npm-token-changes), so it would be a bit inconvenient to stick to our current access token-based method of publishing releases. Meanwhile npm has implemented a more secure publishing method based on OIDC in which you tell the registry that a particular GitHub Actions workflow should be a "trusted publisher" for a given package, and then the CLI will authenticate automatically. (https://docs.npmjs.com/trusted-publishers)

I've already set trusted publishing up on the registry side, and since we're already granting the job permission to generate ID tokens for provenance, there should be no additional lines of config needed to make it work. Let's take away the access token and see how this goes next time we release.
2025-11-03 17:13:03 -05:00
renovate[bot]
a4b1ce52cd Update dependency livekit-client to v2.15.14 2025-11-03 01:02:47 +00:00
renovate[bot]
b22a937e7c Update softprops/action-gh-release digest to 6da8fa9 2025-11-01 02:02:21 +00:00
Valere
0309ceb6aa fix lint 2025-10-27 15:59:01 +01:00
Valere
3ecffccd9e fix prettier 2025-10-27 15:41:16 +01:00
Valere
3dd0b156cf fix: Send close widget action on auto-leave 2025-10-27 14:31:23 +01:00
renovate[bot]
214bb75d27 Update embedded package dependencies to v8.13.0 2025-10-01 02:29:37 +00:00
63 changed files with 3364 additions and 954 deletions

View File

@@ -43,6 +43,29 @@ module.exports = {
// To encourage good usage of RxJS:
"rxjs/no-exposed-subjects": "error",
"rxjs/finnish": ["error", { names: { "^this$": false } }],
"no-restricted-imports": [
"error",
{
paths: ["matrix-widget-api", "matrix-js-sdk"].flatMap((lib) =>
["src", "src/", "src/index", "lib", "lib/", "lib/index"]
.map((path) => `${lib}/${path}`)
.map((name) => ({ name, message: `Please use ${lib} instead` })),
),
patterns: [
...["matrix-widget-api"].map((lib) => ({
group: ["src", "src/", "src/**", "lib", "lib/", "lib/**"].map(
(path) => `${lib}/${path}`,
),
message: `Please use ${lib} instead`,
})),
// XXX: We use /lib in lots of places, so allow for now.
...["matrix-js-sdk"].map((lib) => ({
group: ["src", "src/", "src/**"].map((path) => `${lib}/${path}`),
message: `Please use ${lib} instead`,
})),
],
},
],
},
overrides: [
{

View File

@@ -23,7 +23,7 @@ jobs:
packages: write
steps:
- name: Check it out
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: 📥 Download artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -42,7 +42,7 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@c1e51972afc2121e065aed6d45c65596fe445f3f # v5.8.0
uses: docker/metadata-action@318604b99e75e41977312d83839a89be02ca4893 # v5.9.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: ${{ inputs.docker_tags}}

View File

@@ -32,7 +32,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack
run: corepack enable
- name: Yarn cache

View File

@@ -7,7 +7,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack
run: corepack enable
- name: Yarn cache

View File

@@ -85,7 +85,7 @@ jobs:
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
- name: Upload
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with:
files: |
${{ env.FILENAME_PREFIX }}.tar.gz
@@ -100,10 +100,10 @@ jobs:
ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
permissions:
contents: read
id-token: write # required for the provenance flag on npm publish
id-token: write # Allow npm to authenticate as a trusted publisher
steps:
- name: Checkout
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: 📥 Download built element-call artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -126,8 +126,6 @@ jobs:
npm version ${{ needs.versioning.outputs.PREFIXED_VERSION }} --no-git-tag-version
echo "ARTIFACT_VERSION=$(jq '.version' --raw-output package.json)" >> "$GITHUB_ENV"
npm publish --provenance --access public --tag ${{ needs.versioning.outputs.TAG }} ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_RELEASE_TOKEN }}
- id: artifact_version
name: Output artifact version
@@ -144,7 +142,7 @@ jobs:
contents: read
steps:
- name: Checkout
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: 📥 Download built element-call artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
@@ -199,7 +197,7 @@ jobs:
contents: read
steps:
- name: Checkout
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
path: element-call
@@ -212,7 +210,7 @@ jobs:
path: element-call/embedded/ios/Sources/dist
- name: Checkout element-call-swift
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
repository: element-hq/element-call-swift
path: element-call-swift
@@ -264,7 +262,7 @@ jobs:
echo "iOS: ${{ needs.publish_ios.outputs.ARTIFACT_VERSION }}"
- name: Add release notes
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with:
append_body: true
body: |

View File

@@ -42,7 +42,7 @@ jobs:
- name: Create Checksum
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
- name: Upload
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with:
files: |
${{ env.FILENAME_PREFIX }}.tar.gz
@@ -68,7 +68,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Add release note
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with:
append_body: true
body: |

View File

@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack
run: corepack enable
- name: Yarn cache
@@ -33,7 +33,7 @@ jobs:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack
run: corepack enable
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4

View File

@@ -13,7 +13,7 @@ jobs:
steps:
- name: Checkout the code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Enable Corepack
run: corepack enable
@@ -42,7 +42,7 @@ jobs:
- name: Create Pull Request
id: cpr
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
uses: peter-evans/create-pull-request@84ae59a2cdc2258d6fa0732dd66352dddae2a412 # v7.0.9
with:
token: ${{ secrets.ELEMENT_BOT_TOKEN }}
branch: actions/localazy-download

View File

@@ -14,7 +14,7 @@ jobs:
steps:
- name: Checkout the code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- name: Upload
uses: localazy/upload@27e6b5c0fddf4551596b42226b1c24124335d24a # v1

View File

@@ -1 +1 @@
22
24

View File

@@ -13,7 +13,6 @@ coverage:
informational: true
patch:
default:
# Encourage (but don't enforce) 80% coverage on all lines that a PR
# Enforce 80% coverage on all lines that a PR
# touches
target: 80%
informational: true

View File

@@ -18,7 +18,7 @@
"api_host": "https://posthog-element-call.element.io"
},
"rageshake": {
"submit_url": "https://element.io/bugreports/submit"
"submit_url": "https://rageshakes.element.io/api/submit"
},
"sentry": {
"environment": "netlify-pr-preview",

View File

@@ -136,8 +136,8 @@ handle @jwt_service {
reverse_proxy http://[::1]:8080 {
header_up Host {host}
header_up X-Forwarded-Server {host}
header_up X-Real-IP {remote_addr}
header_up X-Forwarded-For {remote_addr}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
}
}
@@ -146,8 +146,8 @@ handle {
reverse_proxy http://localhost:7880 {
header_up Host {host}
header_up X-Forwarded-Server {host}
header_up X-Real-IP {remote_addr}
header_up X-Forwarded-For {remote_addr}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
}
}
```

View File

@@ -96,6 +96,6 @@ These parameters are only supported in the [embedded](./embedded-standalone.md)
| -------------------- | -------------------------------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `posthogApiHost` | Posthog server URL | No | e.g. `https://posthog-element-call.element.io`. Only supported in embedded package. In full package the value from config is used. |
| `posthogApiKey` | Posthog project API key | No | Only supported in embedded package. In full package the value from config is used. |
| `rageshakeSubmitUrl` | Rageshake server URL endpoint | No | e.g. `https://element.io/bugreports/submit`. In full package the value from config is used. |
| `rageshakeSubmitUrl` | Rageshake server URL endpoint | No | e.g. `https://rageshakes.element.io/api/submit`. In full package the value from config is used. |
| `sentryDsn` | Sentry [DSN](https://docs.sentry.io/concepts/key-terms/dsn-explainer/) | No | In full package the value from config is used. |
| `sentryEnvironment` | Sentry [environment](https://docs.sentry.io/concepts/key-terms/key-terms/) | No | In full package the value from config is used. |

View File

@@ -2,11 +2,11 @@
# https://docs.gradle.org/current/userguide/platforms.html#sub::toml-dependencies-format
[versions]
android_gradle_plugin = "8.11.1"
android_gradle_plugin = "8.13.1"
[libraries]
android_gradle_plugin = { module = "com.android.tools.build:gradle", version.ref = "android_gradle_plugin" }
[plugins]
android_library = { id = "com.android.library", version.ref = "android_gradle_plugin" }
maven_publish = { id = "com.vanniktech.maven.publish", version = "0.34.0" }
maven_publish = { id = "com.vanniktech.maven.publish", version = "0.35.0" }

View File

@@ -64,6 +64,14 @@
"developer_mode": {
"always_show_iphone_earpiece": "Show iPhone earpiece option on all platforms",
"crypto_version": "Crypto version: {{version}}",
"custom_livekit_url": {
"current_url": "Currently set to: ",
"from_config": "Currently, no overwrite is set. Url from well-known or config is used.",
"label": "Custom Livekit-url",
"reset": "Reset overwrite",
"save": "Save",
"saving": "Saving..."
},
"debug_tile_layout_label": "Debug tile layout",
"device_id": "Device ID: {{id}}",
"duplicate_tiles_label": "Number of additional tile copies per participant",
@@ -100,11 +108,14 @@
"connection_lost_description": "You were disconnected from the call.",
"e2ee_unsupported": "Incompatible browser",
"e2ee_unsupported_description": "Your web browser does not support encrypted calls. Supported browsers include Chrome, Safari, and Firefox 117+.",
"failed_to_start_livekit": "Failed to start Livekit connection",
"generic": "Something went wrong",
"generic_description": "Submitting debug logs will help us track down the problem.",
"insufficient_capacity": "Insufficient capacity",
"insufficient_capacity_description": "The server has reached its maximum capacity and you cannot join the call at this time. Try again later, or contact your server admin if the problem persists.",
"matrix_rtc_transport_missing": "The server is not configured to work with {{brand}}. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",
"membership_manager": "Membership Manager Error",
"membership_manager_description": "The Membership Manager had to shut down. This is caused by many consequtive failed network requests.",
"open_elsewhere": "Opened in another tab",
"open_elsewhere_description": "{{brand}} has been opened in another tab. If that doesn't sound right, try reloading the page.",
"room_creation_restricted": "Failed to create call",

View File

@@ -71,7 +71,7 @@
"@types/grecaptcha": "^3.0.9",
"@types/jsdom": "^21.1.7",
"@types/lodash-es": "^4.17.12",
"@types/node": "^22.0.0",
"@types/node": "^24.0.0",
"@types/pako": "^2.0.3",
"@types/qrcode": "^1.5.5",
"@types/react": "^19.0.0",
@@ -109,8 +109,8 @@
"livekit-client": "^2.13.0",
"lodash-es": "^4.17.21",
"loglevel": "^1.9.1",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#head=toger5/sticky-events&commit=e7f5bec51b6f70501a025b79fe5021c933385b21",
"matrix-widget-api": "^1.13.0",
"matrix-js-sdk": "^39.2.0",
"matrix-widget-api": "^1.14.0",
"normalize.css": "^8.0.1",
"observable-hooks": "^4.2.3",
"pako": "^2.0.4",

View File

@@ -75,7 +75,12 @@ test("Should automatically retry non fatal JWT errors", async ({
test("Should show error screen if call creation is restricted", async ({
page,
browserName,
}) => {
test.skip(
browserName === "firefox",
"The is test is not working on firefox CI environment.",
);
await page.goto("/");
// We need the socket connection to fail, but this cannot be done by using the websocket route.

View File

@@ -159,7 +159,11 @@ export const widgetTest = test.extend<MyFixtures>({
} = await registerUser(browser, userB);
// Invite the second user
await ewPage1.getByRole("button", { name: "Add", exact: true }).click();
await ewPage1
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.click();
await ewPage1.getByRole("menuitem", { name: "New Room" }).click();
await ewPage1.getByRole("textbox", { name: "Name" }).fill("Welcome Room");
await ewPage1.getByRole("button", { name: "Create room" }).click();

View File

@@ -332,6 +332,42 @@ describe("UrlParams", () => {
expect(computeUrlParams("?intent=join_existing").skipLobby).toBe(false);
});
});
describe("noiseSuppression", () => {
it("defaults to true", () => {
expect(computeUrlParams().noiseSuppression).toBe(true);
});
it("is parsed", () => {
expect(
computeUrlParams("?intent=start_call&noiseSuppression=true")
.noiseSuppression,
).toBe(true);
expect(
computeUrlParams("?intent=start_call&noiseSuppression&bar=foo")
.noiseSuppression,
).toBe(true);
expect(computeUrlParams("?noiseSuppression=false").noiseSuppression).toBe(
false,
);
});
});
describe("echoCancellation", () => {
it("defaults to true", () => {
expect(computeUrlParams().echoCancellation).toBe(true);
});
it("is parsed", () => {
expect(computeUrlParams("?echoCancellation=true").echoCancellation).toBe(
true,
);
expect(computeUrlParams("?echoCancellation=false").echoCancellation).toBe(
false,
);
});
});
describe("header", () => {
it("uses header if provided", () => {
expect(computeUrlParams("?header=app_bar&hideHeader=true").header).toBe(

View File

@@ -233,6 +233,17 @@ export interface UrlConfiguration {
*/
waitForCallPickup: boolean;
/**
* Whether to enable echo cancellation for audio capture.
* Defaults to true.
*/
echoCancellation?: boolean;
/**
* Whether to enable noise suppression for audio capture.
* Defaults to true.
*/
noiseSuppression?: boolean;
callIntent?: RTCCallIntent;
}
interface IntentAndPlatformDerivedConfiguration {
@@ -525,6 +536,8 @@ export const computeUrlParams = (search = "", hash = ""): UrlParams => {
]),
waitForCallPickup: parser.getFlag("waitForCallPickup"),
autoLeaveWhenOthersLeft: parser.getFlag("autoLeave"),
noiseSuppression: parser.getFlagParam("noiseSuppression", true),
echoCancellation: parser.getFlagParam("echoCancellation", true),
};
// Log the final configuration for debugging purposes.

View File

@@ -122,7 +122,7 @@ export interface ConfigOptions {
delayed_leave_event_delay_ms?: number;
/**
* The time (in milliseconds) after which a we consider a delayed event restart http request to have failed.
* The time (in milliseconds) after which we consider a delayed event restart http request to have failed.
* Setting this to a lower value will result in more frequent retries but also a higher chance of failiour.
*
* In the presence of network packet loss (hurting TCP connections), the custom delayedEventRestartLocalTimeoutMs

View File

@@ -21,7 +21,16 @@ export type OpenIDClientParts = Pick<
MatrixClient,
"getOpenIdToken" | "getDeviceId"
>;
/**
* Gets a bearer token from the homeserver and then use it to authenticate
* to the matrix RTC backend in order to get acces to the SFU.
* It has built-in retry for calls to the homeserver with a backoff policy.
* @param client
* @param serviceUrl
* @param matrixRoomId
* @returns Object containing the token information
* @throws FailToGetOpenIdToken
*/
export async function getSFUConfigWithOpenID(
client: OpenIDClientParts,
serviceUrl: string,

View File

@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, test } from "vitest";
import { beforeEach, describe, expect, test, vi } from "vitest";
import { render, configure } from "@testing-library/react";
import { RaisedHandIndicator } from "./RaisedHandIndicator";
@@ -15,6 +15,13 @@ configure({
});
describe("RaisedHandIndicator", () => {
const fixedTime = new Date("2025-01-01T12:00:00.000Z");
beforeEach(() => {
vi.useFakeTimers();
vi.setSystemTime(fixedTime);
});
test("renders nothing when no hand has been raised", () => {
const { container } = render(<RaisedHandIndicator />);
expect(container.firstChild).toBeNull();

View File

@@ -15,7 +15,7 @@ exports[`RaisedHandIndicator > renders a smaller indicator when miniature is spe
</span>
</div>
<p>
00:01
00:00
</p>
</div>
`;
@@ -35,7 +35,7 @@ exports[`RaisedHandIndicator > renders an indicator when a hand has been raised
</span>
</div>
<p>
00:01
00:00
</p>
</div>
`;
@@ -55,7 +55,7 @@ exports[`RaisedHandIndicator > renders an indicator when a hand has been raised
</span>
</div>
<p>
01:01
01:00
</p>
</div>
`;

View File

@@ -22,6 +22,7 @@ import {
WebBrowserIcon,
} from "@vector-im/compound-design-tokens/assets/web/icons";
import { Button } from "@vector-im/compound-web";
import { logger } from "matrix-js-sdk/lib/logger";
import {
ConnectionLostError,
@@ -53,7 +54,7 @@ const ErrorPage: FC<ErrorPageProps> = ({
widget,
}: ErrorPageProps): ReactElement => {
const { t } = useTranslation();
logger.error("Error boundary caught:", error);
let icon: ComponentType<SVGAttributes<SVGElement>>;
switch (error.category) {
case ErrorCategory.CONFIGURATION_ISSUE:

View File

@@ -29,6 +29,7 @@ import userEvent from "@testing-library/user-event";
import { type RelationsContainer } from "matrix-js-sdk/lib/models/relations-container";
import { useState } from "react";
import { TooltipProvider } from "@vector-im/compound-web";
import { type ITransport } from "matrix-widget-api";
import { prefetchSounds } from "../soundUtils";
import { useAudioContext } from "../useAudioContext";
@@ -43,7 +44,7 @@ import {
MockRTCSession,
} from "../utils/test";
import { GroupCallView } from "./GroupCallView";
import { type WidgetHelpers } from "../widget";
import { ElementWidgetActions, type WidgetHelpers } from "../widget";
import { LazyEventEmitter } from "../LazyEventEmitter";
import { MatrixRTCTransportMissingError } from "../utils/errors";
import { ProcessorProvider } from "../livekit/TrackProcessorContext";
@@ -112,6 +113,10 @@ beforeEach(() => {
return (
<div>
<button onClick={() => onLeave("user")}>Leave</button>
<button onClick={() => onLeave("allOthersLeft")}>
SimulateOtherLeft
</button>
<button onClick={() => onLeave("error")}>SimulateErrorLeft</button>
</div>
);
},
@@ -243,6 +248,112 @@ test.skip("GroupCallView plays a leave sound synchronously in widget mode", asyn
expect(leaveRTCSession).toHaveBeenCalledOnce();
});
test.skip("Should close widget when all other left and have time to play a sound", async () => {
const user = userEvent.setup();
const widgetClosedCalled = Promise.withResolvers<void>();
const widgetSendMock = vi.fn().mockImplementation((action: string) => {
if (action === ElementWidgetActions.Close) {
widgetClosedCalled.resolve();
}
});
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const resolvePlaySound = Promise.withResolvers<void>();
playSound = vi.fn().mockReturnValue(resolvePlaySound);
(useAudioContext as MockedFunction<typeof useAudioContext>).mockReturnValue({
playSound,
playSoundLooping: vitest.fn(),
soundDuration: {},
});
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateOtherLeft");
await user.click(leaveButton);
await flushPromises();
expect(widgetSendMock).not.toHaveBeenCalled();
resolvePlaySound.resolve();
await flushPromises();
expect(playSound).toHaveBeenCalledWith("left");
await widgetClosedCalled.promise;
await flushPromises();
expect(widgetStopMock).toHaveBeenCalledOnce();
});
test("Should close widget when all other left", async () => {
const user = userEvent.setup();
const widgetClosedCalled = Promise.withResolvers<void>();
const widgetSendMock = vi.fn().mockImplementation((action: string) => {
if (action === ElementWidgetActions.Close) {
widgetClosedCalled.resolve();
}
});
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateOtherLeft");
await user.click(leaveButton);
await flushPromises();
await widgetClosedCalled.promise;
await flushPromises();
expect(widgetStopMock).toHaveBeenCalledOnce();
});
test("Should not close widget when auto leave due to error", async () => {
const user = userEvent.setup();
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widgetSendMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const alwaysOnScreenSpy = vi.spyOn(widget.api, "setAlwaysOnScreen");
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateErrorLeft");
await user.click(leaveButton);
await flushPromises();
// When onLeft is called, we first set always on screen to false
await waitFor(() => expect(alwaysOnScreenSpy).toHaveBeenCalledWith(false));
await flushPromises();
// But then we do not close the widget automatically
expect(widgetStopMock).not.toHaveBeenCalledOnce();
expect(widgetSendMock).not.toHaveBeenCalledOnce();
});
test.skip("GroupCallView leaves the session when an error occurs", async () => {
(ActiveCall as MockedFunction<typeof ActiveCall>).mockImplementation(() => {
const [error, setError] = useState<Error | null>(null);

View File

@@ -159,6 +159,7 @@ export const GroupCallView: FC<Props> = ({
};
}, [rtcSession]);
// TODO move this into the callViewModel LocalMembership.ts
useTypedEventEmitter(
rtcSession,
MatrixRTCSessionEvent.MembershipManagerError,
@@ -313,7 +314,9 @@ export const GroupCallView: FC<Props> = ({
const navigate = useNavigate();
const onLeft = useCallback(
(reason: "timeout" | "user" | "allOthersLeft" | "decline"): void => {
(
reason: "timeout" | "user" | "allOthersLeft" | "decline" | "error",
): void => {
let playSound: CallEventSounds = "left";
if (reason === "timeout" || reason === "decline") playSound = reason;
@@ -366,7 +369,7 @@ export const GroupCallView: FC<Props> = ({
}
// On a normal user hangup we can shut down and close the widget. But if an
// error occurs we should keep the widget open until the user reads it.
if (reason === "user" && !getUrlParams().returnToLobby) {
if (reason != "error" && !getUrlParams().returnToLobby) {
try {
await widget.api.transport.send(ElementWidgetActions.Close, {});
} catch (e) {
@@ -518,8 +521,7 @@ export const GroupCallView: FC<Props> = ({
}}
onError={
(/**error*/) => {
// TODO this should not be "user". It needs a new case
if (rtcSession.isJoined()) onLeft("user");
if (rtcSession.isJoined()) onLeft("error");
}
}
>

View File

@@ -115,7 +115,9 @@ export interface ActiveCallProps
extends Omit<InCallViewProps, "vm" | "livekitRoom" | "connState"> {
e2eeSystem: EncryptionSystem;
// TODO refactor those reasons into an enum
onLeft: (reason: "user" | "timeout" | "decline" | "allOthersLeft") => void;
onLeft: (
reason: "user" | "timeout" | "decline" | "allOthersLeft" | "error",
) => void;
}
export const ActiveCall: FC<ActiveCallProps> = (props) => {
@@ -144,6 +146,8 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
reactionsReader.reactions$,
scope.behavior(trackProcessorState$),
);
// TODO move this somewhere else once we use the callViewModel in the lobby as well!
vm.join();
setVm(vm);
vm.leave$.pipe(scope.bind()).subscribe(props.onLeft);
@@ -264,7 +268,7 @@ export const InCallView: FC<InCallViewProps> = ({
const sharingScreen = useBehavior(vm.sharingScreen$);
const ringOverlay = useBehavior(vm.ringOverlay$);
const fatalCallError = useBehavior(vm.configError$);
const fatalCallError = useBehavior(vm.fatalError$);
// Stop the rendering and throw for the error boundary
if (fatalCallError) throw fatalCallError;

View File

@@ -0,0 +1,95 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, vi } from "vitest";
import { render, waitFor } from "@testing-library/react";
import type { MatrixClient } from "matrix-js-sdk";
import type { Room as LivekitRoom } from "livekit-client";
import { DeveloperSettingsTab } from "./DeveloperSettingsTab";
// Mock url params hook to avoid environment-dependent snapshot churn.
vi.mock("../UrlParams", () => ({
useUrlParams: (): { mocked: boolean; answer: number } => ({
mocked: true,
answer: 42,
}),
}));
// Provide a minimal mock of a Livekit Room structure used by the component.
function createMockLivekitRoom(
wsUrl: string,
serverInfo: object,
metadata: string,
): { isLocal: boolean; url: string; room: LivekitRoom } {
const mockRoom = {
serverInfo,
metadata,
engine: { client: { ws: { url: wsUrl } } },
} as unknown as LivekitRoom;
return {
isLocal: true,
url: wsUrl,
room: mockRoom,
};
}
// Minimal MatrixClient mock with only the methods used by the component.
function createMockMatrixClient(): MatrixClient {
return {
doesServerSupportUnstableFeature: vi.fn().mockResolvedValue(true), // ensure stickyEventsSupported eventually becomes true
getCrypto: (): { getVersion: () => string } | undefined => ({
getVersion: () => "crypto-1.0.0",
}),
getUserId: () => "@alice:example.org",
getDeviceId: () => "DEVICE123",
} as unknown as MatrixClient;
}
describe("DeveloperSettingsTab", () => {
it("renders and matches snapshot", async () => {
const client = createMockMatrixClient();
const livekitRooms: {
room: LivekitRoom;
url: string;
isLocal?: boolean;
}[] = [
createMockLivekitRoom(
"wss://local-sfu.example.org",
{ region: "local", version: "1.2.3" },
"local-metadata",
),
{
isLocal: false,
url: "wss://remote-sfu.example.org",
room: {
serverInfo: { region: "remote", version: "4.5.6" },
metadata: "remote-metadata",
engine: { client: { ws: { url: "wss://remote-sfu.example.org" } } },
} as unknown as LivekitRoom,
},
];
const { container } = render(
<DeveloperSettingsTab
client={client}
livekitRooms={livekitRooms}
env={{ MY_MOCK_ENV: 10, ENV: "test" } as unknown as ImportMetaEnv}
/>,
);
// Wait for the async sticky events feature check to resolve so the final UI
// (e.g. enabled Matrix_2_0 radio button) appears deterministically.
await waitFor(() =>
expect(client.doesServerSupportUnstableFeature).toHaveBeenCalled(),
);
expect(container).toMatchSnapshot();
});
});

View File

@@ -11,8 +11,8 @@ import {
useCallback,
useEffect,
useMemo,
useState,
useId,
useState,
} from "react";
import { useTranslation } from "react-i18next";
import {
@@ -21,6 +21,7 @@ import {
} from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import {
EditInPlace,
Root as Form,
Heading,
HelpMessage,
@@ -38,6 +39,7 @@ import {
muteAllAudio as muteAllAudioSetting,
alwaysShowIphoneEarpiece as alwaysShowIphoneEarpieceSetting,
matrixRTCMode as matrixRTCModeSetting,
customLivekitUrl as customLivekitUrlSetting,
MatrixRTCMode,
} from "./settings";
import type { Room as LivekitRoom } from "livekit-client";
@@ -47,9 +49,14 @@ import { useUrlParams } from "../UrlParams";
interface Props {
client: MatrixClient;
livekitRooms?: { room: LivekitRoom; url: string; isLocal?: boolean }[];
env: ImportMetaEnv;
}
export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
export const DeveloperSettingsTab: FC<Props> = ({
client,
livekitRooms,
env,
}) => {
const { t } = useTranslation();
const [duplicateTiles, setDuplicateTiles] = useSetting(duplicateTilesSetting);
const [debugTileLayout, setDebugTileLayout] = useSetting(
@@ -85,6 +92,15 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
alwaysShowIphoneEarpieceSetting,
);
const [customLivekitUrl, setCustomLivekitUrl] = useSetting(
customLivekitUrlSetting,
);
const [customLivekitUrlTextBuffer, setCustomLivekitUrlTextBuffer] =
useState(customLivekitUrl);
useEffect(() => {
setCustomLivekitUrlTextBuffer(customLivekitUrl);
}, [customLivekitUrl]);
const [muteAllAudio, setMuteAllAudio] = useSetting(muteAllAudioSetting);
const urlParams = useUrlParams();
@@ -101,7 +117,7 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
}, [livekitRooms]);
return (
<Form>
<>
<p>
{t("developer_mode.hostname", {
hostname: window.location.hostname || "unknown",
@@ -200,55 +216,93 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
)}
/>{" "}
</FieldRow>
<EditInPlace
onSubmit={(e) => e.preventDefault()}
helpLabel={
customLivekitUrl === null
? t("developer_mode.custom_livekit_url.from_config")
: t("developer_mode.custom_livekit_url.current_url") +
customLivekitUrl
}
label={t("developer_mode.custom_livekit_url.label")}
saveButtonLabel={t("developer_mode.custom_livekit_url.save")}
savingLabel={t("developer_mode.custom_livekit_url.saving")}
cancelButtonLabel={t("developer_mode.custom_livekit_url.reset")}
onSave={useCallback(
(e: React.FormEvent<HTMLFormElement>) => {
setCustomLivekitUrl(
customLivekitUrlTextBuffer === ""
? null
: customLivekitUrlTextBuffer,
);
},
[setCustomLivekitUrl, customLivekitUrlTextBuffer],
)}
value={customLivekitUrlTextBuffer ?? ""}
onChange={useCallback(
(event: ChangeEvent<HTMLInputElement>): void => {
setCustomLivekitUrlTextBuffer(event.target.value);
},
[setCustomLivekitUrlTextBuffer],
)}
onCancel={useCallback(
(e: React.FormEvent<HTMLFormElement>) => {
setCustomLivekitUrl(null);
},
[setCustomLivekitUrl],
)}
/>
<Heading as="h3" type="body" weight="semibold" size="lg">
{t("developer_mode.matrixRTCMode.title")}
</Heading>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Legacy}
value={MatrixRTCMode.Legacy}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Legacy.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Legacy.description")}
</HelpMessage>
</InlineField>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Compatibil}
value={MatrixRTCMode.Compatibil}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Comptibility.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Comptibility.description")}
</HelpMessage>
</InlineField>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Matrix_2_0}
value={MatrixRTCMode.Matrix_2_0}
disabled={!stickyEventsSupported}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Matrix_2_0.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Matrix_2_0.description")}
</HelpMessage>
</InlineField>
<Form>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Legacy}
value={MatrixRTCMode.Legacy}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Legacy.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Legacy.description")}
</HelpMessage>
</InlineField>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Compatibil}
value={MatrixRTCMode.Compatibil}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Comptibility.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Comptibility.description")}
</HelpMessage>
</InlineField>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Matrix_2_0}
value={MatrixRTCMode.Matrix_2_0}
disabled={!stickyEventsSupported}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Matrix_2_0.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Matrix_2_0.description")}
</HelpMessage>
</InlineField>
</Form>
{livekitRooms?.map((livekitRoom) => (
<>
<h3>
@@ -270,9 +324,9 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
</>
))}
<p>{t("developer_mode.environment_variables")}</p>
<pre>{JSON.stringify(import.meta.env, null, 2)}</pre>
<pre>{JSON.stringify(env, null, 2)}</pre>
<p>{t("developer_mode.url_params")}</p>
<pre>{JSON.stringify(urlParams, null, 2)}</pre>
</Form>
</>
);
};

View File

@@ -209,7 +209,11 @@ export const SettingsModal: FC<Props> = ({
key: "developer",
name: t("settings.developer_tab_title"),
content: (
<DeveloperSettingsTab client={client} livekitRooms={livekitRooms} />
<DeveloperSettingsTab
env={import.meta.env}
client={client}
livekitRooms={livekitRooms}
/>
),
};

View File

@@ -0,0 +1,411 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
<div>
<p>
Hostname: localhost
</p>
<p>
Element Call version: dev
</p>
<p>
Crypto version: crypto-1.0.0
</p>
<p>
Matrix ID: @alice:example.org
</p>
<p>
Device ID: DEVICE123
</p>
<div
class="fieldRow"
>
<div
class="field inputField"
>
<input
aria-describedby="«r1»"
id="duplicateTiles"
min="0"
type="number"
value="0"
/>
<label
for="duplicateTiles"
>
Number of additional tile copies per participant
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r2»"
id="debugTileLayout"
type="checkbox"
/>
<label
for="debugTileLayout"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Debug tile layout
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r3»"
id="showConnectionStats"
type="checkbox"
/>
<label
for="showConnectionStats"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Show connection statistics
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r4»"
id="muteAllAudio"
type="checkbox"
/>
<label
for="muteAllAudio"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Mute all audio (participants, reactions, join sounds)
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r5»"
id="alwaysShowIphoneEarpiece"
type="checkbox"
/>
<label
for="alwaysShowIphoneEarpiece"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Show iPhone earpiece option on all platforms
</label>
</div>
</div>
<form
class="_root_19upo_16"
>
<div
class="_field_19upo_26"
>
<label
class="_label_19upo_59"
for="radix-«r6»"
>
Custom Livekit-url
</label>
<div
class="_controls_17lij_8"
>
<input
aria-describedby="radix-«r7»"
class="_control_sqdq4_10"
id="radix-«r6»"
name="input"
title=""
value=""
/>
</div>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«r7»"
>
Currently, no overwrite is set. Url from well-known or config is used.
</span>
</div>
</form>
<h3
class="_typography_6v6n8_153 _font-body-lg-semibold_6v6n8_74"
>
MatrixRTC mode
</h3>
<form
class="_root_19upo_16"
>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
checked=""
class="_input_1e0uz_18"
id="radix-«r8»"
name="«r0»"
title=""
type="radio"
value="legacy"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«r8»"
>
Legacy: state events & oldest membership SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«r9»"
>
Compatible with old versions of EC that do not support multi SFU
</span>
</div>
</div>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
class="_input_1e0uz_18"
id="radix-«ra»"
name="«r0»"
title=""
type="radio"
value="compatibil"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«ra»"
>
Compatibility: state events & multi SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«rb»"
>
Compatible with homeservers that do not support sticky events (but all other EC clients are v0.17.0 or later)
</span>
</div>
</div>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
class="_input_1e0uz_18"
id="radix-«rc»"
name="«r0»"
title=""
type="radio"
value="matrix_2_0"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«rc»"
>
Matrix 2.0: sticky events & multi SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«rd»"
>
Compatible only with homservers supporting sticky events and all EC clients v0.17.0 or later
</span>
</div>
</div>
</form>
<h3>
LiveKit SFU: wss://local-sfu.example.org
</h3>
<p>
ws-url:
wss://local-sfu.example.org/
</p>
<p>
LiveKit Server Info
(
local
)
</p>
<pre
class="pre"
>
{
"region": "local",
"version": "1.2.3"
}
local-metadata
</pre>
<h3>
LiveKit SFU: wss://remote-sfu.example.org
</h3>
<p>
LiveKit Server Info
(
remote
)
</p>
<pre
class="pre"
>
{
"region": "remote",
"version": "4.5.6"
}
remote-metadata
</pre>
<p>
Environment variables
</p>
<pre>
{
"MY_MOCK_ENV": 10,
"ENV": "test"
}
</pre>
<p>
URL parameters
</p>
<pre>
{
"mocked": true,
"answer": 42
}
</pre>
</div>
`;

View File

@@ -134,3 +134,8 @@ export const matrixRTCMode = new Setting<MatrixRTCMode>(
"matrix-rtc-mode",
MatrixRTCMode.Legacy,
);
export const customLivekitUrl = new Setting<string | null>(
"custom-livekit-url",
null,
);

View File

@@ -16,7 +16,10 @@ import { BehaviorSubject } from "rxjs";
* distinction between Behaviors and Observables, see
* https://monoid.dk/post/behaviors-and-streams-why-both/.
*/
export type Behavior<T> = Omit<BehaviorSubject<T>, "next" | "observers">;
export type Behavior<T> = Omit<
BehaviorSubject<T>,
"next" | "observers" | "error"
>;
/**
* Creates a Behavior which never changes in value.

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.

View File

@@ -1,12 +1,12 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { test, vi, onTestFinished, it, describe, expect } from "vitest";
import EventEmitter from "events";
import { test, vi, onTestFinished, it, describe } from "vitest";
import {
BehaviorSubject,
combineLatest,
@@ -18,12 +18,11 @@ import {
of,
switchMap,
} from "rxjs";
import { SyncState, type MatrixClient } from "matrix-js-sdk";
import { SyncState } from "matrix-js-sdk";
import {
ConnectionState,
type LocalTrackPublication,
type RemoteParticipant,
type Room as LivekitRoom,
} from "livekit-client";
import * as ComponentsCore from "@livekit/components-core";
import {
@@ -35,27 +34,18 @@ import {
type LivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
import { deepCompare } from "matrix-js-sdk/lib/utils";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { createCallViewModel$ } from "./CallViewModel";
import { type Layout } from "../layout-types.ts";
import {
mockLocalParticipant,
mockMatrixRoom,
mockMatrixRoomMember,
mockRemoteParticipant,
withTestScheduler,
mockRtcMembership,
MockRTCSession,
mockMediaDevices,
mockMuteStates,
mockConfig,
testScope,
mockLivekitRoom,
exampleTransport,
} from "../../utils/test.ts";
import { E2eeType } from "../../e2ee/e2eeType.ts";
import type { RaisedHandInfo, ReactionInfo } from "../../reactions/index.ts";
import {
aliceId,
aliceParticipant,
@@ -70,10 +60,6 @@ import {
import { MediaDevices } from "../MediaDevices.ts";
import { getValue } from "../../utils/observable.ts";
import { type Behavior, constant } from "../Behavior.ts";
import {
type ElementCallError,
MatrixRTCTransportMissingError,
} from "../../utils/errors.ts";
import { withCallViewModel } from "./CallViewModelTestUtils.ts";
vi.mock("rxjs", async (importOriginal) => ({
@@ -244,71 +230,6 @@ function mockRingEvent(
const mockLegacyRingEvent = {} as { event_id: string } & ICallNotifyContent;
describe("CallViewModel", () => {
// TODO: Restore this test. It requires makeTransport to not be mocked, unlike
// the rest of the tests in this file… what do we do?
it.skip("test missing RTC config error", async () => {
const rtcMemberships$ = new BehaviorSubject<CallMembership[]>([]);
const emitter = new EventEmitter();
const client = vi.mocked<MatrixClient>({
on: emitter.on.bind(emitter),
off: emitter.off.bind(emitter),
getSyncState: vi.fn().mockReturnValue(SyncState.Syncing),
getUserId: vi.fn().mockReturnValue("@user:localhost"),
getUser: vi.fn().mockReturnValue(null),
getDeviceId: vi.fn().mockReturnValue("DEVICE"),
credentials: {
userId: "@user:localhost",
},
getCrypto: vi.fn().mockReturnValue(undefined),
getDomain: vi.fn().mockReturnValue("example.org"),
} as unknown as MatrixClient);
const matrixRoom = mockMatrixRoom({
roomId: "!myRoomId:example.com",
client,
getMember: vi.fn().mockReturnValue(undefined),
});
const fakeRtcSession = new MockRTCSession(matrixRoom).withMemberships(
rtcMemberships$,
);
mockConfig({});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({});
const callVM = createCallViewModel$(
testScope(),
fakeRtcSession.asMockedSession(),
matrixRoom,
mockMediaDevices({}),
mockMuteStates(),
{
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
autoLeaveWhenOthersLeft: false,
livekitRoomFactory: (): LivekitRoom =>
mockLivekitRoom({
localParticipant,
disconnect: async () => Promise.resolve(),
setE2EEEnabled: async () => Promise.resolve(),
}),
},
new BehaviorSubject({} as Record<string, RaisedHandInfo>),
new BehaviorSubject({} as Record<string, ReactionInfo>),
constant({ processor: undefined, supported: false }),
);
const failPromise = Promise.withResolvers<ElementCallError>();
callVM.configError$.subscribe((error) => {
if (error) {
failPromise.resolve(error);
}
});
const error = await failPromise.promise;
expect(error).toBeInstanceOf(MatrixRTCTransportMissingError);
});
test("participants are retained during a focus switch", () => {
withTestScheduler(({ behavior, expectObservable }) => {
// Participants disappear on frame 2 and come back on frame 3
@@ -346,7 +267,7 @@ describe("CallViewModel", () => {
});
});
it.skip("screen sharing activates spotlight layout", () => {
test("screen sharing activates spotlight layout", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => {
// Start with no screen shares, then have Alice and Bob share their screens,
// then return to no screen shares, then have just Alice share for a bit

View File

@@ -1,4 +1,5 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2023, 2024, 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -16,7 +17,6 @@ import { type Room as MatrixRoom } from "matrix-js-sdk";
import {
combineLatest,
distinctUntilChanged,
EMPTY,
filter,
fromEvent,
map,
@@ -27,7 +27,6 @@ import {
pairwise,
race,
scan,
skip,
skipWhile,
startWith,
Subject,
@@ -40,7 +39,10 @@ import {
timer,
} from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type MatrixRTCSession } from "matrix-js-sdk/lib/matrixrtc";
import {
type LivekitTransport,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { type IWidgetApiRequest } from "matrix-widget-api";
import {
@@ -93,9 +95,11 @@ import {
} from "../layout-types.ts";
import { type ElementCallError } from "../../utils/errors.ts";
import { type ObservableScope } from "../ObservableScope.ts";
import { createHomeserverConnected$ } from "./localMember/HomeserverConnected.ts";
import {
createLocalMembership$,
type LocalMemberConnectionState,
enterRTCSession,
RTCBackendState,
} from "./localMember/LocalMembership.ts";
import { createLocalTransport$ } from "./localMember/LocalTransport.ts";
import {
@@ -119,6 +123,9 @@ import {
createMatrixMemberMetadata$,
createRoomMembers$,
} from "./remoteMembers/MatrixMemberMetadata.ts";
import { Publisher } from "./localMember/Publisher.ts";
import { type Connection } from "./remoteMembers/Connection.ts";
import { createLayoutModeSwitch } from "./LayoutSwitch.ts";
const logger = rootLogger.getChild("[CallViewModel]");
//TODO
@@ -193,7 +200,7 @@ export interface CallViewModel {
hangup: () => void;
// joining
join: () => LocalMemberConnectionState;
join: () => void;
// screen sharing
/**
@@ -229,7 +236,7 @@ export interface CallViewModel {
* This is a fatal error that prevents the call from being created/joined.
* Should render a blocking error screen.
*/
configError$: Behavior<ElementCallError | null>;
fatalError$: Behavior<ElementCallError | null>;
// participants and counts
/**
@@ -336,6 +343,7 @@ export interface CallViewModel {
// DISCUSSION own membership manager ALSO this probably can be simplifis
reconnecting$: Behavior<boolean>;
}
/**
* A view model providing all the application logic needed to show the in-call
* UI (may eventually be expanded to cover the lobby and feedback screens in the
@@ -356,9 +364,9 @@ export function createCallViewModel$(
reactionsSubject$: Observable<Record<string, ReactionInfo>>,
trackProcessorState$: Behavior<ProcessorState>,
): CallViewModel {
const userId = matrixRoom.client.getUserId()!;
const deviceId = matrixRoom.client.getDeviceId()!;
const client = matrixRoom.client;
const userId = client.getUserId()!;
const deviceId = client.getDeviceId()!;
const livekitKeyProvider = getE2eeKeyProvider(
options.encryptionSystem,
matrixRTCSession,
@@ -370,8 +378,8 @@ export function createCallViewModel$(
// For mocking purposes it is recommended to only mock the functions creating those outputs.
// All other fields are just temp computations for the mentioned output.
// The class does not need anything except the values underneath the bar.
// The creation of the values under the bar are all tested independently and testing the callViewModel Should
// not test their cretation. Call view model only needs:
// The creations of the values under the bar are all tested independently and testing the callViewModel Should
// not test their creation. Call view model only needs:
// - memberships$ via createMemberships$
// - localMembership via createLocalMembership$
// - callLifecycle via createCallNotificationLifecycle$
@@ -392,7 +400,7 @@ export function createCallViewModel$(
const localTransport$ = createLocalTransport$({
scope: scope,
memberships$: memberships$,
client: matrixRoom.client,
client,
roomId: matrixRoom.roomId,
useOldestMember$: scope.behavior(
matrixRTCMode.value$.pipe(map((v) => v === MatrixRTCMode.Legacy)),
@@ -400,12 +408,14 @@ export function createCallViewModel$(
});
const connectionFactory = new ECConnectionFactory(
matrixRoom.client,
client,
mediaDevices,
trackProcessorState$,
livekitKeyProvider,
getUrlParams().controlledAudioDevices,
options.livekitRoomFactory,
getUrlParams().echoCancellation,
getUrlParams().noiseSuppression,
);
const connectionManager = createConnectionManager$({
@@ -437,7 +447,7 @@ export function createCallViewModel$(
matrixRTCMode.value$.pipe(
map((mode) => ({
encryptMedia: livekitKeyProvider !== undefined,
// TODO. This might need to get called again on each cahnge of matrixRTCMode...
// TODO. This might need to get called again on each change of matrixRTCMode...
matrixRTCMode: mode,
})),
),
@@ -445,15 +455,34 @@ export function createCallViewModel$(
const localMembership = createLocalMembership$({
scope: scope,
homeserverConnected$: createHomeserverConnected$(
scope,
client,
matrixRTCSession,
),
muteStates: muteStates,
mediaDevices: mediaDevices,
joinMatrixRTC: async (transport: LivekitTransport) => {
return enterRTCSession(
matrixRTCSession,
transport,
connectOptions$.value,
);
},
createPublisherFactory: (connection: Connection) => {
return new Publisher(
scope,
connection,
mediaDevices,
muteStates,
trackProcessorState$,
logger.getChild(
"[Publisher" + connection.transport.livekit_service_url + "]",
),
);
},
connectionManager: connectionManager,
matrixRTCSession: matrixRTCSession,
matrixRoom: matrixRoom,
localTransport$: localTransport$,
trackProcessorState$: trackProcessorState$,
widget,
options: connectOptions$,
logger: logger.getChild(`[${Date.now()}]`),
});
@@ -547,15 +576,6 @@ export function createCallViewModel$(
),
);
// CODESMELL?
// This is functionally the same Observable as leave$, except here it's
// hoisted to the top of the class. This enables the cyclic dependency between
// leave$ -> autoLeave$ -> callPickupState$ -> livekitConnectionState$ ->
// localConnection$ -> transports$ -> joined$ -> leave$.
const leaveHoisted$ = new Subject<
"user" | "timeout" | "decline" | "allOthersLeft"
>();
/**
* Whether various media/event sources should pretend to be disconnected from
* all network input, even if their connection still technically works.
@@ -566,7 +586,6 @@ export function createCallViewModel$(
// in a split-brained state.
// DISCUSSION own membership manager ALSO this probably can be simplifis
const reconnecting$ = localMembership.reconnecting$;
const pretendToBeDisconnected$ = reconnecting$;
const audioParticipants$ = scope.behavior(
matrixLivekitMembers$.pipe(
@@ -615,7 +634,7 @@ export function createCallViewModel$(
);
const handsRaised$ = scope.behavior(
handsRaisedSubject$.pipe(pauseWhen(pretendToBeDisconnected$)),
handsRaisedSubject$.pipe(pauseWhen(reconnecting$)),
);
const reactions$ = scope.behavior(
@@ -628,14 +647,13 @@ export function createCallViewModel$(
]),
),
),
pauseWhen(pretendToBeDisconnected$),
pauseWhen(reconnecting$),
),
);
/**
* List of user media (camera feeds) that we want tiles for.
*/
// TODO this also needs the local participant to be added.
const userMedia$ = scope.behavior<UserMedia[]>(
combineLatest([
localMatrixLivekitMember$,
@@ -650,7 +668,7 @@ export function createCallViewModel$(
{ value: matrixLivekitMembers },
duplicateTiles,
]) {
let localParticipantId = undefined;
let localParticipantId: string | undefined = undefined;
// add local member if available
if (localMatrixLivekitMember) {
const { userId, participant$, connection$, membership$ } =
@@ -720,7 +738,7 @@ export function createCallViewModel$(
livekitRoom$,
focusUrl$,
mediaDevices,
pretendToBeDisconnected$,
reconnecting$,
displayName$,
matrixMemberMetadataStore.createAvatarUrlBehavior$(userId),
handsRaised$.pipe(map((v) => v[participantId]?.time ?? null)),
@@ -816,10 +834,7 @@ export function createCallViewModel$(
merge(
autoLeave$,
merge(userHangup$, widgetHangup$).pipe(map(() => "user" as const)),
).pipe(
scope.share,
tap((reason) => leaveHoisted$.next(reason)),
);
).pipe(scope.share);
const spotlightSpeaker$ = scope.behavior<UserMediaViewModel | null>(
userMedia$.pipe(
@@ -968,36 +983,11 @@ export function createCallViewModel$(
spotlightExpandedToggle$.pipe(accumulate(false, (expanded) => !expanded)),
);
const gridModeUserSelection$ = new Subject<GridMode>();
/**
* The layout mode of the media tile grid.
*/
const gridMode$ =
// If the user hasn't selected spotlight and somebody starts screen sharing,
// automatically switch to spotlight mode and reset when screen sharing ends
scope.behavior<GridMode>(
gridModeUserSelection$.pipe(
switchMap((userSelection) =>
(userSelection === "spotlight"
? EMPTY
: combineLatest([hasRemoteScreenShares$, windowMode$]).pipe(
skip(userSelection === null ? 0 : 1),
map(
([hasScreenShares, windowMode]): GridMode =>
hasScreenShares || windowMode === "flat"
? "spotlight"
: "grid",
),
)
).pipe(startWith(userSelection ?? "grid")),
),
),
"grid",
);
const setGridMode = (value: GridMode): void => {
gridModeUserSelection$.next(value);
};
const { setGridMode, gridMode$ } = createLayoutModeSwitch(
scope,
windowMode$,
hasRemoteScreenShares$,
);
const gridLayoutMedia$: Observable<GridLayoutMedia> = combineLatest(
[grid$, spotlight$],
@@ -1424,16 +1414,13 @@ export function createCallViewModel$(
// reassigned here to make it publicly accessible
const toggleScreenSharing = localMembership.toggleScreenSharing;
const join = localMembership.requestConnect;
// TODO-MULTI-SFU: Use this view model for the lobby as well, and only call this once 'join' is clicked?
join();
return {
autoLeave$: autoLeave$,
callPickupState$: callPickupState$,
ringOverlay$: ringOverlay$,
leave$: leave$,
hangup: (): void => userHangup$.next(),
join: join,
join: localMembership.requestConnect,
toggleScreenSharing: toggleScreenSharing,
sharingScreen$: sharingScreen$,
@@ -1442,7 +1429,14 @@ export function createCallViewModel$(
hoverScreen: (): void => screenHover$.next(),
unhoverScreen: (): void => screenUnhover$.next(),
configError$: localMembership.configError$,
fatalError$: scope.behavior(
localMembership.connectionState.livekit$.pipe(
filter((v) => v.state === RTCBackendState.Error),
map((s) => s.error),
),
null,
),
participantCount$: participantCount$,
audioParticipants$: audioParticipants$,
@@ -1474,8 +1468,6 @@ export function createCallViewModel$(
reconnecting$: reconnecting$,
};
}
// TODO-MULTI-SFU // Setup and update the keyProvider which was create by `createRoom` was a thing before. Now we never update if the E2EEsystem changes
// do we need this?
function getE2eeKeyProvider(
e2eeSystem: EncryptionSystem,

View File

@@ -0,0 +1,202 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, test } from "vitest";
import { firstValueFrom, of } from "rxjs";
import { createLayoutModeSwitch } from "./LayoutSwitch";
import { ObservableScope } from "../ObservableScope";
import { constant } from "../Behavior";
import { withTestScheduler } from "../../utils/test";
let scope: ObservableScope;
beforeEach(() => {
scope = new ObservableScope();
});
afterEach(() => {
scope.end();
});
describe("Default mode", () => {
test("Should be in grid layout by default", async () => {
const { gridMode$ } = createLayoutModeSwitch(
scope,
constant("normal"),
of(false),
);
const mode = await firstValueFrom(gridMode$);
expect(mode).toBe("grid");
});
test("Should switch to spotlight mode when window mode is flat", async () => {
const { gridMode$ } = createLayoutModeSwitch(
scope,
constant("flat"),
of(false),
);
const mode = await firstValueFrom(gridMode$);
expect(mode).toBe("spotlight");
});
});
test("Should allow switching modes manually", () => {
withTestScheduler(({ cold, behavior, expectObservable, schedule }): void => {
const { gridMode$, setGridMode } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold("f", { f: false, t: true }),
);
schedule("--sgs", {
s: () => setGridMode("spotlight"),
g: () => setGridMode("grid"),
});
expectObservable(gridMode$).toBe("g-sgs", {
g: "grid",
s: "spotlight",
});
});
});
test("Should switch to spotlight mode when there is a remote screen share", () => {
withTestScheduler(({ cold, behavior, expectObservable }): void => {
const shareMarble = "f--t";
const gridsMarble = "g--s";
const { gridMode$ } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold(shareMarble, { f: false, t: true }),
);
expectObservable(gridMode$).toBe(gridsMarble, {
g: "grid",
s: "spotlight",
});
});
});
test("Can manually force grid when there is a screenshare", () => {
withTestScheduler(({ cold, behavior, expectObservable, schedule }): void => {
const { gridMode$, setGridMode } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold("-ft", { f: false, t: true }),
);
schedule("---g", {
g: () => setGridMode("grid"),
});
expectObservable(gridMode$).toBe("ggsg", {
g: "grid",
s: "spotlight",
});
});
});
test("Should auto-switch after manually selected grid", () => {
withTestScheduler(({ cold, behavior, expectObservable, schedule }): void => {
const { gridMode$, setGridMode } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
// Two screenshares will happen in sequence
cold("-ft-ft", { f: false, t: true }),
);
// There was a screen-share that forced spotlight, then
// the user manually switch back to grid
schedule("---g", {
g: () => setGridMode("grid"),
});
// If we did want to respect manual selection, the expectation would be:
// const expectation = "ggsg";
const expectation = "ggsg-s";
expectObservable(gridMode$).toBe(expectation, {
g: "grid",
s: "spotlight",
});
});
});
test("Should switch back to grid mode when the remote screen share ends", () => {
withTestScheduler(({ cold, behavior, expectObservable }): void => {
const shareMarble = "f--t--f-";
const gridsMarble = "g--s--g-";
const { gridMode$ } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold(shareMarble, { f: false, t: true }),
);
expectObservable(gridMode$).toBe(gridsMarble, {
g: "grid",
s: "spotlight",
});
});
});
test("can auto-switch to spotlight again after first screen share ends", () => {
withTestScheduler(({ cold, behavior, expectObservable }): void => {
const shareMarble = "ftft";
const gridsMarble = "gsgs";
const { gridMode$ } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold(shareMarble, { f: false, t: true }),
);
expectObservable(gridMode$).toBe(gridsMarble, {
g: "grid",
s: "spotlight",
});
});
});
test("can switch manually to grid after screen share while manually in spotlight", () => {
withTestScheduler(({ cold, behavior, schedule, expectObservable }): void => {
// Initially, no one is sharing. Then the user manually switches to
// spotlight. After a screen share starts, the user manually switches to
// grid.
const shareMarbles = " f-t-";
const setModeMarbles = "-s-g";
const expectation = " gs-g";
const { gridMode$, setGridMode } = createLayoutModeSwitch(
scope,
behavior("n", { n: "normal" }),
cold(shareMarbles, { f: false, t: true }),
);
schedule(setModeMarbles, {
g: () => setGridMode("grid"),
s: () => setGridMode("spotlight"),
});
expectObservable(gridMode$).toBe(expectation, {
g: "grid",
s: "spotlight",
});
});
});
test("Should auto-switch to spotlight when in flat window mode", () => {
withTestScheduler(({ cold, behavior, expectObservable }): void => {
const { gridMode$ } = createLayoutModeSwitch(
scope,
behavior("naf", { n: "normal", a: "narrow", f: "flat" }),
cold("f", { f: false, t: true }),
);
expectObservable(gridMode$).toBe("g-s-", {
g: "grid",
s: "spotlight",
});
});
});

View File

@@ -0,0 +1,130 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
BehaviorSubject,
combineLatest,
map,
type Observable,
scan,
} from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { type GridMode, type WindowMode } from "./CallViewModel.ts";
import { type Behavior } from "../Behavior.ts";
import { type ObservableScope } from "../ObservableScope.ts";
/**
* Creates a layout mode switch that allows switching between grid and spotlight modes.
* The actual layout mode can be overridden to spotlight mode if there is a remote screen share active
* or if the window mode is flat.
*
* @param scope - The observable scope to manage subscriptions.
* @param windowMode$ - The current window mode observable.
* @param hasRemoteScreenShares$ - An observable indicating if there are remote screen shares active.
*/
export function createLayoutModeSwitch(
scope: ObservableScope,
windowMode$: Behavior<WindowMode>,
hasRemoteScreenShares$: Observable<boolean>,
): {
gridMode$: Behavior<GridMode>;
setGridMode: (value: GridMode) => void;
} {
const gridModeUserSelection$ = new BehaviorSubject<GridMode>("grid");
// Callback to set the grid mode desired by the user.
// Notice that this is only a preference, the actual grid mode can be overridden
// if there is a remote screen share active.
const setGridMode = (value: GridMode): void => {
gridModeUserSelection$.next(value);
};
/**
* The layout mode of the media tile grid.
*/
const gridMode$ =
// If the user hasn't selected spotlight and somebody starts screen sharing,
// automatically switch to spotlight mode and reset when screen sharing ends
scope.behavior<GridMode>(
combineLatest([
gridModeUserSelection$,
hasRemoteScreenShares$,
windowMode$,
]).pipe(
// Scan to keep track if we have auto-switched already or not.
// To allow the user to override the auto-switch by selecting grid mode again.
scan<
[GridMode, boolean, WindowMode],
{
mode: GridMode;
/** Remember if the change was user driven or not */
hasAutoSwitched: boolean;
/** To know if it is new screen share or an already handled */
hasScreenShares: boolean;
}
>(
(prev, [userSelection, hasScreenShares, windowMode]) => {
const isFlatMode = windowMode === "flat";
// Always force spotlight in flat mode, grid layout is not supported
// in that mode.
// TODO: strange that we do that for flat mode but not for other modes?
// TODO: Why is this not handled in layoutMedia$ like other window modes?
if (isFlatMode) {
logger.debug(`Forcing spotlight mode, windowMode=${windowMode}`);
return {
mode: "spotlight",
hasAutoSwitched: prev.hasAutoSwitched,
hasScreenShares,
};
}
// User explicitly chose spotlight.
// Respect that choice.
if (userSelection === "spotlight") {
return {
mode: "spotlight",
hasAutoSwitched: prev.hasAutoSwitched,
hasScreenShares,
};
}
// User has chosen grid mode. If a screen share starts, we will
// auto-switch to spotlight mode for better experience.
// But we only do it once, if the user switches back to grid mode,
// we respect that choice until they explicitly change it again.
const isNewShare = hasScreenShares && !prev.hasScreenShares;
if (isNewShare && !prev.hasAutoSwitched) {
return {
mode: "spotlight",
hasAutoSwitched: true,
hasScreenShares: true,
};
}
// Respect user's grid choice
// XXX If we want to forbid switching automatically again after we can
// return hasAutoSwitched: acc.hasAutoSwitched here instead of setting to false.
return {
mode: "grid",
hasAutoSwitched: false,
hasScreenShares,
};
},
// initial value
{ mode: "grid", hasAutoSwitched: false, hasScreenShares: false },
),
map(({ mode }) => mode),
),
"grid",
);
return {
gridMode$,
setGridMode,
};
}

View File

@@ -0,0 +1,202 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { EventEmitter } from "events";
import { ClientEvent, SyncState } from "matrix-js-sdk";
import { MembershipManagerEvent, Status } from "matrix-js-sdk/lib/matrixrtc";
import { ObservableScope } from "../../ObservableScope";
import { createHomeserverConnected$ } from "./HomeserverConnected";
/**
* Minimal stub of a Matrix client sufficient for our tests:
```
createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
)
```
*/
class MockMatrixClient extends EventEmitter {
private syncState: SyncState;
public constructor(initial: SyncState) {
super();
this.syncState = initial;
}
public setSyncState(state: SyncState): void {
this.syncState = state;
// Matrix's Sync event in createHomeserverConnected$ expects [SyncState]
this.emit(ClientEvent.Sync, [state]);
}
public getSyncState(): SyncState {
return this.syncState;
}
}
/**
* Minimal stub of MatrixRTCSession (membership manager):
```
createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
)
```
*/
class MockMatrixRTCSession extends EventEmitter {
public membershipStatus: Status;
public probablyLeft: boolean;
public constructor(props: {
membershipStatus: Status;
probablyLeft: boolean;
}) {
super();
this.membershipStatus = props.membershipStatus;
this.probablyLeft = props.probablyLeft;
}
public setMembershipStatus(status: Status): void {
this.membershipStatus = status;
this.emit(MembershipManagerEvent.StatusChanged);
}
public setProbablyLeft(flag: boolean): void {
this.probablyLeft = flag;
this.emit(MembershipManagerEvent.ProbablyLeft);
}
}
describe("createHomeserverConnected$", () => {
let scope: ObservableScope;
let client: MockMatrixClient;
let session: MockMatrixRTCSession;
beforeEach(() => {
scope = new ObservableScope();
client = new MockMatrixClient(SyncState.Error); // start disconnected
session = new MockMatrixRTCSession({
membershipStatus: Status.Disconnected,
probablyLeft: false,
});
});
afterEach(() => {
scope.end();
});
// LLM generated test cases. They are a bit overkill but I improved the mocking so it is
// easy enough to read them so I think they can stay.
it("is false when sync state is not Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
expect(hsConnected$.value).toBe(false);
});
it("remains false while membership status is not Connected even if sync is Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false); // membership still disconnected
});
it("is false when membership status transitions to Connected but ProbablyLeft is true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Make sync loop OK
client.setSyncState(SyncState.Syncing);
// Indicate probable leave before connection
session.setProbablyLeft(true);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(false);
});
it("becomes true only when all three conditions are satisfied", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// 1. Sync loop connected
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false); // not yet membership connected
// 2. Membership connected
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true); // probablyLeft is false
});
it("drops back to false when sync loop leaves Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Reach connected state
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
// Sync loop error => should flip false
client.setSyncState(SyncState.Error);
expect(hsConnected$.value).toBe(false);
});
it("drops back to false when membership status becomes disconnected", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setMembershipStatus(Status.Disconnected);
expect(hsConnected$.value).toBe(false);
});
it("drops to false when ProbablyLeft is emitted after being true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
});
it("recovers to true if ProbablyLeft becomes false again while other conditions remain true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
// Simulate clearing the flag (in realistic scenario membership manager would update)
session.setProbablyLeft(false);
expect(hsConnected$.value).toBe(true);
});
it("composite sequence reflects each individual failure reason", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Initially false (sync error + disconnected + not probably left)
expect(hsConnected$.value).toBe(false);
// Fix sync only
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false);
// Fix membership
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
// Introduce probablyLeft -> false
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
// Restore notProbablyLeft -> true again
session.setProbablyLeft(false);
expect(hsConnected$.value).toBe(true);
// Drop sync -> false
client.setSyncState(SyncState.Error);
expect(hsConnected$.value).toBe(false);
});
});

View File

@@ -0,0 +1,85 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
MembershipManagerEvent,
Status,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { ClientEvent, type MatrixClient, SyncState } from "matrix-js-sdk";
import { fromEvent, startWith, map, tap, type Observable } from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type ObservableScope } from "../../ObservableScope";
import { type Behavior } from "../../Behavior";
import { and$ } from "../../../utils/observable";
import { type NodeStyleEventEmitter } from "../../../utils/test";
/**
* Logger instance (scoped child) for homeserver connection updates.
*/
const logger = rootLogger.getChild("[HomeserverConnected]");
/**
* Behavior representing whether we consider ourselves connected to the Matrix homeserver
* for the purposes of a MatrixRTC session.
*
* Becomes FALSE if ANY sub-condition is fulfilled:
* 1. Sync loop is not in SyncState.Syncing
* 2. membershipStatus !== Status.Connected
* 3. probablyLeft === true
*/
export function createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
): Behavior<boolean> {
const syncing$ = (
fromEvent(client, ClientEvent.Sync) as Observable<[SyncState]>
).pipe(
startWith([client.getSyncState()]),
map(([state]) => state === SyncState.Syncing),
);
const membershipConnected$ = fromEvent(
matrixRTCSession,
MembershipManagerEvent.StatusChanged,
).pipe(
startWith(null),
map(() => matrixRTCSession.membershipStatus === Status.Connected),
);
// This is basically notProbablyLeft$
//
// probablyLeft is computed by a local timer that mimics the server delayed event.
// If we locally predict our server event timed out. We consider ourselves as probablyLeft
// even though we might not yet have received the delayed event leave.
//
// If that is not the case we certainly still have a valid membership on the matrix network
// independet if the sync currently works.
const certainlyConnected$ = fromEvent(
matrixRTCSession,
MembershipManagerEvent.ProbablyLeft,
).pipe(
startWith(null),
map(() => matrixRTCSession.probablyLeft !== true),
);
const connectedCombined$ = and$(
syncing$,
membershipConnected$,
certainlyConnected$,
).pipe(
tap((connected) => {
logger.info(`Homeserver connected update: ${connected}`);
}),
);
return scope.behavior(connectedCombined$);
}

View File

@@ -1,158 +1,492 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type MatrixRTCSession } from "matrix-js-sdk/lib/matrixrtc";
import { expect, test, vi } from "vitest";
import {
type LivekitTransport,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { describe, expect, it, vi } from "vitest";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import EventEmitter from "events";
import { BehaviorSubject, map, of } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import {
ConnectionState as LivekitConnectionState,
type LocalParticipant,
type LocalTrack,
} from "livekit-client";
import { MatrixRTCMode } from "../../../settings/settings";
import { mockConfig } from "../../../utils/test";
import { enterRTCSession } from "./LocalMembership";
import {
flushPromises,
mockConfig,
mockLivekitRoom,
mockMuteStates,
withTestScheduler,
} from "../../../utils/test";
import {
createLocalMembership$,
enterRTCSession,
RTCBackendState,
} from "./LocalMembership";
import { MatrixRTCTransportMissingError } from "../../../utils/errors";
import { Epoch, ObservableScope } from "../../ObservableScope";
import { constant } from "../../Behavior";
import { ConnectionManagerData } from "../remoteMembers/ConnectionManager";
import { type Connection } from "../remoteMembers/Connection";
import { type Publisher } from "./Publisher";
const MATRIX_RTC_MODE = MatrixRTCMode.Legacy;
const getUrlParams = vi.hoisted(() => vi.fn(() => ({})));
vi.mock("../../../UrlParams", () => ({ getUrlParams }));
vi.mock("../../../widget", async (importOriginal) => ({
...(await importOriginal()),
widget: {
api: {
setAlwaysOnScreen: (): void => {},
transport: { send: vi.fn(), reply: vi.fn(), stop: vi.fn() },
},
lazyActions: new EventEmitter(),
},
vi.mock("@livekit/components-core", () => ({
observeParticipantEvents: vi
.fn()
.mockReturnValue(of({ isScreenShareEnabled: false })),
}));
test("It joins the correct Session", async () => {
const focusFromOlderMembership = {
type: "livekit",
livekit_service_url: "http://my-oldest-member-service-url.com",
livekit_alias: "my-oldest-member-service-alias",
};
const focusConfigFromWellKnown = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
};
const focusConfigFromWellKnown2 = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url2.com",
};
const clientWellKnown = {
"org.matrix.msc4143.rtc_foci": [
focusConfigFromWellKnown,
focusConfigFromWellKnown2,
],
};
mockConfig({
livekit: { livekit_service_url: "http://my-default-service-url.com" },
});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockImplementation(
async (domain) => {
if (domain === "example.org") {
return Promise.resolve(clientWellKnown);
}
return Promise.resolve({});
},
);
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn().mockReturnValue(focusFromOlderMembership),
getOldestMembership: vi.fn().mockReturnValue({
getPreferredFoci: vi.fn().mockReturnValue([focusFromOlderMembership]),
}),
joinRoomSession: vi.fn(),
}) as unknown as MatrixRTCSession;
await enterRTCSession(
mockedSession,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
);
expect(mockedSession.joinRoomSession).toHaveBeenLastCalledWith(
[
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
describe("LocalMembership", () => {
describe("enterRTCSession", () => {
it("It joins the correct Session", async () => {
const focusFromOlderMembership = {
type: "livekit",
},
],
undefined,
expect.objectContaining({
manageMediaKeys: true,
useLegacyMemberEvents: false,
}),
);
});
livekit_service_url: "http://my-oldest-member-service-url.com",
livekit_alias: "my-oldest-member-service-alias",
};
test("It should not fail with configuration error if homeserver config has livekit url but not fallback", async () => {
mockConfig({});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({
"org.matrix.msc4143.rtc_foci": [
{
const focusConfigFromWellKnown = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
},
],
};
const focusConfigFromWellKnown2 = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url2.com",
};
const clientWellKnown = {
"org.matrix.msc4143.rtc_foci": [
focusConfigFromWellKnown,
focusConfigFromWellKnown2,
],
};
mockConfig({
livekit: { livekit_service_url: "http://my-default-service-url.com" },
});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockImplementation(
async (domain) => {
if (domain === "example.org") {
return Promise.resolve(clientWellKnown);
}
return Promise.resolve({});
},
);
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn().mockReturnValue(focusFromOlderMembership),
getOldestMembership: vi.fn().mockReturnValue({
getPreferredFoci: vi.fn().mockReturnValue([focusFromOlderMembership]),
}),
joinRoomSession: vi.fn(),
}) as unknown as MatrixRTCSession;
await enterRTCSession(
mockedSession,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
);
expect(mockedSession.joinRoomSession).toHaveBeenLastCalledWith(
[
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
],
undefined,
expect.objectContaining({
manageMediaKeys: true,
useLegacyMemberEvents: false,
}),
);
});
it("It should not fail with configuration error if homeserver config has livekit url but not fallback", async () => {
mockConfig({});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({
"org.matrix.msc4143.rtc_foci": [
{
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
},
],
});
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn(),
joinRoomSession: vi.fn(),
}) as unknown as MatrixRTCSession;
await enterRTCSession(
mockedSession,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
);
});
});
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn(),
joinRoomSession: vi.fn(),
}) as unknown as MatrixRTCSession;
const defaultCreateLocalMemberValues = {
options: constant({
encryptMedia: false,
matrixRTCMode: MatrixRTCMode.Matrix_2_0,
}),
matrixRTCSession: {
updateCallIntent: () => {},
leaveRoomSession: () => {},
} as unknown as MatrixRTCSession,
muteStates: mockMuteStates(),
isHomeserverConnected: constant(true),
trackProcessorState$: constant({
supported: false,
processor: undefined,
}),
logger: logger,
createPublisherFactory: vi.fn(),
joinMatrixRTC: async (): Promise<void> => {},
homeserverConnected$: constant(true),
};
await enterRTCSession(
mockedSession,
it("throws error on missing RTC config error", () => {
withTestScheduler(({ scope, hot, expectObservable }) => {
const goodTransport = {
livekit_service_url: "other",
} as LivekitTransport;
const localTransport$ = scope.behavior<LivekitTransport>(
hot("1ms #", {}, new MatrixRTCTransportMissingError("domain.com")),
goodTransport,
);
const mockConnectionManager = {
transports$: scope.behavior(
localTransport$.pipe(map((t) => new Epoch([t]))),
),
connectionManagerData$: constant(
new Epoch(new ConnectionManagerData()),
),
};
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: mockConnectionManager,
localTransport$,
});
expectObservable(localMembership.connectionState.livekit$).toBe("ne", {
n: { state: RTCBackendState.WaitingForConnection },
e: {
state: RTCBackendState.Error,
error: expect.toSatisfy(
(e) => e instanceof MatrixRTCTransportMissingError,
),
},
});
});
});
const aTransport = {
livekit_service_url: "a",
} as LivekitTransport;
const bTransport = {
livekit_service_url: "b",
} as LivekitTransport;
const connectionManagerData = new ConnectionManagerData();
connectionManagerData.add(
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
livekitRoom: mockLivekitRoom({
localParticipant: {
isScreenShareEnabled: false,
trackPublications: [],
} as unknown as LocalParticipant,
}),
state$: constant({
state: "ConnectedToLkRoom",
livekitConnectionState$: constant(LivekitConnectionState.Connected),
}),
transport: aTransport,
} as unknown as Connection,
[],
);
connectionManagerData.add(
{
state$: constant({
state: "ConnectedToLkRoom",
}),
transport: bTransport,
} as unknown as Connection,
[],
);
it("recreates publisher if new connection is used and ENDS always unpublish and end tracks", async () => {
const scope = new ObservableScope();
const localTransport$ = new BehaviorSubject(aTransport);
const publishers: Publisher[] = [];
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const p = {
stopPublishing: vi.fn(),
stopTracks: vi.fn(),
publishing$: constant(false),
};
publishers.push(p as unknown as Publisher);
return p;
},
);
const publisherFactory =
defaultCreateLocalMemberValues.createPublisherFactory as ReturnType<
typeof vi.fn
>;
createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport$,
});
await flushPromises();
localTransport$.next(bTransport);
await flushPromises();
expect(publisherFactory).toHaveBeenCalledTimes(2);
expect(publishers.length).toBe(2);
// stop the first Publisher and let the second one life.
expect(publishers[0].stopTracks).toHaveBeenCalled();
expect(publishers[1].stopTracks).not.toHaveBeenCalled();
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[1].stopPublishing).not.toHaveBeenCalled();
expect(publisherFactory.mock.calls[0][0].transport).toBe(aTransport);
expect(publisherFactory.mock.calls[1][0].transport).toBe(bTransport);
scope.end();
await flushPromises();
// stop all tracks after ending scopes
expect(publishers[1].stopPublishing).toHaveBeenCalled();
expect(publishers[1].stopTracks).toHaveBeenCalled();
defaultCreateLocalMemberValues.createPublisherFactory.mockReset();
});
it("only start tracks if requested", async () => {
const scope = new ObservableScope();
const localTransport$ = new BehaviorSubject(aTransport);
const publishers: Publisher[] = [];
const tracks$ = new BehaviorSubject<LocalTrack[]>([]);
const publishing$ = new BehaviorSubject<boolean>(false);
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const p = {
stopPublishing: vi.fn(),
stopTracks: vi.fn(),
createAndSetupTracks: vi.fn().mockImplementation(async () => {
tracks$.next([{}, {}] as LocalTrack[]);
return Promise.resolve();
}),
tracks$,
publishing$,
};
publishers.push(p as unknown as Publisher);
return p;
},
);
const publisherFactory =
defaultCreateLocalMemberValues.createPublisherFactory as ReturnType<
typeof vi.fn
>;
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport$,
});
await flushPromises();
expect(publisherFactory).toHaveBeenCalledOnce();
expect(localMembership.tracks$.value.length).toBe(0);
localMembership.startTracks();
await flushPromises();
expect(localMembership.tracks$.value.length).toBe(2);
scope.end();
await flushPromises();
// stop all tracks after ending scopes
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[0].stopTracks).toHaveBeenCalled();
publisherFactory.mockClear();
});
// TODO add an integration test combining publisher and localMembership
//
it("tracks livekit state correctly", async () => {
const scope = new ObservableScope();
const localTransport$ = new BehaviorSubject<null | LivekitTransport>(null);
const connectionManagerData$ = new BehaviorSubject<
Epoch<ConnectionManagerData>
>(new Epoch(new ConnectionManagerData()));
const publishers: Publisher[] = [];
const tracks$ = new BehaviorSubject<LocalTrack[]>([]);
const publishing$ = new BehaviorSubject<boolean>(false);
const createTrackResolver = Promise.withResolvers<void>();
const publishResolver = Promise.withResolvers<void>();
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const p = {
stopPublishing: vi.fn(),
stopTracks: vi.fn().mockImplementation(() => {
logger.info("stopTracks");
tracks$.next([]);
}),
createAndSetupTracks: vi.fn().mockImplementation(async () => {
await createTrackResolver.promise;
tracks$.next([{}, {}] as LocalTrack[]);
}),
startPublishing: vi.fn().mockImplementation(async () => {
await publishResolver.promise;
publishing$.next(true);
}),
tracks$,
publishing$,
};
publishers.push(p as unknown as Publisher);
return p;
},
);
const publisherFactory =
defaultCreateLocalMemberValues.createPublisherFactory as ReturnType<
typeof vi.fn
>;
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: {
connectionManagerData$,
},
localTransport$,
});
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.WaitingForTransport,
});
localTransport$.next(aTransport);
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.WaitingForConnection,
});
connectionManagerData$.next(new Epoch(connectionManagerData));
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.Initialized,
});
expect(publisherFactory).toHaveBeenCalledOnce();
expect(localMembership.tracks$.value.length).toBe(0);
// -------
localMembership.startTracks();
// -------
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.CreatingTracks,
});
createTrackResolver.resolve();
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.ReadyToPublish,
});
// -------
localMembership.requestConnect();
// -------
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.WaitingToPublish,
});
publishResolver.resolve();
await flushPromises();
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.Connected,
});
expect(publishers[0].stopPublishing).not.toHaveBeenCalled();
expect(localMembership.connectionState.livekit$.isStopped).toBe(false);
scope.end();
await flushPromises();
// stays in connected state because it is stopped before the update to tracks update the state.
expect(localMembership.connectionState.livekit$.value).toStrictEqual({
state: RTCBackendState.Connected,
});
// stop all tracks after ending scopes
expect(publishers[0].stopPublishing).toHaveBeenCalled();
expect(publishers[0].stopTracks).toHaveBeenCalled();
});
// TODO add tests for matrix local matrix participation.
});

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -10,20 +10,20 @@ import {
type Participant,
ParticipantEvent,
type LocalParticipant,
type ScreenShareCaptureOptions,
ConnectionState,
} from "livekit-client";
import { observeParticipantEvents } from "@livekit/components-core";
import {
type LivekitTransport,
type MatrixRTCSession,
MembershipManagerEvent,
Status,
} from "matrix-js-sdk/lib/matrixrtc";
import { ClientEvent, SyncState, type Room as MatrixRoom } from "matrix-js-sdk";
import {
BehaviorSubject,
catchError,
combineLatest,
distinctUntilChanged,
fromEvent,
from,
map,
type Observable,
of,
@@ -33,61 +33,73 @@ import {
tap,
} from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { deepCompare } from "matrix-js-sdk/lib/utils";
import { type Behavior } from "../../Behavior";
import { constant, type Behavior } from "../../Behavior";
import { type IConnectionManager } from "../remoteMembers/ConnectionManager";
import { ObservableScope } from "../../ObservableScope";
import { Publisher } from "./Publisher";
import { type Publisher } from "./Publisher";
import { type MuteStates } from "../../MuteStates";
import { type ProcessorState } from "../../../livekit/TrackProcessorContext";
import { type MediaDevices } from "../../MediaDevices";
import { and$ } from "../../../utils/observable";
import { ElementCallError, UnknownCallError } from "../../../utils/errors";
import {
ElementWidgetActions,
widget,
type WidgetHelpers,
} from "../../../widget";
import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers";
ElementCallError,
MembershipManagerError,
UnknownCallError,
} from "../../../utils/errors";
import { ElementWidgetActions, widget } from "../../../widget";
import { getUrlParams } from "../../../UrlParams.ts";
import { PosthogAnalytics } from "../../../analytics/PosthogAnalytics.ts";
import { MatrixRTCMode } from "../../../settings/settings.ts";
import { Config } from "../../../config/Config.ts";
import {
type Connection,
type ConnectionState,
} from "../remoteMembers/Connection.ts";
import { type Connection } from "../remoteMembers/Connection.ts";
export enum LivekitState {
Uninitialized = "uninitialized",
Connecting = "connecting",
Connected = "connected",
export enum RTCBackendState {
Error = "error",
/** Not even a transport is available to the LocalMembership */
WaitingForTransport = "waiting_for_transport",
/** A connection appeared so we can initialise the publisher */
WaitingForConnection = "waiting_for_connection",
/** Connection and transport arrived, publisher Initialized */
Initialized = "Initialized",
CreatingTracks = "creating_tracks",
ReadyToPublish = "ready_to_publish",
WaitingToPublish = "waiting_to_publish",
Connected = "connected",
Disconnected = "disconnected",
Disconnecting = "disconnecting",
}
type LocalMemberLivekitState =
| { state: LivekitState.Error; error: string }
| { state: LivekitState.Connected }
| { state: LivekitState.Connecting }
| { state: LivekitState.Uninitialized }
| { state: LivekitState.Disconnected }
| { state: LivekitState.Disconnecting };
type LocalMemberRtcBackendState =
| { state: RTCBackendState.Error; error: ElementCallError }
| { state: RTCBackendState.WaitingForTransport }
| { state: RTCBackendState.WaitingForConnection }
| { state: RTCBackendState.Initialized }
| { state: RTCBackendState.CreatingTracks }
| { state: RTCBackendState.ReadyToPublish }
| { state: RTCBackendState.WaitingToPublish }
| { state: RTCBackendState.Connected }
| { state: RTCBackendState.Disconnected }
| { state: RTCBackendState.Disconnecting };
export enum MatrixState {
WaitingForTransport = "waiting_for_transport",
Ready = "ready",
Connecting = "connecting",
Connected = "connected",
Disconnected = "disconnected",
Connecting = "connecting",
Error = "Error",
}
type LocalMemberMatrixState =
| { state: MatrixState.Connected }
| { state: MatrixState.WaitingForTransport }
| { state: MatrixState.Ready }
| { state: MatrixState.Connecting }
| { state: MatrixState.Disconnected };
| { state: MatrixState.Disconnected }
| { state: MatrixState.Error; error: Error };
export interface LocalMemberConnectionState {
livekit$: Behavior<LocalMemberLivekitState>;
livekit$: Behavior<LocalMemberRtcBackendState>;
matrix$: Behavior<LocalMemberMatrixState>;
}
@@ -102,17 +114,21 @@ export interface LocalMemberConnectionState {
* - Publisher.publishTracks()
* - send join state/sticky event
*/
interface Props {
options: Behavior<EnterRTCSessionOptions>;
// TODO add a comment into some code style readme or file header callviewmodel
// that the inputs for those createSomething$() functions should NOT contain any js-sdk objectes
scope: ObservableScope;
mediaDevices: MediaDevices;
muteStates: MuteStates;
connectionManager: IConnectionManager;
matrixRTCSession: MatrixRTCSession;
matrixRoom: MatrixRoom;
createPublisherFactory: (connection: Connection) => Publisher;
joinMatrixRTC: (transport: LivekitTransport) => Promise<void>;
homeserverConnected$: Behavior<boolean>;
localTransport$: Behavior<LivekitTransport | null>;
trackProcessorState$: Behavior<ProcessorState>;
widget: WidgetHelpers | null;
matrixRTCSession: Pick<
MatrixRTCSession,
"updateCallIntent" | "leaveRoomSession"
>;
logger: Logger;
}
@@ -131,75 +147,74 @@ interface Props {
*/
export const createLocalMembership$ = ({
scope,
options,
muteStates,
mediaDevices,
connectionManager,
matrixRTCSession,
localTransport$,
matrixRoom,
trackProcessorState$,
widget,
localTransport$: localTransportCanThrow$,
homeserverConnected$,
createPublisherFactory,
joinMatrixRTC,
logger: parentLogger,
muteStates,
matrixRTCSession,
}: Props): {
// publisher: Publisher
requestConnect: () => LocalMemberConnectionState;
/**
* This starts audio and video tracks. They will be reused when calling `requestConnect`.
*/
startTracks: () => Behavior<LocalTrack[]>;
requestDisconnect: () => Observable<LocalMemberLivekitState> | null;
/**
* This sets a inner state (shouldConnect) to true and instructs the js-sdk and livekit to keep the user
* connected to matrix and livekit.
*/
requestConnect: () => void;
requestDisconnect: () => void;
connectionState: LocalMemberConnectionState;
sharingScreen$: Behavior<boolean>;
/**
* Callback to toggle screen sharing. If null, screen sharing is not possible.
*/
toggleScreenSharing: (() => void) | null;
tracks$: Behavior<LocalTrack[]>;
participant$: Behavior<LocalParticipant | null>;
connection$: Behavior<Connection | null>;
// deprecated fields
/** @deprecated use state instead*/
homeserverConnected$: Behavior<boolean>;
/** @deprecated use state instead*/
connected$: Behavior<boolean>;
// this needs to be discussed
/** @deprecated use state instead*/
reconnecting$: Behavior<boolean>;
// also needs to be disccues
/** @deprecated use state instead*/
configError$: Behavior<ElementCallError | null>;
} => {
const logger = parentLogger.getChild("[LocalMembership]");
logger.debug(`Creating local membership..`);
const state = {
livekit$: new BehaviorSubject<LocalMemberLivekitState>({
state: LivekitState.Uninitialized,
}),
matrix$: new BehaviorSubject<LocalMemberMatrixState>({
state: MatrixState.Disconnected,
}),
};
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const trackStartRequested$ = new BehaviorSubject(false);
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const connectRequested$ = new BehaviorSubject(false);
// This should be used in a combineLatest with publisher$ to connect.
const tracks$ = new BehaviorSubject<LocalTrack[]>([]);
// Unwrap the local transport and set the state of the LocalMembership to error in case the transport is an error.
const localTransport$ = scope.behavior(
localTransportCanThrow$.pipe(
catchError((e: unknown) => {
let error: ElementCallError;
if (e instanceof ElementCallError) {
error = e;
} else {
error = new UnknownCallError(
e instanceof Error
? e
: new Error("Unknown error from localTransport"),
);
}
setLivekitError(error);
return of(null);
}),
),
);
// Drop Epoch data here since we will not combine this anymore
const localConnection$ = scope.behavior(
combineLatest([connectionManager.connections$, localTransport$]).pipe(
map(([connections, localTransport]) => {
combineLatest([
connectionManager.connectionManagerData$,
localTransport$,
]).pipe(
map(([{ value: connectionData }, localTransport]) => {
if (localTransport === null) {
return null;
}
return (
connections.value.find((connection) =>
areLivekitTransportsEqual(connection.transport, localTransport),
) ?? null
);
return connectionData.getConnectionForTransport(localTransport);
}),
tap((connection) => {
logger.info(
@@ -208,88 +223,37 @@ export const createLocalMembership$ = ({
}),
),
);
/**
* Whether we are connected to the MatrixRTC session.
*/
const homeserverConnected$ = scope.behavior(
// To consider ourselves connected to MatrixRTC, we check the following:
and$(
// The client is connected to the sync loop
(
fromEvent(matrixRoom.client, ClientEvent.Sync) as Observable<
[SyncState]
>
).pipe(
startWith([matrixRoom.client.getSyncState()]),
map(([state]) => state === SyncState.Syncing),
),
// Room state observed by session says we're connected
fromEvent(matrixRTCSession, MembershipManagerEvent.StatusChanged).pipe(
startWith(null),
map(() => matrixRTCSession.membershipStatus === Status.Connected),
),
// Also watch out for warnings that we've likely hit a timeout and our
// delayed leave event is being sent (this condition is here because it
// provides an earlier warning than the sync loop timeout, and we wouldn't
// see the actual leave event until we reconnect to the sync loop)
fromEvent(matrixRTCSession, MembershipManagerEvent.ProbablyLeft).pipe(
startWith(null),
map(() => matrixRTCSession.probablyLeft !== true),
),
).pipe(
tap((connected) => {
logger.info(`Homeserver connected update: ${connected}`);
}),
),
const localConnectionState$ = localConnection$.pipe(
switchMap((connection) => (connection ? connection.state$ : of(null))),
);
// /**
// * Whether we are "fully" connected to the call. Accounts for both the
// * connection to the MatrixRTC session and the LiveKit publish connection.
// */
// // TODO use this in combination with the MemberState.
const connected$ = scope.behavior(
and$(
homeserverConnected$,
localConnection$.pipe(
switchMap((c) =>
c
? c.state$.pipe(map((state) => state.state === "ConnectedToLkRoom"))
: of(false),
),
homeserverConnected$.pipe(
tap((v) => logger.debug("matrix: Connected state changed", v)),
),
),
);
const publisher$ = new BehaviorSubject<Publisher | null>(null);
localConnection$.subscribe((connection) => {
if (connection !== null && publisher$.value === null) {
// TODO looks strange to not change publisher if connection changes.
publisher$.next(
new Publisher(
scope,
connection,
mediaDevices,
muteStates,
trackProcessorState$,
),
);
}
});
combineLatest([publisher$, trackStartRequested$]).subscribe(
([publisher, shouldStartTracks]) => {
if (publisher && shouldStartTracks) {
publisher
.createAndSetupTracks()
.then((tracks) => {
tracks$.next(tracks);
})
.catch((error) => {
logger.error("Error creating tracks:", error);
});
}
},
localConnectionState$.pipe(
switchMap((state) => {
logger.debug("livekit: Connected state changed", state);
if (!state) return of(false);
if (state.state === "ConnectedToLkRoom") {
logger.debug(
"livekit: Connected state changed (inner livekitConnectionState$)",
state.livekitConnectionState$.value,
);
return state.livekitConnectionState$.pipe(
map((lkState) => lkState === ConnectionState.Connected),
);
}
return of(false);
}),
),
).pipe(tap((v) => logger.debug("combined: Connected state changed", v))),
);
// MATRIX RELATED
@@ -314,83 +278,217 @@ export const createLocalMembership$ = ({
),
);
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const trackStartRequested = Promise.withResolvers<void>();
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const connectRequested$ = new BehaviorSubject(false);
/**
* The publisher is stored in here an abstracts creating and publishing tracks.
*/
const publisher$ = new BehaviorSubject<Publisher | null>(null);
/**
* Extract the tracks from the published. Also reacts to changing publishers.
*/
const tracks$ = scope.behavior(
publisher$.pipe(switchMap((p) => (p?.tracks$ ? p.tracks$ : constant([])))),
);
const publishing$ = scope.behavior(
publisher$.pipe(switchMap((p) => p?.publishing$ ?? constant(false))),
);
const startTracks = (): Behavior<LocalTrack[]> => {
trackStartRequested$.next(true);
trackStartRequested.resolve();
return tracks$;
};
combineLatest([publisher$, tracks$]).subscribe(([publisher, tracks]) => {
if (
tracks.length === 0 ||
// change this to !== Publishing
state.livekit$.value.state !== LivekitState.Uninitialized
) {
return;
const requestConnect = (): void => {
trackStartRequested.resolve();
connectRequested$.next(true);
};
const requestDisconnect = (): void => {
connectRequested$.next(false);
};
// Take care of the publisher$
// create a new one as soon as a local Connection is available
//
// Recreate a new one once the local connection changes
// - stop publishing
// - destruct all current streams
// - overwrite current publisher
scope.reconcile(localConnection$, async (connection) => {
if (connection !== null) {
publisher$.next(createPublisherFactory(connection));
}
state.livekit$.next({ state: LivekitState.Connecting });
publisher
?.startPublishing()
.then(() => {
state.livekit$.next({ state: LivekitState.Connected });
})
.catch((error) => {
state.livekit$.next({ state: LivekitState.Error, error });
});
return Promise.resolve(async (): Promise<void> => {
await publisher$?.value?.stopPublishing();
publisher$?.value?.stopTracks();
});
});
combineLatest([localTransport$, connectRequested$]).subscribe(
// TODO reconnect on options change.
([transport, connectRequested]) => {
if (
transport === null ||
!connectRequested ||
state.matrix$.value.state !== MatrixState.Disconnected
) {
logger.info(
"Not yet connecting because: ",
"transport === null:",
transport === null,
"!connectRequested:",
!connectRequested,
"state.matrix$.value.state !== MatrixState.Disconnected:",
state.matrix$.value.state !== MatrixState.Disconnected,
);
return;
// Use reconcile here to not run concurrent createAndSetupTracks calls
// `tracks$` will update once they are ready.
scope.reconcile(
scope.behavior(
combineLatest([publisher$, tracks$, from(trackStartRequested.promise)]),
null,
),
async (valueIfReady) => {
if (!valueIfReady) return;
const [publisher, tracks] = valueIfReady;
if (publisher && tracks.length === 0) {
await publisher.createAndSetupTracks().catch((e) => logger.error(e));
}
state.matrix$.next({ state: MatrixState.Connecting });
logger.info("Matrix State connecting");
enterRTCSession(matrixRTCSession, transport, options.value).catch(
(error) => {
logger.error(error);
},
);
},
);
const requestConnect = (): LocalMemberConnectionState => {
trackStartRequested$.next(true);
connectRequested$.next(true);
// Based on `connectRequested$` we start publishing tracks. (once they are there!)
scope.reconcile(
scope.behavior(combineLatest([publisher$, tracks$, connectRequested$])),
async ([publisher, tracks, shouldConnect]) => {
if (shouldConnect === publisher?.publishing$.value) return;
if (tracks.length !== 0 && shouldConnect) {
try {
await publisher?.startPublishing();
} catch (error) {
setLivekitError(error as ElementCallError);
}
} else if (tracks.length !== 0 && !shouldConnect) {
try {
await publisher?.stopPublishing();
} catch (error) {
setLivekitError(new UnknownCallError(error as Error));
}
}
},
);
return state;
const fatalLivekitError$ = new BehaviorSubject<ElementCallError | null>(null);
const setLivekitError = (e: ElementCallError): void => {
if (fatalLivekitError$.value !== null)
logger.error("Multiple Livkit Errors:", e);
else fatalLivekitError$.next(e);
};
const livekitState$: Behavior<LocalMemberRtcBackendState> = scope.behavior(
combineLatest([
publisher$,
localTransport$,
tracks$.pipe(
tap((t) => {
logger.info("tracks$: ", t);
}),
),
publishing$,
connectRequested$,
from(trackStartRequested.promise).pipe(
map(() => true),
startWith(false),
),
fatalLivekitError$,
]).pipe(
map(
([
publisher,
localTransport,
tracks,
publishing,
shouldConnect,
shouldStartTracks,
error,
]) => {
// read this:
// if(!<A>) return {state: ...}
// if(!<B>) return {state: <MyState>}
//
// as:
// We do have <A> but not yet <B> so we are in <MyState>
if (error !== null) return { state: RTCBackendState.Error, error };
const hasTracks = tracks.length > 0;
if (!localTransport)
return { state: RTCBackendState.WaitingForTransport };
if (!publisher)
return { state: RTCBackendState.WaitingForConnection };
if (!shouldStartTracks) return { state: RTCBackendState.Initialized };
if (!hasTracks) return { state: RTCBackendState.CreatingTracks };
if (!shouldConnect) return { state: RTCBackendState.ReadyToPublish };
if (!publishing) return { state: RTCBackendState.WaitingToPublish };
return { state: RTCBackendState.Connected };
},
),
distinctUntilChanged(deepCompare),
),
);
const requestDisconnect = (): Behavior<LocalMemberLivekitState> | null => {
if (state.livekit$.value.state !== LivekitState.Connected) return null;
state.livekit$.next({ state: LivekitState.Disconnecting });
combineLatest([publisher$, tracks$], (publisher, tracks) => {
publisher
?.stopPublishing()
.then(() => {
tracks.forEach((track) => track.stop());
state.livekit$.next({ state: LivekitState.Disconnected });
})
.catch((error) => {
state.livekit$.next({ state: LivekitState.Error, error });
});
});
return state.livekit$;
const fatalMatrixError$ = new BehaviorSubject<ElementCallError | null>(null);
const setMatrixError = (e: ElementCallError): void => {
if (fatalMatrixError$.value !== null)
logger.error("Multiple Matrix Errors:", e);
else fatalMatrixError$.next(e);
};
const matrixState$: Behavior<LocalMemberMatrixState> = scope.behavior(
combineLatest([
localTransport$,
connectRequested$,
homeserverConnected$,
]).pipe(
map(([localTransport, connectRequested, homeserverConnected]) => {
if (!localTransport) return { state: MatrixState.WaitingForTransport };
if (!connectRequested) return { state: MatrixState.Ready };
if (!homeserverConnected) return { state: MatrixState.Connecting };
return { state: MatrixState.Connected };
}),
),
);
// Keep matrix rtc session in sync with localTransport$, connectRequested$ and muteStates.video.enabled$
scope.reconcile(
scope.behavior(combineLatest([localTransport$, connectRequested$])),
async ([transport, shouldConnect]) => {
if (!shouldConnect) return;
if (!transport) return;
try {
await joinMatrixRTC(transport);
} catch (error) {
logger.error("Error entering RTC session", error);
if (error instanceof Error)
setMatrixError(new MembershipManagerError(error));
}
// Update our member event when our mute state changes.
const callIntentScope = new ObservableScope();
// because this uses its own scope, we can start another reconciliation for the duration of one connection.
callIntentScope.reconcile(
muteStates.video.enabled$,
async (videoEnabled) =>
matrixRTCSession.updateCallIntent(videoEnabled ? "video" : "audio"),
);
return async (): Promise<void> => {
callIntentScope.end();
try {
// Update matrixRTCSession to allow udpating the transport without leaving the session!
await matrixRTCSession.leaveRoomSession();
} catch (e) {
logger.error("Error leaving RTC session", e);
}
try {
await widget?.api.transport.send(ElementWidgetActions.HangupCall, {});
} catch (e) {
logger.error("Failed to send hangup action", e);
}
};
},
);
const participant$ = scope.behavior(
localConnection$.pipe(map((c) => c?.livekitRoom?.localParticipant ?? null)),
);
// Pause upstream of all local media tracks when we're disconnected from
// MatrixRTC, because it can be an unpleasant surprise for the app to say
@@ -398,12 +496,12 @@ export const createLocalMembership$ = ({
// We use matrixConnected$ rather than reconnecting$ because we want to
// pause tracks during the initial joining sequence too until we're sure
// that our own media is displayed on screen.
combineLatest([localConnection$, homeserverConnected$])
// TODO refactor this based no livekitState$
combineLatest([participant$, homeserverConnected$])
.pipe(scope.bind())
.subscribe(([connection, connected]) => {
if (connection?.state$.value.state !== "ConnectedToLkRoom") return;
const publications =
connection.livekitRoom.localParticipant.trackPublications.values();
.subscribe(([participant, connected]) => {
if (!participant) return;
const publications = participant.trackPublications.values();
if (connected) {
for (const p of publications) {
if (p.track?.isUpstreamPaused === true) {
@@ -441,118 +539,61 @@ export const createLocalMembership$ = ({
}
});
const configError$ = new BehaviorSubject<ElementCallError | null>(null);
// TODO I do not fully understand what this does.
// Is it needed?
// Is this at the right place?
// Can this be simplified?
// Start and stop session membership as needed
scope.reconcile(localTransport$, async (advertised) => {
if (advertised !== null && advertised !== undefined) {
try {
await enterRTCSession(matrixRTCSession, advertised, options.value);
configError$.next(null);
} catch (e) {
logger.error("Error entering RTC session", e);
}
// Update our member event when our mute state changes.
const intentScope = new ObservableScope();
intentScope.reconcile(muteStates.video.enabled$, async (videoEnabled) =>
matrixRTCSession.updateCallIntent(videoEnabled ? "video" : "audio"),
);
return async (): Promise<void> => {
intentScope.end();
// Only sends Matrix leave event. The LiveKit session will disconnect
// as soon as either the stopConnection$ handler above gets to it or
// the view model is destroyed.
try {
await matrixRTCSession.leaveRoomSession();
} catch (e) {
logger.error("Error leaving RTC session", e);
}
try {
await widget?.api.transport.send(ElementWidgetActions.HangupCall, {});
} catch (e) {
logger.error("Failed to send hangup action", e);
}
};
}
});
localConnection$
.pipe(
distinctUntilChanged(),
switchMap((c) =>
c === null ? of({ state: "Initialized" } as ConnectionState) : c.state$,
),
map((s) => {
logger.trace(`Local connection state update: ${s.state}`);
if (s.state == "FailedToStart") {
return s.error instanceof ElementCallError
? s.error
: new UnknownCallError(s.error);
} else {
return null;
}
}),
scope.bind(),
)
.subscribe((fatalError) => {
configError$.next(fatalError);
});
/**
* Whether the user is currently sharing their screen.
*/
const sharingScreen$ = scope.behavior(
localConnection$.pipe(
switchMap((c) =>
c === null
? of(false)
: observeSharingScreen$(c.livekitRoom.localParticipant),
),
participant$.pipe(
switchMap((p) => (p !== null ? observeSharingScreen$(p) : of(false))),
),
);
const toggleScreenSharing =
let toggleScreenSharing: (() => void) | null = null;
if (
"getDisplayMedia" in (navigator.mediaDevices ?? {}) &&
!getUrlParams().hideScreensharing
? (): void =>
// If a connection is ready, toggle screen sharing.
// We deliberately do nothing in the case of a null connection because
// it looks nice for the call control buttons to all become available
// at once upon joining the call, rather than introducing a disabled
// state. The user can just click again.
// We also allow screen sharing to be toggled even if the connection
// is still initializing or publishing tracks, because there's no
// technical reason to disallow this. LiveKit will publish if it can.
void localConnection$.value?.livekitRoom.localParticipant
.setScreenShareEnabled(!sharingScreen$.value, {
audio: true,
selfBrowserSurface: "include",
surfaceSwitching: "include",
systemAudio: "include",
})
.catch(logger.error)
: null;
) {
toggleScreenSharing = (): void => {
const screenshareSettings: ScreenShareCaptureOptions = {
audio: true,
selfBrowserSurface: "include",
surfaceSwitching: "include",
systemAudio: "include",
};
const targetScreenshareState = !sharingScreen$.value;
logger.info(
`toggleScreenSharing called. Switching ${
targetScreenshareState ? "On" : "Off"
}`,
);
// If a connection is ready, toggle screen sharing.
// We deliberately do nothing in the case of a null connection because
// it looks nice for the call control buttons to all become available
// at once upon joining the call, rather than introducing a disabled
// state. The user can just click again.
// We also allow screen sharing to be toggled even if the connection
// is still initializing or publishing tracks, because there's no
// technical reason to disallow this. LiveKit will publish if it can.
participant$.value
?.setScreenShareEnabled(targetScreenshareState, screenshareSettings)
.catch(logger.error);
};
}
const participant$ = scope.behavior(
localConnection$.pipe(map((c) => c?.livekitRoom.localParticipant ?? null)),
);
return {
startTracks,
requestConnect,
requestDisconnect,
connectionState: state,
connectionState: {
livekit$: livekitState$,
matrix$: matrixState$,
},
tracks$,
participant$,
homeserverConnected$,
connected$,
reconnecting$,
configError$,
sharingScreen$,
toggleScreenSharing,
participant$,
connection$: localConnection$,
};
};
@@ -573,7 +614,12 @@ interface EnterRTCSessionOptions {
}
/**
* TODO! document this function properly
* Does the necessary steps to enter the RTC session on the matrix side:
* - Preparing the membership info (FOCUS to use, options)
* - Sends the matrix event to join the call, and starts the membership manager:
* - Delay events management
* - Handles retries (fails only after several attempts)
*
* @param rtcSession
* @param transport
* @param options
@@ -598,6 +644,7 @@ export async function enterRTCSession(
const { sendNotificationType: notificationType, callIntent } = getUrlParams();
const multiSFU = matrixRTCMode !== MatrixRTCMode.Legacy;
// Multi-sfu does not need a preferred foci list. just the focus that is actually used.
// TODO where/how do we track errors originating from the ongoing rtcSession?
rtcSession.joinRoomSession(
multiSFU ? [] : [transport],
multiSFU ? transport : undefined,

View File

@@ -0,0 +1,156 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject } from "rxjs";
import { mockConfig, flushPromises } from "../../../utils/test";
import { createLocalTransport$ } from "./LocalTransport";
import { constant } from "../../Behavior";
import { Epoch, ObservableScope } from "../../ObservableScope";
import {
MatrixRTCTransportMissingError,
FailToGetOpenIdToken,
} from "../../../utils/errors";
import * as openIDSFU from "../../../livekit/openIDSFU";
describe("LocalTransport", () => {
let scope: ObservableScope;
beforeEach(() => (scope = new ObservableScope()));
afterEach(() => scope.end());
it("throws if config is missing", async () => {
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "",
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
await flushPromises();
expect(() => localTransport$.value).toThrow(
new MatrixRTCTransportMissingError(""),
);
});
it("throws FailToGetOpenIdToken when OpenID fetch fails", async () => {
// Provide a valid config so makeTransportInternal resolves a transport
const scope = new ObservableScope();
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const resolver = Promise.withResolvers<void>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockImplementation(
async () => {
await resolver.promise;
throw new FailToGetOpenIdToken(new Error("no openid"));
},
);
const observations: unknown[] = [];
const errors: Error[] = [];
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
// Use empty domain to skip .well-known and use config directly
getDomain: () => "",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
localTransport$.subscribe(
(o) => observations.push(o),
(e) => errors.push(e),
);
resolver.resolve();
await flushPromises();
const expectedError = new FailToGetOpenIdToken(new Error("no openid"));
expect(observations).toStrictEqual([null]);
expect(errors).toStrictEqual([expectedError]);
expect(() => localTransport$.value).toThrow(expectedError);
});
it("emits preferred transport after OpenID resolves", async () => {
// Use config so transport discovery succeeds, but delay OpenID JWT fetch
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const openIdResolver = Promise.withResolvers<openIDSFU.SFUConfig>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockReturnValue(
openIdResolver.promise,
);
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
openIdResolver.resolve?.({ url: "https://lk.example.org", jwt: "jwt" });
expect(localTransport$.value).toBe(null);
await flushPromises();
// final
expect(localTransport$.value).toStrictEqual({
livekit_alias: "!room:example.org",
livekit_service_url: "https://lk.example.org",
type: "livekit",
});
});
it("updates local transport when oldest member changes", async () => {
// Use config so transport discovery succeeds, but delay OpenID JWT fetch
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const memberships$ = new BehaviorSubject(new Epoch([]));
const openIdResolver = Promise.withResolvers<openIDSFU.SFUConfig>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockReturnValue(
openIdResolver.promise,
);
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(true),
memberships$,
client: {
getDomain: () => "",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
openIdResolver.resolve?.({ url: "https://lk.example.org", jwt: "jwt" });
expect(localTransport$.value).toBe(null);
await flushPromises();
// final
expect(localTransport$.value).toStrictEqual({
livekit_alias: "!room:example.org",
livekit_service_url: "https://lk.example.org",
type: "livekit",
});
});
});

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -13,32 +13,39 @@ import {
isLivekitTransportConfig,
} from "matrix-js-sdk/lib/matrixrtc";
import { type MatrixClient } from "matrix-js-sdk";
import { combineLatest, distinctUntilChanged, first, from, map } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import {
combineLatest,
distinctUntilChanged,
first,
from,
map,
switchMap,
} from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { type Behavior } from "../../Behavior.ts";
import { type Epoch, type ObservableScope } from "../../ObservableScope.ts";
import { Config } from "../../../config/Config.ts";
import { MatrixRTCTransportMissingError } from "../../../utils/errors.ts";
import { getSFUConfigWithOpenID } from "../../../livekit/openIDSFU.ts";
import {
getSFUConfigWithOpenID,
type OpenIDClientParts,
} from "../../../livekit/openIDSFU.ts";
import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers.ts";
import { customLivekitUrl } from "../../../settings/settings.ts";
const logger = rootLogger.getChild("[LocalTransport]");
/*
* - get well known
* - get oldest membership
* - get transport to use
* - get openId + jwt token
* - wait for createTrack() call
* - create tracks
* - wait for join() call
* - Publisher.publishTracks()
* - send join state/sticky event
* It figures out “which LiveKit focus URL/alias the local user should use,”
* optionally aligning with the oldest member, and ensures the SFU path is primed
* before advertising that choice.
*/
interface Props {
scope: ObservableScope;
memberships$: Behavior<Epoch<CallMembership[]>>;
client: MatrixClient;
client: Pick<MatrixClient, "getDomain"> & OpenIDClientParts;
roomId: string;
useOldestMember$: Behavior<boolean>;
}
@@ -49,6 +56,8 @@ interface Props {
*
* @prop useOldestMember Whether to use the same transport as the oldest member.
* This will only update once the first oldest member appears. Will not recompute if the oldest member leaves.
*
* @throws MatrixRTCTransportMissingError | FailToGetOpenIdToken
*/
export const createLocalTransport$ = ({
scope,
@@ -75,16 +84,20 @@ export const createLocalTransport$ = ({
/**
* The transport that we would personally prefer to publish on (if not for the
* transport preferences of others, perhaps).
*
* @throws
*/
const preferredTransport$: Behavior<LivekitTransport | null> = scope.behavior(
from(makeTransport(client, roomId)),
customLivekitUrl.value$.pipe(
switchMap((customUrl) => from(makeTransport(client, roomId, customUrl))),
),
null,
);
/**
* The transport we should advertise in our MatrixRTC membership.
* The chosen transport we should advertise in our MatrixRTC membership.
*/
const advertisedTransport$ = scope.behavior(
return scope.behavior(
combineLatest([
useOldestMember$,
oldestMemberTransport$,
@@ -98,82 +111,78 @@ export const createLocalTransport$ = ({
distinctUntilChanged(areLivekitTransportsEqual),
),
);
return advertisedTransport$;
};
const FOCI_WK_KEY = "org.matrix.msc4143.rtc_foci";
async function makeTransportInternal(
client: MatrixClient,
/**
*
* @param client
* @param roomId
* @returns
* @throws MatrixRTCTransportMissingError | FailToGetOpenIdToken
*/
async function makeTransport(
client: Pick<MatrixClient, "getDomain"> & OpenIDClientParts,
roomId: string,
urlFromDevSettings: string | null,
): Promise<LivekitTransport> {
logger.log("Searching for a preferred transport");
let transport: LivekitTransport | undefined;
logger.trace("Searching for a preferred transport");
//TODO refactor this to use the jwt service returned alias.
const livekitAlias = roomId;
// TODO-MULTI-SFU: Either remove this dev tool or make it more official
const urlFromStorage =
localStorage.getItem("robin-matrixrtc-auth") ??
localStorage.getItem("timo-focus-url");
if (urlFromStorage !== null) {
// DEVTOOL: Highest priority: Load from devtool setting
if (urlFromDevSettings !== null) {
const transportFromStorage: LivekitTransport = {
type: "livekit",
livekit_service_url: urlFromStorage,
livekit_service_url: urlFromDevSettings,
livekit_alias: livekitAlias,
};
logger.log(
"Using LiveKit transport from local storage: ",
logger.info(
"Using LiveKit transport from dev tools: ",
transportFromStorage,
);
return transportFromStorage;
transport = transportFromStorage;
}
// Prioritize the .well-known/matrix/client, if available, over the configured SFU
// WELL_KNOWN: Prioritize the .well-known/matrix/client, if available, over the configured SFU
const domain = client.getDomain();
if (domain) {
if (domain && transport === undefined) {
// we use AutoDiscovery instead of relying on the MatrixClient having already
// been fully configured and started
const wellKnownFoci = (await AutoDiscovery.getRawClientConfig(domain))?.[
FOCI_WK_KEY
];
if (Array.isArray(wellKnownFoci)) {
const transport: LivekitTransportConfig | undefined = wellKnownFoci.find(
(f) => f && isLivekitTransportConfig(f),
);
if (transport !== undefined) {
logger.log("Using LiveKit transport from .well-known: ", transport);
return { ...transport, livekit_alias: livekitAlias };
const wellKnownTransport: LivekitTransportConfig | undefined =
wellKnownFoci.find((f) => f && isLivekitTransportConfig(f));
if (wellKnownTransport !== undefined) {
logger.info("Using LiveKit transport from .well-known: ", transport);
transport = { ...wellKnownTransport, livekit_alias: livekitAlias };
}
}
}
// CONFIG: Least prioritized; Load from config file
const urlFromConf = Config.get().livekit?.livekit_service_url;
if (urlFromConf) {
if (urlFromConf && transport === undefined) {
const transportFromConf: LivekitTransport = {
type: "livekit",
livekit_service_url: urlFromConf,
livekit_alias: livekitAlias,
};
logger.log("Using LiveKit transport from config: ", transportFromConf);
return transportFromConf;
logger.info("Using LiveKit transport from config: ", transportFromConf);
transport = transportFromConf;
}
throw new MatrixRTCTransportMissingError(domain ?? "");
}
if (!transport) throw new MatrixRTCTransportMissingError(domain ?? ""); // this will call the jwt/sfu/get endpoint to pre create the livekit room.
await getSFUConfigWithOpenID(
client,
transport.livekit_service_url,
transport.livekit_alias,
);
async function makeTransport(
client: MatrixClient,
roomId: string,
): Promise<LivekitTransport> {
const transport = await makeTransportInternal(client, roomId);
// this will call the jwt/sfu/get endpoint to pre create the livekit room.
try {
await getSFUConfigWithOpenID(
client,
transport.livekit_service_url,
transport.livekit_alias,
);
} catch (e) {
logger.warn(`Failed to get SFU config for transport: ${e}`);
}
return transport;
}

View File

@@ -0,0 +1,140 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
afterEach,
beforeEach,
describe,
expect,
it,
type Mock,
vi,
} from "vitest";
import { ConnectionState as LivekitConenctionState } from "livekit-client";
import { type BehaviorSubject } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { ObservableScope } from "../../ObservableScope";
import { constant } from "../../Behavior";
import {
mockLivekitRoom,
mockLocalParticipant,
mockMediaDevices,
} from "../../../utils/test";
import { Publisher } from "./Publisher";
import {
type Connection,
type ConnectionState,
} from "../remoteMembers/Connection";
import { type MuteStates } from "../../MuteStates";
import { FailToStartLivekitConnection } from "../../../utils/errors";
describe("Publisher", () => {
let scope: ObservableScope;
let connection: Connection;
let muteStates: MuteStates;
beforeEach(() => {
muteStates = {
audio: {
enabled$: constant(false),
unsetHandler: vi.fn(),
setHandler: vi.fn(),
},
video: {
enabled$: constant(false),
unsetHandler: vi.fn(),
setHandler: vi.fn(),
},
} as unknown as MuteStates;
scope = new ObservableScope();
connection = {
state$: constant({
state: "ConnectedToLkRoom",
livekitConnectionState$: constant(LivekitConenctionState.Connected),
}),
livekitRoom: mockLivekitRoom({
localParticipant: mockLocalParticipant({}),
}),
} as unknown as Connection;
});
afterEach(() => scope.end());
it("throws if livekit room could not publish", async () => {
const publisher = new Publisher(
scope,
connection,
mockMediaDevices({}),
muteStates,
constant({ supported: false, processor: undefined }),
logger,
);
// should do nothing if no tracks have been created yet.
await publisher.startPublishing();
expect(
connection.livekitRoom.localParticipant.publishTrack,
).not.toHaveBeenCalled();
await expect(publisher.createAndSetupTracks()).rejects.toThrow(
Error("audio and video is false"),
);
(muteStates.audio.enabled$ as BehaviorSubject<boolean>).next(true);
(
connection.livekitRoom.localParticipant.createTracks as Mock
).mockResolvedValue([{}, {}]);
await expect(publisher.createAndSetupTracks()).resolves.not.toThrow();
expect(
connection.livekitRoom.localParticipant.createTracks,
).toHaveBeenCalledOnce();
// failiour due to localParticipant.publishTrack
(
connection.livekitRoom.localParticipant.publishTrack as Mock
).mockRejectedValue(Error("testError"));
await expect(publisher.startPublishing()).rejects.toThrow(
new FailToStartLivekitConnection("testError"),
);
// does not try other conenction after the first one failed
expect(
connection.livekitRoom.localParticipant.publishTrack,
).toHaveBeenCalledTimes(1);
// failiour due to connection.state$
const beforeState = connection.state$.value;
(connection.state$ as BehaviorSubject<ConnectionState>).next({
state: "FailedToStart",
error: Error("testStartError"),
});
await expect(publisher.startPublishing()).rejects.toThrow(
new FailToStartLivekitConnection("testStartError"),
);
(connection.state$ as BehaviorSubject<ConnectionState>).next(beforeState);
// does not try other conenction after the first one failed
expect(
connection.livekitRoom.localParticipant.publishTrack,
).toHaveBeenCalledTimes(1);
// success case
(
connection.livekitRoom.localParticipant.publishTrack as Mock
).mockResolvedValue({});
await expect(publisher.startPublishing()).resolves.not.toThrow();
expect(
connection.livekitRoom.localParticipant.publishTrack,
).toHaveBeenCalledTimes(3);
});
});

View File

@@ -1,4 +1,5 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -13,6 +14,7 @@ import {
ConnectionState as LivekitConnectionState,
} from "livekit-client";
import {
BehaviorSubject,
map,
NEVER,
type Observable,
@@ -32,6 +34,10 @@ import { getUrlParams } from "../../../UrlParams.ts";
import { observeTrackReference$ } from "../../MediaViewModel.ts";
import { type Connection } from "../remoteMembers/Connection.ts";
import { type ObservableScope } from "../../ObservableScope.ts";
import {
ElementCallError,
FailToStartLivekitConnection,
} from "../../../utils/errors.ts";
/**
* A wrapper for a Connection object.
@@ -39,7 +45,6 @@ import { type ObservableScope } from "../../ObservableScope.ts";
* The Publisher is also responsible for creating the media tracks.
*/
export class Publisher {
public tracks: LocalTrack<Track.Kind>[] = [];
/**
* Creates a new Publisher.
* @param scope - The observable scope to use for managing the publisher.
@@ -51,19 +56,19 @@ export class Publisher {
*/
public constructor(
private scope: ObservableScope,
private connection: Connection,
private connection: Pick<Connection, "livekitRoom" | "state$">, //setE2EEEnabled,
devices: MediaDevices,
private readonly muteStates: MuteStates,
trackerProcessorState$: Behavior<ProcessorState>,
private logger?: Logger,
private logger: Logger,
) {
this.logger?.info("[PublishConnection] Create LiveKit room");
this.logger.info("Create LiveKit room");
const { controlledAudioDevices } = getUrlParams();
const room = connection.livekitRoom;
room.setE2EEEnabled(room.options.e2ee !== undefined)?.catch((e: Error) => {
this.logger?.error("Failed to set E2EE enabled on room", e);
this.logger.error("Failed to set E2EE enabled on room", e);
});
// Setup track processor syncing (blur)
@@ -73,13 +78,31 @@ export class Publisher {
this.workaroundRestartAudioInputTrackChrome(devices, scope);
this.scope.onEnd(() => {
this.logger?.info(
"[PublishConnection] Scope ended -> stop publishing all tracks",
);
this.logger.info("Scope ended -> stop publishing all tracks");
void this.stopPublishing();
});
// TODO move mute state handling here using reconcile (instead of inside the mute state class)
// this.scope.reconcile(
// this.scope.behavior(
// combineLatest([this.muteStates.video.enabled$, this.tracks$]),
// ),
// async ([videoEnabled, tracks]) => {
// const track = tracks.find((t) => t.kind == Track.Kind.Video);
// if (!track) return;
// if (videoEnabled) {
// await track.unmute();
// } else {
// await track.mute();
// }
// },
// );
}
private _tracks$ = new BehaviorSubject<LocalTrack<Track.Kind>[]>([]);
public tracks$ = this._tracks$ as Behavior<LocalTrack<Track.Kind>[]>;
/**
* Start the connection to LiveKit and publish local tracks.
*
@@ -93,51 +116,46 @@ export class Publisher {
* @throws {InsufficientCapacityError} if the LiveKit server indicates that it has insufficient capacity to accept the connection.
* @throws {SFURoomCreationRestrictedError} if the LiveKit server indicates that the room does not exist and cannot be created.
*/
public async createAndSetupTracks(): Promise<LocalTrack[]> {
public async createAndSetupTracks(): Promise<void> {
this.logger.debug("createAndSetupTracks called");
const lkRoom = this.connection.livekitRoom;
// Observe mute state changes and update LiveKit microphone/camera states accordingly
this.observeMuteStates(this.scope);
// TODO: This should be an autostarted connection no need to start here. just check the connection state.
// TODO: This will fetch the JWT token. Perhaps we could keep it preloaded
// instead? This optimization would only be safe for a publish connection,
// because we don't want to leak the user's intent to perhaps join a call to
// remote servers before they actually commit to it.
// const { promise, resolve, reject } = Promise.withResolvers<void>();
// const sub = this.connection.state$.subscribe((s) => {
// if (s.state === "FailedToStart") {
// reject(new Error("Disconnected from LiveKit server"));
// } else if (s.state === "ConnectedToLkRoom") {
// resolve();
// }
// });
// try {
// await promise;
// } catch (e) {
// throw e;
// } finally {
// sub.unsubscribe();
// }
// TODO-MULTI-SFU: Prepublish a microphone track
const audio = this.muteStates.audio.enabled$.value;
const video = this.muteStates.video.enabled$.value;
// createTracks throws if called with audio=false and video=false
if (audio || video) {
// TODO this can still throw errors? It will also prompt for permissions if not already granted
this.tracks =
(await lkRoom.localParticipant
.createTracks({
audio,
video,
})
.catch((error) => {
this.logger?.error("Failed to create tracks", error);
})) ?? [];
return lkRoom.localParticipant
.createTracks({
audio,
video,
})
.then((tracks) => {
this.logger.info(
"created track",
tracks.map((t) => t.kind + ", " + t.id),
);
this._tracks$.next(tracks);
})
.catch((error) => {
this.logger.error("Failed to create tracks", error);
});
}
return this.tracks;
throw Error("audio and video is false");
}
private _publishing$ = new BehaviorSubject<boolean>(false);
public publishing$ = this.scope.behavior(this._publishing$);
/**
*
* @returns
* @throws ElementCallError
*/
public async startPublishing(): Promise<LocalTrack[]> {
this.logger.debug("startPublishing called");
const lkRoom = this.connection.livekitRoom;
const { promise, resolve, reject } = Promise.withResolvers<void>();
const sub = this.connection.state$.subscribe((s) => {
@@ -146,10 +164,14 @@ export class Publisher {
resolve();
break;
case "FailedToStart":
reject(new Error("Failed to connect to LiveKit server"));
reject(
s.error instanceof ElementCallError
? s.error
: new FailToStartLivekitConnection(s.error.message),
);
break;
default:
this.logger?.info("waiting for connection: ", s.state);
this.logger.info("waiting for connection: ", s.state);
}
});
try {
@@ -159,19 +181,27 @@ export class Publisher {
} finally {
sub.unsubscribe();
}
for (const track of this.tracks) {
for (const track of this.tracks$.value) {
this.logger.info("publish ", this.tracks$.value.length, "tracks");
// TODO: handle errors? Needs the signaling connection to be up, but it has some retries internally
// with a timeout.
await lkRoom.localParticipant.publishTrack(track).catch((error) => {
this.logger?.error("Failed to publish track", error);
this.logger.error("Failed to publish track", error);
throw new FailToStartLivekitConnection(
error instanceof Error ? error.message : error,
);
});
this.logger.info("published track ", track.kind, track.id);
// TODO: check if the connection is still active? and break the loop if not?
}
return this.tracks;
this._publishing$.next(true);
return this.tracks$.value;
}
public async stopPublishing(): Promise<void> {
this.logger.debug("stopPublishing called");
// TODO-MULTI-SFU: Move these calls back to ObservableScope.onEnd once scope
// actually has the right lifetime
this.muteStates.audio.unsetHandler();
@@ -183,7 +213,28 @@ export class Publisher {
if (p.track !== undefined) tracks.push(p.track);
};
localParticipant.trackPublications.forEach(addToTracksIfDefined);
await localParticipant.unpublishTracks(tracks);
this.logger.debug(
"list of tracks to unpublish:",
tracks.map((t) => t.kind + ", " + t.id),
"start unpublishing now",
);
await localParticipant.unpublishTracks(tracks).catch((error) => {
this.logger.error("Failed to unpublish tracks", error);
throw error;
});
this.logger.debug(
"unpublished tracks",
tracks.map((t) => t.kind + ", " + t.id),
);
this._publishing$.next(false);
}
/**
* Stops all tracks that are currently running
*/
public stopTracks(): void {
this.tracks$.value.forEach((t) => t.stop());
this._tracks$.next([]);
}
/// Private methods
@@ -220,6 +271,9 @@ export class Publisher {
// the process of being restarted.
activeMicTrack.mediaStreamTrack.readyState !== "ended"
) {
this.logger?.info(
"Restarting audio device track due to active media device changed (workaroundRestartAudioInputTrackChrome)",
);
// Restart the track, which will cause Livekit to do another
// getUserMedia() call with deviceId: default to get the *new* default device.
// Note that room.switchActiveDevice() won't work: Livekit will ignore it because
@@ -228,7 +282,7 @@ export class Publisher {
.getTrackPublication(Track.Source.Microphone)
?.audioTrack?.restartTrack()
.catch((e) => {
this.logger?.error(`Failed to restart audio device track`, e);
this.logger.error(`Failed to restart audio device track`, e);
});
}
});
@@ -248,7 +302,7 @@ export class Publisher {
selected$.pipe(scope.bind()).subscribe((device) => {
if (lkRoom.state != LivekitConnectionState.Connected) return;
// if (this.connectionState$.value !== ConnectionState.Connected) return;
this.logger?.info(
this.logger.info(
"[LivekitRoom] syncDevice room.getActiveDevice(kind) !== d.id :",
lkRoom.getActiveDevice(kind),
" !== ",
@@ -261,7 +315,7 @@ export class Publisher {
lkRoom
.switchActiveDevice(kind, device.id)
.catch((e: Error) =>
this.logger?.error(
this.logger.error(
`Failed to sync ${kind} device with LiveKit`,
e,
),
@@ -286,10 +340,7 @@ export class Publisher {
try {
await lkRoom.localParticipant.setMicrophoneEnabled(desired);
} catch (e) {
this.logger?.error(
"Failed to update LiveKit audio input mute state",
e,
);
this.logger.error("Failed to update LiveKit audio input mute state", e);
}
return lkRoom.localParticipant.isMicrophoneEnabled;
});
@@ -297,10 +348,7 @@ export class Publisher {
try {
await lkRoom.localParticipant.setCameraEnabled(desired);
} catch (e) {
this.logger?.error(
"Failed to update LiveKit video input mute state",
e,
);
this.logger.error("Failed to update LiveKit video input mute state", e);
}
return lkRoom.localParticipant.isCameraEnabled;
});

View File

@@ -1,4 +1,5 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -192,7 +193,7 @@ describe("Start connection states", () => {
capturedState = capturedStates.pop();
if (capturedState!.state === "FailedToStart") {
expect(capturedState!.error.message).toEqual("Something went wrong");
expect(capturedState!.transport.livekit_alias).toEqual(
expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias,
);
} else {
@@ -248,7 +249,7 @@ describe("Start connection states", () => {
expect(capturedState?.error.message).toContain(
"SFU Config fetch failed with exception Error",
);
expect(capturedState?.transport.livekit_alias).toEqual(
expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias,
);
} else {
@@ -312,7 +313,7 @@ describe("Start connection states", () => {
expect(capturedState.error.message).toContain(
"Failed to connect to livekit",
);
expect(capturedState.transport.livekit_alias).toEqual(
expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias,
);
} else {

View File

@@ -1,4 +1,5 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -18,7 +19,7 @@ import {
RoomEvent,
} from "livekit-client";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, map, type Observable } from "rxjs";
import { BehaviorSubject, map } from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger";
import {
@@ -49,16 +50,14 @@ export interface ConnectionOpts {
export type ConnectionState =
| { state: "Initialized" }
| { state: "FetchingConfig"; transport: LivekitTransport }
| { state: "ConnectingToLkRoom"; transport: LivekitTransport }
| { state: "PublishingTracks"; transport: LivekitTransport }
| { state: "FailedToStart"; error: Error; transport: LivekitTransport }
| { state: "FetchingConfig" }
| { state: "ConnectingToLkRoom" }
| {
state: "ConnectedToLkRoom";
livekitConnectionState$: Observable<LivekitConenctionState>;
transport: LivekitTransport;
livekitConnectionState$: Behavior<LivekitConenctionState>;
}
| { state: "Stopped"; transport: LivekitTransport };
| { state: "FailedToStart"; error: Error }
| { state: "Stopped" };
/**
* A connection to a Matrix RTC LiveKit backend.
@@ -76,6 +75,24 @@ export class Connection {
*/
public readonly state$: Behavior<ConnectionState> = this._state$;
/**
* The media transport to connect to.
*/
public readonly transport: LivekitTransport;
public readonly livekitRoom: LivekitRoom;
private scope: ObservableScope;
/**
* An observable of the participants that are publishing on this connection. (Excluding our local participant)
* This is derived from `participantsIncludingSubscribers$` and `remoteTransports$`.
* It filters the participants to only those that are associated with a membership that claims to publish on this connection.
*/
public readonly remoteParticipantsWithTracks$: Behavior<
PublishingParticipant[]
>;
/**
* Whether the connection has been stopped.
* @see Connection.stop
@@ -95,7 +112,6 @@ export class Connection {
* @throws {InsufficientCapacityError} if the LiveKit server indicates that it has insufficient capacity to accept the connection.
* @throws {SFURoomCreationRestrictedError} if the LiveKit server indicates that the room does not exist and cannot be created.
*/
// TODO dont make this throw and instead store a connection error state in this class?
// TODO consider an autostart pattern...
public async start(): Promise<void> {
this.logger.debug("Starting Connection");
@@ -103,7 +119,6 @@ export class Connection {
try {
this._state$.next({
state: "FetchingConfig",
transport: this.transport,
});
const { url, jwt } = await this.getSFUConfigWithOpenID();
// If we were stopped while fetching the config, don't proceed to connect
@@ -111,7 +126,6 @@ export class Connection {
this._state$.next({
state: "ConnectingToLkRoom",
transport: this.transport,
});
try {
await this.livekitRoom.connect(url, jwt);
@@ -142,15 +156,15 @@ export class Connection {
this._state$.next({
state: "ConnectedToLkRoom",
transport: this.transport,
livekitConnectionState$: connectionStateObserver(this.livekitRoom),
livekitConnectionState$: this.scope.behavior(
connectionStateObserver(this.livekitRoom),
),
});
} catch (error) {
this.logger.debug(`Failed to connect to LiveKit room: ${error}`);
this._state$.next({
state: "FailedToStart",
error: error instanceof Error ? error : new Error(`${error}`),
transport: this.transport,
});
throw error;
}
@@ -178,28 +192,11 @@ export class Connection {
await this.livekitRoom.disconnect();
this._state$.next({
state: "Stopped",
transport: this.transport,
});
this.stopped = true;
}
/**
* An observable of the participants that are publishing on this connection. (Excluding our local participant)
* This is derived from `participantsIncludingSubscribers$` and `remoteTransports$`.
* It filters the participants to only those that are associated with a membership that claims to publish on this connection.
*/
public readonly remoteParticipantsWithTracks$: Behavior<
PublishingParticipant[]
>;
/**
* The media transport to connect to.
*/
public readonly transport: LivekitTransport;
private readonly client: OpenIDClientParts;
public readonly livekitRoom: LivekitRoom;
private readonly logger: Logger;
/**
@@ -216,6 +213,7 @@ export class Connection {
);
const { transport, client, scope } = opts;
this.scope = scope;
this.livekitRoom = opts.livekitRoomFactory();
this.transport = transport;
this.client = client;

View File

@@ -7,10 +7,11 @@ Please see LICENSE in the repository root for full details.
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import {
type E2EEOptions,
Room as LivekitRoom,
type RoomOptions,
type BaseKeyProvider,
type E2EEManagerOptions,
type BaseE2EEManager,
} from "livekit-client";
import { type Logger } from "matrix-js-sdk/lib/logger";
import E2EEWorker from "livekit-client/e2ee-worker?worker";
@@ -23,6 +24,7 @@ import type { Behavior } from "../../Behavior.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { defaultLiveKitOptions } from "../../../livekit/options.ts";
// TODO evaluate if this should be done like the Publisher Factory
export interface ConnectionFactory {
createConnection(
transport: LivekitTransport,
@@ -40,8 +42,10 @@ export class ECConnectionFactory implements ConnectionFactory {
* @param client - The OpenID client parts for authentication, needed to get openID and JWT tokens.
* @param devices - Used for video/audio out/in capture options.
* @param processorState$ - Effects like background blur (only for publishing connection?)
* @param e2eeLivekitOptions - The E2EE options to use for the LiveKit Room.
* @param livekitKeyProvider
* @param controlledAudioDevices - Option to indicate whether audio output device is controlled externally (native mobile app).
* @param echoCancellation - Whether to enable echo cancellation for audio capture.
* @param noiseSuppression - Whether to enable noise suppression for audio capture.
* @param livekitRoomFactory - Optional factory function (for testing) to create LivekitRoom instances. If not provided, a default factory is used.
*/
public constructor(
@@ -51,20 +55,24 @@ export class ECConnectionFactory implements ConnectionFactory {
livekitKeyProvider: BaseKeyProvider | undefined,
private controlledAudioDevices: boolean,
livekitRoomFactory?: () => LivekitRoom,
echoCancellation: boolean = true,
noiseSuppression: boolean = true,
) {
const defaultFactory = (): LivekitRoom =>
new LivekitRoom(
generateRoomOption(
this.devices,
this.processorState$.value,
livekitKeyProvider && {
generateRoomOption({
devices: this.devices,
processorState: this.processorState$.value,
e2eeLivekitOptions: livekitKeyProvider && {
keyProvider: livekitKeyProvider,
// It's important that every room use a separate E2EE worker.
// They get confused if given streams from multiple rooms.
worker: new E2EEWorker(),
},
this.controlledAudioDevices,
),
controlledAudioDevices: this.controlledAudioDevices,
echoCancellation,
noiseSuppression,
}),
);
this.livekitRoomFactory = livekitRoomFactory ?? defaultFactory;
}
@@ -89,12 +97,24 @@ export class ECConnectionFactory implements ConnectionFactory {
/**
* Generate the initial LiveKit RoomOptions based on the current media devices and processor state.
*/
function generateRoomOption(
devices: MediaDevices,
processorState: ProcessorState,
e2eeLivekitOptions: E2EEOptions | undefined,
controlledAudioDevices: boolean,
): RoomOptions {
function generateRoomOption({
devices,
processorState,
e2eeLivekitOptions,
controlledAudioDevices,
echoCancellation,
noiseSuppression,
}: {
devices: MediaDevices;
processorState: ProcessorState;
e2eeLivekitOptions:
| E2EEManagerOptions
| { e2eeManager: BaseE2EEManager }
| undefined;
controlledAudioDevices: boolean;
echoCancellation: boolean;
noiseSuppression: boolean;
}): RoomOptions {
return {
...defaultLiveKitOptions,
videoCaptureDefaults: {
@@ -105,6 +125,8 @@ function generateRoomOption(
audioCaptureDefaults: {
...defaultLiveKitOptions.audioCaptureDefaults,
deviceId: devices.audioInput.selected$.value?.id,
echoCancellation,
noiseSuppression,
},
audioOutput: {
// When using controlled audio devices, we don't want to set the

View File

@@ -11,7 +11,7 @@ import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { type Participant as LivekitParticipant } from "livekit-client";
import { logger } from "matrix-js-sdk/lib/logger";
import { Epoch, ObservableScope } from "../../ObservableScope.ts";
import { Epoch, mapEpoch, ObservableScope } from "../../ObservableScope.ts";
import {
createConnectionManager$,
type ConnectionManagerData,
@@ -73,7 +73,7 @@ afterEach(() => {
describe("connections$ stream", () => {
test("Should create and start new connections for each transports", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const { connections$ } = createConnectionManager$({
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("a", {
@@ -82,7 +82,9 @@ describe("connections$ stream", () => {
logger: logger,
});
expectObservable(connections$).toBe("a", {
expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("a", {
a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value;
expect(connections.length).toBe(2);
@@ -110,7 +112,7 @@ describe("connections$ stream", () => {
test("Should start connection only once", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const { connections$ } = createConnectionManager$({
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("abcdef", {
@@ -124,7 +126,9 @@ describe("connections$ stream", () => {
logger: logger,
});
expectObservable(connections$).toBe("xxxxxa", {
expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("xxxxxa", {
x: expect.anything(),
a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value;
@@ -153,7 +157,7 @@ describe("connections$ stream", () => {
test("Should cleanup connections when not needed anymore", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const { connections$ } = createConnectionManager$({
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("abc", {
@@ -164,7 +168,9 @@ describe("connections$ stream", () => {
logger: logger,
});
expectObservable(connections$).toBe("xab", {
expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("xab", {
x: expect.anything(),
a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value;

View File

@@ -1,6 +1,3 @@
// TODOs:
// - make ConnectionManager its own actual class
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd.
@@ -95,9 +92,7 @@ interface Props {
}
// TODO - write test for scopes (do we really need to bind scope)
export interface IConnectionManager {
transports$: Behavior<Epoch<LivekitTransport[]>>;
connectionManagerData$: Behavior<Epoch<ConnectionManagerData>>;
connections$: Behavior<Epoch<Connection[]>>;
}
/**
* Crete a `ConnectionManager`
@@ -220,7 +215,7 @@ export function createConnectionManager$({
new Epoch(new ConnectionManagerData()),
);
return { transports$, connectionManagerData$, connections$ };
return { connectionManagerData$ };
}
function removeDuplicateTransports(

View File

@@ -0,0 +1,133 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { Room as LivekitRoom } from "livekit-client";
import { BehaviorSubject } from "rxjs";
import fetchMock from "fetch-mock";
import { logger } from "matrix-js-sdk/lib/logger";
import EventEmitter from "events";
import { ObservableScope } from "../../ObservableScope.ts";
import { ECConnectionFactory } from "./ConnectionFactory.ts";
import type { OpenIDClientParts } from "../../../livekit/openIDSFU.ts";
import { exampleTransport, mockMediaDevices } from "../../../utils/test.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { constant } from "../../Behavior";
// At the top of your test file, after imports
vi.mock("livekit-client", async (importOriginal) => {
return {
...(await importOriginal()),
Room: vi.fn().mockImplementation(function (this: LivekitRoom, options) {
const emitter = new EventEmitter();
return {
on: emitter.on.bind(emitter),
off: emitter.off.bind(emitter),
emit: emitter.emit.bind(emitter),
disconnect: vi.fn(),
remoteParticipants: new Map(),
} as unknown as LivekitRoom;
}),
};
});
let testScope: ObservableScope;
let mockClient: OpenIDClientParts;
beforeEach(() => {
testScope = new ObservableScope();
mockClient = {
getOpenIdToken: vi.fn().mockReturnValue(""),
getDeviceId: vi.fn().mockReturnValue("DEV000"),
};
});
describe("ECConnectionFactory - Audio inputs options", () => {
test.each([
{ echo: true, noise: true },
{ echo: true, noise: false },
{ echo: false, noise: true },
{ echo: false, noise: false },
])(
"it sets echoCancellation=$echo and noiseSuppression=$noise based on constructor parameters",
({ echo, noise }) => {
// test("it sets echoCancellation and noiseSuppression based on constructor parameters", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
undefined,
echo,
noise,
);
ecConnectionFactory.createConnection(exampleTransport, testScope, logger);
// Check if Room was constructed with expected options
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
audioCaptureDefaults: expect.objectContaining({
echoCancellation: echo,
noiseSuppression: noise,
}),
}),
);
},
);
});
describe("ECConnectionFactory - ControlledAudioDevice", () => {
test.each([{ controlled: true }, { controlled: false }])(
"it sets controlledAudioDevice=$controlled then uses deviceId accordingly",
({ controlled }) => {
// test("it sets echoCancellation and noiseSuppression based on constructor parameters", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
mockMediaDevices({
audioOutput: {
available$: constant(new Map<never, never>()),
selected$: constant({ id: "DEV00", virtualEarpiece: false }),
select: () => {},
},
}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
controlled,
undefined,
false,
false,
);
ecConnectionFactory.createConnection(exampleTransport, testScope, logger);
// Check if Room was constructed with expected options
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
audioOutput: expect.objectContaining({
deviceId: controlled ? undefined : "DEV00",
}),
}),
);
},
);
});
afterEach(() => {
testScope.end();
fetchMock.reset();
});

View File

@@ -104,7 +104,6 @@ describe("MatrixMemberMetadata", () => {
}
}
// TODO this is a regression, now there the own user is not always in the map. Ask Timo if fine
it("should show our own user if present in rtc session and room", () => {
withTestScheduler(({ behavior, expectObservable }) => {
fakeMemberWith({
@@ -517,7 +516,6 @@ describe("MatrixMemberMetadata", () => {
}
}
// TODO this is a regression, now there the own user is not always in the map. Ask Timo if fine
it("should use avatar url from room members", () => {
withTestScheduler(({ behavior, expectObservable }) => {
fakeMemberWith({

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.

View File

@@ -1,13 +1,14 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { BehaviorSubject, combineLatest, Subject } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { sleep } from "matrix-js-sdk/lib/utils";
import {
Epoch,
@@ -102,3 +103,137 @@ describe("Epoch", () => {
s$.complete();
});
});
describe("Reconcile", () => {
beforeEach(() => {
vi.useFakeTimers();
});
afterEach(() => {
vi.useRealTimers();
});
it("should wait clean up before processing next", async () => {
vi.useFakeTimers();
const scope = new ObservableScope();
const behavior$ = new BehaviorSubject<number>(0);
const setup = vi.fn().mockImplementation(async () => await sleep(100));
const cleanup = vi
.fn()
.mockImplementation(async (n: number) => await sleep(100));
scope.reconcile(behavior$, async (value) => {
await setup();
return async (): Promise<void> => {
await cleanup(value);
};
});
// Let the initial setup process
await vi.advanceTimersByTimeAsync(120);
expect(setup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledTimes(0);
// Send next value
behavior$.next(1);
await vi.advanceTimersByTimeAsync(50);
// Should not have started setup for 1 yet
expect(setup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledWith(0);
// Let cleanup finish
await vi.advanceTimersByTimeAsync(50);
// Now setup for 1 should have started
expect(setup).toHaveBeenCalledTimes(2);
});
it("should skip intermediates values that are not setup", async () => {
vi.useFakeTimers();
const scope = new ObservableScope();
const behavior$ = new BehaviorSubject<number>(0);
const setup = vi
.fn()
.mockImplementation(async (n: number) => await sleep(100));
const cleanupLock = Promise.withResolvers();
const cleanup = vi
.fn()
.mockImplementation(async (n: number) => await cleanupLock.promise);
scope.reconcile(behavior$, async (value) => {
await setup(value);
return async (): Promise<void> => {
await cleanup(value);
};
});
// Let the initial setup process (0)
await vi.advanceTimersByTimeAsync(120);
// Send 4 next values quickly
behavior$.next(1);
behavior$.next(2);
behavior$.next(3);
behavior$.next(4);
await vi.advanceTimersByTimeAsync(3000);
// should have only called cleanup for 0
expect(cleanup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledWith(0);
// Let cleanup finish
cleanupLock.resolve(undefined);
await vi.advanceTimersByTimeAsync(120);
// Now setup for 4 should have started, skipping 1,2,3
expect(setup).toHaveBeenCalledTimes(2);
expect(setup).toHaveBeenCalledWith(4);
expect(setup).not.toHaveBeenCalledWith(1);
expect(setup).not.toHaveBeenCalledWith(2);
expect(setup).not.toHaveBeenCalledWith(3);
});
it("should wait for setup to complete before starting cleanup", async () => {
vi.useFakeTimers();
const scope = new ObservableScope();
const behavior$ = new BehaviorSubject<number>(0);
const setup = vi
.fn()
.mockImplementation(async (n: number) => await sleep(3000));
const cleanupLock = Promise.withResolvers();
const cleanup = vi
.fn()
.mockImplementation(async (n: number) => await cleanupLock.promise);
scope.reconcile(behavior$, async (value) => {
await setup(value);
return async (): Promise<void> => {
await cleanup(value);
};
});
await vi.advanceTimersByTimeAsync(500);
// Setup for 0 should be in progress
expect(setup).toHaveBeenCalledTimes(1);
behavior$.next(1);
await vi.advanceTimersByTimeAsync(500);
// Should not have started setup for 1 yet
expect(setup).not.toHaveBeenCalledWith(1);
// Should not have called cleanup yet, because the setup for 0 is not done
expect(cleanup).toHaveBeenCalledTimes(0);
// Let setup for 0 finish
await vi.advanceTimersByTimeAsync(2500 + 100);
// Now cleanup for 0 should have started
expect(cleanup).toHaveBeenCalledTimes(1);
expect(cleanup).toHaveBeenCalledWith(0);
cleanupLock.resolve(undefined);
await vi.advanceTimersByTimeAsync(100);
// Now setup for 1 should have started
expect(setup).toHaveBeenCalledWith(1);
});
});

View File

@@ -123,8 +123,22 @@ export class ObservableScope {
callback: (value: T) => Promise<(() => Promise<void>) | void>,
): void {
let latestValue: T | typeof nothing = nothing;
let reconciledValue: T | typeof nothing = nothing;
let reconcilePromise: Promise<void> | undefined = undefined;
let cleanUp: (() => Promise<void>) | void = undefined;
let prevVal: T | typeof nothing = nothing;
// While this loop runs it will process the latest from `value$` until it caught up with the updates.
// It might skip updates from `value$` and only process the newest value after callback has resolved.
const reconcileLoop = async (): Promise<void> => {
while (latestValue !== prevVal) {
await cleanUp?.(); // Call the previous value's clean-up handler
prevVal = latestValue;
if (latestValue !== nothing) cleanUp = await callback(latestValue); // Sync current value...
// `latestValue` might have gotten updated during the `await callback`. That is why we loop here
}
};
value$
.pipe(
catchError(() => EMPTY), // Ignore errors
@@ -132,23 +146,15 @@ export class ObservableScope {
endWith(nothing), // Clean up when the scope ends
)
.subscribe((value) => {
void (async (): Promise<void> => {
if (latestValue === nothing) {
latestValue = value;
while (latestValue !== reconciledValue) {
await cleanUp?.(); // Call the previous value's clean-up handler
reconciledValue = latestValue;
if (latestValue !== nothing)
cleanUp = await callback(latestValue); // Sync current value
}
// Reset to signal that reconciliation is done for now
latestValue = nothing;
} else {
// There's already an instance of the above 'while' loop running
// concurrently. Just update the latest value and let it be handled.
latestValue = value;
}
})();
// Always track the latest value! The `reconcileLoop` will run until it "processed" the "last" `latestValue`.
latestValue = value;
// There's already an instance of the below 'reconcileLoop' loop running
// concurrently. So lets let the loop handle it. NEVER instanciate two `reconcileLoop`s.
if (reconcilePromise) return;
reconcilePromise = reconcileLoop().finally(() => {
reconcilePromise = undefined;
});
});
}

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.

View File

@@ -13,6 +13,8 @@ export enum ErrorCode {
*/
MISSING_MATRIX_RTC_TRANSPORT = "MISSING_MATRIX_RTC_TRANSPORT",
CONNECTION_LOST_ERROR = "CONNECTION_LOST_ERROR",
INTERNAL_MEMBERSHIP_MANAGER = "INTERNAL_MEMBERSHIP_MANAGER",
FAILED_TO_START_LIVEKIT = "FAILED_TO_START_LIVEKIT",
/** LiveKit indicates that the server has hit its track limits */
INSUFFICIENT_CAPACITY_ERROR = "INSUFFICIENT_CAPACITY_ERROR",
E2EE_NOT_SUPPORTED = "E2EE_NOT_SUPPORTED",
@@ -27,6 +29,7 @@ export enum ErrorCategory {
NETWORK_CONNECTIVITY = "NETWORK_CONNECTIVITY",
CLIENT_CONFIGURATION = "CLIENT_CONFIGURATION",
UNKNOWN = "UNKNOWN",
SYSTEM_FAILURE = "SYSTEM_FAILURE",
// SYSTEM_FAILURE / FEDERATION_FAILURE ..
}
@@ -83,6 +86,18 @@ export class ConnectionLostError extends ElementCallError {
}
}
export class MembershipManagerError extends ElementCallError {
public constructor(error: Error) {
super(
t("error.membership_manager"),
ErrorCode.INTERNAL_MEMBERSHIP_MANAGER,
ErrorCategory.SYSTEM_FAILURE,
t("error.membership_manager_description"),
error,
);
}
}
export class E2EENotSupportedError extends ElementCallError {
public constructor() {
super(
@@ -120,6 +135,17 @@ export class FailToGetOpenIdToken extends ElementCallError {
}
}
export class FailToStartLivekitConnection extends ElementCallError {
public constructor(e?: string) {
super(
t("error.failed_to_start_livekit"),
ErrorCode.FAILED_TO_START_LIVEKIT,
ErrorCategory.NETWORK_CONNECTIVITY,
e,
);
}
}
export class InsufficientCapacityError extends ElementCallError {
public constructor() {
super(

View File

@@ -44,12 +44,12 @@ import {
Track,
} from "livekit-client";
import { randomUUID } from "crypto";
import {
type RoomAndToDeviceEvents,
type RoomAndToDeviceEventsHandlerMap,
} from "matrix-js-sdk/lib/matrixrtc/RoomAndToDeviceKeyTransport";
import { type TrackReference } from "@livekit/components-core";
import EventEmitter from "events";
import {
type KeyTransportEvents,
type KeyTransportEventsHandlerMap,
} from "matrix-js-sdk/lib/matrixrtc/IKeyTransport";
import {
LocalUserMediaViewModel,
@@ -79,6 +79,13 @@ export async function flushPromises(): Promise<void> {
await new Promise<void>((resolve) => window.setTimeout(resolve));
}
export type NodeEventHandler = (...args: unknown[]) => void;
export interface NodeStyleEventEmitter {
addListener(eventName: string | symbol, handler: NodeEventHandler): this;
removeListener(eventName: string | symbol, handler: NodeEventHandler): this;
}
export interface OurRunHelpers extends RunHelpers {
/**
* Schedules a sequence of actions to happen, as described by a marble
@@ -277,6 +284,8 @@ export function mockLivekitRoom(
): LivekitRoom {
const livekitRoom = {
options: {},
setE2EEEnabled: vi.fn(),
...mockEmitter(),
...room,
} as Partial<LivekitRoom> as LivekitRoom;
@@ -299,7 +308,9 @@ export function mockLocalParticipant(
return {
isLocal: true,
trackPublications: new Map(),
unpublishTracks: async () => Promise.resolve(),
publishTrack: vi.fn(),
unpublishTracks: vi.fn().mockResolvedValue([]),
createTracks: vi.fn(),
getTrackPublication: () =>
({}) as Partial<LocalTrackPublication> as LocalTrackPublication,
...mockEmitter(),
@@ -387,9 +398,9 @@ export function mockConfig(
}
export class MockRTCSession extends TypedEventEmitter<
MatrixRTCSessionEvent | RoomAndToDeviceEvents | MembershipManagerEvent,
MatrixRTCSessionEventHandlerMap &
RoomAndToDeviceEventsHandlerMap &
MatrixRTCSessionEvent | MembershipManagerEvent | KeyTransportEvents,
KeyTransportEventsHandlerMap &
MatrixRTCSessionEventHandlerMap &
MembershipManagerEventHandlerMap
> {
public asMockedSession(): MockedObject<MatrixRTCSession> {

View File

@@ -2731,37 +2731,39 @@ __metadata:
languageName: node
linkType: hard
"@livekit/components-core@npm:0.12.10, @livekit/components-core@npm:^0.12.0":
version: 0.12.10
resolution: "@livekit/components-core@npm:0.12.10"
"@livekit/components-core@npm:0.12.11, @livekit/components-core@npm:^0.12.0":
version: 0.12.11
resolution: "@livekit/components-core@npm:0.12.11"
dependencies:
"@floating-ui/dom": "npm:1.6.13"
loglevel: "npm:1.9.1"
rxjs: "npm:7.8.2"
peerDependencies:
livekit-client: ^2.13.3
livekit-client: ^2.15.14
tslib: ^2.6.2
checksum: 10c0/bfd84fb950f72dd037bd5329658c1e750a8ac6b8f2953ea673e3e944b8ea9d412ef9c98eb8b690052323e03c675964b162aacb00e60530cdc5187f77d21979bd
checksum: 10c0/9c2ac3d30bb8cc9067ae0b2049784f81e90e57df9eabf7edbaf3c8ceb65a63f644a4e6abeb6cc38d3ebe52663d8dbb88535e01a965011f365d5ae1f3daf86052
languageName: node
linkType: hard
"@livekit/components-react@npm:^2.0.0":
version: 2.9.15
resolution: "@livekit/components-react@npm:2.9.15"
version: 2.9.16
resolution: "@livekit/components-react@npm:2.9.16"
dependencies:
"@livekit/components-core": "npm:0.12.10"
"@livekit/components-core": "npm:0.12.11"
clsx: "npm:2.1.1"
events: "npm:^3.3.0"
jose: "npm:^6.0.12"
usehooks-ts: "npm:3.1.1"
peerDependencies:
"@livekit/krisp-noise-filter": ^0.2.12 || ^0.3.0
livekit-client: ^2.13.3
livekit-client: ^2.15.14
react: ">=18"
react-dom: ">=18"
tslib: ^2.6.2
peerDependenciesMeta:
"@livekit/krisp-noise-filter":
optional: true
checksum: 10c0/58a93d85c3b8267d0afd00eceb4f34992ce66124f93450e828a78dd825ecc20e254c3123ed22ec33061a3728f50f4f020ff896769fb3a0ff79656ed8cf452a2b
checksum: 10c0/4ba4ff473c5a29d3107412733a6676a3b708d70684ed463e9b34cda26abb3d2f317c2828a52e730837b756de9df3fc248260d6f390aedebfb6ec96ef63c7b151
languageName: node
linkType: hard
@@ -5247,12 +5249,12 @@ __metadata:
languageName: node
linkType: hard
"@types/node@npm:^22.0.0":
version: 22.17.0
resolution: "@types/node@npm:22.17.0"
"@types/node@npm:^24.0.0":
version: 24.10.0
resolution: "@types/node@npm:24.10.0"
dependencies:
undici-types: "npm:~6.21.0"
checksum: 10c0/e1c603b660d3de3243dfc02ded5d40623ff3f36315ffbdd8cdc81bc2c5a8da172035879d437b72e9fa61ca01827f28e9c2b0c32898f411a8e9ba0a5efac0b4ca
undici-types: "npm:~7.16.0"
checksum: 10c0/f82ed7194e16f5590ef7afdc20c6d09068c76d50278b485ede8f0c5749683536e3064ffa8def8db76915196afb3724b854aa5723c64d6571b890b14492943b46
languageName: node
linkType: hard
@@ -7507,7 +7509,7 @@ __metadata:
"@types/grecaptcha": "npm:^3.0.9"
"@types/jsdom": "npm:^21.1.7"
"@types/lodash-es": "npm:^4.17.12"
"@types/node": "npm:^22.0.0"
"@types/node": "npm:^24.0.0"
"@types/pako": "npm:^2.0.3"
"@types/qrcode": "npm:^1.5.5"
"@types/react": "npm:^19.0.0"
@@ -7545,8 +7547,8 @@ __metadata:
livekit-client: "npm:^2.13.0"
lodash-es: "npm:^4.17.21"
loglevel: "npm:^1.9.1"
matrix-js-sdk: "github:matrix-org/matrix-js-sdk#head=toger5/sticky-events&commit=e7f5bec51b6f70501a025b79fe5021c933385b21"
matrix-widget-api: "npm:^1.13.0"
matrix-js-sdk: "npm:^39.2.0"
matrix-widget-api: "npm:^1.14.0"
normalize.css: "npm:^8.0.1"
observable-hooks: "npm:^4.2.3"
pako: "npm:^2.0.4"
@@ -9828,6 +9830,13 @@ __metadata:
languageName: node
linkType: hard
"jose@npm:^6.0.12":
version: 6.1.2
resolution: "jose@npm:6.1.2"
checksum: 10c0/55f79426f43e652ed6d5de938d50f66bb0a10dcae078db81a23f8d3303e889ce226f000e815f3211f9956bb84badce10da892d130d40fe2eca658045a6f1778e
languageName: node
linkType: hard
"jose@npm:^6.1.0":
version: 6.1.0
resolution: "jose@npm:6.1.0"
@@ -10110,8 +10119,8 @@ __metadata:
linkType: hard
"livekit-client@npm:^2.13.0":
version: 2.15.13
resolution: "livekit-client@npm:2.15.13"
version: 2.16.0
resolution: "livekit-client@npm:2.16.0"
dependencies:
"@livekit/mutex": "npm:1.1.1"
"@livekit/protocol": "npm:1.42.2"
@@ -10125,7 +10134,7 @@ __metadata:
webrtc-adapter: "npm:^9.0.1"
peerDependencies:
"@types/dom-mediacapture-record": ^1
checksum: 10c0/5a061df9000461a6d40ef8aa1e72e8aedc640181cc57fe6f2c48c5c7f90ce96a735b125aede377fc43f4692a685e098f17eeae0f42c5b2fed473305867bf2789
checksum: 10c0/5d03adc5d09efde343ab894db397529dff26117598e773b23a5df90a4fb166bde12c6bb1f2cfd1d28dbaf93fe9f275026d7abb75f2ffd2ba816393a2d58e6c7e
languageName: node
linkType: hard
@@ -10343,9 +10352,9 @@ __metadata:
languageName: node
linkType: hard
"matrix-js-sdk@github:matrix-org/matrix-js-sdk#head=toger5/sticky-events&commit=e7f5bec51b6f70501a025b79fe5021c933385b21":
version: 38.4.0
resolution: "matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk.git#commit=e7f5bec51b6f70501a025b79fe5021c933385b21"
"matrix-js-sdk@npm:^39.2.0":
version: 39.2.0
resolution: "matrix-js-sdk@npm:39.2.0"
dependencies:
"@babel/runtime": "npm:^7.12.5"
"@matrix-org/matrix-sdk-crypto-wasm": "npm:^15.3.0"
@@ -10355,23 +10364,23 @@ __metadata:
jwt-decode: "npm:^4.0.0"
loglevel: "npm:^1.9.2"
matrix-events-sdk: "npm:0.0.1"
matrix-widget-api: "npm:^1.10.0"
matrix-widget-api: "npm:^1.14.0"
oidc-client-ts: "npm:^3.0.1"
p-retry: "npm:7"
sdp-transform: "npm:^2.14.1"
sdp-transform: "npm:^3.0.0"
unhomoglyph: "npm:^1.0.6"
uuid: "npm:13"
checksum: 10c0/7adffdc183affd2d3ee1e8497cad6ca7904a37f98328ff7bc15aa6c1829dc9f9a92f8e1bd6260432a33626ff2a839644de938270163e73438b7294675cd954e4
checksum: 10c0/f8b5261de2744305330ba3952821ca9303698170bfd3a0ff8a767b9286d4e8d4ed5aaf6fbaf8a1e8ff9dbd859102a2a47d882787e2da3b3078965bec00157959
languageName: node
linkType: hard
"matrix-widget-api@npm:^1.10.0, matrix-widget-api@npm:^1.13.0":
version: 1.13.1
resolution: "matrix-widget-api@npm:1.13.1"
"matrix-widget-api@npm:^1.14.0":
version: 1.15.0
resolution: "matrix-widget-api@npm:1.15.0"
dependencies:
"@types/events": "npm:^3.0.0"
events: "npm:^3.2.0"
checksum: 10c0/25ded744922755b3eb65f4e171cf6cff1a2e0fe43fc3fecbb13e565e41d8af066daa817dd2c3c7d921b996af399eec3b23df70ab7b682cf422d9cee7ca202512
checksum: 10c0/1c08b5284cd98aed312d95594335e1391d937dfad70ef862a1f90fdbaaa27709e1c44dcda37f8045e4814779d8d5816d240aee396d52cfd9b37fbf243a6baf6a
languageName: node
linkType: hard
@@ -12544,7 +12553,7 @@ __metadata:
languageName: node
linkType: hard
"sdp-transform@npm:^2.14.1, sdp-transform@npm:^2.15.0":
"sdp-transform@npm:^2.15.0":
version: 2.15.0
resolution: "sdp-transform@npm:2.15.0"
bin:
@@ -12553,6 +12562,15 @@ __metadata:
languageName: node
linkType: hard
"sdp-transform@npm:^3.0.0":
version: 3.0.0
resolution: "sdp-transform@npm:3.0.0"
bin:
sdp-verify: checker.js
checksum: 10c0/828a4595041ba64c86b29075aa4007ab384519b1fa29882db59ccb83b54b2b2a33b60848293f8da537fe151c52f5844fc17c8325396cac309fb19e2e81ec5bf4
languageName: node
linkType: hard
"sdp@npm:^3.2.0":
version: 3.2.0
resolution: "sdp@npm:3.2.0"
@@ -13603,10 +13621,10 @@ __metadata:
languageName: node
linkType: hard
"undici-types@npm:~6.21.0":
version: 6.21.0
resolution: "undici-types@npm:6.21.0"
checksum: 10c0/c01ed51829b10aa72fc3ce64b747f8e74ae9b60eafa19a7b46ef624403508a54c526ffab06a14a26b3120d055e1104d7abe7c9017e83ced038ea5cf52f8d5e04
"undici-types@npm:~7.16.0":
version: 7.16.0
resolution: "undici-types@npm:7.16.0"
checksum: 10c0/3033e2f2b5c9f1504bdc5934646cb54e37ecaca0f9249c983f7b1fc2e87c6d18399ebb05dc7fd5419e02b2e915f734d872a65da2e3eeed1813951c427d33cc9a
languageName: node
linkType: hard