Compare commits

...

91 Commits

Author SHA1 Message Date
Valere Fedronic
673ca21575 Fix connection leaks: Ensure that any pending connection open are cancelled/undo when ActiveCall is unmounted (#3255)
* Better logs for connection/component lifecycle

* fix: `AudioCaptureOptions` was causing un-necessary effect render

AudioCaptureOptions was a different object but with same internal values, use directly deviceId so that Object.is works properly

* fix: Livekit openned connection leaks

* review: rename to AbortHandles

* review: rename variable

---------

Co-authored-by: Timo <toger5@hotmail.de>
2025-05-14 18:44:30 +02:00
Robin
a4a6a16482 Merge pull request #2812 from element-hq/toger5/track-processor-blur
Add background blur feature for supported devices
2025-04-29 22:25:02 +02:00
Robin
eeb3a1a991 Add license notices to inlined type declarations 2025-04-29 22:19:00 +02:00
Robin
c0c498aa04 Merge branch 'livekit' into toger5/track-processor-blur 2025-04-29 13:02:23 +02:00
Robin
9f5eae0ac1 Clean up a string 2025-04-29 13:02:02 +02:00
Robin
f92c683bc1 Merge pull request #3237 from element-hq/robin/fix-playwright-invite
Fix Playwright tests in Firefox
2025-04-29 13:01:45 +02:00
Robin
dfb9eb4b1f Upgrade Playwright to fix Element's unsupported browser warning 2025-04-29 12:41:23 +02:00
Robin
fa0af6403e Fix Playwright tests in Firefox
As explained by the comment.
2025-04-29 12:25:35 +02:00
Robin
9df0f58117 Inline a patched version of WebCodecs types 2025-04-29 12:07:16 +02:00
Robin
0b82dcf701 Upgrade TypeScript for ImageBufferSource support 2025-04-28 18:10:58 +02:00
Robin
8f1761442b Fix tests 2025-04-28 16:37:53 +02:00
Robin
7d0117903e Don't lie about the type of the video track 2025-04-28 16:01:34 +02:00
Robin
cd9035b5fc Make background blurring actually react to track restarts 2025-04-28 16:00:03 +02:00
Robin
02ce1f701a Document the background transformer a bit better 2025-04-28 15:59:32 +02:00
Robin
e88e0b1901 Import the segmenter model in a more statically-analyzable way 2025-04-28 15:58:50 +02:00
Robin
c91cc79162 Remove unneeded Knip override for MediaPipe 2025-04-28 15:57:29 +02:00
fkwp
edd8373649 Merge pull request #3236 from element-hq/fkwp/fix_nginx_syntax_highlighting
Fix syntax highlighting of Nginx snippets
2025-04-25 18:00:03 +02:00
Robin
31032994a5 Merge branch 'livekit' into toger5/track-processor-blur 2025-04-25 17:23:20 +02:00
fkwp
984383b60d typo 2025-04-25 00:32:19 +02:00
fkwp
de4875ac7c replace jsonc with nginx highlighting 2025-04-25 00:31:13 +02:00
Robin
dc5d1369bf Merge pull request #3229 from element-hq/robin/upgrade-livekit
Return to latest livekit-client version (2.11.3)
2025-04-24 11:35:29 +02:00
Robin
a6696c2304 Return to latest livekit-client version (2.11.3)
This latest release of livekit-client fixes the regression observed in https://github.com/element-hq/element-call/issues/3219.
2025-04-23 11:59:13 +02:00
Timo
a1d6eef12a Set ratchetWindowSize to > 0 enable rachet compatibility. (#3224) 2025-04-23 09:22:54 +02:00
Timo
78a41c09de Fix MatrixRTC js-sdk logs in rageshakes (#3226) 2025-04-22 21:24:09 +02:00
Robin
3fc4cc908e Cleanup + use a better check for blurring support 2025-04-22 19:20:43 +02:00
Robin
63ad4a3ad3 Get blurring working again 2025-04-22 18:32:01 +02:00
Robin
7142122a74 Dedupe Mediapipe dependencies 2025-04-22 18:05:31 +02:00
Robin
b7b3fde202 Upgrade LiveKit track processors 2025-04-22 18:00:41 +02:00
Robin
8fd99ed23e Merge pull request #3222 from element-hq/robin/revert-livekit
Revert livekit-client to 2.10.0
2025-04-22 17:21:45 +02:00
Robin
e751248fee Revert livekit-client to 2.10.0
Closes https://github.com/element-hq/element-call/issues/3219.
2025-04-22 16:31:44 +02:00
Robin
3243afd061 Merge branch 'livekit' into toger5/track-processor-blur 2025-04-18 12:32:03 +02:00
fkwp
733279f010 Merge pull request #3215 from element-hq/fkwp/add_information_about_synapse_listeners
add information about required synapse listeners
2025-04-17 19:06:59 +02:00
fkwp
a5036ee291 Merge branch 'livekit' into fkwp/add_information_about_synapse_listeners 2025-04-17 18:58:09 +02:00
fkwp
2cf2e090f6 Update docs/self-hosting.md
Co-authored-by: Robin <robin@robin.town>
2025-04-17 16:31:52 +02:00
Timo
78a88a9e0b Add a .links.temp-disabled.yaml state explicitly for when the pre-commit hook disabled the yaml file. otherwise we end up with a linked repo after every commit. (#3210) 2025-04-16 19:39:08 +02:00
fkwp
4694354f69 add informatino about required synapse listeners 2025-04-15 17:53:27 +02:00
Timo
a839cde34f Add PR- label requirement action (#3171)
* Add PR- label requirement action

* also add dependencies

* review
2025-04-15 12:10:48 +02:00
Hugh Nimmo-Smith
8b63813e51 Improve tagging of Release versions of platform specific embedded packages (#3203)
* Refactor version calculation to use separate step and track version for each platform

* Set tag for NPM and SNAPSHOT for AAR appropriately

* Log output versions for each platform

* Fix tarball name

* Just use npm tag of `latest` or `other`

* Attempt to fix AAR snapshot publishing
2025-04-15 11:41:47 +02:00
Timo
e63aff3e9b Bump js-sdk to develop branch (#3205) 2025-04-15 10:05:00 +02:00
Timo
d5407a4fde Update developer key transport setting description (#3194) 2025-04-14 09:56:53 +02:00
fkwp
b813a3c95b Merge pull request #3193 from element-hq/renovate/npm-vite-vulnerability
Update dependency vite to v6.2.6 [SECURITY]
2025-04-11 20:21:00 +02:00
fkwp
0f69ace92a Merge pull request #3187 from element-hq/toger5/packageName-to-depName
Update renovate.json
2025-04-11 20:19:13 +02:00
Hugh Nimmo-Smith
f5615f2e26 Set the correct license on docker images (#3197) 2025-04-11 19:01:04 +02:00
fkwp
cb64c311c5 Merge pull request #3196 from element-hq/hughns/raw-docker-version
Explicitly set docker image version for releases
2025-04-11 18:32:44 +02:00
Hugh Nimmo-Smith
ee45e58903 Meh 2025-04-11 17:23:56 +01:00
Hugh Nimmo-Smith
4b247540ef value!=pattern 2025-04-11 17:16:47 +01:00
Hugh Nimmo-Smith
fc281c516c Fix error
> Unrecognized named-value: 'env'. Located at position 1 within expression: env.VERSION
2025-04-11 17:12:58 +01:00
Hugh Nimmo-Smith
8858e4003a Explicitly set docker image version for releases
Otherwise we end up with inconsistency across RCs and Releases.
2025-04-11 17:08:08 +01:00
renovate[bot]
58e542e15d Update dependency vite to v6.2.6 [SECURITY] 2025-04-11 15:06:47 +00:00
Timo
13f12779dc default for yarn updates 2025-04-11 12:39:51 +02:00
Timo
c305fb8604 use matchFileNames 2025-04-11 12:37:35 +02:00
Timo
054d6a9e48 Update renvovate.json 2025-04-11 12:26:25 +02:00
Timo
6884b19a73 package and lock file 2025-04-05 00:01:56 +02:00
Timo
75933a508c Merge branch 'livekit' into toger5/track-processor-blur 2025-04-05 00:00:00 +02:00
Hugh Nimmo-Smith
00d724a3a9 Bump @livekit/track-processors to ignore empty frames 2025-01-08 14:27:19 +00:00
Hugh Nimmo-Smith
4c4d01211c Fix @matrix-org/matrix-sdk-crypto-wasm version 2025-01-08 14:20:12 +00:00
Hugh Nimmo-Smith
ba2e7a7462 Merge branch 'livekit' into toger5/track-processor-blur 2025-01-08 14:08:49 +00:00
Hugh Nimmo-Smith
38110749cd Merge branch 'livekit' into toger5/track-processor-blur 2025-01-06 17:21:30 +00:00
Hugh Nimmo-Smith
96ea920e98 Try reenabling local implementation of BlurBackgroundTransformer 2025-01-06 15:53:31 +00:00
Hugh Nimmo-Smith
bb0e1f38dc Exclude tasks-vision from knip 2025-01-06 15:48:43 +00:00
Hugh Nimmo-Smith
77ff3583b0 Merge branch 'livekit' into toger5/track-processor-blur 2025-01-06 14:48:54 +00:00
Timo
e0f4bf8eab working version 2024-12-19 17:39:34 +01:00
Timo
7c985b0d9c Revert "Bump matrix-js-sdk for bundle improvements (#2911)"
This reverts commit 6b8dddfaaa.
2024-12-19 17:26:14 +01:00
Hugh Nimmo-Smith
7983adb98e Lint 2024-12-18 09:42:52 +00:00
Hugh Nimmo-Smith
6accf088bf Merge branch 'livekit' into toger5/track-processor-blur 2024-12-18 09:41:38 +00:00
Timo
3f48810034 bump components-js (not yet working) 2024-12-13 09:57:55 +01:00
Timo
843f081d78 Merge branch 'livekit' into toger5/track-processor-blur 2024-12-13 03:37:48 +01:00
Hugh Nimmo-Smith
f18ce5adf1 Format 2024-12-11 10:34:24 +00:00
Hugh Nimmo-Smith
c860be73f6 . 2024-12-11 10:30:45 +00:00
Hugh Nimmo-Smith
5ead61d406 Merge branch 'livekit' into toger5/track-processor-blur 2024-12-11 10:29:49 +00:00
Hugh Nimmo-Smith
b6cc9c534b Lint 2024-12-09 22:12:49 +00:00
Hugh Nimmo-Smith
aa5cf2f591 Wording change 2024-12-09 22:11:30 +00:00
Hugh Nimmo-Smith
18606462bb Lint 2024-12-09 22:09:26 +00:00
Hugh Nimmo-Smith
59bc73cedf Use local assets for blurring 2024-12-09 22:09:16 +00:00
Timo
b77c4afff2 refactor
- still should not work without a fixing upstream LK:
https://github.com/livekit/components-js/pull/1042
https://github.com/livekit/components-js/pull/1043
2024-12-09 22:09:16 +00:00
Timo
574c89529a disable blur in local track again 2024-12-09 22:08:51 +00:00
Timo
c6d48446f4 review 2024-12-09 22:08:51 +00:00
Timo
95c3ca8859 review 2024-12-09 22:08:51 +00:00
Timo
d179db1e8f more detailed error message 2024-12-09 22:08:51 +00:00
Timo
9eae919050 make default false 2024-12-09 22:08:51 +00:00
Timo
7d8e9a7969 fixes and gpu based blurring 2024-12-09 22:08:51 +00:00
Timo
c2b6518027 dont update if processor already set 2024-12-09 22:08:51 +00:00
Timo
a6745ef34a properly add @livekit/track-processors 2024-12-09 22:08:51 +00:00
Timo
ae71c6e679 add @types/dom-mediacapture-transform for the remaining type errors 2024-12-09 22:08:51 +00:00
Timo
aade8f94a6 eslint + prettier 2024-12-09 22:08:51 +00:00
Timo
0be312a5e3 i18n 2024-12-09 22:08:51 +00:00
Timo
95bc7db28a update blur checkbox (disabled state) 2024-12-09 22:08:51 +00:00
Timo
d23beefb60 remove unused dependencies. 2024-12-09 22:08:51 +00:00
Timo
70ff435a8c add types 2024-12-09 22:08:51 +00:00
Timo
140fe484d7 make compatible with unsopported browsers 2024-12-09 22:08:51 +00:00
Timo
c798c79f41 Add background Blur 2024-12-09 22:08:51 +00:00
33 changed files with 1723 additions and 194 deletions

View File

@@ -1,10 +1,10 @@
#!/usr/bin/sh
FILE=.links.disabled.yaml
FILE=.links.temp-disabled.yaml
if test -f "$FILE"; then
# echo "$FILE exists. -> moving to .links.disabled.yaml"
mv .links.disabled.yaml .links.yaml
# echo "running yarn"
# Only do the post-commit hook if the file was temp-disabled by the pre-commit hook.
# Otherwise linking was actively (`yarn links:disable`) disabled and this hook should noop.
mv .links.temp-disabled.yaml .links.yaml
yarnLog=$(yarn)
echo "[yarn-linker] The post-commit hook has re-enabled .links.yaml."
exit 1

View File

@@ -2,8 +2,7 @@
FILE=".links.yaml"
if test -f "$FILE"; then
# echo "$FILE exists. -> moving to .links.disabled.yaml"
mv .links.yaml .links.disabled.yaml
mv .links.yaml .links.temp-disabled.yaml
# echo "running yarn"
x=$(yarn)
y=$(git add yarn.lock)

View File

@@ -46,6 +46,8 @@ jobs:
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: ${{ inputs.docker_tags}}
labels: |
org.opencontainers.image.licenses=AGPL-3.0-only OR LicenseRef-Element-Commercial
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0

14
.github/workflows/changelog-label.yml vendored Normal file
View File

@@ -0,0 +1,14 @@
name: PR changelog label
on:
pull_request_target:
types: [labeled, unlabeled, opened]
jobs:
pr-changelog-label:
runs-on: ubuntu-latest
steps:
- uses: yogevbd/enforce-label-action@2.1.0
with:
REQUIRED_LABELS_ANY: "PR-Bug-Fix,PR-Documentation,PR-Task,PR-Feature,PR-Improvement,PR-Developer-Experience,dependencies"
REQUIRED_LABELS_ANY_DESCRIPTION: "Select at least one 'PR-' label"
BANNED_LABELS: "banned"

View File

@@ -11,19 +11,48 @@ on:
push:
branches: [livekit]
env:
# We perform a dry run for all events except releases.
# This is to help make sure that we notice if the packaging process has become
# broken ahead of a release.
DRY_RUN: ${{ github.event_name != 'release' }}
# We should only use the hard coded test value for a dry run
VERSION: ${{ github.event_name == 'release' && github.event.release.tag_name || 'v0.0.0-pre.0' }}
jobs:
versioning:
name: Versioning
runs-on: ubuntu-latest
outputs:
DRY_RUN: ${{ steps.dry_run.outputs.DRY_RUN }}
PREFIXED_VERSION: ${{ steps.prefixed_version.outputs.PREFIXED_VERSION }}
UNPREFIXED_VERSION: ${{ steps.unprefixed_version.outputs.UNPREFIXED_VERSION }}
TAG: ${{ steps.tag.outputs.TAG }}
steps:
- name: Calculate VERSION
# We should only use the hard coded test value for a dry run
run: echo "VERSION=${{ github.event_name == 'release' && github.event.release.tag_name || 'v0.0.0-pre.0' }}" >> "$GITHUB_ENV"
- id: dry_run
name: Set DRY_RUN
# We perform a dry run for all events except releases.
# This is to help make sure that we notice if the packaging process has become
# broken ahead of a release.
run: echo "DRY_RUN=${{ github.event_name != 'release' }}" >> "$GITHUB_OUTPUT"
- id: prefixed_version
name: Set PREFIXED_VERSION
run: echo "PREFIXED_VERSION=${VERSION}" >> "$GITHUB_OUTPUT"
- id: unprefixed_version
name: Set UNPREFIXED_VERSION
# This just strips the leading character
run: echo "UNPREFIXED_VERSION=${VERSION:1}" >> "$GITHUB_OUTPUT"
- id: tag
# latest = a proper release
# other = anything else
name: Set tag
run: |
if [[ "${VERSION}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "TAG=latest" >> "$GITHUB_OUTPUT"
else
echo "TAG=other" >> "$GITHUB_OUTPUT"
fi
build_element_call:
needs: versioning
uses: ./.github/workflows/build-element-call.yaml
with:
vite_app_version: embedded-${{ github.event.release.tag_name || 'v0.0.0-pre.0' }} # Using ${{ env.VERSION }} here doesn't work
vite_app_version: embedded-${{ needs.versioning.outputs.PREFIXED_VERSION }}
package: embedded
secrets:
SENTRY_ORG: ${{ secrets.SENTRY_ORG }}
@@ -32,7 +61,7 @@ jobs:
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
publish_tarball:
needs: build_element_call
needs: [build_element_call, versioning]
if: always()
name: Publish tarball
runs-on: ubuntu-latest
@@ -40,7 +69,7 @@ jobs:
contents: write # required to upload release asset
steps:
- name: Determine filename
run: echo "FILENAME_PREFIX=element-call-embedded-${VERSION:1}" >> "$GITHUB_ENV"
run: echo "FILENAME_PREFIX=element-call-embedded-${{ needs.versioning.outputs.UNPREFIXED_VERSION }}" >> "$GITHUB_ENV"
- name: 📥 Download built element-call artifact
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4
with:
@@ -53,7 +82,7 @@ jobs:
- name: Create Checksum
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
- name: Upload
if: ${{ env.DRY_RUN == 'false' }}
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@c95fe1489396fe8a9eb87c0abf8aa5b2ef267fda # v2
with:
files: |
@@ -61,10 +90,12 @@ jobs:
${{ env.FILENAME_PREFIX }}.sha256
publish_npm:
needs: build_element_call
needs: [build_element_call, versioning]
if: always()
name: Publish NPM
runs-on: ubuntu-latest
outputs:
ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
permissions:
contents: read
id-token: write # required for the provenance flag on npm publish
@@ -90,17 +121,23 @@ jobs:
- name: Publish npm
working-directory: embedded/web
run: |
npm version ${{ env.VERSION }} --no-git-tag-version
npm version ${{ needs.versioning.outputs.PREFIXED_VERSION }} --no-git-tag-version
echo "ARTIFACT_VERSION=$(jq '.version' --raw-output package.json)" >> "$GITHUB_ENV"
npm publish --provenance --access public ${{ env.DRY_RUN == 'true' && '--dry-run' || '' }}
npm publish --provenance --access public --tag ${{ needs.versioning.outputs.TAG }} ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_RELEASE_TOKEN }}
- id: artifact_version
name: Output artifact version
run: echo "ARTIFACT_VERSION=${{env.ARTIFACT_VERSION}}" >> "$GITHUB_OUTPUT"
publish_android:
needs: build_element_call
needs: [build_element_call, versioning]
if: always()
name: Publish Android AAR
runs-on: ubuntu-latest
outputs:
ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
permissions:
contents: read
steps:
@@ -122,7 +159,13 @@ jobs:
java-version: "17"
- name: Get artifact version
run: echo "ARTIFACT_VERSION=${VERSION:1}" >> "$GITHUB_ENV"
# Anything that is not a final release will be tagged as a snapshot
run: |
if [[ "${{ needs.versioning.outputs.TAG }}" == "latest" ]]; then
echo "ARTIFACT_VERSION=${{ needs.versioning.outputs.UNPREFIXED_VERSION }}" >> "$GITHUB_ENV"
else
echo "ARTIFACT_VERSION=${{ needs.versioning.outputs.UNPREFIXED_VERSION }}-SNAPSHOT" >> "$GITHUB_ENV"
fi
- name: Set version string
run: sed -i "s/0.0.0/${{ env.ARTIFACT_VERSION }}/g" embedded/android/lib/src/main/kotlin/io/element/android/call/embedded/Version.kt
@@ -135,13 +178,19 @@ jobs:
ORG_GRADLE_PROJECT_mavenCentralPassword: ${{ secrets.MAVEN_RELEASE_PASSWORD }}
ORG_GRADLE_PROJECT_signingInMemoryKey: ${{ secrets.GPG_SIGNING_KEY }}
ORG_GRADLE_PROJECT_signingInMemoryKeyPassword: ${{ secrets.GPG_SIGNING_KEY_PASSWORD }}
run: ./gradlew publishAndReleaseToMavenCentral --no-daemon ${{ env.DRY_RUN == 'true' && '--dry-run' || '' }}
run: ./gradlew publishToMavenCentral --no-daemon ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
- id: artifact_version
name: Output artifact version
run: echo "ARTIFACT_VERSION=${{env.ARTIFACT_VERSION}}" >> "$GITHUB_OUTPUT"
publish_ios:
needs: build_element_call
needs: [build_element_call, versioning]
if: always()
name: Publish SwiftPM Library
runs-on: ubuntu-latest
outputs:
ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
permissions:
contents: read
steps:
@@ -169,7 +218,7 @@ jobs:
run: rsync -a --delete --exclude .git element-call/embedded/ios/ element-call-swift
- name: Get artifact version
run: echo "ARTIFACT_VERSION=${VERSION:1}" >> "$GITHUB_ENV"
run: echo "ARTIFACT_VERSION=${{ needs.versioning.outputs.UNPREFIXED_VERSION }}" >> "$GITHUB_ENV"
- name: Set version string
run: sed -i "s/0.0.0/${{ env.ARTIFACT_VERSION }}/g" element-call-swift/Sources/EmbeddedElementCall/EmbeddedElementCall.swift
@@ -184,27 +233,33 @@ jobs:
git config --global user.email "ci@element.io"
git config --global user.name "Element CI"
git add -A
git commit -am "Release ${{ env.VERSION }}"
git commit -am "Release ${{ needs.versioning.outputs.PREFIXED_VERSION }}"
git tag -a ${{ env.ARTIFACT_VERSION }} -m "${{ github.event.release.html_url }}"
- name: Push
working-directory: element-call-swift
run: |
git push --tags ${{ env.DRY_RUN == 'true' && '--dry-run' || '' }}
git push --tags ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
- id: artifact_version
name: Output artifact version
run: echo "ARTIFACT_VERSION=${{env.ARTIFACT_VERSION}}" >> "$GITHUB_OUTPUT"
release_notes:
needs: [publish_npm, publish_android, publish_ios]
needs: [versioning, publish_npm, publish_android, publish_ios]
if: always()
name: Update release notes
runs-on: ubuntu-latest
permissions:
contents: write # to update release notes
steps:
- name: Get artifact version
run: echo "ARTIFACT_VERSION=${VERSION:1}" >> "$GITHUB_ENV"
- name: Log versions
run: |
echo "NPM: ${{ needs.publish_npm.outputs.ARTIFACT_VERSION }}"
echo "Android: ${{ needs.publish_android.outputs.ARTIFACT_VERSION }}"
echo "iOS: ${{ needs.publish_ios.outputs.ARTIFACT_VERSION }}"
- name: Add release notes
if: ${{ env.DRY_RUN == 'false' }}
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@c95fe1489396fe8a9eb87c0abf8aa5b2ef267fda # v2
with:
append_body: true
@@ -218,19 +273,19 @@ jobs:
### NPM
```
npm install @element-hq/element-call-embedded@${{ env.ARTIFACT_VERSION }}
npm install @element-hq/element-call-embedded@${{ needs.publish_npm.outputs.ARTIFACT_VERSION }}
```
### Android AAR
```
dependencies {
implementation 'io.element.android:element-call-embedded:${{ env.ARTIFACT_VERSION }}'
implementation 'io.element.android:element-call-embedded:${{ needs.publish_android.outputs.ARTIFACT_VERSION }}'
}
```
### SwiftPM
```
.package(url: "https://github.com/element-hq/element-call-swift.git", from: "${{ env.ARTIFACT_VERSION }}")
.package(url: "https://github.com/element-hq/element-call-swift.git", from: "${{ needs.publish_ios.outputs.ARTIFACT_VERSION }}")
```

View File

@@ -60,7 +60,8 @@ jobs:
artifact_run_id: ${{ github.event.workflow_run.id || github.run_id }}
docker_tags: |
type=sha,format=short,event=branch
type=semver,pattern=v{{version}}
type=raw,value=${{ github.event.release.tag_name }}
# Like before, using ${{ env.VERSION }} above doesn't work
add_docker_release_note:
needs: publish_docker
name: Add docker release note

1
.gitignore vendored
View File

@@ -21,6 +21,7 @@ yarn-error.log
!/.yarn/versions
/.links.yaml
/.links.disabled.yaml
/.links.temp-disabled.yaml
# Playwright
/test-results/

View File

@@ -63,6 +63,11 @@ rc_delayed_event_mgmt:
burst_count: 20
```
As a prerequisite for the
[Matrix LiveKit JWT auth service](https://github.com/element-hq/lk-jwt-service)
make sure that your Synapse server has either a `federation` or `openid`
[listener configured](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#listeners).
### MatrixRTC Backend
In order to **guarantee smooth operation** of Element Call MatrixRTC backend is
@@ -88,7 +93,7 @@ the example above, this results in:
Using Nginx, you can achieve this by:
```jsonc
```nginx configuration file
server {
...
location ^~ /livekit/jwt/ {
@@ -198,7 +203,7 @@ Because Element Call uses client-side routing, your server must be able to route
any requests to non-existing paths back to `/index.html`. For example, in Nginx
you can achieve this with the `try_files` directive:
```jsonc
```nginx configuration file
server {
...
location / {

View File

@@ -27,7 +27,7 @@ android {
}
mavenPublishing {
publishToMavenCentral(SonatypeHost.S01)
publishToMavenCentral(SonatypeHost.S01, automaticRelease = true)
signAllPublications()

View File

@@ -74,7 +74,7 @@
"show_non_member_tiles": "Show tiles for non-member media",
"url_params": "URL parameters",
"use_new_membership_manager": "Use the new implementation of the call MembershipManager",
"use_to_device_key_transport": "Use to device messages to distribute keys for matrixRTC media"
"use_to_device_key_transport": "Use to device key transport. This will fallback to room key transport when another call member sent a room key"
},
"disconnected_banner": "Connectivity to the server has been lost.",
"error": {
@@ -164,6 +164,9 @@
"effect_volume_description": "Adjust the volume at which reactions and hand raised effects play.",
"effect_volume_label": "Sound effect volume"
},
"background_blur_header": "Background",
"background_blur_label": "Blur the background of the video",
"blur_not_supported_by_browser": "(Background blur is not supported by this device.)",
"developer_tab_title": "Developer",
"devices": {
"camera": "Camera",

View File

@@ -40,6 +40,8 @@
"@livekit/components-core": "^0.12.0",
"@livekit/components-react": "^2.0.0",
"@livekit/protocol": "^1.33.0",
"@livekit/track-processors": "^0.5.5",
"@mediapipe/tasks-vision": "^0.10.18",
"@opentelemetry/api": "^1.4.0",
"@opentelemetry/core": "^1.25.1",
"@opentelemetry/exporter-trace-otlp-http": "^0.57.0",
@@ -47,7 +49,7 @@
"@opentelemetry/sdk-trace-base": "^1.25.1",
"@opentelemetry/sdk-trace-web": "^1.9.1",
"@opentelemetry/semantic-conventions": "^1.25.1",
"@playwright/test": "^1.51.0",
"@playwright/test": "^1.52.0",
"@radix-ui/react-dialog": "^1.0.4",
"@radix-ui/react-slider": "^1.1.2",
"@radix-ui/react-visually-hidden": "^1.0.3",
@@ -70,8 +72,8 @@
"@types/react-dom": "^18.3.0",
"@types/sdp-transform": "^2.4.5",
"@types/uuid": "10",
"@typescript-eslint/eslint-plugin": "^8.0.0",
"@typescript-eslint/parser": "^8.0.0",
"@typescript-eslint/eslint-plugin": "^8.31.0",
"@typescript-eslint/parser": "^8.31.0",
"@use-gesture/react": "^10.2.11",
"@vector-im/compound-design-tokens": "^3.0.0",
"@vector-im/compound-web": "^7.2.0",
@@ -97,10 +99,10 @@
"i18next-parser": "^9.1.0",
"jsdom": "^26.0.0",
"knip": "^5.27.2",
"livekit-client": "2.11.2",
"livekit-client": "^2.11.3",
"lodash-es": "^4.17.21",
"loglevel": "^1.9.1",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#445981f1bc1e13f80b3061fb26d6f42f49161c24",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#19b1b901f575755d29d1fe03ca48cbf7c1cae05c",
"matrix-widget-api": "1.11.0",
"normalize.css": "^8.0.1",
"observable-hooks": "^4.2.3",
@@ -118,7 +120,7 @@
"react-use-measure": "^2.1.1",
"rxjs": "^7.8.1",
"sass": "^1.42.1",
"typescript": "^5.1.6",
"typescript": "^5.8.3",
"typescript-eslint-language-service": "^5.0.5",
"unique-names-generator": "^4.6.0",
"vaul": "^1.0.0",
@@ -131,6 +133,7 @@
},
"resolutions": {
"@livekit/components-core/rxjs": "^7.8.1",
"@livekit/track-processors/@mediapipe/tasks-vision": "^0.10.18",
"matrix-widget-api": "1.11.0"
},
"packageManager": "yarn@4.7.0"

View File

@@ -154,8 +154,13 @@ export const widgetTest = test.extend<MyFixtures>({
ewPage1.getByRole("heading", { name: "Invite to Welcome Room" }),
).toBeVisible();
await ewPage1.getByRole("textbox").fill(whistlerMxId);
await ewPage1.getByRole("textbox").click();
// To get the invite textbox we need to specifically select within the
// dialog, since there is another textbox in the background (the message
// composer). In theory the composer shouldn't be visible to Playwright at
// all because the invite dialog has trapped focus, but the focus trap
// doesn't quite work right on Firefox.
await ewPage1.getByRole("dialog").getByRole("textbox").fill(whistlerMxId);
await ewPage1.getByRole("dialog").getByRole("textbox").click();
await ewPage1.getByRole("button", { name: "Invite" }).click();
// Accept the invite

View File

@@ -50,18 +50,12 @@
"prHeader": "Please review modals on mobile for visual regressions."
},
{
"groupName": "bundled element-call dependencies",
"matchPackageNames": [
"com.vanniktech.maven.publish",
"com.android.library",
"com.android.tools.build",
"gradle"
]
"groupName": "embedded package dependencies",
"matchFileNames": ["embedded/**/*"]
},
{
"groupName": "Yarn",
"matchPackageNames": ["yarn"],
"extends": ["schedule:weekly"]
"matchDepNames": ["yarn"]
}
],
"semanticCommits": "disabled",

View File

@@ -0,0 +1,146 @@
/* eslint-disable */
// The contents of this file below the line are copied from
// @types/dom-mediacapture-transform, which is inlined here into Element Call so
// that we can apply the patch to @types/dom-webcodecs found in
// ./dom-webcodecs.d.ts, which it depends on.
// (https://github.com/DefinitelyTyped/DefinitelyTyped/pull/72625)
// Once that PR is merged and released, we can remove this file and return to
// depending on @types/dom-mediacapture-transform.
// -----------------------------------------------------------------------------
// This project is licensed under the MIT license.
// Copyrights are respective of each contributor listed at the beginning of each definition file.
// Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
// In general, these types are only available behind a command line flag or an origin trial in
// Chrome 90+.
// This API depends on WebCodecs.
// Versioning:
// Until the above-mentioned spec is finalized, the major version number is 0. Although not
// necessary for version 0, consider incrementing the minor version number for breaking changes.
// The following modify existing DOM types to allow defining type-safe APIs on audio and video tracks.
/** Specialize MediaStreamTrack so that we can refer specifically to an audio track. */
interface MediaStreamAudioTrack extends MediaStreamTrack {
readonly kind: "audio";
clone(): MediaStreamAudioTrack;
}
/** Specialize MediaStreamTrack so that we can refer specifically to a video track. */
interface MediaStreamVideoTrack extends MediaStreamTrack {
readonly kind: "video";
clone(): MediaStreamVideoTrack;
}
/** Assert that getAudioTracks and getVideoTracks return the tracks with the appropriate kind. */
interface MediaStream {
getAudioTracks(): MediaStreamAudioTrack[];
getVideoTracks(): MediaStreamVideoTrack[];
}
// The following were originally generated from the spec using
// https://github.com/microsoft/TypeScript-DOM-lib-generator, then heavily modified.
/**
* A track sink that is capable of exposing the unencoded frames from the track to a
* ReadableStream, and exposes a control channel for signals going in the oppposite direction.
*/
interface MediaStreamTrackProcessor<T extends AudioData | VideoFrame> {
/**
* Allows reading the frames flowing through the MediaStreamTrack provided to the constructor.
*/
readonly readable: ReadableStream<T>;
/** Allows sending control signals to the MediaStreamTrack provided to the constructor. */
readonly writableControl: WritableStream<MediaStreamTrackSignal>;
}
declare var MediaStreamTrackProcessor: {
prototype: MediaStreamTrackProcessor<any>;
/** Constructor overrides based on the type of track. */
new (
init: MediaStreamTrackProcessorInit & { track: MediaStreamAudioTrack },
): MediaStreamTrackProcessor<AudioData>;
new (
init: MediaStreamTrackProcessorInit & { track: MediaStreamVideoTrack },
): MediaStreamTrackProcessor<VideoFrame>;
};
interface MediaStreamTrackProcessorInit {
track: MediaStreamTrack;
/**
* If media frames are not read from MediaStreamTrackProcessor.readable quickly enough, the
* MediaStreamTrackProcessor will internally buffer up to maxBufferSize of the frames produced
* by the track. If the internal buffer is full, each time the track produces a new frame, the
* oldest frame in the buffer will be dropped and the new frame will be added to the buffer.
*/
maxBufferSize?: number | undefined;
}
/**
* Takes video frames as input, and emits control signals that result from subsequent processing.
*/
interface MediaStreamTrackGenerator<T extends AudioData | VideoFrame>
extends MediaStreamTrack {
/**
* Allows writing media frames to the MediaStreamTrackGenerator, which is itself a
* MediaStreamTrack. When a frame is written to writable, the frames close() method is
* automatically invoked, so that its internal resources are no longer accessible from
* JavaScript.
*/
readonly writable: WritableStream<T>;
/**
* Allows reading control signals sent from any sinks connected to the
* MediaStreamTrackGenerator.
*/
readonly readableControl: ReadableStream<MediaStreamTrackSignal>;
}
type MediaStreamAudioTrackGenerator = MediaStreamTrackGenerator<AudioData> &
MediaStreamAudioTrack;
type MediaStreamVideoTrackGenerator = MediaStreamTrackGenerator<VideoFrame> &
MediaStreamVideoTrack;
declare var MediaStreamTrackGenerator: {
prototype: MediaStreamTrackGenerator<any>;
/** Constructor overrides based on the type of track. */
new (
init: MediaStreamTrackGeneratorInit & {
kind: "audio";
signalTarget?: MediaStreamAudioTrack | undefined;
},
): MediaStreamAudioTrackGenerator;
new (
init: MediaStreamTrackGeneratorInit & {
kind: "video";
signalTarget?: MediaStreamVideoTrack | undefined;
},
): MediaStreamVideoTrackGenerator;
};
interface MediaStreamTrackGeneratorInit {
kind: MediaStreamTrackGeneratorKind;
/**
* (Optional) track to which the MediaStreamTrackGenerator will automatically forward control
* signals. If signalTarget is provided and signalTarget.kind and kind do not match, the
* MediaStreamTrackGenerators constructor will raise an exception.
*/
signalTarget?: MediaStreamTrack | undefined;
}
type MediaStreamTrackGeneratorKind = "audio" | "video";
type MediaStreamTrackSignalType = "request-frame";
interface MediaStreamTrackSignal {
signalType: MediaStreamTrackSignalType;
}

745
src/@types/dom-webcodecs.d.ts vendored Normal file
View File

@@ -0,0 +1,745 @@
/* eslint-disable */
// The contents of this file below the line are copied from
// @types/dom-webcodecs, which is inlined here into Element Call so that we can
// apply the patch https://github.com/DefinitelyTyped/DefinitelyTyped/pull/72625
// which is needed for TypeScript 5.8 compatibility. Once that PR is merged and
// released, we can remove this file and return to depending on
// @types/dom-webcodecs.
// -----------------------------------------------------------------------------
// This project is licensed under the MIT license.
// Copyrights are respective of each contributor listed at the beginning of each definition file.
// Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
// Versioning:
// Until the WebCodecs spec is finalized, the major version number is 0. I have chosen to use minor
// version 1 to denote the API as defined by the IDL files from the Chromium repo at
// https://chromium.googlesource.com/chromium/src/+/main/third_party/blink/renderer/modules/webcodecs.
// Please use a version number above 0.1 if using the spec at https://w3c.github.io/webcodecs/ as
// the source.
// The declarations in webcodecs.generated.d.ts have been generated using the code in
// https://github.com/yume-chan/webcodecs-lib-generator. See
// https://github.com/yume-chan/webcodecs-lib-generator/blob/main/README.md for more detail.
// The following declarations are copied from
// https://github.com/microsoft/TypeScript-DOM-lib-generator/blob/a75338e1ea8a958bf08a5745141d2ab8f14ba2ca/baselines/dom.generated.d.ts
// and modified to expand the types to include VideoFrame.
/** Shim for OffscreenCanvas, which was removed in TS 4.4 */
interface OffscreenCanvas extends EventTarget {}
/**
* Replaces CanvasImageSource; only applies if WebCodecs is available.
*/
type CanvasImageSourceWebCodecs =
| HTMLOrSVGImageElement
| HTMLVideoElement
| HTMLCanvasElement
| ImageBitmap
| OffscreenCanvas
| VideoFrame;
interface CanvasRenderingContext2D {
drawImage(image: CanvasImageSourceWebCodecs, dx: number, dy: number): void;
drawImage(
image: CanvasImageSourceWebCodecs,
dx: number,
dy: number,
dw: number,
dh: number,
): void;
drawImage(
image: CanvasImageSourceWebCodecs,
sx: number,
sy: number,
sw: number,
sh: number,
dx: number,
dy: number,
dw: number,
dh: number,
): void;
createPattern(
image: CanvasImageSourceWebCodecs,
repetition: string | null,
): CanvasPattern | null;
}
interface OffscreenCanvasRenderingContext2D {
drawImage(image: CanvasImageSourceWebCodecs, dx: number, dy: number): void;
drawImage(
image: CanvasImageSourceWebCodecs,
dx: number,
dy: number,
dw: number,
dh: number,
): void;
drawImage(
image: CanvasImageSourceWebCodecs,
sx: number,
sy: number,
sw: number,
sh: number,
dx: number,
dy: number,
dw: number,
dh: number,
): void;
createPattern(
image: CanvasImageSourceWebCodecs,
repetition: string | null,
): CanvasPattern | null;
}
/**
* Replaces ImageBitmapSource; only applies if WebCodecs is available.
*/
type ImageBitmapSourceWebCodecs = CanvasImageSourceWebCodecs | Blob | ImageData;
declare function createImageBitmap(
image: ImageBitmapSourceWebCodecs,
options?: ImageBitmapOptions,
): Promise<ImageBitmap>;
declare function createImageBitmap(
image: ImageBitmapSourceWebCodecs,
sx: number,
sy: number,
sw: number,
sh: number,
options?: ImageBitmapOptions,
): Promise<ImageBitmap>;
/**
* Replaces TexImageSource; only applies if WebCodecs is available.
*/
type TexImageSourceWebCodecs =
| ImageBitmap
| ImageData
| HTMLImageElement
| HTMLCanvasElement
| HTMLVideoElement
| OffscreenCanvas
| VideoFrame;
interface WebGLRenderingContextOverloads {
texImage2D(
target: GLenum,
level: GLint,
internalformat: GLint,
format: GLenum,
type: GLenum,
source: TexImageSourceWebCodecs,
): void;
texSubImage2D(
target: GLenum,
level: GLint,
xoffset: GLint,
yoffset: GLint,
format: GLenum,
type: GLenum,
source: TexImageSourceWebCodecs,
): void;
}
interface WebGL2RenderingContextBase {
texImage3D(
target: GLenum,
level: GLint,
internalformat: GLint,
width: GLsizei,
height: GLsizei,
depth: GLsizei,
border: GLint,
format: GLenum,
type: GLenum,
source: TexImageSourceWebCodecs,
): void;
texSubImage3D(
target: GLenum,
level: GLint,
xoffset: GLint,
yoffset: GLint,
zoffset: GLint,
width: GLsizei,
height: GLsizei,
depth: GLsizei,
format: GLenum,
type: GLenum,
source: TexImageSourceWebCodecs,
): void;
}
interface WebGL2RenderingContextOverloads {
texImage2D(
target: GLenum,
level: GLint,
internalformat: GLint,
format: GLenum,
type: GLenum,
source: TexImageSourceWebCodecs,
): void;
texImage2D(
target: GLenum,
level: GLint,
internalformat: GLint,
width: GLsizei,
height: GLsizei,
border: GLint,
format: GLenum,
type: GLenum,
source: TexImageSourceWebCodecs,
): void;
texSubImage2D(
target: GLenum,
level: GLint,
xoffset: GLint,
yoffset: GLint,
format: GLenum,
type: GLenum,
source: TexImageSourceWebCodecs,
): void;
texSubImage2D(
target: GLenum,
level: GLint,
xoffset: GLint,
yoffset: GLint,
width: GLsizei,
height: GLsizei,
format: GLenum,
type: GLenum,
source: TexImageSourceWebCodecs,
): void;
}
/////////////////////////////
/// webcodecs APIs
/////////////////////////////
interface AudioDataCopyToOptions {
format?: AudioSampleFormat | undefined;
frameCount?: number | undefined;
frameOffset?: number | undefined;
planeIndex: number;
}
interface AudioDataInit {
data: AllowSharedBufferSource;
format: AudioSampleFormat;
numberOfChannels: number;
numberOfFrames: number;
sampleRate: number;
timestamp: number;
}
interface AudioDecoderConfig {
codec: string;
description?: AllowSharedBufferSource | undefined;
numberOfChannels: number;
sampleRate: number;
}
interface AudioDecoderInit {
error: WebCodecsErrorCallback;
output: AudioDataOutputCallback;
}
interface AudioDecoderSupport {
config?: AudioDecoderConfig;
supported?: boolean;
}
interface AudioEncoderConfig {
bitrate?: number | undefined;
codec: string;
numberOfChannels: number;
sampleRate: number;
}
interface AudioEncoderInit {
error: WebCodecsErrorCallback;
output: EncodedAudioChunkOutputCallback;
}
interface AudioEncoderSupport {
config?: AudioEncoderConfig;
supported?: boolean;
}
interface AvcEncoderConfig {
format?: AvcBitstreamFormat | undefined;
}
interface EncodedAudioChunkInit {
data: AllowSharedBufferSource;
duration?: number | undefined;
timestamp: number;
type: EncodedAudioChunkType;
}
interface EncodedAudioChunkMetadata {
decoderConfig?: AudioDecoderConfig | undefined;
}
interface EncodedVideoChunkInit {
data: AllowSharedBufferSource;
duration?: number | undefined;
timestamp: number;
type: EncodedVideoChunkType;
}
interface EncodedVideoChunkMetadata {
decoderConfig?: VideoDecoderConfig | undefined;
temporalLayerId?: number | undefined;
}
interface ImageDecodeOptions {
completeFramesOnly?: boolean | undefined;
frameIndex?: number | undefined;
}
interface ImageDecodeResult {
complete: boolean;
image: VideoFrame;
}
interface ImageDecoderInit {
colorSpaceConversion?: ColorSpaceConversion | undefined;
data: ImageBufferSource;
desiredHeight?: number | undefined;
desiredWidth?: number | undefined;
preferAnimation?: boolean | undefined;
premultiplyAlpha?: PremultiplyAlpha | undefined;
type: string;
}
interface PlaneLayout {
offset: number;
stride: number;
}
interface VideoColorSpaceInit {
fullRange?: boolean | null | undefined;
matrix?: VideoMatrixCoefficients | null | undefined;
primaries?: VideoColorPrimaries | null | undefined;
transfer?: VideoTransferCharacteristics | null | undefined;
}
interface VideoDecoderConfig {
codec: string;
codedHeight?: number | undefined;
codedWidth?: number | undefined;
colorSpace?: VideoColorSpaceInit | undefined;
description?: AllowSharedBufferSource | undefined;
displayAspectHeight?: number | undefined;
displayAspectWidth?: number | undefined;
hardwareAcceleration?: HardwarePreference | undefined;
optimizeForLatency?: boolean | undefined;
}
interface VideoDecoderInit {
error: WebCodecsErrorCallback;
output: VideoFrameOutputCallback;
}
interface VideoDecoderSupport {
config?: VideoDecoderConfig;
supported?: boolean;
}
interface VideoEncoderConfig {
alpha?: AlphaOption | undefined;
avc?: AvcEncoderConfig | undefined;
bitrate?: number | undefined;
bitrateMode?: VideoEncoderBitrateMode | undefined;
codec: string;
displayHeight?: number | undefined;
displayWidth?: number | undefined;
framerate?: number | undefined;
hardwareAcceleration?: HardwarePreference | undefined;
height: number;
latencyMode?: LatencyMode | undefined;
scalabilityMode?: string | undefined;
width: number;
}
interface VideoEncoderEncodeOptions {
keyFrame?: boolean;
}
interface VideoEncoderInit {
error: WebCodecsErrorCallback;
output: EncodedVideoChunkOutputCallback;
}
interface VideoEncoderSupport {
config?: VideoEncoderConfig;
supported?: boolean;
}
interface VideoFrameBufferInit {
codedHeight: number;
codedWidth: number;
colorSpace?: VideoColorSpaceInit | undefined;
displayHeight?: number | undefined;
displayWidth?: number | undefined;
duration?: number | undefined;
format: VideoPixelFormat;
layout?: PlaneLayout[] | undefined;
timestamp: number;
visibleRect?: DOMRectInit | undefined;
}
interface VideoFrameCopyToOptions {
layout?: PlaneLayout[] | undefined;
rect?: DOMRectInit | undefined;
}
interface VideoFrameInit {
alpha?: AlphaOption | undefined;
displayHeight?: number | undefined;
displayWidth?: number | undefined;
duration?: number | undefined;
timestamp?: number | undefined;
visibleRect?: DOMRectInit | undefined;
}
interface AudioData {
readonly duration: number;
readonly format: AudioSampleFormat | null;
readonly numberOfChannels: number;
readonly numberOfFrames: number;
readonly sampleRate: number;
readonly timestamp: number;
allocationSize(options: AudioDataCopyToOptions): number;
clone(): AudioData;
close(): void;
copyTo(
destination: AllowSharedBufferSource,
options: AudioDataCopyToOptions,
): void;
}
declare var AudioData: {
prototype: AudioData;
new (init: AudioDataInit): AudioData;
};
interface AudioDecoderEventMap {
dequeue: Event;
}
/** Available only in secure contexts. */
interface AudioDecoder {
readonly decodeQueueSize: number;
readonly state: CodecState;
ondequeue: ((this: AudioDecoder, ev: Event) => any) | null;
close(): void;
configure(config: AudioDecoderConfig): void;
decode(chunk: EncodedAudioChunk): void;
flush(): Promise<void>;
reset(): void;
addEventListener<K extends keyof AudioDecoderEventMap>(
type: K,
listener: (this: AudioDecoder, ev: AudioDecoderEventMap[K]) => any,
options?: boolean | AddEventListenerOptions,
): void;
addEventListener(
type: string,
listener: EventListenerOrEventListenerObject,
options?: boolean | AddEventListenerOptions,
): void;
removeEventListener<K extends keyof AudioDecoderEventMap>(
type: K,
listener: (this: AudioDecoder, ev: AudioDecoderEventMap[K]) => any,
options?: boolean | EventListenerOptions,
): void;
removeEventListener(
type: string,
listener: EventListenerOrEventListenerObject,
options?: boolean | EventListenerOptions,
): void;
}
declare var AudioDecoder: {
prototype: AudioDecoder;
new (init: AudioDecoderInit): AudioDecoder;
isConfigSupported(config: AudioDecoderConfig): Promise<AudioDecoderSupport>;
};
interface AudioEncoderEventMap {
dequeue: Event;
}
/** Available only in secure contexts. */
interface AudioEncoder {
readonly encodeQueueSize: number;
readonly state: CodecState;
ondequeue: ((this: AudioEncoder, ev: Event) => any) | null;
close(): void;
configure(config: AudioEncoderConfig): void;
encode(data: AudioData): void;
flush(): Promise<void>;
reset(): void;
addEventListener<K extends keyof AudioEncoderEventMap>(
type: K,
listener: (this: AudioEncoder, ev: AudioEncoderEventMap[K]) => any,
options?: boolean | AddEventListenerOptions,
): void;
addEventListener(
type: string,
listener: EventListenerOrEventListenerObject,
options?: boolean | AddEventListenerOptions,
): void;
removeEventListener<K extends keyof AudioEncoderEventMap>(
type: K,
listener: (this: AudioEncoder, ev: AudioEncoderEventMap[K]) => any,
options?: boolean | EventListenerOptions,
): void;
removeEventListener(
type: string,
listener: EventListenerOrEventListenerObject,
options?: boolean | EventListenerOptions,
): void;
}
declare var AudioEncoder: {
prototype: AudioEncoder;
new (init: AudioEncoderInit): AudioEncoder;
isConfigSupported(config: AudioEncoderConfig): Promise<AudioEncoderSupport>;
};
interface EncodedAudioChunk {
readonly byteLength: number;
readonly duration: number | null;
readonly timestamp: number;
readonly type: EncodedAudioChunkType;
copyTo(destination: AllowSharedBufferSource): void;
}
declare var EncodedAudioChunk: {
prototype: EncodedAudioChunk;
new (init: EncodedAudioChunkInit): EncodedAudioChunk;
};
interface EncodedVideoChunk {
readonly byteLength: number;
readonly duration: number | null;
readonly timestamp: number;
readonly type: EncodedVideoChunkType;
copyTo(destination: AllowSharedBufferSource): void;
}
declare var EncodedVideoChunk: {
prototype: EncodedVideoChunk;
new (init: EncodedVideoChunkInit): EncodedVideoChunk;
};
/** Available only in secure contexts. */
interface ImageDecoder {
readonly complete: boolean;
readonly completed: Promise<void>;
readonly tracks: ImageTrackList;
readonly type: string;
close(): void;
decode(options?: ImageDecodeOptions): Promise<ImageDecodeResult>;
reset(): void;
}
// declare var ImageDecoder: {
// prototype: ImageDecoder;
// new(init: ImageDecoderInit): ImageDecoder;
// isTypeSupported(type: string): Promise<boolean>;
// };
// interface ImageTrack {
// readonly animated: boolean;
// readonly frameCount: number;
// readonly repetitionCount: number;
// selected: boolean;
// }
// declare var ImageTrack: {
// prototype: ImageTrack;
// new(): ImageTrack;
// };
// interface ImageTrackList {
// readonly length: number;
// readonly ready: Promise<void>;
// readonly selectedIndex: number;
// readonly selectedTrack: ImageTrack | null;
// [index: number]: ImageTrack;
// }
// declare var ImageTrackList: {
// prototype: ImageTrackList;
// new(): ImageTrackList;
// };
interface VideoColorSpace {
readonly fullRange: boolean | null;
readonly matrix: VideoMatrixCoefficients | null;
readonly primaries: VideoColorPrimaries | null;
readonly transfer: VideoTransferCharacteristics | null;
toJSON(): VideoColorSpaceInit;
}
declare var VideoColorSpace: {
prototype: VideoColorSpace;
new (init?: VideoColorSpaceInit): VideoColorSpace;
};
interface VideoDecoderEventMap {
dequeue: Event;
}
/** Available only in secure contexts. */
interface VideoDecoder {
readonly decodeQueueSize: number;
readonly state: CodecState;
ondequeue: ((this: VideoDecoder, ev: Event) => any) | null;
close(): void;
configure(config: VideoDecoderConfig): void;
decode(chunk: EncodedVideoChunk): void;
flush(): Promise<void>;
reset(): void;
addEventListener<K extends keyof VideoDecoderEventMap>(
type: K,
listener: (this: VideoDecoder, ev: VideoDecoderEventMap[K]) => any,
options?: boolean | AddEventListenerOptions,
): void;
addEventListener(
type: string,
listener: EventListenerOrEventListenerObject,
options?: boolean | AddEventListenerOptions,
): void;
removeEventListener<K extends keyof VideoDecoderEventMap>(
type: K,
listener: (this: VideoDecoder, ev: VideoDecoderEventMap[K]) => any,
options?: boolean | EventListenerOptions,
): void;
removeEventListener(
type: string,
listener: EventListenerOrEventListenerObject,
options?: boolean | EventListenerOptions,
): void;
}
declare var VideoDecoder: {
prototype: VideoDecoder;
new (init: VideoDecoderInit): VideoDecoder;
isConfigSupported(config: VideoDecoderConfig): Promise<VideoDecoderSupport>;
};
interface VideoEncoderEventMap {
dequeue: Event;
}
/** Available only in secure contexts. */
interface VideoEncoder {
readonly encodeQueueSize: number;
readonly state: CodecState;
close(): void;
ondequeue: ((this: VideoEncoder, ev: Event) => any) | null;
configure(config: VideoEncoderConfig): void;
encode(frame: VideoFrame, options?: VideoEncoderEncodeOptions): void;
flush(): Promise<void>;
reset(): void;
addEventListener<K extends keyof VideoEncoderEventMap>(
type: K,
listener: (this: VideoEncoder, ev: VideoEncoderEventMap[K]) => any,
options?: boolean | AddEventListenerOptions,
): void;
addEventListener(
type: string,
listener: EventListenerOrEventListenerObject,
options?: boolean | AddEventListenerOptions,
): void;
removeEventListener<K extends keyof VideoEncoderEventMap>(
type: K,
listener: (this: VideoEncoder, ev: VideoEncoderEventMap[K]) => any,
options?: boolean | EventListenerOptions,
): void;
removeEventListener(
type: string,
listener: EventListenerOrEventListenerObject,
options?: boolean | EventListenerOptions,
): void;
}
declare var VideoEncoder: {
prototype: VideoEncoder;
new (init: VideoEncoderInit): VideoEncoder;
isConfigSupported(config: VideoEncoderConfig): Promise<VideoEncoderSupport>;
};
interface VideoFrame {
readonly codedHeight: number;
readonly codedRect: DOMRectReadOnly | null;
readonly codedWidth: number;
readonly colorSpace: VideoColorSpace;
readonly displayHeight: number;
readonly displayWidth: number;
readonly duration: number | null;
readonly format: VideoPixelFormat | null;
readonly timestamp: number;
readonly visibleRect: DOMRectReadOnly | null;
allocationSize(options?: VideoFrameCopyToOptions): number;
clone(): VideoFrame;
close(): void;
copyTo(
destination: AllowSharedBufferSource,
options?: VideoFrameCopyToOptions,
): Promise<PlaneLayout[]>;
}
declare var VideoFrame: {
prototype: VideoFrame;
new (source: CanvasImageSource, init?: VideoFrameInit): VideoFrame;
new (data: AllowSharedBufferSource, init: VideoFrameBufferInit): VideoFrame;
};
interface AudioDataOutputCallback {
(output: AudioData): void;
}
interface EncodedAudioChunkOutputCallback {
(output: EncodedAudioChunk, metadata: EncodedAudioChunkMetadata): void;
}
interface EncodedVideoChunkOutputCallback {
(chunk: EncodedVideoChunk, metadata: EncodedVideoChunkMetadata): void;
}
interface VideoFrameOutputCallback {
(output: VideoFrame): void;
}
interface WebCodecsErrorCallback {
(error: DOMException): void;
}
// type AllowSharedBufferSource = ArrayBuffer | ArrayBufferView;
// type BitrateMode = "constant" | "variable";
// type ImageBufferSource = ArrayBuffer | ArrayBufferView | ReadableStream;
// type AlphaOption = "discard" | "keep";
// type AudioSampleFormat = "f32" | "f32-planar" | "s16" | "s16-planar" | "s32" | "s32-planar" | "u8" | "u8-planar";
// type AvcBitstreamFormat = "annexb" | "avc";
// type CodecState = "closed" | "configured" | "unconfigured";
// type EncodedAudioChunkType = "delta" | "key";
// type EncodedVideoChunkType = "delta" | "key";
type HardwarePreference =
| "no-preference"
| "prefer-hardware"
| "prefer-software";
// type LatencyMode = "quality" | "realtime";
// type VideoColorPrimaries = "bt470bg" | "bt709" | "smpte170m";
// type VideoMatrixCoefficients = "bt470bg" | "bt709" | "rgb" | "smpte170m";
// type VideoPixelFormat = "BGRA" | "BGRX" | "I420" | "I420A" | "I422" | "I444" | "NV12" | "RGBA" | "RGBX";
// type VideoTransferCharacteristics = "bt709" | "iec61966-2-1" | "smpte170m";

View File

@@ -22,6 +22,7 @@ import { Initializer } from "./initializer";
import { MediaDevicesProvider } from "./livekit/MediaDevicesContext";
import { widget } from "./widget";
import { useTheme } from "./useTheme";
import { ProcessorProvider } from "./livekit/TrackProcessorContext";
const SentryRoute = Sentry.withSentryReactRouterV7Routing(Route);
@@ -72,22 +73,24 @@ export const App: FC = () => {
<Suspense fallback={null}>
<ClientProvider>
<MediaDevicesProvider>
<Sentry.ErrorBoundary
fallback={(error) => (
<ErrorPage error={error} widget={widget} />
)}
>
<DisconnectedBanner />
<Routes>
<SentryRoute path="/" element={<HomePage />} />
<SentryRoute path="/login" element={<LoginPage />} />
<SentryRoute
path="/register"
element={<RegisterPage />}
/>
<SentryRoute path="*" element={<RoomPage />} />
</Routes>
</Sentry.ErrorBoundary>
<ProcessorProvider>
<Sentry.ErrorBoundary
fallback={(error) => (
<ErrorPage error={error} widget={widget} />
)}
>
<DisconnectedBanner />
<Routes>
<SentryRoute path="/" element={<HomePage />} />
<SentryRoute path="/login" element={<LoginPage />} />
<SentryRoute
path="/register"
element={<RegisterPage />}
/>
<SentryRoute path="*" element={<RoomPage />} />
</Routes>
</Sentry.ErrorBoundary>
</ProcessorProvider>
</MediaDevicesProvider>
</ClientProvider>
</Suspense>

View File

@@ -16,7 +16,7 @@ export class MatrixKeyProvider extends BaseKeyProvider {
private rtcSession?: MatrixRTCSession;
public constructor() {
super({ ratchetWindowSize: 0, keyringSize: 256 });
super({ ratchetWindowSize: 10, keyringSize: 256 });
}
public setRTCSession(rtcSession: MatrixRTCSession): void {

View File

@@ -0,0 +1,80 @@
/*
Copyright 2024-2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only
Please see LICENSE in the repository root for full details.
*/
import {
BackgroundTransformer,
VideoTransformer,
type VideoTransformerInitOptions,
} from "@livekit/track-processors";
import { ImageSegmenter } from "@mediapipe/tasks-vision";
import modelAssetPath from "../mediapipe/imageSegmenter/selfie_segmenter.tflite?url";
interface WasmFileset {
/** The path to the Wasm loader script. */
wasmLoaderPath: string;
/** The path to the Wasm binary. */
wasmBinaryPath: string;
}
// The MediaPipe package, by default, ships some alternative versions of the
// WASM files which avoid SIMD for compatibility with older browsers. But SIMD
// in WASM is actually fine by our support policy, so we include just the SIMD
// versions.
// It's really not ideal that we have to reference these internal files from
// MediaPipe and depend on node_modules having this specific structure. It's
// easy to see this breaking if our dependencies changed and MediaPipe were
// no longer hoisted, or if we switched to another dependency loader such as
// Yarn PnP.
// https://github.com/google-ai-edge/mediapipe/issues/5961
const wasmFileset: WasmFileset = {
wasmLoaderPath: new URL(
"../../node_modules/@mediapipe/tasks-vision/wasm/vision_wasm_internal.js",
import.meta.url,
).href,
wasmBinaryPath: new URL(
"../../node_modules/@mediapipe/tasks-vision/wasm/vision_wasm_internal.wasm",
import.meta.url,
).href,
};
/**
* Track processor that applies effects such as blurring to a user's background.
*
* This is just like LiveKit's prebuilt BackgroundTransformer except that it
* loads the segmentation models from our own bundle rather than as an external
* resource fetched from the public internet.
*/
export class BlurBackgroundTransformer extends BackgroundTransformer {
public async init({
outputCanvas,
inputElement: inputVideo,
}: VideoTransformerInitOptions): Promise<void> {
// Call super.super.init() since we're totally replacing the init method of
// BackgroundTransformer here, rather than extending it
await VideoTransformer.prototype.init.call(this, {
outputCanvas,
inputElement: inputVideo,
});
this.imageSegmenter = await ImageSegmenter.createFromOptions(wasmFileset, {
baseOptions: {
modelAssetPath,
delegate: "GPU",
...this.options.segmenterOptions,
},
canvas: this.canvas,
runningMode: "VIDEO",
outputCategoryMask: true,
outputConfidenceMasks: false,
});
if (this.options.blurRadius) {
this.gl?.setBlurRadius(this.options.blurRadius);
}
}
}

View File

@@ -0,0 +1,84 @@
/*
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only
Please see LICENSE in the repository root for full details.
*/
import {
ProcessorWrapper,
supportsBackgroundProcessors,
type BackgroundOptions,
} from "@livekit/track-processors";
import { createContext, type FC, useContext, useEffect, useMemo } from "react";
import { type LocalVideoTrack } from "livekit-client";
import {
backgroundBlur as backgroundBlurSettings,
useSetting,
} from "../settings/settings";
import { BlurBackgroundTransformer } from "./BlurBackgroundTransformer";
type ProcessorState = {
supported: boolean | undefined;
processor: undefined | ProcessorWrapper<BackgroundOptions>;
};
const ProcessorContext = createContext<ProcessorState | undefined>(undefined);
export function useTrackProcessor(): ProcessorState {
const state = useContext(ProcessorContext);
if (state === undefined)
throw new Error(
"useTrackProcessor must be used within a ProcessorProvider",
);
return state;
}
export const useTrackProcessorSync = (
videoTrack: LocalVideoTrack | null,
): void => {
const { processor } = useTrackProcessor();
useEffect(() => {
if (!videoTrack) return;
if (processor && !videoTrack.getProcessor()) {
void videoTrack.setProcessor(processor);
}
if (!processor && videoTrack.getProcessor()) {
void videoTrack.stopProcessor();
}
}, [processor, videoTrack]);
};
interface Props {
children: JSX.Element;
}
export const ProcessorProvider: FC<Props> = ({ children }) => {
// The setting the user wants to have
const [blurActivated] = useSetting(backgroundBlurSettings);
const supported = useMemo(() => supportsBackgroundProcessors(), []);
const blur = useMemo(
() =>
new ProcessorWrapper(
new BlurBackgroundTransformer({ blurRadius: 15 }),
"background-blur",
),
[],
);
// This is the actual state exposed through the context
const processorState = useMemo(
() => ({
supported,
processor: supported && blurActivated ? blur : undefined,
}),
[supported, blurActivated, blur],
);
return (
<ProcessorContext.Provider value={processorState}>
{children}
</ProcessorContext.Provider>
);
};

View File

@@ -6,7 +6,7 @@ Please see LICENSE in the repository root for full details.
*/
import { type FC, useCallback, useState } from "react";
import { test, vi } from "vitest";
import { describe, expect, test, vi, vitest } from "vitest";
import {
ConnectionError,
ConnectionErrorReason,
@@ -15,6 +15,7 @@ import {
import userEvent from "@testing-library/user-event";
import { render, screen } from "@testing-library/react";
import { MemoryRouter } from "react-router-dom";
import { defer, sleep } from "matrix-js-sdk/lib/utils";
import { useECConnectionState } from "./useECConnectionState";
import { type SFUConfig } from "./openIDSFU";
@@ -57,7 +58,7 @@ test.each<[string, ConnectionError]>([
() => setSfuConfig({ url: "URL", jwt: "JWT token" }),
[],
);
useECConnectionState({}, false, mockRoom, sfuConfig);
useECConnectionState("default", false, mockRoom, sfuConfig);
return <button onClick={connect}>Connect</button>;
};
@@ -73,3 +74,111 @@ test.each<[string, ConnectionError]>([
screen.getByText("Insufficient capacity");
},
);
describe("Leaking connection prevention", () => {
function createTestComponent(mockRoom: Room): FC {
const TestComponent: FC = () => {
const [sfuConfig, setSfuConfig] = useState<SFUConfig | undefined>(
undefined,
);
const connect = useCallback(
() => setSfuConfig({ url: "URL", jwt: "JWT token" }),
[],
);
useECConnectionState("default", false, mockRoom, sfuConfig);
return <button onClick={connect}>Connect</button>;
};
return TestComponent;
}
test("Should cancel pending connections when the component is unmounted", async () => {
const connectCall = vi.fn();
const pendingConnection = defer<void>();
// let pendingDisconnection = defer<void>()
const disconnectMock = vi.fn();
const mockRoom = {
on: () => {},
off: () => {},
once: () => {},
connect: async () => {
connectCall.call(undefined);
return await pendingConnection.promise;
},
disconnect: disconnectMock,
localParticipant: {
getTrackPublication: () => {},
createTracks: () => [],
},
} as unknown as Room;
const TestComponent = createTestComponent(mockRoom);
const { unmount } = render(<TestComponent />);
const user = userEvent.setup();
await user.click(screen.getByRole("button", { name: "Connect" }));
expect(connectCall).toHaveBeenCalled();
// unmount while the connection is pending
unmount();
// resolve the pending connection
pendingConnection.resolve();
await vitest.waitUntil(
() => {
return disconnectMock.mock.calls.length > 0;
},
{
timeout: 1000,
interval: 100,
},
);
// There should be some cleaning up to avoid leaking an open connection
expect(disconnectMock).toHaveBeenCalledTimes(1);
});
test("Should cancel about to open but not yet opened connection", async () => {
const createTracksCall = vi.fn();
const pendingCreateTrack = defer<void>();
// let pendingDisconnection = defer<void>()
const disconnectMock = vi.fn();
const connectMock = vi.fn();
const mockRoom = {
on: () => {},
off: () => {},
once: () => {},
connect: connectMock,
disconnect: disconnectMock,
localParticipant: {
getTrackPublication: () => {},
createTracks: async () => {
createTracksCall.call(undefined);
await pendingCreateTrack.promise;
return [];
},
},
} as unknown as Room;
const TestComponent = createTestComponent(mockRoom);
const { unmount } = render(<TestComponent />);
const user = userEvent.setup();
await user.click(screen.getByRole("button", { name: "Connect" }));
expect(createTracksCall).toHaveBeenCalled();
// unmount while createTracks is pending
unmount();
// resolve createTracks
pendingCreateTrack.resolve();
// Yield to the event loop to let the connection attempt finish
await sleep(100);
// The operation should have been aborted before even calling connect.
expect(connectMock).not.toHaveBeenCalled();
});
});

View File

@@ -6,7 +6,6 @@ Please see LICENSE in the repository root for full details.
*/
import {
type AudioCaptureOptions,
ConnectionError,
ConnectionState,
type LocalTrack,
@@ -25,6 +24,7 @@ import {
InsufficientCapacityError,
UnknownCallError,
} from "../utils/errors.ts";
import { AbortHandle } from "../utils/abortHandle.ts";
declare global {
interface Window {
@@ -59,7 +59,8 @@ async function doConnect(
livekitRoom: Room,
sfuConfig: SFUConfig,
audioEnabled: boolean,
audioOptions: AudioCaptureOptions,
initialDeviceId: string | undefined,
abortHandle: AbortHandle,
): Promise<void> {
// Always create an audio track manually.
// livekit (by default) keeps the mic track open when you mute, but if you start muted,
@@ -82,19 +83,40 @@ async function doConnect(
let preCreatedAudioTrack: LocalTrack | undefined;
try {
const audioTracks = await livekitRoom!.localParticipant.createTracks({
audio: audioOptions,
audio: { deviceId: initialDeviceId },
});
if (audioTracks.length < 1) {
logger.info("Tried to pre-create local audio track but got no tracks");
} else {
preCreatedAudioTrack = audioTracks[0];
}
// There was a yield point previously (awaiting for the track to be created) so we need to check
// if the operation was cancelled and stop connecting if needed.
if (abortHandle.isAborted()) {
logger.info(
"[Lifecycle] Signal Aborted: Pre-created audio track but connection aborted",
);
preCreatedAudioTrack?.stop();
return;
}
logger.info("Pre-created microphone track");
} catch (e) {
logger.error("Failed to pre-create microphone track", e);
}
if (!audioEnabled) await preCreatedAudioTrack?.mute();
if (!audioEnabled) {
await preCreatedAudioTrack?.mute();
// There was a yield point. Check if the operation was cancelled and stop connecting.
if (abortHandle.isAborted()) {
logger.info(
"[Lifecycle] Signal Aborted: Pre-created audio track but connection aborted",
);
preCreatedAudioTrack?.stop();
return;
}
}
// check again having awaited for the track to create
if (
@@ -107,9 +129,18 @@ async function doConnect(
return;
}
logger.info("Connecting & publishing");
logger.info("[Lifecycle] Connecting & publishing");
try {
await connectAndPublish(livekitRoom, sfuConfig, preCreatedAudioTrack, []);
if (abortHandle.isAborted()) {
logger.info(
"[Lifecycle] Signal Aborted: Connected but operation was cancelled. Force disconnect",
);
livekitRoom?.disconnect().catch((err) => {
logger.error("Failed to disconnect from SFU", err);
});
return;
}
} catch (e) {
preCreatedAudioTrack?.stop();
logger.debug("Stopped precreated audio tracks.");
@@ -137,13 +168,16 @@ async function connectAndPublish(
livekitRoom.once(RoomEvent.SignalConnected, tracker.cacheWsConnect);
try {
logger.info(`[Lifecycle] Connecting to livekit room ${sfuConfig!.url} ...`);
await livekitRoom!.connect(sfuConfig!.url, sfuConfig!.jwt, {
// Due to stability issues on Firefox we are testing the effect of different
// timeouts, and allow these values to be set through the console
peerConnectionTimeout: window.peerConnectionTimeout ?? 45000,
websocketTimeout: window.websocketTimeout ?? 45000,
});
logger.info(`[Lifecycle] ... connected to livekit room`);
} catch (e) {
logger.error("[Lifecycle] Failed to connect", e);
// LiveKit uses 503 to indicate that the server has hit its track limits.
// https://github.com/livekit/livekit/blob/fcb05e97c5a31812ecf0ca6f7efa57c485cea9fb/pkg/service/rtcservice.go#L171
// It also errors with a status code of 200 (yes, really) for room
@@ -184,7 +218,7 @@ async function connectAndPublish(
}
export function useECConnectionState(
initialAudioOptions: AudioCaptureOptions,
initialDeviceId: string | undefined,
initialAudioEnabled: boolean,
livekitRoom?: Room,
sfuConfig?: SFUConfig,
@@ -247,6 +281,22 @@ export function useECConnectionState(
const currentSFUConfig = useRef(Object.assign({}, sfuConfig));
// Protection against potential leaks, where the component to be unmounted and there is
// still a pending doConnect promise. This would lead the user to still be in the call even
// if the component is unmounted.
const abortHandlesBag = useRef(new Set<AbortHandle>());
// This is a cleanup function that will be called when the component is about to be unmounted.
// It will cancel all abortHandles in the bag
useEffect(() => {
const bag = abortHandlesBag.current;
return (): void => {
bag.forEach((handle) => {
handle.abort();
});
};
}, []);
// Id we are transitioning from a valid config to another valid one, we need
// to explicitly switch focus
useEffect(() => {
@@ -273,11 +323,14 @@ export function useECConnectionState(
// always capturing audio: it helps keep bluetooth headsets in the right mode and
// mobile browsers to know we're doing a call.
setIsInDoConnect(true);
const abortHandle = new AbortHandle();
abortHandlesBag.current.add(abortHandle);
doConnect(
livekitRoom!,
sfuConfig!,
initialAudioEnabled,
initialAudioOptions,
initialDeviceId,
abortHandle,
)
.catch((e) => {
if (e instanceof ElementCallError) {
@@ -286,14 +339,17 @@ export function useECConnectionState(
setError(new UnknownCallError(e));
} else logger.error("Failed to connect to SFU", e);
})
.finally(() => setIsInDoConnect(false));
.finally(() => {
abortHandlesBag.current.delete(abortHandle);
setIsInDoConnect(false);
});
}
currentSFUConfig.current = Object.assign({}, sfuConfig);
}, [
sfuConfig,
livekitRoom,
initialAudioOptions,
initialDeviceId,
initialAudioEnabled,
doFocusSwitch,
]);

View File

@@ -9,6 +9,7 @@ import {
ConnectionState,
type E2EEManagerOptions,
ExternalE2EEKeyProvider,
LocalVideoTrack,
Room,
type RoomOptions,
Track,
@@ -17,6 +18,8 @@ import { useEffect, useMemo, useRef } from "react";
import E2EEWorker from "livekit-client/e2ee-worker?worker";
import { logger } from "matrix-js-sdk/lib/logger";
import { type MatrixRTCSession } from "matrix-js-sdk/lib/matrixrtc";
import { useObservable, useObservableEagerState } from "observable-hooks";
import { map } from "rxjs";
import { defaultLiveKitOptions } from "./options";
import { type SFUConfig } from "./openIDSFU";
@@ -33,6 +36,12 @@ import {
import { MatrixKeyProvider } from "../e2ee/matrixKeyProvider";
import { E2eeType } from "../e2ee/e2eeType";
import { type EncryptionSystem } from "../e2ee/sharedKeyManagement";
import {
useTrackProcessor,
useTrackProcessorSync,
} from "./TrackProcessorContext";
import { useInitial } from "../useInitial";
import { observeTrackReference$ } from "../state/MediaViewModel";
interface UseLivekitResult {
livekitRoom?: Room;
@@ -82,12 +91,15 @@ export function useLiveKit(
const devices = useMediaDevices();
const initialDevices = useRef<MediaDevices>(devices);
const { processor } = useTrackProcessor();
const initialProcessor = useInitial(() => processor);
const roomOptions = useMemo(
(): RoomOptions => ({
...defaultLiveKitOptions,
videoCaptureDefaults: {
...defaultLiveKitOptions.videoCaptureDefaults,
deviceId: initialDevices.current.videoInput.selectedId,
processor: initialProcessor,
},
audioCaptureDefaults: {
...defaultLiveKitOptions.audioCaptureDefaults,
@@ -98,7 +110,7 @@ export function useLiveKit(
},
e2ee: e2eeOptions,
}),
[e2eeOptions],
[e2eeOptions, initialProcessor],
);
// Store if audio/video are currently updating. If to prohibit unnecessary calls
@@ -123,10 +135,27 @@ export function useLiveKit(
return r;
}, [roomOptions, e2eeSystem]);
// Sync the requested track processors with LiveKit
useTrackProcessorSync(
useObservableEagerState(
useObservable(
(room$) =>
observeTrackReference$(
room$.pipe(map(([room]) => room.localParticipant)),
Track.Source.Camera,
).pipe(
map((trackRef) => {
const track = trackRef?.publication?.track;
return track instanceof LocalVideoTrack ? track : null;
}),
),
[room],
),
),
);
const connectionState = useECConnectionState(
{
deviceId: initialDevices.current.audioInput.selectedId,
},
initialDevices.current.audioInput.selectedId,
initialMuteStates.current.audio.enabled,
room,
sfuConfig,
@@ -198,6 +227,7 @@ export function useLiveKit(
audioMuteUpdating.current = true;
trackPublication = await participant.setMicrophoneEnabled(
buttonEnabled.current.audio,
room.options.audioCaptureDefaults,
);
audioMuteUpdating.current = false;
break;
@@ -205,6 +235,7 @@ export function useLiveKit(
videoMuteUpdating.current = true;
trackPublication = await participant.setCameraEnabled(
buttonEnabled.current.video,
room.options.videoCaptureDefaults,
);
videoMuteUpdating.current = false;
break;

View File

@@ -0,0 +1,5 @@
# Google AI Edge MediaPipe Selfie Segmentation
- See: https://ai.google.dev/edge/mediapipe/solutions/vision/image_segmenter
- Latest: https://storage.googleapis.com/mediapipe-models/image_segmenter/selfie_segmenter/float16/latest/selfie_segmenter.tflite
- License: Apache 2.0 as per https://storage.googleapis.com/mediapipe-assets/Model%20Card%20MediaPipe%20Selfie%20Segmentation.pdf

Binary file not shown.

View File

@@ -38,6 +38,7 @@ import { GroupCallView } from "./GroupCallView";
import { type WidgetHelpers } from "../widget";
import { LazyEventEmitter } from "../LazyEventEmitter";
import { MatrixRTCFocusMissingError } from "../utils/errors";
import { ProcessorProvider } from "../livekit/TrackProcessorContext";
vi.mock("../soundUtils");
vi.mock("../useAudioContext");
@@ -46,6 +47,13 @@ vi.mock("react-use-measure", () => ({
default: (): [() => void, object] => [(): void => {}, {}],
}));
vi.hoisted(
() =>
(global.ImageData = class MockImageData {
public data: number[] = [];
} as unknown as typeof ImageData),
);
const enterRTCSession = vi.hoisted(() => vi.fn(async () => Promise.resolve()));
const leaveRTCSession = vi.hoisted(() =>
vi.fn(
@@ -137,18 +145,20 @@ function createGroupCallView(
const { getByText } = render(
<BrowserRouter>
<TooltipProvider>
<GroupCallView
client={client}
isPasswordlessUser={false}
confineToRoom={false}
preload={false}
skipLobby={false}
hideHeader={true}
rtcSession={rtcSession as unknown as MatrixRTCSession}
isJoined={joined}
muteStates={muteState}
widget={widget}
/>
<ProcessorProvider>
<GroupCallView
client={client}
isPasswordlessUser={false}
confineToRoom={false}
preload={false}
skipLobby={false}
hideHeader={true}
rtcSession={rtcSession as unknown as MatrixRTCSession}
isJoined={joined}
muteStates={muteState}
widget={widget}
/>
</ProcessorProvider>
</TooltipProvider>
</BrowserRouter>,
);

View File

@@ -118,6 +118,13 @@ export const GroupCallView: FC<Props> = ({
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
useEffect(() => {
logger.info("[Lifecycle] GroupCallView Component mounted");
return (): void => {
logger.info("[Lifecycle] GroupCallView Component unmounted");
};
}, []);
useEffect(() => {
window.rtcSession = rtcSession;
return (): void => {

View File

@@ -127,10 +127,23 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
const [vm, setVm] = useState<CallViewModel | null>(null);
useEffect(() => {
logger.info(
`[Lifecycle] InCallView Component mounted, livekitroom state ${livekitRoom?.state}`,
);
return (): void => {
livekitRoom?.disconnect().catch((e) => {
logger.error("Failed to disconnect from livekit room", e);
});
logger.info(
`[Lifecycle] InCallView Component unmounted, livekitroom state ${livekitRoom?.state}`,
);
livekitRoom
?.disconnect()
.then(() => {
logger.info(
`[Lifecycle] Disconnected from livekite room, state:${livekitRoom?.state}`,
);
})
.catch((e) => {
logger.error("[Lifecycle] Failed to disconnect from livekit room", e);
});
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);

View File

@@ -5,14 +5,25 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC, useCallback, useMemo, useState, type JSX } from "react";
import {
type FC,
useCallback,
useMemo,
useState,
type JSX,
useEffect,
} from "react";
import { useTranslation } from "react-i18next";
import { type MatrixClient } from "matrix-js-sdk";
import { Button } from "@vector-im/compound-web";
import classNames from "classnames";
import { logger } from "matrix-js-sdk/lib/logger";
import { usePreviewTracks } from "@livekit/components-react";
import { type LocalVideoTrack, Track } from "livekit-client";
import {
type CreateLocalTracksOptions,
type LocalVideoTrack,
Track,
} from "livekit-client";
import { useObservable } from "observable-hooks";
import { map } from "rxjs";
import { useNavigate } from "react-router-dom";
@@ -36,7 +47,11 @@ import { E2eeType } from "../e2ee/e2eeType";
import { Link } from "../button/Link";
import { useMediaDevices } from "../livekit/MediaDevicesContext";
import { useInitial } from "../useInitial";
import { useSwitchCamera } from "./useSwitchCamera";
import { useSwitchCamera as useShowSwitchCamera } from "./useSwitchCamera";
import {
useTrackProcessor,
useTrackProcessorSync,
} from "../livekit/TrackProcessorContext";
import { usePageTitle } from "../usePageTitle";
interface Props {
@@ -64,6 +79,13 @@ export const LobbyView: FC<Props> = ({
onShareClick,
waitingForInvite,
}) => {
useEffect(() => {
logger.info("[Lifecycle] GroupCallView Component mounted");
return (): void => {
logger.info("[Lifecycle] GroupCallView Component unmounted");
};
}, []);
const { t } = useTranslation();
usePageTitle(matrixInfo.roomName);
@@ -112,7 +134,10 @@ export const LobbyView: FC<Props> = ({
muteStates.audio.enabled && { deviceId: devices.audioInput.selectedId },
);
const localTrackOptions = useMemo(
const { processor } = useTrackProcessor();
const initialProcessor = useInitial(() => processor);
const localTrackOptions = useMemo<CreateLocalTracksOptions>(
() => ({
// The only reason we request audio here is to get the audio permission
// request over with at the same time. But changing the audio settings
@@ -123,12 +148,14 @@ export const LobbyView: FC<Props> = ({
audio: Object.assign({}, initialAudioOptions),
video: muteStates.video.enabled && {
deviceId: devices.videoInput.selectedId,
processor: initialProcessor,
},
}),
[
initialAudioOptions,
devices.videoInput.selectedId,
muteStates.video.enabled,
devices.videoInput.selectedId,
initialProcessor,
],
);
@@ -149,8 +176,8 @@ export const LobbyView: FC<Props> = ({
null) as LocalVideoTrack | null,
[tracks],
);
const switchCamera = useSwitchCamera(
useTrackProcessorSync(videoTrack);
const showSwitchCamera = useShowSwitchCamera(
useObservable(
(inputs$) => inputs$.pipe(map(([video]) => video)),
[videoTrack],
@@ -212,7 +239,9 @@ export const LobbyView: FC<Props> = ({
onClick={onVideoPress}
disabled={muteStates.video.setEnabled === null}
/>
{switchCamera && <SwitchCameraButton onClick={switchCamera} />}
{showSwitchCamera && (
<SwitchCameraButton onClick={showSwitchCamera} />
)}
<SettingsButton onClick={openSettings} />
{!confineToRoom && <EndCallButton onClick={onLeaveClick} />}
</div>

View File

@@ -5,10 +5,10 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC, useState } from "react";
import { type FC, type ReactNode, useState } from "react";
import { useTranslation } from "react-i18next";
import { type MatrixClient } from "matrix-js-sdk";
import { Root as Form } from "@vector-im/compound-web";
import { Root as Form, Separator } from "@vector-im/compound-web";
import { type Room as LivekitRoom } from "livekit-client";
import { Modal } from "../Modal";
@@ -24,12 +24,15 @@ import { widget } from "../widget";
import {
useSetting,
soundEffectVolumeSetting,
backgroundBlur as backgroundBlurSetting,
developerMode,
} from "./settings";
import { PreferencesSettingsTab } from "./PreferencesSettingsTab";
import { Slider } from "../Slider";
import { DeviceSelection } from "./DeviceSelection";
import { useTrackProcessor } from "../livekit/TrackProcessorContext";
import { DeveloperSettingsTab } from "./DeveloperSettingsTab";
import { FieldRow, InputField } from "../input/Input";
import { useSubmitRageshake } from "./submit-rageshake";
type SettingsTab =
@@ -64,6 +67,33 @@ export const SettingsModal: FC<Props> = ({
}) => {
const { t } = useTranslation();
// Generate a `Checkbox` input to turn blur on or off.
const BlurCheckbox: React.FC = (): ReactNode => {
const { supported } = useTrackProcessor();
const [blurActive, setBlurActive] = useSetting(backgroundBlurSetting);
return (
<>
<h4>{t("settings.background_blur_header")}</h4>
<FieldRow>
<InputField
id="activateBackgroundBlur"
label={t("settings.background_blur_label")}
description={
supported ? "" : t("settings.blur_not_supported_by_browser")
}
type="checkbox"
checked={!!blurActive}
onChange={(b): void => setBlurActive(b.target.checked)}
disabled={!supported}
/>
</FieldRow>
</>
);
};
const devices = useMediaDevices();
useMediaDeviceNames(devices, open);
const [soundVolume, setSoundVolume] = useSetting(soundEffectVolumeSetting);
@@ -113,13 +143,17 @@ export const SettingsModal: FC<Props> = ({
key: "video",
name: t("common.video"),
content: (
<Form>
<DeviceSelection
devices={devices.videoInput}
title={t("settings.devices.camera")}
numberedLabel={(n) => t("settings.devices.camera_numbered", { n })}
/>
</Form>
<>
<Form>
<DeviceSelection
devices={devices.videoInput}
title={t("settings.devices.camera")}
numberedLabel={(n) => t("settings.devices.camera_numbered", { n })}
/>
</Form>
<Separator />
<BlurCheckbox />
</>
),
};

View File

@@ -473,11 +473,6 @@ export async function init(): Promise<void> {
// configure loglevel based loggers:
setLogExtension(logger, global.mx_rage_logger.log);
// these are the child/prefixed loggers we want to capture from js-sdk
// there doesn't seem to be an easy way to capture all children
["MatrixRTCSession", "MatrixRTCSessionManager"].forEach((loggerName) => {
setLogExtension(logger.getChild(loggerName), global.mx_rage_logger.log);
});
// intercept console logging so that we can get matrix_sdk logs:
// this is nasty, but no logging hooks are provided

View File

@@ -96,6 +96,8 @@ export const videoInput = new Setting<string | undefined>(
undefined,
);
export const backgroundBlur = new Setting<boolean>("background-blur", false);
export const showHandRaisedTimer = new Setting<boolean>(
"hand-raised-show-timer",
false,

18
src/utils/abortHandle.ts Normal file
View File

@@ -0,0 +1,18 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
export class AbortHandle {
public constructor(private aborted = false) {}
public abort(): void {
this.aborted = true;
}
public isAborted(): boolean {
return this.aborted;
}
}

214
yarn.lock
View File

@@ -2536,6 +2536,17 @@ __metadata:
languageName: node
linkType: hard
"@livekit/track-processors@npm:^0.5.5":
version: 0.5.5
resolution: "@livekit/track-processors@npm:0.5.5"
dependencies:
"@mediapipe/tasks-vision": "npm:0.10.14"
peerDependencies:
livekit-client: ^1.12.0 || ^2.1.0
checksum: 10c0/b86737c4ce6b93c714dfc814c997d07f1d40b0869b5e269b28e00034a3c06da811d9117c4914d8a2706ac760be31f2e9d0705108c07b29f4bbb4626dcd41ed64
languageName: node
linkType: hard
"@matrix-org/matrix-sdk-crypto-wasm@npm:^14.0.1":
version: 14.0.1
resolution: "@matrix-org/matrix-sdk-crypto-wasm@npm:14.0.1"
@@ -2550,6 +2561,13 @@ __metadata:
languageName: node
linkType: hard
"@mediapipe/tasks-vision@npm:^0.10.18":
version: 0.10.21
resolution: "@mediapipe/tasks-vision@npm:0.10.21"
checksum: 10c0/11b2bdf98b8cb6e044f2a954e7c8393169e62c86ff49b3d0b61c3b327d18e1ccd47a187999b023bad48380c9da41bfa66eb165301c80da07746390482cb18a19
languageName: node
linkType: hard
"@nodelib/fs.scandir@npm:2.1.5":
version: 2.1.5
resolution: "@nodelib/fs.scandir@npm:2.1.5"
@@ -3045,14 +3063,14 @@ __metadata:
languageName: node
linkType: hard
"@playwright/test@npm:^1.51.0":
version: 1.51.0
resolution: "@playwright/test@npm:1.51.0"
"@playwright/test@npm:^1.52.0":
version: 1.52.0
resolution: "@playwright/test@npm:1.52.0"
dependencies:
playwright: "npm:1.51.0"
playwright: "npm:1.52.0"
bin:
playwright: cli.js
checksum: 10c0/ae83dd2c3a32133de58f44a9dbcd73a8059155ebd8acc736ba8bd0a7ca99b194afe2e8f5a500861d18b1c8f06b4e4ea8de4a2402297c59053d4becc404b47e0a
checksum: 10c0/1c428b421593eb4f79b7c99783a389c3ab3526c9051ec772749f4fca61414dfa9f2344eba846faac5f238084aa96c836364a91d81d3034ac54924f239a93e247
languageName: node
linkType: hard
@@ -4797,15 +4815,15 @@ __metadata:
languageName: node
linkType: hard
"@typescript-eslint/eslint-plugin@npm:^8.0.0":
version: 8.25.0
resolution: "@typescript-eslint/eslint-plugin@npm:8.25.0"
"@typescript-eslint/eslint-plugin@npm:^8.31.0":
version: 8.31.0
resolution: "@typescript-eslint/eslint-plugin@npm:8.31.0"
dependencies:
"@eslint-community/regexpp": "npm:^4.10.0"
"@typescript-eslint/scope-manager": "npm:8.25.0"
"@typescript-eslint/type-utils": "npm:8.25.0"
"@typescript-eslint/utils": "npm:8.25.0"
"@typescript-eslint/visitor-keys": "npm:8.25.0"
"@typescript-eslint/scope-manager": "npm:8.31.0"
"@typescript-eslint/type-utils": "npm:8.31.0"
"@typescript-eslint/utils": "npm:8.31.0"
"@typescript-eslint/visitor-keys": "npm:8.31.0"
graphemer: "npm:^1.4.0"
ignore: "npm:^5.3.1"
natural-compare: "npm:^1.4.0"
@@ -4813,8 +4831,8 @@ __metadata:
peerDependencies:
"@typescript-eslint/parser": ^8.0.0 || ^8.0.0-alpha.0
eslint: ^8.57.0 || ^9.0.0
typescript: ">=4.8.4 <5.8.0"
checksum: 10c0/11d63850f5f03b29cd31166f8da111788dc74e46877c2e16a5c488d6c4aa4b6c68c0857b9a396ad920aa7f0f3e7166f4faecbb194c19cd2bb9d3f687c5d2b292
typescript: ">=4.8.4 <5.9.0"
checksum: 10c0/7d78e0cdcc967742752d49d2d38986ee38d0b7ca64af247e5fe0816cea9ae5f1bfa5c126154acc0846af515c4fb1c52c96926ee25c73b4c3f7e6fd73cb6d2b0e
languageName: node
linkType: hard
@@ -4829,19 +4847,19 @@ __metadata:
languageName: node
linkType: hard
"@typescript-eslint/parser@npm:^8.0.0":
version: 8.25.0
resolution: "@typescript-eslint/parser@npm:8.25.0"
"@typescript-eslint/parser@npm:^8.31.0":
version: 8.31.0
resolution: "@typescript-eslint/parser@npm:8.31.0"
dependencies:
"@typescript-eslint/scope-manager": "npm:8.25.0"
"@typescript-eslint/types": "npm:8.25.0"
"@typescript-eslint/typescript-estree": "npm:8.25.0"
"@typescript-eslint/visitor-keys": "npm:8.25.0"
"@typescript-eslint/scope-manager": "npm:8.31.0"
"@typescript-eslint/types": "npm:8.31.0"
"@typescript-eslint/typescript-estree": "npm:8.31.0"
"@typescript-eslint/visitor-keys": "npm:8.31.0"
debug: "npm:^4.3.4"
peerDependencies:
eslint: ^8.57.0 || ^9.0.0
typescript: ">=4.8.4 <5.8.0"
checksum: 10c0/9a54539ba297791f23093ff42a885cc57d36b26205d7a390e114d1f01cc584ce91ac6ead01819daa46b48f873cac6c829fcf399a436610bdbfa98e5cd78148a2
typescript: ">=4.8.4 <5.9.0"
checksum: 10c0/9bd903b3ea4e24bfeb444d7a5c2ed82e591ef5cffc0874c609de854c05d34935cd85543e66678ecdb8e0e3eae2cda2df5c1ba66eb72010632cb9f8779031d56d
languageName: node
linkType: hard
@@ -4865,18 +4883,28 @@ __metadata:
languageName: node
linkType: hard
"@typescript-eslint/type-utils@npm:8.25.0":
version: 8.25.0
resolution: "@typescript-eslint/type-utils@npm:8.25.0"
"@typescript-eslint/scope-manager@npm:8.31.0":
version: 8.31.0
resolution: "@typescript-eslint/scope-manager@npm:8.31.0"
dependencies:
"@typescript-eslint/typescript-estree": "npm:8.25.0"
"@typescript-eslint/utils": "npm:8.25.0"
"@typescript-eslint/types": "npm:8.31.0"
"@typescript-eslint/visitor-keys": "npm:8.31.0"
checksum: 10c0/eae758a24cc578fa351b8bf0c30c50de384292c0b05a58762f9b632d65a009bd5d902d806eccb6b678cc0b09686289fb4f1fd67da7f12d59ad43ff033b35cc4f
languageName: node
linkType: hard
"@typescript-eslint/type-utils@npm:8.31.0":
version: 8.31.0
resolution: "@typescript-eslint/type-utils@npm:8.31.0"
dependencies:
"@typescript-eslint/typescript-estree": "npm:8.31.0"
"@typescript-eslint/utils": "npm:8.31.0"
debug: "npm:^4.3.4"
ts-api-utils: "npm:^2.0.1"
peerDependencies:
eslint: ^8.57.0 || ^9.0.0
typescript: ">=4.8.4 <5.8.0"
checksum: 10c0/b7477a2d239cfd337f7d28641666763cf680a43a8d377a09dc42415f715670d35fbb4e772e103dfe8cd620c377e66bce740106bb3983ee65a739c28fab7325d1
typescript: ">=4.8.4 <5.9.0"
checksum: 10c0/f6938413a583430468b259f6823bb2ab1b5cd77cd6d4e21e1803df70e329046b9579aed5bdc9bdcf4046c8091615a911ac3990859db78d00210bb867915ba37f
languageName: node
linkType: hard
@@ -4894,6 +4922,13 @@ __metadata:
languageName: node
linkType: hard
"@typescript-eslint/types@npm:8.31.0":
version: 8.31.0
resolution: "@typescript-eslint/types@npm:8.31.0"
checksum: 10c0/04130a30aac477d36d6a155399b27773457aeb9b485ef8fb56fee05725b6e36768c9fac7e4d1f073fd16988de0eb7dffc743c3f834ae907cf918cabb075e5cd8
languageName: node
linkType: hard
"@typescript-eslint/typescript-estree@npm:5.62.0":
version: 5.62.0
resolution: "@typescript-eslint/typescript-estree@npm:5.62.0"
@@ -4930,6 +4965,24 @@ __metadata:
languageName: node
linkType: hard
"@typescript-eslint/typescript-estree@npm:8.31.0":
version: 8.31.0
resolution: "@typescript-eslint/typescript-estree@npm:8.31.0"
dependencies:
"@typescript-eslint/types": "npm:8.31.0"
"@typescript-eslint/visitor-keys": "npm:8.31.0"
debug: "npm:^4.3.4"
fast-glob: "npm:^3.3.2"
is-glob: "npm:^4.0.3"
minimatch: "npm:^9.0.4"
semver: "npm:^7.6.0"
ts-api-utils: "npm:^2.0.1"
peerDependencies:
typescript: ">=4.8.4 <5.9.0"
checksum: 10c0/0ec074b2b9c49f80fafea716aa0cc4b05085e65730a3ef7c7d2d39db1657a40b38abe83f22bbe15ac4f6fdf576692f47d2d057347242e6cef5be81d070f55064
languageName: node
linkType: hard
"@typescript-eslint/utils@npm:5.62.0":
version: 5.62.0
resolution: "@typescript-eslint/utils@npm:5.62.0"
@@ -4948,7 +5001,22 @@ __metadata:
languageName: node
linkType: hard
"@typescript-eslint/utils@npm:8.25.0, @typescript-eslint/utils@npm:^8.13.0":
"@typescript-eslint/utils@npm:8.31.0":
version: 8.31.0
resolution: "@typescript-eslint/utils@npm:8.31.0"
dependencies:
"@eslint-community/eslint-utils": "npm:^4.4.0"
"@typescript-eslint/scope-manager": "npm:8.31.0"
"@typescript-eslint/types": "npm:8.31.0"
"@typescript-eslint/typescript-estree": "npm:8.31.0"
peerDependencies:
eslint: ^8.57.0 || ^9.0.0
typescript: ">=4.8.4 <5.9.0"
checksum: 10c0/1fd4f62e16a44a5be2de501f70ba4b2d64479e014370bde7bbc6de6897cf1699766a8b7be4deb9b0328e74c2b4171839336ede4e3c60fec6ac8378b623a75275
languageName: node
linkType: hard
"@typescript-eslint/utils@npm:^8.13.0":
version: 8.25.0
resolution: "@typescript-eslint/utils@npm:8.25.0"
dependencies:
@@ -4983,6 +5051,16 @@ __metadata:
languageName: node
linkType: hard
"@typescript-eslint/visitor-keys@npm:8.31.0":
version: 8.31.0
resolution: "@typescript-eslint/visitor-keys@npm:8.31.0"
dependencies:
"@typescript-eslint/types": "npm:8.31.0"
eslint-visitor-keys: "npm:^4.2.0"
checksum: 10c0/e41e2a9e287d11232cda6126377d1df4de69c6e9dc2a14058819cff15280ec654a3877886a6806728196f299766cfbb0b299eb021c2ce168eb15dff5eb07b51b
languageName: node
linkType: hard
"@ungap/structured-clone@npm:^1.2.0":
version: 1.2.0
resolution: "@ungap/structured-clone@npm:1.2.0"
@@ -6853,6 +6931,8 @@ __metadata:
"@livekit/components-core": "npm:^0.12.0"
"@livekit/components-react": "npm:^2.0.0"
"@livekit/protocol": "npm:^1.33.0"
"@livekit/track-processors": "npm:^0.5.5"
"@mediapipe/tasks-vision": "npm:^0.10.18"
"@opentelemetry/api": "npm:^1.4.0"
"@opentelemetry/core": "npm:^1.25.1"
"@opentelemetry/exporter-trace-otlp-http": "npm:^0.57.0"
@@ -6860,7 +6940,7 @@ __metadata:
"@opentelemetry/sdk-trace-base": "npm:^1.25.1"
"@opentelemetry/sdk-trace-web": "npm:^1.9.1"
"@opentelemetry/semantic-conventions": "npm:^1.25.1"
"@playwright/test": "npm:^1.51.0"
"@playwright/test": "npm:^1.52.0"
"@radix-ui/react-dialog": "npm:^1.0.4"
"@radix-ui/react-slider": "npm:^1.1.2"
"@radix-ui/react-visually-hidden": "npm:^1.0.3"
@@ -6883,8 +6963,8 @@ __metadata:
"@types/react-dom": "npm:^18.3.0"
"@types/sdp-transform": "npm:^2.4.5"
"@types/uuid": "npm:10"
"@typescript-eslint/eslint-plugin": "npm:^8.0.0"
"@typescript-eslint/parser": "npm:^8.0.0"
"@typescript-eslint/eslint-plugin": "npm:^8.31.0"
"@typescript-eslint/parser": "npm:^8.31.0"
"@use-gesture/react": "npm:^10.2.11"
"@vector-im/compound-design-tokens": "npm:^3.0.0"
"@vector-im/compound-web": "npm:^7.2.0"
@@ -6910,10 +6990,10 @@ __metadata:
i18next-parser: "npm:^9.1.0"
jsdom: "npm:^26.0.0"
knip: "npm:^5.27.2"
livekit-client: "npm:2.11.2"
livekit-client: "npm:^2.11.3"
lodash-es: "npm:^4.17.21"
loglevel: "npm:^1.9.1"
matrix-js-sdk: "github:matrix-org/matrix-js-sdk#445981f1bc1e13f80b3061fb26d6f42f49161c24"
matrix-js-sdk: "github:matrix-org/matrix-js-sdk#19b1b901f575755d29d1fe03ca48cbf7c1cae05c"
matrix-widget-api: "npm:1.11.0"
normalize.css: "npm:^8.0.1"
observable-hooks: "npm:^4.2.3"
@@ -6931,7 +7011,7 @@ __metadata:
react-use-measure: "npm:^2.1.1"
rxjs: "npm:^7.8.1"
sass: "npm:^1.42.1"
typescript: "npm:^5.1.6"
typescript: "npm:^5.8.3"
typescript-eslint-language-service: "npm:^5.0.5"
unique-names-generator: "npm:^4.6.0"
vaul: "npm:^1.0.0"
@@ -9280,9 +9360,9 @@ __metadata:
languageName: node
linkType: hard
"livekit-client@npm:2.11.2":
version: 2.11.2
resolution: "livekit-client@npm:2.11.2"
"livekit-client@npm:^2.11.3":
version: 2.11.3
resolution: "livekit-client@npm:2.11.3"
dependencies:
"@livekit/mutex": "npm:1.1.1"
"@livekit/protocol": "npm:1.36.1"
@@ -9293,7 +9373,7 @@ __metadata:
tslib: "npm:2.8.1"
typed-emitter: "npm:^2.1.0"
webrtc-adapter: "npm:^9.0.1"
checksum: 10c0/03a184f3a4f81beefb47a78adb37d3ba3b95a5d1f01e2ebbb83fb3769a120f6b8726ee8ccd96eb634164eeec508850cf05d40e2330fe7c1eda0e3d7f481561f7
checksum: 10c0/d56444f31c107b46ccd5532038ac77bd21038042910619008267c17894f1d3f054262ae2354d89df6fe0ba325aba01909b0612ad4c290906487c40d91641f6e4
languageName: node
linkType: hard
@@ -9350,7 +9430,7 @@ __metadata:
languageName: node
linkType: hard
"loglevel@npm:^1.7.1, loglevel@npm:^1.9.1, loglevel@npm:^1.9.2":
"loglevel@npm:^1.9.1, loglevel@npm:^1.9.2":
version: 1.9.2
resolution: "loglevel@npm:1.9.2"
checksum: 10c0/1e317fa4648fe0b4a4cffef6de037340592cee8547b07d4ce97a487abe9153e704b98451100c799b032c72bb89c9366d71c9fb8192ada8703269263ae77acdc7
@@ -9504,9 +9584,9 @@ __metadata:
languageName: node
linkType: hard
"matrix-js-sdk@github:matrix-org/matrix-js-sdk#445981f1bc1e13f80b3061fb26d6f42f49161c24":
version: 37.3.0
resolution: "matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk.git#commit=445981f1bc1e13f80b3061fb26d6f42f49161c24"
"matrix-js-sdk@github:matrix-org/matrix-js-sdk#19b1b901f575755d29d1fe03ca48cbf7c1cae05c":
version: 37.4.0
resolution: "matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk.git#commit=19b1b901f575755d29d1fe03ca48cbf7c1cae05c"
dependencies:
"@babel/runtime": "npm:^7.12.5"
"@matrix-org/matrix-sdk-crypto-wasm": "npm:^14.0.1"
@@ -9515,7 +9595,7 @@ __metadata:
bs58: "npm:^6.0.0"
content-type: "npm:^1.0.4"
jwt-decode: "npm:^4.0.0"
loglevel: "npm:^1.7.1"
loglevel: "npm:^1.9.2"
matrix-events-sdk: "npm:0.0.1"
matrix-widget-api: "npm:^1.10.0"
oidc-client-ts: "npm:^3.0.1"
@@ -9523,7 +9603,7 @@ __metadata:
sdp-transform: "npm:^2.14.1"
unhomoglyph: "npm:^1.0.6"
uuid: "npm:11"
checksum: 10c0/eaeb21abac46c999d2ac25958680ed4c283c9269380678a08815aaf577b8642254eb80de7dd3d69b9ccc71dd31cda51ad0393088d82b5d6e9f0240ee3e90e67e
checksum: 10c0/68a30a113059ba052b2e66502abcd9805f9a18a1bfd1d209203d728b36508af257a57e6248fb237c7018c81bfbe1ec78fa17aea8968c8af0729ea935398dcf8b
languageName: node
linkType: hard
@@ -10295,27 +10375,27 @@ __metadata:
languageName: node
linkType: hard
"playwright-core@npm:1.51.0":
version: 1.51.0
resolution: "playwright-core@npm:1.51.0"
"playwright-core@npm:1.52.0":
version: 1.52.0
resolution: "playwright-core@npm:1.52.0"
bin:
playwright-core: cli.js
checksum: 10c0/8f5de23088c5e97c00327f356b17e0223181e921baf99f4e38d9a3b18d0693db288f8b5389e96d0cb4a1b55f03870f140dd7346128a0c02ce36d11eb92153841
checksum: 10c0/640945507e6ca2144e9f596b2a6ecac042c2fd3683ff99e6271e9a7b38f3602d415f282609d569456f66680aab8b3c5bb1b257d8fb63a7fc0ed648261110421f
languageName: node
linkType: hard
"playwright@npm:1.51.0":
version: 1.51.0
resolution: "playwright@npm:1.51.0"
"playwright@npm:1.52.0":
version: 1.52.0
resolution: "playwright@npm:1.52.0"
dependencies:
fsevents: "npm:2.3.2"
playwright-core: "npm:1.51.0"
playwright-core: "npm:1.52.0"
dependenciesMeta:
fsevents:
optional: true
bin:
playwright: cli.js
checksum: 10c0/e8509ea500e03e8051fd243f2347ac3196ff8dde4c20ae3aba4cf723e2b647a0158d209fba062995dab90590229a483d723562cf1ea8b2fc11698617027416fd
checksum: 10c0/2c6edf1e15e59bbaf77f3fa0fe0ac975793c17cff835d9c8b8bc6395a3b6f1c01898b3058ab37891b2e4d424bcc8f1b4844fe70d943e0143d239d7451408c579
languageName: node
linkType: hard
@@ -12672,13 +12752,13 @@ __metadata:
languageName: node
linkType: hard
"typescript@npm:^5.1.6":
version: 5.7.3
resolution: "typescript@npm:5.7.3"
"typescript@npm:^5.8.3":
version: 5.8.3
resolution: "typescript@npm:5.8.3"
bin:
tsc: bin/tsc
tsserver: bin/tsserver
checksum: 10c0/b7580d716cf1824736cc6e628ab4cd8b51877408ba2be0869d2866da35ef8366dd6ae9eb9d0851470a39be17cbd61df1126f9e211d8799d764ea7431d5435afa
checksum: 10c0/5f8bb01196e542e64d44db3d16ee0e4063ce4f3e3966df6005f2588e86d91c03e1fb131c2581baf0fb65ee79669eea6e161cd448178986587e9f6844446dbb48
languageName: node
linkType: hard
@@ -12692,13 +12772,13 @@ __metadata:
languageName: node
linkType: hard
"typescript@patch:typescript@npm%3A^5.1.6#optional!builtin<compat/typescript>":
version: 5.7.3
resolution: "typescript@patch:typescript@npm%3A5.7.3#optional!builtin<compat/typescript>::version=5.7.3&hash=5786d5"
"typescript@patch:typescript@npm%3A^5.8.3#optional!builtin<compat/typescript>":
version: 5.8.3
resolution: "typescript@patch:typescript@npm%3A5.8.3#optional!builtin<compat/typescript>::version=5.8.3&hash=5786d5"
bin:
tsc: bin/tsc
tsserver: bin/tsserver
checksum: 10c0/6fd7e0ed3bf23a81246878c613423730c40e8bdbfec4c6e4d7bf1b847cbb39076e56ad5f50aa9d7ebd89877999abaee216002d3f2818885e41c907caaa192cc4
checksum: 10c0/39117e346ff8ebd87ae1510b3a77d5d92dae5a89bde588c747d25da5c146603a99c8ee588c7ef80faaf123d89ed46f6dbd918d534d641083177d5fac38b8a1cb
languageName: node
linkType: hard
@@ -13092,8 +13172,8 @@ __metadata:
linkType: hard
"vite@npm:^5.0.0 || ^6.0.0, vite@npm:^6.0.0":
version: 6.2.5
resolution: "vite@npm:6.2.5"
version: 6.2.6
resolution: "vite@npm:6.2.6"
dependencies:
esbuild: "npm:^0.25.0"
fsevents: "npm:~2.3.3"
@@ -13139,7 +13219,7 @@ __metadata:
optional: true
bin:
vite: bin/vite.js
checksum: 10c0/226bb3c1875e1982559007007580e8d083b81f5289f18e28841d622ba030599e1bd9926adccc8264879e319e9f9e4f48a38a0dc52a5dfcdf2a9cb7313bfc1816
checksum: 10c0/68a2ed3e61bdd654c59b817b4f3203065241c66d1739faa707499130f3007bc3a666c7a8320a4198e275e62b5e4d34d9b78a6533f69e321d366e76f5093b2071
languageName: node
linkType: hard