Compare commits

..
Author SHA1 Message Date
Robin df1156b385 Hide Docker and Actions updates from release notes
To make the release notes more scannable.
2026-11-18 14:06:18 -05:00
Timo 7cf6cb4fee Merge pull request #4437 from element-hq/toger5/background-effects-brush-up
Review from the background effects
2026-11-18 17:36:59 +01:00
Timo K. 23c8cb05c8 Make callViewModel construction consume a Props obj. 2026-11-18 17:17:14 +01:00
Timo K. 89e3ff4c5a reveiw: dediated dismiss 2026-11-18 16:52:05 +01:00
Timo K. 586154772e Update track mute comment 2026-11-18 15:23:29 +01:00
Timo K. 20a174c0b4 review: windowSize$ -> viewSize$ 2026-11-18 15:11:52 +01:00
Timo K. 02eb134e40 setEffect -> ClearEffect 2026-11-18 15:01:57 +01:00
Timo K. 3acf5d9a28 review: toggle comments confusing 2026-11-18 14:58:03 +01:00
Timo K. ac461840a3 comment changes to MediaMenuViewModel 2026-11-18 14:55:28 +01:00
renovate[bot]andJohannes Marbach 2e3f40e08c Update ghcr.io/element-hq/element-web:develop Docker digest to 62e9865 (#4469)
* Update ghcr.io/element-hq/element-web:develop Docker digest to f85350f

* Improve failure analysis

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
2026-11-18 08:28:04 +00:00
Timo K. 438d14d1a1 fix facing mode
Use default fallback instead of inversed check.
2026-11-17 21:11:34 +01:00
Timo d04bfdc624 Merge pull request #4471 from element-hq/toger5/bump-dompurify
Raise dompurify override to 3.4.17
2026-11-17 15:01:22 +01:00
Timo b1f8fcd604 Merge pull request #4470 from element-hq/toger5/bump-oxfmt
Update oxfmt to 0.79.0
2026-11-17 14:43:00 +01:00
Timo 001a015a3f Merge pull request #4467 from element-hq/johannes/stable-get-token
Switch to non-unstable method for /rtc/get_token
2026-11-17 14:32:01 +01:00
Timo K. cee9e02617 oxfmt fix 2026-11-17 14:21:58 +01:00
Timo K.andClaude Opus 5.5 b3bbdc79f5 Raise dompurify override to 3.4.17
posthog-js pulls in dompurify and the lockfile still resolved 3.4.12,
which is affected by GHSA-cg2g-xh64-v4q7, GHSA-mrv2-3pqr-j5jr and
GHSA-c6gr-qvv4-5v3f. Raise the override floor to the fixed release so a
lockfile refresh cannot fall back below it. Resolves to 3.4.18.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 14:19:04 +01:00
Timo K.andClaude Opus 5.5 e91d1387b7 Update oxfmt to 0.79.0
oxfmt 0.66.0 pins tinypool to 2.1.0, which has known vulnerabilities.
0.79.0 pins tinypool 2.2.0. Reformat the two files whose output
changed with the new version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 14:03:00 +01:00
Timo K. 34465d7230 Merge branch 'main' into johannes/stable-get-token 2026-11-17 13:29:09 +01:00
Johannes Marbach 512e885bf2 Merge pull request #4449 from element-hq/johannes/claude-failure-analysis
Add workflow for letting Claude analyse CI failures
2026-11-17 12:54:10 +01:00
Johannes Marbach 629031038f Merge pull request #4466 from element-hq/johannes/stable-delegate-delayed
Use non-unstable function for delegating delayed leaves
2026-11-16 18:06:06 +01:00
Johannes Marbach 3879cd65b5 Merge pull request #4357 from element-hq/johannes/sticky-fixes
Sticky event handling fixes
2026-11-16 13:33:36 +01:00
Johannes Marbach 74500f37ea Switch to non-unstable method for /rtc/get_token 2026-11-16 13:31:24 +01:00
Johannes Marbach c9616efc5b Use non-unstable function for delegating delayed leaves 2026-11-16 12:23:22 +01:00
Johannes Marbach b9546b80f5 Start with current sticky events 2026-11-16 10:47:53 +01:00
Johannes Marbach ad316bbe9a Merge pull request #4464 from element-hq/renovate/livekit-components
Update dependency @livekit/components-core to v0.12.16
2026-11-16 08:54:58 +01:00
renovate[bot] 63f31e711c Update ghcr.io/element-hq/element-web:develop Docker digest to bfb9108 (#4463)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-11-16 03:47:10 +00:00
renovate[bot] 3e864c537a Update dependency @livekit/components-core to v0.12.16 2026-11-16 03:40:16 +00:00
renovate[bot] 3ce3d0e266 Update ghcr.io/element-hq/element-web:develop Docker digest to f7b5906 (#4462)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-11-15 01:53:38 +00:00
renovate[bot] 6f328d7304 Update ghcr.io/element-hq/element-web:develop Docker digest to 00133e9 (#4460)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-11-14 02:17:39 +00:00
Johannes Marbach e5d74102e1 Merge branch 'main' into johannes/sticky-fixes 2026-11-13 15:51:31 +01:00
Johannes Marbach 413b795459 Merge pull request #4452 from element-hq/johannes/agents-fixes
Compact AGENTS.md and split hand-off out into a skill
2026-11-13 14:22:39 +01:00
renovate[bot] e9c1e25542 Update ghcr.io/element-hq/element-web:develop Docker digest to 2901117 (#4453)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-11-13 09:06:50 +00:00
Timo K. 60985885d3 Seperate Options and props for CallViewModel
- props are dependencies (otherwise no call can work)
 - options are optional
2026-11-12 18:27:37 +01:00
Timo f88d8d5830 Merge pull request #4456 from element-hq/toger5/getRTCTrasnport_and_getDomain
Use non deprecated non unstable names for getRTCTransports.
2026-11-12 16:12:02 +01:00
Timo K. 6c66f9a3e7 use non deprecated non unstable names for getRTCTransports.
Also remove getDomain where possible
2026-11-12 16:01:11 +01:00
Johannes Marbach 99f48ff987 Merge pull request #4454 from element-hq/johannes/lk-jwt-service-0.10.0
Bump lk-jwt-service to 0.10.0
2026-11-12 13:01:09 +01:00
Timo 6846c8dd3e Merge pull request #4431 from element-hq/toger5/component-no-store-worker
Keep the store worker out of the component package
2026-11-12 12:27:47 +01:00
Johannes Marbach 28ee707ace Bump lk-jwt-service to 0.10.0 2026-11-12 08:29:26 +01:00
Timo K.andClaude Fable 5.1 3100c1c410 Cover the standalone client's initialisation and session restore
Neither had a test before the move, so the moved lines showed up as
uncovered. The client initialisation is tested through a fake client
and mocked stores: which store it syncs into with and without
IndexedDB, with and without the worker, the fallback to memory when
the IndexedDB store cannot start, clearing the stores only when not
restoring, carrying on without crypto, and counting as ready only
after a sync from the network rather than the one restored from the
store. The session restore is tested for no stored session, a
restored one, one the homeserver no longer knows, and any other
failure.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-11 19:24:21 +01:00
Timo K.andClaude Fable 5.1 fc1e91f54c Keep the store worker out of the component package
The package carried the standalone app's IndexedDB store worker: 11 MB
with a copy of the Matrix SDK and its crypto wasm, referenced by an
absolute path no host could serve, and never used by the component,
which takes the host's client. The bundler emits the worker file as
soon as it loads the `?worker` import, used or not, and the import sat
in the Matrix utilities the component reaches for the room URLs.

The client initialisation moves into a module of its own, with the
worker import, which only the standalone login, registration and
session restore paths import. ClientProvider takes the session loader
through a `loadClient` prop instead of importing it itself, since even
a dynamic import keeps a module in the package: the standalone app
passes a lazy `initSPA` from App.tsx, the component passes nothing,
and given neither a client nor a loader no one is signed in.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-11 19:24:21 +01:00
Johannes Marbach 1ea5f0821d Shorten prompt 2026-11-11 16:40:33 +01:00
Johannes Marbach d5f72f464d Compact AGENTS.md and split hand-off out into a skill 2026-11-11 16:33:46 +01:00
Johannes Marbach 1227b0aa18 Add workflow for letting Claude analyse CI failures 2026-11-11 15:56:36 +01:00
renovate[bot] 96949a57da Update ghcr.io/element-hq/element-web:develop Docker digest to 8dc3767 (#4447)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-11-11 14:47:22 +00:00
Johannes Marbach 2e05ef77a0 Merge pull request #4441 from element-hq/johannes/flakes
Try to fix flaky background tests
2026-11-11 15:37:31 +01:00
Johannes Marbach 73826ec4f9 Reformat 2026-11-11 15:29:03 +01:00
Johannes Marbach 1135c02e6f Merge branch 'main' into johannes/flakes 2026-11-11 13:53:30 +01:00
Johannes MarbachandTimo 9dc425b5f8 Apply batched suggestions from code review
Improve comments

Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>
2026-11-11 13:52:05 +01:00
Johannes Marbach c1b94b62dd Merge pull request #4434 from element-hq/johannes/intent
Stabilise m.call.intent
2026-11-11 11:02:22 +01:00
Johannes Marbach cfd8312e46 Try to fix flaky background tests 2026-11-09 12:20:00 +01:00
Timo K. 889828763c Review from the background effects 2026-11-07 16:18:26 +01:00
Johannes Marbach 783bd946e8 Add tests 2026-10-26 11:58:54 +01:00
Johannes Marbach 9c64316b42 Sticky event handling fixes 2026-10-16 14:47:10 +02:00
74 changed files with 2651 additions and 1809 deletions
+35
View File
@@ -0,0 +1,35 @@
---
name: handoff
description: Implement the first cut of a task and stop for a direction check before the quality pass.
argument-hint: <the task>
disable-model-invocation: true
---
Task: $ARGUMENTS
Stop after the first implementation and hand it back before the quality pass.
## Before the hand-off
- Run the narrowest check that rules out an obviously broken hand-off: the test
file for the touched code, `tsc` on its own, a single story. Not the full
`pnpm lint`, not the whole suite.
- If a wider check is needed before feedback can be useful, keep it to the touched
code and say why it was needed.
- Leave wider refactors, extra tests and documentation polish out. They are
follow-up work, not part of the first hand-off.
## The hand-off
Stop and ask whether the direction is right. The message says:
- what was built and where, in a few lines,
- which check ran and what it returned, verbatim if it failed,
- what was left out on purpose,
- the decisions that could have gone another way.
Do not continue past this point on your own.
## After the user confirms
The skill ends here. Continue with the quality pass as usual.
+3
View File
@@ -105,6 +105,9 @@
- name: "Claude-Review"
description: "Request code review from Claude"
color: "e57157"
- name: "Claude-Analyse-Failures"
description: "Request an analysis of failing tests from Claude"
color: "4ADEC0"
- name: "customer-retainer"
color: "F44A5F"
- name: "dependencies"
@@ -0,0 +1,106 @@
name: Claude failure analysis
on:
pull_request:
types: [labeled]
permissions:
# The action exchanges the GitHub OIDC token for a short-lived Anthropic
# access token. GitHub API access goes through CLAUDE_GH_TOKEN instead.
id-token: write
actions: read
jobs:
analysis:
name: Analyse failures
# Only run on labelling and skip forks.
if: github.event.label.name == 'Claude-Analyse-Failures' && github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-24.04
# The turn limit below is generous, so bound the job by time instead.
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
# We may need commit history.
fetch-depth: 0
persist-credentials: false
# Pre-download job logs and artifacts of failed runs so that Claude doesn't spend cycles on it.
- name: Download logs and artifacts of failed workflow runs
env:
GH_TOKEN: ${{ github.token }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -eu
dir=agent-workspace/ci-artifacts
mkdir -p "$dir"
for run in $(gh run list --commit "$HEAD_SHA" --status failure --json databaseId --jq '.[].databaseId'); do
mkdir -p "$dir/$run"
# Fetch run summary, logs of failed jobs and artifacts.
gh run view "$run" > "$dir/$run/run.txt"
gh run view "$run" --log-failed > "$dir/$run/failed-jobs.log"
gh run download "$run" --dir "$dir/$run" || echo "No artifacts for run $run"
done
# Unpack any (nested) zip archives.
while zips=$(find "$dir" -name '*.zip') && [ -n "$zips" ]; do
echo "$zips" | while IFS= read -r zip; do
unzip -oq "$zip" -d "${zip%.zip}" && rm "$zip"
done
done
find "$dir" -maxdepth 3 -type d
- name: Analyse
uses: anthropics/claude-code-action@6fed3ca145920b639991cb756090506e1bcaf515 # v1.0.245
with:
anthropic_federation_rule_id: fdrl_01XQmWo4nka1ZTqZdsdeScyr
anthropic_organization_id: dfbc9a23-d636-4653-8970-035204ef60fa
anthropic_service_account_id: svac_018Ay5qaA7ZW8X7XsbdekqUM
anthropic_workspace_id: wrkspc_01Aw8LHjBS6fZysu9yqzYifW
# The token used here must grant read access to code, issues, and metadata as well as
# read and write access to pull requests.
github_token: ${{ secrets.CLAUDE_GH_TOKEN }}
use_sticky_comment: true
track_progress: true
additional_permissions: |
actions: read
prompt: |
One or more test jobs failed on this pull request. Find out why and
what would need to change to make them pass.
The details, logs and artifacts of every failed workflow run have
already been downloaded into `agent-workspace/ci-artifacts/<run id>/`:
- `run.txt`: Textual run summary with jobs, steps and failure annotations
- `failed-jobs.log`: Logs of the failed jobs
- Any other file: Unpacked run artifacts.
If `agent-workspace/ci-artifacts` is empty, no workflow failed. Say so
and stop in this case.
Read the commits on this branch with `git log` and `git diff` against
the base branch of the PR.
For each failed workflow run , find out what failed and search the
root cause(s). Stop investigating once direct evidence confirms a root
cause. Devise at least one possible resolution for each root cause.
Report your findings as PR comment. Start your message with "# Automated
CI failure analysis". Then add one section per root cause with:
- The affected workflows
- Evidence from the logs
- The fix proposal(s)
Be brief and concise in all your communication. Do not mention things
workflows that did not fail.
Use Markdown links for anything that can be navigated to on GitHub
such as files, commits, issues, pull requests, workflow runs and
jobs.
# Set a generous turn limit. It's bounded by timeout-minutes above.
claude_args: |
--model claude-fable-5-1
--allowedTools "mcp__github_ci__get_ci_status,mcp__github_ci__get_workflow_run_details,mcp__github_ci__download_job_log,mcp__github__list_commits,mcp__github__get_commit,mcp__github__get_file_contents,mcp__github__search_code,mcp__github__list_workflow_runs,mcp__github__get_workflow_run,mcp__github__list_pull_requests,mcp__github__get_pull_request,mcp__github__get_pull_request_files,mcp__github__list_tags,Bash(git log:*),Bash(git diff:*),Bash(git show:*),Bash(gh run list:*),Bash(gh run view:*),Bash(gh run download:*),Bash(gh pr view:*),Bash(gh pr diff:*),Bash(gh api:*),Bash(docker:*)"
--max-turns 150
@@ -1,4 +1,4 @@
name: Claude
name: Claude review
on:
pull_request:
types: [labeled]
+4
View File
@@ -44,3 +44,7 @@ storybook-static
# Nested checkout made by scripts/layered.sh
/matrix-js-sdk/
# Claude
/.claude/*
!/.claude/skills/
+25 -4
View File
@@ -16,19 +16,35 @@ host's page. It is the MatrixRTC reference implementation.
`@media`, global `i18next` or the `widget` global — take it from a provider.
- The change works standalone, as a widget and as a component. Say what you checked.
- Every gate below is green before you push.
- Hand off after the first implementation, before the quality pass. Then commit on
the user's word where the setup allows it, or hand them the message.
## Read before you
| … | … |
| -------------------------------- | ---------------------------------------------------------- |
| start a task of any size | [docs/agents/workflow.md](docs/agents/workflow.md) |
| touch call logic or a view model | [docs/agents/architecture.md](docs/agents/architecture.md) |
| write any code | [docs/agents/code-style.md](docs/agents/code-style.md) |
| write a test, story or e2e spec | [docs/agents/testing.md](docs/agents/testing.md) |
| open a PR | [CONTRIBUTING.md](CONTRIBUTING.md) |
| implement from a feature spec | [FEATURES_SPEC/AGENTS.md](FEATURES_SPEC/AGENTS.md) |
## Scratch files live in `agent-workspace/`
Git-ignored. One kebab-case subfolder per task, matching the branch topic:
`agent-workspace/<slug>/`.
| File | Purpose |
| --------------------------- | -------------------------------------------------------------------------------- |
| `plan.md` | The concrete approach — files, symbols, edit order — once direction is confirmed |
| `commit-msg.txt` | `git commit -F agent-workspace/<slug>/commit-msg.txt` |
| `pr-body.md` | `gh pr create --body-file agent-workspace/<slug>/pr-body.md` |
| `implementation-summary.md` | What was built, decisions, trade-offs |
| `NN-prompt.md` | Raw prompt text, numbered, when worth keeping |
- Nothing here is durable. The folder is ignored, so anything worth keeping is
promoted into the PR body or a doc before the task ends.
- Never write scratch files to the repo root. Stray `load_test_summary.md` and
`config.json_` files are what this folder prevents.
- Start `pr-body.md` from `.github/PULL_REQUEST_TEMPLATE.md` and fill every
section.
## Gates
@@ -42,6 +58,11 @@ pnpm i18n:check
`knip` fails on dead code; a file deliberately inert ahead of its consumer goes in
`knip.ts` `ignoreFiles` with a reason.
Ready means every gate is green and the change is covered at the layers
[testing.md](docs/agents/testing.md) asks for. Read the diff against
[code-style.md](docs/agents/code-style.md) first. Re-run the whole checklist after
any fix.
## A good PR
- One slice, under ~400 changed lines, green on its own, linked to its issue.
+2 -2
View File
@@ -3,7 +3,7 @@ networks:
services:
auth-service:
image: ghcr.io/element-hq/lk-jwt-service:0.8.0
image: ghcr.io/element-hq/lk-jwt-service:0.10.0
pull_policy: always
hostname: auth-server
environment:
@@ -33,7 +33,7 @@ services:
- ecbackend
auth-service-1:
image: ghcr.io/element-hq/lk-jwt-service:0.8.0
image: ghcr.io/element-hq/lk-jwt-service:0.10.0
pull_policy: always
hostname: auth-server-1
environment:
+2 -2
View File
@@ -19,10 +19,10 @@ services:
- ./backend/playwright_homeserver-othersite.yaml:/data/cfg/homeserver.yaml:Z
element-web:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:37d88b4426ff93a884b31c63d5319a7bcee32580f189c724e6b09f793a6de390
image: ghcr.io/element-hq/element-web:develop@sha256:f85350f2b31a7521a800d9050e9f5c0676d04cfb60675242ccc0c6e25cad7d04
element-web-1:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:37d88b4426ff93a884b31c63d5319a7bcee32580f189c724e6b09f793a6de390
image: ghcr.io/element-hq/element-web:develop@sha256:f85350f2b31a7521a800d9050e9f5c0676d04cfb60675242ccc0c6e25cad7d04
volumes:
# The other site opens the embedded package under test in place of the
# copy its image bundles (the first site opens the dev server's app.
+5 -1
View File
@@ -14,7 +14,11 @@ This folder contains documentation for setup, usage, and development of Element
Entry point: [AGENTS.md](../AGENTS.md).
- [Agent workflow](./agents/workflow.md)
- [Architecture](./agents/architecture.md)
- [Code style](./agents/code-style.md)
- [Testing](./agents/testing.md)
Opt-in skills in [`.claude/skills/`](../.claude/skills/):
- `/handoff <task>` — get the first implementation back for a direction check
before lint, the full suite and polish.
+16 -9
View File
@@ -10,6 +10,13 @@
- Owning a resource: a class taking the scope in its constructor — `MediaDevices`,
`MuteStates`, `TileStore`, `Publisher`, `Connection`. Both styles are current;
don't convert one to the other in passing.
- A factory's required parameters are what the thing is made of: no sensible
default. Several of them go in a props interface, so each can carry a doc
comment. Its trailing `options` bag is how the host wants it to behave: every
field optional, each defaulting to standalone behaviour, so `{}` works. A
value with no default is never an option; a test seam is never an option
either — mock the dependency it comes from. See `createCallViewModel$`, with
`CallViewModelProps` and `CallViewModelOptions`.
- `foo$` is an observable. `Behavior<T>`, an observable with a current value, is
the default for anything a view reads.
@@ -36,15 +43,15 @@ Element Call can be mounted several times inside a host's React tree, so it owns
neither window, URL, document nor router. Each seam defaults to the old standalone
and widget behaviour.
| Never | Use |
| --------------------------------------------- | ------------------------------------------------------------- |
| the `widget` global | `useHostBridge()` — `src/HostBridge.ts` |
| `getUrlParams()`, `window.location` | `useUrlParams()` |
| `useNavigate("/")`, `<Link to="/">` | `useLeaveToHome()`, `LeaveToHomeLink` |
| `document.body` | `useRootElement()` |
| `window.innerWidth/Height`, `useMediaQuery` | `useRootSizeMatches()` in views, `windowSize$` in view models |
| global `i18next` | the instance in `src/utils/i18n.ts` |
| config or analytics reading their environment | `Config.initWith()`, `PosthogAnalytics.configure()` |
| Never | Use |
| --------------------------------------------- | ----------------------------------------------------------- |
| the `widget` global | `useHostBridge()` — `src/HostBridge.ts` |
| `getUrlParams()`, `window.location` | `useUrlParams()` |
| `useNavigate("/")`, `<Link to="/">` | `useLeaveToHome()`, `LeaveToHomeLink` |
| `document.body` | `useRootElement()` |
| `window.innerWidth/Height`, `useMediaQuery` | `useRootSizeMatches()` in views, `viewSize$` in view models |
| global `i18next` | the instance in `src/utils/i18n.ts` |
| config or analytics reading their environment | `Config.initWith()`, `PosthogAnalytics.configure()` |
- View models take values as options, never `getUrlParams()`
(`callViewModelOptionsFromParams`, `CallViewModelOptions.hostBridge`).
-50
View File
@@ -1,50 +0,0 @@
# Agent workflow
## Scratch files live in `agent-workspace/`
Git-ignored. One kebab-case subfolder per task, matching the branch topic:
`agent-workspace/<slug>/`.
| File | Purpose |
| --------------------------- | -------------------------------------------------------------------------------- |
| `plan.md` | The concrete approach — files, symbols, edit order — once direction is confirmed |
| `commit-msg.txt` | `git commit -F agent-workspace/<slug>/commit-msg.txt` |
| `pr-body.md` | `gh pr create --body-file agent-workspace/<slug>/pr-body.md` |
| `implementation-summary.md` | What was built, decisions, trade-offs |
| `NN-prompt.md` | Raw prompt text, numbered, when worth keeping |
- Nothing here is durable. The folder is ignored, so anything worth keeping is
promoted into the PR body, a doc, or a feature spec before the task ends.
- Never write scratch files to the repo root. Stray `load_test_summary.md` and
`config.json_` files are what this folder prevents.
- Start `pr-body.md` from `.github/PULL_REQUEST_TEMPLATE.md` and fill every
section.
- Commit subjects are plain imperative English. No conventional-commits prefixes.
- A `plan.md` and a feature spec sit at different altitudes; neither replaces the
other. A spec in `FEATURES_SPEC/` is durable and deliberately abstract —
behaviour, decisions, acceptance criteria — so the feature can be rebuilt against
a `main` nobody has seen yet. A plan is one slice of it landing on today's
`main`: the paths, symbols and edit order the spec's decisions and criteria must
not name. `FEATURES_SPEC/AGENTS.md` wins wherever the two genuinely overlap.
## Hand off before the quality pass
- Implement the change, run the narrowest check that rules out an obviously broken
handoff, then stop and ask whether the direction is right.
- Full `pnpm lint`, the whole suite, coverage and benchmarks come after the
direction is confirmed.
- Wider refactors, extra tests and documentation polish are follow-up work, not
part of the first handoff.
- If a check is needed before feedback, keep it to the touched code and say why.
## Commit and PR readiness
- Commit once the user confirms direction, or asks for one. Not before. Where the
setup has no git identity, a sandbox included, write the message to
`commit-msg.txt` and hand it over instead.
- Ready means every gate in [AGENTS.md](../../AGENTS.md#gates) is green and the
change is covered at the layers [testing.md](./testing.md) asks for. Read the diff
against [code-style.md](./code-style.md) first.
- Re-run the whole checklist after any fix. Only on green, either way.
- Once a human has started reviewing, fix forward — never force-push a
regeneration over a review in progress.
+2 -2
View File
@@ -107,11 +107,11 @@
"livekit-client": "^2.18.1",
"lodash-es": "^4.17.21",
"loglevel": "^1.9.1",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#dc8610e796c92553c658300a4882dde961a0678a",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#6c8b1b418189919bc69fd9df5e57354e7b3c39cb",
"matrix-widget-api": "^1.18.0",
"node-stdlib-browser": "^1.3.1",
"normalize.css": "^8.0.1",
"oxfmt": "^0.66.0",
"oxfmt": "^0.79.0",
"oxlint": "^1.70.0",
"oxlint-tsgolint": "^0.25.0",
"pako": "^2.0.4",
+111 -122
View File
@@ -16,7 +16,7 @@ overrides:
undici: ^6.27.0
flatted: ^3.4.2
ws: ^8.21.0
dompurify: ^3.4.11
dompurify: ^3.4.17
importers:
@@ -39,7 +39,7 @@ importers:
version: 11.7.12
'@livekit/components-core':
specifier: ^0.12.0
version: 0.12.15(livekit-client@2.23.0(@types/dom-mediacapture-record@1.0.22))(tslib@2.8.1)
version: 0.12.16(livekit-client@2.23.0(@types/dom-mediacapture-record@1.0.22))(tslib@2.8.1)
'@livekit/track-processors':
specifier: ^0.7.1
version: 0.7.2(@types/dom-mediacapture-transform@0.1.11)(livekit-client@2.23.0(@types/dom-mediacapture-record@1.0.22))
@@ -191,8 +191,8 @@ importers:
specifier: ^1.9.1
version: 1.9.2
matrix-js-sdk:
specifier: github:matrix-org/matrix-js-sdk#dc8610e796c92553c658300a4882dde961a0678a
version: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/dc8610e796c92553c658300a4882dde961a0678a
specifier: github:matrix-org/matrix-js-sdk#6c8b1b418189919bc69fd9df5e57354e7b3c39cb
version: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/6c8b1b418189919bc69fd9df5e57354e7b3c39cb
matrix-widget-api:
specifier: ^1.18.0
version: 1.20.0
@@ -203,8 +203,8 @@ importers:
specifier: ^8.0.1
version: 8.0.1
oxfmt:
specifier: ^0.66.0
version: 0.66.0
specifier: ^0.79.0
version: 0.79.0
oxlint:
specifier: ^1.70.0
version: 1.81.0(oxlint-tsgolint@0.25.0)
@@ -943,9 +943,6 @@ packages:
'@floating-ui/core@1.8.0':
resolution: {integrity: sha512-0CIZ5itps/8x7BG8dEIhs53BvCUH2PCoogtakwRTut+Arm58sJooJ0AuZhLw2HJYIR5cMLNPBSS728sPho2khQ==}
'@floating-ui/dom@1.7.6':
resolution: {integrity: sha512-9gZSAI5XM36880PPMm//9dfiEngYoC6Am2izES1FF406YFsjvyBMmeJ2g4SAju3xWwtuynNRFL2s9hgxpLI5SQ==}
'@floating-ui/dom@1.8.0':
resolution: {integrity: sha512-yXSrzeHZBTZadLOlfyhCkJHNeLJnHRnRInwdZ40L7ZiaAtrBwoYlsDrX3v5zB1Utk7CLfzcOVnVVWoXEky7Ceg==}
@@ -1176,8 +1173,8 @@ packages:
'@jridgewell/trace-mapping@0.3.31':
resolution: {integrity: sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw==}
'@livekit/components-core@0.12.15':
resolution: {integrity: sha512-bDkK+jkPMgyT4a5lZmdcuc/hZ8fZUrXW+qQPNHVeSpXCBMzk5q9QWzx+HPtTsPJ8hnncWGfoT81RnXHBsy+Z8g==}
'@livekit/components-core@0.12.16':
resolution: {integrity: sha512-jcXuNeHKGkXgRiT4wniWRpg+TKUSoTL9da01tMxxKGmjp98zB2DVgPmk1fBZfgDVjPBF+jsZe/WkeFAbvJu4Gw==}
engines: {node: '>=18'}
peerDependencies:
livekit-client: ^2.20.1
@@ -1195,8 +1192,8 @@ packages:
'@types/dom-mediacapture-transform': ^0.1.9
livekit-client: ^1.12.0 || ^2.1.0
'@matrix-org/matrix-sdk-crypto-wasm@18.9.0':
resolution: {integrity: sha512-NtLzNDTEfNASBjNCTGZHuk7N0jrQhS6Zx2CM857rCHZYYh/rpbKGQNrJ7azktm4S1P5Jy3WU3nwJTGohGw26qQ==}
'@matrix-org/matrix-sdk-crypto-wasm@18.9.2':
resolution: {integrity: sha512-GHeuSeb5Fj6Wg2+NE/WoLJJs9U3nOzttO+2sj7lOaS5gNBpGIksc48N/Q/kK/Z+Lxh7YMhopB7M8b6HyASXnGQ==}
engines: {node: '>= 18'}
'@mdx-js/react@3.1.1':
@@ -1629,124 +1626,124 @@ packages:
cpu: [x64]
os: [win32]
'@oxfmt/binding-android-arm-eabi@0.66.0':
resolution: {integrity: sha512-2Me9eoptv6ERdEuI2P8AOlYdHHraXebJaM6SC0kc2Dfb+mLrep2db+fedBPKaYn673h/vBgvP4tkOdAbaudX6w==}
'@oxfmt/binding-android-arm-eabi@0.79.0':
resolution: {integrity: sha512-xOCQ/rvokUXs8SxOQY5Wy2ukPaZJZ0PmdJovN0ryB1c8DVEtbcvliLdYtINUYh8QOTpmBOctIisSJk9tvh+J6w==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm]
os: [android]
'@oxfmt/binding-android-arm64@0.66.0':
resolution: {integrity: sha512-u7O+bSSF0HGsDKkQQxBqvLGVepu93RA+JKu+ONqvfh4sCnCEbj31wZj4iG5gk3XfRwrmYj0/8catkO2LcblQKQ==}
'@oxfmt/binding-android-arm64@0.79.0':
resolution: {integrity: sha512-WMLDvCjjchUYtI6zXKOdzTDGm72BaYZiz1/VhB1rrxdakUZRlJh1aMr3CT6ui51uHcWVBycULIXSZHpw6u2pTQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [android]
'@oxfmt/binding-darwin-arm64@0.66.0':
resolution: {integrity: sha512-/ikyMIVjX/sdo7KtjxoEsSUosfPzveVhT9RWMx9yGqFDKFJ89JAEKuEeLBmurDjrkb4w8tOnAdSO3SBaplY3bw==}
'@oxfmt/binding-darwin-arm64@0.79.0':
resolution: {integrity: sha512-+LzSN7w7tULz96ntS1Uqe5oI00XZ0o/m/t3+eSfiQp3aTPnj2eGUafqEWryMLk84j7x9NiF+GEP2YT4/BfClgA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [darwin]
'@oxfmt/binding-darwin-x64@0.66.0':
resolution: {integrity: sha512-q5xUsKeFqawa9NXa6ZGXWimFV19m8MogKPdTaSVDAAk2EQKBmBZRDeluwcl1p8ty/OFc9s9888OKEh3xfPVH0g==}
'@oxfmt/binding-darwin-x64@0.79.0':
resolution: {integrity: sha512-KFiMRhKeiiooaS1dKc4r8AZgOrfOhwVe1yf/aRef+8BajJwtVjMtlsQl/BNRxtkFbq13GjEo/4Q+9YhFGU4o6g==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [darwin]
'@oxfmt/binding-freebsd-x64@0.66.0':
resolution: {integrity: sha512-CR+x4VzMY0pRXLK/xFQ/RzsSFkP5t2Z2mef0QY6OP/rTRcMUoMLCOM62/3Fp/t0K+UDoBKxvMyeb6D0zPMjleA==}
'@oxfmt/binding-freebsd-x64@0.79.0':
resolution: {integrity: sha512-lHVIp0g8ZOFKiqVscvttp+3HJkkRH6XkGgfAA0lVexIlE1mMFVp4V7RBdxz/P9p7T8pSNsYdapFWvdT21Mib9w==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [freebsd]
'@oxfmt/binding-linux-arm-gnueabihf@0.66.0':
resolution: {integrity: sha512-ZEYmO/LbH9tTQCADILHGZE4GeOXOAj2VzedHkASNwjmwlwtutJCLpCJbIs37wRGTFgWRoEcD72jpMX+IBJUGjQ==}
'@oxfmt/binding-linux-arm-gnueabihf@0.79.0':
resolution: {integrity: sha512-Pt35GoxiMUfnr+23cuzk07XQVvISDFxxPcp/aFVB87+RQvJoQaBo/XN7Re9XxgElvbELdyxTLnV9XM2aTfJeUA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm]
os: [linux]
'@oxfmt/binding-linux-arm-musleabihf@0.66.0':
resolution: {integrity: sha512-hNtR9/oU0CeTkq7JnRkmBQwqe17v2ZaAMLC4VcN7IIOWeRyWDk0knSPWS9iiLmtbZ2RRBBtsG01jQgkZmKCJeQ==}
'@oxfmt/binding-linux-arm-musleabihf@0.79.0':
resolution: {integrity: sha512-zPi9Dni58VKp3r1QUcFvnbJNfBr49qyecq0vlM4SaX/MSjmguq77VngirghDua5tyhIDfqdUMTCK2KRlnZFp/g==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm]
os: [linux]
'@oxfmt/binding-linux-arm64-gnu@0.66.0':
resolution: {integrity: sha512-uwOVQ8i6I1LT/+eDzfsgrrcZp8Fn6NPVUPn8fF5gdFGekFf0PddF+LEuwsD0/pbNUcKZhDj2rQ5UpITh9gF4iQ==}
'@oxfmt/binding-linux-arm64-gnu@0.79.0':
resolution: {integrity: sha512-XhRehATEWjwX7VVEgDUD+7pfHYncQIJaEU0qsHvEKtu9XX0QYzcKvXhU9+URfiGX9WRh3wy2Ow+YCc3EeHtImg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-arm64-musl@0.66.0':
resolution: {integrity: sha512-tTkF2Dmx4nGAjmBlZb+UtTGqR/EK4ZrW9qBfzte07a9XWqzoGGKzpFFlyNDhQe+Uwql94+ReCTeNbhOXscw1Dg==}
'@oxfmt/binding-linux-arm64-musl@0.79.0':
resolution: {integrity: sha512-gbVjFZYug0MEyqvuTAzXGNxFJmBqBO32o1Fud1lG8bGs6oVz48ZtNQyEcAzqrOwaaG+Sxtv4gVzcsfht9aqmkg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [musl]
'@oxfmt/binding-linux-ppc64-gnu@0.66.0':
resolution: {integrity: sha512-F3cKHUav4yXOHn6GFnwpBhSYsJOYKKf9eqO/9jlEuqPxNw9zb98E9ZFct79gcg8pibUGkbveEu9WDlmXJpDzKw==}
'@oxfmt/binding-linux-ppc64-gnu@0.79.0':
resolution: {integrity: sha512-0N0sD3vnllyuiNh1r6zKvNXWE/CakcPKJAespg7XE2+sfvxpz5Q1iFah8sF5aedsOmVvbZaOPZcSHVmMosT1fA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-riscv64-gnu@0.66.0':
resolution: {integrity: sha512-K5fDaNZfDyQMYA/3qL21bqyN0X9T15LLwwbFPt2aHc94+ZG7bh0vZEsy2y7NlRnjjHFSwN+Hzg6ldJtbOriH4Q==}
'@oxfmt/binding-linux-riscv64-gnu@0.79.0':
resolution: {integrity: sha512-YtjqUHolcX6cv0ZMZuKQCM4H0ae/okcowy0wGcPHJk4O+8zjy3TvNhzDQ1/KGCkMJO8Fwulm1JcPJsraVJ3p+w==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-riscv64-musl@0.66.0':
resolution: {integrity: sha512-44Yc+I+qOmTElRcEhm5hUKIUJEQIOugymz4ua4tB0Wox7tGAfIbjzmXz/HDAtw1Ij6gmBwZlzh4hc9679RhWeA==}
'@oxfmt/binding-linux-riscv64-musl@0.79.0':
resolution: {integrity: sha512-H2gb9bEBr5tcdP5FU8JYTfgOavXDorWcAAlwci/zEKF2jzgh/oEr/SAw6aHB9n3FLEvnKvOh0A1fjku+vgYoUA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@oxfmt/binding-linux-s390x-gnu@0.66.0':
resolution: {integrity: sha512-1e29Eg9hEj2kRBB19M0seIehPbbXHCk35GvImjDvb79rjjYjXCRmtbUNHJcgoktZAMIzXrTbxDBKmTc1V4bg3A==}
'@oxfmt/binding-linux-s390x-gnu@0.79.0':
resolution: {integrity: sha512-aZUbyWW5dSOxT/u6suWc4S9GVDt7YPPNMKQon+z79M8dpXmosadcFHAb7jBhpU7Hdsd2GN6/y3nd0JFTsGddZw==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-x64-gnu@0.66.0':
resolution: {integrity: sha512-vODY1UQo10gngn0+D4xHKU84F1Twm1LqrzV4SqPXvmQKSd87paehvZ6jqA5wKs6XQrlWul9clYMDVHcoW9CPMA==}
'@oxfmt/binding-linux-x64-gnu@0.79.0':
resolution: {integrity: sha512-5pW/ZHYfjjQ71gQR+EKzDz8ZzWIDHlTUtWtxVFNgZbKxVZnRxiSXVrllbjpJtx1q8VtSPZ+Lyg9BLny3yw+PTA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-x64-musl@0.66.0':
resolution: {integrity: sha512-YDzXx2JsT4+HL4MdkVrYjO55NS5lUKNm8rLC4ZPou8+seu0v0jhecSh+ufoO6+xEa8gccEezMlI2WHJi4ApUgw==}
'@oxfmt/binding-linux-x64-musl@0.79.0':
resolution: {integrity: sha512-3IVUerTDB7Dvd90c1EurAzlVnlE9Azl7Wn8e33qX3wL/IqnslF+P7HvwkD+VeMbHxRtuw0EEBGSArZFXpUAZrQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [musl]
'@oxfmt/binding-openharmony-arm64@0.66.0':
resolution: {integrity: sha512-mJjUYd8lj0+j4JkYyEM+5qKBf1Rnrpgjn/SVYKJhicVDqLz566ooa7Fs8zflPqt+dnZDV7X054rVIQX6ZcQNlQ==}
'@oxfmt/binding-openharmony-arm64@0.79.0':
resolution: {integrity: sha512-C5QDgJBtu9+/2BQScoP7ynv54PLOqfoSqhRWwI0D0RPOwYD4tYj0TUdJbXMO1fNegL/5zJiSOxkq6II2xvttWg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [openharmony]
'@oxfmt/binding-win32-arm64-msvc@0.66.0':
resolution: {integrity: sha512-soV+0vESv7e5ntCHWC61x4gg8OSak6IHHnWsZmHrJFlvMj2AK+kmldErCNkVkrvc1Ts2/++rJXn+IuAb2WMXhw==}
'@oxfmt/binding-win32-arm64-msvc@0.79.0':
resolution: {integrity: sha512-zlW1m8M49vNGgqQF/ze80KW8pZMicnW2XDZGBHuYtXTRYn4ewvgpqN9wwoqb6uLZZfb1R1zaQWKEB8cs7KYjwA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [win32]
'@oxfmt/binding-win32-ia32-msvc@0.66.0':
resolution: {integrity: sha512-YCPi23uRIEYuIKTZohAkKbPFpujQ5QBuUM5iDv+UqbCmTPAkaFsxjsSuB8xlBpRT0G7eP/4HMF+cPDSqHtOD9A==}
'@oxfmt/binding-win32-ia32-msvc@0.79.0':
resolution: {integrity: sha512-wbb7hFv2wK3SCTfu8dl1ydNsXM5dmH+PJMU3sojCkfKcG29VDpITL5/AtouCzXjkOc90QRkcAZcg36gP0KqoHA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [ia32]
os: [win32]
'@oxfmt/binding-win32-x64-msvc@0.66.0':
resolution: {integrity: sha512-bwTQcv/JVRPkOqQtMF0X7vpvpncDQiBcXHxZ9S2hR12Hlo8bvBdUR5x5XnxzDZ3kM0qoZw1rv7KaD66Ly+pFWA==}
'@oxfmt/binding-win32-x64-msvc@0.79.0':
resolution: {integrity: sha512-JLaeaQ+4LkQiVXY06JIV1dBFEDJzssvDieK/3Poi6CsY+c4UxghZVrrsfcKGJ5R/0F2Sw3C5Lf5Hu0sM5qJfPw==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [win32]
@@ -3898,8 +3895,8 @@ packages:
resolution: {integrity: sha512-GrwoxYN+uWlzO8uhUXRl0P+kHE4GtVPfYzVLcUxPL7KNdHKj66vvlhiweIHqYYXWlw+T8iLMp42Lm67ghw4WMQ==}
engines: {node: '>= 4'}
dompurify@3.4.12:
resolution: {integrity: sha512-zQvGet8Z2sWbQhCmfFz/T5QWH2oBmjnqK3qvOjaqaNLrLEF912WamU+ohnTp0TCep/MFVHpdJuCZEdFOdTnEFg==}
dompurify@3.4.18:
resolution: {integrity: sha512-ofe3wI/uNm68HInABYFJYqQxRw1EaKAjNdrFVKqjUY+UZurvNd2L/HuQe097s/4RivWiZyP6tT7aoT8atjfhwg==}
domutils@2.8.0:
resolution: {integrity: sha512-w96Cjofp72M5IIhpjgobBimYEfoPjx1Vx0BSX9P30WBdZW2WIKU0T1Bd0kz2eNZ9ikjKgHbEyKx8BB6H1L3h3A==}
@@ -4679,10 +4676,6 @@ packages:
resolution: {integrity: sha512-ja1E3yCr9i/0hmBVaM0bfwDjnGy8I/s6PP4DFp+yP+a+mrHO4Rm7DtmnqROTUkHIkqffC84YY7AeqX6oFk0WFg==}
engines: {node: '>=18'}
loglevel@1.9.1:
resolution: {integrity: sha512-hP3I3kCrDIMuRwAwHltphhDM1r8i55H33GgqjXbrisuJhF4kRhW1dNuxsRklp4bXl8DSdLaNLuiL4A/LWRfxvg==}
engines: {node: '>= 0.6.0'}
loglevel@1.9.2:
resolution: {integrity: sha512-HgMmCqIJSAKqo68l0rS2AanEWfkxaZ5wNiEFb5ggm08lDs9Xl2KxBlX3PTcaD2chBM1gXAYf491/M2Rv8Jwayg==}
engines: {node: '>= 0.6.0'}
@@ -4734,9 +4727,9 @@ packages:
matrix-events-sdk@0.0.1:
resolution: {integrity: sha512-1QEOsXO+bhyCroIe2/A5OwaxHvBm7EsSQ46DEDn8RBIfQwN5HWBpFvyWWR4QY0KHPPnnJdI99wgRiAl7Ad5qaA==}
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/dc8610e796c92553c658300a4882dde961a0678a:
resolution: {gitHosted: true, integrity: sha512-Ldt/VkCzBtre6m1cn0FqgrzRV0GrgWIoARgQUQO58ef2Edu2ERFCINPzp/psFS5RwoN3RIy9Rm4oPVc1ATkEWA==, tarball: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/dc8610e796c92553c658300a4882dde961a0678a}
version: 45.0.0
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/6c8b1b418189919bc69fd9df5e57354e7b3c39cb:
resolution: {gitHosted: true, tarball: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/6c8b1b418189919bc69fd9df5e57354e7b3c39cb}
version: 45.1.0
engines: {node: '>=22.0.0'}
matrix-widget-api@1.20.0:
@@ -4900,14 +4893,17 @@ packages:
oxc-resolver@11.21.2:
resolution: {integrity: sha512-w5tLwYN3Zo24w5EeWJjJWZOwhYqTtC8PS2B1tIt7BZUuqTIcU07sQValbDw+rq7+AuAGzOHklgK+ifsy4lpXfw==}
oxfmt@0.66.0:
resolution: {integrity: sha512-FfvqR8RFtV6JJpRrpkfqyVCQ7HDvZ/VriWFx7veftCgL1B5ZO9qNr+1rvPieycMQnNfVG0PWyJQiy7p0hq1I5w==}
oxfmt@0.79.0:
resolution: {integrity: sha512-92iLHGxAgtYk6jLp/1ZBnEeABveNduHZgMDQj7tyqO9iafuJG3q+YHJq0R5Aa4YT+E6xswMQXdmPZYMwynS8ig==}
engines: {node: ^20.19.0 || >=22.12.0}
hasBin: true
peerDependencies:
'@astrojs/compiler-rs': ^0.5.0
svelte: ^5.0.0
vite-plus: '*'
peerDependenciesMeta:
'@astrojs/compiler-rs':
optional: true
svelte:
optional: true
vite-plus:
@@ -5791,8 +5787,8 @@ packages:
resolution: {integrity: sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g==}
engines: {node: '>=12.0.0'}
tinypool@2.1.0:
resolution: {integrity: sha512-Pugqs6M0m7Lv1I7FtxN4aoyToKg1C4tu+/381vH35y8oENM/Ai7f7C4StcoK4/+BSw9ebcS8jRiVrORFKCALLw==}
tinypool@2.2.0:
resolution: {integrity: sha512-jBrmx4lYmaC9k/mgPbylxs7kBUxHtD8256up+HjLaDFfXScKJQyil+SWXSvhAtT7XHo+yTVlpB81PGmHP8oLSQ==}
engines: {node: ^20.0.0 || >=22.0.0}
tinyrainbow@2.0.0:
@@ -6920,11 +6916,6 @@ snapshots:
dependencies:
'@floating-ui/utils': 0.2.12
'@floating-ui/dom@1.7.6':
dependencies:
'@floating-ui/core': 1.8.0
'@floating-ui/utils': 0.2.12
'@floating-ui/dom@1.8.0':
dependencies:
'@floating-ui/core': 1.8.0
@@ -7158,11 +7149,11 @@ snapshots:
'@jridgewell/resolve-uri': 3.1.2
'@jridgewell/sourcemap-codec': 1.6.0
'@livekit/components-core@0.12.15(livekit-client@2.23.0(@types/dom-mediacapture-record@1.0.22))(tslib@2.8.1)':
'@livekit/components-core@0.12.16(livekit-client@2.23.0(@types/dom-mediacapture-record@1.0.22))(tslib@2.8.1)':
dependencies:
'@floating-ui/dom': 1.7.6
'@floating-ui/dom': 1.8.0
livekit-client: 2.23.0(@types/dom-mediacapture-record@1.0.22)
loglevel: 1.9.1
loglevel: 1.9.2
rxjs: 7.8.2
tslib: 2.8.1
@@ -7178,7 +7169,7 @@ snapshots:
'@types/dom-mediacapture-transform': 0.1.11
livekit-client: 2.23.0(@types/dom-mediacapture-record@1.0.22)
'@matrix-org/matrix-sdk-crypto-wasm@18.9.0': {}
'@matrix-org/matrix-sdk-crypto-wasm@18.9.2': {}
'@mdx-js/react@3.1.1(@types/react@19.2.18)(react@19.3.1)':
dependencies:
@@ -7476,61 +7467,61 @@ snapshots:
'@oxc-resolver/binding-win32-x64-msvc@11.21.2':
optional: true
'@oxfmt/binding-android-arm-eabi@0.66.0':
'@oxfmt/binding-android-arm-eabi@0.79.0':
optional: true
'@oxfmt/binding-android-arm64@0.66.0':
'@oxfmt/binding-android-arm64@0.79.0':
optional: true
'@oxfmt/binding-darwin-arm64@0.66.0':
'@oxfmt/binding-darwin-arm64@0.79.0':
optional: true
'@oxfmt/binding-darwin-x64@0.66.0':
'@oxfmt/binding-darwin-x64@0.79.0':
optional: true
'@oxfmt/binding-freebsd-x64@0.66.0':
'@oxfmt/binding-freebsd-x64@0.79.0':
optional: true
'@oxfmt/binding-linux-arm-gnueabihf@0.66.0':
'@oxfmt/binding-linux-arm-gnueabihf@0.79.0':
optional: true
'@oxfmt/binding-linux-arm-musleabihf@0.66.0':
'@oxfmt/binding-linux-arm-musleabihf@0.79.0':
optional: true
'@oxfmt/binding-linux-arm64-gnu@0.66.0':
'@oxfmt/binding-linux-arm64-gnu@0.79.0':
optional: true
'@oxfmt/binding-linux-arm64-musl@0.66.0':
'@oxfmt/binding-linux-arm64-musl@0.79.0':
optional: true
'@oxfmt/binding-linux-ppc64-gnu@0.66.0':
'@oxfmt/binding-linux-ppc64-gnu@0.79.0':
optional: true
'@oxfmt/binding-linux-riscv64-gnu@0.66.0':
'@oxfmt/binding-linux-riscv64-gnu@0.79.0':
optional: true
'@oxfmt/binding-linux-riscv64-musl@0.66.0':
'@oxfmt/binding-linux-riscv64-musl@0.79.0':
optional: true
'@oxfmt/binding-linux-s390x-gnu@0.66.0':
'@oxfmt/binding-linux-s390x-gnu@0.79.0':
optional: true
'@oxfmt/binding-linux-x64-gnu@0.66.0':
'@oxfmt/binding-linux-x64-gnu@0.79.0':
optional: true
'@oxfmt/binding-linux-x64-musl@0.66.0':
'@oxfmt/binding-linux-x64-musl@0.79.0':
optional: true
'@oxfmt/binding-openharmony-arm64@0.66.0':
'@oxfmt/binding-openharmony-arm64@0.79.0':
optional: true
'@oxfmt/binding-win32-arm64-msvc@0.66.0':
'@oxfmt/binding-win32-arm64-msvc@0.79.0':
optional: true
'@oxfmt/binding-win32-ia32-msvc@0.66.0':
'@oxfmt/binding-win32-ia32-msvc@0.79.0':
optional: true
'@oxfmt/binding-win32-x64-msvc@0.66.0':
'@oxfmt/binding-win32-x64-msvc@0.79.0':
optional: true
'@oxlint-tsgolint/darwin-arm64@0.25.0':
@@ -9515,7 +9506,7 @@ snapshots:
dependencies:
domelementtype: 2.3.0
dompurify@3.4.12:
dompurify@3.4.18:
optionalDependencies:
'@types/trusted-types': 2.0.7
@@ -10029,7 +10020,7 @@ snapshots:
i18next-resources-for-ts@2.1.0:
dependencies:
'@babel/runtime': 7.29.7
'@babel/runtime': 7.29.10
'@swc/core': 1.16.2
chokidar: 5.0.0
yaml: 2.9.0
@@ -10363,8 +10354,6 @@ snapshots:
is-unicode-supported: 2.1.0
yoctocolors: 2.2.0
loglevel@1.9.1: {}
loglevel@1.9.2: {}
loupe@3.2.1: {}
@@ -10413,10 +10402,10 @@ snapshots:
matrix-events-sdk@0.0.1: {}
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/dc8610e796c92553c658300a4882dde961a0678a:
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/6c8b1b418189919bc69fd9df5e57354e7b3c39cb:
dependencies:
'@babel/runtime': 8.0.8
'@matrix-org/matrix-sdk-crypto-wasm': 18.9.0
'@matrix-org/matrix-sdk-crypto-wasm': 18.9.2
another-json: 0.2.0
bs58: 6.0.0
content-type: 3.1.1
@@ -10674,29 +10663,29 @@ snapshots:
'@oxc-resolver/binding-win32-arm64-msvc': 11.21.2
'@oxc-resolver/binding-win32-x64-msvc': 11.21.2
oxfmt@0.66.0:
oxfmt@0.79.0:
dependencies:
tinypool: 2.1.0
tinypool: 2.2.0
optionalDependencies:
'@oxfmt/binding-android-arm-eabi': 0.66.0
'@oxfmt/binding-android-arm64': 0.66.0
'@oxfmt/binding-darwin-arm64': 0.66.0
'@oxfmt/binding-darwin-x64': 0.66.0
'@oxfmt/binding-freebsd-x64': 0.66.0
'@oxfmt/binding-linux-arm-gnueabihf': 0.66.0
'@oxfmt/binding-linux-arm-musleabihf': 0.66.0
'@oxfmt/binding-linux-arm64-gnu': 0.66.0
'@oxfmt/binding-linux-arm64-musl': 0.66.0
'@oxfmt/binding-linux-ppc64-gnu': 0.66.0
'@oxfmt/binding-linux-riscv64-gnu': 0.66.0
'@oxfmt/binding-linux-riscv64-musl': 0.66.0
'@oxfmt/binding-linux-s390x-gnu': 0.66.0
'@oxfmt/binding-linux-x64-gnu': 0.66.0
'@oxfmt/binding-linux-x64-musl': 0.66.0
'@oxfmt/binding-openharmony-arm64': 0.66.0
'@oxfmt/binding-win32-arm64-msvc': 0.66.0
'@oxfmt/binding-win32-ia32-msvc': 0.66.0
'@oxfmt/binding-win32-x64-msvc': 0.66.0
'@oxfmt/binding-android-arm-eabi': 0.79.0
'@oxfmt/binding-android-arm64': 0.79.0
'@oxfmt/binding-darwin-arm64': 0.79.0
'@oxfmt/binding-darwin-x64': 0.79.0
'@oxfmt/binding-freebsd-x64': 0.79.0
'@oxfmt/binding-linux-arm-gnueabihf': 0.79.0
'@oxfmt/binding-linux-arm-musleabihf': 0.79.0
'@oxfmt/binding-linux-arm64-gnu': 0.79.0
'@oxfmt/binding-linux-arm64-musl': 0.79.0
'@oxfmt/binding-linux-ppc64-gnu': 0.79.0
'@oxfmt/binding-linux-riscv64-gnu': 0.79.0
'@oxfmt/binding-linux-riscv64-musl': 0.79.0
'@oxfmt/binding-linux-s390x-gnu': 0.79.0
'@oxfmt/binding-linux-x64-gnu': 0.79.0
'@oxfmt/binding-linux-x64-musl': 0.79.0
'@oxfmt/binding-openharmony-arm64': 0.79.0
'@oxfmt/binding-win32-arm64-msvc': 0.79.0
'@oxfmt/binding-win32-ia32-msvc': 0.79.0
'@oxfmt/binding-win32-x64-msvc': 0.79.0
oxlint-tsgolint@0.25.0:
optionalDependencies:
@@ -11098,7 +11087,7 @@ snapshots:
'@posthog/core': 1.46.4
'@posthog/types': 1.400.0
core-js: 3.49.0
dompurify: 3.4.12
dompurify: 3.4.18
fflate: 0.4.8
preact: 10.29.8
query-selector-shadow-dom: 1.0.1
@@ -11214,7 +11203,7 @@ snapshots:
react-i18next@17.0.13(i18next@26.4.1(typescript@5.9.3))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)(typescript@5.9.3):
dependencies:
'@babel/runtime': 7.29.7
'@babel/runtime': 7.29.10
html-parse-stringify: 4.0.1
i18next: 26.4.1(typescript@5.9.3)
react: 19.2.8
@@ -11671,7 +11660,7 @@ snapshots:
fdir: 6.5.0(picomatch@4.0.5)
picomatch: 4.0.5
tinypool@2.1.0: {}
tinypool@2.2.0: {}
tinyrainbow@2.0.0: {}
+4 -1
View File
@@ -59,4 +59,7 @@ overrides:
# - https://github.com/element-hq/element-call/security/dependabot/196
# - https://github.com/element-hq/element-call/security/dependabot/195
# - https://github.com/element-hq/element-call/security/dependabot/193
"dompurify": "^3.4.11"
# - https://github.com/advisories/GHSA-cg2g-xh64-v4q7
# - https://github.com/advisories/GHSA-mrv2-3pqr-j5jr
# - https://github.com/advisories/GHSA-c6gr-qvv4-5v3f
"dompurify": "^3.4.17"
+5 -3
View File
@@ -5,7 +5,7 @@
"schedule:monthly",
":enableVulnerabilityAlertsWithLabel(security)"
],
"addLabels": ["dependencies"],
"labels": ["dependencies"],
"minimumReleaseAge": "5 days",
"packageRules": [
{
@@ -16,7 +16,8 @@
{
"groupName": "GitHub Actions",
"matchDepTypes": ["action"],
"pinDigests": true
"pinDigests": true,
"labels": ["PR-Task"]
},
{
"description": "Disable Renovate for packages we want to monitor ourselves",
@@ -69,7 +70,8 @@
"matchUpdateTypes": ["digest"],
"schedule": ["* 0-3 * * *"],
"minimumReleaseAge": null,
"automerge": true
"automerge": true,
"labels": ["PR-Task"]
}
],
"semanticCommits": "disabled",
+11 -9
View File
@@ -128,20 +128,22 @@ export async function createMatrixRTCSdk(
});
vm = createCallViewModel$(
scope,
participation,
room,
mediaDevices,
muteStates,
{
rtcParticipation: participation,
matrixRoom: room,
mediaDevices,
muteStates,
encryptionSystem,
// The bundle owns its page, so the body is the space it has
viewSize$: scope.behavior(observeElementSize$(document.body)),
handsRaisedSubject$: of({}),
reactionsSubject$: of({}),
},
{
...callViewModelOptionsFromParams(urlParams),
encryptionSystem,
capture,
hostBridge,
// The bundle owns its page, so the body is the space it has
windowSize$: scope.behavior(observeElementSize$(document.body)),
},
of({}),
of({}),
);
// Once the hang-up has gone through, unstick and ask the host to close us
vm.leave$
+3 -1
View File
@@ -634,7 +634,9 @@ export type MediaSource =
| "unknown";
export type MediaStreamStats =
RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined;
| RTCInboundRtpStreamStats
| RTCOutboundRtpStreamStats
| undefined;
/** One published track of a member. */
export interface MediaTrack {
+11
View File
@@ -152,6 +152,15 @@ describe("createLocalMedia", () => {
});
});
// As the built-in camera of a Mac
it("takes a camera that doesn't say which way it faces to face the user", async () => {
const media = createLocalMedia(fakeGetUserMedia());
const [camera] = (await media.capture([{ source: "camera" }])) as [
LocalVideoMediaTrack,
];
expect(camera.facingMode$.value).toBe("user");
});
it("releases one device and stops the rest", async () => {
const media = createLocalMedia(fakeGetUserMedia());
const [microphone, camera] = await media.capture([
@@ -162,6 +171,7 @@ describe("createLocalMedia", () => {
media.release("camera");
expect(trackBehind(camera).stop).toHaveBeenCalled();
expect(media.tracks$.value).toEqual([microphone]);
expect(media.requests.map((r) => r.source)).toEqual(["microphone"]);
media.stop();
expect(trackBehind(microphone).stop).toHaveBeenCalled();
@@ -197,6 +207,7 @@ function fakeTrack(
const track = {
kind,
id: `${kind}-${nextId++}`,
label: "",
enabled: true,
stop: vi.fn(),
getSettings: () => settings,
+5 -15
View File
@@ -19,6 +19,7 @@ import {
type VideoProcessor,
} from "./media-api";
import { constant } from "./reactive/Behavior";
import { facingModeOf } from "./utils/facingMode";
import { inTurn } from "./utils/processorTurns";
type Source = "microphone" | "camera";
@@ -107,6 +108,7 @@ export function createLocalMedia(
if (source !== "microphone" && source !== "camera") return;
capturedTracks.get(source)?.stop();
capturedTracks.delete(source);
requests.delete(source);
publishCapturedTracks();
};
@@ -160,9 +162,7 @@ function trackConstraints(request: Request): MediaTrackConstraints {
class CapturedTrack {
public readonly mediaTrack: LocalAudioMediaTrack | LocalVideoMediaTrack;
private readonly muted$ = new BehaviorSubject(false);
private readonly facingMode$: BehaviorSubject<
"user" | "environment" | undefined
>;
private readonly facingMode$: BehaviorSubject<"user" | "environment">;
private readonly elements = new Set<HTMLMediaElement>();
private processor: HandedOverTrack["processor"];
/** What the processor reads frames from; a processor needs a playing element. */
@@ -305,10 +305,9 @@ class CapturedTrack {
kind: "video",
facingMode$: this.facingMode$,
switchFacingMode: async () => {
const facingMode = this.facingMode$.value;
if (facingMode === undefined) return;
await this.reacquire({
facingMode: facingMode === "user" ? "environment" : "user",
facingMode:
this.facingMode$.value === "user" ? "environment" : "user",
});
return this.track.getSettings().deviceId;
},
@@ -324,12 +323,3 @@ function play(element: HTMLMediaElement, track: MediaStreamTrack): void {
const playing = element.play() as Promise<void> | undefined;
playing?.catch(() => undefined);
}
function facingModeOf(
track: MediaStreamTrack,
): "user" | "environment" | undefined {
const { facingMode } = track.getSettings();
return facingMode === "user" || facingMode === "environment"
? facingMode
: undefined;
}
+2 -2
View File
@@ -140,8 +140,8 @@ function setup(
getDeviceId: () => deviceId,
getDomain: () => "example.org",
getSyncState: () => SyncState.Syncing,
_unstable_getRTCTransports: vi.fn().mockResolvedValue(homeserverTransports),
_unstable_getLivekitToken: vi.fn().mockResolvedValue({ jwt: publisherJWT }),
getRTCTransports: vi.fn().mockResolvedValue(homeserverTransports),
getLivekitToken: vi.fn().mockResolvedValue({ jwt: publisherJWT }),
} as unknown as MatrixClient;
const members$ = new BehaviorSubject<RTCMember[]>([]);
+2 -2
View File
@@ -31,7 +31,7 @@ function client(
): Parameters<typeof discoverLocalTransport>[0] {
return {
getDomain: () => "example.org",
_unstable_getRTCTransports: vi.fn().mockResolvedValue(advertised),
getRTCTransports: vi.fn().mockResolvedValue(advertised),
};
}
@@ -54,7 +54,7 @@ describe("discoverLocalTransport", () => {
logger,
),
).resolves.toBe(hostTransport);
expect(c._unstable_getRTCTransports).not.toHaveBeenCalled();
expect(c.getRTCTransports).not.toHaveBeenCalled();
});
it("prefers the homeserver's transport over the host's fallback", async () => {
+1 -1
View File
@@ -22,7 +22,7 @@ import { TransportAutoDiscovery } from "./TransportAutoDiscovery";
* @throws {MatrixRTCTransportMissingError} where none of these exists.
*/
export async function discoverLocalTransport(
client: Pick<MatrixClient, "getDomain" | "_unstable_getRTCTransports">,
client: Pick<MatrixClient, "getDomain" | "getRTCTransports">,
transportTypes: readonly string[],
{
transport: override,
@@ -36,11 +36,7 @@ const configTransport: Transport = {
function makeClient(): MockedObject<DiscoveryClient> {
return {
getDomain: vi.fn().mockReturnValue("example.org"),
baseUrl: "https://matrix.example.org",
_unstable_getRTCTransports: vi.fn().mockResolvedValue([]),
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
getRTCTransports: vi.fn().mockResolvedValue([]),
} as unknown as MockedObject<DiscoveryClient>;
}
@@ -57,7 +53,7 @@ describe("TransportAutoDiscovery", () => {
"prefers backend transport other app config $transports",
async ({ transports }) => {
const client = makeClient();
client._unstable_getRTCTransports.mockResolvedValue(transports);
client.getRTCTransports.mockResolvedValue(transports);
const discovery = new TransportAutoDiscovery({
client,
@@ -71,7 +67,7 @@ describe("TransportAutoDiscovery", () => {
expect(discoveredTransport).toStrictEqual(backendTransport);
expect(discoveredTransport).not.toStrictEqual(configTransport);
expect(client._unstable_getRTCTransports).toHaveBeenCalledTimes(1);
expect(client.getRTCTransports).toHaveBeenCalledTimes(1);
},
);
@@ -81,10 +77,7 @@ describe("TransportAutoDiscovery", () => {
url: "wss://sfu.example.org",
};
const client = makeClient();
client._unstable_getRTCTransports.mockResolvedValue([
backendTransport,
mLivekit,
]);
client.getRTCTransports.mockResolvedValue([backendTransport, mLivekit]);
const discovery = new TransportAutoDiscovery({
client,
@@ -100,7 +93,7 @@ describe("TransportAutoDiscovery", () => {
it("Retries limit_exceeded backend transport", async () => {
const client = makeClient();
client._unstable_getRTCTransports
client.getRTCTransports
.mockRejectedValueOnce(
new MatrixError(
{
@@ -124,7 +117,7 @@ describe("TransportAutoDiscovery", () => {
backendTransport,
);
expect(client._unstable_getRTCTransports).toHaveBeenCalledTimes(2);
expect(client.getRTCTransports).toHaveBeenCalledTimes(2);
});
const INVALID_TEST_CASES: Array<{ transports: Transport[] }> = [
@@ -135,7 +128,7 @@ describe("TransportAutoDiscovery", () => {
"falls back to config when backend has no (valid) livekit transports $transports",
async ({ transports }) => {
const client = makeClient();
client._unstable_getRTCTransports.mockResolvedValue(transports);
client.getRTCTransports.mockResolvedValue(transports);
const discovery = new TransportAutoDiscovery({
client,
@@ -152,7 +145,7 @@ describe("TransportAutoDiscovery", () => {
it("returns null when backend and config are all unavailable", async () => {
const client = makeClient();
client._unstable_getRTCTransports.mockResolvedValue([]);
client.getRTCTransports.mockResolvedValue([]);
const discovery = new TransportAutoDiscovery({
client,
+2 -5
View File
@@ -10,10 +10,7 @@ import { type Logger } from "matrix-js-sdk/lib/logger";
import { doNetworkOperationWithRetry } from "../utils/network";
type TransportDiscoveryClient = Pick<
MatrixClient,
"getDomain" | "_unstable_getRTCTransports"
>;
type TransportDiscoveryClient = Pick<MatrixClient, "getRTCTransports">;
export interface TransportAutoDiscoveryProps {
client: TransportDiscoveryClient;
@@ -74,7 +71,7 @@ export class TransportAutoDiscovery {
this.logger.info("First try to use getRTCTransports end point ...");
try {
const transportList = await doNetworkOperationWithRetry(async () =>
client._unstable_getRTCTransports(),
client.getRTCTransports(),
);
for (const type of this.transportTypes) {
const first = transportList.find(
+6 -2
View File
@@ -151,12 +151,16 @@ export interface LocalAudioMediaTrack
extends AudioMediaTrack, LocalMediaTrack {}
export interface LocalVideoMediaTrack extends VideoMediaTrack, LocalMediaTrack {
/** For mirroring; undefined where the camera does not say which way it faces. */
/**
* For mirroring. A camera that does not say which way it faces is taken to
* face the user, unless it is known to film the room, as a phone used as a
* webcam. Undefined while there is no camera track.
*/
facingMode$: Behavior<"user" | "environment" | undefined>;
/**
* Restarts the camera facing the other way, on devices with a front and a
* back camera, and resolves with the id of the device now in use. Does
* nothing where the facing mode is unknown.
* nothing while there is no camera track.
*/
switchFacingMode(): Promise<string | undefined>;
/** Background blur and the like; undefined removes the processor. */
@@ -92,7 +92,7 @@ function setupTest(): void {
matrix_server_name: "example.org",
expires_in: 3600,
}),
_unstable_getLivekitToken: vi.fn().mockResolvedValue({ jwt: testJWTToken }),
getLivekitToken: vi.fn().mockResolvedValue({ jwt: testJWTToken }),
});
localParticipantEventEmiter = new EventEmitter();
@@ -191,7 +191,7 @@ describe("Start connection states", () => {
const deferred = Promise.withResolvers<LivekitGetTokenResponse>();
client._unstable_getLivekitToken.mockImplementation(
client.getLivekitToken.mockImplementation(
async (): Promise<LivekitGetTokenResponse> => {
return await deferred.promise;
},
@@ -235,7 +235,7 @@ describe("Start connection states", () => {
onTestFinished(() => s.unsubscribe());
const deferredSFU = Promise.withResolvers<void>();
client._unstable_getLivekitToken.mockImplementation(async () => {
client.getLivekitToken.mockImplementation(async () => {
await deferredSFU.promise;
throw new MatrixError(
{
@@ -378,7 +378,7 @@ describe("Start connection states", () => {
await connection.start();
expect(client._unstable_getLivekitToken).not.toHaveBeenCalled();
expect(client.getLivekitToken).not.toHaveBeenCalled();
expect(fetchMock.called(`${serviceUrl}/sfu/get`)).toBe(true);
expect(fakeLivekitRoom.connect).toHaveBeenCalledWith(
"wss://sfu",
@@ -33,7 +33,7 @@ const urlTransport: UnstableLivekitTransport = {
function client(): ClientGetTokenParts & ClientOpenIDParts {
return {
getOpenIdToken: vi.fn(),
_unstable_getLivekitToken: vi.fn().mockResolvedValue({ jwt: testJWTToken }),
getLivekitToken: vi.fn().mockResolvedValue({ jwt: testJWTToken }),
};
}
@@ -160,8 +160,8 @@ describe("createLivekitBackend", () => {
const client = {
baseUrl: homeserver,
getOpenIdToken: vi.fn(),
_unstable_getLivekitToken: vi.fn(),
_unstable_delegateDelayedLeave: vi.fn(),
getLivekitToken: vi.fn(),
delegateDelayedLeave: vi.fn(),
};
/** The connections the stubbed factory handed out, with their participants to drive. */
let connections: {
@@ -73,10 +73,7 @@ export interface LivekitBackendOptions {
*/
client: Pick<
MatrixClient,
| "getOpenIdToken"
| "_unstable_getLivekitToken"
| "_unstable_delegateDelayedLeave"
| "baseUrl"
"getOpenIdToken" | "getLivekitToken" | "delegateDelayedLeave" | "baseUrl"
>;
mediaQuality?: MediaQuality;
}
@@ -18,15 +18,7 @@ import {
TrackEvent,
type TrackPublication,
} from "livekit-client";
import {
afterEach,
beforeEach,
describe,
expect,
it,
onTestFinished,
vi,
} from "vitest";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { ObservableScope } from "../../reactive/ObservableScope";
import {
@@ -233,10 +225,6 @@ describe("createLivekitMediaTrack", () => {
])(
"leaves a processed camera alone unless its device changes: $deviceId",
async ({ deviceId, switches }) => {
vi.stubGlobal("MediaStreamTrack", class {});
onTestFinished(() => {
vi.unstubAllGlobals();
});
const camera = Object.assign(Object.create(LocalVideoTrack.prototype), {
source: Track.Source.Camera,
_mediaStreamTrack: {
@@ -270,6 +258,41 @@ describe("createLivekitMediaTrack", () => {
},
);
it("takes the facing mode of a processed camera from the camera", async () => {
const source = {
readyState: "live",
label: "Back Camera",
getSettings: () => ({ facingMode: "environment" }),
};
const camera = Object.assign(Object.create(LocalVideoTrack.prototype), {
source: Track.Source.Camera,
_mediaStreamTrack: source,
mediaStream: { getVideoTracks: () => [source] },
processor: {
processedTrack: { label: "", getSettings: () => ({}) },
},
restartTrack: vi.fn().mockResolvedValue(undefined),
}) as LocalVideoTrack;
const { participant, publication, room } = fakes({
track: camera as unknown as FakeTrack,
});
const track = createLocalLivekitMediaTrack(
scope,
Object.assign(participant, {
isLocal: true,
}) as unknown as LocalParticipant,
publication as unknown as TrackPublication,
room,
vi.fn(),
) as LocalVideoMediaTrack;
expect(track.facingMode$.value).toBe("environment");
await track.switchFacingMode();
expect(camera.restartTrack).toHaveBeenCalledExactlyOnceWith({
facingMode: "user",
});
});
// LiveKit stops a camera that is turned off; the processor is built when it
// is turned on again
it.each([
@@ -279,10 +302,6 @@ describe("createLivekitMediaTrack", () => {
"gives a processor to a camera that $state: $takes",
async ({ isMuted, takes }) => {
// jsdom has none; LiveKit's facing mode checks against it
vi.stubGlobal("MediaStreamTrack", class {});
onTestFinished(() => {
vi.unstubAllGlobals();
});
const camera = Object.assign(Object.create(LocalVideoTrack.prototype), {
isMuted,
_mediaStreamTrack: {
@@ -11,7 +11,6 @@ import {
roomEventSelector,
} from "@livekit/components-core";
import {
facingModeFromLocalTrack,
type LocalParticipant,
LocalTrack,
LocalVideoTrack,
@@ -50,6 +49,7 @@ import {
type VideoMediaTrack,
} from "../../media-api";
import { MatrixRTCError } from "../../errors";
import { facingModeOf } from "../../utils/facingMode";
import { toLivekitProcessor, type LivekitProcessor } from "./videoProcessor";
export const mediaSources: Record<Track.Source, MediaSource> = {
@@ -231,10 +231,9 @@ export function createLocalLivekitMediaTrack(
switchFacingMode: async () => {
const track = videoTrack();
if (!track) return;
const { facingMode } = facingModeFromLocalTrack(track);
if (facingMode !== "user" && facingMode !== "environment") return;
await track.restartTrack({
facingMode: facingMode === "user" ? "environment" : "user",
facingMode:
facingModeOf(sourceTrack(track)) === "user" ? "environment" : "user",
});
return track.mediaStreamTrack.getSettings().deviceId;
},
@@ -288,6 +287,14 @@ async function rtpStreamStats(
return undefined;
}
/**
* The camera's own track. While a processor runs LiveKit's media stream track
* is the processed one, which says nothing of the camera.
*/
function sourceTrack(track: LocalVideoTrack): MediaStreamTrack {
return track.mediaStream?.getVideoTracks()[0] ?? track.mediaStreamTrack;
}
function facingMode$(
scope: ObservableScope,
track$: Behavior<Track | undefined>,
@@ -298,12 +305,7 @@ function facingMode$(
if (!(track instanceof LocalVideoTrack)) return of(undefined);
return fromEvent(track, TrackEvent.Restarted).pipe(
startWith(null),
map(() => {
const { facingMode } = facingModeFromLocalTrack(track);
return facingMode === "user" || facingMode === "environment"
? facingMode
: undefined;
}),
map(() => facingModeOf(sourceTrack(track))),
);
}),
),
@@ -84,7 +84,7 @@ describe("findRemotePublishLivekitTransport", () => {
beforeEach(() => {
client = {
_unstable_getLivekitToken: vi
getLivekitToken: vi
.fn()
.mockResolvedValue({ jwt: jwtWithPublishGrant(true) }),
};
@@ -92,8 +92,8 @@ describe("findRemotePublishLivekitTransport", () => {
it("publishes on the oldest member's SFU with its server_name", async () => {
await expect(find([carol, bob])).resolves.toEqual(bobsTransport);
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
expect(client._unstable_getLivekitToken).toHaveBeenCalledWith({
expect(client.getLivekitToken).toHaveBeenCalledTimes(1);
expect(client.getLivekitToken).toHaveBeenCalledWith({
url: "wss://sfu.bob.com",
server_name: "bob.com",
room_id: "!room:example.org",
@@ -103,70 +103,68 @@ describe("findRemotePublishLivekitTransport", () => {
});
it("asks about an SFU once however many members are on it", async () => {
client._unstable_getLivekitToken.mockResolvedValue({
client.getLivekitToken.mockResolvedValue({
jwt: jwtWithPublishGrant(false),
});
await expect(find([dave, bobsLaptop, bob])).resolves.toBeNull();
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
expect(client.getLivekitToken).toHaveBeenCalledTimes(1);
});
it("skips our own membership", async () => {
await expect(find([ourselves, bob])).resolves.toEqual(bobsTransport);
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
expect(client.getLivekitToken).toHaveBeenCalledTimes(1);
});
it("ignores members publishing through a legacy JWT service", async () => {
await expect(find([legacyErin, bob])).resolves.toEqual(bobsTransport);
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
expect(client.getLivekitToken).toHaveBeenCalledTimes(1);
});
it("moves on when a homeserver does not let us publish", async () => {
client._unstable_getLivekitToken.mockImplementation(
async ({ server_name }) =>
Promise.resolve({
jwt: jwtWithPublishGrant(server_name === "carol.com"),
}),
client.getLivekitToken.mockImplementation(async ({ server_name }) =>
Promise.resolve({
jwt: jwtWithPublishGrant(server_name === "carol.com"),
}),
);
await expect(find([bob, carol])).resolves.toEqual(carolsTransport);
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(2);
expect(client.getLivekitToken).toHaveBeenCalledTimes(2);
});
it("moves on when a homeserver refuses to issue a token", async () => {
client._unstable_getLivekitToken.mockImplementation(
async ({ server_name }) =>
server_name === "bob.com"
? Promise.reject(
new MatrixError(
{ errcode: "M_FORBIDDEN", error: "Not joined" },
403,
),
)
: Promise.resolve({ jwt: jwtWithPublishGrant(true) }),
client.getLivekitToken.mockImplementation(async ({ server_name }) =>
server_name === "bob.com"
? Promise.reject(
new MatrixError(
{ errcode: "M_FORBIDDEN", error: "Not joined" },
403,
),
)
: Promise.resolve({ jwt: jwtWithPublishGrant(true) }),
);
await expect(find([bob, carol])).resolves.toEqual(carolsTransport);
});
it("finds nothing once all candidates are exhausted", async () => {
client._unstable_getLivekitToken.mockResolvedValue({
client.getLivekitToken.mockResolvedValue({
jwt: jwtWithPublishGrant(false),
});
await expect(find([bob, carol])).resolves.toBeNull();
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(2);
expect(client.getLivekitToken).toHaveBeenCalledTimes(2);
});
it("gives up immediately if our homeserver lacks the CS API", async () => {
client._unstable_getLivekitToken.mockRejectedValue(csApiMissing);
client.getLivekitToken.mockRejectedValue(csApiMissing);
await expect(find([bob, carol])).resolves.toBeNull();
expect(client._unstable_getLivekitToken).toHaveBeenCalledTimes(1);
expect(client.getLivekitToken).toHaveBeenCalledTimes(1);
});
it("finds nothing where there are no other members", async () => {
await expect(find([ourselves])).resolves.toBeNull();
expect(client._unstable_getLivekitToken).not.toHaveBeenCalled();
expect(client.getLivekitToken).not.toHaveBeenCalled();
});
});
+4 -7
View File
@@ -20,10 +20,7 @@ function isNetworkError(e: unknown): boolean {
return e instanceof HTTPError || e instanceof ConnectionError;
}
export type ClientGetTokenParts = Pick<
MatrixClient,
"_unstable_getLivekitToken"
>;
export type ClientGetTokenParts = Pick<MatrixClient, "getLivekitToken">;
export interface GetSFUConfigParams {
client: ClientGetTokenParts;
@@ -48,7 +45,7 @@ export async function getSFUConfig({
}: GetSFUConfigParams): Promise<SFUConfig> {
const res = await doNetworkOperationWithRetry(
async () =>
client._unstable_getLivekitToken({
client.getLivekitToken({
url,
server_name: serverName,
room_id: roomId,
@@ -62,7 +59,7 @@ export async function getSFUConfig({
export type ClientDelegationParts = Pick<
MatrixClient,
"_unstable_delegateDelayedLeave" | "baseUrl"
"delegateDelayedLeave" | "baseUrl"
>;
export interface DelegateDelayedLeaveParams {
@@ -90,7 +87,7 @@ export async function delegateDelayedLeave({
}: DelegateDelayedLeaveParams): Promise<void> {
await doNetworkOperationWithRetry(
async () =>
client._unstable_delegateDelayedLeave({
client.delegateDelayedLeave({
url,
room_id: roomId,
slot_id: slotId,
@@ -33,8 +33,8 @@ const client: Mocked<
openID.ClientOpenIDParts
> = {
baseUrl: "https://matrix.example.org",
_unstable_getLivekitToken: vi.fn().mockResolvedValue({ jwt: csApiJWTToken }),
_unstable_delegateDelayedLeave: vi.fn().mockResolvedValue({}),
getLivekitToken: vi.fn().mockResolvedValue({ jwt: csApiJWTToken }),
delegateDelayedLeave: vi.fn().mockResolvedValue({}),
getOpenIdToken: vi
.fn()
.mockResolvedValue({ access_token: "mock-openID-token" }),
@@ -110,7 +110,7 @@ describe("getSFUConfig", () => {
livekitAlias: "via CS API",
}),
);
expect(client._unstable_getLivekitToken).toHaveBeenCalledWith({
expect(client.getLivekitToken).toHaveBeenCalledWith({
url: urlTransport.url,
server_name: "example.org",
room_id: roomId,
@@ -122,11 +122,11 @@ describe("getSFUConfig", () => {
it("does not fall back to OpenID with url transport", async () => {
mockJWTService();
client._unstable_getLivekitToken.mockRejectedValueOnce(unsupportedError);
client.getLivekitToken.mockRejectedValueOnce(unsupportedError);
await expect(
getSFUConfig({ ...defaultParams, transport: urlTransport }),
).rejects.toThrow(expect.any(NoMatrix2AuthorizationService));
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(client.getLivekitToken).toHaveBeenCalled();
expect(jwtServiceCalls()).toEqual([]);
});
@@ -140,7 +140,7 @@ describe("getSFUConfig", () => {
livekitAlias: "via default endpoint",
}),
);
expect(client._unstable_getLivekitToken).not.toHaveBeenCalled();
expect(client.getLivekitToken).not.toHaveBeenCalled();
expect(jwtServiceCalls()).toEqual(["/get_token"]);
});
@@ -157,7 +157,7 @@ describe("getSFUConfig", () => {
livekitAlias: "via default endpoint",
}),
);
expect(client._unstable_getLivekitToken).not.toHaveBeenCalled();
expect(client.getLivekitToken).not.toHaveBeenCalled();
expect(jwtServiceCalls()).toEqual(["/get_token"]);
});
@@ -166,7 +166,7 @@ describe("getSFUConfig", () => {
const error = new Error(
"The rtc_livekit_get_token action is not supported by the client.",
);
client._unstable_getLivekitToken.mockRejectedValueOnce(error);
client.getLivekitToken.mockRejectedValueOnce(error);
await expect(
getSFUConfig({ ...defaultParams, transport: urlTransport }),
).rejects.toThrow(error);
@@ -182,13 +182,13 @@ describe("getSFUConfig", () => {
role: "publisher",
}),
).toEqual(expect.objectContaining({ jwt: csApiJWTToken }));
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(client.getLivekitToken).toHaveBeenCalled();
expect(jwtServiceCalls()).toEqual([]);
});
it("does not fall back to OpenID when publishing with url transport", async () => {
mockJWTService();
client._unstable_getLivekitToken.mockRejectedValueOnce(unsupportedError);
client.getLivekitToken.mockRejectedValueOnce(unsupportedError);
await expect(
getSFUConfig({
...defaultParams,
@@ -196,7 +196,7 @@ describe("getSFUConfig", () => {
role: "publisher",
}),
).rejects.toThrow(expect.any(NoMatrix2AuthorizationService));
expect(client._unstable_getLivekitToken).toHaveBeenCalled();
expect(client.getLivekitToken).toHaveBeenCalled();
expect(jwtServiceCalls()).toEqual([]);
});
@@ -216,7 +216,7 @@ describe("getSFUConfig", () => {
livekitAlias: "via legacy endpoint",
}),
);
expect(client._unstable_getLivekitToken).not.toHaveBeenCalled();
expect(client.getLivekitToken).not.toHaveBeenCalled();
expect(jwtServiceCalls()).toEqual(["/sfu/get"]);
});
});
@@ -245,7 +245,7 @@ describe("delegateDelayedLeave", () => {
it("uses CS API with url transport", async () => {
mockJWTService();
await delegateDelayedLeave({ ...defaultParams, transport: urlTransport });
expect(client._unstable_delegateDelayedLeave).toHaveBeenCalledWith({
expect(client.delegateDelayedLeave).toHaveBeenCalledWith({
url: urlTransport.url,
room_id: roomId,
slot_id: slotId,
@@ -257,13 +257,11 @@ describe("delegateDelayedLeave", () => {
it("does not fall back to OpenID with url transport", async () => {
mockJWTService();
client._unstable_delegateDelayedLeave.mockRejectedValueOnce(
unsupportedError,
);
client.delegateDelayedLeave.mockRejectedValueOnce(unsupportedError);
await expect(
delegateDelayedLeave({ ...defaultParams, transport: urlTransport }),
).rejects.toThrow();
expect(client._unstable_delegateDelayedLeave).toHaveBeenCalled();
expect(client.delegateDelayedLeave).toHaveBeenCalled();
expect(jwtServiceCalls()).toEqual([]);
});
@@ -273,7 +271,7 @@ describe("delegateDelayedLeave", () => {
...defaultParams,
transport: serviceUrlTransport,
});
expect(client._unstable_delegateDelayedLeave).not.toHaveBeenCalled();
expect(client.delegateDelayedLeave).not.toHaveBeenCalled();
expect(delegationViaJWTService()).toEqual({
delay_id: "delay123",
delay_timeout: 3_600_000,
@@ -288,7 +286,7 @@ describe("delegateDelayedLeave", () => {
...defaultParams,
transport: urlAndServiceUrlTransport,
});
expect(client._unstable_delegateDelayedLeave).not.toHaveBeenCalled();
expect(client.delegateDelayedLeave).not.toHaveBeenCalled();
expect(delegationViaJWTService()).toEqual({
delay_id: "delay123",
delay_timeout: 3_600_000,
+43
View File
@@ -0,0 +1,43 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it } from "vitest";
import { facingModeOf } from "./facingMode";
function camera(label: string, facingMode?: string): MediaStreamTrack {
return {
label,
getSettings: () => (facingMode === undefined ? {} : { facingMode }),
} as MediaStreamTrack;
}
describe("facingModeOf", () => {
it.each([
{ label: "Back Camera", facingMode: "environment", faces: "environment" },
{ label: "iPhone Camera", facingMode: "user", faces: "user" },
{ label: "FaceTime HD Camera", facingMode: undefined, faces: "user" },
{ label: "", facingMode: undefined, faces: "user" },
{
label: "Alice's iPhone Camera",
facingMode: undefined,
faces: "environment",
},
{ label: "iPad Camera", facingMode: undefined, faces: "environment" },
{
label: " OBS Virtual Camera ",
facingMode: undefined,
faces: "environment",
},
{ label: "Integrated Camera", facingMode: "left", faces: "user" },
])(
"takes $label ($facingMode) to face $faces",
({ label, facingMode, faces }) => {
expect(facingModeOf(camera(label, facingMode))).toBe(faces);
},
);
});
+23
View File
@@ -0,0 +1,23 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* Which way a camera faces. Laptop and desktop cameras don't say and face the
* user; of those, a phone used as a webcam (Continuity Camera) films with its
* rear camera and OBS shows a scene, so neither is taken to face the user.
* LiveKit makes the same guess.
*/
export function facingModeOf(track: MediaStreamTrack): "user" | "environment" {
const { facingMode } = track.getSettings();
if (facingMode === "user" || facingMode === "environment") return facingMode;
const label = track.label.trim().toLowerCase();
return label === "obs virtual camera" ||
label.includes("iphone") ||
label.includes("ipad")
? "environment"
: "user";
}
+12 -2
View File
@@ -30,7 +30,7 @@ import { HomePage } from "./home/HomePage";
import { LoginPage } from "./auth/LoginPage";
import { RegisterPage } from "./auth/RegisterPage";
import { RoomPage } from "./room/RoomPage";
import { ClientProvider } from "./ClientContext";
import { type ClientLoader, ClientProvider } from "./ClientContext";
import { ErrorPage, LoadingPage } from "./FullScreenView";
import { Initializer } from "./initializer";
import { type WidgetHelpers } from "./widget";
@@ -57,6 +57,16 @@ import { LeaveToHomeProvider } from "./LeaveToHomeContext";
const SentryRoute = Sentry.withSentryReactRouterV7Routing(Route);
// Element Call's own session, for when no host supplies a client.
// - Passed in here, not imported by ClientProvider: this module is only in the
// app builds, so the client setup and the store worker behind it stay out of
// the component package (see the `loadClient` prop).
// - Imported on demand, as it was before, so that a widget, which gets its
// client from the host, never downloads it.
// - Module-level, so that the provider's effect sees one stable loader.
const loadStandaloneClient: ClientLoader = async (...args) =>
(await import("./utils/spa")).initSPA(...args);
interface SimpleProviderProps {
children: JSX.Element;
}
@@ -146,7 +156,7 @@ export const App: FC<Props> = ({ vm, widget }) => {
const content =
loaded && clientReady ? (
<ClientProvider client={widgetClient}>
<ClientProvider client={widgetClient} loadClient={loadStandaloneClient}>
<MediaDevicesContext value={vm.mediaDevices}>
<BackgroundEffectsProvider effects={vm.backgroundEffects}>
<Sentry.ErrorBoundary
+25 -6
View File
@@ -11,7 +11,11 @@ import { BrowserRouter } from "react-router-dom";
import { type MatrixClient } from "matrix-js-sdk";
import { type FC } from "react";
import { ClientProvider, useClientState } from "./ClientContext";
import {
type ClientLoader,
ClientProvider,
useClientState,
} from "./ClientContext";
const mockClient = (userId = "@alice:example.org"): MatrixClient =>
({
@@ -95,13 +99,13 @@ test("follows the client when the host swaps it", () => {
test("finds a client of its own when the host supplies none", async () => {
const client = mockClient();
vi.doMock("./utils/spa", () => ({
initSPA: vi.fn().mockResolvedValue({ client, passwordlessUser: true }),
}));
const loadClient = vi
.fn<ClientLoader>()
.mockResolvedValue({ client, passwordlessUser: true });
const { container } = render(
<BrowserRouter>
<ClientProvider>
<ClientProvider loadClient={loadClient}>
<ShowClientState />
</ClientProvider>
</BrowserRouter>,
@@ -110,6 +114,21 @@ test("finds a client of its own when the host supplies none", async () => {
// Nothing to show until a session has been restored or created
expect(container.textContent).toBe("loading");
await waitFor(() => expect(container.textContent).toBe("@alice:example.org"));
// Given the session store to restore from and to clear on a stale session
expect(loadClient).toHaveBeenCalledWith(
expect.any(Function),
expect.any(Function),
);
});
vi.doUnmock("./utils/spa");
test("has no one signed in when given neither a client nor a way to load one", async () => {
const { container } = render(
<BrowserRouter>
<ClientProvider>
<ShowClientState />
</ClientProvider>
</BrowserRouter>,
);
await waitFor(() => expect(container.textContent).toBe("unauthenticated"));
});
+32 -12
View File
@@ -143,9 +143,34 @@ interface Props {
* own.
*/
client?: MatrixClient;
/**
* How Element Call finds a client when none is supplied: the standalone app
* restores or creates a session of its own. Stable, as a hook dependency.
* Left out along with the client, no one is signed in.
*
* A prop, rather than this module importing `utils/spa` itself, because:
* - This module is in the component build, and so is everything it imports,
* dynamically or not: a dynamic import becomes a chunk in the package,
* not a module left out of it.
* - Behind `utils/spa` sits `utils/initClient` and the client store worker,
* which the bundler emits as soon as it loads the import (see there).
* - `App.tsx`, which the component never imports, is where the standalone
* app passes the loader in; the component passes nothing, having the
* host's client.
*/
loadClient?: ClientLoader;
}
export const ClientProvider: FC<Props> = ({ children, client }) => {
/**
* Restores or creates a session of Element Call's own, kept where the given
* functions keep it. Resolves to null when no one is signed in.
*/
export type ClientLoader = (
loadSession: () => Session | undefined,
clearSession: () => void,
) => Promise<InitResult | null>;
export const ClientProvider: FC<Props> = ({ children, client, loadClient }) => {
const leaveToHome = useLeaveToHome();
const hostBridge = useHostBridge();
@@ -179,9 +204,13 @@ export const ClientProvider: FC<Props> = ({ children, client }) => {
// React does in strict mode), we need to make sure not to doubly initialize
// the client.
if (initializing.current) return;
if (loadClient === undefined) {
setInitClientState(null);
return;
}
initializing.current = true;
loadClient()
loadClient(loadSession, clearSession)
.then((initResult) => {
setInitClientState(initResult);
if (PosthogAnalytics.instance.isEnabled())
@@ -189,7 +218,7 @@ export const ClientProvider: FC<Props> = ({ children, client }) => {
})
.catch((err) => logger.error(err))
.finally(() => (initializing.current = false));
}, [client]);
}, [client, loadClient]);
const changePassword = useCallback(
async (password: string) => {
@@ -360,15 +389,6 @@ export type InitResult = {
passwordlessUser: boolean;
};
/**
* Restores or creates a session of Element Call's own. Only reached when no
* client was supplied for it to use.
*/
async function loadClient(): Promise<InitResult | null> {
const { initSPA } = await import("./utils/spa");
return initSPA(loadSession, clearSession);
}
export interface Session {
user_id: string;
device_id: string;
+1 -1
View File
@@ -13,7 +13,7 @@ import {
type MatrixClient,
} from "matrix-js-sdk";
import { initClient } from "../utils/matrix";
import { initClient } from "../utils/initClient";
import { type Session } from "../ClientContext";
/**
* This provides the login method to login using user credentials.
+1 -1
View File
@@ -14,7 +14,7 @@ import {
} from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import { initClient } from "../utils/matrix";
import { initClient } from "../utils/initClient";
import { type Session } from "../ClientContext";
import { Config } from "../config/Config";
import { useUrlParams } from "../UrlParams";
+1 -1
View File
@@ -45,7 +45,7 @@ interface Props {
/** Offers to add a background of one's own. Omit to leave that tile out. */
onAdd?: () => void;
addLabel?: string;
/** Removes a removable one, never the one in force. Omit to offer none. */
/** Removes a removable one, unless it is the selected one. Omit to offer none. */
onRemove?: (id: string) => void;
removeLabel?: string;
}
+68 -193
View File
@@ -22,7 +22,7 @@ import { MediaDevices } from "../state/MediaDevices";
import { globalScope } from "../state/ObservableScope";
import { constant } from "../state/Behavior";
import { type LayoutMode } from "../state/LayoutSwitchViewModel";
import { shippedBackgrounds } from "../tracks/backgroundEffectCatalog";
import { type MediaMenuSnapshot } from "./MediaMenuViewModel";
// consts for tests
const reactionIdentifier = "@user:example.com:DEVICE";
@@ -35,10 +35,57 @@ const mediaDevices = new MediaDevices(globalScope, {
controlledAudioDevices: false,
});
function menu(
kind: "audio" | "video",
withDevices: boolean,
): MediaMenuSnapshot {
const device = kind === "audio" ? "Microphone" : "Camera";
return {
enabled: true,
busy: false,
toggle: fn(),
options: withDevices
? [
{ id: "1", label: { type: "name", name: `${device} 1` } },
{ id: "2", label: { type: "name", name: `${device} 2` } },
]
: [],
selected: "1",
select: fn(),
outputOptions:
withDevices && kind === "audio"
? [
{
id: "default",
label: { type: "default", name: "Built-in Output" },
},
]
: [],
selectedOutput: "default",
selectOutput: fn(),
backgroundEffect: "none",
backgroundEffects:
kind === "video"
? [
{ id: "none", kind: "none" },
{ id: "blur", kind: "blur" },
]
: [],
backgroundEffectNotice: undefined,
backgroundEffectSettling: false,
backgroundImageRefusal: undefined,
selectBackgroundEffect: fn(),
addBackgroundImage: undefined,
removeBackgroundEffect: undefined,
cameraTrack: null,
};
}
/**
* A wrapper component that is used for:
* - exposing the snapshot via props so the storybook documents the snapshot properties (basically unpack them form the vm)
* - constructing the layout switch view model
* - constructing the layout switch and the menus' view models; the menus'
* own states are in the MediaMenuButton stories
* - Add additional react context
* The paraeters are all params from the FooterSnapshot,
* the Snapshot of the vm, the wrapper will create a mocked vm from it and pass it to the CallFooter.
@@ -49,15 +96,22 @@ function CallFooterStoryWrapper({
children,
layout,
setLayout,
withDevices,
...vmSnapshot
}: Omit<FooterSnapshot, "layoutSwitchVm"> & {
}: Omit<FooterSnapshot, "layoutSwitchVm" | "audioMenuVm" | "cameraMenuVm"> & {
children?: false | JSX.Element | JSX.Element[] | undefined;
layout: LayoutMode | null;
setLayout: (value: LayoutMode) => void;
/** Whether the menus have devices to pick, or are plain mute buttons. */
withDevices: boolean;
}): ReactNode {
const audioMenuVm = useStaticViewModel(menu("audio", withDevices));
const cameraMenuVm = useStaticViewModel(menu("video", withDevices));
const vm = useStaticViewModel({
...vmSnapshot,
layoutSwitchVm: layout && { layout$: constant(layout), setLayout },
audioMenuVm,
cameraMenuVm,
});
return (
<MediaDevicesContext value={mediaDevices}>
@@ -102,8 +156,6 @@ const meta = {
},
toggleScreenSharing: fnArgType,
openSettings: fnArgType,
toggleAudio: fnArgType,
toggleVideo: fnArgType,
hangup: fnArgType,
},
} satisfies Meta<typeof CallFooterStoryWrapper>;
@@ -116,32 +168,9 @@ export const Default: Story = {
showLogo: false,
layout: "grid",
setLayout: fn(),
audioEnabled: true,
audioBusy: false,
videoEnabled: true,
videoBusy: false,
withDevices: false,
openSettings: fn(),
toggleAudio: fn(),
toggleVideo: fn(),
toggleScreenSharing: fn(),
selectBackgroundEffect: fn(),
backgroundEffect: "blur",
backgroundEffects: [
{ id: "none", kind: "none", removable: false },
{ id: "blur", kind: "blur", removable: false },
...shippedBackgrounds.map((background) => ({
id: `image:${background.id}`,
kind: "image" as const,
imageUrl: background.imagePath,
removable: false,
})),
],
backgroundEffectNotice: undefined,
backgroundEffectSettling: false,
backgroundImageRefusal: undefined,
addBackgroundImage: fn(),
removeBackgroundEffect: fn(),
cameraTrack: null,
hangup: fn(),
buttonSize: "lg",
showFooter: true,
@@ -154,49 +183,17 @@ export const Default: Story = {
reactionData: undefined,
debugTileLayout: false,
tileStoreGeneration: undefined,
audioOptions: [],
audioOutputOptions: [],
videoOptions: [],
selectedAudio: undefined,
selectedAudioOutput: undefined,
selectedVideo: undefined,
selectAudioButtonOption: undefined,
selectAudioOutputOption: undefined,
selectVideoButtonOption: undefined,
},
parameters: {
layout: "fullscreen",
},
};
export const WithAudioAndVideoOptions: Story = {
...Default,
args: {
...Default.args,
audioEnabled: false,
videoEnabled: true,
audioOptions: [
{ label: { type: "name", name: "Microphone 1" }, id: "1" },
{ label: { type: "name", name: "Microphone 2" }, id: "2" },
],
audioOutputOptions: [
{ label: { type: "default", name: "Built-in Output" }, id: "default" },
{ label: { type: "name", name: "Headset" }, id: "2" },
],
videoOptions: [
{ label: { type: "name", name: "Camera 1" }, id: "1" },
{ label: { type: "name", name: "Camera 2" }, id: "2" },
],
selectedAudio: "2",
selectedAudioOutput: "default",
selectedVideo: "1",
},
};
export const StaysWhileAMenuIsOpen: Story = {
...Default,
args: {
...WithAudioAndVideoOptions.args,
...Default.args,
withDevices: true,
// In a short window the footer overlays the call and hides itself.
asOverlay: true,
showFooter: true,
@@ -219,25 +216,6 @@ export const StaysWhileAMenuIsOpen: Story = {
},
};
export const AudioBusy: Story = {
...Default,
args: {
...Default.args,
audioEnabled: true,
audioBusy: true,
videoEnabled: true,
},
};
export const VideoBusy: Story = {
...Default,
args: {
...Default.args,
audioEnabled: true,
videoEnabled: true,
videoBusy: true,
},
};
export const WithLogo: Story = {
...Default,
args: {
@@ -246,45 +224,7 @@ export const WithLogo: Story = {
},
};
export const AudioVideoEnabled: Story = {
...Default,
args: {
...Default.args,
audioEnabled: true,
videoEnabled: true,
},
play: async ({ args, canvasElement }) => {
const canvas = within(canvasElement);
const spotlightRadio = canvas.getByRole("radio", { name: "Spotlight" });
await userEvent.click(spotlightRadio);
await expect(args.setLayout).toHaveBeenCalledWith("spotlight");
const micButtonMute = canvas.getByRole("switch", {
name: "Mute microphone",
});
await userEvent.click(micButtonMute);
await expect(args.toggleAudio).toHaveBeenCalled();
const videoMuteButton = canvas.getByRole("switch", {
name: "Stop video",
});
await userEvent.click(videoMuteButton);
await expect(args.toggleVideo).toHaveBeenCalled();
const screenShare = canvas.getByRole("switch", {
name: "Share screen",
});
await userEvent.click(screenShare);
await expect(args.toggleScreenSharing).toHaveBeenCalled();
const endCall = canvas.getByRole("button", {
name: "End call",
});
await userEvent.click(endCall);
await expect(args.hangup).toHaveBeenCalled();
},
};
/** used to test switching to grid mode */
/** Switching to grid, and the footer's own buttons, act. */
export const SpotlightMode: Story = {
...Default,
args: {
@@ -293,10 +233,12 @@ export const SpotlightMode: Story = {
},
play: async ({ args, canvasElement }) => {
const canvas = within(canvasElement);
const spotlightRadio = canvas.getByRole("radio", { name: "Grid" });
await userEvent.click(spotlightRadio);
await userEvent.click(canvas.getByRole("radio", { name: "Grid" }));
await expect(args.setLayout).toHaveBeenCalledWith("grid");
await userEvent.click(canvas.getByRole("switch", { name: "Share screen" }));
await expect(args.toggleScreenSharing).toHaveBeenCalled();
await userEvent.click(canvas.getByRole("button", { name: "End call" }));
await expect(args.hangup).toHaveBeenCalled();
},
};
@@ -330,34 +272,10 @@ export const Pip: Story = {
buttonSize: "md",
layout: null,
},
play: async ({ args, canvasElement }) => {
const canvas = within(canvasElement);
play: async ({ canvasElement }) => {
await expect(
canvas.queryByRole("radio", { name: "Spotlight" }),
within(canvasElement).queryByRole("radio", { name: "Spotlight" }),
).not.toBeInTheDocument();
const micButtonMute = canvas.getByRole("switch", {
name: "Mute microphone",
});
await userEvent.click(micButtonMute);
await expect(args.toggleAudio).toHaveBeenCalled();
const videoMuteButton = canvas.getByRole("switch", {
name: "Stop video",
});
await userEvent.click(videoMuteButton);
await expect(args.toggleVideo).toHaveBeenCalled();
const screenShare = canvas.getByRole("switch", {
name: "Share screen",
});
await userEvent.click(screenShare);
await expect(args.toggleScreenSharing).toHaveBeenCalled();
const endCall = canvas.getByRole("button", {
name: "End call",
});
await userEvent.click(endCall);
await expect(args.hangup).toHaveBeenCalled();
},
};
export const NoControlsWithLogo: Story = {
@@ -378,18 +296,6 @@ export const DebugData: Story = {
},
};
export const UnavailableMediaDevices: Story = {
...Default,
args: {
...Default.args,
audioEnabled: false,
videoEnabled: false,
toggleAudio: undefined,
toggleVideo: undefined,
audioOutputSwitcher: undefined,
},
};
export const MobileLayout: Story = {
...Default,
args: {
@@ -420,37 +326,6 @@ export const Lobby: Story = {
},
};
export const LobbyMobile: Story = {
...Default,
args: {
...Default.args,
showLogo: false,
layout: null,
toggleScreenSharing: undefined,
},
globals: {
viewport: { value: "mobile2", isRotated: false },
},
parameters: {
...Default.parameters,
},
};
export const LobbyRecentButton: Story = {
...Default,
args: {
...Default.args,
children: <Link>Back To Recents</Link>,
showLogo: false,
layout: null,
toggleScreenSharing: undefined,
},
parameters: {
...Default.parameters,
},
};
export const LobbyRecentButtonMobile: Story = {
...Default,
args: {
+20 -165
View File
@@ -7,14 +7,11 @@ Please see LICENSE in the repository root for full details.
import { type FC, type JSX, type Ref, useMemo } from "react";
import classNames from "classnames";
import { useTranslation } from "react-i18next";
import LogoMark from "../icons/LogoMark.svg?react";
import LogoType from "../icons/LogoType.svg?react";
import {
EndCallButton,
MicButton,
VideoButton,
ShareScreenButton,
SettingsButton,
ReactionToggleButton,
@@ -23,23 +20,13 @@ import {
type ReactionData,
} from "../button";
import styles from "./CallFooter.module.css";
import {
MediaMuteAndSwitchButton,
type MenuOptions,
} from "./MediaMuteAndSwitchButton";
import { type Behavior } from "../state/Behavior";
import { type ViewModel } from "../state/ViewModel";
import { useBehavior } from "../useBehavior";
import { type LayoutSwitchViewModel } from "../state/LayoutSwitchViewModel";
import { LayoutSwitch } from "../room/LayoutSwitch";
import { type BackgroundEffectGridItem } from "./BackgroundEffectGrid";
import { SelfPreview } from "./SelfPreview";
import { type UnusableReason } from "../tracks/backgroundImages";
import { type LocalVideoMediaTrack } from "@element-hq/matrixrtc-sdk";
export interface BackgroundImageRefusal {
reason: UnusableReason | "not-kept";
}
import { type MediaMenuViewModel } from "./MediaMenuViewModel";
import { MediaMenuButton } from "./MediaMenuButton";
export interface AudioOutputSwitcher {
targetOutput: string;
@@ -61,16 +48,6 @@ export interface AudioOutputSwitcher {
*/
export type FooterSnapshot = FooterActions & FooterState;
export interface FooterActions {
/** Also controls if the audioMute button is disabled */
toggleAudio: (() => void) | undefined;
/** Also controls if the videoMute button is disabled */
toggleVideo: (() => void) | undefined;
/** Undefined where background effects can't be chosen. */
selectBackgroundEffect: ((id: string) => void) | undefined;
/** Undefined where no background can be added. */
addBackgroundImage: ((file: File) => void) | undefined;
/** Removes an added background by its option id; never the one in force. */
removeBackgroundEffect: ((id: string) => void) | undefined;
toggleScreenSharing: (() => void) | undefined;
/** Also controls if the settings button is visible */
openSettings: (() => void) | undefined;
@@ -80,22 +57,8 @@ export interface FooterActions {
// we do not use any ? optional properties so that the vm type is including all fields.
export interface FooterState {
audioEnabled: boolean;
audioBusy: boolean;
videoEnabled: boolean;
videoBusy: boolean;
/** The background effect in force, as its option id. */
backgroundEffect: string;
/** Every effect on offer, in the order they are shown. */
backgroundEffects: BackgroundEffectGridItem[];
/** What the user is told about effects here, if anything. */
backgroundEffectNotice: "unavailable" | "slow" | undefined;
/** Whether the first effect chosen is still being prepared. */
backgroundEffectSettling: boolean;
/** Why the last file offered couldn't be kept, if it couldn't. */
backgroundImageRefusal: BackgroundImageRefusal | undefined;
/** The camera to preview, with the effect in force; null while it is off. */
cameraTrack: LocalVideoMediaTrack | null;
audioMenuVm: MediaMenuViewModel;
cameraMenuVm: MediaMenuViewModel;
showFooter: boolean;
/* This is needed for WindowMode = "flat" */
@@ -122,19 +85,6 @@ export interface FooterState {
// debug stuff
debugTileLayout: boolean;
tileStoreGeneration: number | undefined;
/** Providing no options `[]` or `undefined` will imply that we dont have a audio fast switcher */
audioOptions: MenuOptions[];
/** Output (speaker) devices. */
audioOutputOptions: MenuOptions[];
/** Providing no options `[]` or `undefined` will imply that we dont have a audio fast switcher */
videoOptions: MenuOptions[];
selectedAudio: string | undefined;
selectedAudioOutput: string | undefined;
selectedVideo: string | undefined;
selectAudioButtonOption: ((deviceId: string) => void) | undefined;
selectAudioOutputOption: ((deviceId: string) => void) | undefined;
selectVideoButtonOption: ((option: string) => void) | undefined;
}
export interface FooterProps {
@@ -150,19 +100,14 @@ export const CallFooter: FC<FooterProps> = ({
children,
vm,
}) => {
const { t } = useTranslation();
const asOverlay = useBehavior(vm.asOverlay$);
const showFooter = useBehavior(vm.showFooter$);
const hideControls = useBehavior(vm.hideControls$);
const showModals = useBehavior(vm.showModals$);
const layoutSwitchVm = useBehavior(vm.layoutSwitchVm$);
const audioMenuVm = useBehavior(vm.audioMenuVm$);
const cameraMenuVm = useBehavior(vm.cameraMenuVm$);
const openSettings = useBehavior(vm.openSettings$);
const audioEnabled = useBehavior(vm.audioEnabled$);
const audioBusy = useBehavior(vm.audioBusy$);
const videoEnabled = useBehavior(vm.videoEnabled$);
const videoBusy = useBehavior(vm.videoBusy$);
const toggleAudio = useBehavior(vm.toggleAudio$);
const toggleVideo = useBehavior(vm.toggleVideo$);
const sharingScreen = useBehavior(vm.sharingScreen$);
const toggleScreenSharing = useBehavior(vm.toggleScreenSharing$);
const reactionIdentifier = useBehavior(vm.reactionIdentifier$);
@@ -170,38 +115,6 @@ export const CallFooter: FC<FooterProps> = ({
const audioOutputSwitcher = useBehavior(vm.audioOutputSwitcher$);
const hangup = useBehavior(vm.hangup$);
const debugTileLayout = useBehavior(vm.debugTileLayout$);
const videoOptions = useBehavior(vm.videoOptions$);
const selectedVideo = useBehavior(vm.selectedVideo$);
const audioOptions = useBehavior(vm.audioOptions$);
const selectedAudio = useBehavior(vm.selectedAudio$);
const selectAudioButtonOption = useBehavior(vm.selectAudioButtonOption$);
const audioOutputOptions = useBehavior(vm.audioOutputOptions$);
const selectedAudioOutput = useBehavior(vm.selectedAudioOutput$);
const selectAudioOutputOption = useBehavior(vm.selectAudioOutputOption$);
const selectVideoButtonOption = useBehavior(vm.selectVideoButtonOption$);
const backgroundEffect = useBehavior(vm.backgroundEffect$);
const selectBackgroundEffect = useBehavior(vm.selectBackgroundEffect$);
const backgroundEffectNotice = useBehavior(vm.backgroundEffectNotice$);
const backgroundEffectSettling = useBehavior(vm.backgroundEffectSettling$);
const addBackgroundImage = useBehavior(vm.addBackgroundImage$);
const removeBackgroundEffect = useBehavior(vm.removeBackgroundEffect$);
const cameraTrack = useBehavior(vm.cameraTrack$);
const refusal = useBehavior(vm.backgroundImageRefusal$);
const refusalMessage = useMemo(() => {
switch (refusal?.reason) {
case undefined:
return undefined;
case "not-an-image":
return { text: t("error.background_not_an_image") };
case "animated":
return { text: t("error.background_animated") };
case "undecodable":
return { text: t("error.background_undecodable") };
case "not-kept":
return { text: t("error.background_not_kept") };
}
}, [refusal, t]);
const backgroundEffects = useBehavior(vm.backgroundEffects$);
const buttonSize = useBehavior(vm.buttonSize$);
const showLogo = useBehavior(vm.showLogo$);
@@ -221,78 +134,20 @@ export const CallFooter: FC<FooterProps> = ({
);
}
if ((audioOptions?.length ?? 0) > 0) {
buttons.push(
<MediaMuteAndSwitchButton
key="audio"
iconsAndLabels="audio"
enabled={audioEnabled ?? false}
busy={audioBusy ?? false}
onMuteClick={toggleAudio}
data-testid="incall_mute"
options={audioOptions}
selectedOption={selectedAudio}
onSelect={selectAudioButtonOption}
outputOptions={audioOutputOptions}
selectedOutputOption={selectedAudioOutput}
onSelectOutput={selectAudioOutputOption}
/>,
);
} else {
buttons.push(
<MicButton
size={buttonSize}
key="audio"
enabled={audioEnabled ?? false}
busy={audioBusy ?? false}
onClick={toggleAudio}
disabled={(audioBusy ?? false) || toggleAudio === undefined}
data-testid="incall_mute"
/>,
);
}
if ((videoOptions?.length ?? 0) > 0) {
buttons.push(
<MediaMuteAndSwitchButton
key="video"
iconsAndLabels="video"
enabled={videoEnabled ?? false}
busy={videoBusy ?? false}
onMuteClick={toggleVideo}
options={videoOptions}
selectedOption={selectedVideo}
onSelect={selectVideoButtonOption}
backgroundEffects={backgroundEffects}
selectedBackgroundEffect={backgroundEffect}
onSelectBackgroundEffect={selectBackgroundEffect}
backgroundEffectSettling={backgroundEffectSettling}
onAddBackgroundImage={addBackgroundImage}
onRemoveBackgroundEffect={removeBackgroundEffect}
selfPreview={<SelfPreview track={cameraTrack} />}
backgroundImageRefusal={refusalMessage}
backgroundEffectNotice={
backgroundEffectNotice === "unavailable"
? t("background_effects.unavailable")
: backgroundEffectNotice === "slow"
? t("background_effects.slow")
: undefined
}
/>,
);
} else {
buttons.push(
<VideoButton
size={buttonSize}
key="video"
enabled={videoEnabled ?? false}
busy={videoBusy ?? false}
onClick={toggleVideo}
disabled={(videoBusy ?? false) || toggleVideo === undefined}
data-testid="incall_videomute"
/>,
);
}
buttons.push(
<MediaMenuButton
key="audio"
kind="audio"
vm={audioMenuVm}
size={buttonSize}
/>,
<MediaMenuButton
key="video"
kind="video"
vm={cameraMenuVm}
size={buttonSize}
/>,
);
if (toggleScreenSharing !== undefined) {
buttons.push(
+63 -446
View File
@@ -5,18 +5,15 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, describe, expect, it, vi } from "vitest";
import { BehaviorSubject } from "rxjs";
import { type LocalVideoMediaTrack } from "@element-hq/matrixrtc-sdk";
import { describe, expect, it, vi } from "vitest";
import {
flushPromises,
testScope,
mockMuteStates,
mockMediaDevices,
mockVideoTrack,
} from "../utils/test";
import { type Behavior, constant } from "../state/Behavior";
import { constant } from "../state/Behavior";
import { type BackgroundEffectsState } from "../state/BackgroundEffects";
import type { CallViewModel } from "../state/CallViewModel/CallViewModel";
import type { Layout } from "../state/layout-types";
@@ -27,15 +24,6 @@ import {
createLobbyFooterViewModel,
} from "./CallFooterViewModel";
import { HeaderStyle } from "../UrlParams";
import { backgroundEffect as backgroundEffectSetting } from "../settings/settings";
import { shippedBackgrounds } from "../tracks/backgroundEffectCatalog";
import type * as BackgroundImages from "../tracks/backgroundImages";
import {
type AddedBackground,
addedBackgrounds,
maxAddedBackgrounds,
UnusableImage,
} from "../tracks/backgroundImages";
const platformMock = vi.hoisted(() => vi.fn(() => "desktop"));
vi.mock("../Platform", () => ({
@@ -44,35 +32,6 @@ vi.mock("../Platform", () => ({
},
}));
// The SDK's own check needs WebGL and canvas APIs jsdom does not have.
const sdkSupportMock = vi.hoisted(() => vi.fn(() => false));
const modernRouteMock = vi.hoisted(() => vi.fn(() => true));
vi.mock("@livekit/track-processors", () => ({
supportsBackgroundProcessors: (): boolean => sdkSupportMock(),
supportsModernBackgroundProcessors: (): boolean => modernRouteMock(),
}));
const store = vi.hoisted(() => ({
add: vi.fn<(file: Blob) => Promise<string>>(),
remove: vi.fn<(id: string) => Promise<void>>(),
}));
vi.mock("../tracks/backgroundImages", async (original) => {
const { BehaviorSubject } = await import("rxjs");
return {
...(await original<typeof BackgroundImages>()),
addedBackgrounds: {
added$: new BehaviorSubject<AddedBackground[] | undefined>([]),
add: store.add,
remove: store.remove,
},
};
});
const outputSelectionMock = vi.hoisted(() => vi.fn(() => true));
vi.mock("livekit-client", () => ({
supportsAudioOutputSelection: (): boolean => outputSelectionMock(),
}));
/**
* Returns the minimum set of CallViewModel fields required by
* createCallFooterViewModel, with all other properties stubbed to
@@ -113,424 +72,82 @@ const pipLayout: Layout = {
spotlight: {} as SpotlightTileViewModel,
};
const noPipeline = constant<BackgroundEffectsState>({
supported: false,
const pipeline = constant<BackgroundEffectsState>({
supported: true,
processor: undefined,
});
/** The provider's verdict, which the SDK's check stands for here. */
function pipelineAsTheSdkSays(): Behavior<BackgroundEffectsState> {
return constant({ supported: sdkSupportMock(), processor: undefined });
}
const twoMicsAndOneCamMediaDevices = mockMediaDevices({
audioInput: {
available$: constant(
new Map<string, DeviceLabel>([
["mic1", { type: "number", number: 1 }],
["mic2", { type: "name", name: "Microphone 2" }],
]),
),
selected$: constant(undefined),
select: vi.fn(),
},
videoInput: {
available$: constant(
new Map<string, DeviceLabel>([
["cam1", { type: "name", name: "Camera 1" }],
]),
),
selected$: constant(undefined),
select: vi.fn(),
},
const oneOfEach = {
available$: constant(
new Map<string, DeviceLabel>([["1", { type: "number", number: 1 }]]),
),
selected$: constant(undefined),
select: vi.fn(),
};
const devices = mockMediaDevices({
audioInput: oneOfEach,
videoInput: oneOfEach,
});
describe("createCallFooterViewModel", () => {
describe("selectAudioOutputOption", () => {
function buildFooterVm(): ReturnType<typeof createCallFooterViewModel> {
platformMock.mockReturnValue("desktop");
return createCallFooterViewModel(
testScope(),
buildMinimalCallViewModel(gridLayout),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
noPipeline,
/* reactionIdentifier */ undefined,
{ showControls: true, header: HeaderStyle.Standard },
);
}
it("is withheld where the platform cannot route audio to a chosen device", () => {
outputSelectionMock.mockReturnValue(false);
expect(buildFooterVm().selectAudioOutputOption$.value).toBeUndefined();
});
it("is offered where the platform can route audio to a chosen device", () => {
outputSelectionMock.mockReturnValue(true);
expect(buildFooterVm().selectAudioOutputOption$.value).toBeDefined();
});
});
describe("audioOutputOptions", () => {
it("is an empty list, not absent, where the platform enumerates no outputs", () => {
platformMock.mockReturnValue("desktop");
outputSelectionMock.mockReturnValue(true);
const vm = createCallFooterViewModel(
testScope(),
buildMinimalCallViewModel(gridLayout),
mockMuteStates(),
mockMediaDevices({
audioInput: {
available$: constant(
new Map<string, DeviceLabel>([
["mic1", { type: "name", name: "Microphone 1" }],
]),
),
selected$: constant(undefined),
select: vi.fn(),
},
// As Safari: no outputs listed.
audioOutput: {
available$: constant(new Map<string, DeviceLabel>()),
selected$: constant(undefined),
select: vi.fn(),
},
}),
noPipeline,
/* reactionIdentifier */ undefined,
{ showControls: true, header: HeaderStyle.Standard },
);
// Empty, not undefined: undefined would hide the section.
expect(vm.audioOutputOptions$.value).toEqual([]);
});
});
describe("audioOptions and videoOptions", () => {
function checkEmptyFor(platform: string, layout: Layout): void {
it.each([
["desktop", "grid", true, gridLayout],
["ios", "grid", false, gridLayout],
["desktop", "pip", false, pipLayout],
] as const)(
"on %s in a %s layout, offers the device menus: %s",
(platform, _, offered, layout) => {
platformMock.mockReturnValue(platform);
const vm = createCallFooterViewModel(
testScope(),
buildMinimalCallViewModel(layout),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
noPipeline,
devices,
pipeline,
/* reactionIdentifier */ undefined,
{ showControls: true, header: HeaderStyle.Standard },
);
const audio = vm.audioMenuVm$.value;
const camera = vm.cameraMenuVm$.value;
expect(audio.options$.value.length > 0).toBe(offered);
expect(camera.options$.value.length > 0).toBe(offered);
expect(camera.selectBackgroundEffect$.value !== undefined).toBe(offered);
},
);
expect(vm.audioOptions$.value).toEqual([]);
expect(vm.videoOptions$.value).toEqual([]);
}
it("are both empty when the platform is iOS", () => {
checkEmptyFor("ios", gridLayout);
});
it("are both empty when the layout is pip", () => {
checkEmptyFor("desktop", pipLayout);
});
it("are populated when the platform is desktop and the layout is not PiP", () => {
platformMock.mockReturnValue("desktop");
const vm = createCallFooterViewModel(
testScope(),
buildMinimalCallViewModel(gridLayout),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
noPipeline,
/* reactionIdentifier */ undefined,
{ showControls: true, header: HeaderStyle.Standard },
);
expect(vm.audioOptions$?.value).toEqual([
{
id: "mic1",
label: {
number: 1,
type: "number",
},
},
{
id: "mic2",
label: {
name: "Microphone 2",
type: "name",
},
},
]);
expect(vm.videoOptions$?.value).toEqual([
{
id: "cam1",
label: {
name: "Camera 1",
type: "name",
},
},
]);
});
it("previews the call's camera", () => {
const track = mockVideoTrack({});
const vm = createCallFooterViewModel(
testScope(),
{ ...buildMinimalCallViewModel(gridLayout), camera$: constant(track) },
mockMuteStates(),
devices,
pipeline,
/* reactionIdentifier */ undefined,
{ showControls: true, header: HeaderStyle.Standard },
);
expect(vm.cameraMenuVm$.value.cameraTrack$.value).toBe(track);
});
describe("background effects", () => {
afterEach(() => backgroundEffectSetting.setValue("none"));
// The lobby, because in a call the whole device switcher is already
// withheld on a phone, whatever the verdict says.
function lobbyOn(
platform: string,
): ReturnType<typeof createLobbyFooterViewModel> {
platformMock.mockReturnValue(platform);
return createLobbyFooterViewModel(
testScope(),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
pipelineAsTheSdkSays(),
/* camera$ */ constant(null),
/* openSettings */ undefined,
/* hangup */ undefined,
/* showLogo */ false,
);
}
it("offers them on a phone whose browser can run them", () => {
sdkSupportMock.mockReturnValue(true);
expect(lobbyOn("ios").selectBackgroundEffect$.value).toBeDefined();
});
it("offers them on a desktop whose browser can run them", () => {
sdkSupportMock.mockReturnValue(true);
expect(lobbyOn("desktop").selectBackgroundEffect$.value).toBeDefined();
});
it("offers nothing where the browser itself cannot run them", () => {
sdkSupportMock.mockReturnValue(false);
expect(lobbyOn("desktop").selectBackgroundEffect$.value).toBeUndefined();
expect(lobbyOn("desktop").addBackgroundImage$.value).toBeUndefined();
});
it("puts no effect in force where the browser cannot run them", () => {
backgroundEffectSetting.setValue("blur");
sdkSupportMock.mockReturnValue(false);
expect(lobbyOn("desktop").backgroundEffect$.value).toBe("none");
sdkSupportMock.mockReturnValue(true);
expect(lobbyOn("desktop").backgroundEffect$.value).toBe("blur");
});
it("stores the effect chosen", () => {
sdkSupportMock.mockReturnValue(true);
const vm = lobbyOn("desktop");
vm.selectBackgroundEffect$.value?.("image:arc");
expect(vm.backgroundEffect$.value).toBe("image:arc");
// What isn't an effect on offer is stored as none.
vm.selectBackgroundEffect$.value?.("image:gone");
expect(backgroundEffectSetting.getValue()).toBe("none");
});
it("says they run slowly where only the slower route exists", () => {
sdkSupportMock.mockReturnValue(true);
modernRouteMock.mockReturnValue(false);
const slow = lobbyOn("desktop");
expect(slow.backgroundEffectNotice$.value).toBe("slow");
expect(slow.selectBackgroundEffect$.value).toBeDefined();
modernRouteMock.mockReturnValue(true);
expect(lobbyOn("desktop").backgroundEffectNotice$.value).toBeUndefined();
});
it("offers every effect in order, the added ones removable", () => {
sdkSupportMock.mockReturnValue(true);
const added$ = addedBackgrounds.added$ as BehaviorSubject<
AddedBackground[] | undefined
>;
added$.next([{ id: "mine", url: "blob:mine" }]);
expect(lobbyOn("desktop").backgroundEffects$.value).toEqual([
{ id: "none", kind: "none", removable: false },
{ id: "blur", kind: "blur", removable: false },
...shippedBackgrounds.map((background) => ({
id: `image:${background.id}`,
kind: "image",
imageUrl: background.imagePath,
removable: false,
})),
{
id: "added:mine",
kind: "image",
imageUrl: "blob:mine",
removable: true,
},
]);
added$.next([]);
});
it("says they are unavailable where they cannot be chosen", () => {
sdkSupportMock.mockReturnValue(false);
expect(lobbyOn("desktop").backgroundEffectNotice$.value).toBe(
"unavailable",
);
});
it("passes on the wait for the first effect", () => {
platformMock.mockReturnValue("desktop");
const pipeline$ = new BehaviorSubject<BackgroundEffectsState>({
supported: true,
processor: undefined,
wait: "settling",
});
const vm = createLobbyFooterViewModel(
testScope(),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
pipeline$,
/* camera$ */ constant(null),
/* openSettings */ undefined,
/* hangup */ undefined,
/* showLogo */ false,
);
expect(vm.backgroundEffectSettling$.value).toBe(true);
pipeline$.next({ ...pipeline$.value, wait: undefined });
expect(vm.backgroundEffectSettling$.value).toBe(false);
});
it("offers nothing once the pipeline fails to build, keeping the choice", () => {
platformMock.mockReturnValue("desktop");
backgroundEffectSetting.setValue("blur");
const pipeline$ = new BehaviorSubject<BackgroundEffectsState>({
supported: true,
processor: undefined,
});
const vm = createLobbyFooterViewModel(
testScope(),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
pipeline$,
/* camera$ */ constant(null),
/* openSettings */ undefined,
/* hangup */ undefined,
/* showLogo */ false,
);
expect(vm.backgroundEffect$.value).toBe("blur");
pipeline$.next({ supported: false, processor: undefined });
expect(vm.selectBackgroundEffect$.value).toBeUndefined();
expect(vm.addBackgroundImage$.value).toBeUndefined();
expect(vm.backgroundEffect$.value).toBe("none");
expect(backgroundEffectSetting.getValue()).toBe("blur");
});
it("adding is unavailable at the limit", () => {
sdkSupportMock.mockReturnValue(true);
const added$ = addedBackgrounds.added$ as BehaviorSubject<
AddedBackground[] | undefined
>;
const vm = lobbyOn("desktop");
expect(vm.addBackgroundImage$.value).toBeDefined();
added$.next(
Array.from({ length: maxAddedBackgrounds }, (_, i) => ({
id: `${i}`,
url: `blob:${i}`,
})),
);
expect(vm.addBackgroundImage$.value).toBeUndefined();
added$.next(added$.value!.slice(1));
expect(vm.addBackgroundImage$.value).toBeDefined();
added$.next([]);
});
it("says why each refused file was refused", async () => {
sdkSupportMock.mockReturnValue(true);
const vm = lobbyOn("desktop");
store.add.mockRejectedValue(new UnusableImage("animated"));
vm.addBackgroundImage$.value!(new File(["x"], "a.gif"));
await flushPromises();
const first = vm.backgroundImageRefusal$.value;
expect(first).toEqual({ reason: "animated" });
// The same again is a refusal of its own, so it is shown again.
vm.addBackgroundImage$.value!(new File(["x"], "a.gif"));
await flushPromises();
expect(vm.backgroundImageRefusal$.value).toEqual({ reason: "animated" });
expect(vm.backgroundImageRefusal$.value).not.toBe(first);
store.add.mockRejectedValue(new Error("quota"));
vm.addBackgroundImage$.value!(new File(["x"], "b.png"));
await flushPromises();
expect(vm.backgroundImageRefusal$.value).toEqual({ reason: "not-kept" });
store.add.mockResolvedValue("kept");
vm.addBackgroundImage$.value!(new File(["x"], "c.png"));
await flushPromises();
expect(vm.backgroundImageRefusal$.value).toBeUndefined();
});
it("never removes the background in force", () => {
sdkSupportMock.mockReturnValue(true);
store.remove.mockResolvedValue();
const vm = lobbyOn("desktop");
backgroundEffectSetting.setValue("added:worn");
vm.removeBackgroundEffect$.value!("added:worn");
expect(store.remove).not.toHaveBeenCalled();
vm.removeBackgroundEffect$.value!("added:spare");
expect(store.remove).toHaveBeenCalledWith("spare");
});
it("previews the lobby's camera while there is one", () => {
const track = mockVideoTrack({});
const camera$ = new BehaviorSubject<LocalVideoMediaTrack | null>(track);
const vm = createLobbyFooterViewModel(
testScope(),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
noPipeline,
camera$,
/* openSettings */ undefined,
/* hangup */ undefined,
/* showLogo */ false,
);
expect(vm.cameraTrack$.value).toBe(track);
// The lobby releases the camera when it is turned off
camera$.next(null);
expect(vm.cameraTrack$.value).toBeNull();
});
it("previews the call's camera only while it is not muted", () => {
const muted$ = new BehaviorSubject(false);
const track = mockVideoTrack({ muted$ });
const vm = createCallFooterViewModel(
testScope(),
{ ...buildMinimalCallViewModel(gridLayout), camera$: constant(track) },
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
noPipeline,
/* reactionIdentifier */ undefined,
{ showControls: true, header: HeaderStyle.Standard },
);
expect(vm.cameraTrack$.value).toBe(track);
muted$.next(true);
expect(vm.cameraTrack$.value).toBeNull();
muted$.next(false);
expect(vm.cameraTrack$.value).toBe(track);
});
it("availability is the same before and during a call", () => {
for (const supported of [true, false]) {
sdkSupportMock.mockReturnValue(supported);
const lobby = lobbyOn("desktop");
const inCall = createCallFooterViewModel(
testScope(),
buildMinimalCallViewModel(gridLayout),
mockMuteStates(),
twoMicsAndOneCamMediaDevices,
pipelineAsTheSdkSays(),
/* reactionIdentifier */ undefined,
{ showControls: true, header: HeaderStyle.Standard },
);
const offeredInLobby =
lobby.selectBackgroundEffect$.value !== undefined;
expect(inCall.selectBackgroundEffect$.value !== undefined).toBe(
offeredInLobby,
);
expect(offeredInLobby).toBe(supported);
}
});
});
describe("createLobbyFooterViewModel", () => {
it("offers the device menus even on a phone, previewing the lobby's camera", () => {
platformMock.mockReturnValue("ios");
const track = mockVideoTrack({});
const vm = createLobbyFooterViewModel(
testScope(),
mockMuteStates(),
devices,
pipeline,
constant(track),
/* openSettings */ undefined,
/* hangup */ undefined,
/* showLogo */ false,
);
const camera = vm.cameraMenuVm$.value;
expect(vm.audioMenuVm$.value.options$.value).not.toEqual([]);
expect(camera.options$.value).not.toEqual([]);
expect(camera.selectBackgroundEffect$.value).toBeDefined();
expect(camera.cameraTrack$.value).toBe(track);
});
});
+54 -305
View File
@@ -5,253 +5,24 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
BehaviorSubject,
combineLatest,
distinctUntilChanged,
map,
type Observable,
of,
switchMap,
} from "rxjs";
import { supportsAudioOutputSelection } from "livekit-client";
import { combineLatest, map, type Observable } from "rxjs";
import { type LocalVideoMediaTrack } from "@element-hq/matrixrtc-sdk";
import { supportsModernBackgroundProcessors } from "@livekit/track-processors";
import { logger } from "matrix-js-sdk/lib/logger";
import {
parseEffect,
serializeEffect,
shippedBackgrounds,
} from "../tracks/backgroundEffectCatalog";
import {
type AddedBackground,
addedBackgrounds,
maxAddedBackgrounds,
UnusableImage,
} from "../tracks/backgroundImages";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { type BackgroundEffectsState } from "../state/BackgroundEffects";
import { type MenuOptions } from "./MediaMuteAndSwitchButton";
import { type MediaDevices } from "../state/MediaDevices";
import {
backgroundEffect as backgroundEffectSetting,
debugTileLayout as debugTileLayoutSetting,
} from "../settings/settings";
import { debugTileLayout as debugTileLayoutSetting } from "../settings/settings";
import { type Behavior, constant } from "../state/Behavior";
import type { ObservableScope } from "../state/ObservableScope";
import { type MuteStates } from "../state/MuteStates";
import { createStaticViewModel, type ViewModel } from "../state/ViewModel";
import { HeaderStyle } from "../UrlParams";
import { platform } from "../Platform";
import { type BackgroundImageRefusal, type FooterSnapshot } from "./CallFooter";
import { type BackgroundEffectGridItem } from "./BackgroundEffectGrid";
/**
* Shared helper: maps MuteStates into the audio/video enabled + toggle behaviors
* needed by FooterSnapshot.
*/
function buildMuteBehaviors(
scope: ObservableScope,
muteStates: MuteStates,
): Pick<
ViewModel<FooterSnapshot>,
| "audioEnabled$"
| "audioBusy$"
| "toggleAudio$"
| "videoEnabled$"
| "videoBusy$"
| "toggleVideo$"
> {
return {
audioEnabled$: muteStates.audio.enabled$,
audioBusy$: muteStates.audio.syncing$,
toggleAudio$: scope.behavior(
muteStates.audio.toggle$.pipe(map((t) => t ?? undefined)),
),
videoEnabled$: muteStates.video.enabled$,
videoBusy$: muteStates.video.syncing$,
toggleVideo$: scope.behavior(
muteStates.video.toggle$.pipe(map((t) => t ?? undefined)),
),
};
}
/**
* Shared helper: maps MediaDevices into the audio/video device-list behaviors
* needed by FooterSnapshot (options, selection, callbacks, background effect).
*/
function buildDeviceBehaviors(
scope: ObservableScope,
mediaDevices: MediaDevices,
/** return empty arrays for audioOptions and videoOptions*/
disableSwitcher$: Behavior<boolean>,
backgroundEffectsState$: Observable<BackgroundEffectsState>,
/** The camera as the pipeline has it, for the menu's preview. */
camera$: Observable<LocalVideoMediaTrack | null | undefined>,
): Pick<
ViewModel<FooterSnapshot>,
| "audioOptions$"
| "selectedAudio$"
| "selectAudioButtonOption$"
| "audioOutputOptions$"
| "selectedAudioOutput$"
| "selectAudioOutputOption$"
| "videoOptions$"
| "selectedVideo$"
| "selectVideoButtonOption$"
| "backgroundEffect$"
| "selectBackgroundEffect$"
| "backgroundEffects$"
| "backgroundEffectNotice$"
| "backgroundEffectSettling$"
| "addBackgroundImage$"
| "backgroundImageRefusal$"
| "removeBackgroundEffect$"
| "cameraTrack$"
> {
const options$ = (
available$: Behavior<Map<string, MenuOptions["label"]>>,
): Observable<MenuOptions[]> =>
disableSwitcher$.pipe(
switchMap((disable) =>
disable
? constant([] as MenuOptions[])
: available$.pipe(
map((available) =>
[...available.entries()].map(([id, label]) => ({ id, label })),
),
),
),
);
// Also false once the pipeline has failed to build this session.
const supported$ = backgroundEffectsState$.pipe(
map(({ supported }) => supported === true),
distinctUntilChanged(),
);
const slow = !supportsModernBackgroundProcessors();
const offered$ = combineLatest([disableSwitcher$, supported$]).pipe(
map(([switcherDisabled, supported]) => !switcherDisabled && supported),
);
// A new object for each refusal, so the same one twice is shown twice.
const refusal$ = new BehaviorSubject<BackgroundImageRefusal | undefined>(
undefined,
);
return {
audioOptions$: scope.behavior(options$(mediaDevices.audioInput.available$)),
selectedAudio$: scope.behavior(
mediaDevices.audioInput.selected$.pipe(map((s) => s?.id)),
),
selectAudioButtonOption$: constant(mediaDevices.audioInput.select),
audioOutputOptions$: scope.behavior(
options$(mediaDevices.audioOutput.available$),
),
selectedAudioOutput$: scope.behavior(
mediaDevices.audioOutput.selected$.pipe(map((s) => s?.id)),
),
// Withheld where the platform can't route audio to a chosen device, which
// disables the speaker section.
selectAudioOutputOption$: constant(
supportsAudioOutputSelection()
? mediaDevices.audioOutput.select
: undefined,
),
videoOptions$: scope.behavior(options$(mediaDevices.videoInput.available$)),
selectedVideo$: scope.behavior(
mediaDevices.videoInput.selected$.pipe(map((s) => s?.id)),
),
selectVideoButtonOption$: constant(mediaDevices.videoInput.select),
backgroundEffect$: scope.behavior(
combineLatest([backgroundEffectSetting.value$, supported$]).pipe(
map(([raw, supported]) =>
supported ? serializeEffect(parseEffect(raw)) : "none",
),
),
),
selectBackgroundEffect$: scope.behavior(
offered$.pipe(
map((offered) =>
offered
? (id: string): void =>
backgroundEffectSetting.setValue(
serializeEffect(parseEffect(id)),
)
: undefined,
),
),
),
backgroundEffects$: scope.behavior(
addedBackgrounds.added$.pipe(
map((added) => backgroundEffectChoices(added ?? [])),
),
),
backgroundEffectNotice$: scope.behavior(
offered$.pipe(
map((offered) =>
!offered ? "unavailable" : slow ? "slow" : undefined,
),
),
),
backgroundEffectSettling$: scope.behavior(
backgroundEffectsState$.pipe(
map(({ wait }) => wait !== undefined),
distinctUntilChanged(),
),
false,
),
// Kept and offered, not put on: that waits for the user to choose it.
addBackgroundImage$: scope.behavior(
combineLatest([offered$, addedBackgrounds.added$]).pipe(
map(([offered, added]) =>
offered && (added?.length ?? 0) < maxAddedBackgrounds
? (file: File): void => {
refusal$.next(undefined);
addedBackgrounds.add(file).catch((e) => {
logger.warn("Could not keep that background", e);
refusal$.next({
reason: e instanceof UnusableImage ? e.reason : "not-kept",
});
});
}
: undefined,
),
),
),
backgroundImageRefusal$: refusal$,
// None while muted: the call mutes its track rather than releasing it.
cameraTrack$: scope.behavior(
camera$.pipe(
switchMap((camera) =>
camera
? camera.muted$.pipe(map((muted) => (muted ? null : camera)))
: of(null),
),
),
null,
),
// Never the one in force: taking it away would leave nothing chosen.
removeBackgroundEffect$: scope.behavior(
offered$.pipe(
map((offered) =>
offered
? (optionId: string): void => {
const effect = parseEffect(optionId);
if (effect.kind !== "added") return;
if (backgroundEffectSetting.getValue() === optionId) return;
addedBackgrounds
.remove(effect.id)
.catch((e) =>
logger.warn("Could not remove that background", e),
);
}
: undefined,
),
),
),
};
}
import { type FooterSnapshot } from "./CallFooter";
import {
createAudioMenuViewModel,
createCameraMenuViewModel,
} from "./MediaMenuViewModel";
/**
* Creates the ViewModel for the CallFooter.
@@ -286,13 +57,23 @@ export function createCallFooterViewModel(
isPip$.pipe(map((isPip) => isPip || platform !== "desktop")),
);
return {
...buildMuteBehaviors(scope, muteStates),
...buildDeviceBehaviors(
scope,
mediaDevices,
disableDeviceSwitcher$,
backgroundEffectsState$,
callModel.camera$,
audioMenuVm$: constant(
createAudioMenuViewModel(
scope,
muteStates,
mediaDevices,
disableDeviceSwitcher$,
),
),
cameraMenuVm$: constant(
createCameraMenuViewModel(
scope,
muteStates,
mediaDevices,
disableDeviceSwitcher$,
backgroundEffectsState$,
callModel.camera$,
),
),
// candidat to move into the FooterViewModel
showFooter$: callModel.showFooter$,
@@ -372,72 +153,40 @@ export function createLobbyFooterViewModel(
hangup: (() => void) | undefined,
showLogo: boolean,
): ViewModel<FooterSnapshot> {
return {
...createStaticViewModel({
// we can safly skip any props that we do not need.
// The view model will then have less keys.
// But as soon as we call `useViewModel` and convert back to a snapshot the missing props will
// be correcty matching the snapshot type.
showLogo,
hideControls: false,
asOverlay: false,
showModals: true,
buttonSize: "lg",
openSettings,
hangup,
debugTileLayout: false,
showFooter: true,
toggleAudio: undefined,
toggleVideo: undefined,
toggleScreenSharing: undefined,
audioEnabled: undefined,
audioBusy: false,
videoEnabled: undefined,
videoBusy: false,
layoutSwitchVm: null,
sharingScreen: false,
audioOutputSwitcher: undefined,
reactionIdentifier: undefined,
reactionData: undefined,
tileStoreGeneration: undefined,
audioOptions: undefined,
audioOutputOptions: undefined,
videoOptions: undefined,
selectedAudio: undefined,
selectedAudioOutput: undefined,
selectedVideo: undefined,
selectAudioButtonOption: undefined,
selectAudioOutputOption: undefined,
selectVideoButtonOption: undefined,
}),
...buildMuteBehaviors(scope, muteStates),
...buildDeviceBehaviors(
return createStaticViewModel({
// we can safly skip any props that we do not need.
// The view model will then have less keys.
// But as soon as we call `useViewModel` and convert back to a snapshot the missing props will
// be correcty matching the snapshot type.
showLogo,
hideControls: false,
asOverlay: false,
showModals: true,
buttonSize: "lg",
openSettings,
hangup,
debugTileLayout: false,
showFooter: true,
toggleScreenSharing: undefined,
layoutSwitchVm: null,
sharingScreen: false,
audioOutputSwitcher: undefined,
reactionIdentifier: undefined,
reactionData: undefined,
tileStoreGeneration: undefined,
audioMenuVm: createAudioMenuViewModel(
scope,
muteStates,
mediaDevices,
constant(false),
),
cameraMenuVm: createCameraMenuViewModel(
scope,
muteStates,
mediaDevices,
constant(false),
backgroundEffectsState$,
camera$,
),
};
}
function backgroundEffectChoices(
added: AddedBackground[],
): BackgroundEffectGridItem[] {
return [
{ id: serializeEffect({ kind: "none" }), kind: "none", removable: false },
{ id: serializeEffect({ kind: "blur" }), kind: "blur", removable: false },
...shippedBackgrounds.map((background) => ({
id: serializeEffect({ kind: "shipped", id: background.id }),
kind: "image" as const,
imageUrl: background.imagePath,
removable: false,
})),
...added.map((background) => ({
id: serializeEffect({ kind: "added", id: background.id }),
kind: "image" as const,
imageUrl: background.url,
removable: true,
})),
];
});
}
+193
View File
@@ -0,0 +1,193 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, fn, userEvent, within } from "storybook/test";
import { type JSX, type ReactNode } from "react";
import type { Meta, StoryObj } from "@storybook/react-vite";
import { MediaMenuButton } from "./MediaMenuButton";
import { type MediaMenuSnapshot } from "./MediaMenuViewModel";
import { useStaticViewModel } from "../state/ViewModel";
import { MediaDevicesContext } from "../MediaDevicesContext";
import { MediaDevices } from "../state/MediaDevices";
import { globalScope } from "../state/ObservableScope";
const mediaDevices = new MediaDevices(globalScope, {
controlledAudioDevices: false,
});
/** Builds the view model from the story's args, so each field is a control. */
function MediaMenuButtonStoryWrapper({
kind,
...snapshot
}: MediaMenuSnapshot & { kind: "audio" | "video" }): ReactNode {
const vm = useStaticViewModel(snapshot);
return <MediaMenuButton kind={kind} vm={vm} size="lg" />;
}
const meta = {
component: MediaMenuButtonStoryWrapper,
decorators: [
(Story): JSX.Element => (
<MediaDevicesContext value={mediaDevices}>
<Story />
</MediaDevicesContext>
),
],
} satisfies Meta<typeof MediaMenuButtonStoryWrapper>;
export default meta;
type Story = StoryObj<typeof meta>;
const empty = {
outputOptions: [],
selectedOutput: undefined,
selectOutput: undefined,
backgroundEffect: undefined,
backgroundEffects: [],
backgroundEffectNotice: undefined,
backgroundEffectSettling: false,
backgroundImageRefusal: undefined,
selectBackgroundEffect: undefined,
addBackgroundImage: undefined,
removeBackgroundEffect: undefined,
cameraTrack: null,
} satisfies Partial<MediaMenuSnapshot>;
export const Microphone: Story = {
args: {
...empty,
kind: "audio",
enabled: true,
busy: false,
toggle: fn(),
options: [
{ id: "mic1", label: { type: "name", name: "Microphone 1" } },
{ id: "mic2", label: { type: "name", name: "Microphone 2" } },
],
selected: "mic1",
select: fn(),
outputOptions: [
{ id: "default", label: { type: "default", name: "Built-in Output" } },
],
selectedOutput: "default",
selectOutput: fn(),
},
play: async ({ args, canvasElement }) => {
await userEvent.click(
within(canvasElement).getByRole("switch", { name: "Mute microphone" }),
);
await expect(args.toggle).toHaveBeenCalled();
},
};
export const Camera: Story = {
args: {
...empty,
kind: "video",
enabled: true,
busy: false,
toggle: fn(),
options: [{ id: "cam1", label: { type: "name", name: "Camera 1" } }],
selected: "cam1",
select: fn(),
backgroundEffect: "none",
backgroundEffects: [
{ id: "none", kind: "none" },
{ id: "blur", kind: "blur" },
],
selectBackgroundEffect: fn(),
},
play: async ({ args, canvasElement }) => {
await userEvent.click(
within(canvasElement).getByRole("switch", { name: "Stop video" }),
);
await expect(args.toggle).toHaveBeenCalled();
},
};
export const CameraBusy: Story = {
args: { ...Camera.args, busy: true },
};
/** Opens the effects, for the stories that check what they say. */
async function openEffects(canvasElement: HTMLElement): Promise<HTMLElement> {
await userEvent.click(
within(canvasElement).getByRole("button", { name: "Camera" }),
);
return within(document.body).findByRole("group", {
name: "Background effects",
});
}
/** The view model says why; the view says it in words. */
export const CameraEffectsUnavailable: Story = {
args: {
...Camera.args,
selectBackgroundEffect: undefined,
backgroundEffectNotice: "unavailable",
},
play: async ({ canvasElement }) => {
const section = await openEffects(canvasElement);
await expect(
within(section).getByRole("group"),
).toHaveAccessibleDescription(
"Background effects are not supported on this platform.",
);
},
};
export const CameraEffectsSlow: Story = {
args: {
...Camera.args,
backgroundEffectNotice: "slow",
},
play: async ({ canvasElement }) => {
const section = await openEffects(canvasElement);
await expect(
within(section).getByRole("group"),
).toHaveAccessibleDescription(
"Background effects run slowly on this platform, which may cause your video to stutter.",
);
},
};
export const CameraFileRefused: Story = {
args: {
...Camera.args,
addBackgroundImage: fn(),
backgroundImageRefusal: { reason: "not-an-image", dismiss: fn() },
},
play: async ({ canvasElement }) => {
await openEffects(canvasElement);
await expect(
await within(document.body).findByText(
"That file is not a supported image",
),
).toBeInTheDocument();
},
};
/** No devices to pick and none to mute: a plain, disabled mute button. */
export const WithoutDevices: Story = {
args: {
...Camera.args,
enabled: false,
toggle: undefined,
options: [],
},
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(
canvas.queryByRole("button", { name: "Camera" }),
).not.toBeInTheDocument();
await expect(canvas.getByTestId("incall_videomute")).toHaveAttribute(
"aria-disabled",
"true",
);
},
};
+124
View File
@@ -0,0 +1,124 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC, useMemo } from "react";
import { useTranslation } from "react-i18next";
import { MicButton, VideoButton } from "../button";
import { MediaMuteAndSwitchButton } from "./MediaMuteAndSwitchButton";
import { type MediaMenuViewModel } from "./MediaMenuViewModel";
import { SelfPreview } from "./SelfPreview";
import { useBehavior } from "../useBehavior";
interface Props {
vm: MediaMenuViewModel;
kind: "audio" | "video";
size: "md" | "lg";
}
/**
* The microphone's or the camera's mute button: with a menu to pick the
* device while there are devices to offer, a plain mute button otherwise.
*/
export const MediaMenuButton: FC<Props> = ({ vm, kind, size }) => {
const { t } = useTranslation();
const enabled = useBehavior(vm.enabled$);
const busy = useBehavior(vm.busy$);
const toggle = useBehavior(vm.toggle$);
const options = useBehavior(vm.options$);
const selected = useBehavior(vm.selected$);
const select = useBehavior(vm.select$);
const outputOptions = useBehavior(vm.outputOptions$);
const selectedOutput = useBehavior(vm.selectedOutput$);
const selectOutput = useBehavior(vm.selectOutput$);
const backgroundEffect = useBehavior(vm.backgroundEffect$);
const backgroundEffects = useBehavior(vm.backgroundEffects$);
const notice = useBehavior(vm.backgroundEffectNotice$);
const settling = useBehavior(vm.backgroundEffectSettling$);
const refusal = useBehavior(vm.backgroundImageRefusal$);
const selectBackgroundEffect = useBehavior(vm.selectBackgroundEffect$);
const addBackgroundImage = useBehavior(vm.addBackgroundImage$);
const removeBackgroundEffect = useBehavior(vm.removeBackgroundEffect$);
const cameraTrack = useBehavior(vm.cameraTrack$);
const testId = kind === "audio" ? "incall_mute" : "incall_videomute";
const noticeText = useMemo(() => {
switch (notice) {
case undefined:
return undefined;
case "unavailable":
return t("background_effects.unavailable");
case "slow":
return t("background_effects.slow");
}
}, [notice, t]);
const refusalText = useMemo(() => {
switch (refusal?.reason) {
case undefined:
return undefined;
case "not-an-image":
return t("error.background_not_an_image");
case "animated":
return t("error.background_animated");
case "undecodable":
return t("error.background_undecodable");
case "not-kept":
return t("error.background_not_kept");
}
}, [refusal, t]);
if (options.length === 0) {
const PlainButton = kind === "audio" ? MicButton : VideoButton;
return (
<PlainButton
size={size}
enabled={enabled}
busy={busy}
onClick={toggle}
disabled={busy || toggle === undefined}
data-testid={testId}
/>
);
}
return kind === "audio" ? (
<MediaMuteAndSwitchButton
iconsAndLabels="audio"
enabled={enabled}
busy={busy}
onMuteClick={toggle}
data-testid={testId}
options={options}
selectedOption={selected}
onSelect={select}
outputOptions={outputOptions}
selectedOutputOption={selectedOutput}
onSelectOutput={selectOutput}
/>
) : (
<MediaMuteAndSwitchButton
iconsAndLabels="video"
enabled={enabled}
busy={busy}
onMuteClick={toggle}
options={options}
selectedOption={selected}
onSelect={select}
backgroundEffects={backgroundEffects}
selectedBackgroundEffect={backgroundEffect}
onSelectBackgroundEffect={selectBackgroundEffect}
backgroundEffectSettling={settling}
onAddBackgroundImage={addBackgroundImage}
onRemoveBackgroundEffect={removeBackgroundEffect}
selfPreview={<SelfPreview track={cameraTrack} />}
backgroundImageRefusal={refusalText}
onDismissBackgroundImageRefusal={refusal?.dismiss}
backgroundEffectNotice={noticeText}
/>
);
};
+294
View File
@@ -0,0 +1,294 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, describe, expect, it, vi } from "vitest";
import { BehaviorSubject, type Observable } from "rxjs";
import { type LocalVideoMediaTrack } from "@element-hq/matrixrtc-sdk";
import {
flushPromises,
testScope,
mockMuteStates,
mockMediaDevices,
mockVideoTrack,
} from "../utils/test";
import { constant } from "../state/Behavior";
import { type BackgroundEffectsState } from "../state/BackgroundEffects";
import type { DeviceLabel } from "../state/MediaDevices";
import {
createAudioMenuViewModel,
createCameraMenuViewModel,
type MediaMenuViewModel,
} from "./MediaMenuViewModel";
import { backgroundEffect as backgroundEffectSetting } from "../settings/settings";
import { shippedBackgrounds } from "../tracks/backgroundEffectCatalog";
import type * as BackgroundImages from "../tracks/backgroundImages";
import {
type AddedBackground,
addedBackgrounds,
maxAddedBackgrounds,
UnusableImage,
} from "../tracks/backgroundImages";
const modernRouteMock = vi.hoisted(() => vi.fn(() => true));
vi.mock("@livekit/track-processors", () => ({
supportsModernBackgroundProcessors: (): boolean => modernRouteMock(),
}));
const outputSelectionMock = vi.hoisted(() => vi.fn(() => true));
vi.mock("livekit-client", () => ({
supportsAudioOutputSelection: (): boolean => outputSelectionMock(),
}));
const store = vi.hoisted(() => ({
add: vi.fn<(file: Blob) => Promise<string>>(),
remove: vi.fn<(id: string) => Promise<void>>(),
}));
vi.mock("../tracks/backgroundImages", async (original) => {
const { BehaviorSubject } = await import("rxjs");
return {
...(await original<typeof BackgroundImages>()),
addedBackgrounds: {
added$: new BehaviorSubject<AddedBackground[] | undefined>([]),
add: store.add,
remove: store.remove,
},
};
});
const added$ = addedBackgrounds.added$ as BehaviorSubject<
AddedBackground[] | undefined
>;
const devices = mockMediaDevices({
audioInput: {
available$: constant(
new Map<string, DeviceLabel>([
["mic1", { type: "number", number: 1 }],
["mic2", { type: "name", name: "Microphone 2" }],
]),
),
selected$: constant(undefined),
select: vi.fn(),
},
// As Safari: no outputs listed.
audioOutput: {
available$: constant(new Map<string, DeviceLabel>()),
selected$: constant(undefined),
select: vi.fn(),
},
});
function audioMenu(disableSwitcher = false): MediaMenuViewModel {
return createAudioMenuViewModel(
testScope(),
mockMuteStates(),
devices,
constant(disableSwitcher),
);
}
function cameraMenu({
supported = true,
disableSwitcher = false,
pipeline$ = constant({ supported, processor: undefined }),
camera$ = constant(null),
}: {
supported?: boolean;
disableSwitcher?: boolean;
pipeline$?: Observable<BackgroundEffectsState>;
camera$?: Observable<LocalVideoMediaTrack | null>;
} = {}): MediaMenuViewModel {
return createCameraMenuViewModel(
testScope(),
mockMuteStates(),
devices,
constant(disableSwitcher),
pipeline$,
camera$,
);
}
describe("createAudioMenuViewModel", () => {
it("lists the devices, unless the switcher is withheld", () => {
expect(audioMenu().options$.value).toEqual([
{ id: "mic1", label: { type: "number", number: 1 } },
{ id: "mic2", label: { type: "name", name: "Microphone 2" } },
]);
expect(audioMenu(true).options$.value).toEqual([]);
});
it("lists no outputs as an empty list, which keeps the section", () => {
expect(audioMenu().outputOptions$.value).toEqual([]);
});
it.each([true, false])(
"offers to pick an output only where the platform can route to it (%s)",
(canRoute) => {
outputSelectionMock.mockReturnValue(canRoute);
expect(audioMenu().selectOutput$.value !== undefined).toBe(canRoute);
},
);
});
describe("createCameraMenuViewModel", () => {
afterEach(() => {
backgroundEffectSetting.setValue("none");
added$.next([]);
});
it("offers effects where they run, saying where they run slowly", () => {
modernRouteMock.mockReturnValue(true);
const vm = cameraMenu();
expect(vm.selectBackgroundEffect$.value).toBeDefined();
expect(vm.addBackgroundImage$.value).toBeDefined();
expect(vm.backgroundEffectNotice$.value).toBeUndefined();
modernRouteMock.mockReturnValue(false);
const slow = cameraMenu();
expect(slow.selectBackgroundEffect$.value).toBeDefined();
expect(slow.backgroundEffectNotice$.value).toBe("slow");
modernRouteMock.mockReturnValue(true);
});
it("offers nothing while the switcher is withheld", () => {
const vm = cameraMenu({ disableSwitcher: true });
expect(vm.options$.value).toEqual([]);
expect(vm.selectBackgroundEffect$.value).toBeUndefined();
expect(vm.backgroundEffectNotice$.value).toBe("unavailable");
});
it("offers nothing once the pipeline fails to build, keeping the choice", () => {
backgroundEffectSetting.setValue("blur");
const pipeline$ = new BehaviorSubject<BackgroundEffectsState>({
supported: true,
processor: undefined,
});
const vm = cameraMenu({ pipeline$ });
expect(vm.backgroundEffect$.value).toBe("blur");
pipeline$.next({ supported: false, processor: undefined });
expect(vm.selectBackgroundEffect$.value).toBeUndefined();
expect(vm.addBackgroundImage$.value).toBeUndefined();
expect(vm.removeBackgroundEffect$.value).toBeUndefined();
expect(vm.backgroundEffectNotice$.value).toBe("unavailable");
expect(vm.backgroundEffect$.value).toBe("none");
expect(backgroundEffectSetting.getValue()).toBe("blur");
});
it("saves the effect chosen, and none for one not on offer", () => {
const vm = cameraMenu();
vm.selectBackgroundEffect$.value!("image:arc");
expect(vm.backgroundEffect$.value).toBe("image:arc");
vm.selectBackgroundEffect$.value!("image:gone");
expect(backgroundEffectSetting.getValue()).toBe("none");
});
it("offers every effect in order, the added ones removable", () => {
added$.next([{ id: "mine", url: "blob:mine" }]);
expect(cameraMenu().backgroundEffects$.value).toEqual([
{ id: "none", kind: "none", removable: false },
{ id: "blur", kind: "blur", removable: false },
...shippedBackgrounds.map((background) => ({
id: `image:${background.id}`,
kind: "image",
imageUrl: background.imagePath,
removable: false,
})),
{
id: "added:mine",
kind: "image",
imageUrl: "blob:mine",
removable: true,
},
]);
});
it("passes on the wait for the first effect", () => {
const pipeline$ = new BehaviorSubject<BackgroundEffectsState>({
supported: true,
processor: undefined,
wait: "settling",
});
const vm = cameraMenu({ pipeline$ });
expect(vm.backgroundEffectSettling$.value).toBe(true);
pipeline$.next({ ...pipeline$.value, wait: undefined });
expect(vm.backgroundEffectSettling$.value).toBe(false);
});
it("offers adding only below the limit", () => {
const vm = cameraMenu();
added$.next(
Array.from({ length: maxAddedBackgrounds }, (_, i) => ({
id: `${i}`,
url: `blob:${i}`,
})),
);
expect(vm.addBackgroundImage$.value).toBeUndefined();
added$.next(added$.value!.slice(1));
expect(vm.addBackgroundImage$.value).toBeDefined();
});
it("says why each refused file was refused", async () => {
const vm = cameraMenu();
store.add.mockRejectedValue(new UnusableImage("animated"));
vm.addBackgroundImage$.value!(new File(["x"], "a.gif"));
await flushPromises();
expect(vm.backgroundImageRefusal$.value?.reason).toBe("animated");
store.add.mockRejectedValue(new Error("quota"));
vm.addBackgroundImage$.value!(new File(["x"], "b.png"));
await flushPromises();
expect(vm.backgroundImageRefusal$.value?.reason).toBe("not-kept");
store.add.mockResolvedValue("kept");
vm.addBackgroundImage$.value!(new File(["x"], "c.png"));
await flushPromises();
expect(vm.backgroundImageRefusal$.value).toBeUndefined();
});
it("drops a refusal once dismissed, and only that one", async () => {
const vm = cameraMenu();
store.add.mockRejectedValue(new UnusableImage("animated"));
vm.addBackgroundImage$.value!(new File(["x"], "a.gif"));
await flushPromises();
const first = vm.backgroundImageRefusal$.value!;
first.dismiss();
expect(vm.backgroundImageRefusal$.value).toBeUndefined();
// The same again is a refusal of its own, which the first can't dismiss.
vm.addBackgroundImage$.value!(new File(["x"], "a.gif"));
await flushPromises();
expect(vm.backgroundImageRefusal$.value?.reason).toBe("animated");
first.dismiss();
expect(vm.backgroundImageRefusal$.value?.reason).toBe("animated");
});
it("never removes the selected background", () => {
store.remove.mockResolvedValue();
const vm = cameraMenu();
backgroundEffectSetting.setValue("added:worn");
vm.removeBackgroundEffect$.value!("added:worn");
expect(store.remove).not.toHaveBeenCalled();
vm.removeBackgroundEffect$.value!("added:spare");
expect(store.remove).toHaveBeenCalledWith("spare");
});
it("previews the camera only while there is one and it is not muted", () => {
const muted$ = new BehaviorSubject(false);
const track = mockVideoTrack({ muted$ });
const camera$ = new BehaviorSubject<LocalVideoMediaTrack | null>(track);
const vm = cameraMenu({ camera$ });
expect(vm.cameraTrack$.value).toBe(track);
muted$.next(true);
expect(vm.cameraTrack$.value).toBeNull();
muted$.next(false);
expect(vm.cameraTrack$.value).toBe(track);
camera$.next(null);
expect(vm.cameraTrack$.value).toBeNull();
});
});
+324
View File
@@ -0,0 +1,324 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
BehaviorSubject,
combineLatest,
distinctUntilChanged,
map,
type Observable,
of,
switchMap,
} from "rxjs";
import { supportsAudioOutputSelection } from "livekit-client";
import { type LocalVideoMediaTrack } from "@element-hq/matrixrtc-sdk";
import { supportsModernBackgroundProcessors } from "@livekit/track-processors";
import { logger } from "matrix-js-sdk/lib/logger";
import {
parseEffect,
serializeEffect,
shippedBackgrounds,
} from "../tracks/backgroundEffectCatalog";
import {
type AddedBackground,
addedBackgrounds,
maxAddedBackgrounds,
type UnusableReason,
UnusableImage,
} from "../tracks/backgroundImages";
import { type BackgroundEffectsState } from "../state/BackgroundEffects";
import { type MenuOptions } from "./MediaMuteAndSwitchButton";
import { type MediaDevices } from "../state/MediaDevices";
import { backgroundEffect as backgroundEffectSetting } from "../settings/settings";
import { type Behavior, constant } from "../state/Behavior";
import type { ObservableScope } from "../state/ObservableScope";
import { type MuteStates } from "../state/MuteStates";
import { createStaticViewModel, type ViewModel } from "../state/ViewModel";
import { type BackgroundEffectGridItem } from "./BackgroundEffectGrid";
export interface BackgroundImageRefusal {
reason: UnusableReason | "not-kept";
/** Done with it: once read, or once the menu showing it closes. */
dismiss: () => void;
}
/**
* A mute button with a menu to pick the device, shared by the microphone and
* the camera. Fields that belong to the other device are left empty.
*/
export interface MediaMenuSnapshot {
/** Whether the device is on. (muted/unmuted state) */
enabled: boolean;
busy: boolean;
/** Turns the device on or off. Undefined while it can't be toggled. (disable interaction state) */
toggle: (() => void) | undefined;
options: MenuOptions[];
selected: string | undefined;
select: ((id: string) => void) | undefined;
/** Output (speaker) devices. Microphone only. */
outputOptions: MenuOptions[];
selectedOutput: string | undefined;
/** Undefined if the output device is not configurable. */
selectOutput: ((id: string) => void) | undefined;
/** The selected background effect, as its option id. Camera only. */
backgroundEffect: string | undefined;
/** Every effect on offer, in the order they are shown. */
backgroundEffects: BackgroundEffectGridItem[];
/** What the user is told about effects here, if anything. */
backgroundEffectNotice: "unavailable" | "slow" | undefined;
/** Whether the first effect chosen is still being prepared. */
backgroundEffectSettling: boolean;
/** Why the last file offered couldn't be kept, if it couldn't. */
backgroundImageRefusal: BackgroundImageRefusal | undefined;
/** Undefined where background effects can't be chosen. */
selectBackgroundEffect: ((id: string) => void) | undefined;
/** Undefined where no background can be added. */
addBackgroundImage: ((file: File) => void) | undefined;
/** Removes an added background by its option id, unless it is the selected one. */
removeBackgroundEffect: ((id: string) => void) | undefined;
/** The camera to preview, with the selected effect; null while it is off. */
cameraTrack: LocalVideoMediaTrack | null;
}
export type MediaMenuViewModel = ViewModel<MediaMenuSnapshot>;
const noBackgroundEffects = {
backgroundEffect: undefined,
backgroundEffects: [],
backgroundEffectNotice: undefined,
backgroundEffectSettling: false,
backgroundImageRefusal: undefined,
selectBackgroundEffect: undefined,
addBackgroundImage: undefined,
removeBackgroundEffect: undefined,
cameraTrack: null,
} satisfies Partial<MediaMenuSnapshot>;
function options$(
disableSwitcher$: Behavior<boolean>,
available$: Behavior<Map<string, MenuOptions["label"]>>,
): Observable<MenuOptions[]> {
return disableSwitcher$.pipe(
switchMap((disable) =>
disable
? constant([] as MenuOptions[])
: available$.pipe(
map((available) =>
[...available.entries()].map(([id, label]) => ({ id, label })),
),
),
),
);
}
/**
* The microphone's mute button and menu, with the speaker choice.
*
* @param disableSwitcher$ - True to leave out the menu.
*/
export function createAudioMenuViewModel(
scope: ObservableScope,
muteStates: MuteStates,
mediaDevices: MediaDevices,
disableSwitcher$: Behavior<boolean>,
): MediaMenuViewModel {
const { audio } = muteStates;
return {
...createStaticViewModel(noBackgroundEffects),
enabled$: audio.enabled$,
busy$: audio.syncing$,
toggle$: scope.behavior(audio.toggle$.pipe(map((t) => t ?? undefined))),
options$: scope.behavior(
options$(disableSwitcher$, mediaDevices.audioInput.available$),
),
selected$: scope.behavior(
mediaDevices.audioInput.selected$.pipe(map((s) => s?.id)),
),
select$: constant(mediaDevices.audioInput.select),
outputOptions$: scope.behavior(
options$(disableSwitcher$, mediaDevices.audioOutput.available$),
),
selectedOutput$: scope.behavior(
mediaDevices.audioOutput.selected$.pipe(map((s) => s?.id)),
),
selectOutput$: constant(
supportsAudioOutputSelection()
? mediaDevices.audioOutput.select
: undefined,
),
};
}
/**
* The camera's mute button and menu, with the background effects and a
* preview of the camera.
*
* @param disableSwitcher$ - True to leave out the menu.
* @param backgroundEffectsState$ - The background pipeline's state: whether
* effects run here, and the wait for the first.
* @param camera$ - The camera as the pipeline has it, for the preview.
*/
export function createCameraMenuViewModel(
scope: ObservableScope,
muteStates: MuteStates,
mediaDevices: MediaDevices,
disableSwitcher$: Behavior<boolean>,
backgroundEffectsState$: Observable<BackgroundEffectsState>,
camera$: Observable<LocalVideoMediaTrack | null | undefined>,
): MediaMenuViewModel {
const { video } = muteStates;
// Also false once the pipeline has failed to build this session.
const supported$ = backgroundEffectsState$.pipe(
map(({ supported }) => supported === true),
distinctUntilChanged(),
);
const slow = !supportsModernBackgroundProcessors();
const offered$ = combineLatest([disableSwitcher$, supported$]).pipe(
map(([switcherDisabled, supported]) => !switcherDisabled && supported),
);
const refusal$ = new BehaviorSubject<BackgroundImageRefusal | undefined>(
undefined,
);
const refuse = (reason: BackgroundImageRefusal["reason"]): void => {
const refusal: BackgroundImageRefusal = {
reason,
// A later refusal is not this one's to dismiss.
dismiss: () => {
if (refusal$.value === refusal) refusal$.next(undefined);
},
};
refusal$.next(refusal);
};
return {
...createStaticViewModel({
outputOptions: [],
selectedOutput: undefined,
selectOutput: undefined,
}),
enabled$: video.enabled$,
busy$: video.syncing$,
toggle$: scope.behavior(video.toggle$.pipe(map((t) => t ?? undefined))),
options$: scope.behavior(
options$(disableSwitcher$, mediaDevices.videoInput.available$),
),
selected$: scope.behavior(
mediaDevices.videoInput.selected$.pipe(map((s) => s?.id)),
),
select$: constant(mediaDevices.videoInput.select),
backgroundEffect$: scope.behavior(
combineLatest([backgroundEffectSetting.value$, supported$]).pipe(
map(([raw, supported]) =>
supported ? serializeEffect(parseEffect(raw)) : "none",
),
),
),
selectBackgroundEffect$: scope.behavior(
offered$.pipe(
map((offered) =>
offered
? (id: string): void =>
backgroundEffectSetting.setValue(
serializeEffect(parseEffect(id)),
)
: undefined,
),
),
),
backgroundEffects$: scope.behavior(
addedBackgrounds.added$.pipe(
map((added) => backgroundEffectChoices(added ?? [])),
),
),
backgroundEffectNotice$: scope.behavior(
offered$.pipe(
map((offered) =>
!offered ? "unavailable" : slow ? "slow" : undefined,
),
),
),
backgroundEffectSettling$: scope.behavior(
backgroundEffectsState$.pipe(
map(({ wait }) => wait !== undefined),
distinctUntilChanged(),
),
false,
),
// Kept and offered, not put on: that waits for the user to choose it.
addBackgroundImage$: scope.behavior(
combineLatest([offered$, addedBackgrounds.added$]).pipe(
map(([offered, added]) =>
offered && (added?.length ?? 0) < maxAddedBackgrounds
? (file: File): void => {
refusal$.next(undefined);
addedBackgrounds.add(file).catch((e) => {
logger.warn("Could not keep that background", e);
refuse(e instanceof UnusableImage ? e.reason : "not-kept");
});
}
: undefined,
),
),
),
backgroundImageRefusal$: refusal$,
// Muting stops the camera, and so its light.
// The track outlives the mute, so its mute state is checked.
cameraTrack$: scope.behavior(
camera$.pipe(
switchMap((camera) =>
camera
? camera.muted$.pipe(map((muted) => (muted ? null : camera)))
: of(null),
),
),
null,
),
removeBackgroundEffect$: scope.behavior(
offered$.pipe(
map((offered) =>
offered
? (optionId: string): void => {
const effect = parseEffect(optionId);
if (effect.kind !== "added") return;
// The selected effect can't be removed: that would leave no
// effect selected.
if (backgroundEffectSetting.getValue() === optionId) return;
addedBackgrounds
.remove(effect.id)
.catch((e) =>
logger.warn("Could not remove that background", e),
);
}
: undefined,
),
),
),
};
}
function backgroundEffectChoices(
added: AddedBackground[],
): BackgroundEffectGridItem[] {
return [
{ id: serializeEffect({ kind: "none" }), kind: "none", removable: false },
{ id: serializeEffect({ kind: "blur" }), kind: "blur", removable: false },
...shippedBackgrounds.map((background) => ({
id: serializeEffect({ kind: "shipped", id: background.id }),
kind: "image" as const,
imageUrl: background.imagePath,
removable: false,
})),
...added.map((background) => ({
id: serializeEffect({ kind: "added", id: background.id }),
kind: "image" as const,
imageUrl: background.url,
removable: true,
})),
];
}
@@ -404,7 +404,7 @@ export const BackgroundImageRefused: Story = {
selectedBackgroundEffect: "blur",
backgroundEffectNotice:
"Background effects run slowly on this platform, which may cause your video to stutter.",
backgroundImageRefusal: { text: "That file is not a supported image" },
backgroundImageRefusal: "That file is not a supported image",
},
decorators: [
(Story): JSX.Element => (
@@ -440,7 +440,7 @@ function swatch(colour: string): string {
)}`;
}
/** One's own backgrounds are removable, never the one in force. */
/** One's own backgrounds are removable, unless one is the selected one. */
export const AddedBackgroundsCanBeRemoved: Story = {
args: {
...VideoUnmute.args,
@@ -437,7 +437,7 @@ describe("MediaMuteAndSwitchButton", () => {
}
});
test("a refusal can be dismissed and does not outlive the menu", async () => {
test("a refusal is dismissed when closed, or with the menu", async () => {
const user = userEvent.setup();
// jsdom lays nothing out, so has nothing to scroll.
const scroll = Element.prototype.scrollIntoView;
@@ -445,7 +445,8 @@ describe("MediaMuteAndSwitchButton", () => {
onTestFinished(() => {
Element.prototype.scrollIntoView = scroll;
});
const menu = (refusal?: { text: string }): JSX.Element =>
const onDismiss = vi.fn();
const menu = (refusal?: string): JSX.Element =>
withProviders(
<MediaMuteAndSwitchButton
iconsAndLabels="video"
@@ -457,28 +458,22 @@ describe("MediaMuteAndSwitchButton", () => {
onSelectBackgroundEffect={vi.fn()}
onAddBackgroundImage={vi.fn()}
backgroundImageRefusal={refusal}
onDismissBackgroundImageRefusal={onDismiss}
/>,
);
const { rerender, getByRole } = render(menu());
await user.click(getByRole("button", { name: "Camera" }));
rerender(menu({ text: "That file is not a supported image" }));
rerender(menu("That file is not a supported image"));
expect(
screen.getByText("That file is not a supported image"),
).toBeVisible();
screen.getByRole("menuitemradio", { name: "Blur", checked: true });
await user.click(screen.getByRole("button", { name: "Close" }));
expect(screen.queryByText("That file is not a supported image")).toBeNull();
// The next refusal is shown, even if it says the same.
rerender(menu({ text: "That file is not a supported image" }));
expect(
screen.getByText("That file is not a supported image"),
).toBeVisible();
expect(onDismiss).toHaveBeenCalledTimes(1);
await user.keyboard("{Escape}");
await user.click(getByRole("button", { name: "Camera" }));
expect(screen.queryByText("That file is not a supported image")).toBeNull();
expect(onDismiss).toHaveBeenCalledTimes(2);
});
test("offers the background effects in a phone's drawer", async () => {
+17 -16
View File
@@ -85,10 +85,12 @@ export interface MediaMuteAndSwitchButtonProps {
backgroundEffectSettling?: boolean;
/** Called with the file chosen from the add tile. Omit to leave it out. */
onAddBackgroundImage?: (file: File) => void;
/** Removes a removable effect, never the one in force. */
/** Removes a removable effect, unless it is the selected one. */
onRemoveBackgroundEffect?: (id: string) => void;
/** Why the last file offered couldn't be used; a new object each time. */
backgroundImageRefusal?: { text: string };
/** Why the last file offered couldn't be used, if it couldn't. */
backgroundImageRefusal?: string;
/** Called as the refusal is closed, or the menu showing it. */
onDismissBackgroundImageRefusal?: () => void;
/** The user's own camera, shown at the top of the camera menu. */
selfPreview?: ReactNode;
/**
@@ -135,6 +137,7 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
onAddBackgroundImage,
onRemoveBackgroundEffect,
backgroundImageRefusal,
onDismissBackgroundImageRefusal,
selfPreview,
onSelect,
}) => {
@@ -149,20 +152,18 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
// elsewhere; it is held open until the picker is done.
const choosingFile = useRef(false);
const fileInput = useRef<HTMLInputElement | null>(null);
// A refusal is about the attempt that caused it: shown until dismissed or
// until the menu closes, and again for the next one.
const [seenRefusal, setSeenRefusal] = useState<object>();
const refusal =
backgroundImageRefusal !== seenRefusal ? backgroundImageRefusal : undefined;
const onOpenChange = useCallback(
(open: boolean): void => {
if (!open && choosingFile.current) return;
setMenuOpen(open);
// Drop a request that never arrived.
if (!open) setPlannedSelection(null);
if (!open) setSeenRefusal(backgroundImageRefusal);
if (!open) {
// Drop a request that never arrived.
setPlannedSelection(null);
// A refusal is about the attempt that caused it.
onDismissBackgroundImageRefusal?.();
}
},
[backgroundImageRefusal],
[onDismissBackgroundImageRefusal],
);
// Scrolled to as it appears, as the list may be scrolled away from it.
const scrollIntoView = useCallback((element: HTMLElement | null): void => {
@@ -512,7 +513,7 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
removeLabel={t("action.remove")}
/>
{/* In the list, so the menu grows no taller for it. */}
{refusal !== undefined && (
{backgroundImageRefusal !== undefined && (
<div
ref={scrollIntoView}
role="none"
@@ -520,12 +521,12 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
>
<Alert
type="critical"
title={refusal.text}
onClose={(): void => setSeenRefusal(refusal)}
title={backgroundImageRefusal}
onClose={onDismissBackgroundImageRefusal}
/>
</div>
)}
{refusal === undefined &&
{backgroundImageRefusal === undefined &&
backgroundEffectNotice !== undefined && (
<div id={noticeId} role="none" className={styles.notice}>
<InfoIcon width={20} height={20} />
+122
View File
@@ -0,0 +1,122 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { renderHook } from "@testing-library/react";
import {
EventType,
JoinRule,
KnownMembership,
type MatrixClient,
MatrixEvent,
type Room,
} from "matrix-js-sdk";
import { describe, expect, test } from "vitest";
import { useGroupCallRooms } from "./useGroupCallRooms";
import { mockMatrixRoom } from "../utils/test";
describe("useGroupCallRooms", () => {
test("lists a joined room with legacy call member state", () => {
const room = mockCallRoom({ legacyMemberState: true });
expect(listedRoomIds([room])).toEqual([room.roomId]);
});
test("lists a joined room with a sticky RTC membership", () => {
const room = mockCallRoom({
stickyEvents: [new MatrixEvent({ type: EventType.RTCMembership })],
});
expect(listedRoomIds([room])).toEqual([room.roomId]);
});
test("lists a joined room with an RTC membership in its timeline", () => {
const room = mockCallRoom({
timelineEvents: [new MatrixEvent({ type: EventType.RTCMembership })],
});
expect(listedRoomIds([room])).toEqual([room.roomId]);
});
test("lists a knocked room without any call events", () => {
const room = mockCallRoom({
membership: KnownMembership.Knock,
joinRule: JoinRule.Knock,
});
expect(listedRoomIds([room])).toEqual([room.roomId]);
});
test("omits a joined room without any call events", () => {
const room = mockCallRoom({
stickyEvents: [new MatrixEvent({ type: EventType.RoomMessage })],
timelineEvents: [new MatrixEvent({ type: EventType.RoomMessage })],
});
expect(listedRoomIds([room])).toEqual([]);
});
test.each([KnownMembership.Invite, KnownMembership.Leave])(
"omits a room with %s membership despite call events",
(membership) => {
const room = mockCallRoom({
membership,
legacyMemberState: true,
stickyEvents: [new MatrixEvent({ type: EventType.RTCMembership })],
timelineEvents: [new MatrixEvent({ type: EventType.RTCMembership })],
});
expect(listedRoomIds([room])).toEqual([]);
},
);
});
function listedRoomIds(rooms: Room[]): string[] {
const client = {
getRooms: () => rooms,
getUserId: () => "@alice:example.org",
on: () => {},
off: () => {},
matrixRTC: {
getRoomSession: () => ({ memberships: [] }),
on: () => {},
off: () => {},
},
} as unknown as MatrixClient;
const { result } = renderHook(() => useGroupCallRooms(client));
return result.current.map(({ room }) => room.roomId);
}
let nextRoomId = 0;
function mockCallRoom({
membership = KnownMembership.Join,
joinRule = JoinRule.Public,
legacyMemberState = false,
stickyEvents = [],
timelineEvents = [],
}: {
membership?: KnownMembership;
joinRule?: JoinRule;
legacyMemberState?: boolean;
stickyEvents?: MatrixEvent[];
timelineEvents?: MatrixEvent[];
}): Room {
const stateEvents = new Map(
legacyMemberState ? [[EventType.GroupCallMemberPrefix, new Map()]] : [],
);
return mockMatrixRoom({
roomId: `!room${nextRoomId++}:example.org`,
name: "Call room",
getMyMembership: () => membership,
getJoinRule: () => joinRule,
hasEncryptionStateEvent: () => true,
getCanonicalAlias: () => null,
getMxcAvatarUrl: () => null,
getLiveTimeline: () =>
({
getState: () => ({ events: stateEvents }),
getEvents: () => timelineEvents,
}) as unknown as ReturnType<Room["getLiveTimeline"]>,
_unstable_getStickyEvents: () =>
stickyEvents as unknown as ReturnType<Room["_unstable_getStickyEvents"]>,
});
}
+4 -6
View File
@@ -141,7 +141,7 @@ const roomHasCallMembershipEvents = (room: Room): boolean => {
return true;
}
// Check for *active* calls using sticky events.
// Check for recent calls using sticky events.
for (const sticky of room._unstable_getStickyEvents()) {
if (sticky.getType() === EventType.RTCMembership) {
return true;
@@ -150,11 +150,9 @@ const roomHasCallMembershipEvents = (room: Room): boolean => {
// Otherwise, check recent event history to see if anyone had
// sent a call membership in here.
return timeline.getEvents().some(
(e) =>
// Membership events only count if both of these are true
e.unstableStickyInfo && e.getType() === EventType.GroupCallMemberPrefix,
);
return timeline
.getEvents()
.some((e) => e.getType() === EventType.RTCMembership);
// Otherwise, it's *unlikely* this room was ever a call.
};
+2 -1
View File
@@ -22,7 +22,8 @@ and their fonts, where a host embedding the component supplies its own. The
@import url("@fontsource/inconsolata/400.css");
@import url("@fontsource/inconsolata/700.css");
@import url("@vector-im/compound-design-tokens/assets/web/css/compound-design-tokens.css") layer(compound);
@import url("@vector-im/compound-design-tokens/assets/web/css/compound-design-tokens.css")
layer(compound);
@import url("@vector-im/compound-web/dist/style.css") layer(compound.components);
body {
+10 -8
View File
@@ -186,13 +186,18 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
const vm = createCallViewModel$(
scope,
rtcParticipation,
props.matrixRoom,
mediaDevices,
props.muteStates,
{
rtcParticipation,
matrixRoom: props.matrixRoom,
mediaDevices,
muteStates: props.muteStates,
encryptionSystem: props.e2eeSystem,
viewSize$: scope.behavior(observeElementSize$(rootElement)),
handsRaisedSubject$: reactionsReader.raisedHands$,
reactionsSubject$: reactionsReader.reactions$,
},
{
...callViewModelOptionsFromParams(urlParams),
encryptionSystem: props.e2eeSystem,
capture,
backgroundEffectsState$,
hostBridge,
@@ -202,10 +207,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
scope,
props.rtcSession,
),
windowSize$: scope.behavior(observeElementSize$(rootElement)),
},
reactionsReader.raisedHands$,
reactionsReader.reactions$,
);
setVm(vm);
+4 -4
View File
@@ -75,7 +75,7 @@ describe("the pipeline's state", () => {
const effects = createBackgroundEffects(testScope(), {
supported: true,
effect$: behavior(effect, { n: "none", b: "blur" }),
setEffect: vi.fn(),
clearEffect: vi.fn(),
added$: new BehaviorSubject<AddedBackground[] | undefined>([]),
canSegment: () => cold(answer, { y: true, n: false }),
pipeline: fakePipeline().pipeline,
@@ -110,7 +110,7 @@ describe("the pipeline's state", () => {
describe("background effects", () => {
let effect$: BehaviorSubject<string>;
let added$: BehaviorSubject<AddedBackground[] | undefined>;
let setEffect: (raw: string) => void;
let clearEffect: () => void;
let fake: ReturnType<typeof fakePipeline>;
function build(
@@ -119,7 +119,7 @@ describe("background effects", () => {
return createBackgroundEffects(testScope(), {
supported: true,
effect$,
setEffect,
clearEffect,
added$,
canSegment: () => of(true),
pipeline: fake.pipeline,
@@ -135,7 +135,7 @@ describe("background effects", () => {
beforeEach(() => {
effect$ = new BehaviorSubject("none");
added$ = new BehaviorSubject<AddedBackground[] | undefined>(undefined);
setEffect = vi.fn((raw: string) => effect$.next(raw));
clearEffect = vi.fn(() => effect$.next("none"));
fake = fakePipeline();
});
+4 -5
View File
@@ -33,7 +33,6 @@ import { type Behavior } from "./Behavior";
import { type ObservableScope } from "./ObservableScope";
import {
type BackgroundEffect,
type EffectId,
imagePathFor,
parseEffect,
} from "../tracks/backgroundEffectCatalog";
@@ -57,8 +56,8 @@ export interface BackgroundEffectsOptions {
supported: boolean;
/** The effect chosen, as the setting stores it. */
effect$: Behavior<string>;
/** Stores a choice, to forget one that can't be honoured. */
setEffect: (id: EffectId) => void;
/** Resets the choice to no effect, for when the chosen one can't run here. */
clearEffect: () => void;
/** The backgrounds the device keeps, undefined until it has said. */
added$: Behavior<AddedBackground[] | undefined>;
/** Builds a segmenter on trial, and answers whether it could. */
@@ -80,7 +79,7 @@ export function createBackgroundEffects(
{
supported,
effect$,
setEffect,
clearEffect,
added$,
canSegment,
pipeline,
@@ -107,7 +106,7 @@ export function createBackgroundEffects(
added !== undefined &&
!added.some(({ id }) => id === effect.id))
)
setEffect("none");
clearEffect();
});
// Asked once an effect is first wanted. A browser update may fix what
@@ -5,12 +5,14 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { test } from "vitest";
import EventEmitter from "events";
import { expect, test } from "vitest";
import {
EventType,
type IRoomTimelineData,
MatrixEvent,
type Room,
RoomEvent,
RoomStickyEventsEvent,
} from "matrix-js-sdk";
import {
RTC_CALL_INTENT_AUDIO,
@@ -29,6 +31,7 @@ import {
import {
type CallNotificationWrapper,
createCallNotificationLifecycle$,
createReceivedDecline$,
type RingAttempt,
} from "./CallNotificationLifecycle";
import { constant } from "../Behavior";
@@ -205,22 +208,16 @@ test("ring attempt can be declined", () => {
a: mockRingEvent("$notif1", 30),
}),
receivedDecline$: hot("--d", {
d: [
new MatrixEvent({
type: EventType.RTCDecline,
sender: alice.userId,
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$notif1",
},
d: new MatrixEvent({
type: EventType.RTCDecline,
sender: alice.userId,
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$notif1",
},
}),
{} as Room,
undefined,
false,
{} as IRoomTimelineData,
],
},
}),
}),
});
@@ -240,22 +237,16 @@ test("ring attempt times out if recipient declines too late", () => {
a: mockRingEvent("$notif1", 30),
}),
receivedDecline$: hot("100ms d", {
d: [
new MatrixEvent({
type: EventType.RTCDecline,
sender: alice.userId,
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$notif1",
},
d: new MatrixEvent({
type: EventType.RTCDecline,
sender: alice.userId,
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$notif1",
},
}),
{} as Room,
undefined,
false,
{} as IRoomTimelineData,
],
},
}),
}),
});
@@ -275,22 +266,16 @@ test("decline event relating to wrong event is ignored (times out)", () => {
a: mockRingEvent("$notif1", 30),
}),
receivedDecline$: hot("--d", {
d: [
new MatrixEvent({
type: EventType.RTCDecline,
sender: alice.userId,
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$other", // <---- WRONG
},
d: new MatrixEvent({
type: EventType.RTCDecline,
sender: alice.userId,
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$other", // <---- WRONG
},
}),
{} as Room,
undefined,
false,
{} as IRoomTimelineData,
],
},
}),
}),
});
@@ -310,22 +295,16 @@ test("decline event from wrong sender is ignored (times out)", () => {
a: mockRingEvent("$notif1", 30),
}),
receivedDecline$: hot("--d", {
d: [
new MatrixEvent({
type: EventType.RTCDecline,
sender: local.userId, // <---- WRONG
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$notif1",
},
d: new MatrixEvent({
type: EventType.RTCDecline,
sender: local.userId, // <---- WRONG
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$notif1",
},
}),
{} as Room,
undefined,
false,
{} as IRoomTimelineData,
],
},
}),
}),
});
@@ -335,3 +314,42 @@ test("decline event from wrong sender is ignored (times out)", () => {
});
});
});
test("declines are received from the timeline and the sticky section", () => {
const event = (eventId: string, type = EventType.RTCDecline): MatrixEvent =>
new MatrixEvent({ event_id: eventId, type, sender: alice.userId });
const room = Object.assign(new EventEmitter(), {
_unstable_getStickyEvents: () => [
event("$existing"),
event("$existingMembership", EventType.RTCMembership),
],
});
const received: string[] = [];
const subscription = createReceivedDecline$(
room as unknown as Room,
).subscribe((event) => received.push(event.getId()!));
const timelineArgs = [room, false, false, {}];
room.emit(RoomEvent.Timeline, event("$timeline"), ...timelineArgs);
room.emit(
RoomEvent.Timeline,
event("$message", EventType.RoomMessage),
...timelineArgs,
);
// A gappy sync delivers the decline only in the sticky section
room.emit(RoomStickyEventsEvent.Update, [event("$sticky")], [], []);
room.emit(
RoomStickyEventsEvent.Update,
[],
[{ previous: event("$old"), current: event("$replacement") }],
[event("$removed")],
);
subscription.unsubscribe();
expect(received).toEqual([
"$existing",
"$timeline",
"$sticky",
"$replacement",
]);
});
@@ -14,7 +14,9 @@ import {
toStableCallIntent,
} from "matrix-js-sdk/lib/matrixrtc";
import {
defer,
filter,
from,
fromEvent,
map,
merge,
@@ -27,13 +29,17 @@ import {
EMPTY,
race,
take,
mergeMap,
} from "rxjs";
import {
type EventTimelineSetHandlerMap,
EventType,
type MatrixEvent,
type Room as MatrixRoom,
RoomEvent,
RoomStickyEventsEvent,
} from "matrix-js-sdk";
import { type RoomStickyEventsMap } from "matrix-js-sdk/lib/models/room-sticky-events";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type Behavior, type ObservableScope } from "@element-hq/matrixrtc-sdk";
@@ -70,12 +76,34 @@ export function createSentCallNotification$(
export function createReceivedDecline$(
matrixRoom: MatrixRoom,
): Observable<Parameters<EventTimelineSetHandlerMap[RoomEvent.Timeline]>> {
return (
): Observable<MatrixEvent> {
// A decline is sticky, so after a gappy sync it arrives only in the sticky
// section, which never reaches the timeline. Racing for the first outcome
// makes a decline seen in both places harmless.
const timeline$ = (
fromEvent(matrixRoom, RoomEvent.Timeline) as Observable<
Parameters<EventTimelineSetHandlerMap[RoomEvent.Timeline]>
>
).pipe(filter(([event]) => event.getType() === EventType.RTCDecline));
).pipe(map(([event]) => event));
// The decline can arrive via sync before the send of our notification
// resolves, which is when we start listening. To guard against this, we
// include what is already in the sticky section.
const currentSticky$ = defer(() =>
from(matrixRoom._unstable_getStickyEvents()),
);
const sticky$ = (
fromEvent(matrixRoom, RoomStickyEventsEvent.Update) as Observable<
Parameters<RoomStickyEventsMap[RoomStickyEventsEvent.Update]>
>
).pipe(
mergeMap(([added, updated]) => [
...added,
...updated.map(({ current }) => current),
]),
);
return merge(currentSticky$, sticky$, timeline$).pipe(
filter((event) => event.getType() === EventType.RTCDecline),
);
}
export interface Props {
@@ -85,9 +113,7 @@ export interface Props {
/** The user ids of the room's members, ours included. */
roomMemberUserIds$: Behavior<string[]>;
sentCallNotification$: Observable<CallNotificationWrapper | null>;
receivedDecline$: Observable<
Parameters<EventTimelineSetHandlerMap[RoomEvent.Timeline]>
>;
receivedDecline$: Observable<MatrixEvent>;
options: { waitForCallPickup?: boolean; autoLeaveWhenOthersLeft?: boolean };
localUser: { deviceId: string; userId: string };
}
@@ -148,7 +174,7 @@ export function createCallNotificationLifecycle$({
// Call is declined when we receive a decline event
const decline$ = receivedDecline$.pipe(
filter(
([event]) =>
(event) =>
event.getRelation()?.rel_type === "m.reference" &&
event.getRelation()?.event_id === notificationEvent.event_id &&
event.getSender() === recipient,
+5 -10
View File
@@ -36,7 +36,6 @@ import {
mockRtcMembership,
testScope,
} from "../../utils/test.ts";
import { E2eeType } from "../../e2ee/e2eeType.ts";
import {
alice,
aliceId,
@@ -506,7 +505,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
videoEnabled: new Map([
[localParticipant, behavior(videoInputMarbles, yesNo)],
]),
windowSize$: constant({ width: 380, height: 700 }), // Mobile phone in portrait
viewSize$: constant({ width: 380, height: 700 }), // Mobile phone in portrait
},
(vm) => {
schedule(tapScreenInputMarbles, { a: () => vm.tapScreen() });
@@ -548,7 +547,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
// Both Alice and Bob are with us in the room
roomMembers: [local, alice, bob],
rtcMembers$: constant([localRtcMember, aliceRtcMember]),
windowSize$: constant({ width: 380, height: 700 }), // Mobile phone
viewSize$: constant({ width: 380, height: 700 }), // Mobile phone
},
(vm) => {
// Uses one-on-one mobile layout
@@ -595,7 +594,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
sharingScreen: new Map([
[aliceParticipant, behavior(aliceSharingInputMarbles, yesNo)],
]),
windowSize$: constant({ width: 700, height: 380 }), // Mobile phone in landscape
viewSize$: constant({ width: 700, height: 380 }), // Mobile phone in landscape
},
(vm) => {
expectObservable(summarizeLayout$(vm.layout$)).toBe(
@@ -783,13 +782,13 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
test("layout reacts to window size", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const windowSizeInputMarbles = "abc";
const viewSizeInputMarbles = "abc";
const expectedLayoutMarbles = " abc";
withCallViewModel(
{
remoteParticipants$: constant([aliceParticipant]),
rtcMembers$: constant([localRtcMember, aliceRtcMember]),
windowSize$: behavior(windowSizeInputMarbles, {
viewSize$: behavior(viewSizeInputMarbles, {
a: { width: 380, height: 700 }, // Start very narrow, like a phone
b: { width: 1000, height: 800 }, // Go to normal desktop window size
c: { width: 200, height: 180 }, // Go to PiP size
@@ -1466,7 +1465,6 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
},
{
autoLeaveWhenOthersLeft: true,
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
},
);
});
@@ -1484,7 +1482,6 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
},
{
autoLeaveWhenOthersLeft: true,
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
},
);
});
@@ -1502,7 +1499,6 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
},
{
autoLeaveWhenOthersLeft: false,
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
},
);
});
@@ -1532,7 +1528,6 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
},
{
autoLeaveWhenOthersLeft: true,
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
},
);
});
+47 -25
View File
@@ -185,8 +185,39 @@ export function initialPublishRequests(
return requests;
}
export interface CallViewModelOptions {
/**
* What the call is made of: the room, the participation, the devices, the
* things with no sensible default. How the host wants it to behave is in
* {@link CallViewModelOptions} instead.
*/
export interface CallViewModelProps {
rtcParticipation: RTCParticipation;
/** A call is permanently tied to a single Matrix room. */
matrixRoom: MatrixRoom;
mediaDevices: MediaDevices;
muteStates: MuteStates;
encryptionSystem: EncryptionSystem;
/**
* The size of the space the call is drawn in: the page when Element Call
* owns it, or the container a host mounted it in when it is a component.
* The layout — whether the call is shown full size, flat, narrow or as a
* picture-in-picture — follows this rather than the size of the window, so
* that a component shrunk by its host adapts even though the window has not
* changed.
*/
viewSize$: Behavior<{ width: number; height: number }>;
handsRaisedSubject$: Observable<Record<string, RaisedHandInfo>>;
reactionsSubject$: Observable<Record<string, ReactionInfo>>;
}
/**
* How the host wants the call to behave. Every field is optional and defaults
* to what standalone Element Call does, so `{}` is a working standalone call.
* What the call is made of is in {@link CallViewModelProps} instead. A test
* that wants to replace behaviour mocks the dependency it comes from rather
* than adding an option here.
*/
export interface CallViewModelOptions {
/**
* How our tracks are captured and encoded; the browser's and LiveKit's
* defaults otherwise.
@@ -222,17 +253,6 @@ export interface CallViewModelOptions {
* Defaults to none sent.
*/
sentCallNotification$?: Observable<CallNotificationWrapper | null>;
/**
* The size of the space the call is drawn in: the page when Element Call
* owns it, or the container a host mounted it in when it is a component.
* The layout — whether the call is shown full size, flat, narrow or as a
* picture-in-picture — follows this rather than the size of the window, so
* that a component shrunk by its host adapts even though the window has not
* changed.
*/
windowSize$: Behavior<{ width: number; height: number }>;
/** Optional behavior overriding for the screensharing, for testing */
toggleScreensharing?: () => void;
}
/**
@@ -469,14 +489,17 @@ export interface CallViewModel {
*/
export function createCallViewModel$(
scope: ObservableScope,
rtcParticipation: RTCParticipation,
// A call is permanently tied to a single Matrix room
matrixRoom: MatrixRoom,
mediaDevices: MediaDevices,
muteStates: MuteStates,
options: CallViewModelOptions,
handsRaisedSubject$: Observable<Record<string, RaisedHandInfo>>,
reactionsSubject$: Observable<Record<string, ReactionInfo>>,
{
rtcParticipation,
matrixRoom,
mediaDevices,
muteStates,
encryptionSystem,
viewSize$,
handsRaisedSubject$,
reactionsSubject$,
}: CallViewModelProps,
options: CallViewModelOptions = {},
): CallViewModel {
const logger = rootLogger.getChild("[CallViewModel]");
const client = matrixRoom.client;
@@ -702,7 +725,7 @@ export function createCallViewModel$(
userId: member.userId,
rtcBackendIdentity: member.rtcBackendIdentity,
member,
encryptionSystem: options.encryptionSystem,
encryptionSystem,
transport$: scope.behavior(
member.transport$.pipe(map((transport) => transport?.raw)),
),
@@ -916,7 +939,7 @@ export function createCallViewModel$(
// A guess at what the window's mode should be based on the size and shape of
// the space we have to draw in.
const naturalWindowMode$ = scope.behavior<WindowMode>(
options.windowSize$.pipe(
viewSize$.pipe(
map(({ width, height }) => {
if (height <= 400 && width <= 340) return "pip";
// Our layouts for flat windows are better at adapting to a small width
@@ -1500,8 +1523,7 @@ export function createCallViewModel$(
* the platform cannot capture a screen, or the host hides the button.
*/
const toggleScreenSharing =
options.toggleScreensharing ??
(hideScreensharing || !("getDisplayMedia" in (navigator.mediaDevices ?? {}))
hideScreensharing || !("getDisplayMedia" in (navigator.mediaDevices ?? {}))
? null
: (): void => {
const track = screenShare$.value;
@@ -1522,7 +1544,7 @@ export function createCallViewModel$(
e instanceof Error ? e : new Error(String(e)),
);
});
});
};
// Tell the host and the analytics about the user's joins and leaves. The
// participation is joined by the time the view model is built on it
@@ -81,7 +81,7 @@ export interface CallViewModelInputs {
sharingScreen: Map<Participant, Behavior<boolean>>;
mediaDevices: MediaDevices;
initialSyncState: SyncState;
windowSize$: Behavior<{ width: number; height: number }>;
viewSize$: Behavior<{ width: number; height: number }>;
}
export const localParticipant: Participant = { identity: "" };
@@ -106,7 +106,7 @@ export function withCallViewModel(mode: MatrixRTCMode) {
sharingScreen = new Map(),
mediaDevices = mockMediaDevices({}),
initialSyncState = SyncState.Syncing,
windowSize$ = constant({ width: 1000, height: 800 }),
viewSize$ = constant({ width: 1000, height: 800 }),
}: Partial<CallViewModelInputs> = {},
continuation: (
vm: CallViewModel,
@@ -146,6 +146,7 @@ export function withCallViewModel(mode: MatrixRTCMode) {
})() as Partial<MatrixClient> as MatrixClient,
getMembers: () => roomMembers,
getMembersWithMembership: () => roomMembers,
_unstable_getStickyEvents: () => [],
});
const rtcSession = new MockRTCSession(room, []).withMemberships(
rtcMembers$,
@@ -261,22 +262,24 @@ export function withCallViewModel(mode: MatrixRTCMode) {
const vm = createCallViewModel$(
scope,
rtcParticipation,
room,
mediaDevices,
muteStates,
{
rtcParticipation,
matrixRoom: room,
mediaDevices,
muteStates,
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
viewSize$,
handsRaisedSubject$: raisedHands$,
reactionsSubject$: reactions$,
},
{
autoLeaveWhenOthersLeft: false,
sentCallNotification$: createSentCallNotification$(
scope,
rtcSession.asMockedSession(),
),
windowSize$,
...options,
},
raisedHands$,
reactions$,
);
void mode;
-2
View File
@@ -17,7 +17,6 @@ import {
type HostRequest,
nullHostBridge,
} from "../HostBridge.ts";
import { E2eeType } from "../e2ee/e2eeType.ts";
import { MatrixRTCMode } from "../config/ConfigOptions.ts";
it.each([[MatrixRTCMode.Compatibility], [MatrixRTCMode.Matrix_2_0]])(
@@ -41,7 +40,6 @@ it.each([[MatrixRTCMode.Compatibility], [MatrixRTCMode.Matrix_2_0]])(
hangUp$.next({ data: {}, reply });
},
{
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
hostBridge,
},
);
+34 -1
View File
@@ -20,6 +20,7 @@ import {
const frame = {} as VideoFrame;
const controller = {} as TransformStreamDefaultController<VideoFrame>;
const stats = { processingTimeMs: 1, segmentationTimeMs: 1, filterTimeMs: 0 };
describe("BackgroundEffectTransformer", () => {
afterEach(() => {
@@ -66,9 +67,10 @@ describe("BackgroundEffectTransformer", () => {
});
it("reports the first frame carrying an effect, once", async () => {
// The library reports every frame it draws.
vi.spyOn(BackgroundTransformer.prototype, "transform").mockImplementation(
async function (this: BackgroundTransformer) {
this.isFirstFrame = false;
this.options.onFrameProcessed?.(stats);
return Promise.resolve();
},
);
@@ -91,6 +93,37 @@ describe("BackgroundEffectTransformer", () => {
expect(transformer.onFirstFrame).not.toHaveBeenCalled();
});
it("keeps reporting drawn frames to whoever asked", async () => {
vi.spyOn(BackgroundTransformer.prototype, "transform").mockImplementation(
async function (this: BackgroundTransformer) {
this.options.onFrameProcessed?.(stats);
return Promise.resolve();
},
);
const onFrameProcessed = vi.fn();
const transformer = new BackgroundEffectTransformer({ onFrameProcessed });
await transformer.transform(frame, controller);
expect(onFrameProcessed).toHaveBeenCalledWith(stats);
});
it("never sends the first frame on untouched", async () => {
// The library copies its first frame through untouched, ahead of the
// drawn one, unless told the first frame is past
const seen: boolean[] = [];
vi.spyOn(BackgroundTransformer.prototype, "transform").mockImplementation(
async function (this: BackgroundTransformer) {
seen.push(this.isFirstFrame);
return Promise.resolve();
},
);
const transformer = new BackgroundEffectTransformer({});
expect(transformer.isFirstFrame).toBe(true);
await transformer.transform(frame, controller);
expect(seen).toEqual([false]);
});
it("passes frames on untouched once disabled, even after blurring", async () => {
const transformer = new BackgroundEffectTransformer({});
await transformer.update({ blurRadius: 15, backgroundDisabled: false });
+28 -5
View File
@@ -41,18 +41,41 @@ const wasmFileset: WasmFileset = { wasmLoaderPath, wasmBinaryPath };
* resource fetched from the public internet.
*/
export class BackgroundEffectTransformer extends BackgroundTransformer {
/** Called once, when the first frame carrying an effect has been drawn. */
/**
* Called once, when the first frame carrying an effect has been drawn.
* This is used for drewAFrame$ to show the correct loading state.
*/
public onFirstFrame: (() => void) | undefined;
private drewAFrame = false;
public constructor(opts: BackgroundOptions) {
super(opts);
// Called by the library for a frame it drew, never for one it passed on
// untouched.
this.options.onFrameProcessed = (stats): void => {
opts.onFrameProcessed?.(stats);
if (this.drewAFrame) return;
this.drewAFrame = true;
this.onFirstFrame?.();
};
}
public override async transform(
frame: VideoFrame,
controller: TransformStreamDefaultController<VideoFrame>,
): Promise<void> {
const priming = this.isFirstFrame;
// From the LK docs:
// > Render a copy of the first frame is rendered to the screen as soon as possible to act
// > as a less jarring initial state than a solid color while the synchronous work below
// > (segmentation + frame rendering) occurs.
// We do not want this copy (it leaks unblurred media).
// So we always set `isFirstFrame` to false -> never show an unprocessed frame.
// It does not have any impact other than setting the unprocessed frame while loading the processor.
//
// To not show the blank frame, we have our own mechanism for hiding the loading overlay:
// `onFirstFrame` is what the EC UI will listen to and only then show the video feed.
this.isFirstFrame = false;
await super.transform(frame, controller);
// Cleared only once a frame has been through the segmenter; a frame
// passed through untouched leaves it set.
if (priming && !this.isFirstFrame) this.onFirstFrame?.();
}
/**
+1 -1
View File
@@ -57,7 +57,7 @@ export function createAppBackgroundEffects(
return createBackgroundEffects(scope, {
supported: supportsBackgroundProcessors(),
effect$: backgroundEffectSetting.value$,
setEffect: backgroundEffectSetting.setValue,
clearEffect: () => backgroundEffectSetting.setValue("none"),
added$: addedBackgrounds.added$,
canSegment,
pipeline: new BackgroundProcessorWrapper(transformer, "background-effect"),
+214
View File
@@ -0,0 +1,214 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { EventEmitter } from "events";
import {
afterEach,
beforeEach,
describe,
expect,
it,
type Mock,
vi,
} from "vitest";
import {
ClientEvent,
createClient,
type ICreateClientOpts,
IndexedDBStore,
MemoryStore,
} from "matrix-js-sdk";
import type * as MatrixSdk from "matrix-js-sdk";
import { initClient } from "./initClient";
import { flushPromises } from "./test";
interface Store {
startup: Mock<() => Promise<void>>;
getSavedSyncToken: Mock<() => Promise<string | null>>;
}
const store = (): Store => ({
startup: vi.fn<() => Promise<void>>().mockResolvedValue(undefined),
getSavedSyncToken: vi
.fn<() => Promise<string | null>>()
.mockResolvedValue(null),
});
// The client is the SDK's business; what these tests are about is how it is
// set up: its stores, its crypto, and when it counts as ready.
vi.mock("matrix-js-sdk", async (importOriginal) => ({
...(await importOriginal<typeof MatrixSdk>()),
createClient: vi.fn(),
IndexedDBStore: vi.fn(function (this: Store) {
Object.assign(this, store());
}),
MemoryStore: vi.fn(function (this: Store) {
Object.assign(this, store());
}),
}));
interface FakeClient extends EventEmitter {
store: Store;
clearStores: Mock<() => Promise<void>>;
initRustCrypto: Mock<() => Promise<void>>;
startClient: Mock<() => Promise<void>>;
}
/**
* A client that, unless told otherwise, reports its first sync a tick after
* being started, as the real one does once the network answers.
*/
function fakeClient(opts: ICreateClientOpts, syncs = true): FakeClient {
const client = Object.assign(new EventEmitter(), {
store: opts.store as unknown as Store,
clearStores: vi.fn<() => Promise<void>>().mockResolvedValue(undefined),
initRustCrypto: vi.fn<() => Promise<void>>().mockResolvedValue(undefined),
startClient: vi.fn(async () => {
if (!syncs) return;
await new Promise((resolve) => setTimeout(resolve, 0));
client.emit(ClientEvent.Sync, "PREPARED", null);
}),
});
return client;
}
const options: ICreateClientOpts = { baseUrl: "https://hs.example.org" };
const clients = vi.mocked(createClient) as unknown as Mock<
(opts: ICreateClientOpts) => FakeClient
>;
const indexedDBStores = vi.mocked(IndexedDBStore);
const memoryStores = vi.mocked(MemoryStore);
describe("initClient", () => {
beforeEach(() => {
clients.mockImplementation((opts) => fakeClient(opts));
});
afterEach(() => {
vi.clearAllMocks();
vi.unstubAllGlobals();
vi.unstubAllEnvs();
});
it("hands back the client once its first sync is done", async () => {
const client = await initClient(options, true);
expect(client).toBe(clients.mock.results[0].value);
expect(client.startClient).toHaveBeenCalledWith({
clientWellKnownPollPeriod: 600,
});
expect(clients).toHaveBeenCalledWith(
expect.objectContaining({
baseUrl: "https://hs.example.org",
useAuthorizationHeader: true,
localTimeoutMs: 5000,
}),
);
});
it("keeps the stores when restoring a session", async () => {
const client = (await initClient(options, true)) as unknown as FakeClient;
expect(client.clearStores).not.toHaveBeenCalled();
});
it("clears the stores before starting them when not restoring", async () => {
const client = (await initClient(options, false)) as unknown as FakeClient;
expect(client.clearStores).toHaveBeenCalled();
expect(client.clearStores.mock.invocationCallOrder[0]).toBeLessThan(
client.store.startup.mock.invocationCallOrder[0],
);
});
it("syncs into memory where there is no IndexedDB", async () => {
// jsdom has no indexedDB
const client = await initClient(options, true);
expect(client.store).toBe(memoryStores.mock.instances[0]);
expect(indexedDBStores).not.toHaveBeenCalled();
});
it("syncs into IndexedDB through a worker, where it was bundled", async () => {
vi.stubGlobal("indexedDB", {});
vi.stubEnv("DEV", false);
const client = await initClient(options, true);
expect(client.store).toBe(indexedDBStores.mock.instances[0]);
const [storeOptions] = indexedDBStores.mock.calls[0];
expect(storeOptions.dbName).toBe("element-call-sync");
expect(storeOptions.workerFactory).toBeTypeOf("function");
});
it("syncs into IndexedDB without a worker in development", async () => {
vi.stubGlobal("indexedDB", {});
vi.stubEnv("DEV", true);
await initClient(options, true);
// The dev server does not bundle the worker
expect(indexedDBStores.mock.calls[0][0].workerFactory).toBeUndefined();
});
it("falls back to memory when the IndexedDB store cannot start", async () => {
vi.stubGlobal("indexedDB", {});
const failing = store();
failing.startup.mockRejectedValue(new Error("quota exceeded"));
indexedDBStores.mockImplementationOnce(function (this: IndexedDBStore) {
Object.assign(this, failing);
});
const client = await initClient(options, true);
expect(client.store).toBe(memoryStores.mock.instances[0]);
expect(client.store.startup).toHaveBeenCalled();
});
it("carries on without crypto when it cannot be initialised", async () => {
clients.mockImplementation((opts) => {
const client = fakeClient(opts);
client.initRustCrypto.mockRejectedValue(new Error("stores in use"));
return client;
});
await expect(initClient(options, true)).resolves.toBeDefined();
});
it("waits for a sync from the network, past one restored from the store", async () => {
let client!: FakeClient;
clients.mockImplementation((opts) => {
client = fakeClient(opts, false);
client.store.getSavedSyncToken.mockResolvedValue("saved");
return client;
});
let ready = false;
const result = initClient(options, true).then(() => (ready = true));
await flushPromises();
// The first sync the client reports is the one restored from the store
client.emit(ClientEvent.Sync, "PREPARED", null);
await flushPromises();
expect(ready).toBe(false);
client.emit(ClientEvent.Sync, "PREPARED", null);
await result;
expect(ready).toBe(true);
});
it("fails when the first sync does", async () => {
let client!: FakeClient;
clients.mockImplementation((opts) => (client = fakeClient(opts, false)));
const result = initClient(options, true);
await flushPromises();
client.emit(ClientEvent.Sync, "ERROR", null, {
error: new Error("offline"),
});
await expect(result).rejects.toThrow("offline");
});
});
+170
View File
@@ -0,0 +1,170 @@
/*
Copyright 2022-2024 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
ClientEvent,
createClient,
type ICreateClientOpts,
IndexedDBStore,
type MatrixClient,
MemoryStore,
} from "matrix-js-sdk";
import { type ISyncStateData, type SyncState } from "matrix-js-sdk/lib/sync";
import { logger } from "matrix-js-sdk/lib/logger";
// This import is why `initClient` has a module of its own:
// - The bundler emits the worker file (11 MB: the Matrix SDK and its crypto
// wasm) the moment it loads this import — whether or not anything ever calls
// `new IndexedDBWorker()`. Tree shaking drops the call, not the file.
// - So it may only sit in a module the component build never loads. It used
// to be in `./matrix`, which the component reaches for the room URL helpers,
// and the worker shipped in the package, unused and unusable by any host.
import IndexedDBWorker from "../IndexedDBWorker?worker";
import { getUrlParams } from "../UrlParams";
import { Config } from "../config/Config";
import { fallbackICEServerAllowed } from "./matrix";
const SYNC_STORE_NAME = "element-call-sync";
/**
* Initialises and returns a new standalone Matrix Client.
* This can only be called safely if no other client is running
* otherwise rust crypto will throw since it is not ready to initialize a new session.
* If another client is running make sure `.logout()` is called before executing this function.
* @param clientOptions Object of options passed through to the client
* @param restore If the rust crypto should be reset before the client initialization or
* if the initialization should try to restore the crypto state from the indexDB.
* @returns The MatrixClient instance
*/
export async function initClient(
clientOptions: ICreateClientOpts,
restore: boolean,
): Promise<MatrixClient> {
let indexedDB: IDBFactory | undefined;
try {
indexedDB = window.indexedDB;
} catch (e) {
logger.warn("Could not get indexDB from window.", e);
}
// options we always pass to the client (stuff that we need in order to work)
const baseOpts = {
fallbackICEServerAllowed: fallbackICEServerAllowed,
isVoipWithNoMediaAllowed:
Config.get().features?.feature_group_calls_without_video_and_audio,
} as ICreateClientOpts;
if (indexedDB && localStorage) {
baseOpts.store = new IndexedDBStore({
indexedDB: window.indexedDB,
localStorage,
dbName: SYNC_STORE_NAME,
// We can't use the worker in dev mode because Vite simply doesn't bundle workers
// in dev mode: it expects them to use native modules. Ours don't, and even then only
// Chrome supports it. (It bundles them fine in production mode.)
workerFactory: import.meta.env.DEV
? undefined
: (): Worker => new IndexedDBWorker(),
});
} else if (localStorage) {
baseOpts.store = new MemoryStore({ localStorage });
}
// XXX: we read from the URL params in RoomPage too:
// it would be much better to read them in one place and pass
// the values around, but we initialise the matrix client in
// many different places so we'd have to pass it into all of
// them.
const { e2eEnabled } = getUrlParams();
if (!e2eEnabled) {
logger.info("Disabling E2E: group call signalling will NOT be encrypted.");
}
const client = createClient({
...baseOpts,
...clientOptions,
useAuthorizationHeader: true,
// Use a relatively low timeout for API calls: this is a realtime app
// so we don't want API calls taking ages, we'd rather they just fail.
localTimeoutMs: 5000,
useE2eForGroupCall: e2eEnabled,
fallbackICEServerAllowed: fallbackICEServerAllowed,
});
// In case of logging in a new matrix account but there is still crypto local store. This is needed for:
// - We lost the auth tokens and cannot restore the client resulting in registering a new user.
// - We start the sign in flow but are registered with a guest user. (It should additionally log out the guest before)
// - A new account is created because of missing LocalStorage: "matrix-auth-store", but the crypto IndexDB is still available.
if (!restore) {
await client.clearStores();
}
// Start client store.
// Note: The `client.store` is used to store things like sync results. It's independent of
// the cryptostore, and uses a separate indexeddb database.
try {
await client.store.startup();
} catch (error) {
logger.error(
"Error starting matrix client indexDB store. Falling back to memory store.",
error,
);
client.store = new MemoryStore({ localStorage });
await client.store.startup();
}
// Also creates and starts any crypto related stores.
try {
await client.initRustCrypto();
} catch (err) {
logger.warn(
err,
"Make sure to clear client stores before initializing the rust crypto.",
);
}
// Once startClient is called, syncs are run asynchronously.
// Also, sync completion is communicated only via events.
// So, apply the event listener *before* starting the client.
// Otherwise, a sync may complete before the listener gets applied,
// and we will miss it.
const syncPromise = waitForSync(client);
await client.startClient({ clientWellKnownPollPeriod: 60 * 10 });
await syncPromise;
return client;
}
async function waitForSync(client: MatrixClient): Promise<void> {
// If there is a saved sync, the client will fire an additional sync event
// for restoring it before it runs the first network sync.
// However, the sync we want to wait for is the network sync,
// as the saved sync may be missing some state.
// Thus, don't resolve on the first sync when we know it's for the saved sync.
let waitForSavedSync = !!(await client.store.getSavedSyncToken());
return new Promise<void>((resolve, reject) => {
const onSync = (
state: SyncState,
_old: SyncState | null,
data?: ISyncStateData,
): void => {
if (state === "PREPARED") {
if (waitForSavedSync) {
waitForSavedSync = false;
} else {
client.removeListener(ClientEvent.Sync, onSync);
resolve();
}
} else if (state === "ERROR") {
client.removeListener(ClientEvent.Sync, onSync);
reject(data?.error);
}
};
client.on(ClientEvent.Sync, onSync);
});
}
+8 -152
View File
@@ -5,21 +5,12 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
ClientEvent,
createClient,
IndexedDBStore,
MemoryStore,
Preset,
Visibility,
} from "matrix-js-sdk";
import { type ISyncStateData, type SyncState } from "matrix-js-sdk/lib/sync";
import { ClientEvent, Preset, Visibility } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import { secureRandomBase64Url } from "matrix-js-sdk/lib/randomstring";
import type { ICreateClientOpts, MatrixClient, Room } from "matrix-js-sdk";
import IndexedDBWorker from "../IndexedDBWorker?worker";
import { generateUrlSearchParams, getUrlParams } from "../UrlParams";
import type { MatrixClient, Room } from "matrix-js-sdk";
import { generateUrlSearchParams } from "../UrlParams";
import { Config } from "../config/Config";
import { E2eeType } from "../e2ee/e2eeType";
import {
@@ -27,149 +18,14 @@ import {
saveKeyForRoom,
} from "../e2ee/sharedKeyManagement";
// Shared with the component, which reaches this module for the room URLs. So
// nothing in here may import the client store worker: the bundler emits the
// worker file on loading the import, used or not. The one function that needs
// it, `initClient`, lives in `./initClient` for that reason.
export const fallbackICEServerAllowed =
import.meta.env.VITE_FALLBACK_STUN_ALLOWED === "true";
const SYNC_STORE_NAME = "element-call-sync";
async function waitForSync(client: MatrixClient): Promise<void> {
// If there is a saved sync, the client will fire an additional sync event
// for restoring it before it runs the first network sync.
// However, the sync we want to wait for is the network sync,
// as the saved sync may be missing some state.
// Thus, don't resolve on the first sync when we know it's for the saved sync.
let waitForSavedSync = !!(await client.store.getSavedSyncToken());
return new Promise<void>((resolve, reject) => {
const onSync = (
state: SyncState,
_old: SyncState | null,
data?: ISyncStateData,
): void => {
if (state === "PREPARED") {
if (waitForSavedSync) {
waitForSavedSync = false;
} else {
client.removeListener(ClientEvent.Sync, onSync);
resolve();
}
} else if (state === "ERROR") {
client.removeListener(ClientEvent.Sync, onSync);
reject(data?.error);
}
};
client.on(ClientEvent.Sync, onSync);
});
}
/**
* Initialises and returns a new standalone Matrix Client.
* This can only be called safely if no other client is running
* otherwise rust crypto will throw since it is not ready to initialize a new session.
* If another client is running make sure `.logout()` is called before executing this function.
* @param clientOptions Object of options passed through to the client
* @param restore If the rust crypto should be reset before the client initialization or
* if the initialization should try to restore the crypto state from the indexDB.
* @returns The MatrixClient instance
*/
export async function initClient(
clientOptions: ICreateClientOpts,
restore: boolean,
): Promise<MatrixClient> {
let indexedDB: IDBFactory | undefined;
try {
indexedDB = window.indexedDB;
} catch (e) {
logger.warn("Could not get indexDB from window.", e);
}
// options we always pass to the client (stuff that we need in order to work)
const baseOpts = {
fallbackICEServerAllowed: fallbackICEServerAllowed,
isVoipWithNoMediaAllowed:
Config.get().features?.feature_group_calls_without_video_and_audio,
} as ICreateClientOpts;
if (indexedDB && localStorage) {
baseOpts.store = new IndexedDBStore({
indexedDB: window.indexedDB,
localStorage,
dbName: SYNC_STORE_NAME,
// We can't use the worker in dev mode because Vite simply doesn't bundle workers
// in dev mode: it expects them to use native modules. Ours don't, and even then only
// Chrome supports it. (It bundles them fine in production mode.)
workerFactory: import.meta.env.DEV
? undefined
: (): Worker => new IndexedDBWorker(),
});
} else if (localStorage) {
baseOpts.store = new MemoryStore({ localStorage });
}
// XXX: we read from the URL params in RoomPage too:
// it would be much better to read them in one place and pass
// the values around, but we initialise the matrix client in
// many different places so we'd have to pass it into all of
// them.
const { e2eEnabled } = getUrlParams();
if (!e2eEnabled) {
logger.info("Disabling E2E: group call signalling will NOT be encrypted.");
}
const client = createClient({
...baseOpts,
...clientOptions,
useAuthorizationHeader: true,
// Use a relatively low timeout for API calls: this is a realtime app
// so we don't want API calls taking ages, we'd rather they just fail.
localTimeoutMs: 5000,
useE2eForGroupCall: e2eEnabled,
fallbackICEServerAllowed: fallbackICEServerAllowed,
});
// In case of logging in a new matrix account but there is still crypto local store. This is needed for:
// - We lost the auth tokens and cannot restore the client resulting in registering a new user.
// - We start the sign in flow but are registered with a guest user. (It should additionally log out the guest before)
// - A new account is created because of missing LocalStorage: "matrix-auth-store", but the crypto IndexDB is still available.
if (!restore) {
await client.clearStores();
}
// Start client store.
// Note: The `client.store` is used to store things like sync results. It's independent of
// the cryptostore, and uses a separate indexeddb database.
try {
await client.store.startup();
} catch (error) {
logger.error(
"Error starting matrix client indexDB store. Falling back to memory store.",
error,
);
client.store = new MemoryStore({ localStorage });
await client.store.startup();
}
// Also creates and starts any crypto related stores.
try {
await client.initRustCrypto();
} catch (err) {
logger.warn(
err,
"Make sure to clear client stores before initializing the rust crypto.",
);
}
// Once startClient is called, syncs are run asynchronously.
// Also, sync completion is communicated only via events.
// So, apply the event listener *before* starting the client.
// Otherwise, a sync may complete before the listener gets applied,
// and we will miss it.
const syncPromise = waitForSync(client);
await client.startClient({ clientWellKnownPollPeriod: 60 * 10 });
await syncPromise;
return client;
}
export function roomAliasLocalpartFromRoomName(roomName: string): string {
return roomName
.trim()
+83
View File
@@ -0,0 +1,83 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { type MatrixClient, MatrixError } from "matrix-js-sdk";
import { initClient } from "./initClient";
import { initSPA } from "./spa";
import { Config } from "../config/Config";
import { type Session } from "../ClientContext";
vi.mock("./initClient", () => ({ initClient: vi.fn() }));
const session: Session = {
user_id: "@alice:example.org",
device_id: "AAAA",
access_token: "secret",
passwordlessUser: true,
};
const client = {} as MatrixClient;
describe("initSPA", () => {
const clearSession = vi.fn();
beforeEach(() => {
vi.spyOn(Config, "defaultHomeserverUrl").mockReturnValue(
"https://hs.example.org",
);
});
afterEach(() => {
vi.restoreAllMocks();
vi.clearAllMocks();
});
it("continues without a client when no session is stored", async () => {
await expect(initSPA(() => undefined, clearSession)).resolves.toBeNull();
expect(initClient).not.toHaveBeenCalled();
expect(clearSession).not.toHaveBeenCalled();
});
it("restores the stored session into a client", async () => {
vi.mocked(initClient).mockResolvedValue(client);
await expect(initSPA(() => session, clearSession)).resolves.toEqual({
client,
passwordlessUser: true,
});
expect(initClient).toHaveBeenCalledWith(
expect.objectContaining({
baseUrl: "https://hs.example.org",
accessToken: "secret",
userId: "@alice:example.org",
deviceId: "AAAA",
}),
// Restoring: the crypto store is the session's, not to be wiped
true,
);
});
it("drops a session the homeserver no longer knows", async () => {
vi.mocked(initClient).mockRejectedValue(
new MatrixError({ errcode: "M_UNKNOWN_TOKEN", error: "Unknown token" }),
);
await expect(initSPA(() => session, clearSession)).resolves.toBeNull();
expect(clearSession).toHaveBeenCalled();
});
it("drops the session and fails on any other error", async () => {
vi.mocked(initClient).mockRejectedValue(new Error("offline"));
await expect(initSPA(() => session, clearSession)).rejects.toThrow(
"offline",
);
expect(clearSession).toHaveBeenCalled();
});
});
+2 -1
View File
@@ -9,7 +9,8 @@ import { type ICreateClientOpts, MatrixError } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import { Config } from "../config/Config";
import { fallbackICEServerAllowed, initClient } from "./matrix";
import { fallbackICEServerAllowed } from "./matrix";
import { initClient } from "./initClient";
import type { InitResult, Session } from "../ClientContext";
export async function initSPA(
+9 -7
View File
@@ -196,21 +196,23 @@ export function getBasicCallViewModelEnvironment(
});
const vm = createCallViewModel$(
scope,
rtcParticipation,
matrixRoom,
mediaDevices,
muteStates,
{
rtcParticipation,
matrixRoom,
mediaDevices,
muteStates,
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
viewSize$: constant({ width: 1000, height: 800 }),
handsRaisedSubject$: handRaisedSubject$,
reactionsSubject$,
},
{
sentCallNotification$: createSentCallNotification$(
scope,
rtcSession.asMockedSession(),
),
windowSize$: constant({ width: 1000, height: 800 }),
...callViewModelOptions,
},
handRaisedSubject$,
reactionsSubject$,
);
const footerVm = createCallFooterViewModel(
testScope(),