Compare commits

..
Author SHA1 Message Date
Robin f287fb3800 Adapt to matrix-js-sdk removal of oldest_membership support 2026-09-30 00:45:19 +02:00
Robin f9b601504b Stop sending livekit_alias in memberships
It's been many versions since anything used the value.
2026-09-30 00:45:19 +02:00
Timo 81233e5cfb Merge pull request #4269 from element-hq/toger5/component-api-entry
Export EC component related types
2026-09-29 16:24:26 +02:00
Robin fbce73ae55 Merge pull request #4302 from element-hq/always-multi-sfu
Remove multi-SFU check which is always true
2026-09-29 09:14:42 +02:00
renovate[bot] ce79274ccc Update ghcr.io/element-hq/element-web:develop Docker digest to cbb7be9 (#4303)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-29 01:07:59 +00:00
Robin 8db2e8ac41 Remove multi-SFU check which is always true
Compatibility and Matrix 2.0 are the only MatrixRTC modes now.
2026-09-28 22:03:16 +02:00
Johannes Marbach 12352b94f5 Merge pull request #4299 from element-hq/johannes/drop-observable-hooks
Drop observable-hooks in favor of useBehavior
2026-09-28 18:12:43 +02:00
Johannes MarbachandRobin e393b6a984 Swap conditional ordering
Co-authored-by: Robin <robin@robin.town>
2026-09-28 17:56:08 +02:00
Johannes Marbach 3f63a6fc3c Drop observable-hooks in favor of useBehavior
With this change, views now read every reactive value through `useBehavior`, as
`architecture.md` already prescribes. Where a view reads a plain stream, the view
model now exposes a `Behavior`.
2026-09-28 17:54:50 +02:00
Johannes Marbach addc1b10a0 Merge pull request #4297 from element-hq/renovate/compound
Update dependency @vector-im/compound-web to v10.1.0
2026-09-28 08:23:32 +02:00
renovate[bot] ee0d7dc0cb Update ghcr.io/element-hq/element-web:develop Docker digest to cfbe7ce (#4296)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-28 01:14:27 +00:00
renovate[bot] 7b1f3f28ed Update dependency @vector-im/compound-web to v10.1.0 2026-09-28 01:03:27 +00:00
renovate[bot] 8c44ecd61b Update ghcr.io/element-hq/element-web:develop Docker digest to 8301993 (#4295)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-27 01:25:15 +00:00
renovate[bot] 7d92ddf0c9 Update ghcr.io/element-hq/element-web:develop Docker digest to d97061c (#4294)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-26 02:21:38 +00:00
Johannes Marbach 5c497cc6a8 Fix build on Windows 2026-09-25 16:15:39 +02:00
fkwpandClaude Opus 5 b7bd7d18b2 [Feature] Quick audio menu (#4275)
* Give the microphone level its own observable
* Add a microphone level meter
* Show speakers and microphones in the quick audio menu
* Wire speaker selection through the call footer
* Add stories for the meter and the device menu
* Add end-to-end specs for the quick audio menu
* Keep the footer while a menu opened from it is open
* Use the shared audio capture stub in the lobby test
---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 09:52:02 +00:00
fkwpandClaude Opus 5.5 b4b72432b9 Tighten the comment guidance in AGENTS.md (#4279)
* Say what earns a comment its place

- Add a comments section to the code style: no comment by default, the reason
  only, and the kinds that go stale.
- Point to it from the PR checklist in AGENTS.md.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 08:44:04 +00:00
Robin ac1f5e2c8c Merge pull request #4277 from element-hq/vitest-warning
Fix Vitest import warning
2026-09-23 16:04:47 +02:00
renovate[bot] 98d77cc816 Update ghcr.io/element-hq/element-web:develop Docker digest to 0636bbe (#4276)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-09-23 01:55:54 +00:00
Robin d5812ddd34 Fix Vitest import warning
Fixes the following warning:

[warn] (!) Your Vite config uses features that are unsupported by `configLoader: 'native'`, which is planned to become the default in a future major version of Vite:
  - import "./vite.config" without a file extension (vitest.config.ts:3:35). Add the file extension
Set `VITE_CONFIG_NATIVE_IGNORE_WARNING=true` to suppress this warning.
2026-09-22 23:34:19 +02:00
Johannes Marbach cf7c275fb7 Merge pull request #4274 from element-hq/hs/bell
Bell reaction sound.
2026-09-22 12:32:38 +02:00
Half-Shot 93b2661513 Switch to 🛎️ 2026-09-22 11:21:11 +01:00
Robin 1f84cb9c5c Merge pull request #4273 from element-hq/simplify-local-transport
Simplify local transport code
2026-09-22 12:15:34 +02:00
Half-Shot 4269aaa163 update snap 2026-09-22 10:04:12 +01:00
Half-Shot bc62bb9565 lint 2026-09-22 10:01:22 +01:00
Half-Shot be56bbd05b lint 2026-09-22 10:01:10 +01:00
Half-Shot eb44fd545d Bell reaction sound. 2026-09-22 09:57:43 +01:00
Robin c9f911b7c9 Swallow local transport errors in connection manager 2026-09-22 10:01:14 +02:00
Robin 3a189d155b Remove references to outdated variables or types 2026-09-22 09:56:02 +02:00
Robin 0211d5c367 Remove unused shareReplay operator 2026-09-22 09:45:48 +02:00
Robin c88f884fc9 Simplify local transport code
As of the removal of 'Legacy' mode, there can no longer be a difference between the transport you advertise in your membership and the transport you publish media on, so the local transport code can be simplified considerably. For instance, the function which gets the local transport can simply return a promise rather than being reactive.

I kept the local transport as an Observable in other modules so that they could easily be tested with existing marble tests.
2026-09-21 12:51:28 +02:00
Johannes Marbach 9f7c35cd8f Merge pull request #4270 from element-hq/renovate/major-compound
Update dependency @vector-im/compound-design-tokens to v11
2026-09-21 07:51:35 +02:00
renovate[bot] ac81a7c0ab Update dependency @vector-im/compound-design-tokens to v11 2026-09-21 02:56:00 +00:00
Timo K. 8ebb0b5a90 Export component related types. 2026-09-19 08:46:30 +02:00
86 changed files with 1997 additions and 2203 deletions
+2
View File
@@ -48,6 +48,8 @@ pnpm i18n:check
- View model + marble tests, thin view, a story per state, an e2e spec.
- Any new shared component called out explicitly, with why nothing existing fit.
- Template filled for real: what, why, before/after screenshots, repro steps.
- Comments only where the code can't give the reason — see
[code-style.md](docs/agents/code-style.md#comments).
- Anything that redraws continuously — an animation frame, a stream — says what it
costs, or better, counts the redraws in a test.
- Branch `<handle>/<topic>`. Plain imperative commit subjects, no prefixes.
+23 -6
View File
@@ -34,16 +34,16 @@ You can find the latest development version continuously deployed to
## ✨ Key Features
✅ **Decentralized & Federated** – No central authority; works across Matrix
homeservers.
✅ **End-to-End Encrypted** – Secure and private calls.
homeservers.
✅ **End-to-End Encrypted** – Secure and private calls.
✅ **Standalone, Widget & Component Mode** – Use as an independent app, embed
in Matrix clients as a widget, or (experimentally) mount it as a React component
inside your own application.
✅ **WebRTC-based** – No additional software required.
inside your own application.
✅ **WebRTC-based** – No additional software required.
✅ **Scalable with LiveKit** – Supports large meetings via SFU
([MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)).
([MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md)).
✅ **Raise Hand** – Participants can signal when they want to speak, helping to
organize the flow of the meeting.
organize the flow of the meeting.
✅ **Emoji Reactions** – Users can react with emojis 👍️ 🎉 👏 🤘, adding
engagement and interactivity to the conversation.
@@ -309,6 +309,23 @@ root. The host imports the component from
`react-dom`, `matrix-js-sdk` and `livekit-client` itself, since the bundle leaves
them external.
The component is large, and a host will usually load it lazily, only once a
call is shown. Everything a host needs in order to talk about a call before
then is also exported
from `@element-hq/element-call-component/api` (a small entry point that does not
load the component).
- props and the types they are made of
- the handle
- the host bridge
- `UserIntent`
- `HeaderStyle`
- `BackgroundStyle` enums
- `configurationForIntent`, the defaults each intent implies
Import from it wherever a value such as
`BackgroundStyle.Solid` is needed on a path that must stay light; the main
entry point re-exports all of it too.
### Backend
A docker compose file `docker-compose-dev.yml` is provided to start the
+124
View File
@@ -0,0 +1,124 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* EXPERIMENTAL
*
* What a host needs in order to talk about Element Call, without Element Call:
* the component's props and the types they are made of, the handle and the
* host bridge, the intents a call can be started with and the configuration
* each implies.
*
* This is an entry point of its own, `@element-hq/element-call-component/api`,
* so that a host can import it on a path that must stay light — a call model
* that needs `BackgroundStyle.Solid` as a value, say — and load the component
* itself, and everything it brings, only when a call is shown. The main entry
* point re-exports all of it, so a host that does not care may import
* everything from there.
*
* Nothing in here may reach anything that is not a type or a constant: the
* point of this module is what it does not import.
*/
import type { Ref } from "react";
import type { MatrixClient } from "matrix-js-sdk";
import type { UrlProperties } from "../src/UrlParams";
import type { ElementCallHandle, ElementCallHostBridge } from "./host";
import {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
} from "../src/UrlConfiguration";
// How the host and Element Call talk to each other, and what they say
export type { ElementCallHandle, ElementCallHostBridge } from "./host";
export type { DeviceMuteRequest, DeviceMuteState } from "../src/HostBridge";
export type { JoinCallData } from "../src/widget";
// The deployment-wide configuration, as distinct from ElementCallConfiguration
// below, which is per call
export type { ConfigOptions } from "../src/config/ConfigOptions";
// The values that appear in ElementCallConfiguration and in the intent, and
// what each intent means by default
export {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
};
/**
* How Element Call should behave. Everything is optional; anything left out
* takes the default that {@link ElementCallProps.intent} implies.
*
* This is the behaviour a widget can be configured with through its URL, plus
* the one fact about the call a host has a say in here, the background. The
* rest of what a widget's URL carries — who the user is, how to reach the
* homeserver, where to report analytics, the shared secret of a room that is
* encrypted with one — a component host supplies by other routes, or not at
* all; and what can change while the call is running, the theme and the
* language, is a prop of its own.
*/
export type ElementCallConfiguration = Partial<UrlConfiguration> &
Partial<Pick<UrlProperties, "background">>;
export interface ElementCallProps {
/**
* The client to place the call with. Element Call does not authenticate
* anyone or manage a session of its own; this one is the host's.
*/
client: MatrixClient;
/** The room to call in. The host's client must already know about it. */
roomId: string;
/**
* What the user asked for — whether they started the call or joined one that
* was already running, and whether it is a call in a group or a DM. Element
* Call decides what each of those means: whether to show the lobby first,
* whether to ring, and so on.
*
* Defaults to joining an existing group call, which is the most conservative
* reading, but a host that knows which button the user pressed should say so.
*/
intent?: UserIntent;
/**
* How Element Call should behave, overriding whatever {@link intent} implies.
* A host that finds itself setting a lot of these probably wants a different
* intent instead.
*
* Compared by value, so it is fine to write this inline; only a change to
* what it says restarts anything.
*/
config?: ElementCallConfiguration;
/**
* What Element Call tells the host while the call is running: that the user
* has joined or hung up, that it would like to be kept on screen, and so on.
* Without one, Element Call assumes nobody is listening.
*/
hostBridge?: ElementCallHostBridge;
/**
* What the host tells Element Call: to hang up, to mute, to join. Available
* once the component has rendered.
*/
ref?: Ref<ElementCallHandle>;
/**
* The theme to show Element Call in, `light` or `dark`. Left out, Element
* Call picks. Changes take effect at once, and cost nothing else.
*/
theme?: string;
/**
* The language to show Element Call in, as a BCP 47 tag: one of the
* `supportedLanguages` the main entry point exports, or something that falls
* back to one (`de-AT` to `de`). Left out, the browser's language is used.
*
* Translations are one thing shared by every Element Call on the page, so
* the most recently set language wins for all of them.
*/
language?: string;
}
+6 -96
View File
@@ -36,14 +36,12 @@ import {
type FC,
type JSX,
type ReactNode,
type Ref,
useEffect,
useLayoutEffect,
useMemo,
useRef,
useState,
} from "react";
import { type MatrixClient } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import { I18nextProvider } from "react-i18next";
import { TooltipProvider } from "@vector-im/compound-web";
@@ -62,10 +60,8 @@ import { RootElementProvider, useRootElement } from "../src/RootElementContext";
import {
configurationForIntent,
componentProperties,
type UrlConfiguration,
type UrlParams,
UrlParamsProvider,
type UrlProperties,
UserIntent,
useUrlParams,
} from "../src/UrlParams";
@@ -79,102 +75,16 @@ import { i18n } from "../src/utils/i18n";
import { useTheme } from "../src/useTheme";
import { useStableValue } from "../src/useStableValue";
import styles from "./ElementCall.module.css";
import {
type ElementCallHandle,
type ElementCallHostBridge,
useComponentHostBridge,
} from "./host";
import { useComponentHostBridge } from "./host";
import { type ElementCallProps } from "./api";
import { supportedLanguages, translationsBackend } from "./localization";
// The languages Element Call can be shown in
export { supportedLanguages } from "./localization";
// How the host and Element Call talk to each other, and what they say
export { type ElementCallHandle, type ElementCallHostBridge } from "./host";
export {
type DeviceMuteRequest,
type DeviceMuteState,
} from "../src/HostBridge";
export { type JoinCallData } from "../src/widget";
// The deployment-wide configuration, as distinct from ElementCallConfiguration
// above, which is per call
export { type ConfigOptions } from "../src/config/ConfigOptions";
// The values that appear in ElementCallConfiguration and in the intent
export {
BackgroundStyle,
HeaderStyle,
UserIntent,
type UrlConfiguration,
} from "../src/UrlParams";
/**
* How Element Call should behave. Everything is optional; anything left out
* takes the default that {@link ElementCallProps.intent} implies.
*
* This is the behaviour a widget can be configured with through its URL, plus
* the one fact about the call a host has a say in here, the background. The
* rest of what a widget's URL carries — who the user is, how to reach the
* homeserver, where to report analytics, the shared secret of a room that is
* encrypted with one — a component host supplies by other routes, or not at
* all; and what can change while the call is running, the theme and the
* language, is a prop of its own.
*/
export type ElementCallConfiguration = Partial<UrlConfiguration> &
Partial<Pick<UrlProperties, "background">>;
export interface ElementCallProps {
/**
* The client to place the call with. Element Call does not authenticate
* anyone or manage a session of its own; this one is the host's.
*/
client: MatrixClient;
/** The room to call in. The host's client must already know about it. */
roomId: string;
/**
* What the user asked for — whether they started the call or joined one that
* was already running, and whether it is a call in a group or a DM. Element
* Call decides what each of those means: whether to show the lobby first,
* whether to ring, and so on.
*
* Defaults to joining an existing group call, which is the most conservative
* reading, but a host that knows which button the user pressed should say so.
*/
intent?: UserIntent;
/**
* How Element Call should behave, overriding whatever {@link intent} implies.
* A host that finds itself setting a lot of these probably wants a different
* intent instead.
*
* Compared by value, so it is fine to write this inline; only a change to
* what it says restarts anything.
*/
config?: ElementCallConfiguration;
/**
* What Element Call tells the host while the call is running: that the user
* has joined or hung up, that it would like to be kept on screen, and so on.
* Without one, Element Call assumes nobody is listening.
*/
hostBridge?: ElementCallHostBridge;
/**
* What the host tells Element Call: to hang up, to mute, to join. Available
* once the component has rendered.
*/
ref?: Ref<ElementCallHandle>;
/**
* The theme to show Element Call in, `light` or `dark`. Left out, Element
* Call picks. Changes take effect at once, and cost nothing else.
*/
theme?: string;
/**
* The language to show Element Call in, as a BCP 47 tag: one of
* {@link supportedLanguages}, or something that falls back to one (`de-AT`
* to `de`). Left out, the browser's language is used.
*
* Translations are one thing shared by every Element Call on the page, so
* the most recently set language wins for all of them.
*/
language?: string;
}
// Everything a host needs to talk about Element Call — the props, the handle,
// the host bridge, the intents and what each means — also available without
// the component itself from `@element-hq/element-call-component/api`
export * from "./api";
/**
* Prepares the things Element Call needs before it can be shown: translations,
+4
View File
@@ -25,6 +25,10 @@
"types": "./dist/types/component/index.d.ts",
"default": "./dist/element-call.js"
},
"./api": {
"types": "./dist/types/component/api.d.ts",
"default": "./dist/api.js"
},
"./style.css": "./dist/element-call.css"
},
"sideEffects": [
+3 -3
View File
@@ -13,8 +13,8 @@
"rootDir": "..",
"outDir": "./dist/types"
},
// The entry point, plus the ambient declarations (CSS modules, `?react` SVGs,
// `import.meta.env`, …) that the sources it reaches rely on.
"include": ["./index.tsx", "../src/@types/*.d.ts"],
// The entry points, plus the ambient declarations (CSS modules, `?react` SVGs,
// `import.meta.env`, …) that the sources they reach rely on.
"include": ["./index.tsx", "./api.ts", "../src/@types/*.d.ts"],
"exclude": []
}
+2 -2
View File
@@ -19,7 +19,7 @@ services:
- ./backend/playwright_homeserver-othersite.yaml:/data/cfg/homeserver.yaml:Z
element-web:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:3993d53bc2c784c0376b86e00230aef0ee2c9a705b2166e9229555117e8db167
image: ghcr.io/element-hq/element-web:develop@sha256:cbb7be9a54352997748d02c36dc70ee877c705ee3518be31497717c9a2ff0395
element-web-1:
# Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
image: ghcr.io/element-hq/element-web:develop@sha256:3993d53bc2c784c0376b86e00230aef0ee2c9a705b2166e9229555117e8db167
image: ghcr.io/element-hq/element-web:develop@sha256:cbb7be9a54352997748d02c36dc70ee877c705ee3518be31497717c9a2ff0395
+4 -1
View File
@@ -76,4 +76,7 @@ Code that builds in only one is a bug.
- `build:component` — `@element-hq/element-call-component`, sources in `component/`
(its own pnpm project; run pnpm from the repo root). Host API is in the README;
`pnpm lint:externals` rejects an import of a `react` / `react-dom` /
`matrix-js-sdk` / `livekit-client` subpath the externals list omits.
`matrix-js-sdk` / `livekit-client` subpath the externals list omits. Two entry
points: `index.tsx` (the component) and `api.ts` (types, enums and
`configurationForIntent`, for a host that must not load the component). `api.ts`
may only reach types and constants — `src/UrlConfiguration.ts`, not `UrlParams`.
+22 -1
View File
@@ -31,7 +31,28 @@ copy, the PR says so.
- **Suites.** Test cases first, helpers below. Older suites invert this; follow the
rule in new ones, don't reorder old ones.
- **Names.** What a thing means, not what it is made of: `naturalLayout$`, not
`computedLayout$`. Comments explain why, never what.
`computedLayout$`.
## Comments
No comment is the default. One earns its place by giving a reason the code
cannot: a platform quirk, an outside constraint, a trap that looks like a
simplification.
- Say the reason, then stop. Test each sentence: does the reason still stand
without it? If so, cut it. Most reasons fit on a line, so a long comment is
worth a second look — it is usually carrying something that isn't one.
- Write about the line the comment sits on, in terms of what the code does
there — leave layout to the stylesheet and measurements to the design.
- Name the condition ("where the platform can't route audio"), not the browser
or the spec entry.
- Put history — what was tried, removed or rejected, the dev journal — in the
commit or the PR description.
- In tests, comment only structure that looks wrong but isn't: batching, timers,
why this scroll position.
Before keeping one: would it still be true after a CSS change or a refactor of
its caller? If not, cut it.
## TypeScript, React and RxJS idiom win on a clash
+1 -1
View File
@@ -24,7 +24,7 @@ homeserver does not support sticky events. This is the default.
MSC4354 and all clients on v0.17.0 or later. The local membership requests its
token from the Matrix 2.0 JWT endpoint of the
[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)
and identifies the room by a hashed identity instead of a `livekit_alias`.
and identifies the room by a hashed identity instead of the Matrix `room_id`.
(Remote memberships always try the new endpoint first and fall back to the
legacy one, so remote participants can be on either.)
+1 -1
View File
@@ -166,7 +166,7 @@
"microphone_level": {
"label": "Microphone level",
"no_device": "No microphone found. Connect one, then try again.",
"permission_denied": "Element Call cannot use your microphone. Allow access in your browser settings.",
"permission_denied": "Microphone access is blocked. Allow it, then try again.",
"value": "{{level}} of {{max}}"
},
"microphone_off": "Microphone off",
+3 -4
View File
@@ -8,7 +8,7 @@
"dev:embedded": "vite --config vite-embedded.config.js",
"dev:component": "vite --config vite-component-dev.config.ts",
"build": "pnpm build:full",
"build:full": "NODE_OPTIONS=--max-old-space-size=16384 vite build",
"build:full": "node --max-old-space-size=16384 node_modules/vite/bin/vite.js build",
"build:full:production": "pnpm build:full",
"build:full:development": "pnpm build:full --mode development",
"build:embedded": "pnpm build:full --config vite-embedded.config.js",
@@ -83,7 +83,7 @@
"@types/sdp-transform": "^2.4.5",
"@typescript-eslint/utils": "^8.61.0",
"@use-gesture/react": "^10.2.11",
"@vector-im/compound-design-tokens": "^10.0.0",
"@vector-im/compound-design-tokens": "^11.0.0",
"@vector-im/compound-web": "^10.0.0",
"@vitejs/plugin-react": "^6.0.2",
"@vitest/browser-playwright": "^4.1.5",
@@ -103,11 +103,10 @@
"livekit-client": "^2.18.1",
"lodash-es": "^4.17.21",
"loglevel": "^1.9.1",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#develop",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#robin/remove-oldest-membership",
"matrix-widget-api": "^1.18.0",
"node-stdlib-browser": "^1.3.1",
"normalize.css": "^8.0.1",
"observable-hooks": "^4.2.3",
"oxfmt": "^0.66.0",
"oxlint": "^1.70.0",
"oxlint-tsgolint": "^0.25.0",
+15 -39
View File
@@ -18,11 +18,9 @@ test.describe("the quick audio menu", () => {
await joinACall(page, "Menu user", "Audio menu");
await openAudioMenu(page);
// The speaker list is what the settings modal used to be the only home of.
await expect(page.getByRole("group", { name: "Speaker" })).toBeVisible();
await expect(page.getByRole("group", { name: "Microphone" })).toBeVisible();
// Named rather than counted: the browser contributes its own fake output
// and a "Default" entry, so a total would be a fact about the browser.
// By name, not count: the browser adds its own fake output and a Default.
for (const n of [1, 2, 3])
await expect(
page
@@ -30,8 +28,6 @@ test.describe("the quick audio menu", () => {
.getByRole("menuitemradio", { name: `Fake Speaker ${n}` }),
).toBeVisible();
// Only one entry of a kind is marked, and the meter reports a number
// rather than a colour.
await expect(
page.getByRole("menuitemradio", { checked: true }),
).toHaveCount(2);
@@ -40,10 +36,7 @@ test.describe("the quick audio menu", () => {
await expect(meter).toHaveAttribute("aria-valuenow", /\d+/);
await expect(meter).toHaveAttribute("aria-valuetext", /\d+ of \d+/);
// Both browsers in the matrix can route audio to a chosen output, so the
// section offers a real choice. The case where a platform cannot — Safari,
// and anything without setSinkId — is covered by a unit check, since no
// browser here can reach it.
// Both browsers here can route audio; the platform that can't is a unit check.
await expect(
page
.getByRole("group", { name: "Speaker" })
@@ -55,7 +48,7 @@ test.describe("the quick audio menu", () => {
test("moves the microphone and the speaker without disturbing the call", async ({
browser,
}) => {
// Two browsers, two joins and a real call between them.
// Two browsers and a real call.
test.slow();
const hostContext = await browser.newContext({ reducedMotion: "reduce" });
const host = await hostContext.newPage();
@@ -73,9 +66,7 @@ test.describe("the quick audio menu", () => {
await openAudioMenu(host);
await selectDevice(host, "Speaker", "Fake Speaker 2");
// The point of the criterion: the switch is not a rejoin. Neither side
// sees the call drop, and the guest still has both tiles — so the host
// never left and came back.
// Not a rejoin: neither side drops, and the guest still has both tiles.
await expect(
host.getByRole("dialog", { name: "Reconnecting…" }),
).not.toBeVisible();
@@ -92,7 +83,7 @@ test.describe("the quick audio menu", () => {
test("keeps the meter moving while muted, and sends nothing", async ({
browser,
}) => {
// Two browsers, two joins and a real call between them.
// Two browsers and a real call.
test.slow();
const hostContext = await browser.newContext({ reducedMotion: "reduce" });
const host = await hostContext.newPage();
@@ -110,15 +101,12 @@ test.describe("the quick audio menu", () => {
await expect(mute).toHaveAttribute("aria-checked", "false");
await openAudioMenu(host);
// The microphone is held open while muted, so the meter still reports the
// hardware. The mute control is what says nothing is being transmitted.
const meter = host.getByRole("meter", { name: "Microphone level" });
await expect(meter).toBeVisible();
// Queried by test id, not by role: the menu is modal, so Radix takes the
// rest of the call out of the accessibility tree while it is open.
// By test id: the modal menu hides the rest of the call from the a11y tree.
await expect(mute).toHaveAttribute("aria-checked", "false");
await expect(mute).toBeVisible();
// And the listener is told so, rather than being left to guess from silence.
// And the guest is shown the mute.
await expect(
guest.getByTestId("videoTile").filter({ hasText: "Muted host" }),
).toBeVisible();
@@ -137,9 +125,8 @@ test.describe("the quick audio menu", () => {
const meter = page.getByRole("meter", { name: "Microphone level" });
await expect(meter).toBeVisible();
// Scrolled so the microphones start at the top of the list and run past its
// bottom: the position that tells a pinned meter from one that merely
// happens to be last.
// Scrolled so the microphones start at the top: only there does a pinned
// meter differ from one that is simply last.
const list = page.locator("[role='menu'] div[role='none']").first();
await list.evaluate((element) => {
const group = element.querySelector("[role='group'][aria-label*='icro']");
@@ -150,7 +137,6 @@ test.describe("the quick audio menu", () => {
await expect(meter).toBeInViewport();
await expectPinnedInside(meter, list);
// Every entry stays reachable, which is what the scroll is for.
await expect(
page.getByRole("menuitemradio", { name: "Fake Microphone 20" }),
).toBeVisible();
@@ -169,16 +155,14 @@ test.describe("the quick audio menu", () => {
await openAudioMenu(page);
const first = page.getByRole("menuitemradio").first();
// Opened by pointer, so no ring, even though Radix has moved focus into the
// menu already.
// Opened by pointer: no ring, though Radix has moved focus into the menu.
await expect.poll(async () => outlineWidth(first)).toBe(0);
await page.keyboard.press("ArrowDown");
const focused = page.locator("[role='menuitemradio']:focus");
await expect.poll(async () => outlineWidth(focused)).toBeGreaterThan(0);
// The pointer takes it away again: the menu focuses whatever it is over, so
// a ring that followed focus alone would trail the mouse.
// And the pointer takes it away again.
await first.hover();
await expect.poll(async () => outlineWidth(focused)).toBe(0);
});
@@ -193,8 +177,7 @@ async function joinACall(
await page.goto("/");
await SpaHelpers.createCall(page, userName, callName, true);
await expect(page.getByTestId("name_tag")).toContainText(userName);
// The media controls stay disabled until the devices have enumerated, and
// every test here drives them.
// The media controls stay disabled until devices have enumerated.
await expect(page.getByTestId("incall_mute")).toBeEnabled({
timeout: 10_000,
});
@@ -214,19 +197,12 @@ async function selectDevice(
.getByRole("group", { name: section })
.getByRole("menuitemradio", { name });
await item.click();
// Selecting does not close the menu — the component prevents the default so
// the list survives a mis-click — so it is dismissed explicitly.
// Selecting doesn't close the menu, so dismiss it.
await page.keyboard.press("Escape");
await expect(page.getByRole("menu")).not.toBeVisible();
}
/**
* Asserts the meter sits within the scrollport, and inside the menu's frame.
*
* The meter is the one opaque element in the menu, so it is the one thing that
* can paint over the border. Whether it actually does needs a screenshot; this
* pins the geometry that decides it.
*/
/** Asserts the meter sits within the scrollport and inside the menu's frame. */
async function expectPinnedInside(
meter: Locator,
list: Locator,
@@ -243,7 +219,7 @@ async function expectPinnedInside(
expect(meterBox.x + meterBox.width).toBeLessThan(frame.x + frame.width);
}
/** The painted outline width in pixels, however the stylesheet spells it. */
/** Painted outline width, in px. */
async function outlineWidth(item: Locator): Promise<number> {
if ((await item.count()) === 0) return 0;
return item.first().evaluate((element) => {
+16 -46
View File
@@ -11,22 +11,12 @@ import { createUserAndRoom, resizeContainer, startHarness } from "./harness.ts";
import { installFakeDevices } from "../utils/fake-devices.ts";
/**
* The device menu where Element Call is a component in a host's page rather
* than the whole of one.
*
* This is the case the stylesheets cannot describe: the menu is portalled to
* the document, so a container query and a viewport unit both measure the wrong
* thing — the first has no container to resolve against out there, the second
* measures a page Element Call does not own. The menu has to be sized against
* the space the call is actually drawn in.
*
* Driven from the lobby rather than a joined call. The footer builds the same
* menu from the same device behaviours in both, and the container is the same
* size either way, so joining would only add two connections' worth of flake.
* The device menu with Element Call as a component in a host page, where the
* portalled menu must be sized against the call, not the window. Driven from
* the lobby: it builds the same menu, without two connections' worth of flake.
*/
// Signing in, setting up crypto and syncing happen twice before anything is on
// screen, as in component-call.spec.ts
// Sign-in, crypto setup and sync happen twice before anything shows.
test.describe.configure({ timeout: 180_000 });
test("sizes the device list against the call, not the window", async ({
@@ -37,8 +27,7 @@ test("sizes the device list against the call, not the window", async ({
const panes = await startHarness(page, username, roomId);
const pane = panes.first();
// A short call in a much taller page: the difference between measuring the
// call and measuring the window.
// A short call in a much taller page.
const container = pane.getByTestId("call-container");
await resizeContainer(container, { width: 900, height: 400 });
await expect(pane.getByTestId("lobby_joinCall")).toBeVisible({
@@ -50,8 +39,6 @@ test("sizes the device list against the call, not the window", async ({
const windowHeight = page.viewportSize()!.height;
const listHeight = (await list.boundingBox())!.height;
// Sized against the call. Were it sized against the window the list would be
// half as tall again, and the assertion below would not be able to tell.
expect(callHeight).toBeLessThan(windowHeight * 0.75);
expect(listHeight).toBeLessThanOrEqual(callHeight);
expect(listHeight).toBeLessThan(windowHeight * 0.6);
@@ -72,9 +59,7 @@ test("follows the call area when the host resizes it", async ({ page }) => {
const list = await openDeviceList(page, pane);
const whenShort = (await list.boundingBox())!.height;
// The host grows the space Element Call is drawn in while the menu is open —
// a panel opening, a window dragged, a phone turned. A bound taken once on
// opening would still describe the smaller call.
// The host grows the call while the menu is open.
await resizeContainer(container, { width: 900, height: 700 });
await expect
.poll(async () => (await list.boundingBox())!.height)
@@ -88,15 +73,13 @@ test("keeps every device reachable in a small container", async ({ page }) => {
const pane = panes.first();
const container = pane.getByTestId("call-container");
// Narrow as well as short, which is where entries get pushed out of reach.
await resizeContainer(container, { width: 400, height: 360 });
await expect(pane.getByTestId("lobby_joinCall")).toBeVisible({
timeout: 60_000,
});
const list = await openDeviceList(page, pane);
// More devices than the space allows, so the list has to scroll rather than
// put entries somewhere they cannot be got at.
// More devices than fit, so the list must scroll.
expect(await list.evaluate((el) => el.scrollHeight > el.clientHeight)).toBe(
true,
);
@@ -104,14 +87,12 @@ test("keeps every device reachable in a small container", async ({ page }) => {
const last = page.getByRole("menuitemradio", { name: "Fake Microphone 20" });
await last.scrollIntoViewIfNeeded();
await expect(last).toBeInViewport();
// Reachable means usable, not merely painted: D11 accepts that the menu may
// be drawn outside the call area, so this asserts reach rather than
// containment.
// Reachable means clickable: the menu may be drawn outside the call area.
await last.click();
await expect(last).toHaveAttribute("aria-checked", "true");
});
test("tracks the focus modality of its own call, not the page", async ({
test("tracks the focus source of its own call, not the page", async ({
page,
}) => {
await installFakeDevices(page);
@@ -124,36 +105,25 @@ test("tracks the focus modality of its own call, not the page", async ({
timeout: 60_000,
});
await openDeviceList(page, pane);
// The menu owns the modality, because every item it can focus has to answer
// to it — the device rows and the camera menu's blur toggle alike.
const menu = page.getByRole("menu");
// Asserted on the attribute rather than the painted ring, which cannot be
// read here: the menu is portalled outside the call root, and the component
// build scopes the stylesheet to it, so neither the ring nor the rule that
// suppresses the browser's own reaches this menu. The paint is asserted
// standalone instead — in the story and in audio-menu.spec.ts. What is on
// trial here is which call the tracking answers for.
await expect(menu).toHaveAttribute("data-focus-modality", "pointer");
// Asserted on the attribute: the component build's scoped stylesheet doesn't
// reach the portalled menu, so its ring can't be read here.
await expect(menu).toHaveAttribute("data-focus-source", "pointer");
// A key pressed in the other call on this page — or anywhere in the host's
// own page — says nothing about how this menu is being used.
// A key pressed in the other call says nothing about this menu.
await other.evaluate((element) =>
element.dispatchEvent(
new KeyboardEvent("keydown", { key: "ArrowDown", bubbles: true }),
),
);
await expect(menu).toHaveAttribute("data-focus-modality", "pointer");
await expect(menu).toHaveAttribute("data-focus-source", "pointer");
// A key pressed in this menu does.
await page.keyboard.press("ArrowDown");
await expect(menu).toHaveAttribute("data-focus-modality", "keyboard");
await expect(menu).toHaveAttribute("data-focus-source", "keyboard");
});
/**
* Opens the microphone menu of one component and returns its scrolling device
* list, which lives outside the component: the menu is portalled to the page.
*/
/** Opens a component's microphone menu and returns its device list. */
async function openDeviceList(page: Page, pane: Locator): Promise<Locator> {
await pane
.getByRole("button", { name: "Microphone" })
+7 -20
View File
@@ -8,20 +8,10 @@ Please see LICENSE in the repository root for full details.
import { type Page } from "@playwright/test";
/**
* Gives the browser more fake devices than it ships with.
*
* A headless browser's fake capture offers one microphone and one speaker,
* which is one short of what a device menu is for: with no choice to make, every
* entry renders disabled. These are synthetic entries on top of the real fake
* device, so the menu has a list to show and a selection to move, and the app
* runs its real device pipeline against them.
*
* What they do not do is route audio: every entry is backed by the same capture,
* and `setSinkId` is accepted rather than honoured. A test can prove that
* choosing a device changes the app's state and does not disturb the call. That
* a listener hears the change needs hardware, and stays a manual check.
*
* Must be called before the page navigates.
* Adds synthetic devices beside the browser's one fake microphone and speaker,
* so the menu has a choice to show. They share one capture and `setSinkId` is
* accepted but not honoured: routing still needs hardware and a manual check.
* Call before the page navigates.
*/
export async function installFakeDevices(
page: Page,
@@ -58,8 +48,7 @@ export async function installFakeDevices(
...synthetic("audiooutput", speakers, "Fake Speaker"),
];
// Our ids name no hardware, so an exact-device constraint on one would be
// rejected. Drop it and let the one real fake device answer.
// Our ids name no hardware, so drop the exact-device constraint.
const getUserMedia = devices.getUserMedia.bind(devices);
devices.getUserMedia = async (
constraints?: MediaStreamConstraints,
@@ -77,10 +66,8 @@ export async function installFakeDevices(
return getUserMedia(constraints);
};
// Routing to a device that does not exist would reject, and the app
// treats that as a failed switch. Both sinks are patched: Element Call
// routes its own AudioContext as well as the media elements, and leaving
// that one alone logs a NotFoundError for every switch.
// Accept routing to our ids on both media elements and the AudioContext,
// which the app also routes.
for (const proto of [
HTMLMediaElement.prototype,
AudioContext.prototype,
+28 -44
View File
@@ -131,11 +131,11 @@ importers:
specifier: ^10.2.11
version: 10.3.1(react@19.2.8)
'@vector-im/compound-design-tokens':
specifier: ^10.0.0
version: 10.2.4(@types/react@19.2.18)(react@19.2.8)
specifier: ^11.0.0
version: 11.0.0(@types/react@19.2.18)(react@19.2.8)
'@vector-im/compound-web':
specifier: ^10.0.0
version: 10.0.1(@fontsource/inconsolata@5.3.0)(@fontsource/inter@5.3.0)(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(@vector-im/compound-design-tokens@10.2.4(@types/react@19.2.18)(react@19.2.8))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
version: 10.1.0(@fontsource/inconsolata@5.3.0)(@fontsource/inter@5.3.0)(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(@vector-im/compound-design-tokens@11.0.0(@types/react@19.2.18)(react@19.2.8))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@vitejs/plugin-react':
specifier: ^6.0.2
version: 6.1.1(@rolldown/plugin-babel@0.2.3(@babel/core@7.29.7(supports-color@7.2.0))(@babel/runtime@8.0.0)(rolldown@1.2.7)(vite@8.2.2(@types/node@24.13.3)(esbuild@0.28.2)(jiti@2.7.0)(sass@1.103.1)(terser@5.46.1)(yaml@2.9.0)))(babel-plugin-react-compiler@1.0.0)(vite@8.2.2(@types/node@24.13.3)(esbuild@0.28.2)(jiti@2.7.0)(sass@1.103.1)(terser@5.46.1)(yaml@2.9.0))
@@ -191,8 +191,8 @@ importers:
specifier: ^1.9.1
version: 1.9.2
matrix-js-sdk:
specifier: github:matrix-org/matrix-js-sdk#develop
version: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/24929be0e741be6a5d0a7226f1c682e245263b8a
specifier: github:matrix-org/matrix-js-sdk#robin/remove-oldest-membership
version: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/6defa47adae28feb7e3906dde25089e0dce42ece
matrix-widget-api:
specifier: ^1.18.0
version: 1.19.0
@@ -202,9 +202,6 @@ importers:
normalize.css:
specifier: ^8.0.1
version: 8.0.1
observable-hooks:
specifier: ^4.2.3
version: 4.2.4(react-dom@19.2.8(react@19.2.8))(react@19.2.8)(rxjs@7.8.2)
oxfmt:
specifier: ^0.66.0
version: 0.66.0
@@ -1197,8 +1194,8 @@ packages:
'@types/dom-mediacapture-transform': ^0.1.9
livekit-client: ^1.12.0 || ^2.1.0
'@matrix-org/matrix-sdk-crypto-wasm@18.5.0':
resolution: {integrity: sha512-E826Hy1rG26LanPjtSsOiVRcVoHfSgPgj2r2Xsb5RPScpaKi9XJADQ0u3dNjRCitZPX4oyLNl1FDd5AfDlmIwQ==}
'@matrix-org/matrix-sdk-crypto-wasm@18.9.0':
resolution: {integrity: sha512-NtLzNDTEfNASBjNCTGZHuk7N0jrQhS6Zx2CM857rCHZYYh/rpbKGQNrJ7azktm4S1P5Jy3WU3nwJTGohGw26qQ==}
engines: {node: '>= 18'}
'@mdx-js/react@3.1.1':
@@ -3229,8 +3226,8 @@ packages:
peerDependencies:
react: '>= 16.8.0'
'@vector-im/compound-design-tokens@10.2.4':
resolution: {integrity: sha512-2aCjhSvJxktPWnvh5s9Xdc0Rd76AfHqbDbN4QjOxo2VIac6zYdLyQbxMUQ5ZeoR9G3vAxILd0H1UvApRfMilpQ==}
'@vector-im/compound-design-tokens@11.0.0':
resolution: {integrity: sha512-pWz0Q67Xbo8gwzaTeFbTTcM6i29ju6PR7SqyuExr2nIV3j9K785cOInft8FA2CM7PEapkr6TRtJKR/4xtWh2RA==}
peerDependencies:
'@adobe/leonardo-contrast-colors': ^1.0.0
'@types/react': '*'
@@ -3246,8 +3243,8 @@ packages:
react:
optional: true
'@vector-im/compound-web@10.0.1':
resolution: {integrity: sha512-CT3aGMalPSf53o3JUbBF7EA10Rfp+auftIuBwOUSna+hfZ/zPxGLTFcuCuOJ7L8hrzijM+V4I8L0PUkfyOczpg==}
'@vector-im/compound-web@10.1.0':
resolution: {integrity: sha512-Lhw9Q+RZo1GtDYjF5PXWTD0zdfbrz/vnSVBYFHC8Hq+YeDule2QfGbTRo8IVHeV7jK6ZzsLc8zb6553eeIRLYg==}
peerDependencies:
'@fontsource/inconsolata': ^5
'@fontsource/inter': ^5
@@ -3662,8 +3659,8 @@ packages:
constants-browserify@1.0.0:
resolution: {integrity: sha512-xFxOwqIzR/e1k1gLiWEophSCMqXcwVHIH7akf7b/vxcUeGunlj3hvZaaqxwHsTgn+IndtkQJgSztIDWeumWJDQ==}
content-type@3.0.0:
resolution: {integrity: sha512-AIi5H6p0xk5uknXcN3/rmhP8jgp69OfSe/JuKiQAFprJ7UGw7mwj7m4XcmDzlrnJDG+cGpphAINGdU3g3g7kDw==}
content-type@3.1.1:
resolution: {integrity: sha512-GW4qUsfFo59d0HbUibDlWv5wPz+vAAcaTWbKIuKCf0JkC7wWkSyf8f13IpXn5JkeMlB2P8iTSSIjwXljorg2vA==}
engines: {node: '>=22'}
convert-source-map@2.0.0:
@@ -4683,9 +4680,9 @@ packages:
matrix-events-sdk@0.0.1:
resolution: {integrity: sha512-1QEOsXO+bhyCroIe2/A5OwaxHvBm7EsSQ46DEDn8RBIfQwN5HWBpFvyWWR4QY0KHPPnnJdI99wgRiAl7Ad5qaA==}
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/24929be0e741be6a5d0a7226f1c682e245263b8a:
resolution: {gitHosted: true, integrity: sha512-0EGwbXzvG88sBOxq4ZlCRwT4VpQj2xkizfyy0A3bTSK0ysb6/gXJ/o1WPFoqCjRGMDgPq4gAsk92xGtHA7EhPg==, tarball: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/24929be0e741be6a5d0a7226f1c682e245263b8a}
version: 42.2.0
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/6defa47adae28feb7e3906dde25089e0dce42ece:
resolution: {gitHosted: true, integrity: sha512-jHFmI2C29HJnzs/FIrqdmQD6ytyunyy627onhrxI+xxTGSfYrWhcTUFstNiCAaMt3fc0nNi9aVvLPD3IIglF2A==, tarball: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/6defa47adae28feb7e3906dde25089e0dce42ece}
version: 42.3.0
engines: {node: '>=22.0.0'}
matrix-widget-api@1.19.0:
@@ -4816,13 +4813,6 @@ packages:
resolution: {integrity: sha512-nK28WOo+QIjBkDduTINE4JkF/UJJKyf2EJxvJKfblDpyg0Q+pkOHNTL0Qwy6NP6FhE/EnzV73BxxqcJaXY9anw==}
engines: {node: '>= 0.4'}
observable-hooks@4.2.4:
resolution: {integrity: sha512-FdTQgyw1h5bG/QHCBIqctdBSnv9VARJCEilgpV6L2qlw1yeLqFIwPm4U15dMtl5kDmNN0hSt+Nl6iYbLFwEcQA==}
peerDependencies:
react: '>=16.8.0'
react-dom: '>=16.8.0'
rxjs: '>=6.0.0'
obug@2.1.4:
resolution: {integrity: sha512-4a+OsYv9UktOJKE+l1A4OufDgdRF9PifWj+tJnHURo/P+WOxpG4GzUFL9qCalmWauao6ogiG+QvnCovwPoyAWA==}
engines: {node: '>=12.20.0'}
@@ -4905,8 +4895,8 @@ packages:
resolution: {integrity: sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw==}
engines: {node: '>=10'}
p-retry@8.0.0:
resolution: {integrity: sha512-kFVqH1HxOHp8LupNsOys7bSV09VYTRLxarH/mokO4Rqhk6wGi70E0jh4VzvVGXfEVNggHoHLAMWsQqHyU1Ey9A==}
p-retry@8.0.1:
resolution: {integrity: sha512-NAigyu8hfvJe7MsS18mnc4is0MZtau3QMdBklaJKK23oBQb6occO3UPM3vHmTckW8KhfyjMZaUOqdTFEYliHgg==}
engines: {node: '>=22'}
p-try@2.2.0:
@@ -7111,7 +7101,7 @@ snapshots:
'@types/dom-mediacapture-transform': 0.1.11
livekit-client: 2.22.3(@types/dom-mediacapture-record@1.0.22)
'@matrix-org/matrix-sdk-crypto-wasm@18.5.0': {}
'@matrix-org/matrix-sdk-crypto-wasm@18.9.0': {}
'@mdx-js/react@3.1.1(@types/react@19.2.18)(react@19.2.8)':
dependencies:
@@ -8695,12 +8685,12 @@ snapshots:
'@use-gesture/core': 10.3.1
react: 19.2.8
'@vector-im/compound-design-tokens@10.2.4(@types/react@19.2.18)(react@19.2.8)':
'@vector-im/compound-design-tokens@11.0.0(@types/react@19.2.18)(react@19.2.8)':
optionalDependencies:
'@types/react': 19.2.18
react: 19.2.8
'@vector-im/compound-web@10.0.1(@fontsource/inconsolata@5.3.0)(@fontsource/inter@5.3.0)(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(@vector-im/compound-design-tokens@10.2.4(@types/react@19.2.18)(react@19.2.8))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
'@vector-im/compound-web@10.1.0(@fontsource/inconsolata@5.3.0)(@fontsource/inter@5.3.0)(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(@vector-im/compound-design-tokens@11.0.0(@types/react@19.2.18)(react@19.2.8))(react-dom@19.2.8(react@19.2.8))(react@19.2.8)':
dependencies:
'@floating-ui/react': 0.27.20(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@floating-ui/react-dom': 2.1.9(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
@@ -8712,7 +8702,7 @@ snapshots:
'@radix-ui/react-progress': 1.1.16(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@radix-ui/react-separator': 1.1.15(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
'@radix-ui/react-slot': 1.3.3(@types/react@19.2.18)(react@19.2.8)
'@vector-im/compound-design-tokens': 10.2.4(@types/react@19.2.18)(react@19.2.8)
'@vector-im/compound-design-tokens': 11.0.0(@types/react@19.2.18)(react@19.2.8)
classnames: 2.5.1
react: 19.2.8
vaul: 1.1.2(@types/react-dom@19.2.5(@types/react@19.2.18))(@types/react@19.2.18)(react-dom@19.2.8(react@19.2.8))(react@19.2.8)
@@ -9173,7 +9163,7 @@ snapshots:
constants-browserify@1.0.0: {}
content-type@3.0.0: {}
content-type@3.1.1: {}
convert-source-map@2.0.0: {}
@@ -10259,17 +10249,17 @@ snapshots:
matrix-events-sdk@0.0.1: {}
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/24929be0e741be6a5d0a7226f1c682e245263b8a:
matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/6defa47adae28feb7e3906dde25089e0dce42ece:
dependencies:
'@babel/runtime': 8.0.0
'@matrix-org/matrix-sdk-crypto-wasm': 18.5.0
'@matrix-org/matrix-sdk-crypto-wasm': 18.9.0
another-json: 0.2.0
bs58: 6.0.0
content-type: 3.0.0
content-type: 3.1.1
loglevel: 1.9.2
matrix-events-sdk: 0.0.1
matrix-widget-api: 1.19.0
p-retry: 8.0.0
p-retry: 8.0.1
sdp-transform: 3.0.0
unhomoglyph: 1.0.6
@@ -10410,12 +10400,6 @@ snapshots:
has-symbols: 1.1.0
object-keys: 1.1.1
observable-hooks@4.2.4(react-dom@19.2.8(react@19.2.8))(react@19.2.8)(rxjs@7.8.2):
dependencies:
react: 19.2.8
react-dom: 19.2.8(react@19.2.8)
rxjs: 7.8.2
obug@2.1.4: {}
once@1.4.0:
@@ -10600,7 +10584,7 @@ snapshots:
dependencies:
p-limit: 3.1.0
p-retry@8.0.0:
p-retry@8.0.1:
dependencies:
is-network-error: 1.3.2
+2 -2
View File
@@ -6,9 +6,9 @@ Please see LICENSE in the repository root for full details.
*/
import { createContext, use, useMemo } from "react";
import { useObservableEagerState } from "observable-hooks";
import { type MediaDevices } from "./state/MediaDevices";
import { useBehavior } from "./useBehavior";
export const MediaDevicesContext = createContext<MediaDevices | undefined>(
undefined,
@@ -35,7 +35,7 @@ export const useEarpieceAudioConfig = (): {
volume: number;
} => {
const devices = useMediaDevices();
const audioOutput = useObservableEagerState(devices.audioOutput.selected$);
const audioOutput = useBehavior(devices.audioOutput.selected$);
const isVirtualEarpiece = audioOutput?.virtualEarpiece ?? false;
return {
// We use only the right speaker (pan = 1) for the earpiece.
+242
View File
@@ -0,0 +1,242 @@
/*
Copyright 2022-2024 New Vector Ltd.
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* How Element Call can be told to behave, and what each intent means.
*
* Kept apart from {@link UrlParams} on purpose: everything here is constants,
* types and one function of them, so that a host of the component can import
* these — the enums it needs as values, the defaults an intent implies — without
* loading Element Call itself. UrlParams, which reads them from a URL, needs
* the router, the config and the rest of the app; this module must not.
* `component/api.ts` is built from it as an entry point of its own.
*/
import {
type RTCCallIntent,
type RTCNotificationType,
} from "matrix-js-sdk/lib/matrixrtc";
import { platform } from "./Platform";
export enum UserIntent {
StartNewCall = "start_call",
JoinExistingCall = "join_existing",
StartNewCallVoice = "start_call_voice",
JoinExistingCallVoice = "join_existing_voice",
StartNewCallDM = "start_call_dm",
StartNewCallDMVoice = "start_call_dm_voice",
JoinExistingCallDM = "join_existing_dm",
JoinExistingCallDMVoice = "join_existing_dm_voice",
Unknown = "unknown",
}
export enum HeaderStyle {
None = "none",
Standard = "standard",
AppBar = "app_bar",
}
export enum BackgroundStyle {
Solid = "solid",
Gradient = "gradient",
}
/**
* The configuration for the app, which can be set via URL parameters.
* Those property are different to the UrlProperties, since they are all optional
* and configure the behavior of the app. Their value is the same if EC is used in
* the same context but with different accounts/users.
*
* Their defaults can be controlled by the `intent` property.
*/
export interface UrlConfiguration {
/**
* Whether the app should keep the user confined to the current call/room.
*/
confineToRoom: boolean;
/**
* Whether the app should pause before joining the call until it sees an
* io.element.join widget action, allowing it to be preloaded.
*/
preload: boolean;
/**
* The style of headers to show. "standard" is the default arrangement, "none"
* hides the header entirely, and "app_bar" produces a header with a back
* button like you might see in mobile apps. The callback for the back button
* is window.controls.onBackButtonPressed.
*/
header: HeaderStyle;
/**
* Whether the controls should be shown. For screen recording no controls can be desired.
*/
showControls: boolean;
/**
* Whether to hide the screen-sharing button.
*/
hideScreensharing: boolean;
/**
* Whether the app is allowed to use fallback STUN servers for ICE in case the
* user's homeserver doesn't provide any.
*/
allowIceFallback: boolean;
/**
* Whether the app should use per participant keys for E2EE.
*/
perParticipantE2EE: boolean;
/**
* Whether the global JS controls for audio output devices should be enabled,
* allowing the list of output devices to be controlled by the app hosting
* Element Call.
*/
controlledAudioDevices: boolean;
/**
* Setting this flag skips the lobby and brings you in the call directly.
* In the widget this can be combined with preload to pass the device settings
* with the join widget action.
*/
skipLobby: boolean;
/**
* Setting this flag makes element call show the lobby after leaving a call.
* This is useful for video rooms.
*/
returnToLobby: boolean;
/**
* Whether and what type of notification EC should send, when the user joins the call.
*/
sendNotificationType?: RTCNotificationType;
/**
* Whether the app should automatically leave the call when there
* is no one left in the call.
* This is one part to make the call matrixRTC session behave like a telephone call.
*/
autoLeaveWhenOthersLeft: boolean;
/**
* If the client should behave like it is awaiting an answer if a notification was sent (wait for call pick up).
* This is a no-op if not combined with sendNotificationType.
*
* This entails:
* - show ui that it is awaiting an answer
* - play a sound that indicates that it is awaiting an answer
* - auto-dismiss the call widget once the notification lifetime expires on the receivers side.
*/
waitForCallPickup: boolean;
/**
* Whether to enable echo cancellation for audio capture.
* Defaults to true.
*/
echoCancellation?: boolean;
/**
* Whether to enable noise suppression for audio capture.
* Defaults to true.
*/
noiseSuppression?: boolean;
callIntent?: RTCCallIntent;
}
// If you need to add a new flag to this interface, prefer a name that describes
// a specific behavior (such as 'confineToRoom'), rather than one that describes
// the situations that call for this behavior ('isEmbedded'). This makes it
// clearer what each flag means, and helps us avoid coupling Element Call's
// behavior to the needs of specific consumers.
/**
* The configuration implied by what the user meant to do — if they pressed a
* Start Call button this would be `start_call`, and if they pressed Join Call,
* `join_existing`.
*
* These are platform-specific defaults, so that a host can start a call by
* saying what the user asked for rather than by setting every parameter itself,
* and so that what each intent means is Element Call's decision, made in one
* place. A host that wants something else states it alongside the intent.
*
* {@link UserIntent.Unknown} means no intent was stated, and gives the
* standalone app's defaults: Element Call owns the whole page, so it offers the
* way out of the room that a hosted call must not.
*/
export function configurationForIntent(intent: UserIntent): UrlConfiguration {
// Only constants and `platform` here, so that this depends on nothing but
// the intent.
let preset: UrlConfiguration = {
confineToRoom: true,
preload: false,
header: platform === "desktop" ? HeaderStyle.None : HeaderStyle.AppBar,
showControls: true,
hideScreensharing: false,
allowIceFallback: true,
perParticipantE2EE: true,
controlledAudioDevices: platform === "desktop" ? false : true,
skipLobby: true,
returnToLobby: false,
sendNotificationType: "notification",
autoLeaveWhenOthersLeft: false,
waitForCallPickup: false,
};
switch (intent) {
case UserIntent.StartNewCall:
preset.skipLobby = false;
preset.callIntent = "video";
break;
case UserIntent.JoinExistingCall:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = false;
preset.callIntent = "video";
break;
case UserIntent.StartNewCallVoice:
preset.skipLobby = false;
preset.callIntent = "audio";
break;
case UserIntent.JoinExistingCallVoice:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = false;
preset.callIntent = "audio";
break;
case UserIntent.StartNewCallDMVoice:
preset.callIntent = "audio";
// Fall through
case UserIntent.StartNewCallDM:
preset.skipLobby = true;
preset.sendNotificationType = "ring";
preset.autoLeaveWhenOthersLeft = true;
preset.waitForCallPickup = true;
preset.callIntent = preset.callIntent ?? "video";
break;
case UserIntent.JoinExistingCallDMVoice:
preset.callIntent = "audio";
// Fall through
case UserIntent.JoinExistingCallDM:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = true;
preset.autoLeaveWhenOthersLeft = true;
preset.callIntent = preset.callIntent ?? "video";
break;
// Non widget usecase defaults
default:
preset = {
confineToRoom: false,
preload: false,
header: HeaderStyle.Standard,
showControls: true,
hideScreensharing: false,
allowIceFallback: false,
perParticipantE2EE: false,
controlledAudioDevices: false,
skipLobby: false,
returnToLobby: false,
sendNotificationType: undefined,
autoLeaveWhenOthersLeft: false,
waitForCallPickup: false,
};
}
return preset;
}
+17 -222
View File
@@ -9,47 +9,36 @@ Please see LICENSE in the repository root for full details.
import { createContext, use, useMemo } from "react";
import { useLocation } from "react-router-dom";
import { logger } from "matrix-js-sdk/lib/logger";
import {
type RTCCallIntent,
type RTCNotificationType,
} from "matrix-js-sdk/lib/matrixrtc";
import { pickBy } from "lodash-es";
import { Config } from "./config/Config";
import { type EncryptionSystem } from "./e2ee/sharedKeyManagement";
import { E2eeType } from "./e2ee/e2eeType";
import { platform } from "./Platform";
import {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
} from "./UrlConfiguration";
import { redact } from "./utils/redact";
// Moved to their own module, so that a host can import them without the rest
// of this one; re-exported here as they always were
export {
BackgroundStyle,
configurationForIntent,
HeaderStyle,
type UrlConfiguration,
UserIntent,
} from "./UrlConfiguration";
interface RoomIdentifier {
roomAlias: string | null;
roomId: string | null;
viaServers: string[];
}
export enum UserIntent {
StartNewCall = "start_call",
JoinExistingCall = "join_existing",
StartNewCallVoice = "start_call_voice",
JoinExistingCallVoice = "join_existing_voice",
StartNewCallDM = "start_call_dm",
StartNewCallDMVoice = "start_call_dm_voice",
JoinExistingCallDM = "join_existing_dm",
JoinExistingCallDMVoice = "join_existing_dm_voice",
Unknown = "unknown",
}
export enum HeaderStyle {
None = "none",
Standard = "standard",
AppBar = "app_bar",
}
export enum BackgroundStyle {
Solid = "solid",
Gradient = "gradient",
}
/**
* The UrlProperties are used to pass required data to the widget.
* Those are different in different rooms, users, devices. They do not configure the behavior of the
@@ -166,109 +155,6 @@ export interface UrlProperties {
*/
background: BackgroundStyle;
}
/**
* The configuration for the app, which can be set via URL parameters.
* Those property are different to the UrlProperties, since they are all optional
* and configure the behavior of the app. Their value is the same if EC is used in
* the same context but with different accounts/users.
*
* Their defaults can be controlled by the `intent` property.
*/
export interface UrlConfiguration {
/**
* Whether the app should keep the user confined to the current call/room.
*/
confineToRoom: boolean;
/**
* Whether the app should pause before joining the call until it sees an
* io.element.join widget action, allowing it to be preloaded.
*/
preload: boolean;
/**
* The style of headers to show. "standard" is the default arrangement, "none"
* hides the header entirely, and "app_bar" produces a header with a back
* button like you might see in mobile apps. The callback for the back button
* is window.controls.onBackButtonPressed.
*/
header: HeaderStyle;
/**
* Whether the controls should be shown. For screen recording no controls can be desired.
*/
showControls: boolean;
/**
* Whether to hide the screen-sharing button.
*/
hideScreensharing: boolean;
/**
* Whether the app is allowed to use fallback STUN servers for ICE in case the
* user's homeserver doesn't provide any.
*/
allowIceFallback: boolean;
/**
* Whether the app should use per participant keys for E2EE.
*/
perParticipantE2EE: boolean;
/**
* Whether the global JS controls for audio output devices should be enabled,
* allowing the list of output devices to be controlled by the app hosting
* Element Call.
*/
controlledAudioDevices: boolean;
/**
* Setting this flag skips the lobby and brings you in the call directly.
* In the widget this can be combined with preload to pass the device settings
* with the join widget action.
*/
skipLobby: boolean;
/**
* Setting this flag makes element call show the lobby after leaving a call.
* This is useful for video rooms.
*/
returnToLobby: boolean;
/**
* Whether and what type of notification EC should send, when the user joins the call.
*/
sendNotificationType?: RTCNotificationType;
/**
* Whether the app should automatically leave the call when there
* is no one left in the call.
* This is one part to make the call matrixRTC session behave like a telephone call.
*/
autoLeaveWhenOthersLeft: boolean;
/**
* If the client should behave like it is awaiting an answer if a notification was sent (wait for call pick up).
* This is a no-op if not combined with sendNotificationType.
*
* This entails:
* - show ui that it is awaiting an answer
* - play a sound that indicates that it is awaiting an answer
* - auto-dismiss the call widget once the notification lifetime expires on the receivers side.
*/
waitForCallPickup: boolean;
/**
* Whether to enable echo cancellation for audio capture.
* Defaults to true.
*/
echoCancellation?: boolean;
/**
* Whether to enable noise suppression for audio capture.
* Defaults to true.
*/
noiseSuppression?: boolean;
callIntent?: RTCCallIntent;
}
// If you need to add a new flag to this interface, prefer a name that describes
// a specific behavior (such as 'confineToRoom'), rather than one that describes
// the situations that call for this behavior ('isEmbedded'). This makes it
// clearer what each flag means, and helps us avoid coupling Element Call's
// behavior to the needs of specific consumers.
export interface UrlParams extends UrlProperties, UrlConfiguration {}
class ParamParser {
@@ -354,97 +240,6 @@ export const getUrlParams = (
return params;
};
/**
* The configuration implied by what the user meant to do — if they pressed a
* Start Call button this would be `start_call`, and if they pressed Join Call,
* `join_existing`.
*
* These are platform-specific defaults, so that a host can start a call by
* saying what the user asked for rather than by setting every parameter itself,
* and so that what each intent means is Element Call's decision, made in one
* place. A host that wants something else states it alongside the intent.
*
* {@link UserIntent.Unknown} means no intent was stated, and gives the
* standalone app's defaults: Element Call owns the whole page, so it offers the
* way out of the room that a hosted call must not.
*/
export function configurationForIntent(intent: UserIntent): UrlConfiguration {
// Only constants and `platform` here, so that this depends on nothing but
// the intent.
let preset: UrlConfiguration = {
confineToRoom: true,
preload: false,
header: platform === "desktop" ? HeaderStyle.None : HeaderStyle.AppBar,
showControls: true,
hideScreensharing: false,
allowIceFallback: true,
perParticipantE2EE: true,
controlledAudioDevices: platform === "desktop" ? false : true,
skipLobby: true,
returnToLobby: false,
sendNotificationType: "notification",
autoLeaveWhenOthersLeft: false,
waitForCallPickup: false,
};
switch (intent) {
case UserIntent.StartNewCall:
preset.skipLobby = false;
preset.callIntent = "video";
break;
case UserIntent.JoinExistingCall:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = false;
preset.callIntent = "video";
break;
case UserIntent.StartNewCallVoice:
preset.skipLobby = false;
preset.callIntent = "audio";
break;
case UserIntent.JoinExistingCallVoice:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = false;
preset.callIntent = "audio";
break;
case UserIntent.StartNewCallDMVoice:
preset.callIntent = "audio";
// Fall through
case UserIntent.StartNewCallDM:
preset.skipLobby = true;
preset.sendNotificationType = "ring";
preset.autoLeaveWhenOthersLeft = true;
preset.waitForCallPickup = true;
preset.callIntent = preset.callIntent ?? "video";
break;
case UserIntent.JoinExistingCallDMVoice:
preset.callIntent = "audio";
// Fall through
case UserIntent.JoinExistingCallDM:
// On desktop this will be overridden based on which button was used to join the call
preset.skipLobby = true;
preset.autoLeaveWhenOthersLeft = true;
preset.callIntent = preset.callIntent ?? "video";
break;
// Non widget usecase defaults
default:
preset = {
confineToRoom: false,
preload: false,
header: HeaderStyle.Standard,
showControls: true,
hideScreensharing: false,
allowIceFallback: false,
perParticipantE2EE: false,
controlledAudioDevices: false,
skipLobby: false,
returnToLobby: false,
sendNotificationType: undefined,
autoLeaveWhenOthersLeft: false,
waitForCallPickup: false,
};
}
return preset;
}
/**
* The {@link UrlProperties} for Element Call running as a component inside a
* host application.
@@ -9,7 +9,7 @@ exports[`Can close reaction dialog 1`] = `
aria-disabled="false"
aria-expanded="true"
aria-haspopup="true"
aria-labelledby="_r_bb_"
aria-labelledby="_r_bh_"
class="_button_1nw83_8 _has-icon_1nw83_60 _icon-only_1nw83_53"
data-kind="primary"
data-size="lg"
+4 -1
View File
@@ -43,7 +43,10 @@ Please see LICENSE in the repository root for full details.
inset-inline: 0;
}
.footer.overlay:has(:focus-visible) {
/* Keep the footer while it is in use: focus inside it, or a menu opened from
it. The menu is portalled out, so the trigger's aria-expanded tracks it. */
.footer.overlay:has(:focus-visible),
.footer.overlay:has([aria-expanded="true"]) {
opacity: 1;
pointer-events: initial;
}
+27
View File
@@ -13,6 +13,7 @@ import { Link } from "@vector-im/compound-web";
import type { Meta, StoryObj } from "@storybook/react-vite";
import { CallFooter, type FooterSnapshot } from "./CallFooter";
import inCallViewStyles from "../room/InCallView.module.css";
import styles from "./CallFooter.module.css";
import { useStaticViewModel } from "../state/ViewModel";
import { ReactionsSenderContext } from "../reactions/useReactionsSender";
import { type ReactionOption } from "../reactions";
@@ -175,6 +176,32 @@ export const WithAudioAndVideoOptions: Story = {
},
};
export const StaysWhileAMenuIsOpen: Story = {
...Default,
args: {
...WithAudioAndVideoOptions.args,
// In a short window the footer overlays the call and hides itself.
asOverlay: true,
showFooter: true,
},
play: async ({ canvasElement }): Promise<void> => {
const footer = canvasElement.querySelector<HTMLElement>(
'[data-testid="footer-container"]',
)!;
await userEvent.click(
within(canvasElement).getByRole("button", { name: "Microphone" }),
);
await expect(document.body.querySelector('[role="menu"]')).not.toBeNull();
// The call hides the footer with this class.
footer.classList.add(styles.hidden);
// Read after the 0.15s fade, not mid-transition.
await new Promise((settled) => setTimeout(settled, 400));
await expect(getComputedStyle(footer).opacity).toBe("1");
},
};
export const AudioBusy: Story = {
...Default,
args: {
+1 -1
View File
@@ -99,7 +99,7 @@ export interface FooterState {
/** Providing no options `[]` or `undefined` will imply that we dont have a audio fast switcher */
audioOptions: MenuOptions[];
/** Output devices shown as their own section in the audio menu. */
/** Output (speaker) devices. */
audioOutputOptions: MenuOptions[];
/** Providing no options `[]` or `undefined` will imply that we dont have a audio fast switcher */
videoOptions: MenuOptions[];
+2 -8
View File
@@ -116,7 +116,6 @@ describe("createCallFooterViewModel", () => {
it("is withheld where the platform cannot route audio to a chosen device", () => {
outputSelectionMock.mockReturnValue(false);
// Undefined is what renders the speaker section disabled.
expect(buildFooterVm().selectAudioOutputOption$.value).toBeUndefined();
});
@@ -145,9 +144,7 @@ describe("createCallFooterViewModel", () => {
selected$: constant(undefined),
select: vi.fn(),
},
// Safari enumerates no output devices whatsoever. Reproduced by the
// condition rather than by the browser, so it is checked on the
// Linux CI runners that have no Safari to check it with.
// As Safari: no outputs listed.
audioOutput: {
available$: constant(new Map<string, DeviceLabel>()),
selected$: constant(undefined),
@@ -158,10 +155,7 @@ describe("createCallFooterViewModel", () => {
{ showControls: true, header: HeaderStyle.Standard },
);
// Empty rather than undefined: undefined means this menu has no notion
// of outputs at all, as the camera menu has none, and hides the section.
// Empty means there are none to list, and the menu still shows the
// section with a default in it, disabled.
// Empty, not undefined: undefined would hide the section.
expect(vm.audioOutputOptions$.value).toEqual([]);
});
});
+2 -2
View File
@@ -104,8 +104,8 @@ function buildDeviceBehaviors(
selectedAudioOutput$: scope.behavior(
mediaDevices.audioOutput.selected$.pipe(map((s) => s?.id)),
),
// Safari and most Firefox builds cannot route audio to a chosen device at
// all. Withholding the callback is what renders the section disabled.
// Withheld where the platform can't route audio to a chosen device, which
// disables the speaker section.
selectAudioOutputOption$: constant(
supportsAudioOutputSelection()
? mediaDevices.audioOutput.select
@@ -41,55 +41,29 @@ Please see LICENSE in the repository root for full details.
flex-direction: column;
}
/* Only the device lists scroll; the level meter stays put beneath them.
The bound comes from the measured height of the call area, set by the
component: the menu is portalled outside the root, so neither a container
query nor a viewport unit describes the space it is allowed to fill. */
/* Only the device lists scroll, bounded by the measured call area: the menu is
portalled out of the root, so CSS can't size it against the call. */
.deviceList {
overflow-y: auto;
min-block-size: 0;
max-block-size: var(--device-list-max-height);
/* The heading stands over the head of this list and the level meter over its
foot. Scrolling a row flush to either edge would put it underneath one of
them, which is how a row reached by the keyboard ends up half-readable;
this keeps both their heights clear for anything the browser scrolls to.
Set by the component, which is the only thing that can measure them. */
/* Keeps a row reached by keyboard clear of the sticky heading and meter. */
scroll-padding-block: var(--device-list-scroll-padding-start, 0)
var(--device-list-scroll-padding-end, 0);
}
/* Each section is headed by its own rule, which Compound draws under a menu
heading already. The heading spans the whole menu — the menu has no padding
of its own — so that rule runs edge to edge, stopping only where the frame
is, rather than inset as a separator between the sections would be.
Compound's spacing around it is the design's too, bar one thing: the design
leaves a section's first device further below the rule than Compound does —
measured off the mock, 27px from the rule to the top of the control, against
the 16px that Compound's 8px margin and the row's own 8px padding give. The
difference goes below the rule, so the rule stays tight under its own text
where Compound put it. */
/* Compound's heading rule heads each section; the design adds space below it. */
.menu h3 {
margin-block-end: var(--cpd-space-5x);
}
/* One section is set much further from the one above it than Compound's 8px
heading margin allows: measured off the mock, 41px from the last device's
control to the next heading's text, against the 22px we had. Only between
sections — the first heading keeps Compound's spacing, because the menu's
own padding is already above it. */
/* And more above a heading that follows another section. */
.deviceList [role="group"] + [role="group"] h3 {
margin-block-start: var(--cpd-space-7x);
}
/* Each section's heading stays at the top of the scrollport while any of that
section is still in view, so a long list never leaves you wondering which
kind of device you are looking at. It leaves with its own section, because
sticky only holds while the group it belongs to is in view.
Opaque, and held a border width clear of the sides, for the same reason as
the meter below: this is a positioned element, so it paints above the
outline the menu draws its frame with, and would swallow it. */
/* Sticky within its own section. Opaque and inset a border width so it doesn't
paint over the menu's outline. */
.sectionHeading {
position: sticky;
inset-block-start: 0;
@@ -98,40 +72,23 @@ Please see LICENSE in the repository root for full details.
margin-block-start: var(--cpd-border-width-1);
}
/* The meter belongs to the microphone section: it stays at the bottom of the
scrollport while that section is in view, and leaves with it when the list
is scrolled up to the speakers. The wrapper is deliberately unpositioned —
a positioned one paints above the menu's outline and swallows the frame
along this whole section. Sticky is resolved against the scroll container,
so it does not need one. */
/* Sticky at the foot of the microphone section; opaque and inset as above. */
.stickyMeter {
position: sticky;
inset-block-end: 0;
/* Opaque, so the list does not show through it as it scrolls past. The menu
draws its frame as an outline inset by one border width, and the device
rows are transparent at rest, so this is the only thing that can cover it:
hold it clear on the sides and the bottom. */
background: var(--cpd-color-bg-canvas-default);
margin-inline: var(--cpd-border-width-1);
margin-block-end: var(--cpd-border-width-1);
}
/* Radix focuses whatever the pointer is over, so the browser's own ring marks
the item under the mouse. It cannot distinguish the two modalities here, so
it is suppressed and replaced by one that can.
Every kind of item the menu can focus, not only the device rows: the camera
menu's blur toggle is a checkbox item and a child of the menu rather than of
the list, and left out it kept the browser's own ring and followed the
pointer with it. */
/* Radix focuses whatever the pointer is over, so the browser's ring would
follow the mouse. Replaced by one shown only for keyboard focus. */
.menu [role^="menuitem"]:focus,
.menu [role^="menuitem"]:focus-visible {
outline: none;
}
/* Shown only when the keyboard is what moved the focus. */
.menu[data-focus-modality="keyboard"] [role^="menuitem"]:focus {
.menu[data-focus-source="keyboard"] [role^="menuitem"]:focus {
outline: var(--cpd-border-width-2) solid var(--cpd-color-border-focused);
outline-offset: calc(-1 * var(--cpd-border-width-2));
}
@@ -21,24 +21,14 @@ const mediaDevices = new MediaDevices(globalScope, {
controlledAudioDevices: false,
});
/**
* Gives these stories a microphone to read.
*
* The menu opens a capture of whichever device it has been told is selected,
* and the devices in a story are invented: asking for one by an id no hardware
* answers to fails, and the meter reports that — correctly — as there being no
* microphone. So the story provides one rather than borrowing the machine's: a
* wavering tone played into a real MediaStream, which the meter then runs its
* own analyser over. Nothing here stands in for the meter itself.
*/
/** Supplies a microphone, a wavering tone: a story's invented device ids match no hardware. */
const WithAMicrophone: FC<{ children: ReactNode }> = ({ children }) => {
useEffect(() => {
const context = new AudioContext();
const microphone = context.createMediaStreamDestination();
const tone = context.createOscillator();
const loudness = context.createGain();
// Swinging between about a third and two thirds of the range, so the meter
// reads as something live rather than as a level someone pinned there.
// Wavers between a third and two thirds of full scale, so it reads as live.
const swing = context.createOscillator();
const depth = context.createGain();
loudness.gain.value = 0.25;
@@ -53,8 +43,7 @@ const WithAMicrophone: FC<{ children: ReactNode }> = ({ children }) => {
const devices = navigator.mediaDevices;
const openedForReal = devices.getUserMedia.bind(devices);
const opened = Promise.resolve(microphone.stream);
// A fresh clone each time, so that a caller stopping its tracks when it is
// done does not take the microphone away from the next one.
// A fresh clone each time, so a caller stopping its tracks doesn't end the next.
devices.getUserMedia = async (): Promise<MediaStream> =>
(await opened).clone();
@@ -69,26 +58,14 @@ const WithAMicrophone: FC<{ children: ReactNode }> = ({ children }) => {
return <>{children}</>;
};
/**
* Gives these stories the call area the menu belongs to.
*
* The menu sizes its device list against the space Element Call is drawn in,
* and takes that from a provider. Without one it falls back to the document
* body — which in a story is the whole of Storybook's frame, so the list is
* bounded by something far larger than the story it is drawn in and runs off
* the top of the canvas. Supplying a root is the same courtesy as supplying the
* devices: the story stands in for the call, so it has to say how big it is.
*/
/** Supplies a call-sized root. Without one the list is bounded by the whole Storybook frame. */
const WithACallArea: FC<{ children: ReactNode }> = ({ children }) => {
const [callArea, setCallArea] = useState<HTMLElement | null>(null);
return (
<div
ref={setCallArea}
style={{
// The size of a call, not of a thumbnail: the device list is bounded to
// a share of this, so a small area makes even a two-device menu scroll,
// which no real call does. Tall enough to leave the menu room to open
// upward and still be wholly on screen in the story's frame.
// A call's size: anything smaller makes a short device list scroll.
blockSize: 720,
display: "flex",
alignItems: "flex-end",
@@ -122,7 +99,6 @@ type Story = StoryObj<typeof meta>;
export const Default: Story = {
args: {
title: "SomeMenu",
iconsAndLabels: "audio",
enabled: true,
options: [
@@ -130,10 +106,7 @@ export const Default: Story = {
{ label: { type: "name", name: "Option 2" }, id: "2" },
],
selectedOption: "1",
// The audio menu always has a speaker section: the footer hands it an
// output list whenever it draws the chevron at all, so a microphone menu
// with no speakers in it is a shape nothing in the app produces. Set here
// rather than in each story, since the others build on these.
// The footer always passes an output list to the audio menu.
outputOptions: [
{ label: { type: "default", name: "Built-in Output" }, id: "default" },
{ label: { type: "name", name: "Headset" }, id: "spk2" },
@@ -148,7 +121,6 @@ export const Default: Story = {
export const AudioMute: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: false,
options: [
@@ -172,7 +144,6 @@ export const AudioMute: Story = {
export const AudioUnmute: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: [
@@ -185,7 +156,6 @@ export const AudioUnmute: Story = {
export const VideoMute: Story = {
args: {
title: "Camera",
iconsAndLabels: "video",
enabled: false,
options: [
@@ -199,7 +169,6 @@ export const VideoMute: Story = {
export const VideoUnmute: Story = {
args: {
title: "Camera",
iconsAndLabels: "video",
enabled: true,
options: [
@@ -215,7 +184,6 @@ export const VideoUnmute: Story = {
export const SpeakerAndMicrophoneSections: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: [
@@ -240,16 +208,13 @@ export const SpeakerAndMicrophoneSections: Story = {
await userEvent.click(headset);
await expect(args.onSelectOutput).toHaveBeenCalledWith("spk2");
// A handful of devices fits: only a list longer than the space it is given
// scrolls, and a menu that scrolled at four devices would be bounded by
// something far smaller than the call it is drawn in.
// A few devices fit, so the list doesn't scroll.
const list = document.body.querySelector<HTMLElement>(
`.${styles.deviceList}`,
)!;
await expect(list.scrollHeight).toBe(list.clientHeight);
// Each section is headed by its own rule, running the full width of the
// menu rather than inset — and nothing divides the sections besides.
// Each section is headed by its own edge-to-edge rule, with no separator.
const menu = document.body.querySelector("[role='menu']")!;
await expect(
document.body.querySelectorAll("[role='separator']"),
@@ -264,16 +229,14 @@ export const SpeakerAndMicrophoneSections: Story = {
await expect(
Number.parseFloat(getComputedStyle(rule).borderBottomWidth),
).toBeGreaterThan(0);
// Edge to edge, stopping only where the menu's frame is drawn.
// Edge to edge, within the frame.
const box = rule.getBoundingClientRect();
await expect(box.left - frame.left).toBeLessThanOrEqual(2);
await expect(frame.right - box.right).toBeLessThanOrEqual(2);
}
// A section's first device sits further below the rule than it does from
// the menu's edge. Stated as the relationship rather than a number: what
// the design asks for is the asymmetry, and Compound's own heading margin
// alone would make the two equal.
// The first device sits further below the rule than from the menu's edge.
// Asserted as a relationship, not pixels: the asymmetry is the design.
const control = document.body.querySelector("input[type='radio']")!;
const ruleBottom = headings[0]
.querySelector("h3")!
@@ -281,8 +244,8 @@ export const SpeakerAndMicrophoneSections: Story = {
const box = control.getBoundingClientRect();
await expect(box.top - ruleBottom).toBeGreaterThan(box.left - frame.left);
// And one section stands further from the one above it than a heading does
// from its own first device — again the relationship, not a number.
// And a section stands further from the one above than from its own first
// device.
const groups = document.body.querySelectorAll("[role='group']");
const speakers = groups[0].querySelectorAll("input[type='radio']");
const lastSpeaker = speakers[speakers.length - 1].getBoundingClientRect();
@@ -296,7 +259,6 @@ export const SpeakerAndMicrophoneSections: Story = {
export const OutputCannotBeChosen: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: [
@@ -326,7 +288,6 @@ export const OutputCannotBeChosen: Story = {
export const OnlyOneDevice: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: [{ label: { type: "name", name: "Microphone 1" }, id: "mic1" }],
@@ -339,7 +300,7 @@ export const OnlyOneDevice: Story = {
const canvas = within(canvasElement);
await userEvent.click(canvas.getByRole("button", { name: "Microphone" }));
// Shown rather than hidden, so the menu keeps its shape everywhere.
// Shown, disabled, rather than hidden.
const only = await within(document.body).findByRole("menuitemradio", {
name: "Microphone 1",
});
@@ -347,14 +308,10 @@ export const OnlyOneDevice: Story = {
},
};
/**
* A device has been asked for and has not arrived. Nothing in either section
* can be picked until it does, so a second request cannot overtake the first.
*/
/** A requested device hasn't arrived: nothing in either section can be picked. */
export const SelectionSettling: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: [
@@ -374,8 +331,7 @@ export const SelectionSettling: Story = {
await userEvent.click(canvas.getByRole("button", { name: "Microphone" }));
const menu = within(document.body);
// The story never changes `selectedOption`, which is what a device that has
// not taken effect yet looks like from here.
// selectedOption never changes, so the request stays in flight.
await userEvent.click(
await menu.findByRole("menuitemradio", { name: "Microphone 2" }),
);
@@ -387,17 +343,12 @@ export const SelectionSettling: Story = {
};
/**
* The focus ring belongs to the keyboard. Radix focuses whatever the pointer is
* over, so a ring that followed focus alone would trail the mouse.
*
* Asserted on the painted outline rather than on `data-focus-modality`: the
* attribute is what the stylesheet keys off, so asserting it would pass even
* with the rule deleted.
* The ring shows for keyboard focus only. Asserted on the painted outline:
* asserting data-focus-source would still pass with the CSS rule deleted.
*/
export const KeyboardFocusRing: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: [
@@ -428,14 +379,10 @@ export const KeyboardFocusRing: Story = {
},
};
/**
* More devices than the menu can show. The list scrolls, and the meter stays at
* the foot of the Microphone section rather than scrolling away with it.
*/
/** The list scrolls; the meter stays at the foot of the microphone section. */
export const ManyDevices: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: Array.from({ length: 20 }, (_, i) => ({
@@ -455,11 +402,8 @@ export const ManyDevices: Story = {
await userEvent.click(canvas.getByRole("button", { name: "Microphone" }));
const menu = within(document.body);
// The scroll container and the opaque sticky wrapper, named rather than
// walked: the nesting between them is layout, and it moves. The wrapper
// rather than the meter itself, because this story is about where the meter
// sits, not what it reads — without a fake microphone, as on WebKit, it
// says it has no permission instead of showing a level.
// The sticky wrapper rather than the meter: without a fake microphone (as on
// WebKit) the meter shows a message instead of a level.
const list = document.body.querySelector<HTMLElement>(
`.${styles.deviceList}`,
)!;
@@ -472,10 +416,8 @@ export const ManyDevices: Story = {
});
await expect(list.scrollHeight).toBeGreaterThan(list.clientHeight);
// Scrolled so the Microphone section starts at the top of the scrollport.
// Its devices then run past the bottom, which is the position that tells a
// pinned meter from one that simply happens to be the last element: at the
// very bottom of the list the two look identical.
// Scrolled so the microphone section starts at the top: only there does a
// pinned meter differ from one that is simply last.
const group = await menu.findByRole("group", { name: "Microphone" });
list.scrollTop +=
group.getBoundingClientRect().top - list.getBoundingClientRect().top;
@@ -488,39 +430,30 @@ export const ManyDevices: Story = {
await expect(pinned.bottom).toBeLessThanOrEqual(scrollport.bottom + 1);
await expect(pinned.top).toBeGreaterThanOrEqual(scrollport.top - 1);
// The whole menu is on screen. It opens upward from the foot of the call,
// so a list bounded by something bigger than the call — the document, say —
// runs off the top and takes the speakers with it.
// The whole menu is on screen, not bounded by the document.
const frame = document.body
.querySelector("[role='menu']")!
.getBoundingClientRect();
await expect(frame.top).toBeGreaterThanOrEqual(0);
await expect(frame.bottom).toBeLessThanOrEqual(window.innerHeight + 1);
// The meter is the one opaque thing in the menu, so it is the one thing
// that can cover the frame. Its box has to stay inside the menu's own. The
// paint itself needs a screenshot; this pins the geometry that decides it.
// The meter is the menu's one opaque part, so it must stay inside the frame.
await expect(pinned.left).toBeGreaterThan(frame.left);
await expect(pinned.right).toBeLessThan(frame.right);
},
};
/** The painted outline width, in pixels, however the stylesheet spells it. */
/** Painted outline width, in px. */
function outlineWidth(element: HTMLElement): number {
const { outlineStyle, outlineWidth } = getComputedStyle(element);
if (outlineStyle === "none") return 0;
return Number.parseFloat(outlineWidth) || 0;
}
/**
* A platform that enumerates no output devices and offers no way to choose one
* — Safari. The section still names where audio is going, disabled, rather than
* leaving a heading with nothing under it.
*/
/** Safari lists no outputs: a disabled, selected Default stands in. */
export const OutputNotEnumerated: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: [
@@ -543,19 +476,10 @@ export const OutputNotEnumerated: Story = {
},
};
/**
* The level meter's icon sits on the same centre line as the radio controls of
* the devices above it.
*
* Held here because it is a fact about two components side by side, and because
* layout decides it: the meter's row is inset to keep the menu's frame clear,
* and its icon is a different size from a radio control, so the padding that
* lines them up is arithmetic that would otherwise go stale in silence.
*/
/** The meter's icon shares the radio controls' centre line; only a real browser lays this out. */
export const MeterAlignsWithTheDeviceRows: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: [
@@ -581,29 +505,19 @@ export const MeterAlignsWithTheDeviceRows: Story = {
},
};
/** Where an element sits on the inline axis, at its middle. */
/** Inline-axis centre of an element. */
function centre(element: Element): number {
const box = element.getBoundingClientRect();
return box.left + box.width / 2;
}
/**
* Walking the device list with the keyboard, all the way to the last entry.
*
* The level meter stands over the foot of the list, so a row scrolled flush to
* the bottom edge arrives underneath it and can only half be read. Nothing in
* the DOM says an element is covered, so this compares where the two were
* actually drawn.
*/
/** Every row the keyboard reaches is fully visible, not under a heading or the meter. */
export const KeyboardReachesEveryDevice: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
// Enough of them that the list scrolls well past its own height, so that
// arrowing back up has to scroll too — which is where the heading can hide
// a row, as the meter can on the way down.
// Enough to scroll both ways, so either end can hide a row.
options: Array.from({ length: 20 }, (_, i) => ({
label: { type: "name" as const, name: `Microphone ${i + 1}` },
id: `mic${i + 1}`,
@@ -628,8 +542,7 @@ export const KeyboardReachesEveryDevice: Story = {
return element;
});
// Everything that is drawn over the scrolling list: a heading holds the top
// while its section is in view, the meter holds the foot.
// Drawn over the list: the sticky headings and the meter.
const overlays = [
meter,
...document.body.querySelectorAll<HTMLElement>(
@@ -638,32 +551,23 @@ export const KeyboardReachesEveryDevice: Story = {
];
const items = within(document.body).getAllByRole("menuitemradio");
// Down to the last device, as someone reading the list would, and back up
// again: a row can be hidden at either end.
// Down to the last device and back up.
for (const key of ["{ArrowDown}", "{ArrowUp}"])
for (let i = 0; i < items.length; i++) {
await userEvent.keyboard(key);
const focused = document.activeElement as HTMLElement;
await expect(focused).toHaveRole("menuitemradio");
// Nothing is drawn over the row the keyboard has just reached. Stated
// as overlap rather than as an edge, because whether a heading is in
// the way depends on whether its section is still on screen.
// Checked as overlap: a heading only covers rows while its section is on
// screen.
await expect(overlapping(focused, overlays)).toBeLessThanOrEqual(1);
}
},
};
/**
* A long list scrolled well into the microphones.
*
* The heading of the section you are in stays at the top of the list, so it is
* always clear which kind of device the rows below are. It leaves with its own
* section rather than stacking with the next one.
*/
/** A section's heading stays at the top while the section is in view, and leaves with it. */
export const HeadingsStayWhileScrolling: Story = {
args: {
...Default.args,
title: "Microphone",
iconsAndLabels: "audio",
enabled: true,
options: Array.from({ length: 20 }, (_, i) => ({
@@ -687,14 +591,12 @@ export const HeadingsStayWhileScrolling: Story = {
`.${styles.deviceList}`,
)!;
const group = await menu.findByRole("group", { name: "Microphone" });
// By class, not role: the heading is aria-hidden decoration, because the
// group it belongs to is what carries the name.
// By class: the heading is aria-hidden, and its group carries the name.
const heading = group.querySelector<HTMLElement>(
`.${styles.sectionHeading}`,
)!;
// Far enough in that the heading's own place in the list is well above the
// top of the scrollport: it is only still on screen if it is stuck there.
// Far enough that the heading is only on screen if it is stuck there.
list.scrollTop +=
group.getBoundingClientRect().top - list.getBoundingClientRect().top + 80;
@@ -705,7 +607,7 @@ export const HeadingsStayWhileScrolling: Story = {
await expect(heading.getBoundingClientRect().bottom).toBeGreaterThan(
scrollport.top,
);
// And it keeps clear of the menu's frame, as the meter does.
// Clear of the menu's frame.
const frame = document.body
.querySelector("[role='menu']")!
.getBoundingClientRect();
@@ -715,13 +617,7 @@ export const HeadingsStayWhileScrolling: Story = {
},
};
/**
* How far an element is covered, in pixels, by the most overlapping of others.
*
* Nothing in the DOM says an element is obscured, and an element scrolled flush
* to an edge of its container looks no different there from one a sticky
* heading is sitting on top of. The boxes are the only witness.
*/
/** How far the most overlapping of `overlays` covers `element`, in px. */
function overlapping(element: Element, overlays: Element[]): number {
const box = element.getBoundingClientRect();
return overlays.reduce((worst, overlay) => {
@@ -732,13 +628,7 @@ function overlapping(element: Element, overlays: Element[]): number {
}, 0);
}
/**
* The camera menu's blur toggle, which the keyboard reaches after the cameras.
*
* It is a checkbox item and a child of the menu rather than of the device list,
* so a focus ring hung on the list alone left it with the browser's own —
* which follows the pointer, and is what the ring exists to replace.
*/
/** The blur toggle gets the keyboard ring too: it is the menu's child, not the list's. */
export const FocusRingCoversTheBlurToggle: Story = {
args: {
...VideoUnmute.args,
@@ -753,16 +643,14 @@ export const FocusRingCoversTheBlurToggle: Story = {
name: /Blur background/,
});
// Arrowed down past the cameras to the toggle, which is the last thing in
// the menu.
// Arrow down past the cameras to the toggle.
for (let i = 0; i < 6 && document.activeElement !== toggle; i++)
await userEvent.keyboard("{ArrowDown}");
await expect(document.activeElement).toBe(toggle);
// The same ring the device rows get.
await expect(outlineWidth(toggle)).toBeGreaterThan(0);
// And the pointer takes it away again, with the toggle still focused — so
// there is something to light up and it is not lit.
// And the pointer takes it away, with the toggle still focused.
await userEvent.hover(toggle);
await expect(document.activeElement).toBe(toggle);
await expect(outlineWidth(toggle)).toBe(0);
+168 -63
View File
@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, test, vi } from "vitest";
import { afterEach, describe, expect, test, vi } from "vitest";
import { axe } from "vitest-axe";
import {
act,
@@ -15,37 +15,51 @@ import {
type RenderResult,
} from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { type JSX, useState, type ReactNode } from "react";
import { Profiler, type JSX, useState, type ReactNode } from "react";
import { TooltipProvider } from "@vector-im/compound-web";
import { MediaMuteAndSwitchButton } from "./MediaMuteAndSwitchButton";
import {
MediaMuteAndSwitchButton,
type MenuOptions,
} from "./MediaMuteAndSwitchButton";
import { MediaDevicesContext } from "../MediaDevicesContext";
import { type MediaDevices } from "../state/MediaDevices";
import { restoreAudioCapture, stubAudioCapture } from "../utils/test";
interface RenderOptions {
requestDeviceNames: () => void;
}
function renderComponent(
function withProviders(
component: ReactNode,
{ requestDeviceNames = (): void => {} }: Partial<RenderOptions> = {},
): RenderResult {
return render(
): JSX.Element {
return (
<TooltipProvider>
<MediaDevicesContext
value={{ requestDeviceNames } as unknown as MediaDevices}
>
{component}
</MediaDevicesContext>
</TooltipProvider>,
</TooltipProvider>
);
}
function renderComponent(
component: ReactNode,
options: Partial<RenderOptions> = {},
): RenderResult {
return render(withProviders(component, options));
}
describe("MediaMuteAndSwitchButton", () => {
// Only one test stubs the capture; don't let it leak into the rest.
afterEach(restoreAudioCapture);
test("renders", () => {
const { container } = renderComponent(
<TooltipProvider>
<MediaMuteAndSwitchButton title={"Switcher"} iconsAndLabels={"audio"} />
<MediaMuteAndSwitchButton iconsAndLabels={"audio"} />
</TooltipProvider>,
);
expect(container).toMatchSnapshot();
@@ -57,11 +71,7 @@ describe("MediaMuteAndSwitchButton", () => {
enabled: boolean,
): RenderResult => {
return renderComponent(
<MediaMuteAndSwitchButton
title={"Switcher"}
iconsAndLabels={type}
enabled={enabled}
/>,
<MediaMuteAndSwitchButton iconsAndLabels={type} enabled={enabled} />,
);
};
const renderAudioEndabled = renderLabels("audio", true);
@@ -88,7 +98,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onMute = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title={"Switcher"}
onMuteClick={onMute}
iconsAndLabels="audio"
enabled={true}
@@ -105,7 +114,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onMute = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title={"Switcher"}
onMuteClick={onMute}
iconsAndLabels="audio"
enabled={true}
@@ -126,7 +134,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onMute = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title={"Switcher"}
onMuteClick={onMute}
iconsAndLabels="video"
enabled={true}
@@ -146,11 +153,7 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const requestDeviceNames = vi.fn();
renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled
/>,
<MediaMuteAndSwitchButton iconsAndLabels="audio" enabled />,
{ requestDeviceNames },
);
@@ -164,7 +167,6 @@ describe("MediaMuteAndSwitchButton", () => {
renderComponent(
<>
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled
options={[
@@ -174,7 +176,6 @@ describe("MediaMuteAndSwitchButton", () => {
selectedOption="mic1"
/>
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="video"
enabled
options={[
@@ -200,7 +201,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onSelect = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -224,7 +224,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onSelect = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -253,7 +252,6 @@ describe("MediaMuteAndSwitchButton", () => {
const [selectedOption, setSelectedOption] = useState("mic1");
return (
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -313,7 +311,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onVideoBlurToggle = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="video"
enabled={true}
videoBlurToggleClick={onVideoBlurToggle}
@@ -338,7 +335,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -362,7 +358,6 @@ describe("MediaMuteAndSwitchButton", () => {
const [selectedOption, setSelectedOption] = useState("mic1");
return (
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -389,8 +384,7 @@ describe("MediaMuteAndSwitchButton", () => {
screen.getByRole("menuitemradio", { name: "Microphone 2" }),
);
// In flight: nothing else can be picked, in either section, so a second
// request cannot overtake the first.
// In flight: nothing else can be picked, in either section.
for (const name of ["Microphone 1", "Speakers", "Headset"]) {
expect(screen.getByRole("menuitemradio", { name })).toHaveAttribute(
"aria-disabled",
@@ -403,7 +397,7 @@ describe("MediaMuteAndSwitchButton", () => {
await promise;
});
// Settled: choosable again.
// Settled: selectable again.
expect(
screen.getByRole("menuitemradio", { name: "Microphone 1" }),
).not.toHaveAttribute("aria-disabled", "true");
@@ -412,11 +406,144 @@ describe("MediaMuteAndSwitchButton", () => {
).not.toHaveAttribute("aria-disabled", "true");
});
test("lets go of a device switch that never arrives", async () => {
const user = userEvent.setup();
// onSelect never reports back, as when a device is removed mid-switch.
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
iconsAndLabels="audio"
enabled={true}
options={[
{ label: { type: "name", name: "Microphone 1" }, id: "mic1" },
{ label: { type: "name", name: "Microphone 2" }, id: "mic2" },
]}
selectedOption="mic1"
onSelect={vi.fn()}
outputOptions={[
{ label: { type: "name", name: "Speakers" }, id: "spk1" },
{ label: { type: "name", name: "Headset" }, id: "spk2" },
]}
selectedOutputOption="spk1"
onSelectOutput={vi.fn()}
/>,
);
await user.click(getByRole("button", { name: "Microphone" }));
await user.click(
screen.getByRole("menuitemradio", { name: "Microphone 2" }),
);
await user.keyboard("{Escape}");
await user.click(getByRole("button", { name: "Microphone" }));
for (const name of ["Microphone 1", "Speakers", "Headset"]) {
expect(screen.getByRole("menuitemradio", { name })).not.toHaveAttribute(
"aria-disabled",
"true",
);
}
});
test("lets go of a device switch whose device is unplugged", async () => {
const user = userEvent.setup();
const mics: MenuOptions[] = [
{ label: { type: "name", name: "Microphone 1" }, id: "mic1" },
{ label: { type: "name", name: "Microphone 2" }, id: "mic2" },
];
const menu = (options: MenuOptions[]): JSX.Element => (
<MediaMuteAndSwitchButton
iconsAndLabels="audio"
enabled={true}
options={options}
selectedOption="mic1"
onSelect={vi.fn()}
outputOptions={[
{ label: { type: "name", name: "Speakers" }, id: "spk1" },
{ label: { type: "name", name: "Headset" }, id: "spk2" },
]}
selectedOutputOption="spk1"
onSelectOutput={vi.fn()}
/>
);
const { getByRole, rerender } = renderComponent(menu(mics));
await user.click(getByRole("button", { name: "Microphone" }));
await user.click(
screen.getByRole("menuitemradio", { name: "Microphone 2" }),
);
// The second microphone is unplugged before the switch lands.
rerender(withProviders(menu(mics.slice(0, 1))));
// Selectable again without closing the menu.
expect(
screen.getByRole("menuitemradio", { name: "Headset" }),
).not.toHaveAttribute("aria-disabled", "true");
});
test("moves the level without re-rendering anything", async () => {
// The level is drawn into the DOM, so a moving level commits nothing.
const capture = stubAudioCapture();
const user = userEvent.setup();
let commits = 0;
const { getByRole } = renderComponent(
<Profiler
id="menu"
onRender={(): void => {
commits++;
}}
>
<MediaMuteAndSwitchButton
iconsAndLabels="audio"
enabled={true}
options={[
{ label: { type: "name", name: "Microphone 1" }, id: "mic1" },
{ label: { type: "name", name: "Microphone 2" }, id: "mic2" },
]}
selectedOption="mic1"
onSelect={vi.fn()}
outputOptions={[
{ label: { type: "name", name: "Speakers" }, id: "spk1" },
{ label: { type: "name", name: "Headset" }, id: "spk2" },
]}
selectedOutputOption="spk1"
onSelectOutput={vi.fn()}
/>
</Profiler>,
);
await user.click(getByRole("button", { name: "Microphone" }));
capture.grant();
await vi.waitFor(() => expect(capture.contexts).toHaveLength(1));
const meter = await screen.findByRole("meter");
// The capture's arrival is one render: the idle level swapped for its own.
await act(async () => {});
const settled = commits;
// The meter smooths by elapsed time, so hand-driven frames need a clock.
let elapsed = performance.now();
const clock = vi
.spyOn(performance, "now")
.mockImplementation(() => (elapsed += 16));
// One frame per task, as a browser delivers them: one act() would batch them.
for (let step = 1; step <= 8; step++) {
capture.speak(step / 8);
await act(async () => {
capture.drawFrames(1);
await Promise.resolve();
});
}
clock.mockRestore();
expect(meter.getAttribute("aria-valuenow")).not.toBe("0");
expect(commits - settled).toBe(0);
});
test("camera menu uses the same selection pattern and keeps the blur toggle", async () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="video"
enabled={true}
options={[
@@ -431,10 +558,8 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Camera" }));
// Same selection pattern as the microphone menu.
screen.getByRole("menuitemradio", { name: "Camera 1", checked: true });
screen.getByRole("menuitemradio", { name: "Camera 2", checked: false });
// And background blur is still reachable from here.
expect(
screen.getByRole("menuitemcheckbox", { name: "Blur background" }),
).toBeInTheDocument();
@@ -444,7 +569,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -459,27 +583,25 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Microphone" }));
const list = screen
.getByRole("menuitemradio", { name: "Microphone 1" })
.closest("[data-focus-modality]");
.closest("[data-focus-source]");
// The menu focuses whatever the pointer is over, so focus alone says
// nothing about how someone is navigating.
expect(list).toHaveAttribute("data-focus-modality", "pointer");
// The menu focuses whatever the pointer is over, so focus alone says nothing.
expect(list).toHaveAttribute("data-focus-source", "pointer");
await user.keyboard("{ArrowDown}");
expect(list).toHaveAttribute("data-focus-modality", "keyboard");
expect(list).toHaveAttribute("data-focus-source", "keyboard");
await user.pointer({
target: screen.getByRole("menuitemradio", { name: "Microphone 2" }),
coords: { clientX: 10, clientY: 10 },
});
expect(list).toHaveAttribute("data-focus-modality", "pointer");
expect(list).toHaveAttribute("data-focus-source", "pointer");
});
test("marks the selected device with the accent fill", async () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -497,8 +619,7 @@ describe("MediaMuteAndSwitchButton", () => {
.getByRole("menuitemradio", { name: "Microphone 2" })
.querySelector("input[type=radio]");
expect(selected).toBeChecked();
// A read-only control is painted muted, which loses the accent fill that
// marks the selection and makes the menu differ from settings.
// readOnly would paint the selected radio muted.
expect(selected).not.toHaveAttribute("readonly");
});
@@ -506,7 +627,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -526,8 +646,6 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Microphone" }));
// Includes the menu's own structure: wrappers between the menu and its
// items break the relationship the roles describe.
const menu = document.querySelector('[role="menu"]');
expect(await axe(menu as HTMLElement)).toHaveNoViolations();
});
@@ -536,7 +654,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -570,7 +687,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -607,7 +723,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onSelectOutput = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -635,7 +750,6 @@ describe("MediaMuteAndSwitchButton", () => {
const onSelect = vi.fn();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -648,7 +762,7 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Microphone" }));
// Shown rather than hidden, so the menu keeps its shape, but not choosable.
// Shown, but not selectable.
const only = screen.getByRole("menuitemradio", { name: "Microphone 1" });
expect(only).toHaveAttribute("aria-disabled", "true");
});
@@ -657,7 +771,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -666,8 +779,7 @@ describe("MediaMuteAndSwitchButton", () => {
]}
selectedOption="mic1"
onSelect={vi.fn()}
// Safari enumerates no output devices at all, and offers no way to
// choose one.
// As Safari: no outputs listed.
outputOptions={[]}
selectedOutputOption={undefined}
onSelectOutput={undefined}
@@ -676,8 +788,6 @@ describe("MediaMuteAndSwitchButton", () => {
await user.click(getByRole("button", { name: "Microphone" }));
// A heading with nothing under it says the feature is broken. Audio is
// playing somewhere, so the section names that somewhere and disables it.
const speakers = screen
.getAllByRole("group")
.find((group) => group.getAttribute("aria-label") === "Speaker")!;
@@ -685,8 +795,6 @@ describe("MediaMuteAndSwitchButton", () => {
expect(entries).toHaveLength(1);
expect(entries[0]).toHaveAccessibleName("Default");
expect(entries[0]).toHaveAttribute("aria-disabled", "true");
// And marked as the selection: it is where audio is going, so an unchecked
// lone entry would read as nothing being chosen at all.
expect(entries[0]).toHaveAttribute("aria-checked", "true");
expect(
within(entries[0]).getByRole("radio", { hidden: true }),
@@ -697,7 +805,6 @@ describe("MediaMuteAndSwitchButton", () => {
const user = userEvent.setup();
const { getByRole } = renderComponent(
<MediaMuteAndSwitchButton
title="Switcher"
iconsAndLabels="audio"
enabled={true}
options={[
@@ -711,7 +818,6 @@ describe("MediaMuteAndSwitchButton", () => {
{ label: { type: "name", name: "Headset" }, id: "spk2" },
]}
selectedOutputOption="spk1"
// No callback: nothing can be picked here.
onSelectOutput={undefined}
/>,
);
@@ -724,7 +830,6 @@ describe("MediaMuteAndSwitchButton", () => {
expect(
screen.getByRole("menuitemradio", { name: "Headset" }),
).toHaveAttribute("aria-disabled", "true");
// The microphone section is unaffected.
expect(
screen.getByRole("menuitemradio", { name: "Microphone 2" }),
).not.toHaveAttribute("aria-disabled", "true");
+79 -161
View File
@@ -39,8 +39,7 @@ import {
import { useMediaDevices } from "../MediaDevicesContext";
import { useRootElement } from "../RootElementContext";
import { observeElementSize$ } from "../utils/elementSize";
import { MicrophoneLevelMeter } from "./MicrophoneLevelMeter";
import { useMicrophoneLevel } from "./useMicrophoneLevel";
import { LiveMicrophoneLevelMeter } from "./MicrophoneLevelMeter";
export interface MenuOptions {
label: DeviceLabel | AudioOutputDeviceLabel;
@@ -48,11 +47,6 @@ export interface MenuOptions {
}
export interface MediaMuteAndSwitchButtonProps {
/**
* The accessible name of the menu. Defaults to a translated name for the
* media kind. Never shown: each section carries its own heading.
*/
title?: string;
/** If the Mute button is enabled */
enabled?: boolean;
/** Callback if the mute button is clicked */
@@ -64,17 +58,11 @@ export interface MediaMuteAndSwitchButtonProps {
options?: MenuOptions[];
/** The option that will currently be rendered as the selected option */
selectedOption?: string;
/**
* Output (speaker) devices, shown as their own section above the input
* section. Audio menu only; omitted entirely for video.
*/
/** Output (speaker) devices. Audio menu only. */
outputOptions?: MenuOptions[];
/** The output option currently rendered as selected */
selectedOutputOption?: string;
/**
* Called when an output device is picked. Undefined means no output can be
* chosen here, and the section renders disabled.
*/
/** Picks an output device. Undefined disables the speaker section. */
onSelectOutput?: (id: string) => void;
videoBlurToggleClick?: () => void;
videoBlurEnabled?: boolean;
@@ -87,33 +75,16 @@ export interface MediaMuteAndSwitchButtonProps {
const BLUR_ID = "blur";
/**
* Stands for wherever the platform is sending audio, where it will not say.
*
* Not a device id the browser would recognise: nothing can be selected on a
* platform that lists no outputs, so this is only ever shown, never sent.
*/
/** Id of the placeholder "Default" row, shown when the platform lists no outputs. */
const DEFAULT_OUTPUT_ID = "default";
/**
* The share of the call area the device list may fill.
*
* The menu carries its headings and the level meter as well, and a list that
* took the whole call would hide the call it belongs to.
*/
/** Largest share of the call area's height the device list may take. */
const LIST_SHARE_OF_CALL = 0.6;
/**
* The shortest the device list may be, whatever the call measures.
*
* A share alone collapses in a small call to a list that shows one device and
* gives no sign that there are others. Scrolling a short list is the better
* failure.
*/
/** Smallest device list height in px, so a short call still shows more than one device. */
const MIN_LIST_HEIGHT = 160;
export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
title,
enabled,
busy,
onMuteClick,
@@ -127,49 +98,34 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
videoBlurToggleClick,
onSelect,
}) => {
// Which device we have asked for but not yet been given. Carries the kind as
// well as the id, because an input and an output can share an id: "default"
// names both on Chrome.
// Requested but not yet selected. Keyed by kind too, since Chrome uses
// "default" for both an input and an output.
const [plannedSelection, setPlannedSelection] = useState<{
kind: "input" | "output";
id: string;
} | null>(null);
const [menuOpen, setMenuOpen] = useState(false);
const onOpenChange = useCallback((open: boolean): void => {
setMenuOpen(open);
// Drop a request that never arrived.
if (!open) setPlannedSelection(null);
}, []);
const isBusy = busy ?? false;
const { t } = useTranslation();
const devices = useMediaDevices();
// Radix focuses whatever the pointer is over, so the browser's own
// :focus-visible cannot tell us whether a person is navigating by keyboard:
// Chromium answers yes to everything after any key press, Firefox answers no
// to programmatic focus. Track it ourselves and let the styling follow.
/**
* Tracks which modality moved the focus, for as long as the list is mounted.
*
* A ref rather than an effect on `menuOpen`: that state is ours, the open
* menu is Radix's, and the two do not commit together — an effect keyed on
* ours can run before Radix has mounted the content, with nothing to attach
* to. The list existing is the honest signal that the menu is open.
*
* Recorded on the menu rather than held in state, because every item the menu
* can focus has to answer to it — the device rows and the camera menu's blur
* toggle, which is the menu's child and not the list's — and because which
* modality someone is using changes nothing that has to be rendered again.
* Records on the menu whether the keyboard or the pointer moved focus, for
* the focus ring. `:focus-visible` can't tell: Radix focuses whatever the
* pointer is over. Listened for on the menu, not the document, so a second
* Element Call on the page doesn't answer for this one.
*/
const trackFocusModality = useCallback(
const trackFocusSource = useCallback(
(list: HTMLDivElement | null): (() => void) | undefined => {
// Watched on the menu, not on the document. Element Call can be mounted
// more than once in a host's page, and the menu is portalled out of the
// call root, so a document listener would also answer for a key pressed
// in the other instance, or in the host's own page. The menu rather than
// the list, because the first arrow key arrives while the menu itself
// holds focus, above anything we render.
const menu = list?.closest<HTMLElement>('[role="menu"]');
if (menu === null || menu === undefined) return;
// Each opening starts over: the modality belongs to whoever is using this
// menu now, not to whoever last used it.
const record = (modality: "keyboard" | "pointer"): void => {
menu.dataset.focusModality = modality;
const record = (source: "keyboard" | "pointer"): void => {
menu.dataset.focusSource = source;
};
record("pointer");
const usedKeyboard = (): void => record("keyboard");
@@ -184,18 +140,12 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
[],
);
// The menu is portalled outside the call root, so nothing in the stylesheets
// can size it against the call. Measure the call area rather than the window,
// or the menu is wrong wherever Element Call is not the whole page.
// Measured on the call area: CSS can't size the portalled menu against it.
const rootElement = useRootElement();
const [listMaxHeight, setListMaxHeight] = useState<number>();
useEffect(() => {
if (!menuOpen) return;
// Followed rather than measured once: a host can resize the space Element
// Call is drawn in while the menu is open — a panel animating, a window
// dragged, a phone turned — and a bound taken on opening then describes a
// call area that no longer exists. Quantised before it reaches React, so a
// resize re-renders only when the bound itself moves.
// Followed, since a host can resize the call while the menu is open.
const subscription = observeElementSize$(rootElement)
.pipe(
map(({ height }) =>
@@ -207,56 +157,17 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
return (): void => subscription.unsubscribe();
}, [menuOpen, rootElement]);
// Only while the menu is open, so nothing holds a second capture of the
// microphone for the length of a call.
// The meter sits over the foot of the scrolling list, so the list has to
// keep that much of itself clear: a row scrolled to by the keyboard would
// otherwise arrive underneath it, half-read. Its own height, measured,
// because the failure states are two lines where a level is one.
const [meterHeight, setMeterHeight] = useState<number>();
const meter = useCallback(
(element: HTMLDivElement | null): (() => void) | undefined => {
if (element === null) return;
const subscription = observeElementSize$(element)
.pipe(
map(({ height }) => height),
distinctUntilChanged(),
)
.subscribe(setMeterHeight);
return (): void => subscription.unsubscribe();
},
[],
);
// Kept clear at the list's foot, so a row reached by keyboard isn't under
// the meter.
const [meterHeight, meter] = useMeasuredHeight();
// The headings stand over the head of the list, so it has to keep their
// height clear too — the same bargain as the meter, at the other end. One
// measurement serves both: the sections are headed alike.
const [headingHeight, setHeadingHeight] = useState<number>();
const heading = useCallback(
(element: HTMLDivElement | null): (() => void) | undefined => {
if (element === null) return;
const subscription = observeElementSize$(element)
.pipe(
map(({ height }) => height),
distinctUntilChanged(),
)
.subscribe(setHeadingHeight);
return (): void => subscription.unsubscribe();
},
[],
);
const microphoneState = useMicrophoneLevel(
selectedOption,
menuOpen && iconsAndLabels === "audio",
);
// Likewise at its head, for the sticky headings.
const [headingHeight, heading] = useMeasuredHeight();
useEffect(() => {
if (menuOpen) devices.requestDeviceNames(); // No-op after the first call
}, [menuOpen, devices]);
// The mute control differs between the two only in which button it is and
// what it is called; how it behaves is the same, and was worth saying once.
const MuteButton = iconsAndLabels === "audio" ? MicButton : VideoButton;
const button = (
<MuteButton
@@ -274,7 +185,6 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
/>
);
// Only the camera menu carries a toggle, and only when the caller offers one.
const toggles =
iconsAndLabels === "video" && videoBlurToggleClick !== undefined
? [
@@ -287,24 +197,23 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
: [];
let optionsButtonLabel: string;
let defaultMenuTitle: string;
let menuTitle: string;
let numberedLabel: (number: number) => string;
switch (iconsAndLabels) {
case "video":
optionsButtonLabel = t("settings.devices.camera");
defaultMenuTitle = t("settings.devices.camera_source");
menuTitle = t("settings.devices.camera_source");
numberedLabel = (n): string =>
t("settings.devices.camera_numbered", { n });
break;
case "audio":
optionsButtonLabel = t("settings.devices.microphone");
defaultMenuTitle = t("settings.devices.mic_source");
menuTitle = t("settings.devices.mic_source");
numberedLabel = (n): string =>
t("settings.devices.microphone_numbered", { n });
break;
}
/** The text shown for a device, whichever kind of label it carries. */
const labelText = (
label: MenuOptions["label"],
numbered: (n: number) => string,
@@ -325,26 +234,24 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
}
};
// A device we asked for that has not arrived yet. Until it does, nothing in
// the menu can be picked, so a second request cannot overtake the first.
// Nothing can be picked while a requested device is on offer but not yet
// selected, so a second request can't overtake it.
const plannedOutput = plannedSelection?.kind === "output";
const selectedOfPlannedKind = plannedOutput
? selectedOutputOption
: selectedOption;
const offeredOfPlannedKind = plannedOutput ? outputOptions : options;
const settling =
plannedSelection !== null &&
plannedSelection.id !==
(plannedSelection.kind === "output"
? selectedOutputOption
: selectedOption);
plannedSelection.id !== selectedOfPlannedKind &&
offeredOfPlannedKind?.some(({ id }) => id === plannedSelection.id) === true;
// Safari enumerates no output devices at all, and offers no way to choose
// one, so the list arrives empty. The section is shown all the same — audio
// is playing somewhere — naming that somewhere and disabling it like any
// single entry. A heading with nothing beneath it reads as a broken feature,
// and leaves the menu a different shape on one browser.
// Safari lists no outputs: show a disabled, selected Default rather than an
// empty section.
const noOutputsListed = outputOptions?.length === 0;
const speakerOptions: MenuOptions[] | undefined = noOutputsListed
? [{ id: DEFAULT_OUTPUT_ID, label: { type: "default", name: null } }]
: outputOptions;
// And it is the selection, not merely the only row: it is where audio is
// going. An unchecked lone entry reads as nothing being chosen at all.
const selectedSpeaker = noOutputsListed
? DEFAULT_OUTPUT_ID
: selectedOutputOption;
@@ -357,30 +264,23 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
numbered: (n: number) => string,
): ReactElement[] => {
const list = items ?? [];
// Shown but not choosable when nothing can be picked here, or when there is
// only one device. The entry stays visible so the menu keeps the same shape
// on every platform.
// Disabled rather than hidden, so the menu keeps its shape.
const disabled = select === undefined || list.length <= 1 || settling;
return list.map(({ label, id }) => (
<MenuItem
// A radio input inside a button is invalid, and the menu needs an
// element it can give menuitemradio semantics to.
// A radio input may not sit inside a button.
as="div"
hideChevron
disabled={disabled}
label={labelText(label, numbered)}
Icon={
// Inert, not aria-hidden: a form control inside a menu item must be
// out of the focus order and out of the accessibility tree, and
// aria-hidden alone leaves it focusable. The item's aria-checked is
// what conveys the selection.
// Inert, not aria-hidden: aria-hidden alone leaves it focusable.
<span inert>
<RadioInput
checked={selected === id}
disabled={disabled}
// Not readOnly: that styles the control as muted, losing the
// accent fill that marks the selection. The menu item owns the
// interaction, so the change handler has nothing to do.
// Not readOnly, which mutes the selected fill. The item handles
// the click.
onChange={(): void => {}}
/>
</span>
@@ -420,12 +320,11 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
{button}
<Menu
className={styles.menu}
title={title ?? defaultMenuTitle}
// Each section carries its own heading, so the menu's own title would
// sit on top of the first one. Kept for the accessible name only.
// Named for screen readers only: each section has its own heading.
title={menuTitle}
showTitle={false}
open={menuOpen}
onOpenChange={setMenuOpen}
onOpenChange={onOpenChange}
side="top"
trigger={
<Button
@@ -442,9 +341,8 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
}
>
<div
ref={trackFocusModality}
// Transparent to assistive technology, so the menu still sees its
// items as its own children.
ref={trackFocusSource}
// Keeps the items the menu's own children for assistive tech.
role="none"
className={styles.deviceList}
style={
@@ -461,11 +359,8 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
{iconsAndLabels === "audio" && speakerOptions && (
<>
{/* A menu may only contain items, separators and groups, so each
heading belongs to a group rather than sitting beside the
items it names. */}
heading is a hidden part of a named group. */}
<div role="group" aria-label={t("settings.devices.speaker")}>
{/* The heading is decoration: the group carries the name, and
a menu may only contain items, separators and groups. */}
<div aria-hidden className={styles.sectionHeading}>
<MenuTitle title={t("settings.devices.speaker")} />
</div>
@@ -483,8 +378,7 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
<div ref={heading} aria-hidden className={styles.sectionHeading}>
<MenuTitle title={optionsButtonLabel} />
</div>
{/* The heading sits outside, so the meter can never ride up over it:
sticky only holds while this block is in view. */}
{/* Apart from the heading, so the sticky meter can't ride over it. */}
<div role="none">
{deviceItems(
"input",
@@ -494,9 +388,11 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
numberedLabel,
)}
{iconsAndLabels === "audio" && (
<MicrophoneLevelMeter
<LiveMicrophoneLevelMeter
ref={meter}
state={microphoneState}
deviceId={selectedOption}
// Only while open, so the microphone isn't held all call.
active={menuOpen}
className={styles.stickyMeter}
/>
)}
@@ -519,3 +415,25 @@ export const MediaMuteAndSwitchButton: FC<MediaMuteAndSwitchButtonProps> = ({
</div>
);
};
/** Follows an element's height. */
function useMeasuredHeight(): [
number | undefined,
(element: HTMLElement | null) => (() => void) | undefined,
] {
const [height, setHeight] = useState<number>();
const ref = useCallback(
(element: HTMLElement | null): (() => void) | undefined => {
if (element === null) return;
const subscription = observeElementSize$(element)
.pipe(
map((size) => size.height),
distinctUntilChanged(),
)
.subscribe(setHeight);
return (): void => subscription.unsubscribe();
},
[],
);
return [height, ref];
}
+18 -33
View File
@@ -6,20 +6,21 @@ Please see LICENSE in the repository root for full details.
*/
.meter {
/* A device row's icon column, so the bars start where the names do. */
--meter-icon-size: 24px;
/* Compound's radio size, whose centre line the icon shares. */
--device-control-size: 20px;
display: flex;
align-items: center;
gap: var(--cpd-space-3x);
padding-block: var(--cpd-space-2x);
/* The icon shares a centre with the radio controls in the device rows above,
which sit one 4x padding in from the menu's edge. Two things pull this row
out of line with them, and both come off the padding: the row is inset by a
border width so that the menu's frame stays visible behind it (see
`.stickyMeter`), and the icon is wider than a radio control, so it has to
start further left for the two to share a centre. Both are
measured in the MeterAlignsWithTheDeviceRows story rather than trusted:
jsdom lays nothing out, so only a real browser can hold this. */
/* Centres the icon on the radio column: less the row's inset border and half
the icon's extra width. Checked in the MeterAlignsWithTheDeviceRows story. */
padding-inline-start: calc(
var(--cpd-space-4x) - var(--cpd-border-width-1) - 2px
var(--cpd-space-4x) - var(--cpd-border-width-1) -
(var(--meter-icon-size) - var(--device-control-size)) / 2
);
padding-inline-end: calc(var(--cpd-space-4x) * 2 + var(--cpd-space-2x));
}
@@ -27,33 +28,20 @@ Please see LICENSE in the repository root for full details.
.icon {
color: var(--cpd-color-icon-secondary);
flex-shrink: 0;
/* Same width as a device row's icon column, so the bars start where the
device names start. */
inline-size: 24px;
inline-size: var(--meter-icon-size);
block-size: var(--meter-icon-size);
}
.segments {
/* A bar and the space beside it are always the same size; what changes with
the width available is how many bars there are. Spreading a fixed number of
bars instead would make the meter a different shape in every place it is
used, and close them up into one solid block wherever the space ran short —
and a meter whose bars touch stops reading as a count, which is what
carries the level for anyone who cannot rely on the colour.
Both sizes are set here and nowhere else: the component reads them back off
the rendered bars to work out how many fit. */
/* Bars keep one size and the width decides how many; the component reads the
sizes back from here. */
flex: 1;
/* The width decides the count, never the other way round. Containment is
what enforces that: without it the bars are both a floor under the width
and the widest thing in the menu, so the meter would set the menu's width
and the device names — which are what a person is reading — would have to
fit around it. */
/* So the bars never set the menu's width. */
contain: inline-size;
min-inline-size: 0;
display: flex;
align-items: center;
/* Half again as wide as a bar, as the design has it: bars set as close as
their own width read as a solid block long before they touch. */
/* Half again a bar's width, per the design. */
gap: var(--cpd-space-1-5x);
}
@@ -65,16 +53,13 @@ Please see LICENSE in the repository root for full details.
background: var(--cpd-color-bg-subtle-primary);
}
/* Filled segments carry the level by count as well as by colour, so the meter
stays readable without relying on hue. */
/* The count of lit bars carries the level as well as their colour. */
.segmentLit {
background: var(--cpd-color-bg-accent-rest);
}
.message {
color: var(--cpd-color-text-secondary);
/* A paragraph brings a margin below it, which in a centred row does not push
the text down but lifts it: the margin box is what gets centred, so the
words end up above the middle and the icon beside them looks low. */
/* A paragraph's margin would lift the text off centre. */
margin-block: 0;
}
+26 -54
View File
@@ -14,17 +14,10 @@ import {
type MicrophoneLevelMeterProps,
} from "./MicrophoneLevelMeter";
import styles from "./MicrophoneLevelMeter.module.css";
import { METER_SEGMENTS } from "../state/MicrophoneLevel";
import { LEVEL_SCALE } from "../state/MicrophoneLevel";
import { constant } from "../state/Behavior";
/**
* A width to show the meter at, close to the menu it lives in.
*
* Not a copy of the menu's width, and nothing depends on the two agreeing: a
* bar and the gap beside it are a fixed size now, so this only decides how many
* bars there is room for. Without a width at all the stories would shrink-wrap
* to almost nothing and show a meter two bars wide, which is no use to anyone
* looking at them.
*/
/** Roughly the menu's width. It only decides how many bars fit. */
const STORY_WIDTH = 256;
const meta = {
@@ -47,29 +40,22 @@ const meta = {
export default meta;
type Story = StoryObj<typeof meta>;
/**
* A quiet room. Nothing is lit: room hiss below the noise floor must not read
* as "it can hear me".
*/
/** A quiet room: hiss below the noise floor lights nothing. */
export const Silent: Story = {
args: { state: { type: "level", level: 0 } },
args: { state: { type: "level", level$: constant(0) } },
play: async ({ canvasElement }) => {
await expect(litSegments(canvasElement)).toBe(0);
},
};
export const QuietSpeech: Story = {
args: { state: { type: "level", level: 5 } },
args: { state: { type: "level", level$: constant(5) } },
};
export const NormalSpeech: Story = {
args: { state: { type: "level", level: 12 } },
args: { state: { type: "level", level$: constant(12) } },
play: async ({ canvasElement }) => {
// Shown at something like the width of the menu, so the meter in a story
// reads like the meter in a call rather than like a handful of bars. The
// design's own mock has sixteen of them at this width; a floor rather than
// a count, because the number follows from the bar and gap sizes and those
// are the design's to change.
// A floor, not a count: the count follows from the design's bar and gap sizes.
await expect(
canvasElement.getElementsByClassName(styles.segment).length,
).toBeGreaterThanOrEqual(15);
@@ -77,48 +63,44 @@ export const NormalSpeech: Story = {
};
export const LoudSpeech: Story = {
args: { state: { type: "level", level: METER_SEGMENTS } },
args: { state: { type: "level", level$: constant(LEVEL_SCALE) } },
};
/**
* The three volumes differ by how many bars are lit, so the level survives
* greyscale and a screen reader as well as it survives colour.
*/
/** The three volumes differ in how many bars are lit, not only in colour. */
export const VolumesAreDistinguishable: Story = {
args: { state: { type: "level", level: 5 } },
args: { state: { type: "level", level$: constant(5) } },
play: async ({ canvasElement, mount }) => {
const lit: number[] = [];
for (const level of [5, 12, METER_SEGMENTS]) {
await mount(<MicrophoneLevelMeter state={{ type: "level", level }} />);
for (const level of [5, 12, LEVEL_SCALE]) {
await mount(
<MicrophoneLevelMeter
state={{ type: "level", level$: constant(level) }}
/>,
);
lit.push(litSegments(canvasElement));
await expect(within(canvasElement).getByRole("meter")).toHaveAttribute(
"aria-valuenow",
String(level),
);
}
// Three different counts, rising: the level is carried by how many bars
// are lit, not by their colour alone.
await expect(new Set(lit).size).toBe(lit.length);
await expect(lit).toEqual([...lit].sort((a, b) => a - b));
},
};
/**
* Permission refused. A message with a next action, never a still meter that
* reads as silence.
*/
/** Permission refused: a message with a next action, not a still meter. */
export const PermissionDenied: Story = {
args: { state: { type: "permission-denied" } },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(canvas.queryByRole("meter")).toBeNull();
await expect(
canvas.getByText(/Allow access in your browser settings/),
canvas.getByText(/Microphone access is blocked/),
).toBeVisible();
},
};
/** No input device at all, told apart from a refusal. */
/** No input device, told apart from a refusal. */
export const NoDevice: Story = {
args: { state: { type: "no-device" } },
play: async ({ canvasElement }) => {
@@ -126,9 +108,7 @@ export const NoDevice: Story = {
await expect(canvas.queryByRole("meter")).toBeNull();
await expect(canvas.getByText(/No microphone found/)).toBeVisible();
// The icon sits on the middle of the words, however many lines they run to.
// A paragraph's own margin would centre its margin box instead, leaving the
// text high and the icon looking low beside it.
// The icon sits on the middle of the text, however many lines it runs to.
const middle = (element: Element): number => {
const box = element.getBoundingClientRect();
return box.top + box.height / 2;
@@ -139,17 +119,9 @@ export const NoDevice: Story = {
},
};
/**
* The same meter at two widths.
*
* A bar and the space beside it are always the same size; what changes is how
* many bars there are. Spreading a fixed number of bars instead would make the
* meter a different shape in every place it is used, and close the bars up into
* one block wherever the space ran short — and bars that touch cannot be
* counted, which is what carries the level without colour.
*/
/** The same meter at two widths: the bars keep their size and only their count changes. */
export const ShapeStaysTheSameAtAnyWidth: Story = {
args: { state: { type: "level", level: 12 } },
args: { state: { type: "level", level$: constant(12) } },
play: async ({ mount, args }) => {
const narrow = await measureAt(mount, args, 180);
const wide = await measureAt(mount, args, 400);
@@ -157,17 +129,17 @@ export const ShapeStaysTheSameAtAnyWidth: Story = {
await expect(narrow.bar).toBe(wide.bar);
await expect(narrow.gap).toBe(wide.gap);
await expect(narrow.count).toBeLessThan(wide.count);
// And the bars are still bars, not one run of colour.
// The bars are still separate, not one run of colour.
await expect(narrow.gap).toBeGreaterThan(0);
},
};
/** How many bars are painted as carrying level, rather than as empty. */
/** How many bars are lit. */
function litSegments(canvasElement: HTMLElement): number {
return canvasElement.getElementsByClassName(styles.segmentLit).length;
}
/** Renders the meter at one width and reports the shape of its bars. */
/** Renders the meter at one width and reports its bars' shape. */
async function measureAt(
mount: (ui: JSX.Element) => Promise<unknown>,
args: MicrophoneLevelMeterProps,
+8 -6
View File
@@ -9,25 +9,27 @@ import { describe, expect, test } from "vitest";
import { render, screen } from "@testing-library/react";
import { MicrophoneLevelMeter } from "./MicrophoneLevelMeter";
import { METER_SEGMENTS } from "../state/MicrophoneLevel";
import { LEVEL_SCALE } from "../state/MicrophoneLevel";
import { constant } from "../state/Behavior";
describe("MicrophoneLevelMeter", () => {
test("announces the level rather than relying on hue", () => {
render(<MicrophoneLevelMeter state={{ type: "level", level: 6 }} />);
render(
<MicrophoneLevelMeter state={{ type: "level", level$: constant(6) }} />,
);
const meter = screen.getByRole("meter", { name: "Microphone level" });
expect(meter).toHaveAttribute("aria-valuenow", "6");
expect(meter).toHaveAttribute("aria-valuemax", String(METER_SEGMENTS));
expect(meter).toHaveAttribute("aria-valuetext", `6 of ${METER_SEGMENTS}`);
expect(meter).toHaveAttribute("aria-valuemax", String(LEVEL_SCALE));
expect(meter).toHaveAttribute("aria-valuetext", `6 of ${LEVEL_SCALE}`);
});
test("shows distinct messages for denied permission and no input device", () => {
const denied = render(
<MicrophoneLevelMeter state={{ type: "permission-denied" }} />,
);
// A next action, not a flat meter that reads as silence.
expect(
denied.getByText(/Allow access in your browser settings/),
denied.getByText(/Microphone access is blocked/),
).toBeInTheDocument();
expect(denied.queryByRole("meter")).toBeNull();
+90 -64
View File
@@ -5,7 +5,14 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { useCallback, useState, type FC, type Ref } from "react";
import {
useCallback,
useLayoutEffect,
useRef,
useState,
type FC,
type Ref,
} from "react";
import { Text } from "@vector-im/compound-web";
import { MicOnIcon } from "@vector-im/compound-design-tokens/assets/web/icons";
import classNames from "classnames";
@@ -14,107 +21,126 @@ import { useTranslation } from "react-i18next";
import { distinctUntilChanged, map } from "rxjs";
import styles from "./MicrophoneLevelMeter.module.css";
import { METER_SEGMENTS, type MicrophoneState } from "../state/MicrophoneLevel";
import { LEVEL_SCALE, type MicrophoneState } from "../state/MicrophoneLevel";
import { observeElementSize$ } from "../utils/elementSize";
import { useMicrophoneLevel } from "./useMicrophoneLevel";
export interface MicrophoneLevelMeterProps {
state: MicrophoneState;
export interface LiveMicrophoneLevelMeterProps {
/** The microphone to listen to. */
deviceId: string | undefined;
/** Whether to hold a capture at all. */
active: boolean;
className?: string;
/**
* The meter's own element. Its height is what a scroll container has to keep
* clear to stop the meter covering the row it has just scrolled to.
*/
ref?: Ref<HTMLDivElement>;
}
/**
* The live input level of the selected microphone, shown beneath it.
*
* It says whether the microphone is picking anything up, which is not the same
* as whether the user is being heard: it keeps moving while muted, and the mute
* control is what says nothing is transmitted.
* The meter, wired to a live capture. Its own component so a changing level
* re-renders the meter, not its parent.
*/
export const LiveMicrophoneLevelMeter: FC<LiveMicrophoneLevelMeterProps> = ({
deviceId,
active,
className,
ref,
}) => {
const state = useMicrophoneLevel(deviceId, active);
return <MicrophoneLevelMeter state={state} className={className} ref={ref} />;
};
export interface MicrophoneLevelMeterProps {
state: MicrophoneState;
className?: string;
ref?: Ref<HTMLDivElement>;
}
/** The input level of a microphone, or why there is none. */
export const MicrophoneLevelMeter: FC<MicrophoneLevelMeterProps> = ({
state,
className,
ref,
}) => {
const { t } = useTranslation();
// How many bars there is room for. The bars never change size, so this is
// what absorbs a change of width. Starts at the full count so that the first
// paint is a meter rather than a single bar, and so that a renderer with no
// layout at all — jsdom — still draws the whole thing.
const [barCount, setBarCount] = useState(METER_SEGMENTS);
// Bars keep one size, so their count follows the width. Starts full so the
// first paint, and jsdom, draw a whole meter.
const [barCount, setBarCount] = useState(LEVEL_SCALE);
const segments = useRef<HTMLDivElement | null>(null);
const track = useCallback(
(element: HTMLDivElement | null): (() => void) | undefined => {
if (element === null) return;
segments.current = element;
const subscription = observeElementSize$(element)
.pipe(
map(({ width }) => barsThatFit(element, width)),
distinctUntilChanged(),
)
.subscribe(setBarCount);
return (): void => subscription.unsubscribe();
return (): void => {
subscription.unsubscribe();
segments.current = null;
};
},
[],
);
if (state.type !== "level")
return (
<div ref={ref} className={classNames(styles.meter, className)}>
<MicOnIcon width={24} height={24} className={styles.icon} aria-hidden />
// Drawn straight into the DOM: the level changes many times a second, and
// re-rendering for each change is what this avoids.
useLayoutEffect(() => {
const element = segments.current;
if (state.type !== "level" || element === null) return;
const subscription = state.level$.subscribe((level) => {
element.setAttribute("aria-valuenow", String(level));
element.setAttribute(
"aria-valuetext",
t("microphone_level.value", { level, max: LEVEL_SCALE }),
);
// The level is a share of the scale, not a bar count.
const lit = Math.round((level / LEVEL_SCALE) * barCount);
Array.from(element.children).forEach((bar, i) =>
bar.classList.toggle(styles.segmentLit, i < lit),
);
});
return (): void => subscription.unsubscribe();
}, [state, barCount, t]);
return (
<div ref={ref} className={classNames(styles.meter, className)}>
<MicOnIcon className={styles.icon} aria-hidden />
{state.type !== "level" ? (
<Text size="sm" className={styles.message}>
{state.type === "permission-denied"
? t("microphone_level.permission_denied")
: t("microphone_level.no_device")}
</Text>
</div>
);
return (
<div ref={ref} className={classNames(styles.meter, className)}>
<MicOnIcon width={24} height={24} className={styles.icon} aria-hidden />
<div
ref={track}
className={styles.segments}
role="meter"
aria-label={t("microphone_level.label")}
aria-valuemin={0}
aria-valuemax={METER_SEGMENTS}
aria-valuenow={state.level}
aria-valuetext={t("microphone_level.value", {
level: state.level,
max: METER_SEGMENTS,
})}
>
{Array.from({ length: barCount }, (_, i) => (
<span
key={i}
aria-hidden
className={classNames(styles.segment, {
// The level is a share of the scale, not a number of bars: how
// many bars stand for it depends on how many there are.
[styles.segmentLit]:
i < Math.round((state.level / METER_SEGMENTS) * barCount),
})}
/>
))}
</div>
) : (
<div
ref={track}
className={styles.segments}
role="meter"
aria-label={t("microphone_level.label")}
aria-valuemin={0}
aria-valuemax={LEVEL_SCALE}
// The first paint's value; the effect above keeps it current.
aria-valuenow={state.level$.value}
aria-valuetext={t("microphone_level.value", {
level: state.level$.value,
max: LEVEL_SCALE,
})}
>
{Array.from({ length: barCount }, (_, i) => (
<span key={i} aria-hidden className={styles.segment} />
))}
</div>
)}
</div>
);
};
/**
* How many bars fit across `width`, at the size the stylesheet draws them.
*
* Measured off a rendered bar rather than told: the size of a bar and of the
* space beside it are a design question, settled in the stylesheet, and reading
* them back is what keeps them from being settled twice.
*/
/** How many bars fit across `width`, measured so the stylesheet alone sets their size. */
function barsThatFit(track: HTMLElement, width: number): number {
const gap = Number.parseFloat(getComputedStyle(track).columnGap);
const bar = track.firstElementChild?.getBoundingClientRect().width ?? 0;
// A renderer that lays nothing out tells us nothing; keep the full count.
if (!(bar > 0) || !(gap >= 0)) return METER_SEGMENTS;
// No layout (jsdom): keep the full count.
if (!(bar > 0) || !(gap >= 0)) return LEVEL_SCALE;
return Math.max(1, Math.floor((width + gap) / (bar + gap)));
}
+4 -5
View File
@@ -11,9 +11,7 @@ import { renderHook } from "@testing-library/react";
import { useMicrophoneLevel } from "./useMicrophoneLevel";
import { restoreAudioCapture, stubAudioCapture } from "../utils/test";
// The capture itself, and releasing it, belong to observeMicrophoneState$ and
// are covered in src/state/MicrophoneLevel.test.ts. What is left here is the
// bridging: when the hook watches, and what it reports before it has an answer.
// Capture and release are covered in MicrophoneLevel.test.ts; this covers the bridge.
describe("useMicrophoneLevel", () => {
afterEach(restoreAudioCapture);
@@ -36,7 +34,8 @@ describe("useMicrophoneLevel", () => {
rerender({ id: "mic2" });
// No level carried over: the meter reads the new device or nothing at all.
expect(result.current).toEqual({ type: "level", level: 0 });
// No level carried over from the previous device.
const state = result.current;
expect(state.type === "level" ? state.level$.value : state.type).toBe(0);
});
});
+4 -11
View File
@@ -11,17 +11,11 @@ import {
type MicrophoneState,
observeMicrophoneState$,
} from "../state/MicrophoneLevel";
import { constant } from "../state/Behavior";
const IDLE: MicrophoneState = { type: "level", level: 0 };
const IDLE: MicrophoneState = { type: "level", level$: constant(0) };
/**
* Reads the live input level of a microphone, while `active`.
*
* A bridge and nothing else: the capture, its lifetime and the maths belong to
* {@link observeMicrophoneState$}. Scoped to `active` so the device is held
* only while whatever shows the meter is on screen, rather than for the length
* of a call.
*/
/** The live level of a microphone, captured only while `active`. */
export function useMicrophoneLevel(
deviceId: string | undefined,
active: boolean,
@@ -30,8 +24,7 @@ export function useMicrophoneLevel(
useEffect(() => {
if (!active) return;
// Idle first, so a new device starts from nothing rather than from the
// level the previous one was reading.
// Idle first, so a new device doesn't start from the previous level.
setState(IDLE);
const subscription = observeMicrophoneState$(deviceId).subscribe(setState);
return (): void => subscription.unsubscribe();
+187
View File
@@ -0,0 +1,187 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { act, render, screen } from "@testing-library/react";
import { BehaviorSubject } from "rxjs";
import { describe, expect, it, vi } from "vitest";
import { arrangeTiles, type Bounds } from "./CallLayout";
import { makeGridLayout } from "./GridLayout";
import { makeSpotlightLandscapeLayout } from "./SpotlightLandscapeLayout";
import { makeSpotlightPortraitLayout } from "./SpotlightPortraitLayout";
import { type DragCallback, type LayoutProps } from "./Grid";
import { constant } from "../state/Behavior";
import {
type Alignment,
type GridLayout,
type SpotlightLandscapeLayout,
type SpotlightPortraitLayout,
} from "../state/layout-types";
import { type UserMediaViewModel } from "../state/media/UserMediaViewModel";
import {
GridTileViewModel,
SpotlightTileViewModel,
type TileViewModel,
} from "../state/TileViewModel";
// Grid supplies these hooks to the layouts it renders; on their own the
// layouts have no grid to report to
vi.mock("./Grid", async (importOriginal) => ({
...(await importOriginal<Record<string, unknown>>()),
useUpdateLayout: (): void => {},
useVisibleTiles: (): void => {},
}));
describe("grid layout", () => {
it("sizes the tiles from the minimum bounds", () => {
const minBounds$ = new BehaviorSubject<Bounds>({ width: 800, height: 600 });
const { scrolling: Scrolling } = makeGridLayout({ minBounds$ });
render(<Scrolling model={gridModel()} Slot={Slot} />);
const layer = screen.getByTestId("slot-0").parentElement!;
expect(layer.style.width).toBe("800px");
expect(layer.style.getPropertyValue("--width")).toBe(
`${Math.floor(arrangeTiles(800, 600, 3).tileWidth)}px`,
);
act(() => minBounds$.next({ width: 400, height: 300 }));
expect(layer.style.width).toBe("400px");
expect(layer.style.getPropertyValue("--width")).toBe(
`${Math.floor(arrangeTiles(400, 300, 3).tileWidth)}px`,
);
});
it("moves the spotlight only when a drag reaches another corner", () => {
const model = gridModel();
const { fixed: Fixed } = makeGridLayout({ minBounds$: bounds });
render(<Fixed model={model} Slot={Slot} />);
const alignments: Alignment[] = [];
model.spotlightAlignment$.subscribe((a) => alignments.push(a));
const drag = drags.get("spotlight")!;
// Still in the bottom right corner
act(() => drag({ x: 0, y: 0, xRatio: 0.9, yRatio: 0.9 }));
// Over to the bottom left
act(() => drag({ x: 0, y: 0, xRatio: 0.1, yRatio: 0.9 }));
// And a little further, still bottom left
act(() => drag({ x: 0, y: 0, xRatio: 0.2, yRatio: 0.8 }));
// Up to the top left
act(() => drag({ x: 0, y: 0, xRatio: 0.2, yRatio: 0.1 }));
expect(alignments).toEqual([
{ inline: "end", block: "end" },
{ inline: "start", block: "end" },
{ inline: "start", block: "start" },
]);
const slot = screen.getByTestId("slot-spotlight");
expect(slot.getAttribute("data-inline-alignment")).toBe("start");
expect(slot.getAttribute("data-block-alignment")).toBe("start");
});
});
describe("spotlight landscape layout", () => {
it("puts the spotlight in the fixed layer and the grid in the scrolling one", () => {
const { fixed: Fixed, scrolling: Scrolling } = makeSpotlightLandscapeLayout(
{ minBounds$: bounds },
);
const model = landscapeModel();
const { unmount } = render(<Fixed model={model} Slot={Slot} />);
expect(screen.getByTestId("slot-spotlight")).toBeInTheDocument();
unmount();
render(<Scrolling model={model} Slot={Slot} />);
expect(slotIds()).toEqual(model.grid.map((tile) => tile.id));
});
});
describe("spotlight portrait layout", () => {
it("sizes the grid tiles from the minimum width", () => {
const minBounds$ = new BehaviorSubject<Bounds>({ width: 300, height: 700 });
const { scrolling: Scrolling } = makeSpotlightPortraitLayout({
minBounds$,
});
const model = portraitModel();
render(<Scrolling model={model} Slot={Slot} />);
expect(slotIds()).toEqual(model.grid.map((tile) => tile.id));
const layer = screen.getByTestId(`slot-${model.grid[0].id}`).parentElement!
.parentElement!;
expect(layer.style.getPropertyValue("--grid-tile-width")).toBe(
`${Math.floor(arrangeTiles(300, 300, 3).tileWidth)}px`,
);
act(() => minBounds$.next({ width: 500, height: 700 }));
expect(layer.style.getPropertyValue("--grid-tile-width")).toBe(
`${Math.floor(arrangeTiles(500, 500, 3).tileWidth)}px`,
);
});
});
const bounds = constant<Bounds>({ width: 800, height: 600 });
// The layouts only pass tile view models through to their slots
const media = {} as UserMediaViewModel;
const spotlight = new SpotlightTileViewModel(
constant([]),
constant(false),
constant("solid"),
);
const tiles = (): GridTileViewModel[] =>
Array.from({ length: 3 }, () => new GridTileViewModel(constant(media)));
function gridModel(): GridLayout {
return {
type: "grid",
spotlight,
grid: tiles(),
spotlightAlignment$: new BehaviorSubject<Alignment>({
inline: "end",
block: "end",
}),
setVisibleTiles: () => {},
};
}
function landscapeModel(): SpotlightLandscapeLayout {
return {
type: "spotlight-landscape",
spotlight,
grid: tiles(),
setVisibleTiles: () => {},
};
}
function portraitModel(): SpotlightPortraitLayout {
return {
type: "spotlight-portrait",
spotlight,
grid: tiles(),
setVisibleTiles: () => {},
};
}
// Slots record their drag callback so a test can drag them, and show the id
// of the tile they hold
const drags = new Map<string, DragCallback | undefined>();
const Slot: LayoutProps<unknown, TileViewModel, HTMLDivElement>["Slot"] = ({
id,
model,
onDrag,
...props
}) => {
drags.set(id, onDrag);
return (
<div data-testid={`slot-${id}`} {...props}>
{model instanceof GridTileViewModel ? model.id : "spotlight"}
</div>
);
};
function slotIds(): string[] {
return screen
.getAllByTestId(/^slot-/)
.map((slot) => slot.textContent ?? "")
.filter((id) => id !== "spotlight");
}
+2 -2
View File
@@ -5,11 +5,11 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type Observable } from "rxjs";
import { type ComponentType } from "react";
import { type LayoutProps } from "./Grid";
import { type TileViewModel } from "../state/TileViewModel";
import { type Behavior } from "../state/Behavior";
export interface Bounds {
width: number;
@@ -20,7 +20,7 @@ export interface CallLayoutInputs {
/**
* The minimum bounds of the layout area.
*/
minBounds$: Observable<Bounds>;
minBounds$: Behavior<Bounds>;
}
export interface CallLayoutOutputs<Model> {
+12 -18
View File
@@ -11,14 +11,12 @@ import {
useCallback,
useMemo,
} from "react";
import { distinctUntilChanged } from "rxjs";
import { useObservableEagerState } from "observable-hooks";
import { type GridLayout as GridLayoutModel } from "../state/layout-types.ts";
import styles from "./GridLayout.module.css";
import { useInitial } from "../useInitial";
import { type CallLayout, arrangeTiles } from "./CallLayout";
import { type DragCallback, useUpdateLayout, useVisibleTiles } from "./Grid";
import { useBehavior } from "../useBehavior";
interface GridCSSProperties extends CSSProperties {
"--gap": string;
@@ -39,22 +37,18 @@ export const makeGridLayout: CallLayout<GridLayoutModel> = ({
// lives
fixed: function GridLayoutFixed({ ref, model, Slot }): ReactNode {
useUpdateLayout();
const alignment = useObservableEagerState(
useInitial(() =>
model.spotlightAlignment$.pipe(
distinctUntilChanged(
(a1, a2) => a1.block === a2.block && a1.inline === a2.inline,
),
),
),
);
const alignment = useBehavior(model.spotlightAlignment$);
const onDragSpotlight: DragCallback = useCallback(
({ xRatio, yRatio }) =>
model.spotlightAlignment$.next({
block: yRatio < 0.5 ? "start" : "end",
inline: xRatio < 0.5 ? "start" : "end",
}),
({ xRatio, yRatio }) => {
const block = yRatio < 0.5 ? "start" : "end";
const inline = xRatio < 0.5 ? "start" : "end";
// A drag reports the same alignment on every move; only a change is
// worth a re-render
const current = model.spotlightAlignment$.value;
if (current.block !== block || current.inline !== inline)
model.spotlightAlignment$.next({ block, inline });
},
[model.spotlightAlignment$],
);
@@ -78,7 +72,7 @@ export const makeGridLayout: CallLayout<GridLayoutModel> = ({
scrolling: function GridLayout({ ref, model, Slot }): ReactNode {
useUpdateLayout();
useVisibleTiles(model.setVisibleTiles);
const { width, height: minHeight } = useObservableEagerState(minBounds$);
const { width, height: minHeight } = useBehavior(minBounds$);
const { gap, tileWidth, tileHeight } = useMemo(
() => arrangeTiles(width, minHeight, model.grid.length),
[width, minHeight, model.grid.length],
+1 -2
View File
@@ -7,7 +7,6 @@ Please see LICENSE in the repository root for full details.
*/
import { type ReactNode, useCallback, useMemo } from "react";
import { useObservableEagerState } from "observable-hooks";
import classNames from "classnames";
import { type OneOnOneDesktopLayout as OneOnOneDesktopLayoutModel } from "../state/layout-types.ts";
@@ -37,7 +36,7 @@ export const makeOneOnOneDesktopLayout: CallLayout<
Slot,
}): ReactNode {
useUpdateLayout();
const { width, height } = useObservableEagerState(minBounds$);
const { width, height } = useBehavior(minBounds$);
const pipAlignment = useBehavior(model.pipAlignment$);
const { tileWidth, tileHeight } = useMemo(
() => arrangeTiles(width, height, 1),
+2 -3
View File
@@ -6,7 +6,6 @@ Please see LICENSE in the repository root for full details.
*/
import { type FC, type ReactNode } from "react";
import { useObservableEagerState } from "observable-hooks";
import classNames from "classnames";
import { type CallLayout } from "./CallLayout";
@@ -33,7 +32,7 @@ export const makeSpotlightLandscapeLayout: CallLayout<
Slot,
}): ReactNode {
useUpdateLayout();
useObservableEagerState(minBounds$);
useBehavior(minBounds$);
return (
<div ref={ref} className={styles.layer}>
@@ -56,7 +55,7 @@ export const makeSpotlightLandscapeLayout: CallLayout<
}): ReactNode {
useUpdateLayout();
useVisibleTiles(model.setVisibleTiles);
useObservableEagerState(minBounds$);
useBehavior(minBounds$);
return (
<div ref={ref} className={styles.layer}>
+1 -2
View File
@@ -6,7 +6,6 @@ Please see LICENSE in the repository root for full details.
*/
import { type ReactNode, type CSSProperties } from "react";
import { useObservableEagerState } from "observable-hooks";
import classNames from "classnames";
import { type CallLayout, arrangeTiles } from "./CallLayout";
@@ -58,7 +57,7 @@ export const makeSpotlightPortraitLayout: CallLayout<
}): ReactNode {
useUpdateLayout();
useVisibleTiles(model.setVisibleTiles);
const { width } = useObservableEagerState(minBounds$);
const { width } = useBehavior(minBounds$);
const { gap, tileWidth, tileHeight } = arrangeTiles(
width,
// TODO: We pretend that the minimum height is the width, because the
+4 -14
View File
@@ -20,8 +20,7 @@ import {
} from "react";
import { type LocalVideoTrack } from "livekit-client";
import { logger } from "matrix-js-sdk/lib/logger";
import { combineLatest, map, type Observable } from "rxjs";
import { useObservable } from "observable-hooks";
import { combineLatest } from "rxjs";
import {
backgroundBlur as backgroundBlurSettings,
@@ -31,6 +30,7 @@ import { BlurBackgroundTransformer } from "./BlurBackgroundTransformer";
import { type Behavior } from "../state/Behavior";
import { type ObservableScope } from "../state/ObservableScope";
import { platform } from "../Platform";
import { useValueBehavior } from "../useValueBehavior";
//TODO-MULTI-SFU: This is not yet fully there.
// it is a combination of exposing observable and react hooks.
@@ -52,18 +52,8 @@ export function useTrackProcessor(): ProcessorState {
return state;
}
export function useTrackProcessorObservable$(): Observable<ProcessorState> {
const state = use(ProcessorContext);
if (state === undefined)
throw new Error(
"useTrackProcessor must be used within a ProcessorProvider",
);
const state$ = useObservable(
(init$) => init$.pipe(map(([init]) => init)),
[state],
);
return state$;
export function useTrackProcessorState$(): Behavior<ProcessorState> {
return useValueBehavior(useTrackProcessor());
}
/**
+11
View File
@@ -29,6 +29,8 @@ import waveSoundOgg from "../sound/reactions/wave.ogg?url";
import waveSoundMp3 from "../sound/reactions/wave.mp3?url";
import baduntssSoundOgg from "../sound/reactions/baduntss.ogg?url";
import baduntssSoundMp3 from "../sound/reactions/baduntss.mp3?url";
import bellSoundOgg from "../sound/reactions/bell.ogg?url";
import bellSoundMp3 from "../sound/reactions/bell.mp3?url";
export const ElementCallReactionEventType = "io.element.call.reaction";
@@ -202,6 +204,15 @@ export const ReactionSet: ReactionOption[] = [
mp3: baduntssSoundMp3,
},
},
{
emoji: "🛎️",
name: "bell",
alias: ["ding"],
sound: {
ogg: bellSoundOgg,
mp3: bellSoundMp3,
},
},
];
export interface RaisedHandInfo {
+8 -12
View File
@@ -20,8 +20,6 @@ import {
import useMeasure from "react-use-measure";
import { type MatrixRTCSession } from "matrix-js-sdk/lib/matrixrtc";
import classNames from "classnames";
import { map } from "rxjs";
import { useObservable } from "observable-hooks";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { useTranslation } from "react-i18next";
@@ -77,10 +75,11 @@ import {
useAppBarSubtitle,
} from "../AppBar.tsx";
import { useBehavior } from "../useBehavior.ts";
import { useValueBehavior } from "../useValueBehavior.ts";
import { constant } from "../state/Behavior.ts";
import { Toast } from "../Toast.tsx";
import overlayStyles from "../Overlay.module.css";
import { useTrackProcessorObservable$ } from "../livekit/TrackProcessorContext.tsx";
import { useTrackProcessorState$ } from "../livekit/TrackProcessorContext.tsx";
import { type Layout } from "../state/layout-types.ts";
import { ObservableScope } from "../state/ObservableScope.ts";
import { CallFooter, type FooterSnapshot } from "../components/CallFooter.tsx";
@@ -121,7 +120,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
const urlParams = useUrlParams();
const hostBridge = useHostBridge();
const mediaDevices = useMediaDevices();
const trackProcessorState$ = useTrackProcessorObservable$();
const trackProcessorState$ = useTrackProcessorState$();
// The element we have to draw the call in: the page, or the container a host
// gave us. Its size, not the window's, decides how the call is laid out.
const rootElement = useRootElement();
@@ -151,7 +150,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
},
reactionsReader.raisedHands$,
reactionsReader.reactions$,
scope.behavior(trackProcessorState$),
trackProcessorState$,
);
// TODO move this somewhere else once we use the callViewModel in the lobby as well!
vm.join();
@@ -252,7 +251,7 @@ export const InCallView: FC<InCallViewProps> = ({
useWakeLock();
// TODO-MULTI-SFU This is unused now??
// const connectionState = useObservableEagerState(vm.livekitConnectionState$);
// const connectionState = useBehavior(vm.livekitConnectionState$);
// annoyingly we don't get the disconnection reason this way,
// only by listening for the emitted event
@@ -374,10 +373,7 @@ export const InCallView: FC<InCallViewProps> = ({
edgeToEdge,
],
);
const gridBoundsObservable$ = useObservable(
(inputs$) => inputs$.pipe(map(([gridBounds]) => gridBounds)),
[gridBounds],
);
const gridBounds$ = useValueBehavior(gridBounds);
useAppBarHidden(!showHeader);
useAppBarSubtitle(
@@ -538,7 +534,7 @@ export const InCallView: FC<InCallViewProps> = ({
);
const layouts = useMemo(() => {
const inputs = { minBounds$: gridBoundsObservable$ };
const inputs = { minBounds$: gridBounds$ };
return {
grid: makeGridLayout(inputs),
"spotlight-landscape": makeSpotlightLandscapeLayout(inputs),
@@ -547,7 +543,7 @@ export const InCallView: FC<InCallViewProps> = ({
"one-on-one-desktop": makeOneOnOneDesktopLayout(inputs),
"one-on-one-mobile": makeOneOnOneMobileLayout(inputs),
};
}, [gridBoundsObservable$]);
}, [gridBounds$]);
const showFooter = useBehavior(footerVm.showFooter$);
const renderContent = (): JSX.Element => {
+10 -46
View File
@@ -20,7 +20,12 @@ import {
import { LobbyView } from "./LobbyView";
import { E2eeType } from "../e2ee/e2eeType";
import { mockMediaDevices, mockMuteStates } from "../utils/test";
import {
mockMediaDevices,
mockMuteStates,
restoreAudioCapture,
stubAudioCapture,
} from "../utils/test";
import { type MediaDevices } from "../state/MediaDevices";
import { MediaDevicesContext } from "../MediaDevicesContext";
import { type ProcessorState } from "../livekit/TrackProcessorContext";
@@ -184,51 +189,11 @@ describe("LobbyView", () => {
});
describe("LobbyView microphone level", () => {
const realMediaDevices = Object.getOwnPropertyDescriptor(
navigator,
"mediaDevices",
);
afterEach(() => {
vi.unstubAllGlobals();
// Put navigator back, or every later test in the run inherits the stub.
if (realMediaDevices === undefined) {
Reflect.deleteProperty(navigator, "mediaDevices");
} else {
Object.defineProperty(navigator, "mediaDevices", realMediaDevices);
}
});
/** Just enough of the Web Audio and capture APIs for the meter to run. */
function stubAudioCapture(): void {
vi.stubGlobal(
"AudioContext",
class {
public readonly state = "running";
public createAnalyser(): object {
return {
fftSize: 1024,
getByteTimeDomainData: (): void => {},
};
}
public createMediaStreamSource(): object {
return { connect: (): void => {} };
}
public close(): void {}
},
);
// Only this property: replacing navigator wholesale drops the getters on
// its prototype, such as userAgent.
Object.defineProperty(navigator, "mediaDevices", {
configurable: true,
value: {
getUserMedia: vi.fn().mockResolvedValue({ getTracks: () => [] }),
},
});
}
afterEach(restoreAudioCapture);
it("shows the microphone level meter", async () => {
stubAudioCapture();
const capture = stubAudioCapture();
capture.grant();
const user = userEvent.setup();
const { getByRole } = renderLobbyView({}, false, "desktop", {
requestDeviceNames: (): void => {},
@@ -241,8 +206,7 @@ describe("LobbyView microphone level", () => {
},
} as unknown as Partial<MediaDevices>);
// The meter lives with the microphone picker, which the pre-join screen
// reaches through the same chevron as a call in progress.
// Pre-join reaches the meter through the same chevron as a call.
await user.click(getByRole("button", { name: "Microphone" }));
expect(
+1 -4
View File
@@ -24,7 +24,6 @@ import {
type LocalVideoTrack,
Track,
} from "livekit-client";
import { useObservableEagerState } from "observable-hooks";
import inCallStyles from "./InCallView.module.css";
import styles from "./LobbyView.module.css";
@@ -125,9 +124,7 @@ export const LobbyView: FC<Props> = ({
);
const devices = useMediaDevices();
const videoInputId = useObservableEagerState(
devices.videoInput.selected$,
)?.id;
const videoInputId = useBehavior(devices.videoInput.selected$)?.id;
// Capture the audio options as they were when we first mounted, because
// we're not doing anything with the audio anyway so we don't need to
+82
View File
@@ -0,0 +1,82 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { render, screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { Form } from "@vector-im/compound-web";
import { describe, expect, it, vi } from "vitest";
import { DeviceSelection } from "./DeviceSelection";
import { constant } from "../state/Behavior";
import {
type AudioOutputDeviceLabel,
type DeviceLabel,
type MediaDevice,
type SelectedDevice,
} from "../state/MediaDevices";
type Device = MediaDevice<DeviceLabel | AudioOutputDeviceLabel, SelectedDevice>;
function mockDevice(
available: [string, DeviceLabel][],
selectedId: string | undefined,
): Device & { select: ReturnType<typeof vi.fn> } {
return {
available$: constant(new Map(available)),
selected$: constant(
selectedId === undefined ? undefined : { id: selectedId },
),
select: vi.fn(),
} as unknown as Device & { select: ReturnType<typeof vi.fn> };
}
const twoDevices: [string, DeviceLabel][] = [
["a", { type: "name", name: "Mic A" }],
["b", { type: "number", number: 2 }],
];
describe("DeviceSelection", () => {
it("marks the selected device and selects another on click", async () => {
const device = mockDevice(twoDevices, "a");
render(
<Form.Root>
<DeviceSelection
device={device}
title="Microphone"
numberedLabel={(n) => `Microphone ${n}`}
/>
</Form.Root>,
);
const first = screen.getByRole<HTMLInputElement>("radio", {
name: "Mic A",
});
const second = screen.getByRole<HTMLInputElement>("radio", {
name: "Microphone 2",
});
expect(first.checked).toBe(true);
expect(second.checked).toBe(false);
await userEvent.click(second);
expect(device.select).toHaveBeenCalledWith("b");
});
it("shows nothing when there is no choice to make", () => {
const device = mockDevice([twoDevices[0]], "a");
render(
<Form.Root>
<DeviceSelection
device={device}
title="Microphone"
numberedLabel={(n) => `Microphone ${n}`}
/>
</Form.Root>,
);
expect(screen.queryByRole("radio")).toBeNull();
expect(screen.queryByText("Microphone")).toBeNull();
});
});
+3 -3
View File
@@ -21,7 +21,6 @@ import {
Separator,
} from "@vector-im/compound-web";
import { Trans, useTranslation } from "react-i18next";
import { useObservableEagerState } from "observable-hooks";
import {
type AudioOutputDeviceLabel,
@@ -30,6 +29,7 @@ import {
type MediaDevice,
} from "../state/MediaDevices";
import styles from "./DeviceSelection.module.css";
import { useBehavior } from "../useBehavior";
interface Props {
device: MediaDevice<DeviceLabel | AudioOutputDeviceLabel, SelectedDevice>;
@@ -44,8 +44,8 @@ export const DeviceSelection: FC<Props> = ({
}) => {
const { t } = useTranslation();
const groupId = useId();
const available = useObservableEagerState(device.available$);
const selectedId = useObservableEagerState(device.selected$)?.id;
const available = useBehavior(device.available$);
const selectedId = useBehavior(device.selected$)?.id;
const onChange = useCallback(
(e: ChangeEvent<HTMLInputElement>) => {
device.select(e.target.value);
+1
View File
@@ -10,6 +10,7 @@ The following sound effects have been licensed from Pixabay, under https://pixab
- `reactions/generic`
- `reactions/lightbulb`
- `reactions/party`
- `reactions/bell`
### Other
Binary file not shown.
Binary file not shown.
+16 -26
View File
@@ -16,7 +16,6 @@ import {
import { type Room as MatrixRoom } from "matrix-js-sdk";
import {
BehaviorSubject,
catchError,
combineLatest,
distinctUntilChanged,
filter,
@@ -39,11 +38,12 @@ import {
throttleTime,
timer,
takeUntil,
from,
} from "rxjs";
import { type Logger, logger as rootLogger } from "matrix-js-sdk/lib/logger";
import {
MembershipManagerEvent,
type LivekitTransportConfig,
type LivekitTransport,
type MatrixRTCSession,
type RTCCallIntent,
type RTCNotificationType,
@@ -112,7 +112,7 @@ import {
TransportState,
} from "./localMember/LocalMember.ts";
import {
createLocalTransport$,
getLocalTransport,
type LocalTransport,
} from "./localMember/LocalTransport.ts";
import {
@@ -573,16 +573,16 @@ export function createCallViewModel$(
: `${userId}:${deviceId}`,
};
const localTransport =
options.localTransport ??
createLocalTransport$({
scope: scope,
memberships$: memberships$,
ownMembershipIdentity,
client,
roomId: matrixRoom.roomId,
matrixRTCMode,
});
const localTransport$ = options.localTransport
? constant(options.localTransport)
: from(
getLocalTransport({
ownMembershipIdentity,
client,
roomId: matrixRoom.roomId,
matrixRTCMode,
}),
);
const connectionFactory =
options.connectionFactory ??
@@ -599,17 +599,7 @@ export function createCallViewModel$(
const connectionManager = createConnectionManager$({
scope: scope,
connectionFactory: connectionFactory,
localTransport$: scope.behavior(
localTransport.active$.pipe(
catchError((e: unknown) => {
logger.info(
"could not pass local transport to createConnectionManager$. localTransport$ threw an error",
e,
);
return of(null);
}),
),
),
localTransport$,
remoteTransports$: membershipsAndTransports.transports$,
logger: logger,
ownMembershipIdentity,
@@ -634,7 +624,7 @@ export function createCallViewModel$(
),
muteStates,
joinMatrixRTC: (
transport: LivekitTransportConfig,
transport: LivekitTransport,
delayedLeaveTimings: ResolvedDelayedLeaveTimings,
) => {
return enterRTCSession(
@@ -665,7 +655,7 @@ export function createCallViewModel$(
connectionManager,
client,
matrixRTCSession,
localTransport,
localTransport$,
roomId: matrixRoom.roomId,
hideScreensharing,
hostBridge,
@@ -210,11 +210,8 @@ export function withCallViewModel(mode: MatrixRTCMode) {
connectionState$,
windowSize$,
localTransport: {
active$: constant({
transport: exampleTransport,
sfuConfig: exampleSfuConfig,
}),
advertised$: constant(exampleTransport),
transport: exampleTransport,
sfuConfig: exampleSfuConfig,
},
connectionFactory: {
createConnection(
@@ -8,7 +8,7 @@ Please see LICENSE in the repository root for full details.
import {
Status as RTCMemberStatus,
type LivekitTransportConfig,
type LivekitTransport,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import {
@@ -21,7 +21,7 @@ import {
beforeEach,
afterEach,
} from "vitest";
import { BehaviorSubject, map, of } from "rxjs";
import { BehaviorSubject, map, of, Subject } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { type LocalParticipant, type LocalTrack } from "livekit-client";
import fetchMock from "fetch-mock";
@@ -50,10 +50,7 @@ import {
TrackState,
watchScreenShareToggle,
} from "./LocalMember";
import {
FailToGetOpenIdToken,
MatrixRTCTransportMissingError,
} from "../../../utils/errors";
import { MatrixRTCTransportMissingError } from "../../../utils/errors";
import { Epoch, ObservableScope } from "../../ObservableScope";
import { constant } from "../../Behavior";
import { ConnectionManagerData } from "../remoteMembers/ConnectionManager";
@@ -61,10 +58,7 @@ import { ConnectionState, type Connection } from "../remoteMembers/Connection";
import { type Publisher } from "./Publisher";
import { initializeWidget } from "../../../widget";
import { nullHostBridge } from "../../../HostBridge";
import {
type LocalTransport,
type LocalTransportWithSFUConfig,
} from "./LocalTransport";
import { type LocalTransport } from "./LocalTransport";
import * as openIDSFU from "../../../livekit/openIDSFU";
initializeWidget();
@@ -127,8 +121,7 @@ const mockedClient = {
};
describe("enterRTCSession", () => {
const transport: LivekitTransportConfig = {
livekit_alias: "roomId",
const transport: LivekitTransport = {
livekit_service_url: "http://my-livekit-service-url.com",
type: "livekit",
};
@@ -163,8 +156,7 @@ describe("enterRTCSession", () => {
memberId: "@alice:example.org:DEVICE",
userId: "@alice:example.org",
},
[],
transport,
[transport],
expect.objectContaining({ manageMediaKeys: true }),
);
});
@@ -184,7 +176,6 @@ describe("enterRTCSession", () => {
expect(mockedSession.joinRTCSession).toHaveBeenLastCalledWith(
expect.any(Object),
[],
expect.any(Object),
expect.objectContaining({
keyRotationParticipantLimit: 50,
@@ -196,7 +187,6 @@ describe("enterRTCSession", () => {
enterRTCSession(mockedSession, ownMemberMock, transport, options);
expect(mockedSession.joinRTCSession).toHaveBeenLastCalledWith(
expect.anything(),
expect.anything(),
expect.anything(),
expect.objectContaining({
@@ -261,8 +251,8 @@ describe("LocalMembership", () => {
});
it("throws error on missing RTC config error", () => {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
const localTransport$ = scope.behavior<null | LivekitTransportConfig>(
withTestScheduler(({ scope, hot, expectObservable }) => {
const localTransport$ = scope.behavior<null | LivekitTransport>(
hot("1ms #", {}, new MatrixRTCTransportMissingError("domain.com")),
null,
);
@@ -277,16 +267,15 @@ describe("LocalMembership", () => {
),
};
const aLocalTransport: LocalTransport = {
advertised$: localTransport$,
active$: behavior("a", { a: null }),
};
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: mockConnectionManager,
localTransport: aLocalTransport,
localTransport$: hot(
"1ms #",
{},
new MatrixRTCTransportMissingError("domain.com"),
),
});
expectObservable(localMembership.localMemberState$).toBe("ne", {
@@ -296,61 +285,9 @@ describe("LocalMembership", () => {
});
});
it("Should not publish to active transport if advertised has errors", () => {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
const advertised$ = scope.behavior<null | LivekitTransportConfig>(
hot("--#", {}, new FailToGetOpenIdToken(new Error("foo"))),
null,
);
// Populate a connection for active
const connectionManagerData = new ConnectionManagerData();
connectionManagerData.add(connectionTransportBConnected, []);
const mockConnectionManager = {
transports$: constant(new Epoch([bTransport])),
connectionManagerData$: constant(new Epoch(connectionManagerData)),
};
const aLocalTransport: LocalTransport = {
advertised$,
active$: behavior("a", { n: null, a: bTransportWithSFUConfig }),
};
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
return {} as unknown as Publisher;
},
);
const publisherFactory =
defaultCreateLocalMemberValues.createPublisherFactory as ReturnType<
typeof vi.fn
>;
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: mockConnectionManager,
localTransport: aLocalTransport,
});
expectObservable(localMembership.localMemberState$).toBe("n-e", {
n: TransportState.Waiting,
e: expect.toSatisfy((e) => e instanceof FailToGetOpenIdToken),
});
// Should not have created any publisher
expect(publisherFactory).toHaveBeenCalledTimes(0);
});
});
it("logs if callIntent cannot be updated", async () => {
const scope = new ObservableScope();
const aLocalTransport: LocalTransport = {
advertised$: constant(aTransport),
active$: constant(aTransportWithSFUConfig),
};
const mockConnectionManager = {
transports$: constant(new Epoch([])),
connectionManagerData$: constant(new Epoch(new ConnectionManagerData())),
@@ -366,7 +303,7 @@ describe("LocalMembership", () => {
leaveRoomSession: vi.fn(),
},
connectionManager: mockConnectionManager,
localTransport: aLocalTransport,
localTransport$: constant(mockTransport),
});
const expextedLog =
"'not connected yet' while updating the call intent (this is expected on startup)";
@@ -378,33 +315,19 @@ describe("LocalMembership", () => {
scope.end();
});
const aTransport = {
const mockTransportConfig = {
livekit_service_url: "a",
} as LivekitTransportConfig;
} as LivekitTransport;
const aTransportWithSFUConfig = {
transport: aTransport,
const mockTransport = {
transport: mockTransportConfig,
sfuConfig: {
jwt: "foo",
livekitAlias: "bar",
livekitIdentity: "baz",
url: "bro",
},
} as LocalTransportWithSFUConfig;
const bTransport = {
livekit_service_url: "b",
} as LivekitTransportConfig;
const bTransportWithSFUConfig = {
transport: bTransport,
sfuConfig: {
jwt: "foo2",
livekitAlias: "bar2",
livekitIdentity: "baz2",
url: "bro2",
},
} as LocalTransportWithSFUConfig;
} as LocalTransport;
const connectionTransportAConnected = {
livekitRoom: mockLivekitRoom({
@@ -414,18 +337,13 @@ describe("LocalMembership", () => {
} as unknown as LocalParticipant,
}),
state$: constant(ConnectionState.LivekitConnected),
transport: aTransport,
} as unknown as Connection;
transport: mockTransportConfig,
} as Connection;
const connectionTransportAConnecting = {
...connectionTransportAConnected,
state$: constant(ConnectionState.LivekitConnecting),
livekitRoom: mockLivekitRoom({}),
} as unknown as Connection;
const connectionTransportBConnected = {
state$: constant(ConnectionState.LivekitConnected),
transport: bTransport,
livekitRoom: mockLivekitRoom({}),
} as unknown as Connection;
const authCallSpy = vi
.spyOn(openIDSFU, "getSFUConfigWithOpenID")
@@ -459,10 +377,7 @@ describe("LocalMembership", () => {
),
},
joinMatrixRTC,
localTransport: {
advertised$: constant(aTransport),
active$: constant(aTransportWithSFUConfig),
},
localTransport$: constant(mockTransport),
delayId$,
});
@@ -471,7 +386,7 @@ describe("LocalMembership", () => {
await flushPromises();
// Joins with timings appropriate for the level of delegation support
expect(joinMatrixRTC).toHaveBeenCalledWith(
aTransport,
mockTransportConfig,
delayedLeaveTimings,
);
@@ -505,75 +420,7 @@ describe("LocalMembership", () => {
},
);
it("recreates publisher if new connection is used, always unpublish and end tracks", async () => {
const scope = new ObservableScope();
const activeTransport$ = new BehaviorSubject(aTransportWithSFUConfig);
const aLocalTransport: LocalTransport = {
advertised$: constant(aTransport),
active$: activeTransport$,
};
const publishers: Publisher[] = [];
let seed = 0;
defaultCreateLocalMemberValues.createPublisherFactory.mockImplementation(
() => {
const a = seed;
seed += 1;
logger.info(`creating [${a}]`);
const p = {
// It is enought to check if destroy is called. Destroy itself is tested in the publisher to make sure it does
// all the cleanup we need.
destroy: vi.fn(),
stopPublishing: vi.fn().mockImplementation(() => {
logger.info(`stopPublishing [${a}]`);
}),
stopTracks: vi.fn(),
};
publishers.push(p as unknown as Publisher);
return p;
},
);
const publisherFactory =
defaultCreateLocalMemberValues.createPublisherFactory as ReturnType<
typeof vi.fn
>;
const connectionManagerData = new ConnectionManagerData();
connectionManagerData.add(connectionTransportAConnected, []);
connectionManagerData.add(connectionTransportBConnected, []);
createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport: aLocalTransport,
});
await flushPromises();
activeTransport$.next({
...aTransportWithSFUConfig,
transport: bTransport,
});
await flushPromises();
expect(publisherFactory).toHaveBeenCalledTimes(2);
expect(publishers.length).toBe(2);
// stop the first Publisher and let the second one life.
expect(publishers[0].destroy).toHaveBeenCalled();
expect(publishers[1].destroy).not.toHaveBeenCalled();
expect(publisherFactory.mock.calls[0][0].transport).toBe(aTransport);
expect(publisherFactory.mock.calls[1][0].transport).toBe(bTransport);
scope.end();
await flushPromises();
// stop all tracks after ending scopes
expect(publishers[1].destroy).toHaveBeenCalled();
// expect(publishers[1].stopTracks).toHaveBeenCalled();
defaultCreateLocalMemberValues.createPublisherFactory.mockReset();
});
it("only start tracks if requested", async () => {
it("only starts tracks if requested", async () => {
const scope = new ObservableScope();
const publishers: Publisher[] = [];
@@ -602,11 +449,6 @@ describe("LocalMembership", () => {
typeof vi.fn
>;
const aLocalTransport: LocalTransport = {
advertised$: constant(aTransport),
active$: constant(aTransportWithSFUConfig),
};
const connectionManagerData = new ConnectionManagerData();
connectionManagerData.add(connectionTransportAConnected, []);
// connectionManagerData.add(connectionTransportB, []);
@@ -616,7 +458,7 @@ describe("LocalMembership", () => {
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport: aLocalTransport,
localTransport$: constant(mockTransport),
});
await flushPromises();
expect(publisherFactory).toHaveBeenCalledOnce();
@@ -637,16 +479,8 @@ describe("LocalMembership", () => {
//
it("tracks livekit state correctly", async () => {
const scope = new ObservableScope();
const connectionManagerData = new ConnectionManagerData();
const activeTransport$ =
new BehaviorSubject<null | LocalTransportWithSFUConfig>(null);
const aLocalTransport: LocalTransport = {
advertised$: constant(aTransport),
active$: activeTransport$,
};
const localTransport$ = new Subject<LocalTransport>();
const connectionManagerData$ = new BehaviorSubject(
new Epoch(connectionManagerData),
@@ -687,14 +521,14 @@ describe("LocalMembership", () => {
connectionManager: {
connectionManagerData$,
},
localTransport: aLocalTransport,
localTransport$,
});
await flushPromises();
expect(localMembership.localMemberState$.value).toStrictEqual(
TransportState.Waiting,
);
activeTransport$.next(aTransportWithSFUConfig);
localTransport$.next(mockTransport);
await flushPromises();
expect(localMembership.localMemberState$.value).toStrictEqual({
matrix: RTCMemberStatus.Connected,
@@ -719,7 +553,7 @@ describe("LocalMembership", () => {
});
(
connectionManagerData2.getConnectionForTransport(aTransport)!
connectionManagerData2.getConnectionForTransport(mockTransportConfig)!
.state$ as BehaviorSubject<ConnectionState>
).next(ConnectionState.LivekitConnected);
expect(localMembership.localMemberState$.value).toStrictEqual({
@@ -822,10 +656,7 @@ describe("LocalMembership", () => {
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport: {
advertised$: constant(aTransport),
active$: constant(aTransportWithSFUConfig),
},
localTransport$: constant(mockTransport),
});
await flushPromises();
@@ -862,10 +693,7 @@ describe("LocalMembership", () => {
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport: {
advertised$: constant(aTransport),
active$: constant(aTransportWithSFUConfig),
},
localTransport$: constant(mockTransport),
});
await flushPromises();
@@ -913,10 +741,7 @@ describe("LocalMembership", () => {
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport: {
advertised$: constant(aTransport),
active$: constant(aTransportWithSFUConfig),
},
localTransport$: constant(mockTransport),
});
await flushPromises();
@@ -957,10 +782,7 @@ describe("LocalMembership", () => {
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport: {
advertised$: constant(aTransport),
active$: constant(aTransportWithSFUConfig),
},
localTransport$: constant(mockTransport),
});
await flushPromises();
@@ -1026,10 +848,7 @@ describe("LocalMembership", () => {
connectionManager: {
connectionManagerData$: constant(new Epoch(connectionManagerData)),
},
localTransport: {
advertised$: constant(aTransport),
active$: constant(aTransportWithSFUConfig),
},
localTransport$: constant(mockTransport),
});
return { scope, localMembership };
};
@@ -1039,7 +858,7 @@ describe("LocalMembership", () => {
const setScreenShareEnabled = vi.fn().mockRejectedValue(error);
const connection = {
state$: constant(ConnectionState.LivekitConnected),
transport: aTransport,
transport: mockTransportConfig,
livekitRoom: mockLivekitRoom({
localParticipant: mockLocalParticipant({
isScreenShareEnabled: false,
+107 -155
View File
@@ -19,7 +19,6 @@ import { type MatrixClient } from "matrix-js-sdk";
import {
Status as RTCSessionStatus,
type LivekitTransport,
type LivekitTransportConfig,
type MatrixRTCSession,
type RTCCallIntent,
type RTCNotificationType,
@@ -38,6 +37,10 @@ import {
startWith,
switchMap,
tap,
NEVER,
concat,
race,
Subject,
} from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { deepCompare } from "matrix-js-sdk/lib/utils";
@@ -77,8 +80,6 @@ import {
} from "../remoteMembers/Connection.ts";
import { type HomeserverConnected } from "./HomeserverConnected.ts";
import { type LocalTransport } from "./LocalTransport.ts";
import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers.ts";
import { or$ } from "../../../utils/observable.ts";
import { getSFUConfigWithOpenID } from "../../../livekit/openIDSFU.ts";
export enum TransportState {
@@ -142,13 +143,13 @@ interface Props {
connectionManager: IConnectionManager;
createPublisherFactory: (connection: Connection) => Publisher;
joinMatrixRTC: (
transport: LivekitTransportConfig,
transport: LivekitTransport,
delayedLeaveTimings: ResolvedDelayedLeaveTimings,
) => void;
homeserverConnected: HomeserverConnected;
roomId: string;
ownMembershipIdentity: CallMembershipIdentityParts;
localTransport: LocalTransport;
localTransport$: Observable<LocalTransport>;
client: Pick<MatrixClient, "getDeviceId" | "getOpenIdToken">;
matrixRTCSession: Pick<
MatrixRTCSession,
@@ -194,7 +195,7 @@ interface Props {
export const createLocalMembership$ = ({
scope,
connectionManager,
localTransport,
localTransport$: localTransportWithErrors$,
homeserverConnected,
createPublisherFactory,
joinMatrixRTC,
@@ -252,20 +253,24 @@ export const createLocalMembership$ = ({
const logger = parentLogger.getChild("[LocalMembership]");
logger.debug(`Creating local membership..`);
// We consider error on the transport as fatal.
// Whether it is the active transport or the preferred transport.
const handleTransportError = (e: unknown): Observable<null> => {
let error: ElementCallError;
if (e instanceof ElementCallError) {
error = e;
} else {
error = new UnknownCallError(
e instanceof Error ? e : new Error("Unknown error from localTransport"),
);
}
setTransportError(error);
return of(null);
};
// Unwrap the local transport and set the state of the LocalMembership to error in case the transport is an error.
const fatalTransportError$ = new Subject<ElementCallError>();
const localTransport$ = localTransportWithErrors$.pipe(
catchError((e: unknown) => {
let error: ElementCallError;
if (e instanceof ElementCallError) {
error = e;
} else {
error = new UnknownCallError(
e instanceof Error
? e
: new Error("Unknown error from localTransport"),
);
}
fatalTransportError$.next(error);
return NEVER; // Make this Observable swallow the error
}),
);
async function checkDelegationSupport(
endpointUrl: string,
@@ -303,58 +308,40 @@ export const createLocalMembership$ = ({
// The transport that we will advertise in our membership, paired with info as
// to whether delayed event delegation is supported
const joinParams$ = localTransport.advertised$.pipe(
catchError(handleTransportError),
distinctUntilChanged(areLivekitTransportsEqual),
switchMap((transport) => {
if (transport === null) return of(null);
const transportSupportsDelegation = checkDelegationSupport(
transport.livekit_service_url + "/delegate_delayed_leave",
`transport ${transport.livekit_service_url}`,
);
return or$(
from(homeserverSupportsDelegation),
from(transportSupportsDelegation),
).pipe(
map((delegationSupported) => ({ transport, delegationSupported })),
startWith(null),
);
}),
);
// Unwrap the local transport and set the state of the LocalMembership to error in case the transport is an error.
const activeTransport$ = scope.behavior(
combineLatest([localTransport.active$, localTransport.advertised$]).pipe(
map(([active, advertised]) => {
// Our policy is to not publish to another transport if our prefered transport is miss-configured
if (advertised == null) return null;
return active?.transport ?? null;
const joinParams$ = scope.behavior(
localTransport$.pipe(
switchMap(async ({ transport }) => {
const transportSupportsDelegation = checkDelegationSupport(
transport.livekit_service_url + "/delegate_delayed_leave",
`transport ${transport.livekit_service_url}`,
);
return {
transport,
delegationSupported:
(await homeserverSupportsDelegation) ||
(await transportSupportsDelegation),
};
}),
catchError(handleTransportError),
distinctUntilChanged(areLivekitTransportsEqual),
),
null,
);
// Drop Epoch data here since we will not combine this anymore
const localConnection$ = scope.behavior(
combineLatest([
connectionManager.connectionManagerData$,
activeTransport$,
localTransport$,
]).pipe(
map(([{ value: connectionData }, localTransport]) => {
if (localTransport === null) {
return null;
}
return connectionData.getConnectionForTransport(localTransport);
}),
map(([{ value: connectionData }, { transport }]) =>
connectionData.getConnectionForTransport(transport),
),
tap((connection) => {
logger.info(
`Local connection updated: ${connection?.transport?.livekit_service_url}`,
);
}),
),
null,
);
// Tracks error that happen when creating the local tracks.
@@ -490,18 +477,6 @@ export const createLocalMembership$ = ({
}
};
const fatalTransportError$ = new BehaviorSubject<ElementCallError | null>(
null,
);
const setTransportError = (e: ElementCallError): void => {
if (fatalTransportError$.value !== null) {
logger.error("Multiple Transport Errors:", e);
} else {
fatalTransportError$.next(e);
}
};
const localConnectionState$ = localConnection$.pipe(
switchMap((connection) => (connection ? connection.state$ : of(null))),
);
@@ -509,38 +484,29 @@ export const createLocalMembership$ = ({
const mediaState$: Behavior<LocalMemberMediaState> = scope.behavior(
combineLatest([
localConnectionState$,
activeTransport$,
joinAndPublishRequested$,
from(trackStartRequested.promise).pipe(
map(() => true),
startWith(false),
),
]).pipe(
map(
([
localConnectionState,
localTransport,
shouldPublish,
shouldStartTracks,
]) => {
if (!localTransport) return null;
const trackState: TrackState = shouldStartTracks
? TrackState.Ready
: TrackState.WaitingForUser;
map(([localConnectionState, shouldPublish, shouldStartTracks]) => {
const trackState: TrackState = shouldStartTracks
? TrackState.Ready
: TrackState.WaitingForUser;
if (
localConnectionState !== ConnectionState.LivekitConnected ||
trackState !== TrackState.Ready
)
return {
connection: localConnectionState,
tracks: trackState,
};
if (!shouldPublish) return PublishState.WaitingForUser;
// if (!publishing) return PublishState.Starting;
return PublishState.Publishing;
},
),
if (
localConnectionState !== ConnectionState.LivekitConnected ||
trackState !== TrackState.Ready
)
return {
connection: localConnectionState,
tracks: trackState,
};
if (!shouldPublish) return PublishState.WaitingForUser;
// if (!publishing) return PublishState.Starting;
return PublishState.Publishing;
}),
distinctUntilChanged(deepCompare),
),
);
@@ -554,30 +520,36 @@ export const createLocalMembership$ = ({
};
const localMemberState$ = scope.behavior<LocalMemberState>(
combineLatest([
mediaState$,
homeserverConnected.rtsSession$,
fatalMatrixError$,
fatalTransportError$,
publishError$,
]).pipe(
map(
([
mediaState,
rtcSessionStatus,
fatalMatrixError,
fatalTransportError,
publishError,
]) => {
if (fatalTransportError !== null) return fatalTransportError;
// `mediaState` will be 'null' until the transport/connection appears.
if (mediaState && rtcSessionStatus)
return {
matrix: fatalMatrixError ?? rtcSessionStatus,
media: publishError ?? mediaState,
};
return TransportState.Waiting;
},
concat(
// Waiting until…
of(TransportState.Waiting),
race(
// either there is a fatal transport error
fatalTransportError$,
// or the transport is available.
localTransport$.pipe(
switchMap(() =>
// Once available, track session/media state.
combineLatest([
mediaState$,
homeserverConnected.rtsSession$,
fatalMatrixError$,
publishError$,
]).pipe(
map(
([
mediaState,
rtcSessionStatus,
fatalMatrixError,
publishError,
]) => ({
matrix: fatalMatrixError ?? rtcSessionStatus,
media: publishError ?? mediaState,
}),
),
),
),
),
),
),
);
@@ -685,7 +657,7 @@ export const createLocalMembership$ = ({
});
});
// Keep matrix rtc session in sync with advertisedTransport$, connectRequested$
// Join and leave the session as needed
scope.reconcile(
scope.behavior(combineLatest([joinParams$, joinAndPublishRequested$])),
async ([joinParams, shouldConnect]) => {
@@ -993,7 +965,7 @@ interface EnterRTCSessionOptions {
export function enterRTCSession(
rtcSession: MatrixRTCSession,
ownMembershipIdentity: CallMembershipIdentityParts,
transport: LivekitTransportConfig,
transport: LivekitTransport,
{
encryptMedia,
matrixRTCMode,
@@ -1014,21 +986,6 @@ export function enterRTCSession(
matrix_rtc_session: sessionConfig,
} = Config.get();
const retryInterval = sessionConfig.network_error_retry_ms;
const multiSFU =
matrixRTCMode === MatrixRTCMode.Compatibility ||
matrixRTCMode === MatrixRTCMode.Matrix_2_0;
// For backwards compatibility with Element Call versions that do not do Matrix 2.0,
// we add the livekit alias to the transport.
let backwardCompatibleTransport: LivekitTransport | LivekitTransportConfig;
if (matrixRTCMode === MatrixRTCMode.Matrix_2_0) {
backwardCompatibleTransport = transport;
} else {
backwardCompatibleTransport = {
livekit_alias: rtcSession.room.roomId,
...transport,
};
}
// Set maximumNetworkErrorRetryCount such that we will consider the client
// disconnected as soon as either it fails to sync for longer than the grace
@@ -1041,24 +998,19 @@ export function enterRTCSession(
// Multi-sfu does not need a preferred foci list. just the focus that is actually used.
// TODO where/how do we track errors originating from the ongoing rtcSession?
rtcSession.joinRTCSession(
ownMembershipIdentity,
multiSFU ? [] : [backwardCompatibleTransport],
multiSFU ? backwardCompatibleTransport : undefined,
{
notificationType,
callIntent,
manageMediaKeys: encryptMedia,
delayedLeaveEventRestartMs: delayedLeaveTimings.restart_ms,
delayedLeaveEventDelayMs: delayedLeaveTimings.delay_ms,
delayedLeaveEventRestartLocalTimeoutMs:
delayedLeaveTimings.restart_timeout_ms,
networkErrorRetryMs: sessionConfig.network_error_retry_ms,
makeKeyDelay: sessionConfig.wait_for_key_rotation_ms,
membershipEventExpiryMs: sessionConfig.membership_event_expiry_ms,
keyRotationParticipantLimit: sessionConfig.key_rotation_participant_limit,
unstableSendStickyEvents: matrixRTCMode === MatrixRTCMode.Matrix_2_0,
maximumNetworkErrorRetryCount: maximumNetworkErrorRetryCount,
},
);
rtcSession.joinRTCSession(ownMembershipIdentity, [transport], {
notificationType,
callIntent,
manageMediaKeys: encryptMedia,
delayedLeaveEventRestartMs: delayedLeaveTimings.restart_ms,
delayedLeaveEventDelayMs: delayedLeaveTimings.delay_ms,
delayedLeaveEventRestartLocalTimeoutMs:
delayedLeaveTimings.restart_timeout_ms,
networkErrorRetryMs: sessionConfig.network_error_retry_ms,
makeKeyDelay: sessionConfig.wait_for_key_rotation_ms,
membershipEventExpiryMs: sessionConfig.membership_event_expiry_ms,
keyRotationParticipantLimit: sessionConfig.key_rotation_participant_limit,
unstableSendStickyEvents: matrixRTCMode === MatrixRTCMode.Matrix_2_0,
maximumNetworkErrorRetryCount: maximumNetworkErrorRetryCount,
});
}
@@ -14,19 +14,10 @@ import {
type MockedObject,
vi,
} from "vitest";
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { lastValueFrom } from "rxjs";
import fetchMock from "fetch-mock";
import {
mockConfig,
flushPromises,
ownMemberMock,
testScope,
} from "../../../utils/test";
import { createLocalTransport$ } from "./LocalTransport";
import { constant } from "../../Behavior";
import { Epoch, ObservableScope } from "../../ObservableScope";
import { mockConfig, ownMemberMock } from "../../../utils/test";
import { getLocalTransport } from "./LocalTransport";
import {
MatrixRTCTransportMissingError,
FailToGetOpenIdToken,
@@ -47,118 +38,79 @@ describe("LocalTransport", () => {
beforeEach(() => vi.clearAllMocks());
it("throws if config is missing", async () => {
const { advertised$, active$ } = createLocalTransport$({
scope: testScope(),
roomId: "!room:example.org",
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getDomain: () => "example.org",
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
ownMembershipIdentity: ownMemberMock,
matrixRTCMode: MatrixRTCMode.Compatibility,
});
await flushPromises();
expect(() => advertised$.value).toThrow(
new MatrixRTCTransportMissingError("example.org"),
);
expect(() => active$.value).toThrow(
new MatrixRTCTransportMissingError("example.org"),
);
await expect(
getLocalTransport({
roomId: "!room:example.org",
client: {
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getDomain: () => "example.org",
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
ownMembershipIdentity: ownMemberMock,
matrixRTCMode: MatrixRTCMode.Compatibility,
}),
).rejects.toThrow(new MatrixRTCTransportMissingError("example.org"));
});
it("throws FailToGetOpenIdToken when OpenID fetch fails", async () => {
// Provide a valid config so makeTransportInternal resolves a transport
const scope = new ObservableScope();
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const resolver = Promise.withResolvers<void>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockImplementation(
async () => {
await resolver.promise;
throw new FailToGetOpenIdToken(new Error("no openid"));
},
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockRejectedValue(
new FailToGetOpenIdToken(new Error("no openid")),
);
const observations: unknown[] = [];
const errors: Error[] = [];
const { advertised$, active$ } = createLocalTransport$({
scope,
roomId: "!example_room_id",
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "example.org",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
ownMembershipIdentity: ownMemberMock,
matrixRTCMode: MatrixRTCMode.Compatibility,
});
active$.subscribe(
(o) => observations.push(o),
(e) => errors.push(e),
);
resolver.resolve();
await flushPromises();
const expectedError = new FailToGetOpenIdToken(new Error("no openid"));
expect(observations).toStrictEqual([null]);
expect(errors).toStrictEqual([expectedError]);
expect(() => advertised$.value).toThrow(expectedError);
expect(() => active$.value).toThrow(expectedError);
await expect(
getLocalTransport({
roomId: "!example_room_id",
client: {
getDomain: () => "example.org",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
ownMembershipIdentity: ownMemberMock,
matrixRTCMode: MatrixRTCMode.Compatibility,
}),
).rejects.toThrow(new FailToGetOpenIdToken(new Error("no openid")));
});
it("emits preferred transport after OpenID resolves", async () => {
it("returns preferred transport", async () => {
// Use config so transport discovery succeeds, but delay OpenID JWT fetch
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const openIdResolver = Promise.withResolvers<openIDSFU.SFUConfig>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockReturnValue(
openIdResolver.promise,
);
const { advertised$, active$ } = createLocalTransport$({
scope: testScope(),
roomId: "!room:example.org",
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getDomain: () => "example.org",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
ownMembershipIdentity: ownMemberMock,
matrixRTCMode: MatrixRTCMode.Compatibility,
});
openIdResolver.resolve?.({
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockResolvedValue({
url: "https://lk.example.org",
jwt: "jwt",
livekitAlias: "Akph4alDMhen",
livekitIdentity: ownMemberMock.userId + ":" + ownMemberMock.deviceId,
});
expect(advertised$.value).toBe(null);
expect(active$.value).toBe(null);
await flushPromises();
// final
const expectedTransport = {
livekit_service_url: "https://lk.example.org",
type: "livekit",
};
expect(advertised$.value).toStrictEqual(expectedTransport);
expect(active$.value).toStrictEqual({
transport: expectedTransport,
expect(
await getLocalTransport({
roomId: "!room:example.org",
client: {
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
getDomain: () => "example.org",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
ownMembershipIdentity: ownMemberMock,
matrixRTCMode: MatrixRTCMode.Compatibility,
}),
).toStrictEqual({
transport: {
livekit_service_url: "https://lk.example.org",
type: "livekit",
},
sfuConfig: {
jwt: "jwt",
livekitAlias: "Akph4alDMhen",
@@ -168,22 +120,19 @@ describe("LocalTransport", () => {
});
});
type LocalTransportProps = Parameters<typeof createLocalTransport$>[0];
type LocalTransportProps = Parameters<typeof getLocalTransport>[0];
describe("transport configuration mechanisms", () => {
let localTransportOpts: LocalTransportProps & {
client: MockedObject<LocalTransportProps["client"]>;
};
let openIdResolver: PromiseWithResolvers<openIDSFU.SFUConfig>;
beforeEach(() => {
mockConfig({});
customLivekitUrl.setValue(customLivekitUrl.defaultValue);
localTransportOpts = {
ownMembershipIdentity: ownMemberMock,
scope: testScope(),
roomId: "!example_room_id",
matrixRTCMode: MatrixRTCMode.Compatibility,
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
// eslint-disable-next-line @typescript-eslint/naming-convention
@@ -192,10 +141,6 @@ describe("LocalTransport", () => {
getDeviceId: vi.fn(),
},
};
openIdResolver = Promise.withResolvers<openIDSFU.SFUConfig>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockReturnValue(
openIdResolver.promise,
);
});
afterEach(() => {
@@ -206,19 +151,15 @@ describe("LocalTransport", () => {
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const { advertised$, active$ } =
createLocalTransport$(localTransportOpts);
openIdResolver.resolve?.(openIdResponse);
expect(advertised$.value).toBe(null);
expect(active$.value).toBe(null);
await flushPromises();
const expectedTransport = {
livekit_service_url: "https://lk.example.org",
type: "livekit",
};
expect(advertised$.value).toStrictEqual(expectedTransport);
expect(active$.value).toStrictEqual({
transport: expectedTransport,
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockResolvedValue(
openIdResponse,
);
expect(await getLocalTransport(localTransportOpts)).toStrictEqual({
transport: {
livekit_service_url: "https://lk.example.org",
type: "livekit",
},
sfuConfig: {
jwt: "e30=.eyJzdWIiOiJAbWU6ZXhhbXBsZS5vcmc6QUJDREVGIiwidmlkZW8iOnsicm9vbSI6IiFleGFtcGxlX3Jvb21faWQifX0=.e30=",
livekitAlias: "Akph4alDMhen",
@@ -230,12 +171,11 @@ describe("LocalTransport", () => {
it("supports getting transport via user settings", async () => {
customLivekitUrl.setValue("https://lk.example.org");
const { advertised$, active$ } =
createLocalTransport$(localTransportOpts);
openIdResolver.resolve?.(openIdResponse);
expect(advertised$.value).toBe(null);
await flushPromises();
expect(active$.value).toStrictEqual({
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockResolvedValue(
openIdResponse,
);
expect(await getLocalTransport(localTransportOpts)).toStrictEqual({
transport: {
livekit_service_url: "https://lk.example.org",
type: "livekit",
@@ -253,19 +193,15 @@ describe("LocalTransport", () => {
localTransportOpts.client._unstable_getRTCTransports.mockResolvedValue([
{ type: "livekit", livekit_service_url: "https://lk.example.org" },
]);
const { advertised$, active$ } =
createLocalTransport$(localTransportOpts);
openIdResolver.resolve?.(openIdResponse);
expect(advertised$.value).toBe(null);
expect(active$.value).toBe(null);
await flushPromises();
const expectedTransport = {
livekit_service_url: "https://lk.example.org",
type: "livekit",
};
expect(advertised$.value).toStrictEqual(expectedTransport);
expect(active$.value).toStrictEqual({
transport: expectedTransport,
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockResolvedValue(
openIdResponse,
);
expect(await getLocalTransport(localTransportOpts)).toStrictEqual({
transport: {
livekit_service_url: "https://lk.example.org",
type: "livekit",
},
sfuConfig: {
jwt: "e30=.eyJzdWIiOiJAbWU6ZXhhbXBsZS5vcmc6QUJDREVGIiwidmlkZW8iOnsicm9vbSI6IiFleGFtcGxlX3Jvb21faWQifX0=.e30=",
livekitAlias: "Akph4alDMhen",
@@ -279,38 +215,31 @@ describe("LocalTransport", () => {
localTransportOpts.client._unstable_getRTCTransports.mockResolvedValue([
{ type: "livekit", livekit_service_url: "https://lk.example.org" },
]);
openIdResolver.reject(
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockRejectedValue(
new FailToGetOpenIdToken(new Error("Test driven error")),
);
await expect(async () =>
lastValueFrom(createLocalTransport$(localTransportOpts).active$),
).rejects.toThrow(expect.any(FailToGetOpenIdToken));
await expect(getLocalTransport(localTransportOpts)).rejects.toThrow(
expect.any(FailToGetOpenIdToken),
);
});
it("throws if no options are available", async () => {
const { advertised$, active$ } = createLocalTransport$({
scope: testScope(),
ownMembershipIdentity: ownMemberMock,
roomId: "!example_room_id",
matrixRTCMode: MatrixRTCMode.Compatibility,
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "example.org",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
await flushPromises();
expect(() => advertised$.value).toThrow(
new MatrixRTCTransportMissingError("example.org"),
);
expect(() => active$.value).toThrow(
new MatrixRTCTransportMissingError("example.org"),
);
await expect(
getLocalTransport({
ownMembershipIdentity: ownMemberMock,
roomId: "!example_room_id",
matrixRTCMode: MatrixRTCMode.Compatibility,
client: {
getDomain: () => "example.org",
// eslint-disable-next-line @typescript-eslint/naming-convention
_unstable_getRTCTransports: async () => Promise.resolve([]),
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
}),
).rejects.toThrow(new MatrixRTCTransportMissingError("example.org"));
});
});
});
@@ -5,17 +5,11 @@ SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type CallMembership,
type LivekitTransportConfig,
} from "matrix-js-sdk/lib/matrixrtc";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { type MatrixClient } from "matrix-js-sdk";
import { distinctUntilChanged, from, map, of, switchMap } from "rxjs";
import { logger as rootLogger, type Logger } from "matrix-js-sdk/lib/logger";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { type Behavior } from "../../Behavior.ts";
import { type Epoch, type ObservableScope } from "../../ObservableScope.ts";
import { Config } from "../../../config/Config.ts";
import {
FailToGetOpenIdToken,
@@ -27,19 +21,12 @@ import {
type SFUConfig,
type OpenIDClientParts,
} from "../../../livekit/openIDSFU.ts";
import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers.ts";
import { customLivekitUrl } from "../../../settings/settings.ts";
import { RtcTransportAutoDiscovery } from "./RtcTransportAutoDiscovery.ts";
import { type MatrixRTCMode } from "../../../config/ConfigOptions.ts";
/*
* It figures out “which LiveKit focus URL/alias the local user should use,”
* and ensures the SFU path is primed before advertising that choice.
*/
interface Props {
scope: ObservableScope;
ownMembershipIdentity: CallMembershipIdentityParts;
memberships$: Behavior<Epoch<CallMembership[]>>;
client: Pick<MatrixClient, "getDomain" | "_unstable_getRTCTransports"> &
OpenIDClientParts;
// Used by the jwt service to create the livekit room and compute the livekit alias.
@@ -47,135 +34,65 @@ interface Props {
matrixRTCMode: MatrixRTCMode;
}
// TODO livekit_alias-cleanup
// 1. We need to move away from transports map to connections!!!
//
// 2. We need to stop sending livekit_alias all together
//
//
// 1.
// Transports are just the jwt service adress but do not contain the information which room on this transport to use.
// That requires slot and roomId.
//
// We need one connection per room on the transport.
//
// We need an object that contains:
// transport
// roomId
// slotId
//
// To map to the connections. Prosposal: `ConnectionIdentifier`
//
// 2.
// We need to make sure we do not sent livekit_alias in sticky events and that we drop all code for sending state events!
export interface LocalTransportWithSFUConfig {
transport: LivekitTransportConfig;
export interface LocalTransport {
transport: LivekitTransport;
sfuConfig: SFUConfig;
}
export function isLocalTransportWithSFUConfig(
obj: LivekitTransportConfig | LocalTransportWithSFUConfig,
): obj is LocalTransportWithSFUConfig {
export function isLocalTransport(
obj: LivekitTransport | LocalTransport,
): obj is LocalTransport {
return "transport" in obj && "sfuConfig" in obj;
}
export interface LocalTransport {
/**
* The transport to be advertised in our MatrixRTC membership. `null` when not
* yet fetched/validated.
*/
advertised$: Behavior<LivekitTransportConfig | null>;
/**
* The transport to connect to and publish media on. `null` when not yet known
* or available.
*/
active$: Behavior<LocalTransportWithSFUConfig | null>;
}
/**
* Connects to the JWT service and determines the transports that the local member should use.
* Connects to the JWT service and determines the transport that the local member should use.
*
* @prop useOldJwtEndpoint Whether to set forceOldJwtEndpoint on the returned transport and to use the old JWT endpoint.
* This is used when the connection manager needs to know if it has to use the legacy endpoint which implies a string concatenated rtcBackendIdentity.
* (which is expected for non sticky event based rtc member events)
* @returns The transport to advertise in the local MatrixRTC membership, along with the transport to actively publish media to.
* @returns The transport to advertise in our MatrixRTC membership and publish media on.
* @throws MatrixRTCTransportMissingError | FailToGetOpenIdToken
*/
export const createLocalTransport$ = ({
scope,
memberships$,
export async function getLocalTransport({
ownMembershipIdentity,
client,
roomId,
matrixRTCMode,
}: Props): LocalTransport => {
}: Props): Promise<LocalTransport> {
const logger = rootLogger.getChild("[LocalTransport]");
const transportDiscovery = new RtcTransportAutoDiscovery({
const discovery = new RtcTransportAutoDiscovery({
client: client,
resolvedConfig: Config.get(),
logger: logger,
});
const customUrl = customLivekitUrl.value$.value;
// Get the preferred transport from the current deployment.
const discoveredTransport$ = from(
transportDiscovery.discoverPreferredTransport(),
);
// Respect the user's custom URL, if set
const transport: LivekitTransport | null = customUrl
? { type: "livekit", livekit_service_url: customUrl }
: await discovery.discoverPreferredTransport();
const preferredConfig$ = customLivekitUrl.value$.pipe(
switchMap((customUrl) => {
if (customUrl) {
return of({
type: "livekit",
livekit_service_url: customUrl,
} as LivekitTransportConfig);
} else {
return discoveredTransport$;
}
}),
map((config) => {
if (!config) {
// Bubbled up from the preferredConfig$ observable.
throw new MatrixRTCTransportMissingError(client.getDomain() ?? "");
}
return config;
}),
distinctUntilChanged(areLivekitTransportsEqual),
);
if (transport === null)
throw new MatrixRTCTransportMissingError(client.getDomain() ?? "");
const preferredTransport$ = preferredConfig$.pipe(
switchMap(async (transport) => {
try {
return await doOpenIdAndJWTFromUrl(
transport,
matrixRTCMode,
ownMembershipIdentity,
roomId,
client,
logger,
);
} catch (e) {
logger.error(
`Failed to authenticate to transport ${transport.livekit_service_url}`,
e,
);
throw mapAuthErrorToUserFriendlyError(e);
}
}),
);
// Always publish on and advertise the preferred transport.
return {
advertised$: scope.behavior(
preferredTransport$.pipe(
map((t) => t.transport),
distinctUntilChanged(areLivekitTransportsEqual),
),
null,
),
active$: scope.behavior(preferredTransport$, null),
};
};
try {
return await doOpenIdAndJWTFromUrl(
transport,
matrixRTCMode,
ownMembershipIdentity,
roomId,
client,
logger,
);
} catch (e) {
logger.error(
`Failed to authenticate to transport ${transport.livekit_service_url}`,
e,
);
throw mapAuthErrorToUserFriendlyError(e);
}
}
/**
* Utility to ensure the user can authenticate with the SFU.
@@ -193,13 +110,13 @@ export const createLocalTransport$ = ({
* @throws FailToGetOpenIdToken, NoMatrix2AuthorizationService
*/
async function doOpenIdAndJWTFromUrl(
transport: LivekitTransportConfig,
transport: LivekitTransport,
matrixRTCMode: MatrixRTCMode,
membership: CallMembershipIdentityParts,
roomId: string,
client: Pick<MatrixClient, "_unstable_getRTCTransports"> & OpenIDClientParts,
logger?: Logger,
): Promise<LocalTransportWithSFUConfig> {
): Promise<LocalTransport> {
const sfuConfig = await getSFUConfigWithOpenID(
client,
membership,
@@ -16,7 +16,7 @@ import {
import { MatrixError } from "matrix-js-sdk";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import {
type LivekitTransportConfig,
type LivekitTransport,
type Transport,
} from "matrix-js-sdk/lib/matrixrtc";
@@ -28,12 +28,12 @@ import {
type DiscoveryClient = RtcTransportAutoDiscoveryProps["client"];
const backendTransport: LivekitTransportConfig = {
const backendTransport: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://backend.example.org",
};
const configTransport: LivekitTransportConfig = {
const configTransport: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://config.example.org",
};
@@ -5,8 +5,8 @@ SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
isLivekitTransportConfig,
type LivekitTransportConfig,
isLivekitTransport,
type LivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
import { type MatrixClient } from "matrix-js-sdk";
import { type Logger } from "matrix-js-sdk/lib/logger";
@@ -40,7 +40,7 @@ export class RtcTransportAutoDiscovery {
this.logger = logger.getChild("[RtcTransportAutoDiscovery]");
}
public async discoverPreferredTransport(): Promise<LivekitTransportConfig | null> {
public async discoverPreferredTransport(): Promise<LivekitTransport | null> {
// 1) backend transports
const backendTransport = await this.tryBackendTransports();
if (backendTransport) {
@@ -67,7 +67,7 @@ export class RtcTransportAutoDiscovery {
* This will not throw errors, but instead just log them and return null if the expected config is not found or malformed.
* @private
*/
private async tryBackendTransports(): Promise<LivekitTransportConfig | null> {
private async tryBackendTransports(): Promise<LivekitTransport | null> {
const client = this.client;
// MSC4143: Attempt to fetch transports from backend.
this.logger.info("First try to use getRTCTransports end point ...");
@@ -75,7 +75,7 @@ export class RtcTransportAutoDiscovery {
const transportList = await doNetworkOperationWithRetry(async () =>
client._unstable_getRTCTransports(),
);
const first = transportList.find(isLivekitTransportConfig);
const first = transportList.find(isLivekitTransport);
if (first) {
return first;
} else {
@@ -90,7 +90,7 @@ export class RtcTransportAutoDiscovery {
return null;
}
private tryConfigTransport(): LivekitTransportConfig | null {
private tryConfigTransport(): LivekitTransport | null {
const url = this.resolvedConfig.livekit?.livekit_service_url;
if (url) {
return {
@@ -30,7 +30,7 @@ import fetchMock from "fetch-mock";
import EventEmitter from "events";
import { type IOpenIDToken } from "matrix-js-sdk";
import { logger, type Logger } from "matrix-js-sdk/lib/logger";
import { type LivekitTransportConfig } from "matrix-js-sdk/lib/matrixrtc";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import {
Connection,
@@ -57,7 +57,7 @@ let fakeLocalParticipant: MockedObject<LocalParticipant>;
const ROOM_ID = "!roomID:example.org";
const livekitFocus: LivekitTransportConfig = {
const livekitFocus: LivekitTransport = {
livekit_service_url: "https://matrix-rtc.example.org/livekit/jwt",
type: "livekit",
};
@@ -228,9 +228,7 @@ describe("Start connection states", () => {
capturedState = capturedStates.pop();
if (capturedState instanceof Error) {
expect(capturedState.message).toEqual("Something went wrong");
expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias,
);
expect(connection.transport).toEqual(livekitFocus);
} else {
expect.fail("Expected FailedToStart state but got " + capturedState);
}
@@ -290,9 +288,7 @@ describe("Start connection states", () => {
expect(capturedState.cause.message).toContain(
"Failed to look up user info from homeserver",
);
expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias,
);
expect(connection.transport).toEqual(livekitFocus);
} else {
expect.fail("Expected FailedToStart state but got " + capturedState);
}
@@ -357,9 +353,7 @@ describe("Start connection states", () => {
expect(capturedState.cause.message).toContain(
"Failed to connect to livekit",
);
expect(connection.transport.livekit_alias).toEqual(
livekitFocus.livekit_alias,
);
expect(connection.transport).toEqual(livekitFocus);
} else {
expect.fail(
"Expected FailedToStart state but got " + JSON.stringify(capturedState),
@@ -22,7 +22,7 @@ import {
type Track,
type TrackPublication,
} from "livekit-client";
import { type LivekitTransportConfig } from "matrix-js-sdk/lib/matrixrtc";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, map } from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
@@ -52,7 +52,7 @@ export interface ConnectionOpts {
/** The identity parts to use on this connection */
ownMembershipIdentity: CallMembershipIdentityParts;
/** The media transport to connect to. */
transport: LivekitTransportConfig;
transport: LivekitTransport;
/** The Matrix client to use for OpenID and SFU config requests. */
client: OpenIDClientParts;
/** The room ID this connection is associated with. */
@@ -107,7 +107,7 @@ export class Connection {
/**
* The media transport to connect to.
*/
public readonly transport: LivekitTransportConfig;
public readonly transport: LivekitTransport;
public readonly livekitRoom: LivekitRoom;
@@ -16,7 +16,7 @@ import { logger, type Logger } from "matrix-js-sdk/lib/logger";
// imported as inline to support worker when loaded from a cdn (cross domain)
import E2EEWorker from "livekit-client/e2ee-worker?worker&inline";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
import { type LivekitTransportConfig } from "matrix-js-sdk/lib/matrixrtc";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { type ObservableScope } from "../../ObservableScope.ts";
import { Connection } from "./Connection.ts";
@@ -44,7 +44,7 @@ import {
export interface ConnectionFactory {
createConnection(
scope: ObservableScope,
transport: LivekitTransportConfig,
transport: LivekitTransport,
ownMembershipIdentity: CallMembershipIdentityParts,
logger: Logger,
sfuConfig?: SFUConfig,
@@ -103,7 +103,7 @@ export class ECConnectionFactory implements ConnectionFactory {
*/
public createConnection(
scope: ObservableScope,
transport: LivekitTransportConfig,
transport: LivekitTransport,
ownMembershipIdentity: CallMembershipIdentityParts,
logger: Logger,
sfuConfig?: SFUConfig,
@@ -6,8 +6,8 @@ Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { BehaviorSubject } from "rxjs";
import { type LivekitTransportConfig } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, NEVER } from "rxjs";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { type RemoteParticipant } from "livekit-client";
import { logger } from "matrix-js-sdk/lib/logger";
@@ -20,16 +20,16 @@ import { type ConnectionFactory } from "./ConnectionFactory.ts";
import { type Connection } from "./Connection.ts";
import { ownMemberMock, withTestScheduler } from "../../../utils/test.ts";
import { areLivekitTransportsEqual } from "./MatrixLivekitMembers.ts";
import { constant, type Behavior } from "../../Behavior.ts";
import { type Behavior } from "../../Behavior.ts";
// Some test constants
const TRANSPORT_1: LivekitTransportConfig = {
const TRANSPORT_1: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
};
const TRANSPORT_2: LivekitTransportConfig = {
const TRANSPORT_2: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.sample.com",
};
@@ -47,7 +47,7 @@ beforeEach(() => {
vi.mocked(fakeConnectionFactory).createConnection = vi
.fn()
.mockImplementation(
(scope: ObservableScope, transport: LivekitTransportConfig) => {
(scope: ObservableScope, transport: LivekitTransport) => {
const mockConnection = {
transport,
remoteParticipants$: new BehaviorSubject([]),
@@ -74,7 +74,7 @@ describe("connections$ stream", () => {
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
localTransport$: constant(null),
localTransport$: NEVER,
remoteTransports$: behavior("a", {
a: new Epoch([TRANSPORT_1, TRANSPORT_2], 0),
}),
@@ -115,7 +115,7 @@ describe("connections$ stream", () => {
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
localTransport$: constant(null),
localTransport$: NEVER,
remoteTransports$: behavior("abcdef", {
a: new Epoch([TRANSPORT_1], 0),
b: new Epoch([TRANSPORT_1], 1),
@@ -162,7 +162,7 @@ describe("connections$ stream", () => {
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
localTransport$: constant(null),
localTransport$: NEVER,
remoteTransports$: behavior("abc", {
a: new Epoch([TRANSPORT_1], 0),
b: new Epoch([TRANSPORT_1, TRANSPORT_2], 1),
@@ -225,7 +225,7 @@ describe("connectionManagerData$ stream", () => {
// Used in test to control fake connections' remoteParticipants$ streams
let fakeRemoteParticipantsStreams: Map<string, Behavior<RemoteParticipant[]>>;
function keyForTransport(transport: LivekitTransportConfig): string {
function keyForTransport(transport: LivekitTransport): string {
return `${transport.livekit_service_url}`;
}
@@ -233,7 +233,7 @@ describe("connectionManagerData$ stream", () => {
fakeRemoteParticipantsStreams = new Map();
function getRemoteParticipantsFor(
transport: LivekitTransportConfig,
transport: LivekitTransport,
): Behavior<RemoteParticipant[]> {
return (
fakeRemoteParticipantsStreams.get(keyForTransport(transport)) ??
@@ -245,7 +245,7 @@ describe("connectionManagerData$ stream", () => {
vi.mocked(fakeConnectionFactory).createConnection = vi
.fn()
.mockImplementation(
(scope: ObservableScope, transport: LivekitTransportConfig) => {
(scope: ObservableScope, transport: LivekitTransport) => {
const fakeRemoteParticipants$ = new BehaviorSubject<
RemoteParticipant[]
>([]);
@@ -297,7 +297,7 @@ describe("connectionManagerData$ stream", () => {
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
localTransport$: constant(null),
localTransport$: NEVER,
remoteTransports$: behavior("a", {
a: new Epoch([TRANSPORT_1, TRANSPORT_2], 0),
}),
@@ -6,8 +6,17 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type LivekitTransportConfig } from "matrix-js-sdk/lib/matrixrtc";
import { combineLatest, map, of, switchMap } from "rxjs";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import {
combineLatest,
map,
type Observable,
of,
switchMap,
startWith,
catchError,
NEVER,
} from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { type RemoteParticipant } from "livekit-client";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
@@ -19,8 +28,8 @@ import { generateItemsWithEpoch } from "../../../utils/observable.ts";
import { areLivekitTransportsEqual } from "./MatrixLivekitMembers.ts";
import { type ConnectionFactory } from "./ConnectionFactory.ts";
import {
isLocalTransportWithSFUConfig,
type LocalTransportWithSFUConfig,
isLocalTransport,
type LocalTransport,
} from "../localMember/LocalTransport.ts";
import { type SFUConfig } from "../../../livekit/openIDSFU.ts";
@@ -47,7 +56,7 @@ export class ConnectionManagerData {
}
}
private getKey(transport: LivekitTransportConfig): string {
private getKey(transport: LivekitTransport): string {
// This is enough as a key because the ConnectionManager is already scoped by room.
// We also do not need to consider the slotId at this point since each `MatrixRTCSession` is already scoped by `slotDescription: {id, application}`.
return transport.livekit_service_url;
@@ -58,13 +67,13 @@ export class ConnectionManagerData {
}
public getConnectionForTransport(
transport: LivekitTransportConfig,
transport: LivekitTransport,
): Connection | null {
return this.store.get(this.getKey(transport))?.connection ?? null;
}
public getParticipantsForTransport(
transport: LivekitTransportConfig,
transport: LivekitTransport,
): RemoteParticipant[] {
const key = this.getKey(transport);
const existing = this.store.get(key);
@@ -78,8 +87,8 @@ export class ConnectionManagerData {
interface Props {
scope: ObservableScope;
connectionFactory: ConnectionFactory;
localTransport$: Behavior<LocalTransportWithSFUConfig | null>;
remoteTransports$: Behavior<Epoch<LivekitTransportConfig[]>>;
localTransport$: Observable<LocalTransport>;
remoteTransports$: Behavior<Epoch<LivekitTransport[]>>;
logger: Logger;
ownMembershipIdentity: CallMembershipIdentityParts;
@@ -121,6 +130,14 @@ export function createConnectionManager$({
const logger = parentLogger.getChild("[ConnectionManager]");
// TODO logger: only construct one logger from the client and make it compatible via a EC specific sing
const localTransportAsArray$ = localTransport$.pipe(
// LocalMember already surfaces local transport errors properly in the UI,
// here we can just swallow them
catchError(() => NEVER),
map((transport) => [transport]),
startWith([]),
);
/**
* All transports currently managed by the ConnectionManager.
*
@@ -130,16 +147,12 @@ export function createConnectionManager$({
* externally this is modified via `registerTransports()`.
*/
const localAndRemoteTransports$: Behavior<
Epoch<(LivekitTransportConfig | LocalTransportWithSFUConfig)[]>
Epoch<(LivekitTransport | LocalTransport)[]>
> = scope.behavior(
combineLatest([remoteTransports$, localTransport$]).pipe(
combineLatest([localTransportAsArray$, remoteTransports$]).pipe(
// Combine local and remote transports into one transport array
// and set the forceOldJwtEndpoint property on the local transport
map(([remoteTransports, localTransport]) => {
let localTransportAsArray: LocalTransportWithSFUConfig[] = [];
if (localTransport) {
localTransportAsArray = [localTransport];
}
map(([localTransportAsArray, remoteTransports]) => {
const dedupedRemote = removeDuplicateTransports(remoteTransports.value);
const remoteWithoutLocal = dedupedRemote.filter(
(transport) =>
@@ -170,8 +183,9 @@ export function createConnectionManager$({
"ConnectionManager connections$",
function* (transports) {
for (const transport of transports) {
if (isLocalTransportWithSFUConfig(transport)) {
// This is the local transport; only the `LocalTransportWithSFUConfig` has a `sfuConfig` field.
if (isLocalTransport(transport)) {
// This is the local transport; only the `LocalTransport`
// interface has a `sfuConfig` field.
yield {
keys: [
transport.transport.livekit_service_url,
@@ -257,7 +271,7 @@ export function createConnectionManager$({
return { connectionManagerData$ };
}
function removeDuplicateTransports<T extends LivekitTransportConfig>(
function removeDuplicateTransports<T extends LivekitTransport>(
transports: T[],
): T[] {
return transports.reduce((acc, transport) => {
@@ -42,13 +42,11 @@ const fallbackMemberId = (userId: string, deviceId: string): string =>
const transportA: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
livekit_alias: "!alias:example.org",
};
const transportB: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.sample.com",
livekit_alias: "!alias:sample.com",
};
const bobMembership = mockRtcMembership("@bob:example.org", "DEV000", {
@@ -126,7 +124,7 @@ function createEpochedMemberships$(m$: Observable<CallMembership[]>): {
const membershipsWithTransport$ = memberships$.pipe(
mapEpoch((members) => {
return members.map((m) => {
const tr = m.getTransport(m);
const tr = m.getTransport();
return {
membership: m,
transport:
@@ -152,7 +150,7 @@ test("should signal participant on a connection that is publishing", async () =>
);
const connection = {
transport: bobMembership.getTransport(bobMembership),
transport: bobMembership.getTransport(),
} as unknown as Connection;
const dataWithPublisher = new ConnectionManagerData();
dataWithPublisher.add(connection, [
@@ -195,7 +193,7 @@ test("should signal participant on a connection that is not publishing", async (
);
const connection = {
transport: bobMembership.getTransport(bobMembership),
transport: bobMembership.getTransport(),
} as unknown as Connection;
const dataWithPublisher = new ConnectionManagerData();
dataWithPublisher.add(connection, []);
@@ -8,7 +8,7 @@ Please see LICENSE in the repository root for full details.
import { type LocalParticipant, type RemoteParticipant } from "livekit-client";
import {
type CallMembership,
type LivekitTransportConfig,
type LivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
import { combineLatest, filter, map } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
@@ -60,7 +60,7 @@ export interface RemoteMatrixLivekitMember extends MatrixLivekitMember {
interface Props {
scope: ObservableScope;
membershipsWithTransport$: Behavior<
Epoch<{ membership: CallMembership; transport?: LivekitTransportConfig }[]>
Epoch<{ membership: CallMembership; transport?: LivekitTransport }[]>
>;
connectionManager: IConnectionManager;
localUser: { deviceId: string; userId: string };
@@ -165,7 +165,7 @@ export function createRemoteMatrixLivekitMembers$({
// TODO add back in the callviewmodel pauseWhen(this.pretendToBeDisconnected$)
// TODO add this to the JS-SDK
export function areLivekitTransportsEqual<T extends LivekitTransportConfig>(
export function areLivekitTransportsEqual<T extends LivekitTransport>(
t1: T | null,
t2: T | null,
): boolean {
@@ -6,11 +6,11 @@ Please see LICENSE in the repository root for full details.
*/
import { test, vi, expect, beforeEach, afterEach } from "vitest";
import { BehaviorSubject } from "rxjs";
import { BehaviorSubject, NEVER } from "rxjs";
import { type Room as LivekitRoom } from "livekit-client";
import EventEmitter from "events";
import fetchMock from "fetch-mock";
import { type LivekitTransportConfig } from "matrix-js-sdk/lib/matrixrtc";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { logger } from "matrix-js-sdk/lib/logger";
import {
@@ -34,7 +34,6 @@ import {
} from "./MatrixLivekitMembers.ts";
import { createConnectionManager$ } from "./ConnectionManager.ts";
import { membershipsAndTransports$ } from "../../SessionBehaviors.ts";
import { constant } from "../../Behavior.ts";
import { localRtcMember, testJWTToken } from "../../../utils/test-fixtures.ts";
// Test the integration of ConnectionManager and MatrixLivekitMerger
@@ -124,7 +123,7 @@ test("bob, carl, then bob joining no tracks yet", () => {
const connectionManager = createConnectionManager$({
scope: testScope,
connectionFactory: ecConnectionFactory,
localTransport$: constant(null),
localTransport$: NEVER,
remoteTransports$: membershipsAndTransports.transports$,
logger: logger,
ownMembershipIdentity: ownMemberMock,
@@ -150,7 +149,7 @@ test("bob, carl, then bob joining no tracks yet", () => {
a: expect.toSatisfy((co) =>
areLivekitTransportsEqual(
co.transport,
bobMembership.transports[0]! as LivekitTransportConfig,
bobMembership.transports[0]! as LivekitTransport,
),
),
});
@@ -187,7 +186,7 @@ test("bob, carl, then bob joining no tracks yet", () => {
expect(
areLivekitTransportsEqual(
connection.transport,
carlMembership.transports[0]! as LivekitTransportConfig,
carlMembership.transports[0]! as LivekitTransport,
),
).toBe(true);
return true;
@@ -217,7 +216,7 @@ test("bob, carl, then bob joining no tracks yet", () => {
expect(
areLivekitTransportsEqual(
connection.transport,
daveMembership.transports[0]! as LivekitTransportConfig,
daveMembership.transports[0]! as LivekitTransport,
),
).toBe(true);
return true;
+4 -12
View File
@@ -14,7 +14,6 @@ vi.mock("../Platform", () => ({
},
isFirefox: (): boolean => false,
}));
// One observer per device kind, so a test can add and remove hardware.
const observers = vi.hoisted(
() => new Map<string, { next: (devices: unknown[]) => void }>(),
);
@@ -75,7 +74,7 @@ function device(deviceId: string, label: string, groupId = deviceId): object {
return { deviceId, label, groupId, kind: "audioinput" };
}
/** Replaces the hardware of one kind, as the browser would report it. */
/** Replaces the devices of one kind, as the browser reports them. */
function setDevices(kind: string, devices: object[]): void {
const observer = observers.get(kind);
if (observer === undefined) throw new Error(`nothing observing ${kind}`);
@@ -103,7 +102,7 @@ describe("MediaDevices selection", () => {
devices.audioInput.select("mic2");
// A later call on the same machine reads the same stored preference.
// A later call reads the same stored preference.
expect(newMediaDevices().audioInput.selected$.value?.id).toBe("mic2");
});
@@ -112,7 +111,6 @@ describe("MediaDevices selection", () => {
setDevices("audioinput", [device("mic1", "Microphone 1")]);
expect([...devices.audioInput.available$.value.keys()]).toEqual(["mic1"]);
// A headset is plugged in.
setDevices("audioinput", [
device("mic1", "Microphone 1"),
device("mic2", "Headset"),
@@ -122,7 +120,6 @@ describe("MediaDevices selection", () => {
"mic2",
]);
// And unplugged again.
setDevices("audioinput", [device("mic1", "Microphone 1")]);
expect([...devices.audioInput.available$.value.keys()]).toEqual(["mic1"]);
});
@@ -136,7 +133,6 @@ describe("MediaDevices selection", () => {
devices.audioInput.select("mic2");
expect(devices.audioInput.selected$.value?.id).toBe("mic2");
// The headset is unplugged mid-call.
setDevices("audioinput", [device("mic1", "Microphone 1")]);
expect(devices.audioInput.selected$.value?.id).toBe("mic1");
@@ -145,7 +141,6 @@ describe("MediaDevices selection", () => {
test("falls back when the remembered device is absent", () => {
const devices = newMediaDevices();
setDevices("audioinput", [device("mic1", "Microphone 1")]);
// Remembered from a previous call, on hardware that is not here now.
devices.audioInput.select("a-device-from-last-time");
expect(devices.audioInput.selected$.value?.id).toBe("mic1");
@@ -153,7 +148,7 @@ describe("MediaDevices selection", () => {
test("falls back to numbered labels when labels are unavailable", () => {
const devices = newMediaDevices();
// The browser withholds names until permission has been granted.
// Names are withheld until permission is granted.
setDevices("audioinput", [device("mic1", ""), device("mic2", "")]);
expect([...devices.audioInput.available$.value.values()]).toEqual([
@@ -167,10 +162,7 @@ describe("MediaDevices selection", () => {
setDevices("audiooutput", [device("spk1", "Speakers")]);
const available = devices.audioOutput.available$.value;
// Default follows the operating system and re-points when it changes, so
// it is its own choice rather than an alias for the device it resolves to.
// It carries no name of its own precisely because which device it resolves
// to is not knowable from here.
// Default is its own entry, unnamed: which device it resolves to isn't knowable.
expect(available.get("spk1")).toEqual({ type: "name", name: "Speakers" });
expect(available.get("")).toEqual({ type: "default", name: null });
});
+45 -24
View File
@@ -9,7 +9,7 @@ import { afterEach, describe, expect, test, vi } from "vitest";
import {
ATTACK_MS,
METER_SEGMENTS,
LEVEL_SCALE,
observeMicrophoneState$,
RELEASE_MS,
segmentsForVolume,
@@ -23,8 +23,6 @@ describe("segmentsForVolume", () => {
});
test("shows nothing for the hiss of a quiet room", () => {
// Without a noise floor these light the first segments permanently, which
// reads as "it can hear me" when nobody is speaking.
expect(segmentsForVolume(0.005)).toBe(0);
expect(segmentsForVolume(0.015)).toBe(0);
});
@@ -40,14 +38,13 @@ describe("segmentsForVolume", () => {
});
test("moves the meter visibly for normal speech", () => {
// Ordinary speech should reach the middle of the meter, not scrape along
// the floor: a meter that barely moves reads as a broken microphone.
expect(segmentsForVolume(0.2)).toBeGreaterThanOrEqual(METER_SEGMENTS / 4);
// Ordinary speech reaches the middle of the meter.
expect(segmentsForVolume(0.2)).toBeGreaterThanOrEqual(LEVEL_SCALE / 4);
});
test("never exceeds the meter", () => {
expect(segmentsForVolume(1)).toBe(METER_SEGMENTS);
expect(segmentsForVolume(4)).toBe(METER_SEGMENTS);
expect(segmentsForVolume(1)).toBe(LEVEL_SCALE);
expect(segmentsForVolume(4)).toBe(LEVEL_SCALE);
});
test("treats a missing reading as silence", () => {
@@ -65,16 +62,14 @@ describe("smoothVolume", () => {
});
test("registers a syllable as it starts", () => {
// Most of the way there within one attack time constant, so speech does
// not lag the speaker.
// Most of the way within one attack time constant.
expect(smoothVolume(0, 1, ATTACK_MS)).toBeGreaterThan(0.6);
});
test("rides over the gaps between words", () => {
// A pause of a few tens of milliseconds should not collapse the meter, or
// it flickers rather than reading as a level.
// A short pause doesn't collapse the meter...
expect(smoothVolume(1, 0, 30)).toBeGreaterThan(0.7);
// A real silence still brings it down.
// ...but a real silence brings it down.
expect(smoothVolume(1, 0, RELEASE_MS * 3)).toBeLessThan(0.1);
});
@@ -98,8 +93,7 @@ describe("observeMicrophoneState$", () => {
const capture = stubAudioCapture();
const subscription = observeMicrophoneState$("mic1").subscribe();
// The user gives up on the permission prompt and closes the menu, and only
// then does the browser hand the microphone over.
// The menu closes before the browser hands the microphone over.
subscription.unsubscribe();
capture.grant();
await vi.waitFor(() => expect(capture.track.stop).toHaveBeenCalled());
@@ -118,6 +112,30 @@ describe("observeMicrophoneState$", () => {
expect(capture.contexts[0].close).toHaveBeenCalled();
});
test("gives the microphone back when the audio graph fails to build", async () => {
const capture = stubAudioCapture();
// Fails only after getUserMedia has granted the device.
vi.stubGlobal(
"AudioContext",
class {
public constructor() {
throw new Error("no audio context for you");
}
},
);
const seen: string[] = [];
const subscription = observeMicrophoneState$("mic1").subscribe((state) =>
seen.push(state.type),
);
capture.grant();
await vi.waitFor(() => expect(seen).toContain("no-device"));
expect(capture.track.stop).toHaveBeenCalled();
subscription.unsubscribe();
});
test("tells denied permission and a missing device apart", async () => {
for (const [name, expected] of [
["NotAllowedError", "permission-denied"],
@@ -139,23 +157,26 @@ describe("observeMicrophoneState$", () => {
test("says nothing on a frame that did not change the level", async () => {
const capture = stubAudioCapture();
let emissions = 0;
const subscription = observeMicrophoneState$("mic1").subscribe(
() => emissions++,
);
const states: string[] = [];
let levels = 0;
const subscription = observeMicrophoneState$("mic1").subscribe((state) => {
states.push(state.type);
if (state.type === "level") state.level$.subscribe(() => levels++);
});
capture.grant();
await vi.waitFor(() => expect(emissions).toBe(1));
await vi.waitFor(() => expect(levels).toBe(1));
// The analyser is read every animation frame, but the meter has only
// METER_SEGMENTS steps: a steady signal must not redraw the meter.
// Read every frame, but a steady signal emits once, and the state itself
// arrives only once.
capture.drawFrames(20);
expect(emissions).toBe(1);
expect(levels).toBe(1);
expect(states).toEqual(["level"]);
subscription.unsubscribe();
});
});
/** An error with the `name` the browser would give it, not just a message. */
/** An error carrying the browser's `name`. */
function named(error: Error, name: string): Error {
error.name = name;
return error;
+37 -86
View File
@@ -5,42 +5,25 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { distinctUntilChanged, Observable } from "rxjs";
import { BehaviorSubject, Observable } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
/**
* What the microphone selector can say about the input, beyond its level.
*
* Silence and a broken microphone look identical on a meter, so the states a
* user has to act on are named rather than drawn as a flat bar.
*/
import { type Behavior } from "./Behavior";
/** What the microphone picks up, or why it can't be read. */
export type MicrophoneState =
| { type: "level"; level: number }
// A Behavior, so a changing level can be drawn without re-rendering.
| { type: "level"; level$: Behavior<number> }
| { type: "permission-denied" }
| { type: "no-device" };
/**
* Steps the meter is drawn in. The level is quantised to these.
*
* Enough of them that they sit close together across the width of the menu:
* the bars keep a fixed size, so too few leaves visible gaps between them.
*/
export const METER_SEGMENTS = 24;
/** Scale a level is reported on. Fixed rather than the bar count, so a level reads the same at any width. */
export const LEVEL_SCALE = 24;
/**
* Observes what a microphone is picking up, as the meter should show it.
*
* Subscribing opens the device and unsubscribing releases it, so it is held for
* exactly as long as something is watching — the menu, and not the call.
*
* Opens a capture of its own rather than reading the track the call already
* holds. That is a shortcut, recorded as S1 in the feature spec: the pre-join
* screen freezes its audio track to the device selected when it mounted, so a
* meter fed from that track could not follow the picker.
*
* The level says whether the microphone is picking anything up, which is not
* the same as whether the user is being heard: it keeps reading while muted,
* and the mute control is what says nothing is transmitted.
* The level of a microphone, captured while subscribed. Its own capture rather
* than the call's track, because pre-join freezes that track to the device
* selected at mount.
*/
export function observeMicrophoneState$(
deviceId: string | undefined,
@@ -49,9 +32,9 @@ export function observeMicrophoneState$(
let stream: MediaStream | undefined;
let context: AudioContext | undefined;
let frame: number | undefined;
let level: BehaviorSubject<number> | undefined;
// Safe to call more than once: unsubscribing runs it, and `start` runs it
// again for anything the browser handed over after that.
// Idempotent: teardown and start can both call it.
const release = (): void => {
if (frame !== undefined) cancelAnimationFrame(frame);
stream?.getTracks().forEach((track) => track.stop());
@@ -59,6 +42,8 @@ export function observeMicrophoneState$(
frame = undefined;
stream = undefined;
context = undefined;
level?.complete();
level = undefined;
};
const start = async (): Promise<void> => {
@@ -66,16 +51,11 @@ export function observeMicrophoneState$(
audio:
deviceId === undefined ? true : { deviceId: { exact: deviceId } },
});
// A permission prompt outlives the subscription that asked for it, so by
// now nobody may be watching — and `release` ran while `stream` was still
// undefined. Nothing will call it again, so release here or the device
// stays held, with the indicator lit and no meter on screen.
// Unsubscribed while the permission prompt was open.
if (subscriber.closed) return release();
context = new AudioContext();
// Chrome starts the context suspended unless it was created during a
// gesture; opening the menu is one, but resume explicitly so the meter
// cannot silently sit at zero.
// Starts suspended outside a user gesture, which would read as silence.
if (context.state === "suspended") await context.resume();
if (subscriber.closed) return release();
@@ -85,11 +65,13 @@ export function observeMicrophoneState$(
const samples = new Uint8Array(analyser.fftSize);
let displayed = 0;
let previousFrame = performance.now();
const current = new BehaviorSubject(0);
level = current;
subscriber.next({ type: "level", level$: current });
const read = (): void => {
analyser.getByteTimeDomainData(samples);
// Root mean square of the waveform around its centre, which is the
// loudness a listener perceives rather than the tallest spike.
// RMS: perceived loudness rather than the peak.
let sum = 0;
for (const sample of samples) {
const centred = (sample - 128) / 128;
@@ -102,24 +84,22 @@ export function observeMicrophoneState$(
now - previousFrame,
);
previousFrame = now;
subscriber.next({ type: "level", level: segmentsForVolume(displayed) });
// Frames that don't move the quantised level say nothing.
const next = segmentsForVolume(displayed);
if (next !== current.value) current.next(next);
frame = requestAnimationFrame(read);
};
read();
};
start().catch((e: unknown) => subscriber.next(stateForFailure(e)));
start().catch((e: unknown) => {
// Building the graph can fail after the device was granted.
release();
subscriber.next(stateForFailure(e));
});
return release;
}).pipe(
// Read every animation frame, but quantised to a whole number of segments,
// so most frames say nothing new and should not reach React.
distinctUntilChanged(
(a, b) =>
a.type === b.type &&
(a.type !== "level" || b.type !== "level" || a.level === b.level),
),
);
});
}
/** What a failure to open the microphone means for the person using it. */
@@ -133,53 +113,24 @@ function stateForFailure(e: unknown): MicrophoneState {
return { type: "no-device" };
}
/**
* Loudness below which the microphone is treated as picking up nothing.
*
* A quiet room is never digitally silent, and without a floor that hiss lights
* the first segments permanently — which reads as "it can hear me" when nobody
* is speaking.
*/
/** Below this counts as silence, so a quiet room's hiss doesn't light the first bars. */
const NOISE_FLOOR = 0.02;
/**
* Quantises a 0..1 volume to a whole number of meter segments.
*
* Exported for the tests: the mapping from loudness to segments is the part
* that decides whether quiet, normal and loud speech look different.
*/
/** Quantises a 0..1 volume onto {@link LEVEL_SCALE}. */
export function segmentsForVolume(volume: number): number {
if (!Number.isFinite(volume) || volume <= NOISE_FLOOR) return 0;
// Volume arrives as amplitude, where speech occupies a small part of the top
// of the range. A square root spreads that out, so ordinary speech moves the
// meter through its middle rather than barely leaving the floor.
// Square root, so ordinary speech reaches the middle of the scale.
const aboveFloor = (Math.min(volume, 1) - NOISE_FLOOR) / (1 - NOISE_FLOOR);
return Math.min(
METER_SEGMENTS,
Math.ceil(Math.sqrt(aboveFloor) * METER_SEGMENTS),
);
return Math.min(LEVEL_SCALE, Math.ceil(Math.sqrt(aboveFloor) * LEVEL_SCALE));
}
/**
* How quickly the meter follows a rise in loudness, as a time constant in
* milliseconds. Short, so a syllable registers the moment it starts.
*/
/** Rise time constant: short, so a syllable registers as it starts. */
export const ATTACK_MS = 50;
/**
* How quickly the meter follows a fall. Longer than the attack: speech is full
* of gaps a few tens of milliseconds long, and a meter that tracked them
* exactly would flicker rather than read as a level.
*/
/** Fall time constant: longer, so the gaps between words don't flicker. */
export const RELEASE_MS = 120;
/**
* Moves a displayed level towards a new reading, fast upwards and slowly
* downwards.
*
* Framed in elapsed time rather than frames, so the meter behaves the same on a
* 60Hz and a 120Hz display, and does not jump when a frame is dropped.
*/
/** Eases towards a reading, by elapsed time so the frame rate doesn't matter. */
export function smoothVolume(
displayed: number,
reading: number,
+12 -19
View File
@@ -7,10 +7,10 @@ Please see LICENSE in the repository root for full details.
import {
type CallMembership,
type LivekitTransportConfig,
type LivekitTransport,
type MatrixRTCSession,
MatrixRTCSessionEvent,
isLivekitTransportConfig,
isLivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
import { fromEvent } from "rxjs";
@@ -22,48 +22,41 @@ import {
} from "./ObservableScope";
import { type Behavior } from "./Behavior";
/**
* Tracks the transports used by ourselves, plus all other MatrixRTC session
* members.
*/
export const membershipsAndTransports$ = (
scope: ObservableScope,
memberships$: Behavior<Epoch<CallMembership[]>>,
): {
membershipsWithTransport$: Behavior<
Epoch<{ membership: CallMembership; transport?: LivekitTransportConfig }[]>
Epoch<{ membership: CallMembership; transport?: LivekitTransport }[]>
>;
transports$: Behavior<Epoch<LivekitTransportConfig[]>>;
transports$: Behavior<Epoch<LivekitTransport[]>>;
} => {
/**
* Lists the transports used by ourselves, plus all other MatrixRTC session
* members.
* For completeness this also lists the preferred transport and
* whether we are in multi-SFU mode or sticky events mode.
* `advertisedTransport$` reads these values together, so bundling them avoids inconsistent state or
* excessive updates when using RxJS.
*/
const membershipsWithTransport$: Behavior<
Epoch<
{
membership: CallMembership;
transport: LivekitTransportConfig | undefined;
transport: LivekitTransport | undefined;
}[]
>
> = scope.behavior(
memberships$.pipe(
mapEpoch((memberships) => {
return memberships.map((membership) => {
const oldestMembership = memberships[0] ?? membership;
const transport = membership.getTransport(oldestMembership);
const transport = membership.getTransport();
return {
membership,
transport: isLivekitTransportConfig(transport)
? transport
: undefined,
transport: isLivekitTransport(transport) ? transport : undefined,
};
});
}),
),
);
const transports$: Behavior<Epoch<LivekitTransportConfig[]>> = scope.behavior(
const transports$: Behavior<Epoch<LivekitTransport[]>> = scope.behavior(
membershipsWithTransport$.pipe(
mapEpoch((mts) => mts.flatMap(({ transport: t }) => (t ? [t] : []))),
),
+41
View File
@@ -25,6 +25,7 @@ import {
mockRemoteScreenShare,
} from "../../utils/test";
import { constant } from "../Behavior";
import { showConnectionStats } from "../../settings/settings";
global.MediaStreamTrack = class {} as unknown as {
new (): MediaStreamTrack;
@@ -42,6 +43,12 @@ vi.mock("../../Platform", () => ({
},
}));
const observeRtpStreamStatsMock = vi.hoisted(() => vi.fn());
vi.mock("./observeRtpStreamStats", async (importOriginal) => ({
...(await importOriginal<Record<string, unknown>>()),
observeRtpStreamStats$: observeRtpStreamStatsMock,
}));
const rtcMembership = mockRtcMembership("@alice:example.org", "AAAA");
test("control a participant's volume", () => {
@@ -289,3 +296,37 @@ test("remote media is not in waiting state when user does not intend to publish
);
expect(vm.waitingForMedia$.value).toBe(false);
});
test("user media polls stream stats only while the setting is on", () => {
const participant = mockRemoteParticipant({});
const vm = mockRemoteMedia(rtcMembership, {}, participant);
onTestFinished(() => showConnectionStats.setValue(false));
withTestScheduler(({ cold, expectObservable, schedule }) => {
const stats = { type: "inbound-rtp" } as RTCInboundRtpStreamStats;
observeRtpStreamStatsMock.mockImplementation(() =>
cold("-s", { s: stats }),
);
schedule("-a-b", {
a() {
showConnectionStats.setValue(true);
},
b() {
showConnectionStats.setValue(false);
},
});
expectObservable(vm.audioStreamStats$).toBe("u-su", {
u: undefined,
s: stats,
});
});
expect(observeRtpStreamStatsMock).toHaveBeenCalledWith(
participant,
Track.Source.Microphone,
"inbound-rtp",
);
expect(observeRtpStreamStatsMock).toHaveBeenCalledWith(
participant,
Track.Source.Camera,
"inbound-rtp",
);
});
+19 -22
View File
@@ -10,7 +10,6 @@ import {
BehaviorSubject,
combineLatest,
map,
type Observable,
of,
Subject,
switchMap,
@@ -54,10 +53,10 @@ export interface BaseUserMediaViewModel extends BaseMemberMediaViewModel {
rtcBackendIdentity: string;
handRaised$: Behavior<Date | null>;
reaction$: Behavior<ReactionOption | null>;
audioStreamStats$: Observable<
audioStreamStats$: Behavior<
RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined
>;
videoStreamStats$: Observable<
videoStreamStats$: Behavior<
RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined
>;
/**
@@ -94,6 +93,21 @@ export function createBaseUserMedia(
);
const toggleCropVideo$ = new Subject<void>();
const videoAspectRatio$ = new BehaviorSubject(NaN);
const streamStats$ = (
scope: ObservableScope,
source: Track.Source,
): Behavior<
RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined
> =>
scope.behavior(
combineLatest([participant$, showConnectionStats.value$]).pipe(
switchMap(([p, showConnectionStats]) =>
p && showConnectionStats
? observeRtpStreamStats$(p, source, statsType)
: of(undefined),
),
),
);
return {
...createMemberMedia(scope, {
@@ -130,25 +144,8 @@ export function createBaseUserMedia(
rtcBackendIdentity,
handRaised$,
reaction$,
audioStreamStats$: combineLatest([
participant$,
showConnectionStats.value$,
]).pipe(
switchMap(([p, showConnectionStats]) => {
//
if (!p || !showConnectionStats) return of(undefined);
return observeRtpStreamStats$(p, Track.Source.Microphone, statsType);
}),
),
videoStreamStats$: combineLatest([
participant$,
showConnectionStats.value$,
]).pipe(
switchMap(([p, showConnectionStats]) => {
if (!p || !showConnectionStats) return of(undefined);
return observeRtpStreamStats$(p, Track.Source.Camera, statsType);
}),
),
audioStreamStats$: streamStats$(scope, Track.Source.Microphone),
videoStreamStats$: streamStats$(scope, Track.Source.Camera),
setVideoAspectRatio: (ratio) => videoAspectRatio$.next(ratio),
};
}
+2 -7
View File
@@ -37,7 +37,6 @@ import {
Menu,
Text,
} from "@vector-im/compound-web";
import { useObservableEagerState } from "observable-hooks";
import styles from "./GridTile.module.css";
import { Slider } from "../Slider";
@@ -133,12 +132,8 @@ const UserMediaTileInner: FC<UserMediaTileProps & { menu: ReactNode }> = ({
const { t } = useTranslation();
const video = useBehavior(vm.video$);
const unencryptedWarning = useBehavior(vm.unencryptedWarning$);
const audioStreamStats = useObservableEagerState<
RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined
>(vm.audioStreamStats$);
const videoStreamStats = useObservableEagerState<
RTCInboundRtpStreamStats | RTCOutboundRtpStreamStats | undefined
>(vm.videoStreamStats$);
const audioStreamStats = useBehavior(vm.audioStreamStats$);
const videoStreamStats = useBehavior(vm.videoStreamStats$);
const audioEnabled = useBehavior(vm.audioEnabled$);
const videoEnabled = useBehavior(vm.videoEnabled$);
const speaking = useBehavior(vm.speaking$);
+8 -3
View File
@@ -26,8 +26,7 @@ import {
VolumeOnSolidIcon,
} from "@vector-im/compound-design-tokens/assets/web/icons";
import { animated } from "@react-spring/web";
import { type Observable, map } from "rxjs";
import { useObservableRef } from "observable-hooks";
import { BehaviorSubject, type Observable, map } from "rxjs";
import { useTranslation } from "react-i18next";
import classNames from "classnames";
import { type TrackReferenceOrPlaceholder } from "@livekit/components-core";
@@ -416,7 +415,13 @@ export const SpotlightTile: FC<Props> = ({
}) => {
const { t } = useTranslation();
const rootElement = useRootElement();
const [ourRef, root$] = useObservableRef<HTMLDivElement | null>(null);
const root$ = useInitial(
() => new BehaviorSubject<HTMLDivElement | null>(null),
);
const ourRef = useCallback(
(root: HTMLDivElement | null) => root$.next(root),
[root$],
);
const ref = useMergedRefs(ourRef, theirRef);
const maximised = useBehavior(vm.maximised$);
const background = useBehavior(vm.background$);
+2 -2
View File
@@ -7,7 +7,6 @@ Please see LICENSE in the repository root for full details.
import { logger } from "matrix-js-sdk/lib/logger";
import { useState, useEffect } from "react";
import { useObservableEagerState } from "observable-hooks";
import {
soundEffectVolume as soundEffectVolumeSetting,
@@ -17,6 +16,7 @@ import { useEarpieceAudioConfig, useMediaDevices } from "./MediaDevicesContext";
import { type PrefetchedSounds } from "./soundUtils";
import { useUrlParams } from "./UrlParams";
import * as controls from "./controls";
import { useBehavior } from "./useBehavior";
/**
* Play a sound though a given AudioContext. Will take
@@ -168,7 +168,7 @@ export function useAudioContext<S extends string>(
};
}, [props.sounds, props.latencyHint]);
const audioOutputId = useObservableEagerState(
const audioOutputId = useBehavior(
useMediaDevices().audioOutput.selected$,
)?.id;
const { controlledAudioDevices } = useUrlParams();
+23
View File
@@ -0,0 +1,23 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { act, renderHook } from "@testing-library/react";
import { BehaviorSubject } from "rxjs";
import { describe, expect, test } from "vitest";
import { useBehavior } from "./useBehavior";
describe("useBehavior", () => {
test("reads the current value and follows changes", () => {
const value$ = new BehaviorSubject(1);
const { result } = renderHook(() => useBehavior(value$));
expect(result.current).toBe(1);
act(() => value$.next(2));
expect(result.current).toBe(2);
});
});
+34
View File
@@ -0,0 +1,34 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { renderHook } from "@testing-library/react";
import { describe, expect, test } from "vitest";
import { useValueBehavior } from "./useValueBehavior";
describe("useValueBehavior", () => {
test("holds the value from the first render", () => {
const { result } = renderHook(({ value }) => useValueBehavior(value), {
initialProps: { value: "a" },
});
expect(result.current.value).toBe("a");
});
test("keeps one identity and takes each new value", () => {
const { result, rerender } = renderHook(
({ value }) => useValueBehavior(value),
{ initialProps: { value: "a" } },
);
const value$ = result.current;
const seen: string[] = [];
value$.subscribe((value) => seen.push(value));
rerender({ value: "b" });
expect(result.current).toBe(value$);
expect(seen).toEqual(["a", "b"]);
});
});
+25
View File
@@ -0,0 +1,25 @@
/*
Copyright 2026 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { useEffect } from "react";
import { BehaviorSubject } from "rxjs";
import { type Behavior } from "./state/Behavior";
import { useInitial } from "./useInitial";
/**
* React hook which mirrors a value from the render cycle into a behavior, so
* that a prop or memoised value can be handed to a view model. The behavior
* takes each new value once the render that produced it has committed.
*/
export function useValueBehavior<T>(value: T): Behavior<T> {
const value$ = useInitial(() => new BehaviorSubject(value));
useEffect(() => {
value$.next(value);
}, [value$, value]);
return value$;
}
+1 -22
View File
@@ -9,30 +9,9 @@ import { expect, test } from "vitest";
import { type Observable, of, Subject, switchMap } from "rxjs";
import { withTestScheduler } from "./test";
import { or$, filterBehavior, generateItems, pauseWhen } from "./observable";
import { filterBehavior, generateItems, pauseWhen } from "./observable";
import { type Behavior } from "../state/Behavior";
const yesNo = {
y: true,
n: false,
};
test("or$", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const input1Marbles = "ny--n--";
const input2Marbles = "n-y--n-";
const input3Marbles = "n--y--n";
const outputMarbles = "nyyyyyn";
expectObservable(
or$(
behavior(input1Marbles, yesNo),
behavior(input2Marbles, yesNo),
behavior(input3Marbles, yesNo),
),
).toBe(outputMarbles, yesNo);
});
});
test("pauseWhen", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const inputMarbles = " abcdefgh-i-jk-";
-9
View File
@@ -8,7 +8,6 @@ Please see LICENSE in the repository root for full details.
import {
type Observable,
audit,
combineLatest,
concat,
defer,
filter,
@@ -113,14 +112,6 @@ export function getValue<T>(state$: Observable<T>): T {
return value;
}
/**
* Creates an Observable that has a value of true whenever some of its inputs
* are true.
*/
export function or$(...inputs: Observable<boolean>[]): Observable<boolean> {
return combineLatest(inputs, (...flags) => flags.some((flag) => flag));
}
/**
* RxJS operator that pauses all changes in the input value whenever a Behavior
* is true. When the Behavior returns to being false, the most recently
+19 -14
View File
@@ -212,7 +212,6 @@ export function mockEmitter<T>(): EmitterMock<T> {
export const exampleTransport: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
livekit_alias: "!alias:example.org",
};
export const exampleSfuConfig: SFUConfig = {
@@ -594,21 +593,19 @@ export class MockConnection extends Connection {
export interface StubbedCapture {
getUserMedia: Mock;
/** Hands the microphone over, as the browser does once permission is given. */
/** Grants the microphone, as the browser does once permission is given. */
grant: () => void;
track: { stop: Mock };
contexts: { close: Mock }[];
/** Runs the animation frames the level meter reads on, in order. */
/** Runs the pending animation frames, in order. */
drawFrames: (count: number) => void;
/** Sets the microphone's loudness from the next frame, 0 to 1. */
speak: (amplitude: number) => void;
}
/**
* Stubs just enough of the capture and Web Audio APIs for a microphone level to
* be read, with the grant held back so a test decides when — or whether — it
* lands, and with animation frames driven by hand rather than by a clock.
*
* Call {@link restoreAudioCapture} afterwards, or every later test in the run
* inherits the stub.
* Stubs the capture and Web Audio APIs, with the grant and the animation
* frames driven by the test. Undo with {@link restoreAudioCapture}.
*/
export function stubAudioCapture(): StubbedCapture {
const track = { stop: vi.fn() };
@@ -620,6 +617,7 @@ export function stubAudioCapture(): StubbedCapture {
});
const frames: FrameRequestCallback[] = [];
let amplitude = 0;
vi.stubGlobal("requestAnimationFrame", (callback: FrameRequestCallback) => {
frames.push(callback);
return frames.length;
@@ -636,10 +634,15 @@ export function stubAudioCapture(): StubbedCapture {
public createAnalyser(): object {
return {
fftSize: 1024,
// Digital silence, which is the midpoint of the range and not zero:
// a buffer left at zero reads as a full-scale waveform.
getByteTimeDomainData: (samples: Uint8Array): void => {
samples.fill(128);
// Silence is the midpoint of the range; a zeroed buffer reads as full scale.
if (amplitude <= 0) {
samples.fill(128);
return;
}
const peak = Math.round(Math.min(1, amplitude) * 127);
for (let i = 0; i < samples.length; i++)
samples[i] = 128 + (i % 2 ? peak : -peak);
},
};
}
@@ -648,8 +651,7 @@ export function stubAudioCapture(): StubbedCapture {
}
},
);
// Only this property: replacing navigator wholesale drops the getters on its
// prototype, such as userAgent.
// Only this property: replacing navigator loses getters like userAgent.
Object.defineProperty(navigator, "mediaDevices", {
configurable: true,
value: { getUserMedia: vi.fn().mockReturnValue(granted) },
@@ -663,6 +665,9 @@ export function stubAudioCapture(): StubbedCapture {
drawFrames: (count): void => {
for (let i = 0; i < count; i++) frames.shift()?.(i);
},
speak: (next): void => {
amplitude = next;
},
};
}
+1 -1
View File
@@ -9,7 +9,7 @@ import { defineConfig, searchForWorkspaceRoot } from "vite";
import { realpathSync } from "node:fs";
import * as fs from "node:fs";
import { vitePluginsConfig } from "./vite.config";
import { vitePluginsConfig } from "./vite.config.ts";
import { scopeStylesToRoot } from "./component/build/scopeStylesToRoot";
// Serves the harness under `component/dev`, which embeds Element Call as a
+14 -3
View File
@@ -7,7 +7,7 @@ Please see LICENSE in the repository root for full details.
import { defineConfig } from "vite";
import { vitePluginsConfig } from "./vite.config";
import { vitePluginsConfig } from "./vite.config.ts";
import { scopeStylesToRoot } from "./component/build/scopeStylesToRoot";
// Config for Element Call as a React component, to be imported by an
@@ -49,8 +49,19 @@ export default defineConfig(({ mode }) => {
cssCodeSplit: false,
lib: {
formats: ["es" as const],
entry: "./component/index.tsx",
fileName: "element-call",
entry: {
// The component
"element-call": "./component/index.tsx",
// What a host needs to talk about it, without it (see component/api.ts).
// Built as an entry point of its own so that importing it does not
// load the component: the bundler gives it a chunk that reaches only
// what it imports.
api: "./component/api.ts",
},
fileName: (_format, entryName) => `${entryName}.js`,
// The one stylesheet (see cssCodeSplit) keeps the name it had when the
// component was the only entry
cssFileName: "element-call",
},
rollupOptions: {
// The host already has these, and a second copy of any of them does not
+1 -1
View File
@@ -8,7 +8,7 @@ Please see LICENSE in the repository root for full details.
import { defineConfig, mergeConfig } from "vite";
import generateFile from "vite-plugin-generate-file";
import fullConfig from "./vite.config";
import fullConfig from "./vite.config.ts";
const base = "./";
+1 -1
View File
@@ -1,6 +1,6 @@
import { defineConfig, mergeConfig } from "vitest/config";
import { playwright } from "@vitest/browser-playwright";
import { vitePluginsConfig } from "./vite.config";
import { vitePluginsConfig } from "./vite.config.ts";
import { storybookTest } from "@storybook/addon-vitest/vitest-plugin";
import path from "node:path";
import { fileURLToPath } from "node:url";