Compare commits

...

77 Commits

Author SHA1 Message Date
Valere
294b90b6dc fix fixture following web change 2025-11-24 11:29:03 +01:00
Timo K
ddd015d696 fix dev EditInPlace 2025-11-24 10:04:01 +01:00
Timo K
7532f72c90 use edit in place (WIP) This is WIP since it will submite -> reload the
page onSave
2025-11-21 18:49:30 +01:00
Timo K
e60bc9e98f add developer tab snapshot test 2025-11-21 17:44:35 +01:00
Timo K
a731981388 Cleanup transport creation and local store hack. 2025-11-21 14:02:15 +01:00
Timo K
4099c4383d move HomeserverConnected 2025-11-21 13:04:28 +01:00
Timo K
d34775fc47 fix linter 2025-11-20 16:40:06 +01:00
Timo K
a29f0162ad remove widget log 2025-11-20 16:34:53 +01:00
Timo K
a69a50d30c add log to error boudnary 2025-11-20 16:32:40 +01:00
Timo K
89a1bfac2d remove unused test helper 2025-11-20 15:25:53 +01:00
Timo K
1fd9ac93c9 refactor local transport testing and local memberhsip initialization 2025-11-20 15:19:00 +01:00
Timo
f6ef872242 Merge pull request #3575 from element-hq/hs/lint-src-imports
Restrict importing src/lib in most cases for matrix- libs
2025-11-20 14:55:41 +01:00
Half-Shot
07de8483d5 Restrict importing src/lib in most cases for matrix- libs 2025-11-20 13:32:09 +00:00
Valere Fedronic
42b4c8301f Merge pull request #3547 from element-hq/valere/fix_blank_widget_auto_leave
fix: Send close widget action on auto-leave
2025-11-20 11:41:43 +01:00
Valere
c2f541ff8c split test in 2 one testing sound and one to just leave 2025-11-20 11:28:24 +01:00
Valere
c1049b6088 fix import causing lint errors in node_modules 2025-11-20 10:46:24 +01:00
Valere
6577db16ce Merge branch 'livekit' into valere/fix_blank_widget_auto_leave 2025-11-20 10:41:31 +01:00
renovate[bot]
cdc3c2b216 Update LiveKit components (#3573)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-11-19 20:18:18 +01:00
renovate[bot]
47324b52f8 Update dependency livekit-client to v2.16.0 (#3574)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-11-19 20:14:37 +01:00
alutarius
82706cca4f Correct Caddy reverse proxy config example (#3572) 2025-11-18 18:46:00 +01:00
Timo
665a6be484 Merge pull request #3560 from element-hq/voip-team/call-viewmodel-refactor
`CallViewmodel` refactor
2025-11-18 13:48:03 +01:00
Valere
0076b54e52 minor comments/TODO cleaning 2025-11-18 12:14:17 +01:00
Valere
614762b5f8 fixup headers 2025-11-18 10:13:10 +01:00
Timo
222982c52c Merge pull request #3571 from element-hq/toger5/createCallViewModelFunction$
`createCallViewModel$` function refactor
2025-11-17 19:46:06 +01:00
Timo K
2a19f06e4a reword docstring 2025-11-17 19:45:14 +01:00
Robin
171c289f2d Merge pull request #3564 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.15.15
2025-11-17 13:41:44 -05:00
Timo K
2e2c799f72 convert CallViewModel into create function pattern. (with much more
minimal changes thanks to the intermediate class refactor)
2025-11-17 18:24:47 +01:00
Valere Fedronic
16e1c59e11 playwright: Fix error boundary mgmt or openId errors (#3570)
* playwright: Fix error boundary mgmt or openId errors

* do not use tap for important logic

* fix lint

---------

Co-authored-by: Timo K <toger5@hotmail.de>
2025-11-17 18:22:37 +01:00
Timo
b51df36a06 Move one-on-one layout into CallViewModel (#3567)
* move ononone layout into CallViewModel

* move even more into the vm.

* tests
2025-11-17 17:42:37 +01:00
Timo
49ec4b4298 Merge pull request #3568 from element-hq/toger5/createCallViewModel$
refactor to createCallViewModel$
2025-11-17 15:04:18 +01:00
Timo K
f518111887 add back docstrings 2025-11-17 14:55:00 +01:00
Timo K
4414fe3c82 remove all /*PUBLIC*/ 2025-11-17 14:39:24 +01:00
Timo K
e9f400e9f1 remove ReturnType 2025-11-17 14:30:16 +01:00
Timo K
7ab90d3c1f Delete unused files 2025-11-17 12:00:28 +01:00
Timo K
348f0dbfa8 CallViewModel structure change. Clear input and outputs. 2025-11-17 11:37:58 +01:00
renovate[bot]
9ad029d4ff Update dependency livekit-client to v2.15.15 2025-11-17 02:42:33 +00:00
Robin
cf8298cd69 Fix lints 2025-11-14 18:57:39 -05:00
Robin
bb746dc8dc Fix a resource leak in the track processor code 2025-11-14 18:45:02 -05:00
Robin
53aaa011b7 Remove stray comma
This comma was showing up in the bottom left corner of the UI.
2025-11-14 16:41:18 -05:00
Robin
494ed7c126 Fix the MatrixRTC mode developer controls
They weren't responding properly to clicks, and make more sense as radio buttons.
2025-11-14 14:15:58 -05:00
Timo K
b9ad83bf3b fix a couple of more tests. 2025-11-14 18:45:07 +01:00
Valere
6d8d2d2384 refact: extract withCallViewModel in test utils file 2025-11-14 17:30:02 +01:00
Timo K
b731c2e9fd dont import withCallViewModel in Lifecycle test 2025-11-14 16:53:43 +01:00
Robin
da85e74596 Restore LocalMembership tests 2025-11-14 10:48:24 -05:00
Valere Fedronic
8ce8067956 Adding more logs (#3563)
* Adding more logs

* post merge fix

* fixup merge error

* review

---------

Co-authored-by: Timo K <toger5@hotmail.de>
2025-11-14 16:18:31 +01:00
Timo K
f6d0d7323e fix call lifecycle and add tests. 2025-11-14 16:01:43 +01:00
Timo K
fdce3ec1aa Fix a couple of CallViewModel tests. 2025-11-14 10:44:16 +01:00
Ben Banfield-Zanin
a3950869fb Use the dedicated bug report sub-domain (#3557)
* Update docs for 'rageshake.submit_url' to use the dedicated subdomain

* Update Netlify preview deployment to use the dedicated subdomain for 'rageshake.submit_url'
2025-11-13 15:33:59 +00:00
Timo K
0115242a2b tests first batch 2025-11-13 11:35:37 +01:00
Robin
c7f50b53f5 Fix decryption errors
The code had regressed to a state where it was attempting to use one encryption worker for all LiveKit rooms, which does not currently work.
2025-11-12 15:41:41 -05:00
Robin
a62d8368a1 Fix and simplify screen sharing 2025-11-12 15:02:19 -05:00
Robin
27b76b4b1d Remove unused Async type 2025-11-12 14:28:26 -05:00
Robin
8d421899a6 Fix formatting of doc comment 2025-11-12 10:16:15 -05:00
Timo K
9f4d954cfa The source of the local participant is the createLocalMembership$ and
not the MatrixLivekitMembers!

Co-authored-by: Valere <bill.carson@valrsoft.com>
2025-11-12 12:09:31 +01:00
Timo K
8671d3fd67 Very bit test overhaul. All displayname tests are now done in the
Metadata file. and not in the CallViewModel anymore.
2025-11-11 15:52:35 +01:00
Timo K
85f659bcc9 Introduce MatrixMemberMetadata and use it to simplify username and
avatar computation This removes member from the tiles entirely!
2025-11-11 15:51:48 +01:00
Timo K
93c4dc5beb make it run 2025-11-10 15:55:01 +01:00
Valere
93659931ca fixup: update integration tests 2025-11-10 11:20:20 +01:00
Valere
5c83e0dce1 test: fixup MatrixLivekitMembers tests 2025-11-10 10:43:53 +01:00
Robin
92ddc4c797 Fix avatar reactivity, simplify display names tracking 2025-11-09 01:16:39 -05:00
Robin
b4c17ed26d Replace generateKeyed$ with a redesigned generateItems operator
And use it to clean up a number of code smells, fix some reactivity bugs, and avoid some resource leaks.
2025-11-09 00:46:16 -05:00
Valere
1f386a1d57 test: fix displayname tests due to Epoch change 2025-11-08 13:24:03 +01:00
Valere
fc842d4be7 test: fixup ConnectionManager tests 2025-11-08 13:02:12 +01:00
Timo K
b8635b52d8 Working (no local feed) 2025-11-07 19:07:45 +01:00
Robin
5afc3c2598 Merge pull request #3556 from element-hq/renovate/node-24.x
Update Node.js to v24
2025-11-05 13:55:24 -05:00
Robin
8ad0cb1955 Merge pull request #3551 from element-hq/renovate/embedded-package-dependencies
Update embedded package dependencies to v8.13.0
2025-11-05 13:53:27 -05:00
Robin
9bd464a7f1 Merge pull request #3550 from element-hq/renovate/github-actions
Update softprops/action-gh-release digest to 6da8fa9
2025-11-05 13:53:08 -05:00
Robin
f0adae41ef Merge pull request #3552 from element-hq/renovate/livekit-client
Update dependency livekit-client to v2.15.14
2025-11-05 13:49:44 -05:00
renovate[bot]
f9cc930155 Update Node.js to v24 2025-11-05 18:49:21 +00:00
Robin
eebcd133f3 Merge pull request #3553 from element-hq/robin/trusted-publishing
Remove use of npm access token now that trusted publishing is set up
2025-11-05 13:45:52 -05:00
Robin
79bd458dc2 Remove use of npm access token now that trusted publishing is set up
npm has recently limited the lifetime of all access tokens to 90 days (https://gh.io/npm-token-changes), so it would be a bit inconvenient to stick to our current access token-based method of publishing releases. Meanwhile npm has implemented a more secure publishing method based on OIDC in which you tell the registry that a particular GitHub Actions workflow should be a "trusted publisher" for a given package, and then the CLI will authenticate automatically. (https://docs.npmjs.com/trusted-publishers)

I've already set trusted publishing up on the registry side, and since we're already granting the job permission to generate ID tokens for provenance, there should be no additional lines of config needed to make it work. Let's take away the access token and see how this goes next time we release.
2025-11-03 17:13:03 -05:00
renovate[bot]
a4b1ce52cd Update dependency livekit-client to v2.15.14 2025-11-03 01:02:47 +00:00
renovate[bot]
b22a937e7c Update softprops/action-gh-release digest to 6da8fa9 2025-11-01 02:02:21 +00:00
Valere
0309ceb6aa fix lint 2025-10-27 15:59:01 +01:00
Valere
3ecffccd9e fix prettier 2025-10-27 15:41:16 +01:00
Valere
3dd0b156cf fix: Send close widget action on auto-leave 2025-10-27 14:31:23 +01:00
renovate[bot]
214bb75d27 Update embedded package dependencies to v8.13.0 2025-10-01 02:29:37 +00:00
78 changed files with 6447 additions and 4548 deletions

View File

@@ -43,6 +43,29 @@ module.exports = {
// To encourage good usage of RxJS:
"rxjs/no-exposed-subjects": "error",
"rxjs/finnish": ["error", { names: { "^this$": false } }],
"no-restricted-imports": [
"error",
{
paths: ["matrix-widget-api", "matrix-js-sdk"].flatMap((lib) =>
["src", "src/", "src/index", "lib", "lib/", "lib/index"]
.map((path) => `${lib}/${path}`)
.map((name) => ({ name, message: `Please use ${lib} instead` })),
),
patterns: [
...["matrix-widget-api"].map((lib) => ({
group: ["src", "src/", "src/**", "lib", "lib/", "lib/**"].map(
(path) => `${lib}/${path}`,
),
message: `Please use ${lib} instead`,
})),
// XXX: We use /lib in lots of places, so allow for now.
...["matrix-js-sdk"].map((lib) => ({
group: ["src", "src/", "src/**"].map((path) => `${lib}/${path}`),
message: `Please use ${lib} instead`,
})),
],
},
],
},
overrides: [
{

View File

@@ -85,7 +85,7 @@ jobs:
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
- name: Upload
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2
uses: softprops/action-gh-release@6da8fa9354ddfdc4aeace5fc48d7f679b5214090 # v2
with:
files: |
${{ env.FILENAME_PREFIX }}.tar.gz
@@ -100,7 +100,7 @@ jobs:
ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
permissions:
contents: read
id-token: write # required for the provenance flag on npm publish
id-token: write # Allow npm to authenticate as a trusted publisher
steps:
- name: Checkout
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
@@ -126,8 +126,6 @@ jobs:
npm version ${{ needs.versioning.outputs.PREFIXED_VERSION }} --no-git-tag-version
echo "ARTIFACT_VERSION=$(jq '.version' --raw-output package.json)" >> "$GITHUB_ENV"
npm publish --provenance --access public --tag ${{ needs.versioning.outputs.TAG }} ${{ needs.versioning.outputs.DRY_RUN == 'true' && '--dry-run' || '' }}
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_RELEASE_TOKEN }}
- id: artifact_version
name: Output artifact version
@@ -264,7 +262,7 @@ jobs:
echo "iOS: ${{ needs.publish_ios.outputs.ARTIFACT_VERSION }}"
- name: Add release notes
if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2
uses: softprops/action-gh-release@6da8fa9354ddfdc4aeace5fc48d7f679b5214090 # v2
with:
append_body: true
body: |

View File

@@ -42,7 +42,7 @@ jobs:
- name: Create Checksum
run: find ${{ env.FILENAME_PREFIX }} -type f -print0 | sort -z | xargs -0 sha256sum | tee ${{ env.FILENAME_PREFIX }}.sha256
- name: Upload
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2
uses: softprops/action-gh-release@6da8fa9354ddfdc4aeace5fc48d7f679b5214090 # v2
with:
files: |
${{ env.FILENAME_PREFIX }}.tar.gz
@@ -68,7 +68,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Add release note
uses: softprops/action-gh-release@aec2ec56f94eb8180ceec724245f64ef008b89f5 # v2
uses: softprops/action-gh-release@6da8fa9354ddfdc4aeace5fc48d7f679b5214090 # v2
with:
append_body: true
body: |

View File

@@ -1 +1 @@
22
24

View File

@@ -18,7 +18,7 @@
"api_host": "https://posthog-element-call.element.io"
},
"rageshake": {
"submit_url": "https://element.io/bugreports/submit"
"submit_url": "https://rageshakes.element.io/api/submit"
},
"sentry": {
"environment": "netlify-pr-preview",

View File

@@ -136,8 +136,8 @@ handle @jwt_service {
reverse_proxy http://[::1]:8080 {
header_up Host {host}
header_up X-Forwarded-Server {host}
header_up X-Real-IP {remote_addr}
header_up X-Forwarded-For {remote_addr}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
}
}
@@ -146,8 +146,8 @@ handle {
reverse_proxy http://localhost:7880 {
header_up Host {host}
header_up X-Forwarded-Server {host}
header_up X-Real-IP {remote_addr}
header_up X-Forwarded-For {remote_addr}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
}
}
```

View File

@@ -96,6 +96,6 @@ These parameters are only supported in the [embedded](./embedded-standalone.md)
| -------------------- | -------------------------------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `posthogApiHost` | Posthog server URL | No | e.g. `https://posthog-element-call.element.io`. Only supported in embedded package. In full package the value from config is used. |
| `posthogApiKey` | Posthog project API key | No | Only supported in embedded package. In full package the value from config is used. |
| `rageshakeSubmitUrl` | Rageshake server URL endpoint | No | e.g. `https://element.io/bugreports/submit`. In full package the value from config is used. |
| `rageshakeSubmitUrl` | Rageshake server URL endpoint | No | e.g. `https://rageshakes.element.io/api/submit`. In full package the value from config is used. |
| `sentryDsn` | Sentry [DSN](https://docs.sentry.io/concepts/key-terms/dsn-explainer/) | No | In full package the value from config is used. |
| `sentryEnvironment` | Sentry [environment](https://docs.sentry.io/concepts/key-terms/key-terms/) | No | In full package the value from config is used. |

View File

@@ -2,7 +2,7 @@
# https://docs.gradle.org/current/userguide/platforms.html#sub::toml-dependencies-format
[versions]
android_gradle_plugin = "8.11.1"
android_gradle_plugin = "8.13.0"
[libraries]
android_gradle_plugin = { module = "com.android.tools.build:gradle", version.ref = "android_gradle_plugin" }

View File

@@ -64,6 +64,14 @@
"developer_mode": {
"always_show_iphone_earpiece": "Show iPhone earpiece option on all platforms",
"crypto_version": "Crypto version: {{version}}",
"custom_livekit_url": {
"current_url": "Currently set to: ",
"from_config": "Currently, no overwrite is set. Url from well-known or config is used.",
"label": "Custom Livekit-url",
"reset": "Reset overwrite",
"save": "Save",
"saving": "Saving..."
},
"debug_tile_layout_label": "Debug tile layout",
"device_id": "Device ID: {{id}}",
"duplicate_tiles_label": "Number of additional tile copies per participant",
@@ -74,17 +82,18 @@
"matrix_id": "Matrix ID: {{id}}",
"matrixRTCMode": {
"Comptibility": {
"label": "Compatibility: state events & multi SFU",
"description": "Compatible with homeservers that do not support sticky events (but all other EC clients are v0.17.0 or later)"
"description": "Compatible with homeservers that do not support sticky events (but all other EC clients are v0.17.0 or later)",
"label": "Compatibility: state events & multi SFU"
},
"Legacy": {
"label": "Legacy: state events & oldest membership SFU",
"description": "Compatible with old versions of EC that do not support multi SFU"
"description": "Compatible with old versions of EC that do not support multi SFU",
"label": "Legacy: state events & oldest membership SFU"
},
"Matrix_2_0": {
"label": "Matrix 2.0: sticky events & multi SFU",
"description": "Compatible only with homservers supporting sticky events and all EC clients v0.17.0 or later"
}
"description": "Compatible only with homservers supporting sticky events and all EC clients v0.17.0 or later",
"label": "Matrix 2.0: sticky events & multi SFU"
},
"title": "MatrixRTC mode"
},
"mute_all_audio": "Mute all audio (participants, reactions, join sounds)",
"show_connection_stats": "Show connection statistics",

View File

@@ -71,7 +71,7 @@
"@types/grecaptcha": "^3.0.9",
"@types/jsdom": "^21.1.7",
"@types/lodash-es": "^4.17.12",
"@types/node": "^22.0.0",
"@types/node": "^24.0.0",
"@types/pako": "^2.0.3",
"@types/qrcode": "^1.5.5",
"@types/react": "^19.0.0",

View File

@@ -159,7 +159,11 @@ export const widgetTest = test.extend<MyFixtures>({
} = await registerUser(browser, userB);
// Invite the second user
await ewPage1.getByRole("button", { name: "Add", exact: true }).click();
await ewPage1
.getByRole("navigation", { name: "Room list" })
.getByRole("button", { name: "New conversation" })
.click();
await ewPage1.getByRole("menuitem", { name: "New Room" }).click();
await ewPage1.getByRole("textbox", { name: "Name" }).fill("Welcome Room");
await ewPage1.getByRole("button", { name: "Create room" }).click();

View File

@@ -122,7 +122,7 @@ export interface ConfigOptions {
delayed_leave_event_delay_ms?: number;
/**
* The time (in milliseconds) after which a we consider a delayed event restart http request to have failed.
* The time (in milliseconds) after which we consider a delayed event restart http request to have failed.
* Setting this to a lower value will result in more frequent retries but also a higher chance of failiour.
*
* In the presence of network packet loss (hurting TCP connections), the custom delayedEventRestartLocalTimeoutMs

View File

@@ -28,6 +28,7 @@ import {
} from "../settings/settings";
import { BlurBackgroundTransformer } from "./BlurBackgroundTransformer";
import { type Behavior } from "../state/Behavior";
import { type ObservableScope } from "../state/ObservableScope";
//TODO-MULTI-SFU: This is not yet fully there.
// it is a combination of exposing observable and react hooks.
@@ -63,13 +64,17 @@ export function useTrackProcessorObservable$(): Observable<ProcessorState> {
return state$;
}
/**
* Updates your video tracks to always use the given processor.
*/
export const trackProcessorSync = (
scope: ObservableScope,
videoTrack$: Behavior<LocalVideoTrack | null>,
processor$: Behavior<ProcessorState>,
): void => {
// TODO-MULTI-SFU: Bind to an ObservableScope to avoid leaking resources.
combineLatest([videoTrack$, processor$]).subscribe(
([videoTrack, processorState]) => {
combineLatest([videoTrack$, processor$])
.pipe(scope.bind())
.subscribe(([videoTrack, processorState]) => {
if (!processorState) return;
if (!videoTrack) return;
const { processor } = processorState;
@@ -79,8 +84,7 @@ export const trackProcessorSync = (
if (!processor && videoTrack.getProcessor()) {
void videoTrack.stopProcessor();
}
},
);
});
};
export const useTrackProcessorSync = (

View File

@@ -21,7 +21,16 @@ export type OpenIDClientParts = Pick<
MatrixClient,
"getOpenIdToken" | "getDeviceId"
>;
/**
* Gets a bearer token from the homeserver and then use it to authenticate
* to the matrix RTC backend in order to get acces to the SFU.
* It has built-in retry for calls to the homeserver with a backoff policy.
* @param client
* @param serviceUrl
* @param matrixRoomId
* @returns Object containing the token information
* @throws FailToGetOpenIdToken
*/
export async function getSFUConfigWithOpenID(
client: OpenIDClientParts,
serviceUrl: string,

View File

@@ -62,7 +62,7 @@ Initializer.initBeforeReact()
.then(() => {
root.render(
<StrictMode>
<App vm={new AppViewModel(globalScope)} />,
<App vm={new AppViewModel(globalScope)} />
</StrictMode>,
);
})

View File

@@ -266,6 +266,7 @@ export class ReactionsReader {
);
return;
}
// TODO refactor to use memer id `membershipEvent.membershipID` (needs to happen in combination with other memberId refactors)
const identifier = `${membershipEvent.userId}:${membershipEvent.deviceId}`;
if (!content.emoji) {

View File

@@ -22,6 +22,7 @@ import {
WebBrowserIcon,
} from "@vector-im/compound-design-tokens/assets/web/icons";
import { Button } from "@vector-im/compound-web";
import { logger } from "matrix-js-sdk/lib/logger";
import {
ConnectionLostError,
@@ -53,7 +54,7 @@ const ErrorPage: FC<ErrorPageProps> = ({
widget,
}: ErrorPageProps): ReactElement => {
const { t } = useTranslation();
logger.error("Error boundary caught:", error);
let icon: ComponentType<SVGAttributes<SVGElement>>;
switch (error.category) {
case ErrorCategory.CONFIGURATION_ISSUE:

View File

@@ -29,6 +29,7 @@ import userEvent from "@testing-library/user-event";
import { type RelationsContainer } from "matrix-js-sdk/lib/models/relations-container";
import { useState } from "react";
import { TooltipProvider } from "@vector-im/compound-web";
import { type ITransport } from "matrix-widget-api";
import { prefetchSounds } from "../soundUtils";
import { useAudioContext } from "../useAudioContext";
@@ -43,7 +44,7 @@ import {
MockRTCSession,
} from "../utils/test";
import { GroupCallView } from "./GroupCallView";
import { type WidgetHelpers } from "../widget";
import { ElementWidgetActions, type WidgetHelpers } from "../widget";
import { LazyEventEmitter } from "../LazyEventEmitter";
import { MatrixRTCTransportMissingError } from "../utils/errors";
import { ProcessorProvider } from "../livekit/TrackProcessorContext";
@@ -77,13 +78,13 @@ const leaveRTCSession = vi.hoisted(() =>
),
);
vi.mock("../rtcSessionHelpers", async (importOriginal) => {
// TODO: perhaps there is a more elegant way to manage the type import here?
// eslint-disable-next-line @typescript-eslint/consistent-type-imports
const orig = await importOriginal<typeof import("../rtcSessionHelpers")>();
// TODO: leaveRTCSession no longer exists! Tests need adapting.
return { ...orig, enterRTCSession, leaveRTCSession };
});
// vi.mock("../rtcSessionHelpers", async (importOriginal) => {
// // TODO: perhaps there is a more elegant way to manage the type import here?
// // eslint-disable-next-line @typescript-eslint/consistent-type-imports
// const orig = await importOriginal<typeof import("../rtcSessionHelpers")>();
// // TODO: leaveRTCSession no longer exists! Tests need adapting.
// return { ...orig, enterRTCSession, leaveRTCSession };
// });
let playSound: MockedFunction<
NonNullable<ReturnType<typeof useAudioContext>>["playSound"]
@@ -112,6 +113,10 @@ beforeEach(() => {
return (
<div>
<button onClick={() => onLeave("user")}>Leave</button>
<button onClick={() => onLeave("allOthersLeft")}>
SimulateOtherLeft
</button>
<button onClick={() => onLeave("error")}>SimulateErrorLeft</button>
</div>
);
},
@@ -243,6 +248,112 @@ test.skip("GroupCallView plays a leave sound synchronously in widget mode", asyn
expect(leaveRTCSession).toHaveBeenCalledOnce();
});
test.skip("Should close widget when all other left and have time to play a sound", async () => {
const user = userEvent.setup();
const widgetClosedCalled = Promise.withResolvers<void>();
const widgetSendMock = vi.fn().mockImplementation((action: string) => {
if (action === ElementWidgetActions.Close) {
widgetClosedCalled.resolve();
}
});
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const resolvePlaySound = Promise.withResolvers<void>();
playSound = vi.fn().mockReturnValue(resolvePlaySound);
(useAudioContext as MockedFunction<typeof useAudioContext>).mockReturnValue({
playSound,
playSoundLooping: vitest.fn(),
soundDuration: {},
});
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateOtherLeft");
await user.click(leaveButton);
await flushPromises();
expect(widgetSendMock).not.toHaveBeenCalled();
resolvePlaySound.resolve();
await flushPromises();
expect(playSound).toHaveBeenCalledWith("left");
await widgetClosedCalled.promise;
await flushPromises();
expect(widgetStopMock).toHaveBeenCalledOnce();
});
test("Should close widget when all other left", async () => {
const user = userEvent.setup();
const widgetClosedCalled = Promise.withResolvers<void>();
const widgetSendMock = vi.fn().mockImplementation((action: string) => {
if (action === ElementWidgetActions.Close) {
widgetClosedCalled.resolve();
}
});
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateOtherLeft");
await user.click(leaveButton);
await flushPromises();
await widgetClosedCalled.promise;
await flushPromises();
expect(widgetStopMock).toHaveBeenCalledOnce();
});
test("Should not close widget when auto leave due to error", async () => {
const user = userEvent.setup();
const widgetStopMock = vi.fn().mockResolvedValue(undefined);
const widgetSendMock = vi.fn().mockResolvedValue(undefined);
const widget = {
api: {
setAlwaysOnScreen: vi.fn().mockResolvedValue(true),
transport: {
send: widgetSendMock,
reply: vi.fn().mockResolvedValue(undefined),
stop: widgetStopMock,
} as unknown as ITransport,
} as Partial<WidgetHelpers["api"]>,
lazyActions: new LazyEventEmitter(),
};
const alwaysOnScreenSpy = vi.spyOn(widget.api, "setAlwaysOnScreen");
const { getByText } = createGroupCallView(widget as WidgetHelpers);
const leaveButton = getByText("SimulateErrorLeft");
await user.click(leaveButton);
await flushPromises();
// When onLeft is called, we first set always on screen to false
await waitFor(() => expect(alwaysOnScreenSpy).toHaveBeenCalledWith(false));
await flushPromises();
// But then we do not close the widget automatically
expect(widgetStopMock).not.toHaveBeenCalledOnce();
expect(widgetSendMock).not.toHaveBeenCalledOnce();
});
test.skip("GroupCallView leaves the session when an error occurs", async () => {
(ActiveCall as MockedFunction<typeof ActiveCall>).mockImplementation(() => {
const [error, setError] = useState<Error | null>(null);
@@ -266,6 +377,7 @@ test.skip("GroupCallView leaves the session when an error occurs", async () => {
test.skip("GroupCallView shows errors that occur during joining", async () => {
const user = userEvent.setup();
// This should not mock this error that deep. it should only mock the CallViewModel.
enterRTCSession.mockRejectedValue(new MatrixRTCTransportMissingError(""));
onTestFinished(() => {
enterRTCSession.mockReset();

View File

@@ -159,6 +159,7 @@ export const GroupCallView: FC<Props> = ({
};
}, [rtcSession]);
// TODO move this into the callViewModel LocalMembership.ts
useTypedEventEmitter(
rtcSession,
MatrixRTCSessionEvent.MembershipManagerError,
@@ -313,7 +314,9 @@ export const GroupCallView: FC<Props> = ({
const navigate = useNavigate();
const onLeft = useCallback(
(reason: "timeout" | "user" | "allOthersLeft" | "decline"): void => {
(
reason: "timeout" | "user" | "allOthersLeft" | "decline" | "error",
): void => {
let playSound: CallEventSounds = "left";
if (reason === "timeout" || reason === "decline") playSound = reason;
@@ -366,7 +369,7 @@ export const GroupCallView: FC<Props> = ({
}
// On a normal user hangup we can shut down and close the widget. But if an
// error occurs we should keep the widget open until the user reads it.
if (reason === "user" && !getUrlParams().returnToLobby) {
if (reason != "error" && !getUrlParams().returnToLobby) {
try {
await widget.api.transport.send(ElementWidgetActions.Close, {});
} catch (e) {
@@ -518,8 +521,7 @@ export const GroupCallView: FC<Props> = ({
}}
onError={
(/**error*/) => {
// TODO this should not be "user". It needs a new case
if (rtcSession.isJoined()) onLeft("user");
if (rtcSession.isJoined()) onLeft("error");
}
}
>

View File

@@ -43,9 +43,6 @@ import { LivekitRoomAudioRenderer } from "../livekit/MatrixAudioRenderer";
import { MediaDevicesContext } from "../MediaDevicesContext";
import { HeaderStyle } from "../UrlParams";
// vi.hoisted(() => {
// localStorage = {} as unknown as Storage;
// });
vi.hoisted(
() =>
(global.ImageData = class MockImageData {
@@ -109,6 +106,7 @@ function createInCallView(): RenderResult & {
getUserId: () => localRtcMember.userId,
getDeviceId: () => localRtcMember.deviceId,
getRoom: (rId) => (rId === roomId ? room : null),
getDomain: () => "example.com",
} as Partial<MatrixClient> as MatrixClient;
const room = mockMatrixRoom({
relations: {
@@ -119,7 +117,8 @@ function createInCallView(): RenderResult & {
} as unknown as RelationsContainer,
client,
roomId,
getMember: (userId) => roomMembers.get(userId) ?? null,
// getMember: (userId) => roomMembers.get(userId) ?? null,
getMembers: () => Array.from(roomMembers.values()),
getMxcAvatarUrl: () => null,
hasEncryptionStateEvent: vi.fn().mockReturnValue(true),
getCanonicalAlias: () => null,

View File

@@ -59,7 +59,8 @@ import { type MatrixInfo } from "./VideoPreview";
import { InviteButton } from "../button/InviteButton";
import { LayoutToggle } from "./LayoutToggle";
import {
CallViewModel,
type CallViewModel,
createCallViewModel$,
type GridMode,
} from "../state/CallViewModel/CallViewModel.ts";
import { Grid, type TileProps } from "../grid/Grid";
@@ -114,21 +115,23 @@ export interface ActiveCallProps
extends Omit<InCallViewProps, "vm" | "livekitRoom" | "connState"> {
e2eeSystem: EncryptionSystem;
// TODO refactor those reasons into an enum
onLeft: (reason: "user" | "timeout" | "decline" | "allOthersLeft") => void;
onLeft: (
reason: "user" | "timeout" | "decline" | "allOthersLeft" | "error",
) => void;
}
export const ActiveCall: FC<ActiveCallProps> = (props) => {
const mediaDevices = useMediaDevices();
const [vm, setVm] = useState<CallViewModel | null>(null);
const { autoLeaveWhenOthersLeft, waitForCallPickup, sendNotificationType } =
useUrlParams();
const urlParams = useUrlParams();
const mediaDevices = useMediaDevices();
const trackProcessorState$ = useTrackProcessorObservable$();
useEffect(() => {
const scope = new ObservableScope();
const reactionsReader = new ReactionsReader(scope, props.rtcSession);
const vm = new CallViewModel(
const { autoLeaveWhenOthersLeft, waitForCallPickup, sendNotificationType } =
urlParams;
const vm = createCallViewModel$(
scope,
props.rtcSession,
props.matrixRoom,
@@ -152,13 +155,11 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
}, [
props.rtcSession,
props.matrixRoom,
mediaDevices,
props.muteStates,
props.e2eeSystem,
autoLeaveWhenOthersLeft,
sendNotificationType,
waitForCallPickup,
props.onLeft,
urlParams,
mediaDevices,
trackProcessorState$,
]);
@@ -250,7 +251,6 @@ export const InCallView: FC<InCallViewProps> = ({
() => void toggleRaisedHand(),
);
// const allLivekitRooms = useBehavior(vm.allLivekitRooms$);
const audioParticipants = useBehavior(vm.audioParticipants$);
const participantCount = useBehavior(vm.participantCount$);
const reconnecting = useBehavior(vm.reconnecting$);
@@ -265,7 +265,8 @@ export const InCallView: FC<InCallViewProps> = ({
const audioOutputSwitcher = useBehavior(vm.audioOutputSwitcher$);
const sharingScreen = useBehavior(vm.sharingScreen$);
const fatalCallError = useBehavior(vm.configError$);
const ringOverlay = useBehavior(vm.ringOverlay$);
const fatalCallError = useBehavior(vm.fatalError$);
// Stop the rendering and throw for the error boundary
if (fatalCallError) throw fatalCallError;
@@ -301,47 +302,26 @@ export const InCallView: FC<InCallViewProps> = ({
// Waiting UI overlay
const waitingOverlay: JSX.Element | null = useMemo(() => {
// No overlay if not in ringing state
if (callPickupState !== "ringing") return null;
// Use room state for other participants data (the one that we likely want to reach)
// TODO: this screams it wants to be a behavior in the vm.
const roomOthers = [
...matrixRoom.getMembersWithMembership("join"),
...matrixRoom.getMembersWithMembership("invite"),
].filter((m) => m.userId !== client.getUserId());
// Yield if there are not other members in the room.
if (roomOthers.length === 0) return null;
const otherMember = roomOthers.length > 0 ? roomOthers[0] : undefined;
const isOneOnOne = roomOthers.length === 1 && otherMember;
const text = isOneOnOne
? `Waiting for ${otherMember.name ?? otherMember.userId} to join…`
: "Waiting for other participants…";
const avatarMxc = isOneOnOne
? (otherMember.getMxcAvatarUrl?.() ?? undefined)
: (matrixRoom.getMxcAvatarUrl() ?? undefined);
return (
return ringOverlay ? (
<div className={classNames(overlayStyles.bg, waitingStyles.overlay)}>
<div
className={classNames(overlayStyles.content, waitingStyles.content)}
>
<div className={waitingStyles.pulse}>
<Avatar
id={isOneOnOne ? otherMember.userId : matrixRoom.roomId}
name={isOneOnOne ? otherMember.name : matrixRoom.name}
src={avatarMxc}
id={ringOverlay.idForAvatar}
name={ringOverlay.name}
src={ringOverlay.avatarMxc}
size={AvatarSize.XL}
/>
</div>
<Text size="md" className={waitingStyles.text}>
{text}
{ringOverlay.text}
</Text>
</div>
</div>
);
}, [callPickupState, client, matrixRoom]);
) : null;
}, [ringOverlay]);
// Ideally we could detect taps by listening for click events and checking
// that the pointerType of the event is "touch", but this isn't yet supported
@@ -734,8 +714,7 @@ export const InCallView: FC<InCallViewProps> = ({
<ShareScreenButton
key="share_screen"
className={styles.shareScreen}
disabled={sharingScreen === undefined}
enabled={sharingScreen === true}
enabled={sharingScreen}
onClick={vm.toggleScreenSharing}
onTouchEnd={onControlsTouchEnd}
data-testid="incall_screenshare"

View File

@@ -10,8 +10,9 @@ import {
afterAll,
afterEach,
beforeEach,
describe,
expect,
test,
it,
vitest,
type MockedFunction,
type Mock,
@@ -49,122 +50,125 @@ vitest.mock("livekit-client/e2ee-worker?worker");
vitest.mock("../useAudioContext");
vitest.mock("../soundUtils");
afterEach(() => {
vitest.resetAllMocks();
playReactionsSoundSetting.setValue(playReactionsSoundSetting.defaultValue);
soundEffectVolumeSetting.setValue(soundEffectVolumeSetting.defaultValue);
});
afterAll(() => {
vitest.restoreAllMocks();
});
let playSound: Mock<
NonNullable<ReturnType<typeof useAudioContext>>["playSound"]
>;
beforeEach(() => {
(prefetchSounds as MockedFunction<typeof prefetchSounds>).mockResolvedValue({
sound: new ArrayBuffer(0),
describe("ReactionAudioRenderer", () => {
afterEach(() => {
playReactionsSoundSetting.setValue(playReactionsSoundSetting.defaultValue);
soundEffectVolumeSetting.setValue(soundEffectVolumeSetting.defaultValue);
});
playSound = vitest.fn();
(useAudioContext as MockedFunction<typeof useAudioContext>).mockReturnValue({
playSound,
playSoundLooping: vitest.fn(),
soundDuration: {},
});
});
test("preloads all audio elements", () => {
const { vm } = getBasicCallViewModelEnvironment([local, alice]);
playReactionsSoundSetting.setValue(true);
render(<TestComponent vm={vm} />);
expect(prefetchSounds).toHaveBeenCalledOnce();
});
test("will play an audio sound when there is a reaction", () => {
const { vm, reactionsSubject$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
playReactionsSoundSetting.setValue(true);
render(<TestComponent vm={vm} />);
// Find the first reaction with a sound effect
const chosenReaction = ReactionSet.find((r) => !!r.sound);
if (!chosenReaction) {
throw Error(
"No reactions have sounds configured, this test cannot succeed",
beforeEach(() => {
(prefetchSounds as MockedFunction<typeof prefetchSounds>).mockResolvedValue(
{
sound: new ArrayBuffer(0),
},
);
}
act(() => {
reactionsSubject$.next({
[aliceRtcMember.deviceId]: {
reactionOption: chosenReaction,
expireAfter: new Date(0),
playSound = vitest.fn();
(useAudioContext as MockedFunction<typeof useAudioContext>).mockReturnValue(
{
playSound,
playSoundLooping: vitest.fn(),
soundDuration: {},
},
});
});
expect(playSound).toHaveBeenCalledWith(chosenReaction.name);
});
test("will play the generic audio sound when there is soundless reaction", () => {
const { vm, reactionsSubject$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
playReactionsSoundSetting.setValue(true);
render(<TestComponent vm={vm} />);
// Find the first reaction with a sound effect
const chosenReaction = ReactionSet.find((r) => !r.sound);
if (!chosenReaction) {
throw Error(
"No reactions have sounds configured, this test cannot succeed",
);
}
act(() => {
reactionsSubject$.next({
[aliceRtcMember.deviceId]: {
reactionOption: chosenReaction,
expireAfter: new Date(0),
},
});
});
expect(playSound).toHaveBeenCalledWith(GenericReaction.name);
});
test("will play multiple audio sounds when there are multiple different reactions", () => {
const { vm, reactionsSubject$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
playReactionsSoundSetting.setValue(true);
render(<TestComponent vm={vm} />);
// Find the first reaction with a sound effect
const [reaction1, reaction2] = ReactionSet.filter((r) => !!r.sound);
if (!reaction1 || !reaction2) {
throw Error(
"No reactions have sounds configured, this test cannot succeed",
);
}
act(() => {
reactionsSubject$.next({
[aliceRtcMember.deviceId]: {
reactionOption: reaction1,
expireAfter: new Date(0),
},
[bobRtcMember.deviceId]: {
reactionOption: reaction2,
expireAfter: new Date(0),
},
[localRtcMember.deviceId]: {
reactionOption: reaction1,
expireAfter: new Date(0),
},
});
afterAll(() => {
vitest.restoreAllMocks();
});
it("preloads all audio elements", () => {
const { vm } = getBasicCallViewModelEnvironment([local, alice]);
playReactionsSoundSetting.setValue(true);
render(<TestComponent vm={vm} />);
expect(prefetchSounds).toHaveBeenCalledOnce();
});
it("will play an audio sound when there is a reaction", () => {
const { vm, reactionsSubject$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
playReactionsSoundSetting.setValue(true);
render(<TestComponent vm={vm} />);
// Find the first reaction with a sound effect
const chosenReaction = ReactionSet.find((r) => !!r.sound);
if (!chosenReaction) {
throw Error(
"No reactions have sounds configured, this test cannot succeed",
);
}
act(() => {
reactionsSubject$.next({
[aliceRtcMember.deviceId]: {
reactionOption: chosenReaction,
expireAfter: new Date(0),
},
});
});
expect(playSound).toHaveBeenCalledWith(chosenReaction.name);
});
it("will play the generic audio sound when there is soundless reaction", () => {
const { vm, reactionsSubject$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
playReactionsSoundSetting.setValue(true);
render(<TestComponent vm={vm} />);
// Find the first reaction with a sound effect
const chosenReaction = ReactionSet.find((r) => !r.sound);
if (!chosenReaction) {
throw Error(
"No reactions have sounds configured, this test cannot succeed",
);
}
act(() => {
reactionsSubject$.next({
[aliceRtcMember.deviceId]: {
reactionOption: chosenReaction,
expireAfter: new Date(0),
},
});
});
expect(playSound).toHaveBeenCalledWith(GenericReaction.name);
});
it("will play multiple audio sounds when there are multiple different reactions", () => {
const { vm, reactionsSubject$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
playReactionsSoundSetting.setValue(true);
render(<TestComponent vm={vm} />);
// Find the first reaction with a sound effect
const [reaction1, reaction2] = ReactionSet.filter((r) => !!r.sound);
if (!reaction1 || !reaction2) {
throw Error(
"No reactions have sounds configured, this test cannot succeed",
);
}
act(() => {
reactionsSubject$.next({
[aliceRtcMember.deviceId]: {
reactionOption: reaction1,
expireAfter: new Date(0),
},
[bobRtcMember.deviceId]: {
reactionOption: reaction2,
expireAfter: new Date(0),
},
[localRtcMember.deviceId]: {
reactionOption: reaction1,
expireAfter: new Date(0),
},
});
});
expect(playSound).toHaveBeenCalledWith(reaction1.name);
expect(playSound).toHaveBeenCalledWith(reaction2.name);
});
expect(playSound).toHaveBeenCalledWith(reaction1.name);
expect(playSound).toHaveBeenCalledWith(reaction2.name);
});

View File

@@ -83,9 +83,6 @@ exports[`InCallView > rendering > renders 1`] = `
class="nav rightNav"
/>
</header>
<div>
mocked: MatrixAudioRenderer
</div>
<div
class="scrollingGrid grid"
>

View File

@@ -1,159 +0,0 @@
/*
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type MatrixRTCSession } from "matrix-js-sdk/lib/matrixrtc";
import { expect, test, vi } from "vitest";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import EventEmitter from "events";
import { enterRTCSession } from "../src/rtcSessionHelpers";
import { mockConfig } from "./utils/test";
import { MatrixRTCMode } from "./settings/settings";
const MATRIX_RTC_MODE = MatrixRTCMode.Legacy;
const getUrlParams = vi.hoisted(() => vi.fn(() => ({})));
vi.mock("./UrlParams", () => ({ getUrlParams }));
const actualWidget = await vi.hoisted(async () => vi.importActual("./widget"));
vi.mock("./widget", () => ({
...actualWidget,
widget: {
api: {
setAlwaysOnScreen: (): void => {},
transport: { send: vi.fn(), reply: vi.fn(), stop: vi.fn() },
},
lazyActions: new EventEmitter(),
},
}));
test("It joins the correct Session", async () => {
const focusFromOlderMembership = {
type: "livekit",
livekit_service_url: "http://my-oldest-member-service-url.com",
livekit_alias: "my-oldest-member-service-alias",
};
const focusConfigFromWellKnown = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
};
const focusConfigFromWellKnown2 = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url2.com",
};
const clientWellKnown = {
"org.matrix.msc4143.rtc_foci": [
focusConfigFromWellKnown,
focusConfigFromWellKnown2,
],
};
mockConfig({
livekit: { livekit_service_url: "http://my-default-service-url.com" },
});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockImplementation(
async (domain) => {
if (domain === "example.org") {
return Promise.resolve(clientWellKnown);
}
return Promise.resolve({});
},
);
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn().mockReturnValue(focusFromOlderMembership),
getOldestMembership: vi.fn().mockReturnValue({
getPreferredFoci: vi.fn().mockReturnValue([focusFromOlderMembership]),
}),
joinRoomSession: vi.fn(),
}) as unknown as MatrixRTCSession;
await enterRTCSession(
mockedSession,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
);
expect(mockedSession.joinRoomSession).toHaveBeenLastCalledWith(
[
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
],
undefined,
expect.objectContaining({
manageMediaKeys: true,
useLegacyMemberEvents: false,
}),
);
});
test("It should not fail with configuration error if homeserver config has livekit url but not fallback", async () => {
mockConfig({});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({
"org.matrix.msc4143.rtc_foci": [
{
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
},
],
});
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn(),
joinRoomSession: vi.fn(),
}) as unknown as MatrixRTCSession;
await enterRTCSession(
mockedSession,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
);
});

View File

@@ -1,160 +0,0 @@
/*
Copyright 2023, 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type MatrixRTCSession,
isLivekitTransportConfig,
type LivekitTransportConfig,
type LivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
import { logger } from "matrix-js-sdk/lib/logger";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { type MatrixClient } from "matrix-js-sdk";
import { PosthogAnalytics } from "./analytics/PosthogAnalytics";
import { Config } from "./config/Config";
import { ElementWidgetActions, widget } from "./widget";
import { MatrixRTCTransportMissingError } from "./utils/errors";
import { getUrlParams } from "./UrlParams";
import { getSFUConfigWithOpenID } from "./livekit/openIDSFU.ts";
import { MatrixRTCMode } from "./settings/settings.ts";
const FOCI_WK_KEY = "org.matrix.msc4143.rtc_foci";
async function makeTransportInternal(
client: MatrixClient,
roomId: string,
): Promise<LivekitTransport> {
logger.log("Searching for a preferred transport");
//TODO refactor this to use the jwt service returned alias.
const livekitAlias = roomId;
// TODO-MULTI-SFU: Either remove this dev tool or make it more official
const urlFromStorage =
localStorage.getItem("robin-matrixrtc-auth") ??
localStorage.getItem("timo-focus-url");
if (urlFromStorage !== null) {
const transportFromStorage: LivekitTransport = {
type: "livekit",
livekit_service_url: urlFromStorage,
livekit_alias: livekitAlias,
};
logger.log(
"Using LiveKit transport from local storage: ",
transportFromStorage,
);
return transportFromStorage;
}
// Prioritize the .well-known/matrix/client, if available, over the configured SFU
const domain = client.getDomain();
if (domain) {
// we use AutoDiscovery instead of relying on the MatrixClient having already
// been fully configured and started
const wellKnownFoci = (await AutoDiscovery.getRawClientConfig(domain))?.[
FOCI_WK_KEY
];
if (Array.isArray(wellKnownFoci)) {
const transport: LivekitTransportConfig | undefined = wellKnownFoci.find(
(f) => f && isLivekitTransportConfig(f),
);
if (transport !== undefined) {
logger.log("Using LiveKit transport from .well-known: ", transport);
return { ...transport, livekit_alias: livekitAlias };
}
}
}
const urlFromConf = Config.get().livekit?.livekit_service_url;
if (urlFromConf) {
const transportFromConf: LivekitTransport = {
type: "livekit",
livekit_service_url: urlFromConf,
livekit_alias: livekitAlias,
};
logger.log("Using LiveKit transport from config: ", transportFromConf);
return transportFromConf;
}
throw new MatrixRTCTransportMissingError(domain ?? "");
}
export async function makeTransport(
client: MatrixClient,
roomId: string,
): Promise<LivekitTransport> {
const transport = await makeTransportInternal(client, roomId);
// this will call the jwt/sfu/get endpoint to pre create the livekit room.
await getSFUConfigWithOpenID(
client,
transport.livekit_service_url,
transport.livekit_alias,
);
return transport;
}
export interface EnterRTCSessionOptions {
encryptMedia: boolean;
matrixRTCMode: MatrixRTCMode;
}
/**
* TODO! document this function properly
* @param rtcSession
* @param transport
* @param options
*/
export async function enterRTCSession(
rtcSession: MatrixRTCSession,
transport: LivekitTransport,
{ encryptMedia, matrixRTCMode }: EnterRTCSessionOptions,
): Promise<void> {
PosthogAnalytics.instance.eventCallEnded.cacheStartCall(new Date());
PosthogAnalytics.instance.eventCallStarted.track(rtcSession.room.roomId);
// This must be called before we start trying to join the call, as we need to
// have started tracking by the time calls start getting created.
// groupCallOTelMembership?.onJoinCall();
const { features, matrix_rtc_session: matrixRtcSessionConfig } = Config.get();
const useDeviceSessionMemberEvents =
features?.feature_use_device_session_member_events;
const { sendNotificationType: notificationType, callIntent } = getUrlParams();
const multiSFU = matrixRTCMode !== MatrixRTCMode.Legacy;
// Multi-sfu does not need a preferred foci list. just the focus that is actually used.
rtcSession.joinRoomSession(
multiSFU ? [] : [transport],
multiSFU ? transport : undefined,
{
notificationType,
callIntent,
manageMediaKeys: encryptMedia,
...(useDeviceSessionMemberEvents !== undefined && {
useLegacyMemberEvents: !useDeviceSessionMemberEvents,
}),
delayedLeaveEventRestartMs:
matrixRtcSessionConfig?.delayed_leave_event_restart_ms,
delayedLeaveEventDelayMs:
matrixRtcSessionConfig?.delayed_leave_event_delay_ms,
delayedLeaveEventRestartLocalTimeoutMs:
matrixRtcSessionConfig?.delayed_leave_event_restart_local_timeout_ms,
networkErrorRetryMs: matrixRtcSessionConfig?.network_error_retry_ms,
makeKeyDelay: matrixRtcSessionConfig?.wait_for_key_rotation_ms,
membershipEventExpiryMs:
matrixRtcSessionConfig?.membership_event_expiry_ms,
useExperimentalToDeviceTransport: true,
unstableSendStickyEvents: matrixRTCMode === MatrixRTCMode.Matrix_2_0,
},
);
if (widget) {
try {
await widget.api.transport.send(ElementWidgetActions.JoinCall, {});
} catch (e) {
logger.error("Failed to send join action", e);
}
}
}

View File

@@ -0,0 +1,95 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, vi } from "vitest";
import { render, waitFor } from "@testing-library/react";
import type { MatrixClient } from "matrix-js-sdk";
import type { Room as LivekitRoom } from "livekit-client";
import { DeveloperSettingsTab } from "./DeveloperSettingsTab";
// Mock url params hook to avoid environment-dependent snapshot churn.
vi.mock("../UrlParams", () => ({
useUrlParams: (): { mocked: boolean; answer: number } => ({
mocked: true,
answer: 42,
}),
}));
// Provide a minimal mock of a Livekit Room structure used by the component.
function createMockLivekitRoom(
wsUrl: string,
serverInfo: object,
metadata: string,
): { isLocal: boolean; url: string; room: LivekitRoom } {
const mockRoom = {
serverInfo,
metadata,
engine: { client: { ws: { url: wsUrl } } },
} as unknown as LivekitRoom;
return {
isLocal: true,
url: wsUrl,
room: mockRoom,
};
}
// Minimal MatrixClient mock with only the methods used by the component.
function createMockMatrixClient(): MatrixClient {
return {
doesServerSupportUnstableFeature: vi.fn().mockResolvedValue(true), // ensure stickyEventsSupported eventually becomes true
getCrypto: (): { getVersion: () => string } | undefined => ({
getVersion: () => "crypto-1.0.0",
}),
getUserId: () => "@alice:example.org",
getDeviceId: () => "DEVICE123",
} as unknown as MatrixClient;
}
describe("DeveloperSettingsTab", () => {
it("renders and matches snapshot", async () => {
const client = createMockMatrixClient();
const livekitRooms: {
room: LivekitRoom;
url: string;
isLocal?: boolean;
}[] = [
createMockLivekitRoom(
"wss://local-sfu.example.org",
{ region: "local", version: "1.2.3" },
"local-metadata",
),
{
isLocal: false,
url: "wss://remote-sfu.example.org",
room: {
serverInfo: { region: "remote", version: "4.5.6" },
metadata: "remote-metadata",
engine: { client: { ws: { url: "wss://remote-sfu.example.org" } } },
} as unknown as LivekitRoom,
},
];
const { container } = render(
<DeveloperSettingsTab
client={client}
livekitRooms={livekitRooms}
env={{ MY_MOCK_ENV: 10, ENV: "test" } as unknown as ImportMetaEnv}
/>,
);
// Wait for the async sticky events feature check to resolve so the final UI
// (e.g. enabled Matrix_2_0 radio button) appears deterministically.
await waitFor(() =>
expect(client.doesServerSupportUnstableFeature).toHaveBeenCalled(),
);
expect(container).toMatchSnapshot();
});
});

View File

@@ -11,6 +11,7 @@ import {
useCallback,
useEffect,
useMemo,
useId,
useState,
} from "react";
import { useTranslation } from "react-i18next";
@@ -19,6 +20,15 @@ import {
type MatrixClient,
} from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import {
EditInPlace,
Root as Form,
Heading,
HelpMessage,
InlineField,
Label,
RadioControl,
} from "@vector-im/compound-web";
import { FieldRow, InputField } from "../input/Input";
import {
@@ -29,6 +39,7 @@ import {
muteAllAudio as muteAllAudioSetting,
alwaysShowIphoneEarpiece as alwaysShowIphoneEarpieceSetting,
matrixRTCMode as matrixRTCModeSetting,
customLivekitUrl as customLivekitUrlSetting,
MatrixRTCMode,
} from "./settings";
import type { Room as LivekitRoom } from "livekit-client";
@@ -38,9 +49,14 @@ import { useUrlParams } from "../UrlParams";
interface Props {
client: MatrixClient;
livekitRooms?: { room: LivekitRoom; url: string; isLocal?: boolean }[];
env: ImportMetaEnv;
}
export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
export const DeveloperSettingsTab: FC<Props> = ({
client,
livekitRooms,
env,
}) => {
const { t } = useTranslation();
const [duplicateTiles, setDuplicateTiles] = useSetting(duplicateTilesSetting);
const [debugTileLayout, setDebugTileLayout] = useSetting(
@@ -60,6 +76,13 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
}, [client]);
const [matrixRTCMode, setMatrixRTCMode] = useSetting(matrixRTCModeSetting);
const matrixRTCModeRadioGroup = useId();
const onMatrixRTCModeChange = useCallback(
(e: ChangeEvent<HTMLInputElement>) => {
setMatrixRTCMode(e.target.value as MatrixRTCMode);
},
[setMatrixRTCMode],
);
const [showConnectionStats, setShowConnectionStats] = useSetting(
showConnectionStatsSetting,
@@ -69,6 +92,15 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
alwaysShowIphoneEarpieceSetting,
);
const [customLivekitUrl, setCustomLivekitUrl] = useSetting(
customLivekitUrlSetting,
);
const [customLivekitUrlTextBuffer, setCustomLivekitUrlTextBuffer] =
useState(customLivekitUrl);
useEffect(() => {
setCustomLivekitUrlTextBuffer(customLivekitUrl);
}, [customLivekitUrl]);
const [muteAllAudio, setMuteAllAudio] = useSetting(muteAllAudioSetting);
const urlParams = useUrlParams();
@@ -142,52 +174,6 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
}
/>
</FieldRow>
<FieldRow>
<InputField
id="matrixRTCMode"
type="checkbox"
label={t("developer_mode.matrixRTCMode.Legacy.label")}
description={t("developer_mode.matrixRTCMode.Legacy.description")}
checked={matrixRTCMode === MatrixRTCMode.Legacy}
onChange={useCallback(
(event: ChangeEvent<HTMLInputElement>) => {
if (event.target.checked) setMatrixRTCMode(MatrixRTCMode.Legacy);
},
[setMatrixRTCMode],
)}
/>
<InputField
id="matrixRTCMode"
type="checkbox"
label={t("developer_mode.matrixRTCMode.Comptibility.label")}
description={t(
"developer_mode.matrixRTCMode.Comptibility.description",
)}
checked={matrixRTCMode === MatrixRTCMode.Compatibil}
onChange={useCallback(
(event: ChangeEvent<HTMLInputElement>) => {
if (event.target.checked)
setMatrixRTCMode(MatrixRTCMode.Compatibil);
},
[setMatrixRTCMode],
)}
/>
<InputField
id="matrixRTCMode"
type="checkbox"
label={t("developer_mode.matrixRTCMode.Matrix_2_0.label")}
description={t("developer_mode.matrixRTCMode.Matrix_2_0.description")}
disabled={!stickyEventsSupported}
checked={matrixRTCMode === MatrixRTCMode.Matrix_2_0}
onChange={useCallback(
(event: ChangeEvent<HTMLInputElement>) => {
if (event.target.checked)
setMatrixRTCMode(MatrixRTCMode.Matrix_2_0);
},
[setMatrixRTCMode],
)}
/>
</FieldRow>
<FieldRow>
<InputField
id="showConnectionStats"
@@ -230,6 +216,93 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
)}
/>{" "}
</FieldRow>
<EditInPlace
onSubmit={(e) => e.preventDefault()}
helpLabel={
customLivekitUrl === null
? t("developer_mode.custom_livekit_url.from_config")
: t("developer_mode.custom_livekit_url.current_url") +
customLivekitUrl
}
label={t("developer_mode.custom_livekit_url.label")}
saveButtonLabel={t("developer_mode.custom_livekit_url.save")}
savingLabel={t("developer_mode.custom_livekit_url.saving")}
cancelButtonLabel={t("developer_mode.custom_livekit_url.reset")}
onSave={useCallback(
(e: React.FormEvent<HTMLFormElement>) => {
setCustomLivekitUrl(
customLivekitUrlTextBuffer === ""
? null
: customLivekitUrlTextBuffer,
);
},
[setCustomLivekitUrl, customLivekitUrlTextBuffer],
)}
value={customLivekitUrlTextBuffer ?? ""}
onChange={useCallback(
(event: ChangeEvent<HTMLInputElement>): void => {
setCustomLivekitUrlTextBuffer(event.target.value);
},
[setCustomLivekitUrlTextBuffer],
)}
onCancel={useCallback(
(e: React.FormEvent<HTMLFormElement>) => {
setCustomLivekitUrl(null);
},
[setCustomLivekitUrl],
)}
/>
<Heading as="h3" type="body" weight="semibold" size="lg">
{t("developer_mode.matrixRTCMode.title")}
</Heading>
<Form>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Legacy}
value={MatrixRTCMode.Legacy}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Legacy.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Legacy.description")}
</HelpMessage>
</InlineField>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Compatibil}
value={MatrixRTCMode.Compatibil}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Comptibility.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Comptibility.description")}
</HelpMessage>
</InlineField>
<InlineField
name={matrixRTCModeRadioGroup}
control={
<RadioControl
checked={matrixRTCMode === MatrixRTCMode.Matrix_2_0}
value={MatrixRTCMode.Matrix_2_0}
disabled={!stickyEventsSupported}
onChange={onMatrixRTCModeChange}
/>
}
>
<Label>{t("developer_mode.matrixRTCMode.Matrix_2_0.label")}</Label>
<HelpMessage>
{t("developer_mode.matrixRTCMode.Matrix_2_0.description")}
</HelpMessage>
</InlineField>
</Form>
{livekitRooms?.map((livekitRoom) => (
<>
<h3>
@@ -251,7 +324,7 @@ export const DeveloperSettingsTab: FC<Props> = ({ client, livekitRooms }) => {
</>
))}
<p>{t("developer_mode.environment_variables")}</p>
<pre>{JSON.stringify(import.meta.env, null, 2)}</pre>
<pre>{JSON.stringify(env, null, 2)}</pre>
<p>{t("developer_mode.url_params")}</p>
<pre>{JSON.stringify(urlParams, null, 2)}</pre>
</>

View File

@@ -209,7 +209,11 @@ export const SettingsModal: FC<Props> = ({
key: "developer",
name: t("settings.developer_tab_title"),
content: (
<DeveloperSettingsTab client={client} livekitRooms={livekitRooms} />
<DeveloperSettingsTab
env={import.meta.env}
client={client}
livekitRooms={livekitRooms}
/>
),
};

View File

@@ -0,0 +1,411 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html
exports[`DeveloperSettingsTab > renders and matches snapshot 1`] = `
<div>
<p>
Hostname: localhost
</p>
<p>
Element Call version: dev
</p>
<p>
Crypto version: crypto-1.0.0
</p>
<p>
Matrix ID: @alice:example.org
</p>
<p>
Device ID: DEVICE123
</p>
<div
class="fieldRow"
>
<div
class="field inputField"
>
<input
aria-describedby="«r1»"
id="duplicateTiles"
min="0"
type="number"
value="0"
/>
<label
for="duplicateTiles"
>
Number of additional tile copies per participant
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r2»"
id="debugTileLayout"
type="checkbox"
/>
<label
for="debugTileLayout"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Debug tile layout
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r3»"
id="showConnectionStats"
type="checkbox"
/>
<label
for="showConnectionStats"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Show connection statistics
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r4»"
id="muteAllAudio"
type="checkbox"
/>
<label
for="muteAllAudio"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Mute all audio (participants, reactions, join sounds)
</label>
</div>
</div>
<div
class="fieldRow"
>
<div
class="field checkboxField"
>
<input
aria-describedby="«r5»"
id="alwaysShowIphoneEarpiece"
type="checkbox"
/>
<label
for="alwaysShowIphoneEarpiece"
>
<div
class="checkbox"
>
<svg
fill="none"
height="24"
stroke="#000"
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
viewBox="0 0 24 24"
width="24"
xmlns="http://www.w3.org/2000/svg"
>
<path
d="m20 6-11 11-5-5"
/>
</svg>
</div>
Show iPhone earpiece option on all platforms
</label>
</div>
</div>
<form
class="_root_19upo_16"
>
<div
class="_field_19upo_26"
>
<label
class="_label_19upo_59"
for="radix-«r6»"
>
Custom Livekit-url
</label>
<div
class="_controls_17lij_8"
>
<input
aria-describedby="radix-«r7»"
class="_control_sqdq4_10"
id="radix-«r6»"
name="input"
title=""
value=""
/>
</div>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«r7»"
>
Currently, no overwrite is set. Url from well-known or config is used.
</span>
</div>
</form>
<h3
class="_typography_6v6n8_153 _font-body-lg-semibold_6v6n8_74"
>
MatrixRTC mode
</h3>
<form
class="_root_19upo_16"
>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
checked=""
class="_input_1e0uz_18"
id="radix-«r8»"
name="«r0»"
title=""
type="radio"
value="legacy"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«r8»"
>
Legacy: state events & oldest membership SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«r9»"
>
Compatible with old versions of EC that do not support multi SFU
</span>
</div>
</div>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
class="_input_1e0uz_18"
id="radix-«ra»"
name="«r0»"
title=""
type="radio"
value="compatibil"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«ra»"
>
Compatibility: state events & multi SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«rb»"
>
Compatible with homeservers that do not support sticky events (but all other EC clients are v0.17.0 or later)
</span>
</div>
</div>
<div
class="_inline-field_19upo_32"
>
<div
class="_inline-field-control_19upo_44"
>
<div
class="_container_1e0uz_10"
>
<input
aria-describedby="radix-«r9» radix-«rb» radix-«rd»"
class="_input_1e0uz_18"
id="radix-«rc»"
name="«r0»"
title=""
type="radio"
value="matrix_2_0"
/>
<div
class="_ui_1e0uz_19"
/>
</div>
</div>
<div
class="_inline-field-body_19upo_38"
>
<label
class="_label_19upo_59"
for="radix-«rc»"
>
Matrix 2.0: sticky events & multi SFU
</label>
<span
class="_message_19upo_85 _help-message_19upo_91"
id="radix-«rd»"
>
Compatible only with homservers supporting sticky events and all EC clients v0.17.0 or later
</span>
</div>
</div>
</form>
<h3>
LiveKit SFU: wss://local-sfu.example.org
</h3>
<p>
ws-url:
wss://local-sfu.example.org/
</p>
<p>
LiveKit Server Info
(
local
)
</p>
<pre
class="pre"
>
{
"region": "local",
"version": "1.2.3"
}
local-metadata
</pre>
<h3>
LiveKit SFU: wss://remote-sfu.example.org
</h3>
<p>
LiveKit Server Info
(
remote
)
</p>
<pre
class="pre"
>
{
"region": "remote",
"version": "4.5.6"
}
remote-metadata
</pre>
<p>
Environment variables
</p>
<pre>
{
"MY_MOCK_ENV": 10,
"ENV": "test"
}
</pre>
<p>
URL parameters
</p>
<pre>
{
"mocked": true,
"answer": 42
}
</pre>
</div>
`;

View File

@@ -134,3 +134,8 @@ export const matrixRTCMode = new Setting<MatrixRTCMode>(
"matrix-rtc-mode",
MatrixRTCMode.Legacy,
);
export const customLivekitUrl = new Setting<string | null>(
"custom-livekit-url",
null,
);

View File

@@ -1,59 +0,0 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { catchError, from, map, type Observable, of, startWith } from "rxjs";
/**
* Data that may need to be loaded asynchronously.
*
* This type is for when you need to represent the current state of an operation
* involving Promises as **immutable data**. See the async$ function below.
*/
export type Async<A> =
| { state: "loading" }
| { state: "error"; value: Error }
| { state: "ready"; value: A };
export const loading: Async<never> = { state: "loading" };
export function error(value: Error): Async<never> {
return { state: "error", value };
}
export function ready<A>(value: A): Async<A> {
return { state: "ready", value };
}
/**
* Turn a Promise into an Observable async value. The Observable will have the
* value "loading" while the Promise is pending, "ready" when the Promise
* resolves, and "error" when the Promise rejects.
*/
export function async$<A>(promise: Promise<A>): Observable<Async<A>> {
return from(promise).pipe(
map(ready),
startWith(loading),
catchError((e: unknown) =>
of(error((e as Error) ?? new Error("Unknown error"))),
),
);
}
/**
* If the async value is ready, apply the given function to the inner value.
*/
export function mapAsync<A, B>(
async: Async<A>,
project: (value: A) => B,
): Async<B> {
return async.state === "ready" ? ready(project(async.value)) : async;
}
export function unwrapAsync<A>(fallback: A): (async: Async<A>) => A {
return (async: Async<A>) => {
return async.state === "ready" ? async.value : fallback;
};
}

View File

@@ -16,11 +16,10 @@ import { BehaviorSubject } from "rxjs";
* distinction between Behaviors and Observables, see
* https://monoid.dk/post/behaviors-and-streams-why-both/.
*/
export type Behavior<T> = Omit<BehaviorSubject<T>, "next" | "observers">;
export type BehaviorWithEpoch<T> = Behavior<T> & {
pipeEpoch(): Behavior<{ value: T; epoch: number }>;
};
export type Behavior<T> = Omit<
BehaviorSubject<T>,
"next" | "observers" | "error"
>;
/**
* Creates a Behavior which never changes in value.

View File

@@ -0,0 +1,354 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type ICallNotifyContent,
type IRTCNotificationContent,
} from "matrix-js-sdk/lib/matrixrtc";
import { describe, it } from "vitest";
import {
EventType,
type IEvent,
type IRoomTimelineData,
MatrixEvent,
type Room,
} from "matrix-js-sdk";
import { withTestScheduler } from "../../utils/test";
import {
aliceRtcMember,
local,
localRtcMember,
} from "../../utils/test-fixtures";
import {
createCallNotificationLifecycle$,
type Props as CallNotificationLifecycleProps,
} from "./CallNotificationLifecycle";
import { trackEpoch } from "../ObservableScope";
const mockLegacyRingEvent = {} as { event_id: string } & ICallNotifyContent;
function mockRingEvent(
eventId: string,
lifetimeMs: number | undefined,
sender = local.userId,
): { event_id: string } & IRTCNotificationContent {
return {
event_id: eventId,
...(lifetimeMs === undefined ? {} : { lifetime: lifetimeMs }),
notification_type: "ring",
sender,
} as unknown as { event_id: string } & IRTCNotificationContent;
}
describe("waitForCallPickup$", () => {
it("unknown -> ringing -> timeout when notified and nobody joins", () => {
withTestScheduler(({ scope, expectObservable, behavior, hot }) => {
// No one ever joins (only local user)
const props: CallNotificationLifecycleProps = {
scope,
memberships$: scope.behavior(
behavior("a", { a: [] }).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$notif1", 30), mockLegacyRingEvent],
}),
receivedDecline$: hot(""),
options: {
waitForCallPickup: true,
autoLeaveWhenOthersLeft: false,
},
localUser: localRtcMember,
};
const lifecycle = createCallNotificationLifecycle$(props);
expectObservable(lifecycle.callPickupState$).toBe("a 9ms b 29ms c", {
a: "unknown",
b: "ringing",
c: "timeout",
});
});
});
it("ringing -> success if someone joins before timeout is reached", () => {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
// Someone joins at 20ms (both LiveKit participant and MatrixRTC member)
const props: CallNotificationLifecycleProps = {
scope,
memberships$: scope.behavior(
behavior("a 19ms b", {
a: [localRtcMember],
b: [localRtcMember, aliceRtcMember],
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("5ms a", {
a: [mockRingEvent("$notif2", 100), mockLegacyRingEvent],
}),
receivedDecline$: hot(""),
options: {
waitForCallPickup: true,
autoLeaveWhenOthersLeft: false,
},
localUser: localRtcMember,
};
const lifecycle = createCallNotificationLifecycle$(props);
expectObservable(lifecycle.callPickupState$).toBe("a 4ms b 14ms c", {
a: "unknown",
b: "ringing",
c: "success",
});
});
});
it("success when someone joins before we notify", () => {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
// Someone joins at 20ms (both LiveKit participant and MatrixRTC member)
const props: CallNotificationLifecycleProps = {
scope,
memberships$: scope.behavior(
behavior("a 9ms b", {
a: [localRtcMember],
b: [localRtcMember, aliceRtcMember],
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("20ms a", {
a: [mockRingEvent("$notif2", 50), mockLegacyRingEvent],
}),
receivedDecline$: hot(""),
options: {
waitForCallPickup: true,
autoLeaveWhenOthersLeft: false,
},
localUser: localRtcMember,
};
const lifecycle = createCallNotificationLifecycle$(props);
expectObservable(lifecycle.callPickupState$).toBe("a 9ms b", {
a: "unknown",
b: "success",
});
});
});
it("notify without lifetime -> immediate timeout", () => {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
// Someone joins at 20ms (both LiveKit participant and MatrixRTC member)
const props: CallNotificationLifecycleProps = {
scope,
memberships$: scope.behavior(
behavior("a", {
a: [localRtcMember],
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$notif2", undefined), mockLegacyRingEvent],
}),
receivedDecline$: hot(""),
options: {
waitForCallPickup: true,
autoLeaveWhenOthersLeft: false,
},
localUser: localRtcMember,
};
const lifecycle = createCallNotificationLifecycle$(props);
expectObservable(lifecycle.callPickupState$).toBe("a 9ms b", {
a: "unknown",
b: "timeout",
});
});
});
it("stays null when waitForCallPickup=false", () => {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
// Someone joins at 20ms (both LiveKit participant and MatrixRTC member)
const validProps: CallNotificationLifecycleProps = {
scope,
memberships$: scope.behavior(
behavior("a--b", {
a: [localRtcMember],
b: [localRtcMember, aliceRtcMember],
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$notif5", 30), mockLegacyRingEvent],
}),
receivedDecline$: hot(""),
options: {
waitForCallPickup: true,
autoLeaveWhenOthersLeft: false,
},
localUser: localRtcMember,
};
const propsDeactivated = {
...validProps,
options: {
...validProps.options,
waitForCallPickup: false,
},
};
const lifecycle = createCallNotificationLifecycle$(propsDeactivated);
expectObservable(lifecycle.callPickupState$).toBe("n", {
n: null,
});
const lifecycleReference = createCallNotificationLifecycle$(validProps);
expectObservable(lifecycleReference.callPickupState$).toBe("u--s", {
u: "unknown",
s: "success",
});
});
});
it("decline before timeout window ends -> decline", () => {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
// Someone joins at 20ms (both LiveKit participant and MatrixRTC member)
const props: CallNotificationLifecycleProps = {
scope,
memberships$: scope.behavior(
behavior("a", {
a: [localRtcMember],
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$decl1", 50), mockLegacyRingEvent],
}),
receivedDecline$: hot("40ms d", {
d: [
new MatrixEvent({
type: EventType.RTCDecline,
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$decl1",
},
},
}),
{} as Room,
undefined,
false,
{} as IRoomTimelineData,
],
}),
options: {
waitForCallPickup: true,
autoLeaveWhenOthersLeft: false,
},
localUser: localRtcMember,
};
const lifecycle = createCallNotificationLifecycle$(props);
expectObservable(lifecycle.callPickupState$).toBe("a 9ms b 29ms e", {
a: "unknown",
b: "ringing",
e: "decline",
});
});
});
it("decline after timeout window ends -> stays timeout", () => {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
// Someone joins at 20ms (both LiveKit participant and MatrixRTC member)
const props: CallNotificationLifecycleProps = {
scope,
memberships$: scope.behavior(
behavior("a", {
a: [localRtcMember],
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$decl", 20), mockLegacyRingEvent],
}),
receivedDecline$: hot("40ms d", {
d: [
new MatrixEvent({
type: EventType.RTCDecline,
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: "$decl",
},
},
}),
{} as Room,
undefined,
false,
{} as IRoomTimelineData,
],
}),
options: {
waitForCallPickup: true,
autoLeaveWhenOthersLeft: false,
},
localUser: localRtcMember,
};
const lifecycle = createCallNotificationLifecycle$(props);
expectObservable(lifecycle.callPickupState$, "50ms !").toBe(
"a 9ms b 19ms e",
{
a: "unknown",
b: "ringing",
e: "timeout",
},
);
});
});
//
function testStaysRinging(
declineEvent: Partial<IEvent>,
expectDecline: boolean,
): void {
withTestScheduler(({ scope, hot, behavior, expectObservable }) => {
// Someone joins at 20ms (both LiveKit participant and MatrixRTC member)
const props: CallNotificationLifecycleProps = {
scope,
memberships$: scope.behavior(
behavior("a", {
a: [localRtcMember],
}).pipe(trackEpoch()),
),
sentCallNotification$: hot("10ms a", {
a: [mockRingEvent("$right", 50), mockLegacyRingEvent],
}),
receivedDecline$: hot("20ms d", {
d: [
new MatrixEvent(declineEvent),
{} as Room,
undefined,
false,
{} as IRoomTimelineData,
],
}),
options: {
waitForCallPickup: true,
autoLeaveWhenOthersLeft: false,
},
localUser: localRtcMember,
};
const lifecycle = createCallNotificationLifecycle$(props);
const marbles = expectDecline ? "a 9ms b 9ms d" : "a 9ms b";
expectObservable(lifecycle.callPickupState$, "21ms !").toBe(marbles, {
a: "unknown",
b: "ringing",
d: "decline",
});
});
}
const reference = (refId?: string, sender?: string): Partial<IEvent> => ({
event_id: "$decline",
type: EventType.RTCDecline,
sender: sender ?? "@other:example.org",
content: {
"m.relates_to": {
rel_type: "m.reference",
event_id: refId ?? "$right",
},
},
});
it("decline reference works", () => {
testStaysRinging(reference(), true);
});
it("decline with wrong id is ignored (stays ringing)", () => {
testStaysRinging(reference("$wrong"), false);
});
it("decline with wrong id is ignored (stays ringing)", () => {
testStaysRinging(reference(undefined, local.userId), false);
});
});

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -70,7 +70,7 @@ export function createReceivedDecline$(
).pipe(filter(([event]) => event.getType() === EventType.RTCDecline));
}
interface Props {
export interface Props {
scope: ObservableScope;
memberships$: Behavior<Epoch<CallMembership[]>>;
sentCallNotification$: Observable<CallNotificationWrapper | null>;
@@ -109,7 +109,6 @@ export function createCallNotificationLifecycle$({
callPickupState$: Behavior<CallPickupState>;
autoLeave$: Observable<AutoLeaveReason>;
} {
// TODO convert all ring and all others left logic into one callLifecycleTracker$(didSendCallNotification$,matrixLivekitItem$): {autoLeave$,callPickupState$}
const allOthersLeft$ = memberships$.pipe(
pairwise(),
filter(
@@ -170,7 +169,8 @@ export function createCallNotificationLifecycle$({
event.getRelation()?.rel_type === "m.reference" &&
event.getRelation()?.event_id ===
notificationEvent.event_id &&
event.getSender() !== localUser.userId,
event.getSender() !== localUser.userId &&
callPickupState$.value !== "timeout",
),
),
),
@@ -186,11 +186,12 @@ export function createCallNotificationLifecycle$({
? combineLatest(
[someoneElseJoined$, remoteRingState$],
(someoneElseJoined, ring) => {
if (someoneElseJoined) {
if (someoneElseJoined.value === true) {
return "success" as const;
}
// Show the ringing state of the most recent ringing attempt.
// as long as we have not yet sent an RTC notification event, ring will be null -> callPickupState$ = unknown.
// as long as we have not yet sent an RTC notification event or noone else joined,
// ring will be null -> callPickupState$ = unknown.
return ring ?? ("unknown" as const);
},
)

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,193 @@
/*
Copyright 2025 Element Corp.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
ConnectionState,
type LocalParticipant,
type Participant,
ParticipantEvent,
type RemoteParticipant,
type Room as LivekitRoom,
} from "livekit-client";
import { SyncState } from "matrix-js-sdk/lib/sync";
import { BehaviorSubject, type Observable, map, of } from "rxjs";
import { onTestFinished, vi } from "vitest";
import { ClientEvent, type MatrixClient } from "matrix-js-sdk";
import EventEmitter from "events";
import * as ComponentsCore from "@livekit/components-core";
import type { CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { E2eeType } from "../../e2ee/e2eeType";
import { type RaisedHandInfo, type ReactionInfo } from "../../reactions";
import {
type CallViewModel,
createCallViewModel$,
type CallViewModelOptions,
} from "./CallViewModel";
import {
mockConfig,
mockLivekitRoom,
mockLocalParticipant,
mockMatrixRoom,
mockMatrixRoomMember,
mockMediaDevices,
mockMuteStates,
MockRTCSession,
testScope,
} from "../../utils/test";
import {
alice,
aliceDoppelganger,
bob,
bobZeroWidthSpace,
daveRTL,
daveRTLRtcMember,
local,
localRtcMember,
} from "../../utils/test-fixtures";
import { type Behavior, constant } from "../Behavior";
import { type ProcessorState } from "../../livekit/TrackProcessorContext";
import { type MediaDevices } from "../MediaDevices";
mockConfig({
livekit: { livekit_service_url: "http://my-default-service-url.com" },
});
const carol = local;
const dave = mockMatrixRoomMember(daveRTLRtcMember, { rawDisplayName: "Dave" });
const roomMembers = new Map(
[alice, aliceDoppelganger, bob, bobZeroWidthSpace, carol, dave, daveRTL].map(
(p) => [p.userId, p],
),
);
export interface CallViewModelInputs {
remoteParticipants$: Behavior<RemoteParticipant[]>;
rtcMembers$: Behavior<Partial<CallMembership>[]>;
livekitConnectionState$: Behavior<ConnectionState>;
speaking: Map<Participant, Observable<boolean>>;
mediaDevices: MediaDevices;
initialSyncState: SyncState;
}
const localParticipant = mockLocalParticipant({ identity: "" });
export function withCallViewModel(
{
remoteParticipants$ = constant([]),
rtcMembers$ = constant([localRtcMember]),
livekitConnectionState$: connectionState$ = constant(
ConnectionState.Connected,
),
speaking = new Map(),
mediaDevices = mockMediaDevices({}),
initialSyncState = SyncState.Syncing,
}: Partial<CallViewModelInputs> = {},
continuation: (
vm: CallViewModel,
rtcSession: MockRTCSession,
subjects: { raisedHands$: BehaviorSubject<Record<string, RaisedHandInfo>> },
setSyncState: (value: SyncState) => void,
) => void,
options: CallViewModelOptions = {
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
autoLeaveWhenOthersLeft: false,
},
): void {
let syncState = initialSyncState;
const setSyncState = (value: SyncState): void => {
const prev = syncState;
syncState = value;
room.client.emit(ClientEvent.Sync, value, prev);
};
const room = mockMatrixRoom({
client: new (class extends EventEmitter {
public getUserId(): string | undefined {
return localRtcMember.userId;
}
public getDeviceId(): string {
return localRtcMember.deviceId;
}
public getDomain(): string {
return "example.com";
}
public getSyncState(): SyncState {
return syncState;
}
})() as Partial<MatrixClient> as MatrixClient,
getMembers: () => Array.from(roomMembers.values()),
getMembersWithMembership: () => Array.from(roomMembers.values()),
});
const rtcSession = new MockRTCSession(room, []).withMemberships(rtcMembers$);
const participantsSpy = vi
.spyOn(ComponentsCore, "connectedParticipantsObserver")
.mockReturnValue(remoteParticipants$);
const mediaSpy = vi
.spyOn(ComponentsCore, "observeParticipantMedia")
.mockImplementation((p) =>
of({ participant: p } as Partial<
ComponentsCore.ParticipantMedia<LocalParticipant>
> as ComponentsCore.ParticipantMedia<LocalParticipant>),
);
const eventsSpy = vi
.spyOn(ComponentsCore, "observeParticipantEvents")
.mockImplementation((p, ...eventTypes) => {
if (eventTypes.includes(ParticipantEvent.IsSpeakingChanged)) {
return (speaking.get(p) ?? of(false)).pipe(
map((s): Participant => ({ ...p, isSpeaking: s }) as Participant),
);
} else {
return of(p);
}
});
const roomEventSelectorSpy = vi
.spyOn(ComponentsCore, "roomEventSelector")
.mockImplementation((_room, _eventType) => of());
const muteStates = mockMuteStates();
const raisedHands$ = new BehaviorSubject<Record<string, RaisedHandInfo>>({});
const reactions$ = new BehaviorSubject<Record<string, ReactionInfo>>({});
const vm = createCallViewModel$(
testScope(),
rtcSession.asMockedSession(),
room,
mediaDevices,
muteStates,
{
...options,
livekitRoomFactory: (): LivekitRoom =>
mockLivekitRoom({
localParticipant,
disconnect: async () => Promise.resolve(),
setE2EEEnabled: async () => Promise.resolve(),
}),
connectionState$,
},
raisedHands$,
reactions$,
new BehaviorSubject<ProcessorState>({
processor: undefined,
supported: undefined,
}),
);
onTestFinished(() => {
participantsSpy.mockRestore();
mediaSpy.mockRestore();
eventsSpy.mockRestore();
roomEventSelectorSpy.mockRestore();
});
continuation(vm, rtcSession, { raisedHands$: raisedHands$ }, setSyncState);
}

View File

@@ -0,0 +1,202 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { EventEmitter } from "events";
import { ClientEvent, SyncState } from "matrix-js-sdk";
import { MembershipManagerEvent, Status } from "matrix-js-sdk/lib/matrixrtc";
import { ObservableScope } from "../../ObservableScope";
import { createHomeserverConnected$ } from "./HomeserverConnected";
/**
* Minimal stub of a Matrix client sufficient for our tests:
```
createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
)
```
*/
class MockMatrixClient extends EventEmitter {
private syncState: SyncState;
public constructor(initial: SyncState) {
super();
this.syncState = initial;
}
public setSyncState(state: SyncState): void {
this.syncState = state;
// Matrix's Sync event in createHomeserverConnected$ expects [SyncState]
this.emit(ClientEvent.Sync, [state]);
}
public getSyncState(): SyncState {
return this.syncState;
}
}
/**
* Minimal stub of MatrixRTCSession (membership manager):
```
createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
)
```
*/
class MockMatrixRTCSession extends EventEmitter {
public membershipStatus: Status;
public probablyLeft: boolean;
public constructor(props: {
membershipStatus: Status;
probablyLeft: boolean;
}) {
super();
this.membershipStatus = props.membershipStatus;
this.probablyLeft = props.probablyLeft;
}
public setMembershipStatus(status: Status): void {
this.membershipStatus = status;
this.emit(MembershipManagerEvent.StatusChanged);
}
public setProbablyLeft(flag: boolean): void {
this.probablyLeft = flag;
this.emit(MembershipManagerEvent.ProbablyLeft);
}
}
describe("createHomeserverConnected$", () => {
let scope: ObservableScope;
let client: MockMatrixClient;
let session: MockMatrixRTCSession;
beforeEach(() => {
scope = new ObservableScope();
client = new MockMatrixClient(SyncState.Error); // start disconnected
session = new MockMatrixRTCSession({
membershipStatus: Status.Disconnected,
probablyLeft: false,
});
});
afterEach(() => {
scope.end();
});
// LLM generated test cases. They are a bit overkill but I improved the mocking so it is
// easy enough to read them so I think they can stay.
it("is false when sync state is not Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
expect(hsConnected$.value).toBe(false);
});
it("remains false while membership status is not Connected even if sync is Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false); // membership still disconnected
});
it("is false when membership status transitions to Connected but ProbablyLeft is true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Make sync loop OK
client.setSyncState(SyncState.Syncing);
// Indicate probable leave before connection
session.setProbablyLeft(true);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(false);
});
it("becomes true only when all three conditions are satisfied", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// 1. Sync loop connected
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false); // not yet membership connected
// 2. Membership connected
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true); // probablyLeft is false
});
it("drops back to false when sync loop leaves Syncing", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Reach connected state
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
// Sync loop error => should flip false
client.setSyncState(SyncState.Error);
expect(hsConnected$.value).toBe(false);
});
it("drops back to false when membership status becomes disconnected", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setMembershipStatus(Status.Disconnected);
expect(hsConnected$.value).toBe(false);
});
it("drops to false when ProbablyLeft is emitted after being true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
});
it("recovers to true if ProbablyLeft becomes false again while other conditions remain true", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
client.setSyncState(SyncState.Syncing);
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
// Simulate clearing the flag (in realistic scenario membership manager would update)
session.setProbablyLeft(false);
expect(hsConnected$.value).toBe(true);
});
it("composite sequence reflects each individual failure reason", () => {
const hsConnected$ = createHomeserverConnected$(scope, client, session);
// Initially false (sync error + disconnected + not probably left)
expect(hsConnected$.value).toBe(false);
// Fix sync only
client.setSyncState(SyncState.Syncing);
expect(hsConnected$.value).toBe(false);
// Fix membership
session.setMembershipStatus(Status.Connected);
expect(hsConnected$.value).toBe(true);
// Introduce probablyLeft -> false
session.setProbablyLeft(true);
expect(hsConnected$.value).toBe(false);
// Restore notProbablyLeft -> true again
session.setProbablyLeft(false);
expect(hsConnected$.value).toBe(true);
// Drop sync -> false
client.setSyncState(SyncState.Error);
expect(hsConnected$.value).toBe(false);
});
});

View File

@@ -0,0 +1,85 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
MembershipManagerEvent,
Status,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { ClientEvent, type MatrixClient, SyncState } from "matrix-js-sdk";
import { fromEvent, startWith, map, tap, type Observable } from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type ObservableScope } from "../../ObservableScope";
import { type Behavior } from "../../Behavior";
import { and$ } from "../../../utils/observable";
import { type NodeStyleEventEmitter } from "../../../utils/test";
/**
* Logger instance (scoped child) for homeserver connection updates.
*/
const logger = rootLogger.getChild("[HomeserverConnected]");
/**
* Behavior representing whether we consider ourselves connected to the Matrix homeserver
* for the purposes of a MatrixRTC session.
*
* Becomes FALSE if ANY sub-condition is fulfilled:
* 1. Sync loop is not in SyncState.Syncing
* 2. membershipStatus !== Status.Connected
* 3. probablyLeft === true
*/
export function createHomeserverConnected$(
scope: ObservableScope,
client: NodeStyleEventEmitter & Pick<MatrixClient, "getSyncState">,
matrixRTCSession: NodeStyleEventEmitter &
Pick<MatrixRTCSession, "membershipStatus" | "probablyLeft">,
): Behavior<boolean> {
const syncing$ = (
fromEvent(client, ClientEvent.Sync) as Observable<[SyncState]>
).pipe(
startWith([client.getSyncState()]),
map(([state]) => state === SyncState.Syncing),
);
const membershipConnected$ = fromEvent(
matrixRTCSession,
MembershipManagerEvent.StatusChanged,
).pipe(
startWith(null),
map(() => matrixRTCSession.membershipStatus === Status.Connected),
);
// This is basically notProbablyLeft$
//
// probablyLeft is computed by a local timer that mimics the server delayed event.
// If we locally predict our server event timed out. We consider ourselves as probablyLeft
// even though we might not yet have received the delayed event leave.
//
// If that is not the case we certainly still have a valid membership on the matrix network
// independet if the sync currently works.
const certainlyConnected$ = fromEvent(
matrixRTCSession,
MembershipManagerEvent.ProbablyLeft,
).pipe(
startWith(null),
map(() => matrixRTCSession.probablyLeft !== true),
);
const connectedCombined$ = and$(
syncing$,
membershipConnected$,
certainlyConnected$,
).pipe(
tap((connected) => {
logger.info(`Homeserver connected update: ${connected}`);
}),
);
return scope.behavior(connectedCombined$);
}

View File

@@ -0,0 +1,229 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2024 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type LivekitTransport,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
import { describe, expect, it, vi } from "vitest";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { map } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { MatrixRTCMode } from "../../../settings/settings";
import {
mockConfig,
mockMuteStates,
withTestScheduler,
} from "../../../utils/test";
import {
createLocalMembership$,
enterRTCSession,
LivekitState,
} from "./LocalMembership";
import { MatrixRTCTransportMissingError } from "../../../utils/errors";
import { Epoch } from "../../ObservableScope";
import { constant } from "../../Behavior";
import { ConnectionManagerData } from "../remoteMembers/ConnectionManager";
import { type Publisher } from "./Publisher";
const MATRIX_RTC_MODE = MatrixRTCMode.Legacy;
const getUrlParams = vi.hoisted(() => vi.fn(() => ({})));
vi.mock("../../../UrlParams", () => ({ getUrlParams }));
describe("LocalMembership", () => {
describe("enterRTCSession", () => {
it("It joins the correct Session", async () => {
const focusFromOlderMembership = {
type: "livekit",
livekit_service_url: "http://my-oldest-member-service-url.com",
livekit_alias: "my-oldest-member-service-alias",
};
const focusConfigFromWellKnown = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
};
const focusConfigFromWellKnown2 = {
type: "livekit",
livekit_service_url: "http://my-well-known-service-url2.com",
};
const clientWellKnown = {
"org.matrix.msc4143.rtc_foci": [
focusConfigFromWellKnown,
focusConfigFromWellKnown2,
],
};
mockConfig({
livekit: { livekit_service_url: "http://my-default-service-url.com" },
});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockImplementation(
async (domain) => {
if (domain === "example.org") {
return Promise.resolve(clientWellKnown);
}
return Promise.resolve({});
},
);
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn().mockReturnValue(focusFromOlderMembership),
getOldestMembership: vi.fn().mockReturnValue({
getPreferredFoci: vi.fn().mockReturnValue([focusFromOlderMembership]),
}),
joinRoomSession: vi.fn(),
}) as unknown as MatrixRTCSession;
await enterRTCSession(
mockedSession,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
);
expect(mockedSession.joinRoomSession).toHaveBeenLastCalledWith(
[
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
],
undefined,
expect.objectContaining({
manageMediaKeys: true,
useLegacyMemberEvents: false,
}),
);
});
it("It should not fail with configuration error if homeserver config has livekit url but not fallback", async () => {
mockConfig({});
vi.spyOn(AutoDiscovery, "getRawClientConfig").mockResolvedValue({
"org.matrix.msc4143.rtc_foci": [
{
type: "livekit",
livekit_service_url: "http://my-well-known-service-url.com",
},
],
});
const mockedSession = vi.mocked({
room: {
roomId: "roomId",
client: {
getDomain: vi.fn().mockReturnValue("example.org"),
getOpenIdToken: vi.fn().mockResolvedValue({
access_token: "ACCCESS_TOKEN",
token_type: "Bearer",
matrix_server_name: "localhost",
expires_in: 10000,
}),
},
},
memberships: [],
getFocusInUse: vi.fn(),
joinRoomSession: vi.fn(),
}) as unknown as MatrixRTCSession;
await enterRTCSession(
mockedSession,
{
livekit_alias: "roomId",
livekit_service_url: "http://my-well-known-service-url.com",
type: "livekit",
},
{
encryptMedia: true,
matrixRTCMode: MATRIX_RTC_MODE,
},
);
});
});
const defaultCreateLocalMemberValues = {
options: constant({
encryptMedia: false,
matrixRTCMode: MatrixRTCMode.Matrix_2_0,
}),
matrixRTCSession: {
updateCallIntent: () => {},
leaveRoomSession: () => {},
} as unknown as MatrixRTCSession,
muteStates: mockMuteStates(),
isHomeserverConnected: constant(true),
trackProcessorState$: constant({
supported: false,
processor: undefined,
}),
logger: logger,
createPublisherFactory: (): Publisher => ({}) as unknown as Publisher,
joinMatrixRTC: async (): Promise<void> => {},
homeserverConnected$: constant(true),
};
it("throws error on missing RTC config error", () => {
withTestScheduler(({ scope, hot, expectObservable }) => {
const goodTransport = {
livekit_service_url: "other",
} as LivekitTransport;
const localTransport$ = scope.behavior<LivekitTransport>(
hot("1ms #", {}, new MatrixRTCTransportMissingError("domain.com")),
goodTransport,
);
const mockConnectionManager = {
transports$: scope.behavior(
localTransport$.pipe(map((t) => new Epoch([t]))),
),
connectionManagerData$: constant(
new Epoch(new ConnectionManagerData()),
),
};
const localMembership = createLocalMembership$({
scope,
...defaultCreateLocalMemberValues,
connectionManager: mockConnectionManager,
localTransport$,
});
expectObservable(localMembership.connectionState.livekit$).toBe("ne", {
n: { state: LivekitState.Uninitialized },
e: {
state: LivekitState.Error,
error: expect.toSatisfy(
(e) => e instanceof MatrixRTCTransportMissingError,
),
},
});
});
});
});

View File

@@ -1,52 +1,51 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type LocalTrack, type E2EEOptions } from "livekit-client";
import {
type LocalTrack,
type Participant,
ParticipantEvent,
type LocalParticipant,
} from "livekit-client";
import { observeParticipantEvents } from "@livekit/components-core";
import {
type LivekitTransport,
type MatrixRTCSession,
MembershipManagerEvent,
Status,
} from "matrix-js-sdk/lib/matrixrtc";
import { ClientEvent, SyncState, type Room as MatrixRoom } from "matrix-js-sdk";
import {
BehaviorSubject,
catchError,
combineLatest,
fromEvent,
distinctUntilChanged,
map,
NEVER,
type Observable,
of,
scan,
startWith,
switchMap,
take,
takeWhile,
tap,
} from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { sharingScreen$ as observeSharingScreen$ } from "../../UserMedia.ts";
import { type Behavior } from "../../Behavior";
import { type IConnectionManager } from "../remoteMembers/ConnectionManager";
import { ObservableScope } from "../../ObservableScope";
import { Publisher } from "./Publisher";
import { type Publisher } from "./Publisher";
import { type MuteStates } from "../../MuteStates";
import { type ProcessorState } from "../../../livekit/TrackProcessorContext";
import { type MediaDevices } from "../../MediaDevices";
import { and$ } from "../../../utils/observable";
import {
enterRTCSession,
type EnterRTCSessionOptions,
} from "../../../rtcSessionHelpers";
import { type ElementCallError } from "../../../utils/errors";
import { ElementWidgetActions, type WidgetHelpers } from "../../../widget";
import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers";
import { ElementCallError, UnknownCallError } from "../../../utils/errors";
import { ElementWidgetActions, widget } from "../../../widget";
import { getUrlParams } from "../../../UrlParams.ts";
import { PosthogAnalytics } from "../../../analytics/PosthogAnalytics.ts";
import { MatrixRTCMode } from "../../../settings/settings.ts";
import { Config } from "../../../config/Config.ts";
import {
type Connection,
type ConnectionState,
} from "../remoteMembers/Connection.ts";
export enum LivekitState {
Uninitialized = "uninitialized",
@@ -56,8 +55,9 @@ export enum LivekitState {
Disconnected = "disconnected",
Disconnecting = "disconnecting",
}
type LocalMemberLivekitState =
| { state: LivekitState.Error; error: string }
| { state: LivekitState.Error; error: ElementCallError }
| { state: LivekitState.Connected }
| { state: LivekitState.Connecting }
| { state: LivekitState.Uninitialized }
@@ -68,15 +68,18 @@ export enum MatrixState {
Connected = "connected",
Disconnected = "disconnected",
Connecting = "connecting",
Error = "Error",
}
type LocalMemberMatrixState =
| { state: MatrixState.Connected }
| { state: MatrixState.Connecting }
| { state: MatrixState.Disconnected };
| { state: MatrixState.Disconnected }
| { state: MatrixState.Error; error: Error };
export interface LocalMemberConnectionState {
livekit$: BehaviorSubject<LocalMemberLivekitState>;
matrix$: BehaviorSubject<LocalMemberMatrixState>;
livekit$: Behavior<LocalMemberLivekitState>;
matrix$: Behavior<LocalMemberMatrixState>;
}
/*
@@ -90,18 +93,22 @@ export interface LocalMemberConnectionState {
* - Publisher.publishTracks()
* - send join state/sticky event
*/
interface Props {
options: Behavior<EnterRTCSessionOptions>;
// TODO add a comment into some code style readme or file header callviewmodel
// that the inputs for those createSomething$() functions should NOT contain any js-sdk objectes
scope: ObservableScope;
mediaDevices: MediaDevices;
muteStates: MuteStates;
connectionManager: IConnectionManager;
matrixRTCSession: MatrixRTCSession;
matrixRoom: MatrixRoom;
localTransport$: Behavior<LivekitTransport | undefined>;
e2eeLivekitOptions: E2EEOptions | undefined;
trackProcessorState$: Behavior<ProcessorState>;
widget: WidgetHelpers | null;
createPublisherFactory: (connection: Connection) => Publisher;
joinMatrixRTC: (trasnport: LivekitTransport) => Promise<void>;
homeserverConnected$: Behavior<boolean>;
localTransport$: Behavior<LivekitTransport | null>;
matrixRTCSession: Pick<
MatrixRTCSession,
"updateCallIntent" | "leaveRoomSession"
>;
logger: Logger;
}
/**
@@ -119,38 +126,36 @@ interface Props {
*/
export const createLocalMembership$ = ({
scope,
options,
muteStates,
mediaDevices,
connectionManager,
localTransport$: localTransportCanThrow$,
homeserverConnected$,
createPublisherFactory,
joinMatrixRTC,
logger: parentLogger,
muteStates,
matrixRTCSession,
localTransport$,
matrixRoom,
e2eeLivekitOptions,
trackProcessorState$,
widget,
}: Props): {
// publisher: Publisher
requestConnect: () => LocalMemberConnectionState;
startTracks: () => Behavior<LocalTrack[]>;
requestDisconnect: () => Observable<LocalMemberLivekitState> | null;
connectionState: LocalMemberConnectionState;
// Use null here since behavior cannot be initialised with undefined.
sharingScreen$: Behavior<boolean | null>;
sharingScreen$: Behavior<boolean>;
/**
* Callback to toggle screen sharing. If null, screen sharing is not possible.
*/
toggleScreenSharing: (() => void) | null;
// deprecated fields
/** @deprecated use state instead*/
participant$: Behavior<LocalParticipant | null>;
connection$: Behavior<Connection | null>;
homeserverConnected$: Behavior<boolean>;
// deprecated fields
/** @deprecated use state instead*/
connected$: Behavior<boolean>;
// this needs to be discussed
/** @deprecated use state instead*/
reconnecting$: Behavior<boolean>;
// also needs to be disccues
/** @deprecated use state instead*/
configError$: Behavior<ElementCallError | null>;
} => {
const logger = parentLogger.getChild("[LocalMembership]");
logger.debug(`Creating local membership..`);
const state = {
livekit$: new BehaviorSubject<LocalMemberLivekitState>({
state: LivekitState.Uninitialized,
@@ -162,53 +167,53 @@ export const createLocalMembership$ = ({
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const shouldStartTracks$ = new BehaviorSubject(false);
const trackStartRequested$ = new BehaviorSubject(false);
// This should be used in a combineLatest with publisher$ to connect.
// to make it possible to call startTracks before the preferredTransport$ has resolved.
const connectRequested$ = new BehaviorSubject(false);
// This should be used in a combineLatest with publisher$ to connect.
const tracks$ = new BehaviorSubject<LocalTrack[]>([]);
// Drop Epoch data here since we will not combine this anymore
const connection$ = scope.behavior(
combineLatest(
[connectionManager.connections$, localTransport$],
(connections, transport) => {
if (transport === null) return null;
return (
connections.value.find((connection) =>
areLivekitTransportsEqual(connection.transport, transport),
) ?? null
);
},
// unwrap the local transport and set the state of the LocalMembership to error in case the transport is an error.
const localTransport$ = scope.behavior(
localTransportCanThrow$.pipe(
catchError((e: unknown) => {
let error: ElementCallError;
if (e instanceof ElementCallError) {
error = e;
} else {
error = new UnknownCallError(
e instanceof Error
? e
: new Error("Unknown error from localTransport"),
);
}
state.livekit$.next({ state: LivekitState.Error, error });
return of(null);
}),
),
);
/**
* Whether we are connected to the MatrixRTC session.
*/
const homeserverConnected$ = scope.behavior(
// To consider ourselves connected to MatrixRTC, we check the following:
and$(
// The client is connected to the sync loop
(
fromEvent(matrixRoom.client, ClientEvent.Sync) as Observable<
[SyncState]
>
).pipe(
startWith([matrixRoom.client.getSyncState()]),
map(([state]) => state === SyncState.Syncing),
),
// Room state observed by session says we're connected
fromEvent(matrixRTCSession, MembershipManagerEvent.StatusChanged).pipe(
startWith(null),
map(() => matrixRTCSession.membershipStatus === Status.Connected),
),
// Also watch out for warnings that we've likely hit a timeout and our
// delayed leave event is being sent (this condition is here because it
// provides an earlier warning than the sync loop timeout, and we wouldn't
// see the actual leave event until we reconnect to the sync loop)
fromEvent(matrixRTCSession, MembershipManagerEvent.ProbablyLeft).pipe(
startWith(null),
map(() => matrixRTCSession.probablyLeft !== true),
),
// Drop Epoch data here since we will not combine this anymore
const localConnection$ = scope.behavior(
combineLatest([
connectionManager.connectionManagerData$,
localTransport$,
]).pipe(
map(([connectionData, localTransport]) => {
if (localTransport === null) {
return null;
}
return connectionData.value.getConnectionForTransport(localTransport);
}),
tap((connection) => {
logger.info(
`Local connection updated: ${connection?.transport?.livekit_service_url}`,
);
}),
),
);
@@ -220,7 +225,7 @@ export const createLocalMembership$ = ({
const connected$ = scope.behavior(
and$(
homeserverConnected$,
connection$.pipe(
localConnection$.pipe(
switchMap((c) =>
c
? c.state$.pipe(map((state) => state.state === "ConnectedToLkRoom"))
@@ -230,26 +235,21 @@ export const createLocalMembership$ = ({
),
);
const publisher$ = scope.behavior(
connection$.pipe(
map((connection) =>
connection
? new Publisher(
scope,
connection,
mediaDevices,
muteStates,
e2eeLivekitOptions,
trackProcessorState$,
)
: null,
),
),
);
const publisher$ = new BehaviorSubject<Publisher | null>(null);
localConnection$.pipe(scope.bind()).subscribe((connection) => {
if (connection !== null && publisher$.value === null) {
// TODO looks strange to not change publisher if connection changes.
// @toger5 will take care of this!
publisher$.next(createPublisherFactory(connection));
}
});
combineLatest(
[publisher$, shouldStartTracks$],
(publisher, shouldStartTracks) => {
// const mutestate= publisher$.pipe(switchMap((publisher) => {
// return publisher.muteState$
// });
combineLatest([publisher$, trackStartRequested$]).subscribe(
([publisher, shouldStartTracks]) => {
if (publisher && shouldStartTracks) {
publisher
.createAndSetupTracks()
@@ -286,41 +286,69 @@ export const createLocalMembership$ = ({
);
const startTracks = (): Behavior<LocalTrack[]> => {
shouldStartTracks$.next(true);
trackStartRequested$.next(true);
return tracks$;
};
const requestConnect = (): LocalMemberConnectionState => {
if (state.livekit$.value.state === LivekitState.Uninitialized) {
startTracks();
state.livekit$.next({ state: LivekitState.Connecting });
combineLatest([publisher$, tracks$], (publisher, tracks) => {
publisher
?.startPublishing()
.then(() => {
state.livekit$.next({ state: LivekitState.Connected });
})
.catch((error) => {
state.livekit$.next({ state: LivekitState.Error, error });
});
combineLatest([publisher$, tracks$]).subscribe(([publisher, tracks]) => {
if (
tracks.length === 0 ||
// change this to !== Publishing
state.livekit$.value.state !== LivekitState.Uninitialized
) {
return;
}
state.livekit$.next({ state: LivekitState.Connecting });
publisher
?.startPublishing()
.then(() => {
state.livekit$.next({ state: LivekitState.Connected });
})
.catch((error) => {
state.livekit$.next({ state: LivekitState.Error, error });
});
}
if (state.matrix$.value.state === MatrixState.Disconnected) {
});
combineLatest([localTransport$, connectRequested$]).subscribe(
// TODO reconnect when transport changes => create test.
([transport, connectRequested]) => {
if (
transport === null ||
!connectRequested ||
state.matrix$.value.state !== MatrixState.Disconnected
) {
logger.info(
"Not yet connecting because: ",
"transport === null:",
transport === null,
"!connectRequested:",
!connectRequested,
"state.matrix$.value.state !== MatrixState.Disconnected:",
state.matrix$.value.state !== MatrixState.Disconnected,
);
return;
}
state.matrix$.next({ state: MatrixState.Connecting });
localTransport$.pipe(
tap((transport) => {
if (transport !== undefined) {
enterRTCSession(matrixRTCSession, transport, options.value).catch(
(error) => {
logger.error(error);
},
);
} else {
logger.info("Waiting for transport to enter rtc session");
}
}),
);
}
logger.info("Matrix State connecting");
joinMatrixRTC(transport).catch((error) => {
logger.error(error);
state.matrix$.next({ state: MatrixState.Error, error });
});
},
);
// TODO add this and update `state.matrix$` based on it.
// useTypedEventEmitter(
// rtcSession,
// MatrixRTCSessionEvent.MembershipManagerError,
// (error) => setExternalError(new ConnectionLostError()),
// );
const requestConnect = (): LocalMemberConnectionState => {
trackStartRequested$.next(true);
connectRequested$.next(true);
return state;
};
@@ -348,7 +376,7 @@ export const createLocalMembership$ = ({
// We use matrixConnected$ rather than reconnecting$ because we want to
// pause tracks during the initial joining sequence too until we're sure
// that our own media is displayed on screen.
combineLatest([connection$, homeserverConnected$])
combineLatest([localConnection$, homeserverConnected$])
.pipe(scope.bind())
.subscribe(([connection, connected]) => {
if (connection?.state$.value.state !== "ConnectedToLkRoom") return;
@@ -358,7 +386,9 @@ export const createLocalMembership$ = ({
for (const p of publications) {
if (p.track?.isUpstreamPaused === true) {
const kind = p.track.kind;
logger.log(`Resuming ${kind} track (MatrixRTC connection present)`);
logger.info(
`Resuming ${kind} track (MatrixRTC connection present)`,
);
p.track
.resumeUpstream()
.catch((e) =>
@@ -373,7 +403,7 @@ export const createLocalMembership$ = ({
for (const p of publications) {
if (p.track?.isUpstreamPaused === false) {
const kind = p.track.kind;
logger.log(
logger.info(
`Pausing ${kind} track (uncertain MatrixRTC connection)`,
);
p.track
@@ -388,18 +418,25 @@ export const createLocalMembership$ = ({
}
}
});
// TODO: Refactor updateCallIntent to sth like this:
// combineLatest([muteStates.video.enabled$,localTransport$, state.matrix$]).pipe(map(()=>{
// matrixRTCSession.updateCallIntent(videoEnabled ? "video" : "audio"),
// }))
//
const configError$ = new BehaviorSubject<ElementCallError | null>(null);
// TODO I do not fully understand what this does.
// Is it needed?
// Is this at the right place?
// Can this be simplified?
// Start and stop session membership as needed
scope.reconcile(localTransport$, async (advertised) => {
if (advertised !== null && advertised !== undefined) {
// Discussed in statndup -> It seems we can remove this (there is another call to enterRTCSession in this file)
// MAKE SURE TO UNDERSTAND why reconcile is needed and what is potentially missing from the alternative enterRTCSession block.
// @toger5 will try to take care of this.
scope.reconcile(localTransport$, async (transport) => {
if (transport !== null && transport !== undefined) {
try {
configError$.next(null);
await enterRTCSession(matrixRTCSession, advertised, options.value);
state.matrix$.next({ state: MatrixState.Connecting });
await joinMatrixRTC(transport);
} catch (e) {
logger.error("Error entering RTC session", e);
}
@@ -429,74 +466,65 @@ export const createLocalMembership$ = ({
}
});
localConnection$
.pipe(
distinctUntilChanged(),
switchMap((c) =>
c === null ? of({ state: "Initialized" } as ConnectionState) : c.state$,
),
map((s) => {
logger.trace(`Local connection state update: ${s.state}`);
if (s.state == "FailedToStart") {
return s.error instanceof ElementCallError
? s.error
: new UnknownCallError(s.error);
}
}),
scope.bind(),
)
.subscribe((error) => {
if (error !== undefined)
state.livekit$.next({ state: LivekitState.Error, error });
});
/**
* Returns undefined if scrennSharing is not yet ready.
* Whether the user is currently sharing their screen.
*/
const sharingScreen$ = scope.behavior(
connection$.pipe(
switchMap((c) => {
if (!c) return of(null);
if (c.state$.value.state === "ConnectedToLkRoom")
return observeSharingScreen$(c.livekitRoom.localParticipant);
return of(false);
}),
localConnection$.pipe(
switchMap((c) =>
c !== null
? observeSharingScreen$(c.livekitRoom.localParticipant)
: of(false),
),
),
null,
);
const toggleScreenSharing =
"getDisplayMedia" in (navigator.mediaDevices ?? {}) &&
!getUrlParams().hideScreensharing
? (): void =>
// If a connection is ready...
void connection$
.pipe(
// I dont see why we need this. isnt the check later on superseeding it?
takeWhile(
(c) =>
c !== undefined && c.state$.value.state !== "FailedToStart",
),
switchMap((c) =>
c?.state$.value.state === "ConnectedToLkRoom" ? of(c) : NEVER,
),
take(1),
scope.bind(),
)
// ...toggle screen sharing.
.subscribe(
(c) =>
void c.livekitRoom.localParticipant
.setScreenShareEnabled(!sharingScreen$.value, {
audio: true,
selfBrowserSurface: "include",
surfaceSwitching: "include",
systemAudio: "include",
})
.catch(logger.error),
)
// If a connection is ready, toggle screen sharing.
// We deliberately do nothing in the case of a null connection because
// it looks nice for the call control buttons to all become available
// at once upon joining the call, rather than introducing a disabled
// state. The user can just click again.
// We also allow screen sharing to be toggled even if the connection
// is still initializing or publishing tracks, because there's no
// technical reason to disallow this. LiveKit will publish if it can.
void localConnection$.value?.livekitRoom.localParticipant
.setScreenShareEnabled(!sharingScreen$.value, {
audio: true,
selfBrowserSurface: "include",
surfaceSwitching: "include",
systemAudio: "include",
})
.catch(logger.error)
: null;
// we do not need all the auto waiting since we can just check via sharingScreen$.value !== undefined
let alternativeScreenshareToggle: (() => void) | null = null;
if (
"getDisplayMedia" in (navigator.mediaDevices ?? {}) &&
!getUrlParams().hideScreensharing
) {
alternativeScreenshareToggle = (): void =>
void connection$.value?.livekitRoom.localParticipant
.setScreenShareEnabled(!sharingScreen$.value, {
audio: true,
selfBrowserSurface: "include",
surfaceSwitching: "include",
systemAudio: "include",
})
.catch(logger.error);
}
logger.log(
"alternativeScreenshareToggle so that it is used",
alternativeScreenshareToggle,
const participant$ = scope.behavior(
localConnection$.pipe(map((c) => c?.livekitRoom?.localParticipant ?? null)),
);
return {
startTracks,
requestConnect,
@@ -505,8 +533,86 @@ export const createLocalMembership$ = ({
homeserverConnected$,
connected$,
reconnecting$,
configError$,
sharingScreen$,
toggleScreenSharing,
participant$,
connection$: localConnection$,
};
};
export function observeSharingScreen$(p: Participant): Observable<boolean> {
return observeParticipantEvents(
p,
ParticipantEvent.TrackPublished,
ParticipantEvent.TrackUnpublished,
ParticipantEvent.LocalTrackPublished,
ParticipantEvent.LocalTrackUnpublished,
).pipe(map((p) => p.isScreenShareEnabled));
}
interface EnterRTCSessionOptions {
encryptMedia: boolean;
matrixRTCMode: MatrixRTCMode;
}
/**
* Does the necessary steps to enter the RTC session on the matrix side:
* - Preparing the membership info (FOCUS to use, options)
* - Sends the matrix event to join the call, and starts the membership manager:
* - Delay events management
* - Handles retries (fails only after several attempts)
*
* @param rtcSession
* @param transport
* @param options
* @throws If the widget could not send ElementWidgetActions.JoinCall action.
*/
// Exported for unit testing
export async function enterRTCSession(
rtcSession: MatrixRTCSession,
transport: LivekitTransport,
{ encryptMedia, matrixRTCMode }: EnterRTCSessionOptions,
): Promise<void> {
PosthogAnalytics.instance.eventCallEnded.cacheStartCall(new Date());
PosthogAnalytics.instance.eventCallStarted.track(rtcSession.room.roomId);
// This must be called before we start trying to join the call, as we need to
// have started tracking by the time calls start getting created.
// groupCallOTelMembership?.onJoinCall();
const { features, matrix_rtc_session: matrixRtcSessionConfig } = Config.get();
const useDeviceSessionMemberEvents =
features?.feature_use_device_session_member_events;
const { sendNotificationType: notificationType, callIntent } = getUrlParams();
const multiSFU = matrixRTCMode !== MatrixRTCMode.Legacy;
// Multi-sfu does not need a preferred foci list. just the focus that is actually used.
// TODO where/how do we track errors originating from the ongoing rtcSession?
rtcSession.joinRoomSession(
multiSFU ? [] : [transport],
multiSFU ? transport : undefined,
{
notificationType,
callIntent,
manageMediaKeys: encryptMedia,
...(useDeviceSessionMemberEvents !== undefined && {
useLegacyMemberEvents: !useDeviceSessionMemberEvents,
}),
delayedLeaveEventRestartMs:
matrixRtcSessionConfig?.delayed_leave_event_restart_ms,
delayedLeaveEventDelayMs:
matrixRtcSessionConfig?.delayed_leave_event_delay_ms,
delayedLeaveEventRestartLocalTimeoutMs:
matrixRtcSessionConfig?.delayed_leave_event_restart_local_timeout_ms,
networkErrorRetryMs: matrixRtcSessionConfig?.network_error_retry_ms,
makeKeyDelay: matrixRtcSessionConfig?.wait_for_key_rotation_ms,
membershipEventExpiryMs:
matrixRtcSessionConfig?.membership_event_expiry_ms,
useExperimentalToDeviceTransport: true,
unstableSendStickyEvents: matrixRTCMode === MatrixRTCMode.Matrix_2_0,
},
);
if (widget) {
await widget.api.transport.send(ElementWidgetActions.JoinCall, {});
}
}

View File

@@ -0,0 +1,120 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { mockConfig, flushPromises } from "../../../utils/test";
import { createLocalTransport$ } from "./LocalTransport";
import { constant } from "../../Behavior";
import { Epoch, ObservableScope } from "../../ObservableScope";
import {
MatrixRTCTransportMissingError,
FailToGetOpenIdToken,
} from "../../../utils/errors";
import * as openIDSFU from "../../../livekit/openIDSFU";
describe("LocalTransport", () => {
let scope: ObservableScope;
beforeEach(() => (scope = new ObservableScope()));
afterEach(() => scope.end());
it("throws if config is missing", async () => {
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "",
// These won't be called in this error path but satisfy the type
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
await flushPromises();
expect(() => localTransport$.value).toThrow(
new MatrixRTCTransportMissingError(""),
);
});
it("throws FailToGetOpenIdToken when OpenID fetch fails", async () => {
// Provide a valid config so makeTransportInternal resolves a transport
const scope = new ObservableScope();
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const resolver = Promise.withResolvers<void>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockImplementation(
async () => {
await resolver.promise;
throw new FailToGetOpenIdToken(new Error("no openid"));
},
);
const observations: unknown[] = [];
const errors: Error[] = [];
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
// Use empty domain to skip .well-known and use config directly
getDomain: () => "",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
localTransport$.subscribe(
(o) => observations.push(o),
(e) => errors.push(e),
);
resolver.resolve();
await flushPromises();
const expectedError = new FailToGetOpenIdToken(new Error("no openid"));
expect(observations).toStrictEqual([null]);
expect(errors).toStrictEqual([expectedError]);
expect(() => localTransport$.value).toThrow(expectedError);
});
it("emits preferred transport after OpenID resolves", async () => {
// Use config so transport discovery succeeds, but delay OpenID JWT fetch
mockConfig({
livekit: { livekit_service_url: "https://lk.example.org" },
});
const openIdResolver = Promise.withResolvers<openIDSFU.SFUConfig>();
vi.spyOn(openIDSFU, "getSFUConfigWithOpenID").mockReturnValue(
openIdResolver.promise,
);
const localTransport$ = createLocalTransport$({
scope,
roomId: "!room:example.org",
useOldestMember$: constant(false),
memberships$: constant(new Epoch<CallMembership[]>([])),
client: {
getDomain: () => "",
getOpenIdToken: vi.fn(),
getDeviceId: vi.fn(),
},
});
openIdResolver.resolve?.({ url: "https://lk.example.org", jwt: "jwt" });
expect(localTransport$.value).toBe(null);
await flushPromises();
// final
expect(localTransport$.value).toStrictEqual({
livekit_alias: "!room:example.org",
livekit_service_url: "https://lk.example.org",
type: "livekit",
});
});
});

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-IdFentifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -13,36 +13,39 @@ import {
isLivekitTransportConfig,
} from "matrix-js-sdk/lib/matrixrtc";
import { type MatrixClient } from "matrix-js-sdk";
import { combineLatest, distinctUntilChanged, first, from } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import {
combineLatest,
distinctUntilChanged,
first,
from,
map,
switchMap,
} from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { AutoDiscovery } from "matrix-js-sdk/lib/autodiscovery";
import { deepCompare } from "matrix-js-sdk/lib/utils";
import { type Behavior } from "../../Behavior.ts";
import {
type Epoch,
mapEpoch,
type ObservableScope,
} from "../../ObservableScope.ts";
import { type Epoch, type ObservableScope } from "../../ObservableScope.ts";
import { Config } from "../../../config/Config.ts";
import { MatrixRTCTransportMissingError } from "../../../utils/errors.ts";
import { getSFUConfigWithOpenID } from "../../../livekit/openIDSFU.ts";
import {
getSFUConfigWithOpenID,
type OpenIDClientParts,
} from "../../../livekit/openIDSFU.ts";
import { areLivekitTransportsEqual } from "../remoteMembers/MatrixLivekitMembers.ts";
import { customLivekitUrl } from "../../../settings/settings.ts";
const logger = rootLogger.getChild("[LocalTransport]");
/*
* - get well known
* - get oldest membership
* - get transport to use
* - get openId + jwt token
* - wait for createTrack() call
* - create tracks
* - wait for join() call
* - Publisher.publishTracks()
* - send join state/sticky event
* It figures out “which LiveKit focus URL/alias the local user should use,”
* optionally aligning with the oldest member, and ensures the SFU path is primed
* before advertising that choice.
*/
interface Props {
scope: ObservableScope;
memberships$: Behavior<Epoch<CallMembership[]>>;
client: MatrixClient;
client: Pick<MatrixClient, "getDomain"> & OpenIDClientParts;
roomId: string;
useOldestMember$: Behavior<boolean>;
}
@@ -53,6 +56,8 @@ interface Props {
*
* @prop useOldestMember Whether to use the same transport as the oldest member.
* This will only update once the first oldest member appears. Will not recompute if the oldest member leaves.
*
* @throws MatrixRTCTransportMissingError | FailToGetOpenIdToken
*/
export const createLocalTransport$ = ({
scope,
@@ -60,15 +65,16 @@ export const createLocalTransport$ = ({
client,
roomId,
useOldestMember$,
}: Props): Behavior<LivekitTransport | undefined> => {
}: Props): Behavior<LivekitTransport | null> => {
/**
* The transport over which we should be actively publishing our media.
* undefined when not joined.
*/
const oldestMemberTransport$ = scope.behavior(
memberships$.pipe(
mapEpoch(
(memberships) => memberships[0]?.getTransport(memberships[0]) ?? null,
map(
(memberships) =>
memberships.value[0]?.getTransport(memberships.value[0]) ?? null,
),
first((t) => t != null && isLivekitTransport(t)),
),
@@ -78,96 +84,105 @@ export const createLocalTransport$ = ({
/**
* The transport that we would personally prefer to publish on (if not for the
* transport preferences of others, perhaps).
*
* @throws
*/
const preferredTransport$: Behavior<LivekitTransport | null> = scope.behavior(
from(makeTransport(client, roomId)),
customLivekitUrl.value$.pipe(
switchMap((customUrl) => from(makeTransport(client, roomId, customUrl))),
),
null,
);
/**
* The transport we should advertise in our MatrixRTC membership.
* The chosen transport we should advertise in our MatrixRTC membership.
*/
const advertisedTransport$ = scope.behavior(
combineLatest(
[useOldestMember$, oldestMemberTransport$, preferredTransport$],
(useOldestMember, oldestMemberTransport, preferredTransport) =>
return scope.behavior(
combineLatest([
useOldestMember$,
oldestMemberTransport$,
preferredTransport$,
]).pipe(
map(([useOldestMember, oldestMemberTransport, preferredTransport]) =>
useOldestMember
? (oldestMemberTransport ?? preferredTransport)
: preferredTransport,
).pipe<LivekitTransport>(distinctUntilChanged(deepCompare)),
),
distinctUntilChanged(areLivekitTransportsEqual),
),
);
return advertisedTransport$;
};
const FOCI_WK_KEY = "org.matrix.msc4143.rtc_foci";
async function makeTransportInternal(
client: MatrixClient,
/**
*
* @param client
* @param roomId
* @returns
* @throws MatrixRTCTransportMissingError | FailToGetOpenIdToken
*/
async function makeTransport(
client: Pick<MatrixClient, "getDomain"> & OpenIDClientParts,
roomId: string,
urlFromDevSettings: string | null,
): Promise<LivekitTransport> {
logger.log("Searching for a preferred transport");
let transport: LivekitTransport | undefined;
logger.trace("Searching for a preferred transport");
//TODO refactor this to use the jwt service returned alias.
const livekitAlias = roomId;
// TODO-MULTI-SFU: Either remove this dev tool or make it more official
const urlFromStorage =
localStorage.getItem("robin-matrixrtc-auth") ??
localStorage.getItem("timo-focus-url");
if (urlFromStorage !== null) {
// DEVTOOL: Highest priority: Load from devtool setting
if (urlFromDevSettings !== null) {
const transportFromStorage: LivekitTransport = {
type: "livekit",
livekit_service_url: urlFromStorage,
livekit_service_url: urlFromDevSettings,
livekit_alias: livekitAlias,
};
logger.log(
"Using LiveKit transport from local storage: ",
logger.info(
"Using LiveKit transport from dev tools: ",
transportFromStorage,
);
return transportFromStorage;
transport = transportFromStorage;
}
// Prioritize the .well-known/matrix/client, if available, over the configured SFU
// WELL_KNOWN: Prioritize the .well-known/matrix/client, if available, over the configured SFU
const domain = client.getDomain();
if (domain) {
if (domain && transport === undefined) {
// we use AutoDiscovery instead of relying on the MatrixClient having already
// been fully configured and started
const wellKnownFoci = (await AutoDiscovery.getRawClientConfig(domain))?.[
FOCI_WK_KEY
];
if (Array.isArray(wellKnownFoci)) {
const transport: LivekitTransportConfig | undefined = wellKnownFoci.find(
(f) => f && isLivekitTransportConfig(f),
);
if (transport !== undefined) {
logger.log("Using LiveKit transport from .well-known: ", transport);
return { ...transport, livekit_alias: livekitAlias };
const wellKnownTransport: LivekitTransportConfig | undefined =
wellKnownFoci.find((f) => f && isLivekitTransportConfig(f));
if (wellKnownTransport !== undefined) {
logger.info("Using LiveKit transport from .well-known: ", transport);
transport = { ...wellKnownTransport, livekit_alias: livekitAlias };
}
}
}
// CONFIG: Least prioritized; Load from config file
const urlFromConf = Config.get().livekit?.livekit_service_url;
if (urlFromConf) {
if (urlFromConf && transport === undefined) {
const transportFromConf: LivekitTransport = {
type: "livekit",
livekit_service_url: urlFromConf,
livekit_alias: livekitAlias,
};
logger.log("Using LiveKit transport from config: ", transportFromConf);
return transportFromConf;
logger.info("Using LiveKit transport from config: ", transportFromConf);
transport = transportFromConf;
}
throw new MatrixRTCTransportMissingError(domain ?? "");
}
if (!transport) throw new MatrixRTCTransportMissingError(domain ?? ""); // this will call the jwt/sfu/get endpoint to pre create the livekit room.
async function makeTransport(
client: MatrixClient,
roomId: string,
): Promise<LivekitTransport> {
const transport = await makeTransportInternal(client, roomId);
// this will call the jwt/sfu/get endpoint to pre create the livekit room.
await getSFUConfigWithOpenID(
client,
transport.livekit_service_url,
transport.livekit_alias,
);
return transport;
}

View File

@@ -1,11 +1,11 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type E2EEOptions,
LocalVideoTrack,
type Room as LivekitRoom,
Track,
@@ -55,7 +55,6 @@ export class Publisher {
private connection: Connection,
devices: MediaDevices,
private readonly muteStates: MuteStates,
e2eeLivekitOptions: E2EEOptions | undefined,
trackerProcessorState$: Behavior<ProcessorState>,
private logger?: Logger,
) {
@@ -64,7 +63,7 @@ export class Publisher {
const room = connection.livekitRoom;
room.setE2EEEnabled(e2eeLivekitOptions !== undefined)?.catch((e: Error) => {
room.setE2EEEnabled(room.options.e2ee !== undefined)?.catch((e: Error) => {
this.logger?.error("Failed to set E2EE enabled on room", e);
});
@@ -74,6 +73,12 @@ export class Publisher {
this.observeMediaDevices(scope, devices, controlledAudioDevices);
this.workaroundRestartAudioInputTrackChrome(devices, scope);
this.scope.onEnd(() => {
this.logger?.info(
"[PublishConnection] Scope ended -> stop publishing all tracks",
);
void this.stopPublishing();
});
}
/**
@@ -99,31 +104,36 @@ export class Publisher {
// instead? This optimization would only be safe for a publish connection,
// because we don't want to leak the user's intent to perhaps join a call to
// remote servers before they actually commit to it.
const { promise, resolve, reject } = Promise.withResolvers<void>();
const sub = this.connection.state$.subscribe((s) => {
if (s.state !== "FailedToStart") {
reject(new Error("Disconnected from LiveKit server"));
} else {
resolve();
}
});
try {
await promise;
} catch (e) {
throw e;
} finally {
sub.unsubscribe();
}
// const { promise, resolve, reject } = Promise.withResolvers<void>();
// const sub = this.connection.state$.subscribe((s) => {
// if (s.state === "FailedToStart") {
// reject(new Error("Disconnected from LiveKit server"));
// } else if (s.state === "ConnectedToLkRoom") {
// resolve();
// }
// });
// try {
// await promise;
// } catch (e) {
// throw e;
// } finally {
// sub.unsubscribe();
// }
// TODO-MULTI-SFU: Prepublish a microphone track
const audio = this.muteStates.audio.enabled$.value;
const video = this.muteStates.video.enabled$.value;
// createTracks throws if called with audio=false and video=false
if (audio || video) {
// TODO this can still throw errors? It will also prompt for permissions if not already granted
this.tracks = await lkRoom.localParticipant.createTracks({
audio,
video,
});
this.tracks =
(await lkRoom.localParticipant
.createTracks({
audio,
video,
})
.catch((error) => {
this.logger?.error("Failed to create tracks", error);
})) ?? [];
}
return this.tracks;
}
@@ -153,7 +163,9 @@ export class Publisher {
for (const track of this.tracks) {
// TODO: handle errors? Needs the signaling connection to be up, but it has some retries internally
// with a timeout.
await lkRoom.localParticipant.publishTrack(track);
await lkRoom.localParticipant.publishTrack(track).catch((error) => {
this.logger?.error("Failed to publish track", error);
});
// TODO: check if the connection is still active? and break the loop if not?
}
@@ -309,6 +321,6 @@ export class Publisher {
),
null,
);
trackProcessorSync(track$, trackerProcessorState$);
trackProcessorSync(scope, track$, trackerProcessorState$);
}
}

View File

@@ -1,4 +1,5 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -10,28 +11,23 @@ import {
describe,
expect,
it,
type Mock,
type MockedObject,
onTestFinished,
vi,
} from "vitest";
import { BehaviorSubject, of } from "rxjs";
import {
type LocalParticipant,
type RemoteParticipant,
type Room as LivekitRoom,
RoomEvent,
type RoomOptions,
ConnectionState as LivekitConnectionState,
} from "livekit-client";
import fetchMock from "fetch-mock";
import EventEmitter from "events";
import { type IOpenIDToken } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import type {
CallMembership,
LivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
import type { LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import {
Connection,
type ConnectionOpts,
@@ -41,9 +37,6 @@ import {
import { ObservableScope } from "../../ObservableScope.ts";
import { type OpenIDClientParts } from "../../../livekit/openIDSFU.ts";
import { FailToGetOpenIdToken } from "../../../utils/errors.ts";
import { mockMediaDevices, mockMuteStates } from "../../../utils/test.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { type MuteStates } from "../../MuteStates.ts";
let testScope: ObservableScope;
@@ -55,9 +48,9 @@ let localParticipantEventEmiter: EventEmitter;
let fakeLocalParticipant: MockedObject<LocalParticipant>;
let fakeRoomEventEmiter: EventEmitter;
let fakeMembershipsFocusMap$: BehaviorSubject<
{ membership: CallMembership; transport: LivekitTransport }[]
>;
// let fakeMembershipsFocusMap$: BehaviorSubject<
// { membership: CallMembership; transport: LivekitTransport }[]
// >;
const livekitFocus: LivekitTransport = {
livekit_alias: "!roomID:example.org",
@@ -76,9 +69,6 @@ function setupTest(): void {
}),
getDeviceId: vi.fn().mockReturnValue("ABCDEF"),
} as unknown as OpenIDClientParts);
fakeMembershipsFocusMap$ = new BehaviorSubject<
{ membership: CallMembership; transport: LivekitTransport }[]
>([]);
localParticipantEventEmiter = new EventEmitter();
@@ -137,7 +127,7 @@ function setupRemoteConnection(): Connection {
fakeLivekitRoom.connect.mockResolvedValue(undefined);
return new Connection(opts);
return new Connection(opts, logger);
}
afterEach(() => {
@@ -156,7 +146,7 @@ describe("Start connection states", () => {
scope: testScope,
livekitRoomFactory: () => fakeLivekitRoom,
};
const connection = new Connection(opts);
const connection = new Connection(opts, logger);
expect(connection.state$.getValue().state).toEqual("Initialized");
});
@@ -172,7 +162,7 @@ describe("Start connection states", () => {
livekitRoomFactory: () => fakeLivekitRoom,
};
const connection = new Connection(opts, undefined);
const connection = new Connection(opts, logger);
const capturedStates: ConnectionState[] = [];
const s = connection.state$.subscribe((value) => {
@@ -224,7 +214,7 @@ describe("Start connection states", () => {
livekitRoomFactory: () => fakeLivekitRoom,
};
const connection = new Connection(opts, undefined);
const connection = new Connection(opts, logger);
const capturedStates: ConnectionState[] = [];
const s = connection.state$.subscribe((value) => {
@@ -280,7 +270,7 @@ describe("Start connection states", () => {
livekitRoomFactory: () => fakeLivekitRoom,
};
const connection = new Connection(opts, undefined);
const connection = new Connection(opts, logger);
const capturedStates: ConnectionState[] = [];
const s = connection.state$.subscribe((value) => {
@@ -373,21 +363,18 @@ describe("Start connection states", () => {
});
});
function fakeRemoteLivekitParticipant(id: string): RemoteParticipant {
function fakeRemoteLivekitParticipant(
id: string,
publications: number = 1,
): RemoteParticipant {
return {
identity: id,
getTrackPublications: () => Array(publications),
} as unknown as RemoteParticipant;
}
function fakeRtcMemberShip(userId: string, deviceId: string): CallMembership {
return {
userId,
deviceId,
} as unknown as CallMembership;
}
describe("Publishing participants observations", () => {
it("should emit the list of publishing participants", async () => {
it("should emit the list of publishing participants", () => {
setupTest();
const connection = setupRemoteConnection();
@@ -395,135 +382,53 @@ describe("Publishing participants observations", () => {
const bobIsAPublisher = Promise.withResolvers<void>();
const danIsAPublisher = Promise.withResolvers<void>();
const observedPublishers: PublishingParticipant[][] = [];
const s = connection.allLivekitParticipants$.subscribe((publishers) => {
observedPublishers.push(publishers);
if (
publishers.some(
(p) => p.participant?.identity === "@bob:example.org:DEV111",
)
) {
bobIsAPublisher.resolve();
}
if (
publishers.some(
(p) => p.participant?.identity === "@dan:example.org:DEV333",
)
) {
danIsAPublisher.resolve();
}
});
const s = connection.remoteParticipantsWithTracks$.subscribe(
(publishers) => {
observedPublishers.push(publishers);
if (publishers.some((p) => p.identity === "@bob:example.org:DEV111")) {
bobIsAPublisher.resolve();
}
if (publishers.some((p) => p.identity === "@dan:example.org:DEV333")) {
danIsAPublisher.resolve();
}
},
);
onTestFinished(() => s.unsubscribe());
// The publishingParticipants$ observable is derived from the current members of the
// livekitRoom and the rtc membership in order to publish the members that are publishing
// on this connection.
let participants: RemoteParticipant[] = [
fakeRemoteLivekitParticipant("@alice:example.org:DEV000"),
fakeRemoteLivekitParticipant("@bob:example.org:DEV111"),
fakeRemoteLivekitParticipant("@carol:example.org:DEV222"),
fakeRemoteLivekitParticipant("@dan:example.org:DEV333"),
fakeRemoteLivekitParticipant("@alice:example.org:DEV000", 0),
fakeRemoteLivekitParticipant("@bob:example.org:DEV111", 0),
fakeRemoteLivekitParticipant("@carol:example.org:DEV222", 0),
fakeRemoteLivekitParticipant("@dan:example.org:DEV333", 0),
];
// Let's simulate 3 members on the livekitRoom
vi.spyOn(fakeLivekitRoom, "remoteParticipants", "get").mockReturnValue(
new Map(participants.map((p) => [p.identity, p])),
vi.spyOn(fakeLivekitRoom, "remoteParticipants", "get").mockImplementation(
() => new Map(participants.map((p) => [p.identity, p])),
);
for (const participant of participants) {
fakeRoomEventEmiter.emit(RoomEvent.ParticipantConnected, participant);
}
participants.forEach((p) =>
fakeRoomEventEmiter.emit(RoomEvent.ParticipantConnected, p),
);
// At this point there should be no publishers
expect(observedPublishers.pop()!.length).toEqual(0);
const otherFocus: LivekitTransport = {
livekit_alias: "!roomID:example.org",
livekit_service_url: "https://other-matrix-rtc.example.org/livekit/jwt",
type: "livekit",
};
const rtcMemberships = [
// Say bob is on the same focus
{
membership: fakeRtcMemberShip("@bob:example.org", "DEV111"),
transport: livekitFocus,
},
// Alice and carol is on a different focus
{
membership: fakeRtcMemberShip("@alice:example.org", "DEV000"),
transport: otherFocus,
},
{
membership: fakeRtcMemberShip("@carol:example.org", "DEV222"),
transport: otherFocus,
},
// NO DAVE YET
participants = [
fakeRemoteLivekitParticipant("@alice:example.org:DEV000", 1),
fakeRemoteLivekitParticipant("@bob:example.org:DEV111", 1),
fakeRemoteLivekitParticipant("@carol:example.org:DEV222", 1),
fakeRemoteLivekitParticipant("@dan:example.org:DEV333", 2),
];
// signal this change in rtc memberships
fakeMembershipsFocusMap$.next(rtcMemberships);
// We should have bob has a publisher now
await bobIsAPublisher.promise;
const publishers = observedPublishers.pop();
expect(publishers?.length).toEqual(1);
expect(publishers?.[0].participant?.identity).toEqual(
"@bob:example.org:DEV111",
participants.forEach((p) =>
fakeRoomEventEmiter.emit(RoomEvent.ParticipantConnected, p),
);
// Now let's make dan join the rtc memberships
rtcMemberships.push({
membership: fakeRtcMemberShip("@dan:example.org", "DEV333"),
transport: livekitFocus,
});
fakeMembershipsFocusMap$.next(rtcMemberships);
// We should have bob and dan has publishers now
await danIsAPublisher.promise;
const twoPublishers = observedPublishers.pop();
expect(twoPublishers?.length).toEqual(2);
expect(
twoPublishers?.some(
(p) => p.participant?.identity === "@bob:example.org:DEV111",
),
).toBeTruthy();
expect(
twoPublishers?.some(
(p) => p.participant?.identity === "@dan:example.org:DEV333",
),
).toBeTruthy();
// Now let's make bob leave the livekit room
participants = participants.filter(
(p) => p.identity !== "@bob:example.org:DEV111",
);
vi.spyOn(fakeLivekitRoom, "remoteParticipants", "get").mockReturnValue(
new Map(participants.map((p) => [p.identity, p])),
);
fakeRoomEventEmiter.emit(
RoomEvent.ParticipantDisconnected,
fakeRemoteLivekitParticipant("@bob:example.org:DEV111"),
);
const updatedPublishers = observedPublishers.pop();
// Bob is not connected to the room but he is still in the rtc memberships declaring that
// he is using that focus to publish, so he should still appear as a publisher
expect(updatedPublishers?.length).toEqual(2);
const pp = updatedPublishers?.find(
(p) => p.membership.userId == "@bob:example.org",
);
expect(pp).toBeDefined();
expect(pp!.participant).not.toBeDefined();
expect(
updatedPublishers?.some(
(p) => p.participant?.identity === "@dan:example.org:DEV333",
),
).toBeTruthy();
// Now if bob is not in the rtc memberships, he should disappear
const noBob = rtcMemberships.filter(
({ membership }) => membership.userId !== "@bob:example.org",
);
fakeMembershipsFocusMap$.next(noBob);
expect(observedPublishers.pop()?.length).toEqual(1);
// At this point there should be no publishers
expect(observedPublishers.pop()!.length).toEqual(4);
});
it("should be scoped to parent scope", (): void => {
@@ -532,18 +437,20 @@ describe("Publishing participants observations", () => {
const connection = setupRemoteConnection();
let observedPublishers: PublishingParticipant[][] = [];
const s = connection.allLivekitParticipants$.subscribe((publishers) => {
observedPublishers.push(publishers);
});
const s = connection.remoteParticipantsWithTracks$.subscribe(
(publishers) => {
observedPublishers.push(publishers);
},
);
onTestFinished(() => s.unsubscribe());
let participants: RemoteParticipant[] = [
fakeRemoteLivekitParticipant("@bob:example.org:DEV111"),
fakeRemoteLivekitParticipant("@bob:example.org:DEV111", 0),
];
// Let's simulate 3 members on the livekitRoom
vi.spyOn(fakeLivekitRoom, "remoteParticipants", "get").mockReturnValue(
new Map(participants.map((p) => [p.identity, p])),
vi.spyOn(fakeLivekitRoom, "remoteParticipants", "get").mockImplementation(
() => new Map(participants.map((p) => [p.identity, p])),
);
for (const participant of participants) {
@@ -553,22 +460,16 @@ describe("Publishing participants observations", () => {
// At this point there should be no publishers
expect(observedPublishers.pop()!.length).toEqual(0);
const rtcMemberships = [
// Say bob is on the same focus
{
membership: fakeRtcMemberShip("@bob:example.org", "DEV111"),
transport: livekitFocus,
},
];
// signal this change in rtc memberships
fakeMembershipsFocusMap$.next(rtcMemberships);
participants = [fakeRemoteLivekitParticipant("@bob:example.org:DEV111", 1)];
for (const participant of participants) {
fakeRoomEventEmiter.emit(RoomEvent.ParticipantConnected, participant);
}
// We should have bob has a publisher now
const publishers = observedPublishers.pop();
expect(publishers?.length).toEqual(1);
expect(publishers?.[0].participant?.identity).toEqual(
"@bob:example.org:DEV111",
);
expect(publishers?.[0]?.identity).toEqual("@bob:example.org:DEV111");
// end the parent scope
testScope.end();
@@ -578,9 +479,7 @@ describe("Publishing participants observations", () => {
participants = participants.filter(
(p) => p.identity !== "@bob:example.org:DEV111",
);
vi.spyOn(fakeLivekitRoom, "remoteParticipants", "get").mockReturnValue(
new Map(participants.map((p) => [p.identity, p])),
);
fakeRoomEventEmiter.emit(
RoomEvent.ParticipantDisconnected,
fakeRemoteLivekitParticipant("@bob:example.org:DEV111"),
@@ -590,108 +489,112 @@ describe("Publishing participants observations", () => {
});
});
describe("PublishConnection", () => {
// let fakeBlurProcessor: ProcessorWrapper<BackgroundOptions>;
let roomFactoryMock: Mock<() => LivekitRoom>;
let muteStates: MockedObject<MuteStates>;
//
// NOT USED ANYMORE ?
//
// This setup look like sth for the Publisher. Not a connection.
function setUpPublishConnection(): void {
setupTest();
// describe("PublishConnection", () => {
// // let fakeBlurProcessor: ProcessorWrapper<BackgroundOptions>;
// let roomFactoryMock: Mock<() => LivekitRoom>;
// let muteStates: MockedObject<MuteStates>;
roomFactoryMock = vi.fn().mockReturnValue(fakeLivekitRoom);
// function setUpPublishConnection(): void {
// setupTest();
muteStates = mockMuteStates();
// roomFactoryMock = vi.fn().mockReturnValue(fakeLivekitRoom);
// fakeBlurProcessor = vi.mocked<ProcessorWrapper<BackgroundOptions>>({
// name: "BackgroundBlur",
// restart: vi.fn().mockResolvedValue(undefined),
// setOptions: vi.fn().mockResolvedValue(undefined),
// getOptions: vi.fn().mockReturnValue({ strength: 0.5 }),
// isRunning: vi.fn().mockReturnValue(false)
// });
}
// muteStates = mockMuteStates();
describe("Livekit room creation", () => {
function createSetup(): void {
setUpPublishConnection();
// // fakeBlurProcessor = vi.mocked<ProcessorWrapper<BackgroundOptions>>({
// // name: "BackgroundBlur",
// // restart: vi.fn().mockResolvedValue(undefined),
// // setOptions: vi.fn().mockResolvedValue(undefined),
// // getOptions: vi.fn().mockReturnValue({ strength: 0.5 }),
// // isRunning: vi.fn().mockReturnValue(false)
// // });
// }
const fakeTrackProcessorSubject$ = new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
});
// describe("Livekit room creation", () => {
// function createSetup(): void {
// setUpPublishConnection();
const opts: ConnectionOpts = {
client: client,
transport: livekitFocus,
remoteTransports$: fakeMembershipsFocusMap$,
scope: testScope,
livekitRoomFactory: roomFactoryMock,
};
// const fakeTrackProcessorSubject$ = new BehaviorSubject<ProcessorState>({
// supported: true,
// processor: undefined,
// });
const audioInput = {
available$: of(new Map([["mic1", { id: "mic1" }]])),
selected$: new BehaviorSubject({ id: "mic1" }),
select(): void {},
};
// const opts: ConnectionOpts = {
// client: client,
// transport: livekitFocus,
// scope: testScope,
// livekitRoomFactory: roomFactoryMock,
// };
const videoInput = {
available$: of(new Map([["cam1", { id: "cam1" }]])),
selected$: new BehaviorSubject({ id: "cam1" }),
select(): void {},
};
// const audioInput = {
// available$: of(new Map([["mic1", { id: "mic1" }]])),
// selected$: new BehaviorSubject({ id: "mic1" }),
// select(): void {},
// };
const audioOutput = {
available$: of(new Map([["speaker", { id: "speaker" }]])),
selected$: new BehaviorSubject({ id: "speaker" }),
select(): void {},
};
// const videoInput = {
// available$: of(new Map([["cam1", { id: "cam1" }]])),
// selected$: new BehaviorSubject({ id: "cam1" }),
// select(): void {},
// };
// TODO understand what is wrong with our mocking that requires ts-expect-error
const fakeDevices = mockMediaDevices({
// @ts-expect-error Mocking only
audioInput,
// @ts-expect-error Mocking only
videoInput,
// @ts-expect-error Mocking only
audioOutput,
});
// const audioOutput = {
// available$: of(new Map([["speaker", { id: "speaker" }]])),
// selected$: new BehaviorSubject({ id: "speaker" }),
// select(): void {},
// };
new PublishConnection(
opts,
fakeDevices,
muteStates,
undefined,
fakeTrackProcessorSubject$,
);
}
// // TODO understand what is wrong with our mocking that requires ts-expect-error
// const fakeDevices = mockMediaDevices({
// // @ts-expect-error Mocking only
// audioInput,
// // @ts-expect-error Mocking only
// videoInput,
// // @ts-expect-error Mocking only
// audioOutput,
// });
it("should create room with proper initial audio and video settings", () => {
createSetup();
// new Connection(
// opts,
// fakeDevices,
// muteStates,
// undefined,
// fakeTrackProcessorSubject$,
// );
// }
expect(roomFactoryMock).toHaveBeenCalled();
// it("should create room with proper initial audio and video settings", () => {
// createSetup();
const lastCallArgs =
roomFactoryMock.mock.calls[roomFactoryMock.mock.calls.length - 1];
// expect(roomFactoryMock).toHaveBeenCalled();
const roomOptions = lastCallArgs.pop() as unknown as RoomOptions;
expect(roomOptions).toBeDefined();
// const lastCallArgs =
// roomFactoryMock.mock.calls[roomFactoryMock.mock.calls.length - 1];
expect(roomOptions!.videoCaptureDefaults?.deviceId).toEqual("cam1");
expect(roomOptions!.audioCaptureDefaults?.deviceId).toEqual("mic1");
expect(roomOptions!.audioOutput?.deviceId).toEqual("speaker");
});
// const roomOptions = lastCallArgs.pop() as unknown as RoomOptions;
// expect(roomOptions).toBeDefined();
it("respect controlledAudioDevices", () => {
// TODO: Refactor the code to make it testable.
// The UrlParams module is a singleton has a cache and is very hard to test.
// This breaks other tests as well if not handled properly.
// vi.mock(import("./../UrlParams"), () => {
// return {
// getUrlParams: vi.fn().mockReturnValue({
// controlledAudioDevices: true
// })
// };
// });
});
});
});
// expect(roomOptions!.videoCaptureDefaults?.deviceId).toEqual("cam1");
// expect(roomOptions!.audioCaptureDefaults?.deviceId).toEqual("mic1");
// expect(roomOptions!.audioOutput?.deviceId).toEqual("speaker");
// });
// it("respect controlledAudioDevices", () => {
// // TODO: Refactor the code to make it testable.
// // The UrlParams module is a singleton has a cache and is very hard to test.
// // This breaks other tests as well if not handled properly.
// // vi.mock(import("./../UrlParams"), () => {
// // return {
// // getUrlParams: vi.fn().mockReturnValue({
// // controlledAudioDevices: true
// // })
// // };
// // });
// });
// });
// });

View File

@@ -1,4 +1,5 @@
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
@@ -18,7 +19,7 @@ import {
RoomEvent,
} from "livekit-client";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, type Observable } from "rxjs";
import { BehaviorSubject, map, type Observable } from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger";
import {
@@ -98,6 +99,7 @@ export class Connection {
// TODO dont make this throw and instead store a connection error state in this class?
// TODO consider an autostart pattern...
public async start(): Promise<void> {
this.logger.debug("Starting Connection");
this.stopped = false;
try {
this._state$.next({
@@ -145,6 +147,7 @@ export class Connection {
livekitConnectionState$: connectionStateObserver(this.livekitRoom),
});
} catch (error) {
this.logger.debug(`Failed to connect to LiveKit room: ${error}`);
this._state$.next({
state: "FailedToStart",
error: error instanceof Error ? error : new Error(`${error}`),
@@ -169,6 +172,9 @@ export class Connection {
* If the connection is already stopped, this is a no-op.
*/
public async stop(): Promise<void> {
this.logger.debug(
`Stopping connection to ${this.transport.livekit_service_url}`,
);
if (this.stopped) return;
await this.livekitRoom.disconnect();
this._state$.next({
@@ -179,12 +185,13 @@ export class Connection {
}
/**
* An observable of the participants that are publishing on this connection.
* An observable of the participants that are publishing on this connection. (Excluding our local participant)
* This is derived from `participantsIncludingSubscribers$` and `remoteTransports$`.
* It filters the participants to only those that are associated with a membership that claims to publish on this connection.
*/
public readonly participantsWithTrack$: Behavior<PublishingParticipant[]>;
public readonly remoteParticipantsWithTracks$: Behavior<
PublishingParticipant[]
>;
/**
* The media transport to connect to.
@@ -194,15 +201,18 @@ export class Connection {
private readonly client: OpenIDClientParts;
public readonly livekitRoom: LivekitRoom;
private readonly logger: Logger;
/**
* Creates a new connection to a matrix RTC LiveKit backend.
*
* @param livekitRoom - LiveKit room instance to use.
* @param opts - Connection options {@link ConnectionOpts}.
*
* @param logger
*/
public constructor(opts: ConnectionOpts, logger?: Logger) {
logger?.info(
public constructor(opts: ConnectionOpts, logger: Logger) {
this.logger = logger.getChild("[Connection]");
this.logger.info(
`[Connection] Creating new connection to ${opts.transport.livekit_service_url} ${opts.transport.livekit_alias}`,
);
const { transport, client, scope } = opts;
@@ -211,16 +221,28 @@ export class Connection {
this.transport = transport;
this.client = client;
this.participantsWithTrack$ = scope.behavior(
// REMOTE participants with track!!!
// this.remoteParticipantsWithTracks$
this.remoteParticipantsWithTracks$ = scope.behavior(
// only tracks remote participants
connectedParticipantsObserver(this.livekitRoom, {
additionalRoomEvents: [
RoomEvent.TrackPublished,
RoomEvent.TrackUnpublished,
],
}),
}).pipe(
map((participants) => {
return participants.filter(
(participant) => participant.getTrackPublications().length > 0,
);
}),
),
[],
);
scope.onEnd(() => void this.stop());
scope.onEnd(() => {
this.logger.info(`Connection scope ended, stopping connection`);
void this.stop();
});
}
}

View File

@@ -10,8 +10,10 @@ import {
type E2EEOptions,
Room as LivekitRoom,
type RoomOptions,
type BaseKeyProvider,
} from "livekit-client";
import { type Logger } from "matrix-js-sdk/lib/logger";
import E2EEWorker from "livekit-client/e2ee-worker?worker";
import { type ObservableScope } from "../../ObservableScope.ts";
import { Connection } from "./Connection.ts";
@@ -21,6 +23,7 @@ import type { Behavior } from "../../Behavior.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { defaultLiveKitOptions } from "../../../livekit/options.ts";
// TODO evaluate if this should be done like the Publisher Factory
export interface ConnectionFactory {
createConnection(
transport: LivekitTransport,
@@ -46,7 +49,7 @@ export class ECConnectionFactory implements ConnectionFactory {
private client: OpenIDClientParts,
private devices: MediaDevices,
private processorState$: Behavior<ProcessorState>,
private e2eeLivekitOptions: E2EEOptions | undefined,
livekitKeyProvider: BaseKeyProvider | undefined,
private controlledAudioDevices: boolean,
livekitRoomFactory?: () => LivekitRoom,
) {
@@ -55,7 +58,12 @@ export class ECConnectionFactory implements ConnectionFactory {
generateRoomOption(
this.devices,
this.processorState$.value,
this.e2eeLivekitOptions,
livekitKeyProvider && {
keyProvider: livekitKeyProvider,
// It's important that every room use a separate E2EE worker.
// They get confused if given streams from multiple rooms.
worker: new E2EEWorker(),
},
this.controlledAudioDevices,
),
);

View File

@@ -9,16 +9,18 @@ import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { BehaviorSubject } from "rxjs";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { type Participant as LivekitParticipant } from "livekit-client";
import { logger } from "matrix-js-sdk/lib/logger";
import { ObservableScope } from "../../ObservableScope.ts";
import { Epoch, mapEpoch, ObservableScope } from "../../ObservableScope.ts";
import {
type IConnectionManager,
createConnectionManager$,
type ConnectionManagerData,
} from "./ConnectionManager.ts";
import { type ConnectionFactory } from "./ConnectionFactory.ts";
import { type Connection } from "./Connection.ts";
import { flushPromises, withTestScheduler } from "../../../utils/test.ts";
import { withTestScheduler } from "../../../utils/test.ts";
import { areLivekitTransportsEqual } from "./MatrixLivekitMembers.ts";
import { type Behavior } from "../../Behavior.ts";
// Some test constants
@@ -34,23 +36,15 @@ const TRANSPORT_2: LivekitTransport = {
livekit_alias: "!alias:sample.com",
};
// const TRANSPORT_3: LivekitTransport = {
// type: "livekit",
// livekit_service_url: "https://lk-other.sample.com",
// livekit_alias: "!alias:sample.com",
// };
let fakeConnectionFactory: ConnectionFactory;
let testScope: ObservableScope;
let testTransportStream$: BehaviorSubject<LivekitTransport[]>;
let connectionManagerInputs: {
scope: ObservableScope;
connectionFactory: ConnectionFactory;
inputTransports$: BehaviorSubject<LivekitTransport[]>;
};
let manager: IConnectionManager;
// Can be useful to track all created connections in tests, even the disposed ones
let allCreatedConnections: Connection[];
beforeEach(() => {
testScope = new ObservableScope();
allCreatedConnections = [];
fakeConnectionFactory = {} as unknown as ConnectionFactory;
vi.mocked(fakeConnectionFactory).createConnection = vi
.fn()
@@ -58,6 +52,7 @@ beforeEach(() => {
(transport: LivekitTransport, scope: ObservableScope) => {
const mockConnection = {
transport,
remoteParticipantsWithTracks$: new BehaviorSubject([]),
} as unknown as Connection;
vi.mocked(mockConnection).start = vi.fn();
vi.mocked(mockConnection).stop = vi.fn();
@@ -65,17 +60,10 @@ beforeEach(() => {
scope.onEnd(() => {
void mockConnection.stop();
});
allCreatedConnections.push(mockConnection);
return mockConnection;
},
);
testTransportStream$ = new BehaviorSubject<LivekitTransport[]>([]);
connectionManagerInputs = {
scope: testScope,
connectionFactory: fakeConnectionFactory,
inputTransports$: testTransportStream$,
};
manager = createConnectionManager$(connectionManagerInputs);
});
afterEach(() => {
@@ -83,101 +71,139 @@ afterEach(() => {
});
describe("connections$ stream", () => {
test("Should create and start new connections for each transports", async () => {
const managedConnections = Promise.withResolvers<Connection[]>();
manager.connections$.subscribe((connections) => {
if (connections.length > 0) managedConnections.resolve(connections);
test("Should create and start new connections for each transports", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("a", {
a: new Epoch([TRANSPORT_1, TRANSPORT_2], 0),
}),
logger: logger,
});
expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("a", {
a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value;
expect(connections.length).toBe(2);
expect(
vi.mocked(fakeConnectionFactory).createConnection,
).toHaveBeenCalledTimes(2);
const conn1 = connections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_1),
);
expect(conn1).toBeDefined();
expect(conn1!.start).toHaveBeenCalled();
const conn2 = connections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_2),
);
expect(conn2).toBeDefined();
expect(conn2!.start).toHaveBeenCalled();
return true;
}),
});
});
connectionManagerInputs.inputTransports$.next([TRANSPORT_1, TRANSPORT_2]);
const connections = await managedConnections.promise;
expect(connections.length).toBe(2);
expect(
vi.mocked(fakeConnectionFactory).createConnection,
).toHaveBeenCalledTimes(2);
const conn1 = connections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_1),
);
expect(conn1).toBeDefined();
expect(conn1!.start).toHaveBeenCalled();
const conn2 = connections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_2),
);
expect(conn2).toBeDefined();
expect(conn2!.start).toHaveBeenCalled();
});
test("Should start connection only once", async () => {
const observedConnections: Connection[][] = [];
manager.connections$.subscribe((connections) => {
observedConnections.push(connections);
test("Should start connection only once", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("abcdef", {
a: new Epoch([TRANSPORT_1], 0),
b: new Epoch([TRANSPORT_1], 1),
c: new Epoch([TRANSPORT_1], 2),
d: new Epoch([TRANSPORT_1], 3),
e: new Epoch([TRANSPORT_1], 4),
f: new Epoch([TRANSPORT_1, TRANSPORT_2], 5),
}),
logger: logger,
});
expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("xxxxxa", {
x: expect.anything(),
a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value;
expect(connections.length).toBe(2);
expect(
vi.mocked(fakeConnectionFactory).createConnection,
).toHaveBeenCalledTimes(2);
const conn2 = connections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_2),
);
expect(conn2).toBeDefined();
const conn1 = connections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_1),
);
expect(conn1).toBeDefined();
expect(conn1!.start).toHaveBeenCalledOnce();
return true;
}),
});
});
testTransportStream$.next([TRANSPORT_1]);
testTransportStream$.next([TRANSPORT_1]);
testTransportStream$.next([TRANSPORT_1]);
testTransportStream$.next([TRANSPORT_1]);
testTransportStream$.next([TRANSPORT_1]);
testTransportStream$.next([TRANSPORT_1, TRANSPORT_2]);
await flushPromises();
const connections = observedConnections.pop()!;
expect(connections.length).toBe(2);
expect(
vi.mocked(fakeConnectionFactory).createConnection,
).toHaveBeenCalledTimes(2);
const conn2 = connections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_2),
);
expect(conn2).toBeDefined();
const conn1 = connections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_1),
);
expect(conn1).toBeDefined();
expect(conn1!.start).toHaveBeenCalledOnce();
});
test("Should cleanup connections when not needed anymore", async () => {
const observedConnections: Connection[][] = [];
manager.connections$.subscribe((connections) => {
observedConnections.push(connections);
test("Should cleanup connections when not needed anymore", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("abc", {
a: new Epoch([TRANSPORT_1], 0),
b: new Epoch([TRANSPORT_1, TRANSPORT_2], 1),
c: new Epoch([TRANSPORT_1], 2),
}),
logger: logger,
});
expectObservable(
connectionManagerData$.pipe(mapEpoch((d) => d.getConnections())),
).toBe("xab", {
x: expect.anything(),
a: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value;
expect(connections.length).toBe(2);
return true;
}),
b: expect.toSatisfy((e: Epoch<Connection[]>) => {
const connections = e.value;
expect(connections.length).toBe(1);
// The second connection should have been stopped has it is no longer needed.
const connection2 = allCreatedConnections.find((c) =>
areLivekitTransportsEqual(c.transport, TRANSPORT_2),
);
expect(connection2).toBeDefined();
expect(connection2!.stop).toHaveBeenCalled();
// The first connection should still be active
const conn1 = connections[0];
expect(conn1.stop).not.toHaveBeenCalledOnce();
return true;
}),
});
});
testTransportStream$.next([TRANSPORT_1]);
testTransportStream$.next([TRANSPORT_1, TRANSPORT_2]);
await flushPromises();
const conn2 = observedConnections
.pop()!
.find((c) => areLivekitTransportsEqual(c.transport, TRANSPORT_2))!;
testTransportStream$.next([TRANSPORT_1]);
await flushPromises();
// The second connection should have been stopped has it is no longer needed
expect(conn2.stop).toHaveBeenCalled();
// The first connection should still be active
const conn1 = observedConnections.pop()![0];
expect(conn1.stop).not.toHaveBeenCalledOnce();
});
});
describe("connectionManagerData$ stream", () => {
// Used in test to control fake connections' participantsWithTrack$ streams
// Used in test to control fake connections' remoteParticipantsWithTracks$ streams
let fakePublishingParticipantsStreams: Map<
string,
BehaviorSubject<LivekitParticipant[]>
Behavior<LivekitParticipant[]>
>;
function keyForTransport(transport: LivekitTransport): string {
@@ -186,6 +212,16 @@ describe("connectionManagerData$ stream", () => {
beforeEach(() => {
fakePublishingParticipantsStreams = new Map();
function getPublishingParticipantsFor(
transport: LivekitTransport,
): Behavior<LivekitParticipant[]> {
return (
fakePublishingParticipantsStreams.get(keyForTransport(transport)) ??
new BehaviorSubject([])
);
}
// need a more advanced fake connection factory
vi.mocked(fakeConnectionFactory).createConnection = vi
.fn()
@@ -196,7 +232,8 @@ describe("connectionManagerData$ stream", () => {
>([]);
const mockConnection = {
transport,
participantsWithTrack$: fakePublishingParticipants$,
remoteParticipantsWithTracks$:
getPublishingParticipantsFor(transport),
} as unknown as Connection;
vi.mocked(mockConnection).start = vi.fn();
vi.mocked(mockConnection).stop = vi.fn();
@@ -216,80 +253,84 @@ describe("connectionManagerData$ stream", () => {
test("Should report connections with the publishing participants", () => {
withTestScheduler(({ expectObservable, schedule, behavior }) => {
manager = createConnectionManager$({
...connectionManagerInputs,
inputTransports$: behavior("a", {
a: [TRANSPORT_1, TRANSPORT_2],
}),
});
const conn1Participants$ = fakePublishingParticipantsStreams.get(
// Setup the fake participants streams behavior
// ==============================
fakePublishingParticipantsStreams.set(
keyForTransport(TRANSPORT_1),
)!;
schedule("-a-b", {
a: () => {
conn1Participants$.next([
{ identity: "user1A" } as LivekitParticipant,
]);
},
b: () => {
conn1Participants$.next([
behavior("oa-b", {
o: [],
a: [{ identity: "user1A" } as LivekitParticipant],
b: [
{ identity: "user1A" } as LivekitParticipant,
{ identity: "user1B" } as LivekitParticipant,
]);
},
});
],
}),
);
const conn2Participants$ = fakePublishingParticipantsStreams.get(
fakePublishingParticipantsStreams.set(
keyForTransport(TRANSPORT_2),
)!;
behavior("o-a", {
o: [],
a: [{ identity: "user2A" } as LivekitParticipant],
}),
);
// ==============================
schedule("--a", {
a: () => {
conn2Participants$.next([
{ identity: "user2A" } as LivekitParticipant,
]);
},
const { connectionManagerData$ } = createConnectionManager$({
scope: testScope,
connectionFactory: fakeConnectionFactory,
inputTransports$: behavior("a", {
a: new Epoch([TRANSPORT_1, TRANSPORT_2], 0),
}),
logger,
});
expectObservable(manager.connectionManagerData$).toBe("abcd", {
a: expect.toSatisfy((data) => {
return (
data.getConnections().length == 2 &&
data.getParticipantForTransport(TRANSPORT_1).length == 0 &&
data.getParticipantForTransport(TRANSPORT_2).length == 0
);
expectObservable(connectionManagerData$).toBe("abcd", {
a: expect.toSatisfy((e) => {
const data: ConnectionManagerData = e.value;
expect(data.getConnections().length).toBe(2);
expect(data.getParticipantForTransport(TRANSPORT_1).length).toBe(0);
expect(data.getParticipantForTransport(TRANSPORT_2).length).toBe(0);
return true;
}),
b: expect.toSatisfy((data) => {
return (
data.getConnections().length == 2 &&
data.getParticipantForTransport(TRANSPORT_1).length == 1 &&
data.getParticipantForTransport(TRANSPORT_2).length == 0 &&
data.getParticipantForTransport(TRANSPORT_1)[0].identity == "user1A"
b: expect.toSatisfy((e) => {
const data: ConnectionManagerData = e.value;
expect(data.getConnections().length).toBe(2);
expect(data.getParticipantForTransport(TRANSPORT_1).length).toBe(1);
expect(data.getParticipantForTransport(TRANSPORT_2).length).toBe(0);
expect(data.getParticipantForTransport(TRANSPORT_1)[0].identity).toBe(
"user1A",
);
return true;
}),
c: expect.toSatisfy((data) => {
return (
data.getConnections().length == 2 &&
data.getParticipantForTransport(TRANSPORT_1).length == 1 &&
data.getParticipantForTransport(TRANSPORT_2).length == 1 &&
data.getParticipantForTransport(TRANSPORT_1)[0].identity ==
"user1A" &&
data.getParticipantForTransport(TRANSPORT_2)[0].identity == "user2A"
c: expect.toSatisfy((e) => {
const data: ConnectionManagerData = e.value;
expect(data.getConnections().length).toBe(2);
expect(data.getParticipantForTransport(TRANSPORT_1).length).toBe(1);
expect(data.getParticipantForTransport(TRANSPORT_2).length).toBe(1);
expect(data.getParticipantForTransport(TRANSPORT_1)[0].identity).toBe(
"user1A",
);
expect(data.getParticipantForTransport(TRANSPORT_2)[0].identity).toBe(
"user2A",
);
return true;
}),
d: expect.toSatisfy((data) => {
return (
data.getConnections().length == 2 &&
data.getParticipantForTransport(TRANSPORT_1).length == 2 &&
data.getParticipantForTransport(TRANSPORT_2).length == 1 &&
data.getParticipantForTransport(TRANSPORT_1)[0].identity ==
"user1A" &&
data.getParticipantForTransport(TRANSPORT_1)[1].identity ==
"user1B" &&
data.getParticipantForTransport(TRANSPORT_2)[0].identity == "user2A"
d: expect.toSatisfy((e) => {
const data: ConnectionManagerData = e.value;
expect(data.getConnections().length).toBe(2);
expect(data.getParticipantForTransport(TRANSPORT_1).length).toBe(2);
expect(data.getParticipantForTransport(TRANSPORT_2).length).toBe(1);
expect(data.getParticipantForTransport(TRANSPORT_1)[0].identity).toBe(
"user1A",
);
expect(data.getParticipantForTransport(TRANSPORT_1)[1].identity).toBe(
"user1B",
);
expect(data.getParticipantForTransport(TRANSPORT_2)[0].identity).toBe(
"user2A",
);
return true;
}),
});
});

View File

@@ -1,6 +1,3 @@
// TODOs:
// - make ConnectionManager its own actual class
/*
Copyright 2025 Element Creations Ltd.
Copyright 2025 New Vector Ltd.
@@ -13,14 +10,14 @@ import {
type LivekitTransport,
type ParticipantId,
} from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, combineLatest, map, of, switchMap } from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { BehaviorSubject, combineLatest, map, of, switchMap, tap } from "rxjs";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { type LocalParticipant, type RemoteParticipant } from "livekit-client";
import { type Behavior } from "../../Behavior.ts";
import { type Connection } from "./Connection.ts";
import { Epoch, type ObservableScope } from "../../ObservableScope.ts";
import { generateKeyed$ } from "../../../utils/observable.ts";
import { generateItemsWithEpoch } from "../../../utils/observable.ts";
import { areLivekitTransportsEqual } from "./MatrixLivekitMembers.ts";
import { type ConnectionFactory } from "./ConnectionFactory.ts";
@@ -55,8 +52,8 @@ export class ConnectionManagerData {
public getConnectionForTransport(
transport: LivekitTransport,
): Connection | undefined {
return this.store.get(this.getKey(transport))?.[0];
): Connection | null {
return this.store.get(this.getKey(transport))?.[0] ?? null;
}
public getParticipantForTransport(
@@ -91,12 +88,12 @@ interface Props {
scope: ObservableScope;
connectionFactory: ConnectionFactory;
inputTransports$: Behavior<Epoch<LivekitTransport[]>>;
logger: Logger;
}
// TODO - write test for scopes (do we really need to bind scope)
export interface IConnectionManager {
transports$: Behavior<Epoch<LivekitTransport[]>>;
connectionManagerData$: Behavior<Epoch<ConnectionManagerData>>;
connections$: Behavior<Epoch<Connection[]>>;
}
/**
* Crete a `ConnectionManager`
@@ -116,8 +113,9 @@ export function createConnectionManager$({
scope,
connectionFactory,
inputTransports$,
logger: parentLogger,
}: Props): IConnectionManager {
const logger = rootLogger.getChild("ConnectionManager");
const logger = parentLogger.getChild("[ConnectionManager]");
const running$ = new BehaviorSubject(true);
scope.onEnd(() => running$.next(false));
@@ -137,6 +135,11 @@ export function createConnectionManager$({
transports.mapInner((transport) => (running ? transport : [])),
),
map((transports) => transports.mapInner(removeDuplicateTransports)),
tap(({ value: transports }) => {
logger.trace(
`Managing transports: ${transports.map((t) => t.livekit_service_url).join(", ")}`,
);
}),
),
);
@@ -144,34 +147,33 @@ export function createConnectionManager$({
* Connections for each transport in use by one or more session members.
*/
const connections$ = scope.behavior(
generateKeyed$<Epoch<LivekitTransport[]>, Connection, Epoch<Connection[]>>(
transports$,
(transports, createOrGet) => {
const createConnection =
(
transport: LivekitTransport,
): ((scope: ObservableScope) => Connection) =>
(scope) => {
const connection = connectionFactory.createConnection(
transport,
scope,
logger,
);
// Start the connection immediately
// Use connection state to track connection progress
void connection.start();
// TODO subscribe to connection state to retry or log issues?
return connection;
};
return transports.mapInner((transports) => {
return transports.map((transport) => {
const key =
transport.livekit_service_url + "|" + transport.livekit_alias;
return createOrGet(key, createConnection(transport));
});
});
},
transports$.pipe(
generateItemsWithEpoch(
function* (transports) {
for (const transport of transports)
yield {
keys: [transport.livekit_service_url, transport.livekit_alias],
data: undefined,
};
},
(scope, _data$, serviceUrl, alias) => {
logger.debug(`Creating connection to ${serviceUrl} (${alias})`);
const connection = connectionFactory.createConnection(
{
type: "livekit",
livekit_service_url: serviceUrl,
livekit_alias: alias,
},
scope,
logger,
);
// Start the connection immediately
// Use connection state to track connection progress
void connection.start();
// TODO subscribe to connection state to retry or log issues?
return connection;
},
),
),
);
@@ -183,7 +185,7 @@ export function createConnectionManager$({
// Map the connections to list of {connection, participants}[]
const listOfConnectionsWithPublishingParticipants =
connections.value.map((connection) => {
return connection.participantsWithTrack$.pipe(
return connection.remoteParticipantsWithTracks$.pipe(
map((participants) => ({
connection,
participants,
@@ -214,7 +216,7 @@ export function createConnectionManager$({
new Epoch(new ConnectionManagerData()),
);
return { transports$, connectionManagerData$, connections$ };
return { transports$, connectionManagerData$ };
}
function removeDuplicateTransports(

View File

@@ -5,161 +5,196 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, test, vi, expect, beforeEach, afterEach } from "vitest";
import { describe, test, expect, beforeEach, afterEach } from "vitest";
import {
type CallMembership,
type LivekitTransport,
} from "matrix-js-sdk/lib/matrixrtc";
import { type Room as MatrixRoom, type RoomMember } from "matrix-js-sdk";
import { getParticipantId } from "matrix-js-sdk/lib/matrixrtc/utils";
import { combineLatest, map, type Observable } from "rxjs";
import { type IConnectionManager } from "./ConnectionManager.ts";
import {
type MatrixLivekitMember,
createMatrixLivekitMembers$,
areLivekitTransportsEqual,
} from "./MatrixLivekitMembers.ts";
import { ObservableScope } from "../../ObservableScope.ts";
import {
Epoch,
mapEpoch,
ObservableScope,
trackEpoch,
} from "../../ObservableScope.ts";
import { ConnectionManagerData } from "./ConnectionManager.ts";
import {
mockCallMembership,
mockRemoteParticipant,
type OurRunHelpers,
withTestScheduler,
} from "../../../utils/test.ts";
import { type Connection } from "./Connection.ts";
let testScope: ObservableScope;
let mockMatrixRoom: MatrixRoom;
// The merger beeing tested
const transportA: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
livekit_alias: "!alias:example.org",
};
const transportB: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.sample.com",
livekit_alias: "!alias:sample.com",
};
const bobMembership = mockCallMembership(
"@bob:example.org",
"DEV000",
transportA,
);
const carlMembership = mockCallMembership(
"@carl:sample.com",
"DEV111",
transportB,
);
beforeEach(() => {
testScope = new ObservableScope();
mockMatrixRoom = vi.mocked<MatrixRoom>({
getMember: vi.fn().mockImplementation((userId: string) => {
return {
userId,
rawDisplayName: userId.replace("@", "").replace(":example.org", ""),
getMxcAvatarUrl: vi.fn().mockReturnValue(null),
} as unknown as RoomMember;
}),
addEventListener: vi.fn(),
removeEventListener: vi.fn(),
} as unknown as MatrixRoom);
});
afterEach(() => {
testScope.end();
});
function epochMeWith$<T, U>(
source$: Observable<Epoch<U>>,
me$: Observable<T>,
): Observable<Epoch<T>> {
return combineLatest([source$, me$]).pipe(
map(([ep, cd]) => {
return new Epoch(cd, ep.epoch);
}),
);
}
test("should signal participant not yet connected to livekit", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const bobMembership = {
userId: "@bob:example.org",
deviceId: "DEV000",
transports: [
{
type: "livekit",
livekit_service_url: "https://lk.example.org",
livekit_alias: "!alias:example.org",
},
],
} as unknown as CallMembership;
const { memberships$, membershipsWithTransport$ } = fromMemberships$(
behavior("a", {
a: [bobMembership],
}),
);
const connectionManagerData$ = epochMeWith$(
memberships$,
behavior("a", {
a: new ConnectionManagerData(),
}),
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: behavior("a", {
a: [
{
membership: bobMembership,
},
],
}),
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
connectionManager: {
connectionManagerData$: behavior("a", {
a: new ConnectionManagerData(),
}),
transports$: behavior("a", { a: [] }),
connections$: behavior("a", { a: [] }),
},
matrixRoom: mockMatrixRoom,
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
});
expectObservable(matrixLivekitMember$).toBe("a", {
expectObservable(matrixLivekitMember$.pipe(map((e) => e.value))).toBe("a", {
a: expect.toSatisfy((data: MatrixLivekitMember[]) => {
return (
data.length == 1 &&
data[0].membership === bobMembership &&
data[0].participant === undefined &&
data[0].connection === undefined
);
expect(data.length).toEqual(1);
expectObservable(data[0].membership$).toBe("a", {
a: bobMembership,
});
expectObservable(data[0].participant$).toBe("a", {
a: null,
});
expectObservable(data[0].connection$).toBe("a", {
a: null,
});
return true;
}),
});
});
});
function aConnectionManager(
data: ConnectionManagerData,
behavior: OurRunHelpers["behavior"],
): IConnectionManager {
return {
connectionManagerData$: behavior("a", { a: data }),
transports$: behavior("a", {
a: data.getConnections().map((connection) => connection.transport),
// Helper to create epoch'ed memberships$ and membershipsWithTransport$ from memberships observable.
function fromMemberships$(m$: Observable<CallMembership[]>): {
memberships$: Observable<Epoch<CallMembership[]>>;
membershipsWithTransport$: Observable<
Epoch<{ membership: CallMembership; transport?: LivekitTransport }[]>
>;
} {
const memberships$ = m$.pipe(trackEpoch());
const membershipsWithTransport$ = memberships$.pipe(
mapEpoch((members) => {
return members.map((m) => {
const tr = m.getTransport(m);
return {
membership: m,
transport:
tr?.type === "livekit" ? (tr as LivekitTransport) : undefined,
};
});
}),
connections$: behavior("a", { a: data.getConnections() }),
);
return {
memberships$,
membershipsWithTransport$,
};
}
test("should signal participant on a connection that is publishing", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const transport: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
livekit_alias: "!alias:example.org",
};
const bobMembership = mockCallMembership(
"@bob:example.org",
"DEV000",
transport,
);
const connectionWithPublisher = new ConnectionManagerData();
const bobParticipantId = getParticipantId(
bobMembership.userId,
bobMembership.deviceId,
);
const { memberships$, membershipsWithTransport$ } = fromMemberships$(
behavior("a", {
a: [bobMembership],
}),
);
const connection = {
transport: transport,
transport: bobMembership.getTransport(bobMembership),
} as unknown as Connection;
connectionWithPublisher.add(connection, [
const dataWithPublisher = new ConnectionManagerData();
dataWithPublisher.add(connection, [
mockRemoteParticipant({ identity: bobParticipantId }),
]);
const connectionManagerData$ = epochMeWith$(
memberships$,
behavior("a", {
a: dataWithPublisher,
}),
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: behavior("a", {
a: [
{
membership: bobMembership,
transport,
},
],
}),
connectionManager: aConnectionManager(connectionWithPublisher, behavior),
matrixRoom: mockMatrixRoom,
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
});
expectObservable(matrixLivekitMember$).toBe("a", {
expectObservable(matrixLivekitMember$.pipe(map((e) => e.value))).toBe("a", {
a: expect.toSatisfy((data: MatrixLivekitMember[]) => {
expect(data.length).toEqual(1);
expect(data[0].participant).toBeDefined();
expect(data[0].connection).toBeDefined();
expect(data[0].membership).toEqual(bobMembership);
expect(
areLivekitTransportsEqual(data[0].connection!.transport, transport),
).toBe(true);
expectObservable(data[0].membership$).toBe("a", {
a: bobMembership,
});
expectObservable(data[0].participant$).toBe("a", {
a: expect.toSatisfy((participant) => {
expect(participant).toBeDefined();
expect(participant!.identity).toEqual(bobParticipantId);
return true;
}),
});
expectObservable(data[0].connection$).toBe("a", {
a: connection,
});
return true;
}),
});
@@ -168,47 +203,45 @@ test("should signal participant on a connection that is publishing", () => {
test("should signal participant on a connection that is not publishing", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const transport: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
livekit_alias: "!alias:example.org",
};
const bobMembership = mockCallMembership(
"@bob:example.org",
"DEV000",
transport,
const { memberships$, membershipsWithTransport$ } = fromMemberships$(
behavior("a", {
a: [bobMembership],
}),
);
const connectionWithPublisher = new ConnectionManagerData();
// const bobParticipantId = getParticipantId(bobMembership.userId, bobMembership.deviceId);
const connection = {
transport: transport,
transport: bobMembership.getTransport(bobMembership),
} as unknown as Connection;
connectionWithPublisher.add(connection, []);
const dataWithPublisher = new ConnectionManagerData();
dataWithPublisher.add(connection, []);
const connectionManagerData$ = epochMeWith$(
memberships$,
behavior("a", {
a: dataWithPublisher,
}),
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: behavior("a", {
a: [
{
membership: bobMembership,
transport,
},
],
}),
connectionManager: aConnectionManager(connectionWithPublisher, behavior),
matrixRoom: mockMatrixRoom,
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
});
expectObservable(matrixLivekitMember$).toBe("a", {
expectObservable(matrixLivekitMember$.pipe(map((e) => e.value))).toBe("a", {
a: expect.toSatisfy((data: MatrixLivekitMember[]) => {
expect(data.length).toEqual(1);
expect(data[0].participant).not.toBeDefined();
expect(data[0].connection).toBeDefined();
expect(data[0].membership).toEqual(bobMembership);
expect(
areLivekitTransportsEqual(data[0].connection!.transport, transport),
).toBe(true);
expectObservable(data[0].membership$).toBe("a", {
a: bobMembership,
});
expectObservable(data[0].participant$).toBe("a", {
a: null,
});
expectObservable(data[0].connection$).toBe("a", {
a: connection,
});
return true;
}),
});
@@ -218,22 +251,10 @@ test("should signal participant on a connection that is not publishing", () => {
describe("Publication edge case", () => {
test("bob is publishing in several connections", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const transportA: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
livekit_alias: "!alias:example.org",
};
const transportB: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.sample.com",
livekit_alias: "!alias:sample.com",
};
const bobMembership = mockCallMembership(
"@bob:example.org",
"DEV000",
transportA,
const { memberships$, membershipsWithTransport$ } = fromMemberships$(
behavior("a", {
a: [bobMembership, carlMembership],
}),
);
const connectionWithPublisher = new ConnectionManagerData();
@@ -254,60 +275,56 @@ describe("Publication edge case", () => {
connectionWithPublisher.add(connectionB, [
mockRemoteParticipant({ identity: bobParticipantId }),
]);
const connectionManagerData$ = epochMeWith$(
memberships$,
behavior("a", {
a: connectionWithPublisher,
}),
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: behavior("a", {
a: [
{
membership: bobMembership,
transport: transportA,
},
],
}),
connectionManager: aConnectionManager(
connectionWithPublisher,
behavior,
membershipsWithTransport$: testScope.behavior(
membershipsWithTransport$,
),
matrixRoom: mockMatrixRoom,
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
});
expectObservable(matrixLivekitMember$).toBe("a", {
a: expect.toSatisfy((data: MatrixLivekitMember[]) => {
expect(data.length).toEqual(1);
expect(data[0].participant).toBeDefined();
expect(data[0].participant!.identity).toEqual(bobParticipantId);
expect(data[0].connection).toBeDefined();
expect(data[0].membership).toEqual(bobMembership);
expect(
areLivekitTransportsEqual(
data[0].connection!.transport,
transportA,
),
).toBe(true);
return true;
}),
});
expectObservable(matrixLivekitMember$.pipe(map((e) => e.value))).toBe(
"a",
{
a: expect.toSatisfy((data: MatrixLivekitMember[]) => {
expect(data.length).toEqual(2);
expectObservable(data[0].membership$).toBe("a", {
a: bobMembership,
});
expectObservable(data[0].connection$).toBe("a", {
// The real connection should be from transportA as per the membership
a: connectionA,
});
expectObservable(data[0].participant$).toBe("a", {
a: expect.toSatisfy((participant) => {
expect(participant).toBeDefined();
expect(participant!.identity).toEqual(bobParticipantId);
return true;
}),
});
return true;
}),
},
);
});
});
test("bob is publishing in the wrong connection", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const transportA: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.example.org",
livekit_alias: "!alias:example.org",
};
const transportB: LivekitTransport = {
type: "livekit",
livekit_service_url: "https://lk.sample.com",
livekit_alias: "!alias:sample.com",
};
const bobMembership = mockCallMembership(
"@bob:example.org",
"DEV000",
transportA,
const { memberships$, membershipsWithTransport$ } = fromMemberships$(
behavior("a", {
a: [bobMembership, carlMembership],
}),
);
const connectionWithPublisher = new ConnectionManagerData();
@@ -315,86 +332,53 @@ describe("Publication edge case", () => {
bobMembership.userId,
bobMembership.deviceId,
);
const connectionA = {
transport: transportA,
} as unknown as Connection;
const connectionB = {
transport: transportB,
} as unknown as Connection;
const connectionA = { transport: transportA } as unknown as Connection;
const connectionB = { transport: transportB } as unknown as Connection;
// Bob is not publishing on A
connectionWithPublisher.add(connectionA, []);
// Bob is publishing on B but his membership says A
connectionWithPublisher.add(connectionB, [
mockRemoteParticipant({ identity: bobParticipantId }),
]);
const connectionManagerData$ = epochMeWith$(
memberships$,
behavior("a", {
a: connectionWithPublisher,
}),
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: behavior("a", {
a: [
{
membership: bobMembership,
transport: transportA,
},
],
}),
connectionManager: aConnectionManager(
connectionWithPublisher,
behavior,
membershipsWithTransport$: testScope.behavior(
membershipsWithTransport$,
),
matrixRoom: mockMatrixRoom,
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
});
expectObservable(matrixLivekitMember$).toBe("a", {
a: expect.toSatisfy((data: MatrixLivekitMember[]) => {
expect(data.length).toEqual(1);
expect(data[0].participant).not.toBeDefined();
expect(data[0].connection).toBeDefined();
expect(data[0].membership).toEqual(bobMembership);
expect(
areLivekitTransportsEqual(
data[0].connection!.transport,
transportA,
),
).toBe(true);
return true;
}),
});
expectObservable(matrixLivekitMember$.pipe(map((e) => e.value))).toBe(
"a",
{
a: expect.toSatisfy((data: MatrixLivekitMember[]) => {
expect(data.length).toEqual(2);
expectObservable(data[0].membership$).toBe("a", {
a: bobMembership,
});
expectObservable(data[0].connection$).toBe("a", {
// The real connection should be from transportA as per the membership
a: connectionA,
});
expectObservable(data[0].participant$).toBe("a", {
// No participant as Bob is not publishing on his membership transport
a: null,
});
return true;
}),
},
);
});
// let lastMatrixLkItems: MatrixLivekitMember[] = [];
// matrixLivekitMerger.matrixLivekitMember$.subscribe((items) => {
// lastMatrixLkItems = items;
// });
// vi.mocked(bobMembership).getTransport = vi
// .fn()
// .mockReturnValue(connectionA.transport);
// fakeMemberships$.next([bobMembership]);
// const lkMap = new ConnectionManagerData();
// lkMap.add(connectionA, []);
// lkMap.add(connectionB, [
// mockRemoteParticipant({ identity: bobParticipantId })
// ]);
// fakeManagerData$.next(lkMap);
// const items = lastMatrixLkItems;
// expect(items).toHaveLength(1);
// const item = items[0];
// // Assert the expected membership
// expect(item.membership.userId).toEqual(bobMembership.userId);
// expect(item.membership.deviceId).toEqual(bobMembership.deviceId);
// expect(item.participant).not.toBeDefined();
// // The transport info should come from the membership transports and not only from the publishing connection
// expect(item.connection?.transport?.livekit_service_url).toEqual(
// bobMembership.transports[0]?.livekit_service_url
// );
// expect(item.connection?.transport?.livekit_alias).toEqual(
// bobMembership.transports[0]?.livekit_alias
// );
});
});

View File

@@ -14,34 +14,30 @@ import {
type CallMembership,
} from "matrix-js-sdk/lib/matrixrtc";
import { combineLatest, filter, map } from "rxjs";
// eslint-disable-next-line rxjs/no-internal
import { type NodeStyleEventEmitter } from "rxjs/internal/observable/fromEvent";
import { type Room as MatrixRoom, type RoomMember } from "matrix-js-sdk";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type Behavior } from "../../Behavior";
import { type IConnectionManager } from "./ConnectionManager";
import { Epoch, mapEpoch, type ObservableScope } from "../../ObservableScope";
import { getRoomMemberFromRtcMember, memberDisplaynames$ } from "./displayname";
import { Epoch, type ObservableScope } from "../../ObservableScope";
import { type Connection } from "./Connection";
import { generateItemsWithEpoch } from "../../../utils/observable";
const logger = rootLogger.getChild("[MatrixLivekitMembers]");
/**
* Represent a matrix call member and his associated livekit participation.
* Represents a Matrix call member and their associated LiveKit participation.
* `livekitParticipant` can be undefined if the member is not yet connected to the livekit room
* or if it has no livekit transport at all.
*/
export interface MatrixLivekitMember {
membership: CallMembership;
displayName?: string;
participant?: LocalLivekitParticipant | RemoteLivekitParticipant;
connection?: Connection;
/**
* TODO Try to remove this! Its waaay to much information.
* Just get the member's avatar
* @deprecated
*/
member: RoomMember;
mxcAvatarUrl?: string;
participantId: string;
membership$: Behavior<CallMembership>;
participant$: Behavior<
LocalLivekitParticipant | RemoteLivekitParticipant | null
>;
connection$: Behavior<Connection | null>;
// participantId: string; We do not want a participantId here since it will be generated by the jwt
// TODO decide if we can also drop the userId. Its in the matrix membership anyways.
userId: string;
}
interface Props {
@@ -50,16 +46,7 @@ interface Props {
Epoch<{ membership: CallMembership; transport?: LivekitTransport }[]>
>;
connectionManager: IConnectionManager;
// TODO this is too much information for that class,
// apparently needed to get a room member to later get the Avatar
// => Extract an AvatarService instead?
// Better with just `getMember`
matrixRoom: Pick<MatrixRoom, "getMember"> & NodeStyleEventEmitter;
// roomMember$: Behavior<Pick<RoomMember, "userId" | "getMxcAvatarUrl">>;
}
// Alternative structure idea:
// const livekitMatrixMember$ = (callMemberships$,connectionManager,scope): Observable<MatrixLivekitMember[]> => {
/**
* Combines MatrixRTC and Livekit worlds.
*
@@ -74,23 +61,15 @@ export function createMatrixLivekitMembers$({
scope,
membershipsWithTransport$,
connectionManager,
matrixRoom,
}: Props): Behavior<Epoch<MatrixLivekitMember[]>> {
/**
* Stream of all the call members and their associated livekit data (if available).
*/
const displaynameMap$ = memberDisplaynames$(
scope,
matrixRoom,
membershipsWithTransport$.pipe(mapEpoch((v) => v.map((v) => v.membership))),
);
return scope.behavior(
combineLatest([
membershipsWithTransport$,
connectionManager.connectionManagerData$,
displaynameMap$,
]).pipe(
filter((values) =>
values.every((value) => value.epoch === values[0].epoch),
@@ -99,45 +78,48 @@ export function createMatrixLivekitMembers$({
([
{ value: membershipsWithTransports, epoch },
{ value: managerData },
{ value: displaynames },
]) => {
const items: MatrixLivekitMember[] = membershipsWithTransports.map(
({ membership, transport }) => {
// TODO! cannot use membership.membershipID yet, Currently its hardcoded by the jwt service to
const participantId = /*membership.membershipID*/ `${membership.userId}:${membership.deviceId}`;
]) =>
new Epoch([membershipsWithTransports, managerData] as const, epoch),
),
generateItemsWithEpoch(
// Generator function.
// creates an array of `{key, data}[]`
// Each change in the keys (new key, missing key) will result in a call to the factory function.
function* ([membershipsWithTransports, managerData]) {
for (const { membership, transport } of membershipsWithTransports) {
// TODO! cannot use membership.membershipID yet, Currently its hardcoded by the jwt service to
const participantId = /*membership.membershipID*/ `${membership.userId}:${membership.deviceId}`;
const participants = transport
? managerData.getParticipantForTransport(transport)
: [];
const participant = participants.find(
(p) => p.identity == participantId,
);
const member = getRoomMemberFromRtcMember(
membership,
matrixRoom,
)?.member;
const connection = transport
? managerData.getConnectionForTransport(transport)
: undefined;
const displayName = displaynames.get(participantId);
return {
participant,
membership,
connection,
// This makes sense to add to the js-sdk callMembership (we only need the avatar so probably the call memberhsip just should aquire the avatar)
// TODO Ugh this is hidign that it might be undefined!! best we remove the member entirely.
member: member as RoomMember,
displayName,
mxcAvatarUrl: member?.getMxcAvatarUrl(),
participantId,
};
},
const participants = transport
? managerData.getParticipantForTransport(transport)
: [];
const participant =
participants.find((p) => p.identity == participantId) ?? null;
const connection = transport
? managerData.getConnectionForTransport(transport)
: null;
yield {
keys: [participantId, membership.userId],
data: { membership, participant, connection },
};
}
},
// Each update where the key of the generator array do not change will result in updates to the `data$` observable in the factory.
(scope, data$, participantId, userId) => {
logger.debug(
`Updating data$ for participantId: ${participantId}, userId: ${userId}`,
);
return new Epoch(items, epoch);
// will only get called once per `participantId, userId` pair.
// updates to data$ and as a result to displayName$ and mxcAvatarUrl$ are more frequent.
return {
participantId,
userId,
...scope.splitBehavior(data$),
};
},
),
),
// new Epoch([]),
);
}
@@ -145,8 +127,8 @@ export function createMatrixLivekitMembers$({
// TODO add this to the JS-SDK
export function areLivekitTransportsEqual(
t1?: LivekitTransport,
t2?: LivekitTransport,
t1: LivekitTransport | null,
t2: LivekitTransport | null,
): boolean {
if (t1 && t2) return t1.livekit_service_url === t2.livekit_service_url;
// In case we have different lk rooms in the same SFU (depends on the livekit authorization service)

View File

@@ -0,0 +1,613 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, vi } from "vitest";
import {
type MatrixEvent,
type RoomMember,
type RoomState,
RoomStateEvent,
} from "matrix-js-sdk";
import EventEmitter from "events";
import { it } from "vitest";
import { ObservableScope } from "../../ObservableScope.ts";
import type { Room as MatrixRoom } from "matrix-js-sdk/lib/models/room";
import {
mockCallMembership,
mockMatrixRoomMember,
withTestScheduler,
} from "../../../utils/test.ts";
import {
createMatrixMemberMetadata$,
createRoomMembers$,
} from "./MatrixMemberMetadata.ts";
let testScope: ObservableScope;
let mockMatrixRoom: MatrixRoom;
describe("MatrixMemberMetadata", () => {
/*
* To be populated in the test setup.
* Maps userId to a partial/mock RoomMember object.
*/
let fakeMembersMap: Map<string, Partial<RoomMember>>;
beforeEach(() => {
testScope = new ObservableScope();
fakeMembersMap = new Map<string, Partial<RoomMember>>();
const roomEmitter = new EventEmitter();
mockMatrixRoom = {
on: roomEmitter.on.bind(roomEmitter),
off: roomEmitter.off.bind(roomEmitter),
emit: roomEmitter.emit.bind(roomEmitter),
// addListener: roomEmitter.addListener.bind(roomEmitter),
// removeListener: roomEmitter.removeListener.bind(roomEmitter),
getMember: vi.fn().mockImplementation((userId: string) => {
const member = fakeMembersMap.get(userId);
if (member) {
return member as RoomMember;
}
return null;
}),
getMembers: vi.fn().mockImplementation(() => {
const members = Array.from(fakeMembersMap.values());
return members;
}),
getMembersWithMembership: vi.fn().mockImplementation(() => {
const members = Array.from(fakeMembersMap.values());
return members;
}),
} as unknown as MatrixRoom;
});
function fakeMemberWith(data: Partial<RoomMember>): void {
const userId = data.userId || "@alice:example.com";
const member: Partial<RoomMember> = {
userId: userId,
rawDisplayName: data.rawDisplayName ?? userId,
getMxcAvatarUrl:
data.getMxcAvatarUrl ||
vi.fn().mockImplementation(() => {
return `mxc://example.com/${userId}`;
}),
...data,
} as unknown as RoomMember;
fakeMembersMap.set(userId, member);
}
afterEach(() => {
fakeMembersMap.clear();
});
describe("displayname", () => {
function updateDisplayName(
userId: `@${string}:${string}`,
newDisplayName: string,
): void {
const member = fakeMembersMap.get(userId);
if (member) {
member.rawDisplayName = newDisplayName;
// Emit the event to notify listeners
mockMatrixRoom.emit(
RoomStateEvent.Members,
{} as unknown as MatrixEvent,
{} as unknown as RoomState,
member as RoomMember,
);
} else {
throw new Error(`No member found with userId: ${userId}`);
}
}
it("should show our own user if present in rtc session and room", () => {
withTestScheduler(({ behavior, expectObservable }) => {
fakeMemberWith({
userId: "@local:example.com",
rawDisplayName: "it's a me",
});
const memberships$ = behavior("a", {
a: [mockCallMembership("@local:example.com", "DEVICE1")],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
const dn$ =
metadataStore.createDisplayNameBehavior$("@local:example.com");
expectObservable(dn$).toBe("a", {
a: "it's a me",
});
expectObservable(metadataStore.displaynameMap$).toBe("a", {
a: new Map<string, string>([["@local:example.com", "it's a me"]]),
});
});
});
function setUpBasicRoom(): void {
fakeMemberWith({
userId: "@local:example.com",
rawDisplayName: "it's a me",
});
fakeMemberWith({ userId: "@alice:example.com", rawDisplayName: "Alice" });
fakeMemberWith({ userId: "@bob:example.com", rawDisplayName: "Bob" });
fakeMemberWith({ userId: "@carl:example.com", rawDisplayName: "Carl" });
fakeMemberWith({ userId: "@evil:example.com", rawDisplayName: "Carl" });
fakeMemberWith({ userId: "@bob:foo.bar", rawDisplayName: "Bob" });
fakeMemberWith({ userId: "@no-name:foo.bar" });
}
it("should get displayName for users", () => {
setUpBasicRoom();
withTestScheduler(({ behavior, expectObservable }) => {
const memberships$ = behavior("a", {
a: [
mockCallMembership("@alice:example.com", "DEVICE1"),
mockCallMembership("@bob:example.com", "DEVICE1"),
],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
const aliceDispName$ =
metadataStore.createDisplayNameBehavior$("@alice:example.com");
expectObservable(aliceDispName$).toBe("a", {
a: "Alice",
});
expectObservable(metadataStore.displaynameMap$).toBe("a", {
a: new Map<string, string>([
["@alice:example.com", "Alice"],
["@bob:example.com", "Bob"],
]),
});
});
});
it("should use userId if no display name", () => {
withTestScheduler(({ behavior, expectObservable }) => {
setUpBasicRoom();
const memberships$ = behavior("a", {
a: [mockCallMembership("@no-name:foo.bar", "D000")],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
expectObservable(metadataStore.displaynameMap$).toBe("a", {
a: new Map<string, string>([
["@no-name:foo.bar", "@no-name:foo.bar"],
]),
});
});
});
it("should disambiguate users with same display name", () => {
withTestScheduler(({ behavior, expectObservable }) => {
setUpBasicRoom();
const memberships$ = behavior("a", {
a: [
mockCallMembership("@bob:example.com", "DEVICE1"),
mockCallMembership("@bob:example.com", "DEVICE2"),
mockCallMembership("@bob:foo.bar", "BOB000"),
mockCallMembership("@carl:example.com", "C000"),
mockCallMembership("@evil:example.com", "E000"),
],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
expectObservable(metadataStore.displaynameMap$).toBe("a", {
a: new Map<string, string>([
// ["@local:example.com", "it's a me"],
["@bob:example.com", "Bob (@bob:example.com)"],
["@bob:example.com", "Bob (@bob:example.com)"],
["@bob:foo.bar", "Bob (@bob:foo.bar)"],
["@carl:example.com", "Carl (@carl:example.com)"],
["@evil:example.com", "Carl (@evil:example.com)"],
]),
});
});
});
it("should start to disambiguate reactivly when needed", () => {
withTestScheduler(({ behavior, expectObservable }) => {
setUpBasicRoom();
const memberships$ = behavior("ab", {
a: [mockCallMembership("@bob:example.com", "DEVICE1")],
b: [
mockCallMembership("@bob:example.com", "DEVICE1"),
mockCallMembership("@bob:foo.bar", "BOB000"),
],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
expectObservable(metadataStore.displaynameMap$).toBe("ab", {
a: new Map<string, string>([["@bob:example.com", "Bob"]]),
b: new Map<string, string>([
["@bob:example.com", "Bob (@bob:example.com)"],
["@bob:foo.bar", "Bob (@bob:foo.bar)"],
]),
});
});
});
it("should keep disambiguated name when other leave", () => {
withTestScheduler(({ behavior, expectObservable }) => {
setUpBasicRoom();
const memberships$ = behavior("ab", {
a: [
mockCallMembership("@bob:example.com", "DEVICE1"),
mockCallMembership("@bob:foo.bar", "BOB000"),
],
b: [mockCallMembership("@bob:example.com", "DEVICE1")],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
expectObservable(metadataStore.displaynameMap$).toBe("ab", {
a: new Map<string, string>([
["@bob:example.com", "Bob (@bob:example.com)"],
["@bob:foo.bar", "Bob (@bob:foo.bar)"],
]),
b: new Map<string, string>([
["@bob:example.com", "Bob (@bob:example.com)"],
]),
});
});
});
it("should disambiguate on name change", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => {
setUpBasicRoom();
const memberships$ = behavior("a", {
a: [
mockCallMembership("@bob:example.com", "B000"),
mockCallMembership("@carl:example.com", "C000"),
],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
schedule("-a", {
a: () => {
updateDisplayName("@carl:example.com", "Bob");
},
});
expectObservable(metadataStore.displaynameMap$).toBe("ab", {
a: new Map<string, string>([
["@bob:example.com", "Bob"],
["@carl:example.com", "Carl"],
]),
b: new Map<string, string>([
["@bob:example.com", "Bob (@bob:example.com)"],
["@carl:example.com", "Bob (@carl:example.com)"],
]),
});
});
});
it("should track individual member id with createDisplayNameBehavior", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => {
setUpBasicRoom();
const BOB = "@bob:example.com";
const CARL = "@carl:example.com";
// for this test we build a mock environment that does all possible changes:
// - memberships join/leave
// - room join/leave
// - disambiguate
const memberships$ = behavior("ab-d", {
a: [mockCallMembership(CARL, "C000")],
b: [
mockCallMembership(CARL, "C000"),
// bob joins
mockCallMembership(BOB, "B000"),
],
// c carl gets renamed to BOB
d: [
// carl leaves
mockCallMembership(BOB, "B000"),
],
});
schedule("--a-", {
a: () => {
// carl renames
updateDisplayName(CARL, "Bob");
},
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
const bob$ = metadataStore.createDisplayNameBehavior$(BOB);
const carl$ = metadataStore.createDisplayNameBehavior$(CARL);
expectObservable(bob$).toBe("abc-", {
a: undefined,
b: "Bob",
c: "Bob (@bob:example.com)",
// bob stays disambiguate even though carl left
// d: "Bob (@bob:example.com)",
});
expectObservable(carl$).toBe("a-cd", {
a: "Carl",
// b: "Carl",
// carl gets renamed and disambiguate
c: "Bob (@carl:example.com)",
d: undefined,
});
});
});
it("should disambiguate users with invisible characters", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const bobRtcMember = mockCallMembership("@bob:example.org", "BBBB");
const bobZeroWidthSpaceRtcMember = mockCallMembership(
"@bob2:example.org",
"BBBB",
);
const bob = mockMatrixRoomMember(bobRtcMember, {
rawDisplayName: "Bob",
});
const bobZeroWidthSpace = mockMatrixRoomMember(
bobZeroWidthSpaceRtcMember,
{
rawDisplayName: "Bo\u200bb",
},
);
fakeMemberWith(bob);
fakeMemberWith(bobZeroWidthSpace);
fakeMemberWith({ userId: "@carol:example.org" });
const memberships$ = behavior("ab", {
a: [mockCallMembership("@carol:example.org", "1111"), bobRtcMember],
b: [
mockCallMembership("@carol:example.org", "1111"),
bobRtcMember,
bobZeroWidthSpaceRtcMember,
],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
const bob$ =
metadataStore.createDisplayNameBehavior$("@bob:example.org");
const bob2$ =
metadataStore.createDisplayNameBehavior$("@bob2:example.org");
const carol$ =
metadataStore.createDisplayNameBehavior$("@carol:example.org");
expectObservable(bob$).toBe("ab", {
a: "Bob",
b: "Bob (@bob:example.org)",
});
expectObservable(bob2$).toBe("ab", {
a: undefined,
b: "Bo\u200bb (@bob2:example.org)",
});
expectObservable(carol$).toBe("a-", {
a: "@carol:example.org",
});
expectObservable(metadataStore.displaynameMap$).toBe("ab", {
// Carol has no displayname - So userId is used.
a: new Map([
["@carol:example.org", "@carol:example.org"],
["@bob:example.org", "Bob"],
]),
// Other Bob joins, and should handle zero width hacks.
b: new Map([
["@carol:example.org", "@carol:example.org"],
[bobRtcMember.userId, `Bob (@bob:example.org)`],
[
bobZeroWidthSpace.userId,
`${bobZeroWidthSpace.rawDisplayName} (${bobZeroWidthSpace.userId})`,
],
]),
});
});
});
it("should strip RTL characters from displayname", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const daveRtcMember = mockCallMembership("@dave:example.org", "DDDD");
const daveRTLRtcMember = mockCallMembership(
"@dave2:example.org",
"DDDD",
);
const dave = mockMatrixRoomMember(daveRtcMember, {
rawDisplayName: "Dave",
});
const daveRTL = mockMatrixRoomMember(daveRTLRtcMember, {
rawDisplayName: "\u202eevaD",
});
fakeMemberWith({ userId: "@carol:example.org" });
fakeMemberWith(daveRTL);
fakeMemberWith(dave);
const memberships$ = behavior("ab", {
a: [mockCallMembership("@carol:example.org", "DDDD")],
b: [
mockCallMembership("@carol:example.org", "DDDD"),
daveRtcMember,
daveRTLRtcMember,
],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
expectObservable(metadataStore.displaynameMap$).toBe("ab", {
// Carol has no displayname - So userId is used.
a: new Map([["@carol:example.org", "@carol:example.org"]]),
// Both Dave's join. Since after stripping
b: new Map([
["@carol:example.org", "@carol:example.org"],
// Not disambiguated
["@dave:example.org", "Dave"],
// This one is, since it's using RTL.
["@dave2:example.org", "evaD (@dave2:example.org)"],
]),
});
});
});
});
describe("avatarUrl", () => {
function updateAvatarUrl(
userId: `@${string}:${string}`,
avatarUrl: string,
): void {
const member = fakeMembersMap.get(userId);
if (member) {
member.getMxcAvatarUrl = vi.fn().mockReturnValue(avatarUrl);
// Emit the event to notify listeners
mockMatrixRoom.emit(
RoomStateEvent.Members,
{} as unknown as MatrixEvent,
{} as unknown as RoomState,
member as RoomMember,
);
} else {
throw new Error(`No member found with userId: ${userId}`);
}
}
it("should use avatar url from room members", () => {
withTestScheduler(({ behavior, expectObservable }) => {
fakeMemberWith({
userId: "@local:example.com",
});
fakeMemberWith({
userId: "@alice:example.com",
getMxcAvatarUrl: vi.fn().mockReturnValue("mxc://custom.url/avatar"),
});
const memberships$ = behavior("a", {
a: [
mockCallMembership("@local:example.com", "DEVICE1"),
mockCallMembership("@alice:example.com", "DEVICE1"),
],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
const local$ =
metadataStore.createAvatarUrlBehavior$("@local:example.com");
const alice$ =
metadataStore.createAvatarUrlBehavior$("@alice:example.com");
expectObservable(local$).toBe("a", {
a: "mxc://example.com/@local:example.com",
});
expectObservable(alice$).toBe("a", {
a: "mxc://custom.url/avatar",
});
expectObservable(metadataStore.avatarMap$).toBe("a", {
a: new Map<string, string>([
["@local:example.com", "mxc://example.com/@local:example.com"],
["@alice:example.com", "mxc://custom.url/avatar"],
]),
});
});
});
it("should update on avatar change and user join/leave", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => {
fakeMemberWith({ userId: "@carl:example.com" });
fakeMemberWith({ userId: "@bob:example.com" });
const memberships$ = behavior("ab-d", {
a: [mockCallMembership("@bob:example.com", "B000")],
b: [
mockCallMembership("@bob:example.com", "B000"),
mockCallMembership("@carl:example.com", "C000"),
],
d: [mockCallMembership("@carl:example.com", "C000")],
});
const metadataStore = createMatrixMemberMetadata$(
testScope,
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
schedule("--c-", {
c: () => {
updateAvatarUrl(
"@carl:example.com",
"mxc://updated.me/updatedAvatar",
);
},
});
const bob$ = metadataStore.createAvatarUrlBehavior$("@bob:example.com");
const carl$ =
metadataStore.createAvatarUrlBehavior$("@carl:example.com");
expectObservable(bob$).toBe("a---", {
a: "mxc://example.com/@bob:example.com",
});
expectObservable(carl$).toBe("a-c-", {
a: "mxc://example.com/@carl:example.com",
c: "mxc://updated.me/updatedAvatar",
});
expectObservable(metadataStore.avatarMap$).toBe("a-c-", {
a: new Map<string, string>([
["@bob:example.com", "mxc://example.com/@bob:example.com"],
["@carl:example.com", "mxc://example.com/@carl:example.com"],
]),
// expect an update once we update the avatar URL
c: new Map<string, string>([
["@bob:example.com", "mxc://example.com/@bob:example.com"],
["@carl:example.com", "mxc://updated.me/updatedAvatar"],
]),
});
});
});
});
});

View File

@@ -0,0 +1,180 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type RoomMember, RoomStateEvent } from "matrix-js-sdk";
import { combineLatest, fromEvent, map } from "rxjs";
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import {
KnownMembership,
type Room as MatrixRoom,
} from "matrix-js-sdk/lib/matrix";
// eslint-disable-next-line rxjs/no-internal
import { type ObservableScope } from "../../ObservableScope";
import {
calculateDisplayName,
shouldDisambiguate,
} from "../../../utils/displayname";
import { type Behavior } from "../../Behavior";
const logger = rootLogger.getChild("[MatrixMemberMetadata]");
export type RoomMemberMap = Map<
string,
Pick<RoomMember, "userId" | "getMxcAvatarUrl" | "rawDisplayName">
>;
export function roomToMembersMap(matrixRoom: MatrixRoom): RoomMemberMap {
const members = matrixRoom
.getMembersWithMembership(KnownMembership.Join)
.concat(matrixRoom.getMembersWithMembership(KnownMembership.Invite));
return members.reduce((acc, member) => {
acc.set(member.userId, {
userId: member.userId,
getMxcAvatarUrl: member.getMxcAvatarUrl.bind(member),
rawDisplayName: member.rawDisplayName,
});
return acc;
}, new Map());
}
export function createRoomMembers$(
scope: ObservableScope,
matrixRoom: MatrixRoom,
): Behavior<RoomMemberMap> {
return scope.behavior(
fromEvent(matrixRoom, RoomStateEvent.Members).pipe(
map(() => roomToMembersMap(matrixRoom)),
),
roomToMembersMap(matrixRoom),
);
}
/**
* creates the member that this DM is with in case it is a DM (two members) otherwise null
*/
export function createDMMember$(
scope: ObservableScope,
roomMembers$: Behavior<RoomMemberMap>,
matrixRoom: MatrixRoom,
): Behavior<Pick<
RoomMember,
"userId" | "getMxcAvatarUrl" | "rawDisplayName"
> | null> {
// We cannot use the normal direct check from matrix since we do not have access to the account data.
// use primitive member count === 2 check instead.
return scope.behavior(
roomMembers$.pipe(
map((membersMap) => {
// primitive appraoch do to no access to account data.
const isDM = membersMap.size === 2;
if (!isDM) return null;
return matrixRoom.getMember(matrixRoom.guessDMUserId());
}),
),
);
}
/**
* Displayname for each member of the call. This will disambiguate
* any displayname that clashes with another member. Only members
* joined to the call are considered here.
*
* @returns Map<userId, displayname> uses the Matrix user ID as the key.
*/
// don't do this work more times than we need to. This is achieved by converting to a behavior:
export const memberDisplaynames$ = (
scope: ObservableScope,
memberships$: Behavior<Pick<CallMembership, "userId">[]>,
roomMembers$: Behavior<RoomMemberMap>,
): Behavior<Map<string, string>> => {
// This map tracks userIds that at some point needed disambiguation.
// This is a memory leak bound to the number of participants.
// A call application will always increase the memory if there have been more members in a call.
// Its capped by room member participants.
const shouldDisambiguateTrackerMap = new Set<string>();
return scope.behavior(
combineLatest([
// Handle call membership changes
memberships$,
// Additionally handle display name changes (implicitly reacting to them)
roomMembers$,
// TODO: do we need: pauseWhen(this.pretendToBeDisconnected$),
]).pipe(
map(([memberships, roomMembers]) => {
const displaynameMap = new Map<string, string>();
// We only consider RTC members for disambiguation as they are the only visible members.
for (const rtcMember of memberships) {
const member = roomMembers.get(rtcMember.userId);
if (!member) {
logger.error(`Could not find member for user ${rtcMember.userId}`);
continue;
}
const disambiguateComputed = shouldDisambiguate(
member,
memberships,
roomMembers,
);
const disambiguate =
shouldDisambiguateTrackerMap.has(rtcMember.userId) ||
disambiguateComputed;
if (disambiguate) shouldDisambiguateTrackerMap.add(rtcMember.userId);
displaynameMap.set(
rtcMember.userId,
calculateDisplayName(member, disambiguate),
);
}
return displaynameMap;
}),
),
);
};
export const createMatrixMemberMetadata$ = (
scope: ObservableScope,
memberships$: Behavior<Pick<CallMembership, "userId">[]>,
roomMembers$: Behavior<RoomMemberMap>,
): {
createDisplayNameBehavior$: (userId: string) => Behavior<string | undefined>;
createAvatarUrlBehavior$: (userId: string) => Behavior<string | undefined>;
displaynameMap$: Behavior<Map<string, string>>;
avatarMap$: Behavior<Map<string, string | undefined>>;
} => {
const displaynameMap$ = memberDisplaynames$(
scope,
memberships$,
roomMembers$,
);
const avatarMap$ = scope.behavior(
roomMembers$.pipe(
map((roomMembers) =>
Array.from(roomMembers.keys()).reduce((acc, key) => {
acc.set(key, roomMembers.get(key)?.getMxcAvatarUrl());
return acc;
}, new Map<string, string | undefined>()),
),
),
);
return {
createDisplayNameBehavior$: (userId: string) =>
scope.behavior(
displaynameMap$.pipe(
map((displaynameMap) => displaynameMap.get(userId)),
),
),
createAvatarUrlBehavior$: (userId: string) =>
scope.behavior(
roomMembers$.pipe(
map((roomMembers) => roomMembers.get(userId)?.getMxcAvatarUrl()),
),
),
// mostly for testing purposes
displaynameMap$,
avatarMap$,
};
};

View File

@@ -1,285 +0,0 @@
/*
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, test, vi } from "vitest";
import {
type MatrixEvent,
type RoomMember,
type RoomState,
RoomStateEvent,
} from "matrix-js-sdk";
import EventEmitter from "events";
import { ObservableScope, trackEpoch } from "../../ObservableScope.ts";
import type { Room as MatrixRoom } from "matrix-js-sdk/lib/models/room";
import { mockCallMembership, withTestScheduler } from "../../../utils/test.ts";
import { memberDisplaynames$ } from "./displayname.ts";
let testScope: ObservableScope;
let mockMatrixRoom: MatrixRoom;
/*
* To be populated in the test setup.
* Maps userId to a partial/mock RoomMember object.
*/
let fakeMembersMap: Map<string, Partial<RoomMember>>;
beforeEach(() => {
testScope = new ObservableScope();
fakeMembersMap = new Map<string, Partial<RoomMember>>();
const roomEmitter = new EventEmitter();
mockMatrixRoom = {
on: roomEmitter.on.bind(roomEmitter),
off: roomEmitter.off.bind(roomEmitter),
emit: roomEmitter.emit.bind(roomEmitter),
// addListener: roomEmitter.addListener.bind(roomEmitter),
// removeListener: roomEmitter.removeListener.bind(roomEmitter),
getMember: vi.fn().mockImplementation((userId: string) => {
const member = fakeMembersMap.get(userId);
if (member) {
return member as RoomMember;
}
return null;
}),
} as unknown as MatrixRoom;
});
function fakeMemberWith(data: Partial<RoomMember>): void {
const userId = data.userId || "@alice:example.com";
const member: Partial<RoomMember> = {
userId: userId,
rawDisplayName: data.rawDisplayName ?? userId,
...data,
} as unknown as RoomMember;
fakeMembersMap.set(userId, member);
// return member as RoomMember;
}
function updateDisplayName(
userId: `@${string}:${string}`,
newDisplayName: string,
): void {
const member = fakeMembersMap.get(userId);
if (member) {
member.rawDisplayName = newDisplayName;
// Emit the event to notify listeners
mockMatrixRoom.emit(
RoomStateEvent.Members,
{} as unknown as MatrixEvent,
{} as unknown as RoomState,
member as RoomMember,
);
} else {
throw new Error(`No member found with userId: ${userId}`);
}
}
afterEach(() => {
fakeMembersMap.clear();
});
test("should always have our own user", () => {
withTestScheduler(({ cold, schedule, expectObservable }) => {
const dn$ = memberDisplaynames$(
testScope,
mockMatrixRoom,
cold("a", {
a: [],
}).pipe(trackEpoch()),
);
expectObservable(dn$).toBe("a", {
a: new Map<string, string>([
["@local:example.com:DEVICE000", "@local:example.com"],
]),
});
});
});
function setUpBasicRoom(): void {
fakeMemberWith({ userId: "@local:example.com", rawDisplayName: "it's a me" });
fakeMemberWith({ userId: "@alice:example.com", rawDisplayName: "Alice" });
fakeMemberWith({ userId: "@bob:example.com", rawDisplayName: "Bob" });
fakeMemberWith({ userId: "@carl:example.com", rawDisplayName: "Carl" });
fakeMemberWith({ userId: "@evil:example.com", rawDisplayName: "Carl" });
fakeMemberWith({ userId: "@bob:foo.bar", rawDisplayName: "Bob" });
fakeMemberWith({ userId: "@no-name:foo.bar" });
}
test("should get displayName for users", () => {
setUpBasicRoom();
withTestScheduler(({ cold, schedule, expectObservable }) => {
const dn$ = memberDisplaynames$(
testScope,
mockMatrixRoom,
cold("a", {
a: [
mockCallMembership("@alice:example.com", "DEVICE1"),
mockCallMembership("@bob:example.com", "DEVICE1"),
],
}).pipe(trackEpoch()),
);
expectObservable(dn$).toBe("a", {
a: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@alice:example.com:DEVICE1", "Alice"],
["@bob:example.com:DEVICE1", "Bob"],
]),
});
});
});
test("should use userId if no display name", () => {
withTestScheduler(({ cold, schedule, expectObservable }) => {
setUpBasicRoom();
const dn$ = memberDisplaynames$(
testScope,
mockMatrixRoom,
cold("a", {
a: [mockCallMembership("@no-name:foo.bar", "D000")],
}).pipe(trackEpoch()),
);
expectObservable(dn$).toBe("a", {
a: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@no-name:foo.bar:D000", "@no-name:foo.bar"],
]),
});
});
});
test("should disambiguate users with same display name", () => {
withTestScheduler(({ cold, schedule, expectObservable }) => {
setUpBasicRoom();
const dn$ = memberDisplaynames$(
testScope,
mockMatrixRoom,
cold("a", {
a: [
mockCallMembership("@bob:example.com", "DEVICE1"),
mockCallMembership("@bob:example.com", "DEVICE2"),
mockCallMembership("@bob:foo.bar", "BOB000"),
mockCallMembership("@carl:example.com", "C000"),
mockCallMembership("@evil:example.com", "E000"),
],
}).pipe(trackEpoch()),
);
expectObservable(dn$).toBe("a", {
a: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@bob:example.com:DEVICE1", "Bob (@bob:example.com)"],
["@bob:example.com:DEVICE2", "Bob (@bob:example.com)"],
["@bob:foo.bar:BOB000", "Bob (@bob:foo.bar)"],
["@carl:example.com:C000", "Carl (@carl:example.com)"],
["@evil:example.com:E000", "Carl (@evil:example.com)"],
]),
});
});
});
test("should disambiguate when needed", () => {
withTestScheduler(({ cold, schedule, expectObservable }) => {
setUpBasicRoom();
const dn$ = memberDisplaynames$(
testScope,
mockMatrixRoom,
cold("ab", {
a: [mockCallMembership("@bob:example.com", "DEVICE1")],
b: [
mockCallMembership("@bob:example.com", "DEVICE1"),
mockCallMembership("@bob:foo.bar", "BOB000"),
],
}).pipe(trackEpoch()),
);
expectObservable(dn$).toBe("ab", {
a: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@bob:example.com:DEVICE1", "Bob"],
]),
b: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@bob:example.com:DEVICE1", "Bob (@bob:example.com)"],
["@bob:foo.bar:BOB000", "Bob (@bob:foo.bar)"],
]),
});
});
});
test.skip("should keep disambiguated name when other leave", () => {
withTestScheduler(({ cold, schedule, expectObservable }) => {
setUpBasicRoom();
const dn$ = memberDisplaynames$(
testScope,
mockMatrixRoom,
cold("ab", {
a: [
mockCallMembership("@bob:example.com", "DEVICE1"),
mockCallMembership("@bob:foo.bar", "BOB000"),
],
b: [mockCallMembership("@bob:example.com", "DEVICE1")],
}).pipe(trackEpoch()),
);
expectObservable(dn$).toBe("ab", {
a: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@bob:example.com:DEVICE1", "Bob (@bob:example.com)"],
["@bob:foo.bar:BOB000", "Bob (@bob:foo.bar)"],
]),
b: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@bob:example.com:DEVICE1", "Bob (@bob:example.com)"],
]),
});
});
});
test("should disambiguate on name change", () => {
withTestScheduler(({ cold, schedule, expectObservable }) => {
setUpBasicRoom();
const dn$ = memberDisplaynames$(
testScope,
mockMatrixRoom,
cold("a", {
a: [
mockCallMembership("@bob:example.com", "B000"),
mockCallMembership("@carl:example.com", "C000"),
],
}).pipe(trackEpoch()),
);
schedule("-a", {
a: () => {
updateDisplayName("@carl:example.com", "Bob");
},
});
expectObservable(dn$).toBe("ab", {
a: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@bob:example.com:B000", "Bob"],
["@carl:example.com:C000", "Carl"],
]),
b: new Map<string, string>([
["@local:example.com:DEVICE000", "it's a me"],
["@bob:example.com:B000", "Bob (@bob:example.com)"],
["@carl:example.com:C000", "Bob (@carl:example.com)"],
]),
});
});
});

View File

@@ -1,99 +0,0 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type RoomMember, RoomStateEvent } from "matrix-js-sdk";
import {
combineLatest,
fromEvent,
map,
type Observable,
startWith,
} from "rxjs";
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { logger } from "matrix-js-sdk/lib/logger";
import { type Room as MatrixRoom } from "matrix-js-sdk/lib/matrix";
// eslint-disable-next-line rxjs/no-internal
import { type NodeStyleEventEmitter } from "rxjs/internal/observable/fromEvent";
import { Epoch, type ObservableScope } from "../../ObservableScope";
import {
calculateDisplayName,
shouldDisambiguate,
} from "../../../utils/displayname";
import { type Behavior } from "../../Behavior";
export function createRoomMembers$(
scope: ObservableScope,
matrixRoom: MatrixRoom,
): Behavior<Pick<RoomMember, "userId" | "getMxcAvatarUrl">[]> {
return scope.behavior(
fromEvent(matrixRoom, RoomStateEvent.Members).pipe(
map(() => matrixRoom.getMembers()),
),
[],
);
}
/**
* Displayname for each member of the call. This will disambiguate
* any displayname that clashes with another member. Only members
* joined to the call are considered here.
*
* @returns Map<member.id, displayname> uses the rtc member idenitfier as the key.
*/
// don't do this work more times than we need to. This is achieved by converting to a behavior:
export const memberDisplaynames$ = (
scope: ObservableScope,
matrixRoom: Pick<MatrixRoom, "getMember"> & NodeStyleEventEmitter,
// roomMember$: Behavior<Pick<RoomMember, "userId" | "getMxcAvatarUrl">>;
memberships$: Observable<Epoch<CallMembership[]>>,
): Behavior<Epoch<Map<string, string>>> =>
scope.behavior(
combineLatest([
// Handle call membership changes
memberships$,
// Additionally handle display name changes (implicitly reacting to them)
fromEvent(matrixRoom, RoomStateEvent.Members).pipe(startWith(null)),
// TODO: do we need: pauseWhen(this.pretendToBeDisconnected$),
]).pipe(
map(([epochMemberships, _displayNames]) => {
const { epoch, value: memberships } = epochMemberships;
const displaynameMap = new Map<string, string>();
const room = matrixRoom;
// We only consider RTC members for disambiguation as they are the only visible members.
for (const rtcMember of memberships) {
// TODO a hard-coded participant ID ? should use rtcMember.membershipID instead?
const matrixIdentifier = `${rtcMember.userId}:${rtcMember.deviceId}`;
const { member } = getRoomMemberFromRtcMember(rtcMember, room);
if (!member) {
logger.error(
"Could not find member for participant id:",
matrixIdentifier,
);
continue;
}
const disambiguate = shouldDisambiguate(member, memberships, room);
displaynameMap.set(
matrixIdentifier,
calculateDisplayName(member, disambiguate),
);
}
return new Epoch(displaynameMap, epoch);
}),
),
new Epoch(new Map<string, string>()),
);
export function getRoomMemberFromRtcMember(
rtcMember: CallMembership,
room: Pick<MatrixRoom, "getMember">,
): { id: string; member: RoomMember | undefined } {
return {
id: rtcMember.userId + ":" + rtcMember.deviceId,
member: room.getMember(rtcMember.userId) ?? undefined,
};
}

View File

@@ -11,7 +11,6 @@ import { type Room as LivekitRoom } from "livekit-client";
import EventEmitter from "events";
import fetchMock from "fetch-mock";
import { type LivekitTransport } from "matrix-js-sdk/lib/matrixrtc";
import { type Room as MatrixRoom, type RoomMember } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import {
@@ -41,7 +40,6 @@ let testScope: ObservableScope;
let ecConnectionFactory: ECConnectionFactory;
let mockClient: OpenIDClientParts;
let lkRoomFactory: () => LivekitRoom;
let mockMatrixRoom: MatrixRoom;
const createdMockLivekitRooms: Map<string, LivekitRoom> = new Map();
@@ -91,18 +89,6 @@ beforeEach(() => {
},
};
});
mockMatrixRoom = vi.mocked<MatrixRoom>({
getMember: vi.fn().mockImplementation((userId: string) => {
return {
userId,
rawDisplayName: userId.replace("@", "").replace(":example.org", ""),
getMxcAvatarUrl: vi.fn().mockReturnValue(null),
} as unknown as RoomMember;
}),
addEventListener: vi.fn(),
removeEventListener: vi.fn(),
} as unknown as MatrixRoom);
});
afterEach(() => {
@@ -135,6 +121,7 @@ test("bob, carl, then bob joining no tracks yet", () => {
scope: testScope,
connectionFactory: ecConnectionFactory,
inputTransports$: membershipsAndTransports.transports$,
logger: logger,
});
const matrixLivekitItems$ = createMatrixLivekitMembers$({
@@ -142,7 +129,6 @@ test("bob, carl, then bob joining no tracks yet", () => {
membershipsWithTransport$:
membershipsAndTransports.membershipsWithTransport$,
connectionManager,
matrixRoom: mockMatrixRoom,
});
expectObservable(matrixLivekitItems$).toBe(vMarble, {
@@ -150,14 +136,20 @@ test("bob, carl, then bob joining no tracks yet", () => {
const items = e.value;
expect(items.length).toBe(1);
const item = items[0]!;
expect(item.membership).toStrictEqual(bobMembership);
expect(
areLivekitTransportsEqual(
item.connection!.transport,
bobMembership.transports[0]! as LivekitTransport,
expectObservable(item.membership$).toBe("a", {
a: bobMembership,
});
expectObservable(item.connection$).toBe("a", {
a: expect.toSatisfy((co) =>
areLivekitTransportsEqual(
co.transport,
bobMembership.transports[0]! as LivekitTransport,
),
),
).toBe(true);
expect(item.participant).toBeUndefined();
});
expectObservable(item.participant$).toBe("a", {
a: null,
});
return true;
}),
b: expect.toSatisfy((e: Epoch<MatrixLivekitMember[]>) => {
@@ -166,51 +158,67 @@ test("bob, carl, then bob joining no tracks yet", () => {
{
const item = items[0]!;
expect(item.membership).toStrictEqual(bobMembership);
expect(item.participant).toBeUndefined();
expectObservable(item.membership$).toBe("a", {
a: bobMembership,
});
expectObservable(item.participant$).toBe("a", {
a: null,
});
}
{
const item = items[1]!;
expect(item.membership).toStrictEqual(carlMembership);
expect(item.participantId).toStrictEqual(
`${carlMembership.userId}:${carlMembership.deviceId}`,
);
expect(
areLivekitTransportsEqual(
item.connection!.transport,
carlMembership.transports[0]! as LivekitTransport,
),
).toBe(true);
expect(item.participant).toBeUndefined();
expectObservable(item.membership$).toBe("a", {
a: carlMembership,
});
expectObservable(item.participant$).toBe("a", {
a: null,
});
expectObservable(item.connection$).toBe("a", {
a: expect.toSatisfy((connection) => {
expect(
areLivekitTransportsEqual(
connection.transport,
carlMembership.transports[0]! as LivekitTransport,
),
).toBe(true);
return true;
}),
});
}
return true;
}),
c: expect.toSatisfy((e: Epoch<MatrixLivekitMember[]>) => {
const items = e.value;
logger.info(`E Items length: ${items.length}`);
expect(items.length).toBe(3);
{
expect(items[0]!.membership).toStrictEqual(bobMembership);
}
{
expect(items[1]!.membership).toStrictEqual(carlMembership);
}
expectObservable(items[0].membership$).toBe("a", {
a: bobMembership,
});
expectObservable(items[1].membership$).toBe("b", {
a: carlMembership,
});
{
const item = items[2]!;
expect(item.membership).toStrictEqual(daveMembership);
expect(item.participantId).toStrictEqual(
`${daveMembership.userId}:${daveMembership.deviceId}`,
);
expect(
areLivekitTransportsEqual(
item.connection!.transport,
daveMembership.transports[0]! as LivekitTransport,
),
).toBe(true);
expect(item.participant).toBeUndefined();
expectObservable(item.membership$).toBe("a", {
a: daveMembership,
});
expectObservable(item.connection$).toBe("a", {
a: expect.toSatisfy((connection) => {
expect(
areLivekitTransportsEqual(
connection.transport,
daveMembership.transports[0]! as LivekitTransport,
),
).toBe(true);
return true;
}),
});
expectObservable(item.participant$).toBe("a", {
a: null,
});
}
return true;
}),

View File

@@ -5,15 +5,18 @@ Copyright 2025 Element Creations Ltd.
Please see LICENSE in the repository root for full details.
*/
import { test, vi, expect } from "vitest";
import { it, vi, expect } from "vitest";
import EventEmitter from "events";
// import * as ComponentsCore from "@livekit/components-core";
import { withCallViewModel } from "./CallViewModel/CallViewModelTestUtils.ts";
import { type CallViewModel } from "./CallViewModel/CallViewModel.ts";
import { constant } from "./Behavior.ts";
import { withCallViewModel } from "./CallViewModel.test.ts";
import { aliceParticipant, localRtcMember } from "../utils/test-fixtures.ts";
import { ElementWidgetActions, widget } from "../widget.ts";
import { E2eeType } from "../e2ee/e2eeType.ts";
import { type CallViewModel } from "./CallViewModel.ts";
vi.mock("@livekit/components-core", { spy: true });
vi.mock("../widget", () => ({
ElementWidgetActions: {
@@ -31,7 +34,7 @@ vi.mock("../widget", () => ({
},
}));
test("expect leave when ElementWidgetActions.HangupCall is called", async () => {
it("expect leave when ElementWidgetActions.HangupCall is called", async () => {
const pr = Promise.withResolvers<string>();
withCallViewModel(
{

View File

@@ -27,7 +27,6 @@ import {
RoomEvent as LivekitRoomEvent,
RemoteTrack,
} from "livekit-client";
import { type RoomMember } from "matrix-js-sdk";
import { logger } from "matrix-js-sdk/lib/logger";
import {
BehaviorSubject,
@@ -44,6 +43,7 @@ import {
startWith,
switchMap,
throttleTime,
distinctUntilChanged,
} from "rxjs";
import { alwaysShowSelf } from "../settings/settings";
@@ -180,29 +180,35 @@ function observeRemoteTrackReceivingOkay$(
}
function encryptionErrorObservable$(
room: LivekitRoom,
room$: Behavior<LivekitRoom | undefined>,
participant: Participant,
encryptionSystem: EncryptionSystem,
criteria: string,
): Observable<boolean> {
return roomEventSelector(room, LivekitRoomEvent.EncryptionError).pipe(
map((e) => {
const [err] = e;
if (encryptionSystem.kind === E2eeType.PER_PARTICIPANT) {
return (
// Ideally we would pull the participant identity from the field on the error.
// However, it gets lost in the serialization process between workers.
// So, instead we do a string match
(err?.message.includes(participant.identity) &&
err?.message.includes(criteria)) ??
false
);
} else if (encryptionSystem.kind === E2eeType.SHARED_KEY) {
return !!err?.message.includes(criteria);
}
return room$.pipe(
switchMap((room) => {
if (room === undefined) return of(false);
return roomEventSelector(room, LivekitRoomEvent.EncryptionError).pipe(
map((e) => {
const [err] = e;
if (encryptionSystem.kind === E2eeType.PER_PARTICIPANT) {
return (
// Ideally we would pull the participant identity from the field on the error.
// However, it gets lost in the serialization process between workers.
// So, instead we do a string match
(err?.message.includes(participant.identity) &&
err?.message.includes(criteria)) ??
false
);
} else if (encryptionSystem.kind === E2eeType.SHARED_KEY) {
return !!err?.message.includes(criteria);
}
return false;
return false;
}),
);
}),
distinctUntilChanged(),
throttleTime(1000), // Throttle to avoid spamming the UI
startWith(false),
);
@@ -220,7 +226,7 @@ abstract class BaseMediaViewModel {
/**
* The LiveKit video track for this media.
*/
public readonly video$: Behavior<TrackReferenceOrPlaceholder | undefined>;
public readonly video$: Behavior<TrackReferenceOrPlaceholder | null>;
/**
* Whether there should be a warning that this media is unencrypted.
*/
@@ -235,12 +241,10 @@ abstract class BaseMediaViewModel {
private observeTrackReference$(
source: Track.Source,
): Behavior<TrackReferenceOrPlaceholder | undefined> {
): Behavior<TrackReferenceOrPlaceholder | null> {
return this.scope.behavior(
this.participant$.pipe(
switchMap((p) =>
p === undefined ? of(undefined) : observeTrackReference$(p, source),
),
switchMap((p) => (!p ? of(null) : observeTrackReference$(p, source))),
),
);
}
@@ -252,23 +256,22 @@ abstract class BaseMediaViewModel {
*/
public readonly id: string,
/**
* The Matrix room member to which this media belongs.
* The Matrix user to which this media belongs.
*/
// TODO: Fully separate the data layer from the UI layer by keeping the
// member object internal
public readonly member: RoomMember,
public readonly userId: string,
// We don't necessarily have a participant if a user connects via MatrixRTC but not (yet) through
// livekit.
protected readonly participant$: Observable<
LocalParticipant | RemoteParticipant | undefined
LocalParticipant | RemoteParticipant | null
>,
encryptionSystem: EncryptionSystem,
audioSource: AudioSource,
videoSource: VideoSource,
livekitRoom: LivekitRoom,
public readonly focusURL: string,
livekitRoom$: Behavior<LivekitRoom | undefined>,
public readonly focusUrl$: Behavior<string | undefined>,
public readonly displayName$: Behavior<string>,
public readonly mxcAvatarUrl$: Behavior<string | undefined>,
) {
const audio$ = this.observeTrackReference$(audioSource);
this.video$ = this.observeTrackReference$(videoSource);
@@ -296,13 +299,13 @@ abstract class BaseMediaViewModel {
} else if (encryptionSystem.kind === E2eeType.PER_PARTICIPANT) {
return combineLatest([
encryptionErrorObservable$(
livekitRoom,
livekitRoom$,
participant,
encryptionSystem,
"MissingKey",
),
encryptionErrorObservable$(
livekitRoom,
livekitRoom$,
participant,
encryptionSystem,
"InvalidKey",
@@ -322,7 +325,7 @@ abstract class BaseMediaViewModel {
} else {
return combineLatest([
encryptionErrorObservable$(
livekitRoom,
livekitRoom$,
participant,
encryptionSystem,
"InvalidKey",
@@ -404,26 +407,28 @@ abstract class BaseUserMediaViewModel extends BaseMediaViewModel {
public constructor(
scope: ObservableScope,
id: string,
member: RoomMember,
participant$: Observable<LocalParticipant | RemoteParticipant | undefined>,
userId: string,
participant$: Observable<LocalParticipant | RemoteParticipant | null>,
encryptionSystem: EncryptionSystem,
livekitRoom: LivekitRoom,
focusUrl: string,
livekitRoom$: Behavior<LivekitRoom | undefined>,
focusUrl$: Behavior<string | undefined>,
displayName$: Behavior<string>,
mxcAvatarUrl$: Behavior<string | undefined>,
public readonly handRaised$: Behavior<Date | null>,
public readonly reaction$: Behavior<ReactionOption | null>,
) {
super(
scope,
id,
member,
userId,
participant$,
encryptionSystem,
Track.Source.Microphone,
Track.Source.Camera,
livekitRoom,
focusUrl,
livekitRoom$,
focusUrl$,
displayName$,
mxcAvatarUrl$,
);
const media$ = this.scope.behavior(
@@ -540,25 +545,27 @@ export class LocalUserMediaViewModel extends BaseUserMediaViewModel {
public constructor(
scope: ObservableScope,
id: string,
member: RoomMember,
participant$: Behavior<LocalParticipant | undefined>,
userId: string,
participant$: Behavior<LocalParticipant | null>,
encryptionSystem: EncryptionSystem,
livekitRoom: LivekitRoom,
focusURL: string,
livekitRoom$: Behavior<LivekitRoom | undefined>,
focusUrl$: Behavior<string | undefined>,
private readonly mediaDevices: MediaDevices,
displayName$: Behavior<string>,
mxcAvatarUrl$: Behavior<string | undefined>,
handRaised$: Behavior<Date | null>,
reaction$: Behavior<ReactionOption | null>,
) {
super(
scope,
id,
member,
userId,
participant$,
encryptionSystem,
livekitRoom,
focusURL,
livekitRoom$,
focusUrl$,
displayName$,
mxcAvatarUrl$,
handRaised$,
reaction$,
);
@@ -650,25 +657,27 @@ export class RemoteUserMediaViewModel extends BaseUserMediaViewModel {
public constructor(
scope: ObservableScope,
id: string,
member: RoomMember,
participant$: Observable<RemoteParticipant | undefined>,
userId: string,
participant$: Observable<RemoteParticipant | null>,
encryptionSystem: EncryptionSystem,
livekitRoom: LivekitRoom,
focusUrl: string,
livekitRoom$: Behavior<LivekitRoom | undefined>,
focusUrl$: Behavior<string | undefined>,
private readonly pretendToBeDisconnected$: Behavior<boolean>,
displayname$: Behavior<string>,
displayName$: Behavior<string>,
mxcAvatarUrl$: Behavior<string | undefined>,
handRaised$: Behavior<Date | null>,
reaction$: Behavior<ReactionOption | null>,
) {
super(
scope,
id,
member,
userId,
participant$,
encryptionSystem,
livekitRoom,
focusUrl,
displayname$,
livekitRoom$,
focusUrl$,
displayName$,
mxcAvatarUrl$,
handRaised$,
reaction$,
);
@@ -749,26 +758,28 @@ export class ScreenShareViewModel extends BaseMediaViewModel {
public constructor(
scope: ObservableScope,
id: string,
member: RoomMember,
userId: string,
participant$: Observable<LocalParticipant | RemoteParticipant>,
encryptionSystem: EncryptionSystem,
livekitRoom: LivekitRoom,
focusUrl: string,
livekitRoom$: Behavior<LivekitRoom | undefined>,
focusUrl$: Behavior<string | undefined>,
private readonly pretendToBeDisconnected$: Behavior<boolean>,
displayname$: Behavior<string>,
displayName$: Behavior<string>,
mxcAvatarUrl$: Behavior<string | undefined>,
public readonly local: boolean,
) {
super(
scope,
id,
member,
userId,
participant$,
encryptionSystem,
Track.Source.ScreenShareAudio,
Track.Source.ScreenShare,
livekitRoom,
focusUrl,
displayname$,
livekitRoom$,
focusUrl$,
displayName$,
mxcAvatarUrl$,
);
}
}

View File

@@ -1,11 +1,13 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it } from "vitest";
import { BehaviorSubject, combineLatest, Subject } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import {
Epoch,
@@ -14,7 +16,6 @@ import {
trackEpoch,
} from "./ObservableScope";
import { withTestScheduler } from "../utils/test";
import { BehaviorSubject, timer } from "rxjs";
describe("Epoch", () => {
it("should map the value correctly", () => {
@@ -59,17 +60,45 @@ describe("Epoch", () => {
});
});
});
it("obs", () => {
const nothing = Symbol("nothing");
const scope = new ObservableScope();
it("diamonds emits in a predictable order", () => {
const sb$ = new BehaviorSubject("initial");
const su$ = new BehaviorSubject(undefined);
expect(sb$.value).toBe("initial");
expect(su$.value).toBe(undefined);
expect(su$.value === nothing).toBe(false);
const root$ = sb$.pipe(trackEpoch());
const derivedA$ = root$.pipe(mapEpoch((e) => e + "-A"));
const derivedB$ = root$.pipe(mapEpoch((e) => e + "-B"));
combineLatest([root$, derivedB$, derivedA$]).subscribe(
([root, derivedA, derivedB]) => {
logger.log(
"combined" +
root.epoch +
root.value +
"\n" +
derivedA.epoch +
derivedA.value +
"\n" +
derivedB.epoch +
derivedB.value,
);
},
);
sb$.next("updated");
sb$.next("ANOTERUPDATE");
});
const a$ = timer(10);
scope.behavior(a$, undefined);
it("behavior test", () => {
const scope = new ObservableScope();
const s$ = new Subject();
const behavior$ = scope.behavior(s$, 0);
behavior$.subscribe((value) => {
logger.log(`Received value: ${value}`);
});
s$.next(1);
s$.next(2);
s$.next(3);
s$.next(3);
s$.next(3);
s$.next(3);
s$.next(3);
s$.complete();
});
});

View File

@@ -24,7 +24,11 @@ import { type Behavior } from "./Behavior";
type MonoTypeOperator = <T>(o: Observable<T>) => Observable<T>;
export const noInitialValue = Symbol("nothing");
type SplitBehavior<T> = keyof T extends string | number
? { [K in keyof T as `${K}$`]: Behavior<T[K]> }
: never;
const nothing = Symbol("nothing");
/**
* A scope which limits the execution lifetime of its bound Observables.
@@ -59,7 +63,10 @@ export class ObservableScope {
* Converts an Observable to a Behavior. If no initial value is specified, the
* Observable must synchronously emit an initial value.
*/
public behavior<T>(setValue$: Observable<T>, initialValue?: T): Behavior<T> {
public behavior<T>(
setValue$: Observable<T>,
initialValue: T | typeof nothing = nothing,
): Behavior<T> {
const subject$ = new BehaviorSubject(initialValue);
// Push values from the Observable into the BehaviorSubject.
// BehaviorSubjects have an undesirable feature where if you call 'complete',
@@ -74,7 +81,7 @@ export class ObservableScope {
subject$.error(err);
},
});
if (subject$.value === noInitialValue)
if (subject$.value === nothing)
throw new Error("Behavior failed to synchronously emit an initial value");
return subject$ as Behavior<T>;
}
@@ -115,27 +122,27 @@ export class ObservableScope {
value$: Behavior<T>,
callback: (value: T) => Promise<(() => Promise<void>) | void>,
): void {
let latestValue: T | typeof noInitialValue = noInitialValue;
let reconciledValue: T | typeof noInitialValue = noInitialValue;
let latestValue: T | typeof nothing = nothing;
let reconciledValue: T | typeof nothing = nothing;
let cleanUp: (() => Promise<void>) | void = undefined;
value$
.pipe(
catchError(() => EMPTY), // Ignore errors
this.bind(), // Limit to the duration of the scope
endWith(noInitialValue), // Clean up when the scope ends
endWith(nothing), // Clean up when the scope ends
)
.subscribe((value) => {
void (async (): Promise<void> => {
if (latestValue === noInitialValue) {
if (latestValue === nothing) {
latestValue = value;
while (latestValue !== reconciledValue) {
await cleanUp?.(); // Call the previous value's clean-up handler
reconciledValue = latestValue;
if (latestValue !== noInitialValue)
if (latestValue !== nothing)
cleanUp = await callback(latestValue); // Sync current value
}
// Reset to signal that reconciliation is done for now
latestValue = noInitialValue;
latestValue = nothing;
} else {
// There's already an instance of the above 'while' loop running
// concurrently. Just update the latest value and let it be handled.
@@ -144,6 +151,24 @@ export class ObservableScope {
})();
});
}
/**
* Splits a Behavior of objects with static properties into an object with
* Behavior properties.
*
* For example, splitting a `Behavior<{ name: string; age: number }>` results
* in an object of type `{ name$: Behavior<string>; age$: Behavior<number> }`.
*/
public splitBehavior<T extends object>(
input$: Behavior<T>,
): SplitBehavior<T> {
return Object.fromEntries(
Object.keys(input$.value).map((key) => [
`${key}$`,
this.behavior(input$.pipe(map((input) => input[key as keyof T]))),
]),
) as SplitBehavior<T>;
}
}
/**

View File

@@ -4,7 +4,7 @@ Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { of, type Observable } from "rxjs";
import { of } from "rxjs";
import {
type LocalParticipant,
type RemoteParticipant,
@@ -13,7 +13,6 @@ import {
import { type ObservableScope } from "./ObservableScope.ts";
import { ScreenShareViewModel } from "./MediaViewModel.ts";
import type { RoomMember } from "matrix-js-sdk";
import type { EncryptionSystem } from "../e2ee/sharedKeyManagement.ts";
import type { Behavior } from "./Behavior.ts";
@@ -28,24 +27,26 @@ export class ScreenShare {
public constructor(
private readonly scope: ObservableScope,
id: string,
member: RoomMember,
userId: string,
participant: LocalParticipant | RemoteParticipant,
encryptionSystem: EncryptionSystem,
livekitRoom: LivekitRoom,
focusUrl: string,
livekitRoom$: Behavior<LivekitRoom | undefined>,
focusUrl$: Behavior<string | undefined>,
pretendToBeDisconnected$: Behavior<boolean>,
displayName$: Observable<string>,
displayName$: Behavior<string>,
mxcAvatarUrl$: Behavior<string | undefined>,
) {
this.vm = new ScreenShareViewModel(
this.scope,
id,
member,
userId,
of(participant),
encryptionSystem,
livekitRoom,
focusUrl,
livekitRoom$,
focusUrl$,
pretendToBeDisconnected$,
this.scope.behavior(displayName$),
displayName$,
mxcAvatarUrl$,
participant.isLocal,
);
}

View File

@@ -1,5 +1,5 @@
/*
Copyright 2025 New Vector Ltd.
Copyright 2025 Element Creations Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -76,6 +76,6 @@ export const createMemberships$ = (
MatrixRTCSessionEvent.MembershipsChanged,
(_, memberships: CallMembership[]) => memberships,
).pipe(trackEpoch()),
new Epoch([]),
new Epoch(matrixRTCSession.memberships),
);
};

View File

@@ -14,7 +14,7 @@ import { fillGaps } from "../utils/iter";
import { debugTileLayout } from "../settings/settings";
function debugEntries(entries: GridTileData[]): string[] {
return entries.map((e) => e.media.member?.rawDisplayName ?? "[👻]");
return entries.map((e) => e.media.displayName$.value);
}
let DEBUG_ENABLED = false;
@@ -156,7 +156,7 @@ export class TileStoreBuilder {
public registerSpotlight(media: MediaViewModel[], maximised: boolean): void {
if (DEBUG_ENABLED)
logger.debug(
`[TileStore, ${this.generation}] register spotlight: ${media.map((m) => m.member?.rawDisplayName ?? "[👻]")}`,
`[TileStore, ${this.generation}] register spotlight: ${media.map((m) => m.displayName$.value)}`,
);
if (this.spotlight !== null) throw new Error("Spotlight already set");
@@ -180,7 +180,7 @@ export class TileStoreBuilder {
public registerGridTile(media: UserMediaViewModel): void {
if (DEBUG_ENABLED)
logger.debug(
`[TileStore, ${this.generation}] register grid tile: ${media.member?.rawDisplayName ?? "[👻]"}`,
`[TileStore, ${this.generation}] register grid tile: ${media.displayName$.value}`,
);
if (this.spotlight !== null) {
@@ -263,7 +263,7 @@ export class TileStoreBuilder {
public registerPipTile(media: UserMediaViewModel): void {
if (DEBUG_ENABLED)
logger.debug(
`[TileStore, ${this.generation}] register PiP tile: ${media.member?.rawDisplayName ?? "[👻]"}`,
`[TileStore, ${this.generation}] register PiP tile: ${media.displayName$.value}`,
);
// If there is a single grid tile that we can reuse

View File

@@ -5,17 +5,9 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
BehaviorSubject,
combineLatest,
map,
type Observable,
of,
switchMap,
} from "rxjs";
import { combineLatest, map, type Observable, of, switchMap } from "rxjs";
import {
type LocalParticipant,
type Participant,
ParticipantEvent,
type RemoteParticipant,
type Room as LivekitRoom,
@@ -29,11 +21,12 @@ import {
type UserMediaViewModel,
} from "./MediaViewModel.ts";
import type { Behavior } from "./Behavior.ts";
import type { RoomMember } from "matrix-js-sdk";
import type { EncryptionSystem } from "../e2ee/sharedKeyManagement.ts";
import type { MediaDevices } from "./MediaDevices.ts";
import type { ReactionOption } from "../reactions";
import { observeSpeaker$ } from "./observeSpeaker.ts";
import { generateItems } from "../utils/observable.ts";
import { ScreenShare } from "./ScreenShare.ts";
/**
* Sorting bins defining the order in which media tiles appear in the layout.
@@ -72,35 +65,35 @@ enum SortingBin {
/**
* A user media item to be presented in a tile. This is a thin wrapper around
* UserMediaViewModel which additionally determines the media item's sorting bin
* for inclusion in the call layout.
* for inclusion in the call layout and tracks associated screen shares.
*/
export class UserMedia {
private readonly participant$ = new BehaviorSubject(this.initialParticipant);
public readonly vm: UserMediaViewModel = this.participant$.value?.isLocal
? new LocalUserMediaViewModel(
this.scope,
this.id,
this.member,
this.participant$ as Behavior<LocalParticipant>,
this.userId,
this.participant$ as Behavior<LocalParticipant | null>,
this.encryptionSystem,
this.livekitRoom,
this.focusURL,
this.livekitRoom$,
this.focusUrl$,
this.mediaDevices,
this.scope.behavior(this.displayname$),
this.displayName$,
this.mxcAvatarUrl$,
this.scope.behavior(this.handRaised$),
this.scope.behavior(this.reaction$),
)
: new RemoteUserMediaViewModel(
this.scope,
this.id,
this.member,
this.participant$ as Observable<RemoteParticipant | undefined>,
this.userId,
this.participant$ as Behavior<RemoteParticipant | null>,
this.encryptionSystem,
this.livekitRoom,
this.focusURL,
this.livekitRoom$,
this.focusUrl$,
this.pretendToBeDisconnected$,
this.scope.behavior(this.displayname$),
this.displayName$,
this.mxcAvatarUrl$,
this.scope.behavior(this.handRaised$),
this.scope.behavior(this.reaction$),
);
@@ -109,12 +102,55 @@ export class UserMedia {
observeSpeaker$(this.vm.speaking$),
);
private readonly presenter$ = this.scope.behavior(
/**
* All screen share media associated with this user media.
*/
public readonly screenShares$ = this.scope.behavior(
this.participant$.pipe(
switchMap((p) => (p === null ? of(false) : sharingScreen$(p))),
switchMap((p) =>
p === null
? of([])
: observeParticipantEvents(
p,
ParticipantEvent.TrackPublished,
ParticipantEvent.TrackUnpublished,
ParticipantEvent.LocalTrackPublished,
ParticipantEvent.LocalTrackUnpublished,
).pipe(
// Technically more than one screen share might be possible... our
// MediaViewModels don't support it though since they look for a unique
// track for the given source. So generateItems here is a bit overkill.
generateItems(
function* (p) {
if (p.isScreenShareEnabled)
yield {
keys: ["screen-share"],
data: undefined,
};
},
(scope, _data$, key) =>
new ScreenShare(
scope,
`${this.id}:${key}`,
this.userId,
p,
this.encryptionSystem,
this.livekitRoom$,
this.focusUrl$,
this.pretendToBeDisconnected$,
this.displayName$,
this.mxcAvatarUrl$,
),
),
),
),
),
);
private readonly presenter$ = this.scope.behavior(
this.screenShares$.pipe(map((screenShares) => screenShares.length > 0)),
);
/**
* Which sorting bin the media item should be placed in.
*/
@@ -147,37 +183,18 @@ export class UserMedia {
public constructor(
private readonly scope: ObservableScope,
public readonly id: string,
private readonly member: RoomMember,
private readonly initialParticipant:
| LocalParticipant
| RemoteParticipant
| null = null,
private readonly userId: string,
private readonly participant$: Behavior<
LocalParticipant | RemoteParticipant | null
>,
private readonly encryptionSystem: EncryptionSystem,
private readonly livekitRoom: LivekitRoom,
private readonly focusURL: string,
private readonly livekitRoom$: Behavior<LivekitRoom | undefined>,
private readonly focusUrl$: Behavior<string | undefined>,
private readonly mediaDevices: MediaDevices,
private readonly pretendToBeDisconnected$: Behavior<boolean>,
private readonly displayname$: Observable<string>,
private readonly displayName$: Behavior<string>,
private readonly mxcAvatarUrl$: Behavior<string | undefined>,
private readonly handRaised$: Observable<Date | null>,
private readonly reaction$: Observable<ReactionOption | null>,
) {}
public updateParticipant(
newParticipant: LocalParticipant | RemoteParticipant | null = null,
): void {
if (this.participant$.value !== newParticipant) {
// Update the BehaviourSubject in the UserMedia.
this.participant$.next(newParticipant);
}
}
}
export function sharingScreen$(p: Participant): Observable<boolean> {
return observeParticipantEvents(
p,
ParticipantEvent.TrackPublished,
ParticipantEvent.TrackUnpublished,
ParticipantEvent.LocalTrackPublished,
ParticipantEvent.LocalTrackUnpublished,
).pipe(map((p) => p.isScreenShareEnabled));
}

View File

@@ -58,7 +58,9 @@ interface TileProps {
style?: ComponentProps<typeof animated.div>["style"];
targetWidth: number;
targetHeight: number;
focusUrl: string | undefined;
displayName: string;
mxcAvatarUrl: string | undefined;
showSpeakingIndicators: boolean;
focusable: boolean;
}
@@ -81,7 +83,9 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
menuStart,
menuEnd,
className,
focusUrl,
displayName,
mxcAvatarUrl,
focusable,
...props
}) => {
@@ -144,8 +148,8 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
const tile = (
<MediaView
ref={ref}
video={video}
member={vm.member}
video={video ?? undefined}
userId={vm.userId}
unencryptedWarning={unencryptedWarning}
encryptionStatus={encryptionStatus}
videoEnabled={videoEnabled}
@@ -164,6 +168,7 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
/>
}
displayName={displayName}
mxcAvatarUrl={mxcAvatarUrl}
focusable={focusable}
primaryButton={
primaryButton ?? (
@@ -190,7 +195,7 @@ const UserMediaTile: FC<UserMediaTileProps> = ({
currentReaction={reaction ?? undefined}
raisedHandOnClick={raisedHandOnClick}
localParticipant={vm.local}
focusUrl={vm.focusURL}
focusUrl={focusUrl}
audioStreamStats={audioStreamStats}
videoStreamStats={videoStreamStats}
{...props}
@@ -359,7 +364,9 @@ export const GridTile: FC<GridTileProps> = ({
const ourRef = useRef<HTMLDivElement | null>(null);
const ref = useMergedRefs(ourRef, theirRef);
const media = useBehavior(vm.media$);
const focusUrl = useBehavior(media.focusUrl$);
const displayName = useBehavior(media.displayName$);
const mxcAvatarUrl = useBehavior(media.mxcAvatarUrl$);
if (media instanceof LocalUserMediaViewModel) {
return (
@@ -367,7 +374,9 @@ export const GridTile: FC<GridTileProps> = ({
ref={ref}
vm={media}
onOpenProfile={onOpenProfile}
focusUrl={focusUrl}
displayName={displayName}
mxcAvatarUrl={mxcAvatarUrl}
{...props}
/>
);
@@ -376,7 +385,9 @@ export const GridTile: FC<GridTileProps> = ({
<RemoteUserMediaTile
ref={ref}
vm={media}
focusUrl={focusUrl}
displayName={displayName}
mxcAvatarUrl={mxcAvatarUrl}
{...props}
/>
);

View File

@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, test, vi } from "vitest";
import { describe, expect, it, test } from "vitest";
import { render, screen } from "@testing-library/react";
import { axe } from "vitest-axe";
import { TooltipProvider } from "@vector-im/compound-web";
@@ -16,7 +16,6 @@ import {
import { LocalTrackPublication, Track } from "livekit-client";
import { TrackInfo } from "@livekit/protocol";
import { type ComponentProps } from "react";
import { type RoomMember } from "matrix-js-sdk";
import { MediaView } from "./MediaView";
import { EncryptionStatus } from "../state/MediaViewModel";
@@ -46,10 +45,8 @@ describe("MediaView", () => {
mirror: false,
unencryptedWarning: false,
video: trackReference,
member: vi.mocked<RoomMember>({
userId: "@alice:example.com",
getMxcAvatarUrl: vi.fn().mockReturnValue(undefined),
} as unknown as RoomMember),
userId: "@alice:example.com",
mxcAvatarUrl: undefined,
localParticipant: false,
focusable: true,
};

View File

@@ -7,7 +7,6 @@ Please see LICENSE in the repository root for full details.
import { type TrackReferenceOrPlaceholder } from "@livekit/components-core";
import { animated } from "@react-spring/web";
import { type RoomMember } from "matrix-js-sdk";
import { type FC, type ComponentProps, type ReactNode } from "react";
import { useTranslation } from "react-i18next";
import classNames from "classnames";
@@ -32,12 +31,13 @@ interface Props extends ComponentProps<typeof animated.div> {
video: TrackReferenceOrPlaceholder | undefined;
videoFit: "cover" | "contain";
mirror: boolean;
member: RoomMember;
userId: string;
videoEnabled: boolean;
unencryptedWarning: boolean;
encryptionStatus: EncryptionStatus;
nameTagLeadingIcon?: ReactNode;
displayName: string;
mxcAvatarUrl: string | undefined;
focusable: boolean;
primaryButton?: ReactNode;
raisedHandTime?: Date;
@@ -59,11 +59,12 @@ export const MediaView: FC<Props> = ({
video,
videoFit,
mirror,
member,
userId,
videoEnabled,
unencryptedWarning,
nameTagLeadingIcon,
displayName,
mxcAvatarUrl,
focusable,
primaryButton,
encryptionStatus,
@@ -94,10 +95,10 @@ export const MediaView: FC<Props> = ({
>
<div className={styles.bg}>
<Avatar
id={member?.userId ?? displayName}
id={userId}
name={displayName}
size={avatarSize}
src={member?.getMxcAvatarUrl()}
src={mxcAvatarUrl}
className={styles.avatar}
style={{ display: video && videoEnabled ? "none" : "initial" }}
/>

View File

@@ -27,7 +27,6 @@ import { useObservableRef } from "observable-hooks";
import { useTranslation } from "react-i18next";
import classNames from "classnames";
import { type TrackReferenceOrPlaceholder } from "@livekit/components-core";
import { type RoomMember } from "matrix-js-sdk";
import FullScreenMaximiseIcon from "../icons/FullScreenMaximise.svg?react";
import FullScreenMinimiseIcon from "../icons/FullScreenMinimise.svg?react";
@@ -55,10 +54,12 @@ interface SpotlightItemBaseProps {
targetHeight: number;
video: TrackReferenceOrPlaceholder | undefined;
videoEnabled: boolean;
member: RoomMember;
userId: string;
unencryptedWarning: boolean;
encryptionStatus: EncryptionStatus;
focusUrl: string | undefined;
displayName: string;
mxcAvatarUrl: string | undefined;
focusable: boolean;
"aria-hidden"?: boolean;
localParticipant: boolean;
@@ -78,7 +79,7 @@ const SpotlightLocalUserMediaItem: FC<SpotlightLocalUserMediaItemProps> = ({
...props
}) => {
const mirror = useBehavior(vm.mirror$);
return <MediaView mirror={mirror} focusUrl={vm.focusURL} {...props} />;
return <MediaView mirror={mirror} {...props} />;
};
SpotlightLocalUserMediaItem.displayName = "SpotlightLocalUserMediaItem";
@@ -134,7 +135,9 @@ const SpotlightItem: FC<SpotlightItemProps> = ({
}) => {
const ourRef = useRef<HTMLDivElement | null>(null);
const ref = useMergedRefs(ourRef, theirRef);
const focusUrl = useBehavior(vm.focusUrl$);
const displayName = useBehavior(vm.displayName$);
const mxcAvatarUrl = useBehavior(vm.mxcAvatarUrl$);
const video = useBehavior(vm.video$);
const videoEnabled = useBehavior(vm.videoEnabled$);
const unencryptedWarning = useBehavior(vm.unencryptedWarning$);
@@ -161,11 +164,13 @@ const SpotlightItem: FC<SpotlightItemProps> = ({
className: classNames(styles.item, { [styles.snap]: snap }),
targetWidth,
targetHeight,
video,
video: video ?? undefined,
videoEnabled,
member: vm.member,
userId: vm.userId,
unencryptedWarning,
focusUrl,
displayName,
mxcAvatarUrl,
focusable,
encryptionStatus,
"aria-hidden": ariaHidden,

View File

@@ -6,10 +6,10 @@ Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeAll, describe, expect, test, vi } from "vitest";
import { type RoomMember } from "matrix-js-sdk";
import { shouldDisambiguate } from "./displayname";
import { alice } from "./test-fixtures";
import { mockMatrixRoom } from "./test";
// Ideally these tests would be in ./displayname.test.ts but I can't figure out how to
// just spy on the removeHiddenChars() function without impacting the other tests.
@@ -29,7 +29,7 @@ describe("shouldDisambiguate", () => {
});
test("should only call removeHiddenChars once for a single displayname", () => {
const room = mockMatrixRoom({});
const room: Map<string, Pick<RoomMember, "userId">> = new Map([]);
shouldDisambiguate(alice, [], room);
expect(jsUtils.removeHiddenChars).toHaveBeenCalledTimes(1);
for (let i = 0; i < 10; i++) {

View File

@@ -20,62 +20,70 @@ import {
daveRTL,
} from "./test-fixtures";
import { mockMatrixRoom } from "./test";
import { roomToMembersMap } from "../state/CallViewModel/remoteMembers/MatrixMemberMetadata";
describe("shouldDisambiguate", () => {
test("should not disambiguate a solo member", () => {
const room = mockMatrixRoom({});
expect(shouldDisambiguate(alice, [], room)).toEqual(false);
const room = mockMatrixRoom({
getMembersWithMembership: () => [],
});
expect(shouldDisambiguate(alice, [], roomToMembersMap(room))).toEqual(
false,
);
});
test("should not disambiguate a member with an empty displayname", () => {
const room = mockMatrixRoom({
getMember: (u) =>
[alice, aliceDoppelganger].find((m) => m.userId === u) ?? null,
getMembersWithMembership: () => [alice, aliceDoppelganger],
});
expect(
shouldDisambiguate(
{ rawDisplayName: "", userId: alice.userId },
[aliceRtcMember, aliceDoppelgangerRtcMember],
room,
roomToMembersMap(room),
),
).toEqual(false);
});
test("should disambiguate a member with RTL characters", () => {
const room = mockMatrixRoom({});
expect(shouldDisambiguate(daveRTL, [], room)).toEqual(true);
const room = mockMatrixRoom({ getMembersWithMembership: () => [] });
expect(shouldDisambiguate(daveRTL, [], roomToMembersMap(room))).toEqual(
true,
);
});
test("should disambiguate a member with a matching displayname", () => {
const room = mockMatrixRoom({
getMember: (u) =>
[alice, aliceDoppelganger].find((m) => m.userId === u) ?? null,
getMembersWithMembership: () => [alice, aliceDoppelganger],
});
expect(
shouldDisambiguate(
alice,
[aliceRtcMember, aliceDoppelgangerRtcMember],
room,
roomToMembersMap(room),
),
).toEqual(true);
expect(
shouldDisambiguate(
aliceDoppelganger,
[aliceRtcMember, aliceDoppelgangerRtcMember],
room,
roomToMembersMap(room),
),
).toEqual(true);
});
test("should disambiguate a member with a matching displayname with hidden spaces", () => {
const room = mockMatrixRoom({
getMember: (u) =>
[bob, bobZeroWidthSpace].find((m) => m.userId === u) ?? null,
getMembersWithMembership: () => [bob, bobZeroWidthSpace],
});
expect(
shouldDisambiguate(bob, [bobRtcMember, bobZeroWidthSpaceRtcMember], room),
shouldDisambiguate(
bob,
[bobRtcMember, bobZeroWidthSpaceRtcMember],
roomToMembersMap(room),
),
).toEqual(true);
expect(
shouldDisambiguate(
bobZeroWidthSpace,
[bobRtcMember, bobZeroWidthSpaceRtcMember],
room,
roomToMembersMap(room),
),
).toEqual(true);
});
@@ -83,11 +91,14 @@ describe("shouldDisambiguate", () => {
"should disambiguate a member with a displayname containing a mxid-like string '%s'",
(rawDisplayName) => {
const room = mockMatrixRoom({
getMember: (u) =>
[alice, aliceDoppelganger].find((m) => m.userId === u) ?? null,
getMembersWithMembership: () => [alice, aliceDoppelganger],
});
expect(
shouldDisambiguate({ rawDisplayName, userId: alice.userId }, [], room),
shouldDisambiguate(
{ rawDisplayName, userId: alice.userId },
[],
roomToMembersMap(room),
),
).toEqual(true);
},
);

View File

@@ -10,7 +10,7 @@ import {
removeHiddenChars as removeHiddenCharsUncached,
} from "matrix-js-sdk/lib/utils";
import type { Room } from "matrix-js-sdk";
import type { RoomMember } from "matrix-js-sdk";
import type { CallMembership } from "matrix-js-sdk/lib/matrixrtc";
// Calling removeHiddenChars() can be slow on Safari, so we cache the results.
@@ -40,8 +40,8 @@ function removeHiddenChars(str: string): string {
// Borrowed from https://github.com/matrix-org/matrix-js-sdk/blob/f10deb5ef2e8f061ff005af0476034382ea128ca/src/models/room-member.ts#L409
export function shouldDisambiguate(
member: { rawDisplayName?: string; userId: string },
memberships: CallMembership[],
room: Pick<Room, "getMember">,
memberships: Pick<CallMembership, "userId">[],
roomMembers: Map<string, Pick<RoomMember, "userId">>,
): boolean {
const { rawDisplayName: displayName, userId } = member;
if (!displayName || displayName === userId) return false;
@@ -65,7 +65,7 @@ export function shouldDisambiguate(
// displayname, after hidden character removal.
return (
memberships
.map((m) => m.userId && room.getMember(m.userId))
.map((m) => m.userId && roomMembers.get(m.userId))
// NOTE: We *should* have a room member for everyone.
.filter((m) => !!m)
.filter((m) => m.userId !== userId)

View File

@@ -9,7 +9,7 @@ import { test } from "vitest";
import { Subject } from "rxjs";
import { withTestScheduler } from "./test";
import { generateKeyed$, pauseWhen } from "./observable";
import { generateItems, pauseWhen } from "./observable";
test("pauseWhen", () => {
withTestScheduler(({ behavior, expectObservable }) => {
@@ -24,7 +24,7 @@ test("pauseWhen", () => {
});
});
test("generateKeyed$ has the right output and ends scopes at the right times", () => {
test("generateItems", () => {
const scope1$ = new Subject<string>();
const scope2$ = new Subject<string>();
const scope3$ = new Subject<string>();
@@ -44,18 +44,27 @@ test("generateKeyed$ has the right output and ends scopes at the right times", (
const scope4Marbles = " ----yn";
expectObservable(
generateKeyed$(hot<string>(inputMarbles), (input, createOrGet) => {
for (let i = 1; i <= +input; i++) {
createOrGet(i.toString(), (scope) => {
hot<string>(inputMarbles).pipe(
generateItems(
function* (input) {
for (let i = 1; i <= +input; i++) {
yield { keys: [i], data: undefined };
}
},
(scope, data$, i) => {
scopeSubjects[i - 1].next("y");
scope.onEnd(() => scopeSubjects[i - 1].next("n"));
return i.toString();
});
}
return "abcd"[+input - 1];
}),
},
),
),
subscriptionMarbles,
).toBe(outputMarbles);
).toBe(outputMarbles, {
a: ["1"],
b: ["1", "2"],
c: ["1", "2", "3"],
d: ["1", "2", "3", "4"],
});
expectObservable(scope1$).toBe(scope1Marbles);
expectObservable(scope2$).toBe(scope2Marbles);

View File

@@ -20,10 +20,12 @@ import {
takeWhile,
tap,
withLatestFrom,
BehaviorSubject,
type OperatorFunction,
} from "rxjs";
import { type Behavior } from "../state/Behavior";
import { ObservableScope } from "../state/ObservableScope";
import { Epoch, ObservableScope } from "../state/ObservableScope";
const nothing = Symbol("nothing");
@@ -119,70 +121,156 @@ export function pauseWhen<T>(pause$: Behavior<boolean>) {
);
}
interface ItemHandle<Data, Item> {
scope: ObservableScope;
data$: BehaviorSubject<Data>;
item: Item;
}
/**
* Maps a changing input value to an output value consisting of items that have
* automatically generated ObservableScopes tied to a key. Items will be
* automatically created when their key is requested for the first time, reused
* when the same key is requested at a later time, and destroyed (have their
* scope ended) when the key is no longer requested.
* Maps a changing input value to a collection of items that each capture some
* dynamic data and are tied to a key. Items will be automatically created when
* their key is requested for the first time, reused when the same key is
* requested at a later time, and destroyed (have their scope ended) when the
* key is no longer requested.
*
* @param input$ The input value to be mapped.
* @param project A function mapping input values to output values. This
* function receives an additional callback `createOrGet` which can be used
* within the function body to request that an item be generated for a certain
* key. The caller provides a factory which will be used to create the item if
* it is being requested for the first time. Otherwise, the item previously
* existing under that key will be returned.
* @param generator A generator function yielding a tuple of keys and the
* currently associated data for each item that it wants to exist.
* @param factory A function constructing an individual item, given the item's key,
* dynamic data, and an automatically managed ObservableScope for the item.
*/
export function generateKeyed$<In, Item, Out>(
input$: Observable<In>,
project: (
input: In,
createOrGet: (
key: string,
factory: (scope: ObservableScope) => Item,
) => Item,
) => Out,
): Observable<Out> {
return input$.pipe(
// Keep track of the existing items over time, so we can reuse them
scan<
In,
{
items: Map<string, { item: Item; scope: ObservableScope }>;
output: Out;
},
{ items: Map<string, { item: Item; scope: ObservableScope }> }
>(
(state, data) => {
const nextItems = new Map<
string,
{ item: Item; scope: ObservableScope }
>();
export function generateItems<
Input,
Keys extends [unknown, ...unknown[]],
Data,
Item,
>(
generator: (
input: Input,
) => Generator<{ keys: readonly [...Keys]; data: Data }, void, void>,
factory: (
scope: ObservableScope,
data$: Behavior<Data>,
...keys: Keys
) => Item,
): OperatorFunction<Input, Item[]> {
return generateItemsInternal(generator, factory, (items) => items);
}
const output = project(data, (key, factory) => {
let item = state.items.get(key);
if (item === undefined) {
// First time requesting the key; create the item
const scope = new ObservableScope();
item = { item: factory(scope), scope };
}
nextItems.set(key, item);
return item.item;
});
// Destroy all items that are no longer being requested
for (const [key, { scope }] of state.items)
if (!nextItems.has(key)) scope.end();
return { items: nextItems, output };
},
{ items: new Map() },
),
finalizeValue((state) => {
// Destroy all remaining items when no longer subscribed
for (const { scope } of state.items.values()) scope.end();
}),
map(({ output }) => output),
/**
* Same as generateItems, but preserves epoch data.
*/
export function generateItemsWithEpoch<
Input,
Keys extends [unknown, ...unknown[]],
Data,
Item,
>(
generator: (
input: Input,
) => Generator<{ keys: readonly [...Keys]; data: Data }, void, void>,
factory: (
scope: ObservableScope,
data$: Behavior<Data>,
...keys: Keys
) => Item,
): OperatorFunction<Epoch<Input>, Epoch<Item[]>> {
return generateItemsInternal(
function* (input) {
yield* generator(input.value);
},
factory,
(items, input) => new Epoch(items, input.epoch),
);
}
function generateItemsInternal<
Input,
Keys extends [unknown, ...unknown[]],
Data,
Item,
Output,
>(
generator: (
input: Input,
) => Generator<{ keys: readonly [...Keys]; data: Data }, void, void>,
factory: (
scope: ObservableScope,
data$: Behavior<Data>,
...keys: Keys
) => Item,
project: (items: Item[], input: Input) => Output,
): OperatorFunction<Input, Output> {
/* eslint-disable @typescript-eslint/no-explicit-any */
return (input$) =>
input$.pipe(
// Keep track of the existing items over time, so they can persist
scan<
Input,
{
map: Map<any, any>;
items: Set<ItemHandle<Data, Item>>;
input: Input;
},
{ map: Map<any, any>; items: Set<ItemHandle<Data, Item>> }
>(
({ map: prevMap, items: prevItems }, input) => {
const nextMap = new Map();
const nextItems = new Set<ItemHandle<Data, Item>>();
for (const { keys, data } of generator(input)) {
// Disable type checks for a second to grab the item out of a nested map
let i: any = prevMap;
for (const key of keys) i = i?.get(key);
let item = i as ItemHandle<Data, Item> | undefined;
if (item === undefined) {
// First time requesting the key; create the item
const scope = new ObservableScope();
const data$ = new BehaviorSubject(data);
item = { scope, data$, item: factory(scope, data$, ...keys) };
} else {
item.data$.next(data);
}
// Likewise, disable type checks to insert the item in the nested map
let m: Map<any, any> = nextMap;
for (let i = 0; i < keys.length - 1; i++) {
let inner = m.get(keys[i]);
if (inner === undefined) {
inner = new Map();
m.set(keys[i], inner);
}
m = inner;
}
const finalKey = keys[keys.length - 1];
if (m.has(finalKey))
throw new Error(
`Keys must be unique (tried to generate multiple items for key ${keys})`,
);
m.set(keys[keys.length - 1], item);
nextItems.add(item);
}
// Destroy all items that are no longer being requested
for (const item of prevItems)
if (!nextItems.has(item)) item.scope.end();
return { map: nextMap, items: nextItems, input };
},
{ map: new Map(), items: new Set() },
),
finalizeValue(({ items }) => {
// Destroy all remaining items when no longer subscribed
for (const { scope } of items) scope.end();
}),
map(({ items, input }) =>
project(
[...items].map(({ item }) => item),
input,
),
),
);
/* eslint-enable @typescript-eslint/no-explicit-any */
}

View File

@@ -11,9 +11,9 @@ import {
mockRemoteParticipant,
} from "./test";
export const localRtcMember = mockRtcMembership("@carol:example.org", "1111");
export const localRtcMember = mockRtcMembership("@local:example.org", "1111");
export const localRtcMemberDevice2 = mockRtcMembership(
"@carol:example.org",
"@local:example.org",
"2222",
);
export const local = mockMatrixRoomMember(localRtcMember);
@@ -37,7 +37,6 @@ export const aliceDoppelganger = mockMatrixRoomMember(
rawDisplayName: "Alice",
},
);
export const aliceDoppelgangerId = `${aliceDoppelganger.userId}:${aliceDoppelgangerRtcMember.deviceId}`;
export const bobRtcMember = mockRtcMembership("@bob:example.org", "BBBB");
export const bob = mockMatrixRoomMember(bobRtcMember, {
@@ -55,10 +54,8 @@ export const bobZeroWidthSpace = mockMatrixRoomMember(
rawDisplayName: "Bo\u200bb",
},
);
export const bobZeroWidthSpaceId = `${bobZeroWidthSpace.userId}:${bobZeroWidthSpaceRtcMember.deviceId}`;
export const daveRTLRtcMember = mockRtcMembership("@dave2:example.org", "DDDD");
export const daveRTL = mockMatrixRoomMember(daveRTLRtcMember, {
rawDisplayName: "\u202eevaD",
});
export const daveRTLId = `${daveRTL.userId}:${daveRTLRtcMember.deviceId}`;

View File

@@ -6,7 +6,7 @@ Please see LICENSE in the repository root for full details.
*/
import { type CallMembership } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, of } from "rxjs";
import { BehaviorSubject } from "rxjs";
import { vitest } from "vitest";
import { type RelationsContainer } from "matrix-js-sdk/lib/models/relations-container";
import EventEmitter from "events";
@@ -20,10 +20,12 @@ import { ConnectionState, type Room as LivekitRoom } from "livekit-client";
import { E2eeType } from "../e2ee/e2eeType";
import {
CallViewModel,
type CallViewModel,
createCallViewModel$,
type CallViewModelOptions,
} from "../state/CallViewModel/CallViewModel";
import {
mockConfig,
mockLivekitRoom,
mockLocalParticipant,
mockMatrixRoom,
@@ -36,6 +38,8 @@ import { aliceRtcMember, localRtcMember } from "./test-fixtures";
import { type RaisedHandInfo, type ReactionInfo } from "../reactions";
import { constant } from "../state/Behavior";
mockConfig({ livekit: { livekit_service_url: "https://example.com" } });
export function getBasicRTCSession(
members: RoomMember[],
initialRtcMemberships: CallMembership[] = [localRtcMember, aliceRtcMember],
@@ -57,6 +61,7 @@ export function getBasicRTCSession(
getUserId: () => localRtcMember.userId,
getDeviceId: () => localRtcMember.deviceId,
getSyncState: () => SyncState.Syncing,
getDomain: () => null,
sendEvent: vitest.fn().mockResolvedValue({ event_id: "$fake:event" }),
redactEvent: vitest.fn().mockResolvedValue({ event_id: "$fake:event" }),
decryptEventIfNeeded: vitest.fn().mockResolvedValue(undefined),
@@ -78,6 +83,9 @@ export function getBasicRTCSession(
),
} as Partial<MatrixClient> as MatrixClient,
getMember: (userId) => matrixRoomMembers.get(userId) ?? null,
getMembers: () => Array.from(matrixRoomMembers.values()),
getMembersWithMembership: () => Array.from(matrixRoomMembers.values()),
guessDMUserId: vitest.fn(),
roomId: matrixRoomId,
on: vitest
.fn()
@@ -138,7 +146,7 @@ export function getBasicCallViewModelEnvironment(
// const remoteParticipants$ = of([aliceParticipant]);
const vm = new CallViewModel(
const vm = createCallViewModel$(
testScope(),
rtcSession.asMockedSession(),
matrixRoom,
@@ -158,7 +166,7 @@ export function getBasicCallViewModelEnvironment(
},
handRaisedSubject$,
reactionsSubject$,
of({ processor: undefined, supported: false }),
constant({ processor: undefined, supported: false }),
);
return {
vm,

View File

@@ -6,7 +6,14 @@ Please see LICENSE in the repository root for full details.
*/
import { map, type Observable, of, type SchedulerLike } from "rxjs";
import { type RunHelpers, TestScheduler } from "rxjs/testing";
import { expect, type MockedObject, onTestFinished, vi, vitest } from "vitest";
import {
expect,
type MockedObject,
type MockInstance,
onTestFinished,
vi,
vitest,
} from "vitest";
import {
MatrixEvent,
type Room as MatrixRoom,
@@ -72,6 +79,13 @@ export async function flushPromises(): Promise<void> {
await new Promise<void>((resolve) => window.setTimeout(resolve));
}
export type NodeEventHandler = (...args: unknown[]) => void;
export interface NodeStyleEventEmitter {
addListener(eventName: string | symbol, handler: NodeEventHandler): this;
removeListener(eventName: string | symbol, handler: NodeEventHandler): this;
}
export interface OurRunHelpers extends RunHelpers {
/**
* Schedules a sequence of actions to happen, as described by a marble
@@ -269,6 +283,7 @@ export function mockLivekitRoom(
}: { remoteParticipants$?: Observable<RemoteParticipant[]> } = {},
): LivekitRoom {
const livekitRoom = {
options: {},
...mockEmitter(),
...room,
} as Partial<LivekitRoom> as LivekitRoom;
@@ -291,6 +306,7 @@ export function mockLocalParticipant(
return {
isLocal: true,
trackPublications: new Map(),
unpublishTracks: async () => Promise.resolve(),
getTrackPublication: () =>
({}) as Partial<LocalTrackPublication> as LocalTrackPublication,
...mockEmitter(),
@@ -304,18 +320,20 @@ export function createLocalMedia(
localParticipant: LocalParticipant,
mediaDevices: MediaDevices,
): LocalUserMediaViewModel {
const member = mockMatrixRoomMember(localRtcMember, roomMember);
return new LocalUserMediaViewModel(
testScope(),
"local",
mockMatrixRoomMember(localRtcMember, roomMember),
member.userId,
constant(localParticipant),
{
kind: E2eeType.PER_PARTICIPANT,
},
mockLivekitRoom({ localParticipant }),
"https://rtc-example.org",
constant(mockLivekitRoom({ localParticipant })),
constant("https://rtc-example.org"),
mediaDevices,
constant(roomMember.rawDisplayName ?? "nodisplayname"),
constant(member.rawDisplayName ?? "nodisplayname"),
constant(member.getMxcAvatarUrl()),
constant(null),
constant(null),
);
@@ -329,6 +347,8 @@ export function mockRemoteParticipant(
setVolume() {},
getTrackPublication: () =>
({}) as Partial<RemoteTrackPublication> as RemoteTrackPublication,
// this will only get used for `getTrackPublications().length`
getTrackPublications: () => [0],
...mockEmitter(),
...participant,
} as RemoteParticipant;
@@ -339,31 +359,38 @@ export function createRemoteMedia(
roomMember: Partial<RoomMember>,
participant: Partial<RemoteParticipant>,
): RemoteUserMediaViewModel {
const member = mockMatrixRoomMember(localRtcMember, roomMember);
const remoteParticipant = mockRemoteParticipant(participant);
return new RemoteUserMediaViewModel(
testScope(),
"remote",
mockMatrixRoomMember(localRtcMember, roomMember),
member.userId,
of(remoteParticipant),
{
kind: E2eeType.PER_PARTICIPANT,
},
mockLivekitRoom({}, { remoteParticipants$: of([remoteParticipant]) }),
"https://rtc-example.org",
constant(
mockLivekitRoom({}, { remoteParticipants$: of([remoteParticipant]) }),
),
constant("https://rtc-example.org"),
constant(false),
constant(roomMember.rawDisplayName ?? "nodisplayname"),
constant(member.rawDisplayName ?? "nodisplayname"),
constant(member.getMxcAvatarUrl()),
constant(null),
constant(null),
);
}
export function mockConfig(config: Partial<ResolvedConfigOptions> = {}): void {
vi.spyOn(Config, "get").mockReturnValue({
export function mockConfig(
config: Partial<ResolvedConfigOptions> = {},
): MockInstance<() => ResolvedConfigOptions> {
const spy = vi.spyOn(Config, "get").mockReturnValue({
...DEFAULT_CONFIG,
...config,
});
// simulate loading the config
vi.spyOn(Config, "init").mockResolvedValue(void 0);
return spy;
}
export class MockRTCSession extends TypedEventEmitter<

View File

@@ -50,5 +50,6 @@
"plugins": [{ "name": "typescript-eslint-language-service" }]
},
"include": ["./src/**/*.ts", "./src/**/*.tsx", "./playwright/**/*.ts"]
"include": ["./src/**/*.ts", "./src/**/*.tsx", "./playwright/**/*.ts"],
"exclude": ["**.test.ts"]
}

View File

@@ -2731,37 +2731,39 @@ __metadata:
languageName: node
linkType: hard
"@livekit/components-core@npm:0.12.10, @livekit/components-core@npm:^0.12.0":
version: 0.12.10
resolution: "@livekit/components-core@npm:0.12.10"
"@livekit/components-core@npm:0.12.11, @livekit/components-core@npm:^0.12.0":
version: 0.12.11
resolution: "@livekit/components-core@npm:0.12.11"
dependencies:
"@floating-ui/dom": "npm:1.6.13"
loglevel: "npm:1.9.1"
rxjs: "npm:7.8.2"
peerDependencies:
livekit-client: ^2.13.3
livekit-client: ^2.15.14
tslib: ^2.6.2
checksum: 10c0/bfd84fb950f72dd037bd5329658c1e750a8ac6b8f2953ea673e3e944b8ea9d412ef9c98eb8b690052323e03c675964b162aacb00e60530cdc5187f77d21979bd
checksum: 10c0/9c2ac3d30bb8cc9067ae0b2049784f81e90e57df9eabf7edbaf3c8ceb65a63f644a4e6abeb6cc38d3ebe52663d8dbb88535e01a965011f365d5ae1f3daf86052
languageName: node
linkType: hard
"@livekit/components-react@npm:^2.0.0":
version: 2.9.15
resolution: "@livekit/components-react@npm:2.9.15"
version: 2.9.16
resolution: "@livekit/components-react@npm:2.9.16"
dependencies:
"@livekit/components-core": "npm:0.12.10"
"@livekit/components-core": "npm:0.12.11"
clsx: "npm:2.1.1"
events: "npm:^3.3.0"
jose: "npm:^6.0.12"
usehooks-ts: "npm:3.1.1"
peerDependencies:
"@livekit/krisp-noise-filter": ^0.2.12 || ^0.3.0
livekit-client: ^2.13.3
livekit-client: ^2.15.14
react: ">=18"
react-dom: ">=18"
tslib: ^2.6.2
peerDependenciesMeta:
"@livekit/krisp-noise-filter":
optional: true
checksum: 10c0/58a93d85c3b8267d0afd00eceb4f34992ce66124f93450e828a78dd825ecc20e254c3123ed22ec33061a3728f50f4f020ff896769fb3a0ff79656ed8cf452a2b
checksum: 10c0/4ba4ff473c5a29d3107412733a6676a3b708d70684ed463e9b34cda26abb3d2f317c2828a52e730837b756de9df3fc248260d6f390aedebfb6ec96ef63c7b151
languageName: node
linkType: hard
@@ -5247,12 +5249,12 @@ __metadata:
languageName: node
linkType: hard
"@types/node@npm:^22.0.0":
version: 22.17.0
resolution: "@types/node@npm:22.17.0"
"@types/node@npm:^24.0.0":
version: 24.10.0
resolution: "@types/node@npm:24.10.0"
dependencies:
undici-types: "npm:~6.21.0"
checksum: 10c0/e1c603b660d3de3243dfc02ded5d40623ff3f36315ffbdd8cdc81bc2c5a8da172035879d437b72e9fa61ca01827f28e9c2b0c32898f411a8e9ba0a5efac0b4ca
undici-types: "npm:~7.16.0"
checksum: 10c0/f82ed7194e16f5590ef7afdc20c6d09068c76d50278b485ede8f0c5749683536e3064ffa8def8db76915196afb3724b854aa5723c64d6571b890b14492943b46
languageName: node
linkType: hard
@@ -7507,7 +7509,7 @@ __metadata:
"@types/grecaptcha": "npm:^3.0.9"
"@types/jsdom": "npm:^21.1.7"
"@types/lodash-es": "npm:^4.17.12"
"@types/node": "npm:^22.0.0"
"@types/node": "npm:^24.0.0"
"@types/pako": "npm:^2.0.3"
"@types/qrcode": "npm:^1.5.5"
"@types/react": "npm:^19.0.0"
@@ -9828,6 +9830,13 @@ __metadata:
languageName: node
linkType: hard
"jose@npm:^6.0.12":
version: 6.1.2
resolution: "jose@npm:6.1.2"
checksum: 10c0/55f79426f43e652ed6d5de938d50f66bb0a10dcae078db81a23f8d3303e889ce226f000e815f3211f9956bb84badce10da892d130d40fe2eca658045a6f1778e
languageName: node
linkType: hard
"jose@npm:^6.1.0":
version: 6.1.0
resolution: "jose@npm:6.1.0"
@@ -10110,8 +10119,8 @@ __metadata:
linkType: hard
"livekit-client@npm:^2.13.0":
version: 2.15.13
resolution: "livekit-client@npm:2.15.13"
version: 2.16.0
resolution: "livekit-client@npm:2.16.0"
dependencies:
"@livekit/mutex": "npm:1.1.1"
"@livekit/protocol": "npm:1.42.2"
@@ -10125,7 +10134,7 @@ __metadata:
webrtc-adapter: "npm:^9.0.1"
peerDependencies:
"@types/dom-mediacapture-record": ^1
checksum: 10c0/5a061df9000461a6d40ef8aa1e72e8aedc640181cc57fe6f2c48c5c7f90ce96a735b125aede377fc43f4692a685e098f17eeae0f42c5b2fed473305867bf2789
checksum: 10c0/5d03adc5d09efde343ab894db397529dff26117598e773b23a5df90a4fb166bde12c6bb1f2cfd1d28dbaf93fe9f275026d7abb75f2ffd2ba816393a2d58e6c7e
languageName: node
linkType: hard
@@ -13603,10 +13612,10 @@ __metadata:
languageName: node
linkType: hard
"undici-types@npm:~6.21.0":
version: 6.21.0
resolution: "undici-types@npm:6.21.0"
checksum: 10c0/c01ed51829b10aa72fc3ce64b747f8e74ae9b60eafa19a7b46ef624403508a54c526ffab06a14a26b3120d055e1104d7abe7c9017e83ced038ea5cf52f8d5e04
"undici-types@npm:~7.16.0":
version: 7.16.0
resolution: "undici-types@npm:7.16.0"
checksum: 10c0/3033e2f2b5c9f1504bdc5934646cb54e37ecaca0f9249c983f7b1fc2e87c6d18399ebb05dc7fd5419e02b2e915f734d872a65da2e3eeed1813951c427d33cc9a
languageName: node
linkType: hard