Compare commits

..

34 Commits

Author SHA1 Message Date
Andrew Ferrazzutti
9e89c4ced3 Add example docker compose; fix dynamic reg 2025-09-03 15:36:42 -04:00
Andrew Ferrazzutti
e52083e29a Start supporting OAuth 2.0 authentication API 2025-08-29 12:44:49 -04:00
Robin
c1694fe706 Merge pull request #3470 from element-hq/af/migrate-defer
Migrate from js-sdk's `defer` in test
2025-08-28 18:21:05 +02:00
Andrew Ferrazzutti
a775bcd62a Migrate from js-sdk's defer in test
See: matrix-org/matrix-js-sdk#4959
2025-08-28 10:14:32 -04:00
renovate[bot]
38e60208ef Update dependency livekit-client to v2.15.6 (#3468)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-08-26 13:54:25 +02:00
Timo
9486ed5d77 Merge pull request #3464 from robintown/reconnecting-test
Test sync loop status and membership status in reconnection test as well
2025-08-25 14:03:32 +02:00
Robin
7ba4df7781 Test sync loop status and membership status in reconnection test as well 2025-08-22 18:15:21 +02:00
Robin
db65a5308a Make most params optional for call view model test setup 2025-08-22 18:14:25 +02:00
fkwp
482e2f8f6f Refine MatrixRTC connection wording (#3463) 2025-08-22 12:34:18 +02:00
ElementRobot
cd5b7cdc47 Merge pull request #3461 from element-hq/actions/localazy-download
Localazy Download
2025-08-21 19:09:14 +02:00
fkwp
e097e58212 Translations updates 2025-08-21 15:39:08 +00:00
Timo
31892f46cd Merge pull request #3453 from robintown/reconnecting
Pause media tracks and show a message when reconnecting to MatrixRTC
2025-08-20 22:24:30 +02:00
Timo K
c0dccf89cb lint
Signed-off-by: Timo K <toger5@hotmail.de>
2025-08-20 22:07:09 +02:00
Robin
44ddb40eea Iterate in pairing session 2025-08-20 20:47:20 +02:00
Robin
5b8839b577 Pause hand raise updates while in 'reconnecting' state 2025-08-20 19:08:44 +02:00
Robin
33bf63d8d2 Disable a bunch of media/event sources when reconnecting 2025-08-20 18:51:03 +02:00
Robin
a7cc11ac81 Merge branch 'livekit' into reconnecting 2025-08-20 14:13:03 +02:00
Robin
25f04030fc Merge pull request #3336 from element-hq/robin/switch-camera-tile
Move the switch camera button to the local user's tile
2025-08-20 13:38:44 +02:00
Robin
5d0289f3c9 Show 'reconnecting' message when session warns we've hit a timeout
By using the new 'probablyLeft' state provided by MatrixRTCSession.
2025-08-20 13:34:41 +02:00
Robin
3da9013426 Show 'reconnecting' message when disconnected from LiveKit 2025-08-20 13:34:36 +02:00
Robin
1a1e5a9db8 Show 'reconnecting' message when sync loop is disconnected
With this change I'm also taking care to not show the standard "Connection to the server has been lost" banner in the call view, since that is now covered by the 'reconnecting' message.
2025-08-20 13:34:26 +02:00
fkwp
87574451a9 Doc: Use new naming of jwt auth service (#3455)
* use new naming of jwt auth service

* use new name of jwt service

* added links the https://github.com/element-hq/lk-jwt-service
2025-08-20 12:45:00 +02:00
fkwp
a0d778083d Allow multiple MatrixRTC sessions per room (#3451)
* bump js-sdk to Allow multiple rtc sessions per room (with different sessionDescriptions)

* Allow new state key string-packing format for widget mode

* bump js-sdk to latest version (with merged multi session PR)

Signed-off-by: Timo K <toger5@hotmail.de>

---------

Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Timo K <toger5@hotmail.de>
2025-08-19 16:13:56 +02:00
Robin
8de6ddceb0 Test that media tracks are paused while reconnecting to MatrixRTC 2025-08-15 20:20:00 +02:00
Robin
8b3b72dfb4 Consolidate yes/no marbles into one definition 2025-08-15 20:19:18 +02:00
Robin
db59679ad4 Allow the local participant's RTC membership to be absent in tests 2025-08-15 20:18:21 +02:00
Robin
f08ae36f9e Pause media tracks and show a message when reconnecting to MatrixRTC 2025-08-15 18:38:52 +02:00
Robin
dc789e63f2 Avoid using the deprecated 'room' field on MatrixRTCSession 2025-08-15 18:32:37 +02:00
renovate[bot]
94085bff00 Update dependency livekit-client to v2.15.5 (#3452)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-08-14 16:47:34 +02:00
Robin
770346676c Merge pull request #3443 from element-hq/toger5/js-to-ts
Use `.ts` for remaining js config files
2025-08-14 15:06:12 +02:00
Robin
8c1fa3fd70 Fix inaccurate copyright headers 2025-08-14 14:57:39 +02:00
Robin
217fa40ddb Only forbid using console in application code 2025-08-14 14:57:20 +02:00
Timo K
02f9d0822f update knip.ts
Signed-off-by: Timo K <toger5@hotmail.de>
2025-08-05 09:17:50 +02:00
Timo K
e133289a7f Make convert remaining js config files to ts
Co-authored-by: hughns
Signed-off-by: Timo K <toger5@hotmail.de>
2025-08-04 19:09:37 +02:00
58 changed files with 1592 additions and 514 deletions

View File

@@ -28,8 +28,6 @@ module.exports = {
rules: {
"matrix-org/require-copyright-header": ["error", COPYRIGHT_HEADER],
"jsx-a11y/media-has-caption": "off",
// We should use the js-sdk logger, never console directly.
"no-console": ["error"],
"react/display-name": "error",
// Encourage proper usage of Promises:
"@typescript-eslint/no-floating-promises": "error",
@@ -46,6 +44,15 @@ module.exports = {
"rxjs/no-exposed-subjects": "error",
"rxjs/finnish": ["error", { names: { "^this$": false } }],
},
overrides: [
{
files: ["src/*/**"],
rules: {
// In application code we should use the js-sdk logger, never console directly.
"no-console": ["error"],
},
},
],
settings: {
react: {
version: "detect",

View File

@@ -143,7 +143,8 @@ via the `org.matrix.msc4143.rtc_foci` key, e.g.:
where the format for MatrixRTC using LiveKit backend is defined in
[MSC4195](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md).
In the example above Matrix clients do discover a focus of type `livekit` which
points them to a Matrix LiveKit JWT Auth Service via `livekit_service_url`.
points them to a [MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)
via `livekit_service_url`.
### Backend Selection
@@ -154,9 +155,9 @@ points them to a Matrix LiveKit JWT Auth Service via `livekit_service_url`.
the **first participant who joined the call** defines via the `foci_preferred`
key in their `org.matrix.msc3401.call.member` which actual MatrixRTC backend
will be used for this call.
- During the actual call join flow, the **LiveKit JWT Auth Service** provides
the client with the **LiveKit SFU WebSocket URL** and an **access JWT token**
in order to exchange media via WebRTC.
- During the actual call join flow, the **[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)**
provides the client with the **LiveKit SFU WebSocket URL** and an
**access JWT token** in order to exchange media via WebRTC.
The example below illustrates how backend selection works across **Matrix
federation**, using the setup from sites A, B, and C. It demonstrates backend
@@ -208,7 +209,8 @@ A docker compose file `dev-backend-docker-compose.yml` is provided to start the
whole stack of components which is required for a local development environment:
- Minimum Synapse Setup (servername: `synapse.m.localhost`)
- LiveKit Authorization Service (Note requires Federation API and hence a TLS reverse proxy)
- Matrix Authentication Service Setup (issuer: `mas.m.localhost`)
- MatrixRTC Authorization Service (Note requires Federation API and hence a TLS reverse proxy)
- Minimum LiveKit SFU Setup using dev defaults for config
- Redis db for completeness
- Minimum `localhost` Certificate Authority (CA) for Transport Layer Security (TLS)
@@ -217,6 +219,7 @@ whole stack of components which is required for a local development environment:
certificates
- Minimum TLS reverse proxy for
- Synapse homeserver: `synapse.m.localhost`
- Matrix Authentication Service: `mas.m.localhost`
- MatrixRTC backend: `matrix-rtc.m.localhost`
- Local Element Call development `call.m.localhost` via `yarn dev --host `
- Element Web `app.m.localhost`

View File

@@ -12,6 +12,11 @@ listeners:
- names: [client, federation, openid]
compress: false
matrix_authentication_service:
enabled: true
secret: "mas-matrix-secret"
endpoint: http://mas:8080
database:
name: sqlite3
args:
@@ -46,9 +51,5 @@ rc_message:
per_second: 0.5
burst_count: 30
# Required for Element Call in Single Page Mode due to on-the-fly user registration
enable_registration: true
enable_registration_without_verification: true
report_stats: false
serve_server_wellknown: true

96
backend/dev_mas.yaml Normal file
View File

@@ -0,0 +1,96 @@
http:
listeners:
- name: web
resources:
- name: discovery
- name: human
- name: oauth
- name: assets
- name: adminapi
binds:
- address: '[::]:8080'
proxy_protocol: false
trusted_proxies:
- 192.168.0.0/16
- 172.16.0.0/12
- 10.0.0.0/10
- 127.0.0.1/8
- fd00::/8
- ::1/128
public_base: https://mas.m.localhost/
database:
uri: postgres://postgres@mas-db/mas
max_connections: 10
min_connections: 0
connect_timeout: 30
idle_timeout: 600
max_lifetime: 1800
email:
from: '"Authentication Service" <root@localhost>'
reply_to: '"Authentication Service" <root@localhost>'
transport: blackhole
secrets:
encryption: 91c9eda308d874d1b8ba51c0fe3b7cbb868638c8fbb82d7eec0e6912586bdabd
keys:
- kid: H30QE7M5eX
key: |
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEA3+t9XjM8LhYMpagIKXnpTXVWxIo5zwM/R8wYRPg0MPGCrOUB
i/L/Vof4yK7lIMWtCT724e989PLR4YmG2pXBpB8P7SZ3feggLoUMi8+QNzyKsGD+
lYmZd0D/2aVmxBxK83JP3LLodmoHduva2qSBF9YdZ6Greg/IcEdG2UY3iGYeojsQ
1Wx7V2+WPeUCtaIONBPk/rwOgWmzAhqGyCXONbvGazElNEuM0fwI278qveP5kNoh
aL6HvlaYQbMSGAg1tf06AKOjsJG0CRsvsMdFxuG9GEwd4pJr9+v+OqwnaFJHJ70z
kQFC3s/w+xCk4NnO+jSopBptu8ycwjZYMuq2kQIDAQABAoIBAHbDqFL2Sc0H1N1o
KiwVhTCYM9U6mz65Mi8aiSTLoKL09aJONGvODrAOnl2SpeSj9AsbYkajh1tEDx3Y
m7YECBjMgN3/sREOtUL3PphJFuy1J7o1N9KIkOU3jHwbxk3t07MbxlAAdFuaESt6
HTIqXm4OGrqEfTbYeC9VHrbPD1VAFj/OGHsYDurJzhfIlFSlZWZqHNjdNh2HAOJM
FElqJWqqR9fj2pYYdpo+oaheI/iIAuWpAgcZOJzaZ4iui4R9i4od+qqQ3EVECPvS
/QnezvDpiobShG6WOmrRj3WBtheiPLdNlNB0sVW9h3dHcrkE/l2n5pfArVbHB+wg
4e5FEAECgYEA4XjB6hpN1iX66ADf27L5mHymu1hTojZLoQdy76OcjgG+4ZoQl71U
OAww4ek9I6Alz+aQqTAnRtLHBKH+xSuO2VoxQfimk71mUmByCr25vIU8mGtXRIJO
rtWEVE4HQPhK2LODLm5zbp3I1GsrfNgMCsuA/yse5MIczC8bRFyf+DECgYEA/jzx
ddl1asjArcFcQyKFJxdobNqYJ4P+rbDLIOmC9IQ8n/v+ETERCzj/y93yaXXqCBlR
uHDzo72F/+SYDDWGYanpRmN2cUv1A0XTUs+dWYjfrscFJUEx8CZh0GeoLE4H3uru
GlwqPnc9sMPee98mj4yDMyrNqLx/VaXV+wnpbGECgYAh4JoKSa9+SLCdYVxBT2/v
OHN43LmcOto8NLlRRl0EfUCn9xUdJ4Za8YH6v6e/DZYA2dzMfv63xn2+tXRpPbU1
9TZHekvVEPUp1XHtKTqaF87V+/LdyVJ3NH+whxTR7zyXuMkyFchkS3Lcb8nV9URB
7vfP3zPCHWRkTYOkTuJ+UQKBgFCrj8ZgMOSoPJMlppvayTtFLypTFjJ7rIT6cwnH
bnkduIrfD5fu5MSV2nyauT+DXbYiKo8GsBhFm849f41oMnKs0ks2Zi++9UiLkGlX
XUs6phc0KUrP7AOSejkBmxgrzk2KZ/DPS8w0U8vR6reNcBPedwb2Tvl6jkDj9QjJ
9VohAoGAISjlufqw3y8F/0on1AhqyROlJghTBsQ+xDEBZ9txx/HcKVghqMhBCbFj
LRf8B4vH9QXtlZVtPFj0wE3INxsYtilsbD8wbwkxsLeGUFdPDucPacfBSyX+wLRh
S1/twrPS9KVhkU5d0TbyfOlEB1OSXZTWZ1n9NaqOZUPf6FAwm78=
-----END RSA PRIVATE KEY-----
- kid: VrRd3Y2OeF
key: |
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIB3zBxhuh275A3piMMDZ2BEM6vsoxswNLrTJRiaY+m80oAoGCCqGSM49
AwEHoUQDQgAECqFotDpyEYNrWf2UZaUB0CZz6KiptQL2wi8oRNkKlarjgDDNCBzR
dgCokx9C8bLpfqhTJE/6aSe6T19qkaPHIg==
-----END EC PRIVATE KEY-----
- kid: SHaCwxflXU
key: |
-----BEGIN EC PRIVATE KEY-----
MIGkAgEBBDCdchyP7aFxQce7vA+QMPkMkOaYKbmNoN1fnlKviKsJK1riq+1eKSEe
UeUF5BOczfugBwYFK4EEACKhZANiAAR1pIE4xN9xkULiCgMd/uztt4Lnu8FhvEZD
3BhUfy5kdBVbYyk1khgKy3k+dQvXaTVkzsHkQN8K78WxlUDlF5zKXLjgkeEiqgz7
HU0rr2e8geUiaEE2AkzWhvmIikvhuMo=
-----END EC PRIVATE KEY-----
- kid: ngjUaMfCuT
key: |
-----BEGIN EC PRIVATE KEY-----
MHQCAQEEIE11jnxjUvPk93ylMuIcwcayJsFUhsSH2EqAn97CiHf8oAcGBSuBBAAK
oUQDQgAE1XySwFNBUkzZ946MBf2/3ecXVptrauZEQ8d8zqUdBS7wOe5pZwZ15Jx4
aZhlusZ3BPl0KiTlWwOlaRDMrw9EGA==
-----END EC PRIVATE KEY-----
passwords:
enabled: true
schemes:
- version: 1
algorithm: argon2id
minimum_complexity: 0
matrix:
kind: synapse
homeserver: synapse.m.localhost
secret: "mas-matrix-secret"
endpoint: http://homeserver:8008/

View File

@@ -40,6 +40,23 @@ server {
}
# Matrix Authentication Server reverse proxy
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name mas.m.localhost;
ssl_certificate /root/ssl/cert.pem;
ssl_certificate_key /root/ssl/key.pem;
location / {
proxy_pass "http://mas:8080";
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
error_page 500 502 503 504 /50x.html;
}
# MatrixRTC reverse proxy
# - MatrixRTC Authorization Service
# - LiveKit SFU websocket signaling connection

View File

@@ -15,5 +15,8 @@
"delayed_leave_event_delay_ms": 18000,
"delayed_leave_event_restart_ms": 4000,
"network_error_retry_ms": 100
},
"oidc_metadata": {
"client_name": "Element Call (dev)"
}
}

View File

@@ -68,6 +68,31 @@ services:
volumes:
- ./backend/synapse_tmp:/data:Z
- ./backend/dev_homeserver.yaml:/data/cfg/homeserver.yaml:Z
depends_on:
- mas
networks:
- ecbackend
mas:
# To add users, see `docker exec element-call-mas-1 mas-cli manage register-user -h`
image: ghcr.io/element-hq/matrix-authentication-service:latest
pull_policy: always
hostname: mas
volumes:
- ./backend/dev_mas.yaml:/config.yaml:ro,Z
depends_on:
- mas-db
networks:
- ecbackend
mas-db:
image: docker.io/postgres:16-alpine
hostname: mas-db
restart: always
shm_size: 128mb
environment:
- POSTGRES_HOST_AUTH_METHOD=trust
- POSTGRES_DB=mas
networks:
- ecbackend
@@ -101,7 +126,9 @@ services:
- "host.docker.internal:host-gateway"
depends_on:
- synapse
- mas
networks:
ecbackend:
aliases:
- mas.m.localhost
- matrix-rtc.m.localhost

Binary file not shown.

Before

Width:  |  Height:  |  Size: 86 KiB

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 929 KiB

After

Width:  |  Height:  |  Size: 941 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 886 KiB

After

Width:  |  Height:  |  Size: 908 KiB

View File

@@ -64,7 +64,7 @@ rc_delayed_event_mgmt:
```
As a prerequisite for the
[Matrix LiveKit JWT auth service](https://github.com/element-hq/lk-jwt-service)
[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)
make sure that your Synapse server has either a `federation` or `openid`
[listener configured](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#listeners).
@@ -77,7 +77,7 @@ required for each site deployment.
As depicted above in the `example.com` site deployment, Element Call requires a
[Livekit SFU](https://github.com/livekit/livekit) alongside a
[Matrix Livekit JWT auth service](https://github.com/element-hq/lk-jwt-service)
[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)
to implement
[MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md).
@@ -89,7 +89,7 @@ the example above, this results in:
| Service | Endpoint | Example |
| -------- | ------- | ------- |
| [Livekit SFU](https://github.com/livekit/livekit) WebSocket signalling connection | `/livekit/sfu` | `matrix-rtc.example.com/livekit/sfu` |
| [Matrix Livekit JWT auth service](https://github.com/element-hq/lk-jwt-service) | `/livekit/jwt` | `matrix-rtc.example.com/livekit/jwt` |
| [MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service) | `/livekit/jwt` | `matrix-rtc.example.com/livekit/jwt` |
Using Nginx, you can achieve this by:
@@ -102,7 +102,7 @@ server {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# JWT Service running at port 8080
# MatrixRTC Authorization Service running at port 8080
proxy_pass http://localhost:8080/;
}

View File

@@ -1,4 +1,6 @@
export default {
import type { UserConfig } from "i18next-parser";
const config: UserConfig = {
keySeparator: ".",
namespaceSeparator: false,
contextSeparator: "|",
@@ -26,3 +28,5 @@ export default {
input: ["src/**/*.{ts,tsx}"],
sort: true,
};
export default config;

11
knip.ts
View File

@@ -1,8 +1,15 @@
import { KnipConfig } from "knip";
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type KnipConfig } from "knip";
export default {
vite: {
config: ["vite.config.js", "vite-embedded.config.js"],
config: ["vite.config.ts", "vite-embedded.config.ts"],
},
entry: ["src/main.tsx", "i18next-parser.config.ts"],
ignoreBinaries: [

View File

@@ -82,7 +82,7 @@
"error": {
"call_is_not_supported": "Volání není podporováno",
"call_not_found": "Volání nebylo nalezeno",
"call_not_found_description": "<0>Zdá se, že tento odkaz nepatří k žádnému existujícímu volání. Zkontrolujte, zda máte správný odkaz, nebo <1>vytvořte nový</1>.</0>",
"call_not_found_description": "<0>Zdá se, že tento odkaz nepatří k žádnému existujícímu hovoru. Zkontrolujte, zda máte správný odkaz, nebo<2> vytvořte nový</2>.</0>",
"connection_lost": "Spojení ztraceno",
"connection_lost_description": "Hovor byl přerušen.",
"e2ee_unsupported": "Nekompatibilní prohlížeč",
@@ -105,6 +105,11 @@
"knock_reject_heading": "Přístup odepřen",
"reason": "Důvod"
},
"handset": {
"overlay_back_button": "Zpět do režimu reproduktoru",
"overlay_description": "Funguje pouze při používání aplikace",
"overlay_title": "Režim sluchátka"
},
"hangup_button_label": "Ukončit hovor",
"header_label": "Domov Element Call",
"header_participants_label": "Účastníci",
@@ -177,6 +182,8 @@
"change_device_button": "Změnit zvukové zařízení",
"default": "Výchozí",
"default_named": "Výchozí <2> ({{name}}) </2>",
"handset": "Sluchátko",
"loudspeaker": "Reproduktor",
"microphone": "Mikrofon",
"microphone_numbered": "Mikrofon {{n}}",
"speaker": "Reproduktor",

View File

@@ -94,6 +94,8 @@
"matrix_rtc_focus_missing": "Serveren er ikke konfigureret til at arbejde med {{brand}}{{domain}}. Kontakt venligst din serveradministrator (domæne:{{domain}}, fejlkode: {{ errorCode }}).",
"open_elsewhere": "Åbnet i en anden fane",
"open_elsewhere_description": "{{brand}} er blevet åbnet i en anden fane. Hvis det ikke lyder rigtigt, kan du prøve at genindlæse siden.",
"room_creation_restricted": "Kunne ikke oprette opkald",
"room_creation_restricted_description": "Oprettelse af opkald er muligvis begrænset til autoriserede brugere. Prøv igen senere, eller kontakt din serveradministrator, hvis problemet fortsætter.",
"unexpected_ec_error": "Der opstod en uventet fejl (<0>Fejlkode:</0> <1> {{ errorCode }}</1>). Kontakt venligst din serveradministrator."
},
"group_call_loader": {
@@ -105,6 +107,11 @@
"knock_reject_heading": "Adgang nægtet",
"reason": "Årsag: {{reason}}"
},
"handset": {
"overlay_back_button": "Tilbage til højttalertilstand",
"overlay_description": "Virker kun, når du bruger appen",
"overlay_title": "Telefon-højtaler"
},
"hangup_button_label": "Afslut opkald",
"header_label": "Element Ring hjem",
"header_participants_label": "Deltagere",
@@ -176,6 +183,7 @@
"change_device_button": "Skift lydenhed",
"default": "Standard",
"default_named": "Standard <2>({{name}})</2>",
"handset": "Telefon",
"loudspeaker": "Højttaler",
"microphone": "Mikrofon",
"microphone_numbered": "Mikrofon {{n}}",

View File

@@ -55,6 +55,7 @@
"profile": "Profil",
"reaction": "Reaktion",
"reactions": "Reaktionen",
"reconnecting": "Verbindung wird wiederhergestellt...",
"settings": "Einstellungen",
"unencrypted": "Nicht verschlüsselt",
"username": "Benutzername",

View File

@@ -55,6 +55,7 @@
"profile": "Profile",
"reaction": "Reaction",
"reactions": "Reactions",
"reconnecting": "Reconnecting…",
"settings": "Settings",
"unencrypted": "Not encrypted",
"username": "Username",
@@ -89,6 +90,7 @@
"e2ee_unsupported_description": "Your web browser does not support encrypted calls. Supported browsers include Chrome, Safari, and Firefox 117+.",
"generic": "Something went wrong",
"generic_description": "Submitting debug logs will help us track down the problem.",
"homeserver_misconfig": "Misconfigured homeserver",
"insufficient_capacity": "Insufficient capacity",
"insufficient_capacity_description": "The server has reached its maximum capacity and you cannot join the call at this time. Try again later, or contact your server admin if the problem persists.",
"matrix_rtc_focus_missing": "The server is not configured to work with {{brand}}. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",

View File

@@ -82,7 +82,7 @@
"error": {
"call_is_not_supported": "Call stöds inte",
"call_not_found": "Samtal hittades inte",
"call_not_found_description": "<0>Den länken verkar inte tillhöra något befintligt samtal. Kontrollera att du har rätt länk, eller <1>skapa en ny</1>.</0>",
"call_not_found_description": "<0>Den länken verkar inte tillhöra något befintligt samtal. Kontrollera att du har rätt länk, eller <2>skapa en ny</2>.</0>",
"connection_lost": "Anslutning förlorad",
"connection_lost_description": "Du kopplades bort från samtalet.",
"e2ee_unsupported": "Inkompatibel webbläsare",
@@ -94,6 +94,8 @@
"matrix_rtc_focus_missing": "Servern är inte konfigurerad för att fungera med {{brand}}. Vänligen kontakta serveradministratören (Domän: {{domain}}, Felkod: {{ errorCode }}).",
"open_elsewhere": "Öppnades i en annan flik",
"open_elsewhere_description": "{{brand}} har öppnats i en annan flik. Om det inte låter rätt, pröva att ladda om sidan.",
"room_creation_restricted": "Misslyckades att skapa samtal",
"room_creation_restricted_description": "Samtalsskapande kan vara begränsat till endast behöriga användare. Försök igen senare eller kontakta serveradministratören om problemet kvarstår.",
"unexpected_ec_error": "Ett oväntat fel inträffade (<0>Felkod:</0> <1>{{ errorCode }}</1>). Kontakta din serveradministratör."
},
"group_call_loader": {
@@ -105,6 +107,11 @@
"knock_reject_heading": "Inte tillåten att gå med",
"reason": "Anledning"
},
"handset": {
"overlay_back_button": "Tillbaka till högtalarläge",
"overlay_description": "Fungerar bara när appen används",
"overlay_title": "Telefonläge"
},
"hangup_button_label": "Avsluta samtal",
"header_label": "Element Call Hem",
"header_participants_label": "Deltagare",
@@ -176,6 +183,8 @@
"change_device_button": "Byt ljudenhet",
"default": "Förval",
"default_named": "Förval <2>({{name}})</2>",
"handset": "Telefonlur",
"loudspeaker": "Högtalare",
"microphone": "Mikrofon",
"microphone_numbered": "Mikrofon {{n}}",
"speaker": "Högtalare",

View File

@@ -20,11 +20,11 @@ import { logger } from "matrix-js-sdk/lib/logger";
import { HomePage } from "./home/HomePage";
import { LoginPage } from "./auth/LoginPage";
import { OidcRedirectPage } from "./auth/OidcRedirectPage";
import { RegisterPage } from "./auth/RegisterPage";
import { RoomPage } from "./room/RoomPage";
import { ClientProvider } from "./ClientContext";
import { ErrorPage, LoadingPage } from "./FullScreenView";
import { DisconnectedBanner } from "./DisconnectedBanner";
import { Initializer } from "./initializer";
import { widget } from "./widget";
import { useTheme } from "./useTheme";
@@ -86,10 +86,10 @@ export const App: FC<Props> = ({ vm }) => {
<Sentry.ErrorBoundary
fallback={(error) => <ErrorPage error={error} widget={widget} />}
>
<DisconnectedBanner />
<Routes>
<SentryRoute path="/" element={<HomePage />} />
<SentryRoute path="/login" element={<LoginPage />} />
<SentryRoute path="/after-login" element={<OidcRedirectPage />} />
<SentryRoute path="/register" element={<RegisterPage />} />
<SentryRoute path="*" element={<RoomPage />} />
</Routes>

View File

@@ -61,7 +61,11 @@ export const AppBar: FC<Props> = ({ children }) => {
style={{ display: hidden ? "none" : "block" }}
className={styles.bar}
>
<Header>
<Header
// App bar is mainly seen in the call view, which has its own
// 'reconnecting' toast
disconnectedBanner={false}
>
<LeftNav>
<Tooltip label={t("common.back")}>
<IconButton onClick={onBackClick}>

View File

@@ -21,6 +21,7 @@ const TestComponent: FC<
<ClientContextProvider
value={{
state: "valid",
oidcClientConfig: null,
disconnected: false,
supportedFeatures: {
reactions: true,

View File

@@ -1,5 +1,5 @@
/*
Copyright 2021-2024 New Vector Ltd.
Copyright 2021-2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -19,7 +19,7 @@ import {
import { useNavigate } from "react-router-dom";
import { logger } from "matrix-js-sdk/lib/logger";
import { type ISyncStateData, type SyncState } from "matrix-js-sdk/lib/sync";
import { ClientEvent, type MatrixClient } from "matrix-js-sdk";
import { ClientEvent, type OidcClientConfig, type MatrixClient } from "matrix-js-sdk";
import type { WidgetApi } from "matrix-widget-api";
import { ErrorPage } from "./FullScreenView";
@@ -29,6 +29,7 @@ import {
RegistrationType,
} from "./analytics/PosthogAnalytics";
import { useEventTarget } from "./useEvents";
import { getAuthMetadata } from "./utils/oidc/discovery";
import { OpenElsewhereError } from "./RichError";
declare global {
@@ -42,6 +43,7 @@ export type ClientState = ValidClientState | ErrorState;
export type ValidClientState = {
state: "valid";
oidcClientConfig: OidcClientConfig | null,
authenticated?: AuthenticatedClient;
// 'Disconnected' rather than 'connected' because it tracks specifically
// whether the client is supposed to be connected but is not
@@ -74,17 +76,20 @@ export const useClientState = (): ClientState | undefined => use(ClientContext);
export function useClient(): {
client?: MatrixClient;
setClient?: (client: MatrixClient, session: Session) => void;
oidcClientConfig: OidcClientConfig | null | undefined;
} {
let client;
let setClient;
let oidcClientConfig;
const clientState = useClientState();
if (clientState?.state === "valid") {
client = clientState.authenticated?.client;
setClient = clientState.setClient;
oidcClientConfig = clientState.oidcClientConfig;
}
return { client, setClient };
return { client, setClient, oidcClientConfig };
}
// Plain representation of the `ClientContext` as a helper for old components that expected an object with multiple fields.
@@ -140,6 +145,12 @@ interface Props {
export const ClientProvider: FC<Props> = ({ children }) => {
const navigate = useNavigate();
// null = no OIDC config, undefined = loading
const [oidcClientConfig, setOidcClientConfig] = useState<
OidcClientConfig | null | undefined
>(undefined);
const [oidcErr, setOidcErr] = useState<Error | undefined>(undefined);
// null = signed out, undefined = loading
const [initClientState, setInitClientState] = useState<
InitResult | null | undefined
@@ -153,6 +164,11 @@ export const ClientProvider: FC<Props> = ({ children }) => {
if (initializing.current) return;
initializing.current = true;
if (!widget) {
// TODO: spec says this may change over time & should be refreshed upon cache expiry
getAuthMetadata().then(setOidcClientConfig, setOidcErr);
}
loadClient()
.then((initResult) => {
setInitClientState(initResult);
@@ -251,12 +267,18 @@ export const ClientProvider: FC<Props> = ({ children }) => {
const [supportsReactions, setSupportsReactions] = useState(false);
const [supportsThumbnails, setSupportsThumbnails] = useState(false);
const state: ClientState | undefined = useMemo(() => {
if (alreadyOpenedErr) {
return { state: "error", error: alreadyOpenedErr };
const state = useMemo((): ClientState | undefined => {
const error = alreadyOpenedErr || oidcErr;
if (error) {
return { state: "error", error };
}
if (initClientState === undefined) return undefined;
if (
initClientState === undefined ||
oidcClientConfig === undefined
) {
return undefined;
}
const authenticated =
initClientState === null
@@ -270,6 +292,7 @@ export const ClientProvider: FC<Props> = ({ children }) => {
return {
state: "valid",
oidcClientConfig,
authenticated,
setClient,
disconnected: isDisconnected,
@@ -280,6 +303,8 @@ export const ClientProvider: FC<Props> = ({ children }) => {
};
}, [
alreadyOpenedErr,
oidcErr,
oidcClientConfig,
changePassword,
initClientState,
logout,
@@ -345,8 +370,9 @@ export const ClientProvider: FC<Props> = ({ children }) => {
};
}, [initClientState, onSync]);
if (alreadyOpenedErr) {
return <ErrorPage widget={widget} error={alreadyOpenedErr} />;
const error = alreadyOpenedErr || oidcErr;
if (error) {
return <ErrorPage widget={widget} error={error} />;
}
return <ClientContext value={state}>{children}</ClientContext>;

View File

@@ -17,27 +17,38 @@ import Logo from "./icons/Logo.svg?react";
import { Avatar, Size } from "./Avatar";
import { EncryptionLock } from "./room/EncryptionLock";
import { useMediaQuery } from "./useMediaQuery";
import { DisconnectedBanner } from "./DisconnectedBanner";
interface HeaderProps extends HTMLAttributes<HTMLElement> {
ref?: Ref<HTMLElement>;
children: ReactNode;
className?: string;
/**
* Whether the header should display an informational banner whenever the
* client is disconnected from the homeserver.
* @default true
*/
disconnectedBanner?: boolean;
}
export const Header: FC<HeaderProps> = ({
ref,
children,
className,
disconnectedBanner = true,
...rest
}) => {
return (
<header
ref={ref}
className={classNames(styles.header, className)}
{...rest}
>
{children}
</header>
<>
<header
ref={ref}
className={classNames(styles.header, className)}
{...rest}
>
{children}
</header>
{disconnectedBanner && <DisconnectedBanner />}
</>
);
};

View File

@@ -6,7 +6,7 @@ Please see LICENSE in the repository root for full details.
*/
import { useTranslation } from "react-i18next";
import { PopOutIcon } from "@vector-im/compound-design-tokens/assets/web/icons";
import { ErrorIcon, PopOutIcon } from "@vector-im/compound-design-tokens/assets/web/icons";
import type { FC, ReactNode } from "react";
import { ErrorView } from "./ErrorView";
@@ -51,3 +51,29 @@ export class OpenElsewhereError extends RichError {
super("App opened in another tab", <OpenElsewhere />);
}
}
const HomeserverMisconfig: FC<{message?: string}> = (props) => {
const { t } = useTranslation();
// TODO: don't want to show "Return to home screen" button for an error as fatal as this
return (
<ErrorView
widget={widget}
Icon={ErrorIcon}
title={t("error.homeserver_misconfig")}
fatal={true}
>
{props.message && (
<p>
{props.message}
</p>
)}
</ErrorView>
);
};
export class HomeserverMisconfigError extends RichError {
public constructor(message?: string) {
super(message || "Unknown error", <HomeserverMisconfig message={message} />);
}
}

View File

@@ -45,6 +45,12 @@ interface Props {
* A supporting icon to display within the toast.
*/
Icon?: ComponentType<SVGAttributes<SVGElement>>;
/**
* Whether the toast should be portaled into the root of the document (rather
* than rendered in-place within the component tree).
* @default true
*/
portal?: boolean;
}
/**
@@ -56,6 +62,7 @@ export const Toast: FC<Props> = ({
autoDismiss,
children,
Icon,
portal = true,
}) => {
const onOpenChange = useCallback(
(open: boolean) => {
@@ -71,29 +78,33 @@ export const Toast: FC<Props> = ({
}
}, [open, autoDismiss, onDismiss]);
const content = (
<>
<DialogOverlay
className={classNames(overlayStyles.bg, overlayStyles.animate)}
/>
<DialogContent aria-describedby={undefined} asChild>
<DialogClose
className={classNames(
overlayStyles.overlay,
overlayStyles.animate,
styles.toast,
)}
>
<DialogTitle asChild>
<Text as="h3" size="sm" weight="semibold">
{children}
</Text>
</DialogTitle>
{Icon && <Icon width={20} height={20} aria-hidden />}
</DialogClose>
</DialogContent>
</>
);
return (
<DialogRoot open={open} onOpenChange={onOpenChange}>
<DialogPortal>
<DialogOverlay
className={classNames(overlayStyles.bg, overlayStyles.animate)}
/>
<DialogContent aria-describedby={undefined} asChild>
<DialogClose
className={classNames(
overlayStyles.overlay,
overlayStyles.animate,
styles.toast,
)}
>
<DialogTitle asChild>
<Text as="h3" size="sm" weight="semibold">
{children}
</Text>
</DialogTitle>
{Icon && <Icon width={20} height={20} aria-hidden />}
</DialogClose>
</DialogContent>
</DialogPortal>
{portal ? <DialogPortal>{content}</DialogPortal> : content}
</DialogRoot>
);
};

View File

@@ -1,11 +1,11 @@
/*
Copyright 2021-2024 New Vector Ltd.
Copyright 2021-2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC, type FormEvent, useCallback, useRef, useState } from "react";
import { type FC, type FormEvent, type ReactElement, useCallback, useRef, useState } from "react";
import { useNavigate, useLocation } from "react-router-dom";
import { Trans, useTranslation } from "react-i18next";
import { Button } from "@vector-im/compound-web";
@@ -19,13 +19,14 @@ import { usePageTitle } from "../usePageTitle";
import { PosthogAnalytics } from "../analytics/PosthogAnalytics";
import { Config } from "../config/Config";
import { Link } from "../button/Link";
import { LoadingPage } from "../FullScreenView";
import { getOidcClientId } from "../utils/oidc/registerClient";
import { startOidcLogin } from "../utils/oidc/authorize";
export const LoginPage: FC = () => {
const { t } = useTranslation();
usePageTitle(t("login_title"));
const { client, setClient } = useClient();
const login = useInteractiveLogin(client);
const homeserver = Config.defaultHomeserverUrl(); // TODO: Make this configurable
const usernameRef = useRef<HTMLInputElement>(null);
const passwordRef = useRef<HTMLInputElement>(null);
@@ -36,19 +37,23 @@ export const LoginPage: FC = () => {
// TODO: Handle hitting login page with authenticated client
const onSubmitLoginForm = useCallback(
(e: FormEvent<HTMLFormElement>) => {
e.preventDefault();
setLoading(true);
let node: ReactElement;
const { client, setClient, oidcClientConfig } = useClient();
if (oidcClientConfig === null) {
const login = useInteractiveLogin(client);
const onSubmitLoginForm = useCallback(
async (e: FormEvent<HTMLFormElement>): Promise<void> => {
e.preventDefault();
setLoading(true);
if (!homeserver || !usernameRef.current || !passwordRef.current) {
setError(Error("Login parameters are undefined"));
setLoading(false);
return;
}
if (!homeserver) {
setError(Error("Login parameters are undefined"));
setLoading(false);
return;
}
login(homeserver, usernameRef.current.value, passwordRef.current.value)
.then(async ([client, session]) => {
try {
const [client, session] = await login(homeserver, usernameRef.current?.value ?? "", passwordRef.current?.value ?? "");
if (!setClient) {
return;
}
@@ -66,20 +71,84 @@ export const LoginPage: FC = () => {
await navigate("/");
}
PosthogAnalytics.instance.eventLogin.track();
})
.catch((error) => {
} catch (error: any) {
setError(error);
setLoading(false);
});
},
[login, location, navigate, homeserver, setClient],
);
// we need to limit the length of the homserver name to not cover the whole loginview input with the string.
let shortendHomeserverName = Config.defaultServerName()?.slice(0, 25);
shortendHomeserverName =
shortendHomeserverName?.length !== Config.defaultServerName()?.length
? shortendHomeserverName + "..."
: shortendHomeserverName;
}
},
[login, location, navigate, homeserver, setClient],
);
// we need to limit the length of the homserver name to not cover the whole loginview input with the string.
let shortendHomeserverName = Config.defaultServerName()?.slice(0, 25);
shortendHomeserverName =
shortendHomeserverName?.length !== Config.defaultServerName()?.length
? shortendHomeserverName + "..."
: shortendHomeserverName;
node = (
<form onSubmit={onSubmitLoginForm}>
<FieldRow>
<InputField
type="text"
ref={usernameRef}
placeholder={t("common.username")}
label={t("common.username")}
autoCorrect="off"
autoCapitalize="none"
prefix="@"
suffix={`:${shortendHomeserverName}`}
data-testid="login_username"
/>
</FieldRow>
<FieldRow>
<InputField
type="password"
ref={passwordRef}
placeholder={t("common.password")}
label={t("common.password")}
data-testid="login_password"
/>
</FieldRow>
{error && (
<FieldRow>
<ErrorMessage error={error} />
</FieldRow>
)}
<FieldRow>
<Button
type="submit"
disabled={loading}
data-testid="login_login"
>
{loading ? t("logging_in") : t("login_title")}
</Button>
</FieldRow>
</form>
);
} else if (oidcClientConfig !== undefined) {
if (!homeserver) {
setError(Error("No homeserver is configured"));
setLoading(false);
return;
}
node = <Button
kind="primary"
onClick={async () => {
// TODO: get the clientId at the same time as doing OIDC discovery
const clientId = await getOidcClientId(oidcClientConfig);
await startOidcLogin(
oidcClientConfig,
clientId,
homeserver,
);
}}
>
{t("room_auth_view_continue_button") /* TODO: make a dedicated word for this */}
</Button>;
} else {
return <LoadingPage />;
}
return (
<>
<div className={styles.container}>
@@ -89,44 +158,7 @@ export const LoginPage: FC = () => {
<h2>{t("log_in")}</h2>
<h4>{t("login_subheading")}</h4>
<form onSubmit={onSubmitLoginForm}>
<FieldRow>
<InputField
type="text"
ref={usernameRef}
placeholder={t("common.username")}
label={t("common.username")}
autoCorrect="off"
autoCapitalize="none"
prefix="@"
suffix={`:${shortendHomeserverName}`}
data-testid="login_username"
/>
</FieldRow>
<FieldRow>
<InputField
type="password"
ref={passwordRef}
placeholder={t("common.password")}
label={t("common.password")}
data-testid="login_password"
/>
</FieldRow>
{error && (
<FieldRow>
<ErrorMessage error={error} />
</FieldRow>
)}
<FieldRow>
<Button
type="submit"
disabled={loading}
data-testid="login_login"
>
{loading ? t("logging_in") : t("login_title")}
</Button>
</FieldRow>
</form>
{node}
</div>
<div className={styles.authLinks}>
<p>{t("login_auth_links_prompt")}</p>

View File

@@ -0,0 +1,104 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { useState, type FC } from "react";
import { useLocation, useNavigate } from "react-router-dom";
import { useTranslation } from "react-i18next";
import { createClient, type MatrixClient } from "matrix-js-sdk/src/matrix";
import { useClient } from "../ClientContext";
import { usePageTitle } from "../usePageTitle";
import { completeOidcLogin } from "../utils/oidc/authorize";
import { initClient } from "../utils/matrix";
/**
* TODO: Yoinked from element-web, so move to SDK if possible
* Gets information about the owner of a given access token.
* @returns Promise that resolves with whoami response
* @throws when whoami request fails
*/
async function getUserIdFromAccessToken(
accessToken: string,
homeserverUrl: string,
): Promise<ReturnType<MatrixClient["whoami"]>> {
try {
const client = createClient({
baseUrl: homeserverUrl,
accessToken: accessToken,
});
return await client.whoami();
} catch (error) {
throw new Error("Failed to retrieve userId using accessToken");
}
}
export const OidcRedirectPage: FC = async () => {
const { t } = useTranslation();
// TODO: probably want a new page title
usePageTitle(t("login_title"));
const navigate = useNavigate();
const location = useLocation();
const [_, setError] = useState<Error>();
const { setClient } = useClient();
if (!setClient) {
return;
}
// TODO: make reactive
try {
const queryParams = new URLSearchParams(location.search);
const { accessToken, refreshToken, homeserverUrl, idToken, clientId, issuer } =
await completeOidcLogin(queryParams);
const {
user_id: userId,
device_id: deviceId,
} = await getUserIdFromAccessToken(accessToken, homeserverUrl);
const session = {
user_id: userId,
access_token: accessToken,
device_id: deviceId!, // TODO: make sure this really is always defined
passwordlessUser: false,
};
console.debug(`TODO: use ${refreshToken}`);
const client = await initClient(
{
baseUrl: homeserverUrl,
accessToken,
userId,
deviceId,
},
false,
);
setClient(client, session);
console.debug(`TODO: use ${clientId}, ${issuer}, ${idToken}`);
// persistOidcAuthenticatedSettings(clientId, issuer, idToken);
const locationState = location.state;
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
if (locationState && locationState.from) {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
await navigate(locationState.from);
} else {
await navigate("/");
}
} catch (error: any) {
setError(error);
return;
}
return (
<></>
);
}

View File

@@ -1,5 +1,5 @@
/*
Copyright 2022-2024 New Vector Ltd.
Copyright 2022-2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
@@ -151,6 +151,23 @@ export interface ConfigOptions {
*/
membership_event_expiry_ms?: number;
};
/**
* Configuration for OIDC issuers where a static client_id has been issued for the app.
* Otherwise dynamic client registration is attempted.
*/
oidc_static_clients?: {
[issuer: string]: { client_id: string };
};
oidc_metadata?: {
client_name?: string;
client_uri?: string;
logo_uri?: string;
tos_uri?: string;
policy_uri?: string;
contacts?: string[];
}
}
// Overrides members from ConfigOptions that are always provided by the

View File

@@ -1,7 +1,7 @@
/*
Copyright 2024-2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/

View File

@@ -1,7 +1,7 @@
/*
Copyright 2024 New Vector Ltd.
Copyright 2024-2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/

View File

@@ -15,7 +15,7 @@ import {
import userEvent from "@testing-library/user-event";
import { render, screen } from "@testing-library/react";
import { MemoryRouter } from "react-router-dom";
import { defer, sleep } from "matrix-js-sdk/lib/utils";
import { sleep } from "matrix-js-sdk/lib/utils";
import { useECConnectionState } from "./useECConnectionState";
import { type SFUConfig } from "./openIDSFU";
@@ -93,7 +93,7 @@ describe("Leaking connection prevention", () => {
test("Should cancel pending connections when the component is unmounted", async () => {
const connectCall = vi.fn();
const pendingConnection = defer<void>();
const pendingConnection = Promise.withResolvers<void>();
// let pendingDisconnection = defer<void>()
const disconnectMock = vi.fn();
@@ -141,7 +141,7 @@ describe("Leaking connection prevention", () => {
test("Should cancel about to open but not yet opened connection", async () => {
const createTracksCall = vi.fn();
const pendingCreateTrack = defer<void>();
const pendingCreateTrack = Promise.withResolvers<void>();
// let pendingDisconnection = defer<void>()
const disconnectMock = vi.fn();
const connectMock = vi.fn();

View File

@@ -31,6 +31,7 @@ import {
aliceRtcMember,
bobRtcMember,
local,
localRtcMember,
} from "../utils/test-fixtures";
vitest.mock("../useAudioContext");
@@ -66,7 +67,7 @@ beforeEach(() => {
* a noise every time.
*/
test("plays one sound when entering a call", () => {
const { vm, remoteRtcMemberships$ } = getBasicCallViewModelEnvironment([
const { vm, rtcMemberships$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
@@ -74,56 +75,58 @@ test("plays one sound when entering a call", () => {
// Joining a call usually means remote participants are added later.
act(() => {
remoteRtcMemberships$.next([aliceRtcMember, bobRtcMember]);
rtcMemberships$.next([localRtcMember, aliceRtcMember, bobRtcMember]);
});
expect(playSound).toHaveBeenCalledOnce();
});
test("plays a sound when a user joins", () => {
const { vm, remoteRtcMemberships$ } = getBasicCallViewModelEnvironment([
const { vm, rtcMemberships$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
render(<CallEventAudioRenderer vm={vm} />);
act(() => {
remoteRtcMemberships$.next([aliceRtcMember, bobRtcMember]);
rtcMemberships$.next([localRtcMember, aliceRtcMember, bobRtcMember]);
});
// Play a sound when joining a call.
expect(playSound).toBeCalledWith("join");
});
test("plays a sound when a user leaves", () => {
const { vm, remoteRtcMemberships$ } = getBasicCallViewModelEnvironment([
const { vm, rtcMemberships$ } = getBasicCallViewModelEnvironment([
local,
alice,
]);
render(<CallEventAudioRenderer vm={vm} />);
act(() => {
remoteRtcMemberships$.next([]);
rtcMemberships$.next([localRtcMember]);
});
expect(playSound).toBeCalledWith("left");
});
test("plays no sound when the participant list is more than the maximum size", () => {
const mockRtcMemberships: CallMembership[] = [];
const mockRtcMemberships: CallMembership[] = [localRtcMember];
for (let i = 0; i < MAX_PARTICIPANT_COUNT_FOR_SOUND; i++) {
mockRtcMemberships.push(
mockRtcMembership(`@user${i}:example.org`, `DEVICE${i}`),
);
}
const { vm, remoteRtcMemberships$ } = getBasicCallViewModelEnvironment(
const { vm, rtcMemberships$ } = getBasicCallViewModelEnvironment(
[local, alice],
mockRtcMemberships,
);
render(<CallEventAudioRenderer vm={vm} />);
expect(playSound).not.toBeCalled();
// Remove the last membership in the array to test the leaving sound
// (The array has length MAX_PARTICIPANT_COUNT_FOR_SOUND + 1)
act(() => {
remoteRtcMemberships$.next(
mockRtcMemberships.slice(0, MAX_PARTICIPANT_COUNT_FOR_SOUND - 1),
rtcMemberships$.next(
mockRtcMemberships.slice(0, MAX_PARTICIPANT_COUNT_FOR_SOUND),
);
});
expect(playSound).toBeCalledWith("left");

View File

@@ -137,11 +137,9 @@ function createGroupCallView(
getJoinRule: () => JoinRule.Invite,
} as Partial<RoomState> as RoomState,
});
const rtcSession = new MockRTCSession(
room,
localRtcMember,
[],
).withMemberships(constant([]));
const rtcSession = new MockRTCSession(room, []).withMemberships(
constant([localRtcMember]),
);
rtcSession.joined = joined;
const muteState = {
audio: { enabled: false },

View File

@@ -452,6 +452,7 @@ export const GroupCallView: FC<Props> = ({
client={client}
matrixInfo={matrixInfo}
rtcSession={rtcSession as MatrixRTCSession}
matrixRoom={room}
participantCount={participantCount}
onLeave={onLeave}
header={header}

View File

@@ -175,6 +175,7 @@ function createInCallView(): RenderResult & {
kind: E2eeType.NONE,
},
}}
matrixRoom={room}
livekitRoom={livekitRoom}
participantCount={0}
onLeave={function (): void {

View File

@@ -7,8 +7,8 @@ Please see LICENSE in the repository root for full details.
import { RoomContext, useLocalParticipant } from "@livekit/components-react";
import { IconButton, Text, Tooltip } from "@vector-im/compound-web";
import { ConnectionState, type Room } from "livekit-client";
import { type MatrixClient } from "matrix-js-sdk";
import { ConnectionState, type Room as LivekitRoom } from "livekit-client";
import { type MatrixClient, type Room as MatrixRoom } from "matrix-js-sdk";
import {
type FC,
type PointerEvent,
@@ -111,6 +111,7 @@ import { useMediaDevices } from "../MediaDevicesContext.ts";
import { EarpieceOverlay } from "./EarpieceOverlay.tsx";
import { useAppBarHidden, useAppBarSecondaryButton } from "../AppBar.tsx";
import { useBehavior } from "../useBehavior.ts";
import { Toast } from "../Toast.tsx";
const canScreenshare = "getDisplayMedia" in (navigator.mediaDevices ?? {});
@@ -164,6 +165,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
const reactionsReader = new ReactionsReader(props.rtcSession);
const vm = new CallViewModel(
props.rtcSession,
props.matrixRoom,
livekitRoom,
mediaDevices,
{
@@ -182,6 +184,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
}
}, [
props.rtcSession,
props.matrixRoom,
livekitRoom,
mediaDevices,
props.e2eeSystem,
@@ -210,7 +213,8 @@ export interface InCallViewProps {
vm: CallViewModel;
matrixInfo: MatrixInfo;
rtcSession: MatrixRTCSession;
livekitRoom: Room;
matrixRoom: MatrixRoom;
livekitRoom: LivekitRoom;
muteStates: MuteStates;
participantCount: number;
/** Function to call when the user explicitly ends the call */
@@ -226,6 +230,7 @@ export const InCallView: FC<InCallViewProps> = ({
vm,
matrixInfo,
rtcSession,
matrixRoom,
livekitRoom,
muteStates,
participantCount,
@@ -270,7 +275,7 @@ export const InCallView: FC<InCallViewProps> = ({
const [useExperimentalToDeviceTransport] = useSetting(
useExperimentalToDeviceTransportSetting,
);
const encryptionSystem = useRoomEncryptionSystem(rtcSession.room.roomId);
const encryptionSystem = useRoomEncryptionSystem(matrixRoom.roomId);
const memberships = useMatrixRTCSessionMemberships(rtcSession);
const showToDeviceEncryption = useMemo(
@@ -307,6 +312,7 @@ export const InCallView: FC<InCallViewProps> = ({
() => void toggleRaisedHand(),
);
const reconnecting = useBehavior(vm.reconnecting$);
const windowMode = useBehavior(vm.windowMode$);
const layout = useBehavior(vm.layout$);
const tileStoreGeneration = useBehavior(vm.tileStoreGeneration$);
@@ -499,7 +505,11 @@ export const InCallView: FC<InCallViewProps> = ({
break;
case "standard":
header = (
<Header className={styles.header} ref={headerRef}>
<Header
className={styles.header}
ref={headerRef}
disconnectedBanner={false} // This screen has its own 'reconnecting' toast
>
<LeftNav>
<RoomHeaderInfo
id={matrixInfo.roomId}
@@ -639,7 +649,7 @@ export const InCallView: FC<InCallViewProps> = ({
};
const rageshakeRequestModalProps = useRageshakeRequestModal(
rtcSession.room.roomId,
matrixRoom.roomId,
);
const toggleScreensharing = useCallback(() => {
@@ -750,6 +760,9 @@ export const InCallView: FC<InCallViewProps> = ({
</div>
);
// The reconnecting toast cannot be dismissed
const onDismissReconnectingToast = useCallback(() => {}, []);
return (
<div
className={styles.inRoom}
@@ -777,8 +790,15 @@ export const InCallView: FC<InCallViewProps> = ({
{renderContent()}
<CallEventAudioRenderer vm={vm} muted={muteAllAudio} />
<ReactionsAudioRenderer vm={vm} muted={muteAllAudio} />
<Toast
onDismiss={onDismissReconnectingToast}
open={reconnecting}
portal={false}
>
{t("common.reconnecting")}
</Toast>
<EarpieceOverlay
show={earpieceMode}
show={earpieceMode && !reconnecting}
onBackToVideoPressed={audioOutputSwitcher?.switch}
/>
<ReactionsOverlay vm={vm} />
@@ -788,7 +808,7 @@ export const InCallView: FC<InCallViewProps> = ({
<RageshakeRequestModal {...rageshakeRequestModalProps} />
<SettingsModal
client={client}
roomId={rtcSession.room.roomId}
roomId={matrixRoom.roomId}
open={settingsModalOpen}
onDismiss={closeSettings}
tab={settingsTab}

View File

@@ -256,7 +256,7 @@ exports[`InCallView > rendering > renders 1`] = `
>
<button
aria-disabled="false"
aria-labelledby="«r5»"
aria-labelledby="«r8»"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
data-kind="primary"
data-size="lg"
@@ -279,7 +279,7 @@ exports[`InCallView > rendering > renders 1`] = `
</button>
<button
aria-disabled="false"
aria-labelledby="«ra»"
aria-labelledby="«rd»"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
data-kind="primary"
data-size="lg"
@@ -301,7 +301,7 @@ exports[`InCallView > rendering > renders 1`] = `
</svg>
</button>
<button
aria-labelledby="«rf»"
aria-labelledby="«ri»"
class="_button_vczzf_8 _has-icon_vczzf_57 _icon-only_vczzf_50"
data-kind="secondary"
data-size="lg"
@@ -322,7 +322,7 @@ exports[`InCallView > rendering > renders 1`] = `
</svg>
</button>
<button
aria-labelledby="«rk»"
aria-labelledby="«rn»"
class="_button_vczzf_8 endCall _has-icon_vczzf_57 _icon-only_vczzf_50 _destructive_vczzf_107"
data-kind="primary"
data-size="lg"
@@ -348,7 +348,7 @@ exports[`InCallView > rendering > renders 1`] = `
class="toggle layout"
>
<input
aria-labelledby="«rp»"
aria-labelledby="«rs»"
name="layout"
type="radio"
value="spotlight"
@@ -366,7 +366,7 @@ exports[`InCallView > rendering > renders 1`] = `
/>
</svg>
<input
aria-labelledby="«ru»"
aria-labelledby="«r11»"
checked=""
name="layout"
type="radio"

View File

@@ -75,6 +75,7 @@ function renderWithMockClient(
<ClientContextProvider
value={{
state: "valid",
oidcClientConfig: null,
disconnected: false,
supportedFeatures: {
reactions: true,

View File

@@ -6,6 +6,7 @@ Please see LICENSE in the repository root for full details.
*/
import { test, vi, onTestFinished, it } from "vitest";
import EventEmitter from "events";
import {
BehaviorSubject,
combineLatest,
@@ -17,16 +18,18 @@ import {
of,
switchMap,
} from "rxjs";
import { type MatrixClient } from "matrix-js-sdk";
import { ClientEvent, SyncState, type MatrixClient } from "matrix-js-sdk";
import {
ConnectionState,
type LocalParticipant,
type LocalTrackPublication,
type Participant,
ParticipantEvent,
type RemoteParticipant,
} from "livekit-client";
import * as ComponentsCore from "@livekit/components-core";
import {
Status,
type CallMembership,
type MatrixRTCSession,
} from "matrix-js-sdk/lib/matrixrtc";
@@ -94,6 +97,11 @@ vi.mock("rxjs", async (importOriginal) => ({
vi.mock("@livekit/components-core");
const yesNo = {
y: true,
n: false,
};
const daveRtcMember = mockRtcMembership("@dave:example.org", "DDDD");
const carol = local;
@@ -226,33 +234,56 @@ function summarizeLayout$(l$: Observable<Layout>): Observable<LayoutSummary> {
);
}
interface CallViewModelInputs {
remoteParticipants$: Behavior<RemoteParticipant[]>;
rtcMembers$: Behavior<Partial<CallMembership>[]>;
connectionState$: Observable<ECConnectionState>;
speaking: Map<Participant, Observable<boolean>>;
mediaDevices: MediaDevices;
initialSyncState: SyncState;
}
function withCallViewModel(
remoteParticipants$: Behavior<RemoteParticipant[]>,
rtcMembers$: Behavior<Partial<CallMembership>[]>,
connectionState$: Observable<ECConnectionState>,
speaking: Map<Participant, Observable<boolean>>,
mediaDevices: MediaDevices,
{
remoteParticipants$ = constant([]),
rtcMembers$ = constant([localRtcMember]),
connectionState$ = of(ConnectionState.Connected),
speaking = new Map(),
mediaDevices = mockMediaDevices({}),
initialSyncState = SyncState.Syncing,
}: Partial<CallViewModelInputs>,
continuation: (
vm: CallViewModel,
rtcSession: MockRTCSession,
subjects: { raisedHands$: BehaviorSubject<Record<string, RaisedHandInfo>> },
setSyncState: (value: SyncState) => void,
) => void,
options: CallViewModelOptions = {
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
autoLeaveWhenOthersLeft: false,
},
): void {
let syncState = initialSyncState;
const setSyncState = (value: SyncState): void => {
const prev = syncState;
syncState = value;
room.client.emit(ClientEvent.Sync, value, prev);
};
const room = mockMatrixRoom({
client: {
getUserId: () => localRtcMember.sender,
getDeviceId: () => localRtcMember.deviceId,
} as Partial<MatrixClient> as MatrixClient,
client: new (class extends EventEmitter {
public getUserId(): string | undefined {
return localRtcMember.sender;
}
public getDeviceId(): string {
return localRtcMember.deviceId;
}
public getSyncState(): SyncState {
return syncState;
}
})() as Partial<MatrixClient> as MatrixClient,
getMember: (userId) => roomMembers.get(userId) ?? null,
});
const rtcSession = new MockRTCSession(
room,
localRtcMember,
[],
).withMemberships(rtcMembers$);
const rtcSession = new MockRTCSession(room, []).withMemberships(rtcMembers$);
const participantsSpy = vi
.spyOn(ComponentsCore, "connectedParticipantsObserver")
.mockReturnValue(remoteParticipants$);
@@ -279,7 +310,7 @@ function withCallViewModel(
.spyOn(ComponentsCore, "roomEventSelector")
.mockImplementation((room, eventType) => of());
const liveKitRoom = mockLivekitRoom(
const livekitRoom = mockLivekitRoom(
{ localParticipant },
{ remoteParticipants$ },
);
@@ -288,7 +319,8 @@ function withCallViewModel(
const vm = new CallViewModel(
rtcSession as unknown as MatrixRTCSession,
liveKitRoom,
room,
livekitRoom,
mediaDevices,
options,
connectionState$,
@@ -304,7 +336,7 @@ function withCallViewModel(
roomEventSelectorSpy!.mockRestore();
});
continuation(vm, { raisedHands$: raisedHands$ });
continuation(vm, rtcSession, { raisedHands$: raisedHands$ }, setSyncState);
}
test("participants are retained during a focus switch", () => {
@@ -317,17 +349,17 @@ test("participants are retained during a focus switch", () => {
const expectedLayoutMarbles = " a";
withCallViewModel(
behavior(participantInputMarbles, {
a: [aliceParticipant, bobParticipant],
b: [],
}),
constant([aliceRtcMember, bobRtcMember]),
behavior(connectionInputMarbles, {
c: ConnectionState.Connected,
s: ECAddonConnectionState.ECSwitchingFocus,
}),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: behavior(participantInputMarbles, {
a: [aliceParticipant, bobParticipant],
b: [],
}),
rtcMembers$: constant([localRtcMember, aliceRtcMember, bobRtcMember]),
connectionState$: behavior(connectionInputMarbles, {
c: ConnectionState.Connected,
s: ECAddonConnectionState.ECSwitchingFocus,
}),
},
(vm) => {
expectObservable(summarizeLayout$(vm.layout$)).toBe(
expectedLayoutMarbles,
@@ -358,16 +390,15 @@ test("screen sharing activates spotlight layout", () => {
const expectedLayoutMarbles = " abcdaefeg";
const expectedShowSpeakingMarbles = "y----nyny";
withCallViewModel(
behavior(participantInputMarbles, {
a: [aliceParticipant, bobParticipant],
b: [aliceSharingScreen, bobParticipant],
c: [aliceSharingScreen, bobSharingScreen],
d: [aliceParticipant, bobSharingScreen],
}),
constant([aliceRtcMember, bobRtcMember]),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: behavior(participantInputMarbles, {
a: [aliceParticipant, bobParticipant],
b: [aliceSharingScreen, bobParticipant],
c: [aliceSharingScreen, bobSharingScreen],
d: [aliceParticipant, bobSharingScreen],
}),
rtcMembers$: constant([localRtcMember, aliceRtcMember, bobRtcMember]),
},
(vm) => {
schedule(modeInputMarbles, {
s: () => vm.setGridMode("spotlight"),
@@ -419,10 +450,7 @@ test("screen sharing activates spotlight layout", () => {
);
expectObservable(vm.showSpeakingIndicators$).toBe(
expectedShowSpeakingMarbles,
{
y: true,
n: false,
},
yesNo,
);
},
);
@@ -443,24 +471,24 @@ test("participants stay in the same order unless to appear/disappear", () => {
const expectedLayoutMarbles = " a 1999ms b 1999ms a 57999ms c 1999ms a";
withCallViewModel(
constant([aliceParticipant, bobParticipant, daveParticipant]),
constant([aliceRtcMember, bobRtcMember, daveRtcMember]),
of(ConnectionState.Connected),
new Map([
[
{
remoteParticipants$: constant([
aliceParticipant,
behavior(aSpeakingInputMarbles, { y: true, n: false }),
],
[
bobParticipant,
behavior(bSpeakingInputMarbles, { y: true, n: false }),
],
[
daveParticipant,
behavior(dSpeakingInputMarbles, { y: true, n: false }),
],
]),
mockMediaDevices({}),
]),
rtcMembers$: constant([
localRtcMember,
aliceRtcMember,
bobRtcMember,
daveRtcMember,
]),
speaking: new Map([
[aliceParticipant, behavior(aSpeakingInputMarbles, yesNo)],
[bobParticipant, behavior(bSpeakingInputMarbles, yesNo)],
[daveParticipant, behavior(dSpeakingInputMarbles, yesNo)],
]),
},
(vm) => {
schedule(visibilityInputMarbles, {
a: () => {
@@ -510,20 +538,23 @@ test("participants adjust order when space becomes constrained", () => {
const expectedLayoutMarbles = " a-b";
withCallViewModel(
constant([aliceParticipant, bobParticipant, daveParticipant]),
constant([aliceRtcMember, bobRtcMember, daveRtcMember]),
of(ConnectionState.Connected),
new Map([
[
{
remoteParticipants$: constant([
aliceParticipant,
bobParticipant,
behavior(bSpeakingInputMarbles, { y: true, n: false }),
],
[
daveParticipant,
behavior(dSpeakingInputMarbles, { y: true, n: false }),
],
]),
mockMediaDevices({}),
]),
rtcMembers$: constant([
localRtcMember,
aliceRtcMember,
bobRtcMember,
daveRtcMember,
]),
speaking: new Map([
[bobParticipant, behavior(bSpeakingInputMarbles, yesNo)],
[daveParticipant, behavior(dSpeakingInputMarbles, yesNo)],
]),
},
(vm) => {
let setVisibleTiles: ((value: number) => void) | null = null;
vm.layout$.subscribe((layout) => {
@@ -569,24 +600,24 @@ test("spotlight speakers swap places", () => {
const expectedLayoutMarbles = "abcd";
withCallViewModel(
constant([aliceParticipant, bobParticipant, daveParticipant]),
constant([aliceRtcMember, bobRtcMember, daveRtcMember]),
of(ConnectionState.Connected),
new Map([
[
{
remoteParticipants$: constant([
aliceParticipant,
behavior(aSpeakingInputMarbles, { y: true, n: false }),
],
[
bobParticipant,
behavior(bSpeakingInputMarbles, { y: true, n: false }),
],
[
daveParticipant,
behavior(dSpeakingInputMarbles, { y: true, n: false }),
],
]),
mockMediaDevices({}),
]),
rtcMembers$: constant([
localRtcMember,
aliceRtcMember,
bobRtcMember,
daveRtcMember,
]),
speaking: new Map([
[aliceParticipant, behavior(aSpeakingInputMarbles, yesNo)],
[bobParticipant, behavior(bSpeakingInputMarbles, yesNo)],
[daveParticipant, behavior(dSpeakingInputMarbles, yesNo)],
]),
},
(vm) => {
schedule(modeInputMarbles, { s: () => vm.setGridMode("spotlight") });
@@ -628,11 +659,10 @@ test("layout enters picture-in-picture mode when requested", () => {
const expectedLayoutMarbles = " aba";
withCallViewModel(
constant([aliceParticipant, bobParticipant]),
constant([aliceRtcMember, bobRtcMember]),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: constant([aliceParticipant, bobParticipant]),
rtcMembers$: constant([localRtcMember, aliceRtcMember, bobRtcMember]),
},
(vm) => {
schedule(pipControlInputMarbles, {
e: () => window.controls.enablePip(),
@@ -670,11 +700,10 @@ test("spotlight remembers whether it's expanded", () => {
const expectedLayoutMarbles = "abcbada";
withCallViewModel(
constant([aliceParticipant, bobParticipant]),
constant([aliceRtcMember, bobRtcMember]),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: constant([aliceParticipant, bobParticipant]),
rtcMembers$: constant([localRtcMember, aliceRtcMember, bobRtcMember]),
},
(vm) => {
schedule(modeInputMarbles, {
s: () => vm.setGridMode("spotlight"),
@@ -727,23 +756,22 @@ test("participants must have a MatrixRTCSession to be visible", () => {
const expectedLayoutMarbles = "a-bc-b";
withCallViewModel(
behavior(scenarioInputMarbles, {
a: [],
b: [bobParticipant],
c: [aliceParticipant, bobParticipant],
d: [aliceParticipant, daveParticipant, bobParticipant],
e: [aliceParticipant, daveParticipant, bobSharingScreen],
}),
behavior(scenarioInputMarbles, {
a: [],
b: [],
c: [aliceRtcMember],
d: [aliceRtcMember, daveRtcMember],
e: [aliceRtcMember, daveRtcMember],
}),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: behavior(scenarioInputMarbles, {
a: [],
b: [bobParticipant],
c: [aliceParticipant, bobParticipant],
d: [aliceParticipant, daveParticipant, bobParticipant],
e: [aliceParticipant, daveParticipant, bobSharingScreen],
}),
rtcMembers$: behavior(scenarioInputMarbles, {
a: [localRtcMember],
b: [localRtcMember],
c: [localRtcMember, aliceRtcMember],
d: [localRtcMember, aliceRtcMember, daveRtcMember],
e: [localRtcMember, aliceRtcMember, daveRtcMember],
}),
},
(vm) => {
vm.setGridMode("grid");
expectObservable(summarizeLayout$(vm.layout$)).toBe(
@@ -780,15 +808,14 @@ test("shows participants without MatrixRTCSession when enabled in settings", ()
const expectedLayoutMarbles = "abc";
withCallViewModel(
behavior(scenarioInputMarbles, {
a: [],
b: [aliceParticipant],
c: [aliceParticipant, bobParticipant],
}),
constant([]), // No one joins the MatrixRTC session
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: behavior(scenarioInputMarbles, {
a: [],
b: [aliceParticipant],
c: [aliceParticipant, bobParticipant],
}),
rtcMembers$: constant([localRtcMember]), // No one else joins the MatrixRTC session
},
(vm) => {
vm.setGridMode("grid");
expectObservable(summarizeLayout$(vm.layout$)).toBe(
@@ -827,16 +854,14 @@ it("should show at least one tile per MatrixRTCSession", () => {
const expectedLayoutMarbles = "abcd";
withCallViewModel(
constant([]),
behavior(scenarioInputMarbles, {
a: [],
b: [aliceRtcMember],
c: [aliceRtcMember, daveRtcMember],
d: [daveRtcMember],
}),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
rtcMembers$: behavior(scenarioInputMarbles, {
a: [localRtcMember],
b: [localRtcMember, aliceRtcMember],
c: [localRtcMember, aliceRtcMember, daveRtcMember],
d: [localRtcMember, daveRtcMember],
}),
},
(vm) => {
vm.setGridMode("grid");
expectObservable(summarizeLayout$(vm.layout$)).toBe(
@@ -875,17 +900,20 @@ test("should disambiguate users with the same displayname", () => {
const expectedLayoutMarbles = "abcde";
withCallViewModel(
constant([]),
behavior(scenarioInputMarbles, {
a: [],
b: [aliceRtcMember],
c: [aliceRtcMember, aliceDoppelgangerRtcMember],
d: [aliceRtcMember, aliceDoppelgangerRtcMember, bobRtcMember],
e: [aliceDoppelgangerRtcMember, bobRtcMember],
}),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
rtcMembers$: behavior(scenarioInputMarbles, {
a: [localRtcMember],
b: [localRtcMember, aliceRtcMember],
c: [localRtcMember, aliceRtcMember, aliceDoppelgangerRtcMember],
d: [
localRtcMember,
aliceRtcMember,
aliceDoppelgangerRtcMember,
bobRtcMember,
],
e: [localRtcMember, aliceDoppelgangerRtcMember, bobRtcMember],
}),
},
(vm) => {
expectObservable(vm.memberDisplaynames$).toBe(expectedLayoutMarbles, {
// Carol has no displayname - So userId is used.
@@ -925,14 +953,12 @@ test("should disambiguate users with invisible characters", () => {
const expectedLayoutMarbles = "ab";
withCallViewModel(
constant([]),
behavior(scenarioInputMarbles, {
a: [],
b: [bobRtcMember, bobZeroWidthSpaceRtcMember],
}),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
rtcMembers$: behavior(scenarioInputMarbles, {
a: [localRtcMember],
b: [localRtcMember, bobRtcMember, bobZeroWidthSpaceRtcMember],
}),
},
(vm) => {
expectObservable(vm.memberDisplaynames$).toBe(expectedLayoutMarbles, {
// Carol has no displayname - So userId is used.
@@ -958,14 +984,12 @@ test("should strip RTL characters from displayname", () => {
const expectedLayoutMarbles = "ab";
withCallViewModel(
constant([]),
behavior(scenarioInputMarbles, {
a: [],
b: [daveRtcMember, daveRTLRtcMember],
}),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
rtcMembers$: behavior(scenarioInputMarbles, {
a: [localRtcMember],
b: [localRtcMember, daveRtcMember, daveRTLRtcMember],
}),
},
(vm) => {
expectObservable(vm.memberDisplaynames$).toBe(expectedLayoutMarbles, {
// Carol has no displayname - So userId is used.
@@ -985,17 +1009,16 @@ test("should strip RTL characters from displayname", () => {
});
it("should rank raised hands above video feeds and below speakers and presenters", () => {
withTestScheduler(({ schedule, expectObservable, behavior }) => {
withTestScheduler(({ schedule, expectObservable }) => {
// There should always be one tile for each MatrixRTCSession
const expectedLayoutMarbles = "ab";
withCallViewModel(
constant([aliceParticipant, bobParticipant]),
constant([aliceRtcMember, bobRtcMember]),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
(vm, { raisedHands$ }) => {
{
remoteParticipants$: constant([aliceParticipant, bobParticipant]),
rtcMembers$: constant([localRtcMember, aliceRtcMember, bobRtcMember]),
},
(vm, _rtcSession, { raisedHands$ }) => {
schedule("ab", {
a: () => {
// We imagine that only two tiles (the first two) will be visible on screen at a time
@@ -1076,10 +1099,10 @@ function rtcMemberJoinLeave$(
) => Observable<CallMembership[]>,
): Observable<CallMembership[]> {
return hot("a-b-c-d", {
a: [], // Start empty
b: [aliceRtcMember], // Alice joins
c: [aliceRtcMember], // Alice still there
d: [], // Alice leaves
a: [localRtcMember], // Start empty
b: [localRtcMember, aliceRtcMember], // Alice joins
c: [localRtcMember, aliceRtcMember], // Alice still there
d: [localRtcMember], // Alice leaves
});
}
@@ -1087,11 +1110,7 @@ test("allOthersLeft$ emits only when someone joined and then all others left", (
withTestScheduler(({ hot, expectObservable, scope }) => {
// Test scenario 1: No one ever joins - should only emit initial false and never emit again
withCallViewModel(
scope.behavior(nooneEverThere$(hot), []),
scope.behavior(nooneEverThere$(hot), []),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{ remoteParticipants$: scope.behavior(nooneEverThere$(hot), []) },
(vm) => {
expectObservable(vm.allOthersLeft$).toBe("n------", { n: false });
},
@@ -1102,11 +1121,10 @@ test("allOthersLeft$ emits only when someone joined and then all others left", (
test("allOthersLeft$ emits true when someone joined and then all others left", () => {
withTestScheduler(({ hot, expectObservable, scope }) => {
withCallViewModel(
scope.behavior(participantJoinLeave$(hot), []),
scope.behavior(rtcMemberJoinLeave$(hot), []),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: scope.behavior(participantJoinLeave$(hot), []),
rtcMembers$: scope.behavior(rtcMemberJoinLeave$(hot), []),
},
(vm) => {
expectObservable(vm.allOthersLeft$).toBe(
"n-----u", // false initially, then at frame 6: true then false emissions in same frame
@@ -1120,11 +1138,10 @@ test("allOthersLeft$ emits true when someone joined and then all others left", (
test("autoLeaveWhenOthersLeft$ emits only when autoLeaveWhenOthersLeft option is enabled", () => {
withTestScheduler(({ hot, expectObservable, scope }) => {
withCallViewModel(
scope.behavior(participantJoinLeave$(hot), []),
scope.behavior(rtcMemberJoinLeave$(hot), []),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: scope.behavior(participantJoinLeave$(hot), []),
rtcMembers$: scope.behavior(rtcMemberJoinLeave$(hot), []),
},
(vm) => {
expectObservable(vm.autoLeaveWhenOthersLeft$).toBe(
"------e", // false initially, then at frame 6: true then false emissions in same frame
@@ -1142,11 +1159,10 @@ test("autoLeaveWhenOthersLeft$ emits only when autoLeaveWhenOthersLeft option is
test("autoLeaveWhenOthersLeft$ never emits autoLeaveWhenOthersLeft option is enabled but no-one is there", () => {
withTestScheduler(({ hot, expectObservable, scope }) => {
withCallViewModel(
scope.behavior(nooneEverThere$(hot), []),
scope.behavior(nooneEverThere$(hot), []),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: scope.behavior(nooneEverThere$(hot), []),
rtcMembers$: scope.behavior(nooneEverThere$(hot), []),
},
(vm) => {
expectObservable(vm.autoLeaveWhenOthersLeft$).toBe("-------");
},
@@ -1161,11 +1177,10 @@ test("autoLeaveWhenOthersLeft$ never emits autoLeaveWhenOthersLeft option is ena
test("autoLeaveWhenOthersLeft$ doesn't emit when autoLeaveWhenOthersLeft option is disabled and all others left", () => {
withTestScheduler(({ hot, expectObservable, scope }) => {
withCallViewModel(
scope.behavior(participantJoinLeave$(hot), []),
scope.behavior(rtcMemberJoinLeave$(hot), []),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: scope.behavior(participantJoinLeave$(hot), []),
rtcMembers$: scope.behavior(rtcMemberJoinLeave$(hot), []),
},
(vm) => {
expectObservable(vm.autoLeaveWhenOthersLeft$).toBe("-------");
},
@@ -1180,27 +1195,26 @@ test("autoLeaveWhenOthersLeft$ doesn't emit when autoLeaveWhenOthersLeft option
test("autoLeaveWhenOthersLeft$ doesn't emits when autoLeaveWhenOthersLeft option is enabled and all others left", () => {
withTestScheduler(({ hot, expectObservable, scope }) => {
withCallViewModel(
scope.behavior(
hot("a-b-c-d", {
a: [], // Alone
b: [aliceParticipant], // Alice joins
c: [aliceParticipant],
d: [], // Local joins with a second device
}),
[], //Alice leaves
),
scope.behavior(
hot("a-b-c-d", {
a: [localRtcMember], // Start empty
b: [localRtcMember, aliceRtcMember], // Alice joins
c: [localRtcMember, aliceRtcMember, localRtcMemberDevice2], // Alice still there
d: [localRtcMember, localRtcMemberDevice2], // The second Alice leaves
}),
[],
),
of(ConnectionState.Connected),
new Map(),
mockMediaDevices({}),
{
remoteParticipants$: scope.behavior(
hot("a-b-c-d", {
a: [], // Alone
b: [aliceParticipant], // Alice joins
c: [aliceParticipant],
d: [], // Local joins with a second device
}),
[], //Alice leaves
),
rtcMembers$: scope.behavior(
hot("a-b-c-d", {
a: [localRtcMember], // Start empty
b: [localRtcMember, aliceRtcMember], // Alice joins
c: [localRtcMember, aliceRtcMember, localRtcMemberDevice2], // Alice still there
d: [localRtcMember, localRtcMemberDevice2], // The second Alice leaves
}),
[],
),
},
(vm) => {
expectObservable(vm.autoLeaveWhenOthersLeft$).toBe("------e", {
e: undefined,
@@ -1234,25 +1248,91 @@ test("audio output changes when toggling earpiece mode", () => {
const expectedEarpieceModeMarbles = "n-yn";
const expectedTargetStateMarbles = " sese";
withCallViewModel({ mediaDevices: devices }, (vm) => {
schedule(toggleInputMarbles, {
a: () => getValue(vm.audioOutputSwitcher$)?.switch(),
});
expectObservable(vm.earpieceMode$).toBe(
expectedEarpieceModeMarbles,
yesNo,
);
expectObservable(
vm.audioOutputSwitcher$.pipe(map((switcher) => switcher?.targetOutput)),
).toBe(expectedTargetStateMarbles, { s: "speaker", e: "earpiece" });
});
});
});
test("media tracks are paused while reconnecting to MatrixRTC", () => {
withTestScheduler(({ schedule, expectObservable }) => {
const trackRunning$ = new BehaviorSubject(true);
const originalPublications = localParticipant.trackPublications;
localParticipant.trackPublications = new Map([
[
"video",
{
track: new (class {
public get isUpstreamPaused(): boolean {
return !trackRunning$.value;
}
public async pauseUpstream(): Promise<void> {
trackRunning$.next(false);
return Promise.resolve();
}
public async resumeUpstream(): Promise<void> {
trackRunning$.next(true);
return Promise.resolve();
}
})(),
} as unknown as LocalTrackPublication,
],
]);
onTestFinished(() => {
localParticipant.trackPublications = originalPublications;
});
// There are three indicators that the client might be disconnected from
// MatrixRTC: whether the sync loop is connected, whether the membership is
// present in local room state, and whether the membership manager thinks
// we've hit the timeout for the delayed leave event. Let's test all
// combinations of these conditions.
const syncingMarbles = " nyny----n--y";
const membershipStatusMarbles = " y---ny-n-yn-y";
const probablyLeftMarbles = " n-----y-ny---n";
const expectedReconnectingMarbles = "n-ynyny------n";
const expectedTrackRunningMarbles = "nynynyn------y";
withCallViewModel(
constant([]),
constant([]),
of(ConnectionState.Connected),
new Map(),
devices,
(vm) => {
schedule(toggleInputMarbles, {
a: () => getValue(vm.audioOutputSwitcher$)?.switch(),
{ initialSyncState: SyncState.Reconnecting },
(vm, rtcSession, _subjects, setSyncState) => {
schedule(syncingMarbles, {
y: () => setSyncState(SyncState.Syncing),
n: () => setSyncState(SyncState.Reconnecting),
});
expectObservable(vm.earpieceMode$).toBe(expectedEarpieceModeMarbles, {
n: false,
y: true,
schedule(membershipStatusMarbles, {
y: () => {
rtcSession.membershipStatus = Status.Connected;
},
n: () => {
rtcSession.membershipStatus = Status.Reconnecting;
},
});
expectObservable(
vm.audioOutputSwitcher$.pipe(
map((switcher) => switcher?.targetOutput),
),
).toBe(expectedTargetStateMarbles, { s: "speaker", e: "earpiece" });
schedule(probablyLeftMarbles, {
y: () => {
rtcSession.probablyLeft = true;
},
n: () => {
rtcSession.probablyLeft = false;
},
});
expectObservable(vm.reconnecting$).toBe(
expectedReconnectingMarbles,
yesNo,
);
expectObservable(trackRunning$).toBe(
expectedTrackRunningMarbles,
yesNo,
);
},
);
});

View File

@@ -11,12 +11,19 @@ import {
observeParticipantMedia,
} from "@livekit/components-core";
import {
ConnectionState,
type Room as LivekitRoom,
type LocalParticipant,
ParticipantEvent,
type RemoteParticipant,
} from "livekit-client";
import { RoomStateEvent, type Room, type RoomMember } from "matrix-js-sdk";
import {
ClientEvent,
RoomStateEvent,
SyncState,
type Room as MatrixRoom,
type RoomMember,
} from "matrix-js-sdk";
import {
BehaviorSubject,
EMPTY,
@@ -48,6 +55,8 @@ import {
type CallMembership,
type MatrixRTCSession,
MatrixRTCSessionEvent,
MembershipManagerEvent,
Status,
} from "matrix-js-sdk/lib/matrixrtc";
import { ViewModel } from "./ViewModel";
@@ -62,7 +71,12 @@ import {
ScreenShareViewModel,
type UserMediaViewModel,
} from "./MediaViewModel";
import { accumulate, finalizeValue } from "../utils/observable";
import {
accumulate,
and$,
finalizeValue,
pauseWhen,
} from "../utils/observable";
import { ObservableScope } from "./ObservableScope";
import {
duplicateTiles,
@@ -261,6 +275,7 @@ class UserMedia {
encryptionSystem: EncryptionSystem,
livekitRoom: LivekitRoom,
mediaDevices: MediaDevices,
pretendToBeDisconnected$: Behavior<boolean>,
displayname$: Observable<string>,
handRaised$: Observable<Date | null>,
reaction$: Observable<ReactionOption | null>,
@@ -288,6 +303,7 @@ class UserMedia {
>,
encryptionSystem,
livekitRoom,
pretendToBeDisconnected$,
this.scope.behavior(displayname$),
this.scope.behavior(handRaised$),
this.scope.behavior(reaction$),
@@ -341,7 +357,8 @@ class ScreenShare {
member: RoomMember | undefined,
participant: LocalParticipant | RemoteParticipant,
encryptionSystem: EncryptionSystem,
liveKitRoom: LivekitRoom,
livekitRoom: LivekitRoom,
pretendToBeDisconnected$: Behavior<boolean>,
displayName$: Observable<string>,
) {
this.participant$ = new BehaviorSubject(participant);
@@ -351,7 +368,8 @@ class ScreenShare {
member,
this.participant$.asObservable(),
encryptionSystem,
liveKitRoom,
livekitRoom,
pretendToBeDisconnected$,
this.scope.behavior(displayName$),
participant.isLocal,
);
@@ -367,7 +385,7 @@ type MediaItem = UserMedia | ScreenShare;
function getRoomMemberFromRtcMember(
rtcMember: CallMembership,
room: Room,
room: MatrixRoom,
): { id: string; member: RoomMember | undefined } {
// WARN! This is not exactly the sender but the user defined in the state key.
// This will be available once we change to the new "member as object" format in the MatrixRTC object.
@@ -389,6 +407,79 @@ function getRoomMemberFromRtcMember(
// TODO: Move wayyyy more business logic from the call and lobby views into here
export class CallViewModel extends ViewModel {
private readonly userId = this.matrixRoom.client.getUserId();
private readonly matrixConnected$ = this.scope.behavior(
// To consider ourselves connected to MatrixRTC, we check the following:
and$(
// The client is connected to the sync loop
(
fromEvent(this.matrixRoom.client, ClientEvent.Sync) as Observable<
[SyncState]
>
).pipe(
startWith([this.matrixRoom.client.getSyncState()]),
map(([state]) => state === SyncState.Syncing),
),
// Room state observed by session says we're connected
fromEvent(
this.matrixRTCSession,
MembershipManagerEvent.StatusChanged,
).pipe(
startWith(null),
map(() => this.matrixRTCSession.membershipStatus === Status.Connected),
),
// Also watch out for warnings that we've likely hit a timeout and our
// delayed leave event is being sent (this condition is here because it
// provides an earlier warning than the sync loop timeout, and we wouldn't
// see the actual leave event until we reconnect to the sync loop)
fromEvent(
this.matrixRTCSession,
MembershipManagerEvent.ProbablyLeft,
).pipe(
startWith(null),
map(() => this.matrixRTCSession.probablyLeft !== true),
),
),
);
private readonly connected$ = this.scope.behavior(
and$(
this.matrixConnected$,
this.livekitConnectionState$.pipe(
map((state) => state === ConnectionState.Connected),
),
),
);
/**
* Whether we should tell the user that we're reconnecting to the call.
*/
public readonly reconnecting$ = this.scope.behavior(
this.connected$.pipe(
// We are reconnecting if we previously had some successful initial
// connection but are now disconnected
scan(
({ connectedPreviously, reconnecting }, connectedNow) => ({
connectedPreviously: connectedPreviously || connectedNow,
reconnecting: connectedPreviously && !connectedNow,
}),
{ connectedPreviously: false, reconnecting: false },
),
map(({ reconnecting }) => reconnecting),
),
);
/**
* Whether various media/event sources should pretend to be disconnected from
* all network input, even if their connection still technically works.
*/
// We do this when the app is in the 'reconnecting' state, because it might be
// that the LiveKit connection is still functional while the homeserver is
// down, for example, and we want to avoid making people worry that the app is
// in a split-brained state.
private readonly pretendToBeDisconnected$ = this.reconnecting$;
/**
* The raw list of RemoteParticipants as reported by LiveKit
*/
@@ -403,7 +494,7 @@ export class CallViewModel extends ViewModel {
private readonly remoteParticipantHolds$ = this.scope.behavior<
RemoteParticipant[][]
>(
this.connectionState$.pipe(
this.livekitConnectionState$.pipe(
withLatestFrom(this.rawRemoteParticipants$),
mergeMap(([s, ps]) => {
// Whenever we switch focuses, we should retain all the previous
@@ -416,7 +507,7 @@ export class CallViewModel extends ViewModel {
// Wait for time to pass and the connection state to have changed
forkJoin([
timer(POST_FOCUS_PARTICIPANT_UPDATE_DELAY_MS),
this.connectionState$.pipe(
this.livekitConnectionState$.pipe(
filter((s) => s !== ECAddonConnectionState.ECSwitchingFocus),
take(1),
),
@@ -440,74 +531,80 @@ export class CallViewModel extends ViewModel {
/**
* The RemoteParticipants including those that are being "held" on the screen
*/
private readonly remoteParticipants$ = this.scope.behavior<
RemoteParticipant[]
>(
combineLatest(
[this.rawRemoteParticipants$, this.remoteParticipantHolds$],
(raw, holds) => {
const result = [...raw];
const resultIds = new Set(result.map((p) => p.identity));
private readonly remoteParticipants$ = this.scope
.behavior<RemoteParticipant[]>(
combineLatest(
[this.rawRemoteParticipants$, this.remoteParticipantHolds$],
(raw, holds) => {
const result = [...raw];
const resultIds = new Set(result.map((p) => p.identity));
// Incorporate the held participants into the list
for (const hold of holds) {
for (const p of hold) {
if (!resultIds.has(p.identity)) {
result.push(p);
resultIds.add(p.identity);
// Incorporate the held participants into the list
for (const hold of holds) {
for (const p of hold) {
if (!resultIds.has(p.identity)) {
result.push(p);
resultIds.add(p.identity);
}
}
}
}
return result;
},
),
);
private readonly memberships$: Observable<CallMembership[]> = merge(
// Handle call membership changes.
fromEvent(this.matrixRTCSession, MatrixRTCSessionEvent.MembershipsChanged),
// Handle room membership changes (and displayname updates)
fromEvent(this.matrixRTCSession.room, RoomStateEvent.Members),
).pipe(
startWith(this.matrixRTCSession.memberships),
map(() => {
return this.matrixRTCSession.memberships;
}),
);
return result;
},
),
)
.pipe(pauseWhen(this.pretendToBeDisconnected$));
/**
* Displaynames for each member of the call. This will disambiguate
* any displaynames that clashes with another member. Only members
* joined to the call are considered here.
*/
public readonly memberDisplaynames$ = this.memberships$.pipe(
map((memberships) => {
const displaynameMap = new Map<string, string>();
const { room } = this.matrixRTCSession;
// It turns out that doing the disambiguation above is rather expensive on Safari (10x slower
// than on Chrome/Firefox). This means it is important that we multicast the result so that we
// don't do this work more times than we need to. This is achieved by converting to a behavior:
public readonly memberDisplaynames$ = this.scope.behavior(
merge(
// Handle call membership changes.
fromEvent(
this.matrixRTCSession,
MatrixRTCSessionEvent.MembershipsChanged,
),
// Handle room membership changes (and displayname updates)
fromEvent(this.matrixRoom, RoomStateEvent.Members),
).pipe(
startWith(null),
map(() => {
const memberships = this.matrixRTCSession.memberships;
const displaynameMap = new Map<string, string>();
const room = this.matrixRoom;
// We only consider RTC members for disambiguation as they are the only visible members.
for (const rtcMember of memberships) {
const matrixIdentifier = `${rtcMember.sender}:${rtcMember.deviceId}`;
const { member } = getRoomMemberFromRtcMember(rtcMember, room);
if (!member) {
logger.error("Could not find member for media id:", matrixIdentifier);
continue;
// We only consider RTC members for disambiguation as they are the only visible members.
for (const rtcMember of memberships) {
const matrixIdentifier = `${rtcMember.sender}:${rtcMember.deviceId}`;
const { member } = getRoomMemberFromRtcMember(rtcMember, room);
if (!member) {
logger.error(
"Could not find member for media id:",
matrixIdentifier,
);
continue;
}
const disambiguate = shouldDisambiguate(member, memberships, room);
displaynameMap.set(
matrixIdentifier,
calculateDisplayName(member, disambiguate),
);
}
const disambiguate = shouldDisambiguate(member, memberships, room);
displaynameMap.set(
matrixIdentifier,
calculateDisplayName(member, disambiguate),
);
}
return displaynameMap;
}),
// It turns out that doing the disambiguation above is rather expensive on Safari (10x slower
// than on Chrome/Firefox). This means it is important that we multicast the result so that we
// don't do this work more times than we need to. This is achieved by converting to a behavior:
return displaynameMap;
}),
pauseWhen(this.pretendToBeDisconnected$),
),
);
public readonly handsRaised$ = this.scope.behavior(this.handsRaisedSubject$);
public readonly handsRaised$ = this.scope.behavior(
this.handsRaisedSubject$.pipe(pauseWhen(this.pretendToBeDisconnected$)),
);
public readonly reactions$ = this.scope.behavior(
this.reactionsSubject$.pipe(
@@ -519,6 +616,7 @@ export class CallViewModel extends ViewModel {
]),
),
),
pauseWhen(this.pretendToBeDisconnected$),
),
);
@@ -536,7 +634,7 @@ export class CallViewModel extends ViewModel {
fromEvent(
this.matrixRTCSession,
MatrixRTCSessionEvent.MembershipsChanged,
).pipe(startWith(null)),
).pipe(startWith(null), pauseWhen(this.pretendToBeDisconnected$)),
showNonMemberTiles.value$,
]).pipe(
scan(
@@ -552,7 +650,7 @@ export class CallViewModel extends ViewModel {
) => {
const newItems = new Map(
function* (this: CallViewModel): Iterable<[string, MediaItem]> {
const room = this.matrixRTCSession.room;
const room = this.matrixRoom;
// m.rtc.members are the basis for calculating what is visible in the call
for (const rtcMember of this.matrixRTCSession.memberships) {
const { member, id: livekitParticipantId } =
@@ -604,6 +702,7 @@ export class CallViewModel extends ViewModel {
this.options.encryptionSystem,
this.livekitRoom,
this.mediaDevices,
this.pretendToBeDisconnected$,
this.memberDisplaynames$.pipe(
map((m) => m.get(matrixIdentifier) ?? "[👻]"),
),
@@ -627,6 +726,7 @@ export class CallViewModel extends ViewModel {
participant,
this.options.encryptionSystem,
this.livekitRoom,
this.pretendToBeDisconnected$,
this.memberDisplaynames$.pipe(
map((m) => m.get(matrixIdentifier) ?? "[👻]"),
),
@@ -669,6 +769,7 @@ export class CallViewModel extends ViewModel {
this.options.encryptionSystem,
this.livekitRoom,
this.mediaDevices,
this.pretendToBeDisconnected$,
this.memberDisplaynames$.pipe(
map(
(m) => m.get(participant.identity) ?? "[👻]",
@@ -772,12 +873,13 @@ export class CallViewModel extends ViewModel {
public readonly allOthersLeft$ = this.matrixUserChanges$.pipe(
map(({ userIds, leftUserIds }) => {
const userId = this.matrixRTCSession.room.client.getUserId();
if (!userId) {
logger.warn("Could access client.getUserId to compute allOthersLeft");
if (!this.userId) {
logger.warn("Could not access user ID to compute allOthersLeft");
return false;
}
return userIds.size === 1 && userIds.has(userId) && leftUserIds.size > 0;
return (
userIds.size === 1 && userIds.has(this.userId) && leftUserIds.size > 0
);
}),
startWith(false),
distinctUntilChanged(),
@@ -889,7 +991,7 @@ export class CallViewModel extends ViewModel {
map((speaker) => (speaker ? [speaker] : [])),
);
}),
distinctUntilChanged(shallowEquals),
distinctUntilChanged<MediaViewModel[]>(shallowEquals),
),
);
@@ -1474,10 +1576,11 @@ export class CallViewModel extends ViewModel {
public constructor(
// A call is permanently tied to a single Matrix room and LiveKit room
private readonly matrixRTCSession: MatrixRTCSession,
private readonly matrixRoom: MatrixRoom,
private readonly livekitRoom: LivekitRoom,
private readonly mediaDevices: MediaDevices,
private readonly options: CallViewModelOptions,
private readonly connectionState$: Observable<ECConnectionState>,
private readonly livekitConnectionState$: Observable<ECConnectionState>,
private readonly handsRaisedSubject$: Observable<
Record<string, RaisedHandInfo>
>,
@@ -1486,5 +1589,51 @@ export class CallViewModel extends ViewModel {
>,
) {
super();
// Pause upstream of all local media tracks when we're disconnected from
// MatrixRTC, because it can be an unpleasant surprise for the app to say
// 'reconnecting' and yet still be transmitting your media to others.
// We use matrixConnected$ rather than reconnecting$ because we want to
// pause tracks during the initial joining sequence too until we're sure
// that our own media is displayed on screen.
this.matrixConnected$.pipe(this.scope.bind()).subscribe((connected) => {
const publications =
this.livekitRoom.localParticipant.trackPublications.values();
if (connected) {
for (const p of publications) {
if (p.track?.isUpstreamPaused === true) {
const kind = p.track.kind;
logger.log(
`Resumming ${kind} track (MatrixRTC connection present)`,
);
p.track
.resumeUpstream()
.catch((e) =>
logger.error(
`Failed to resume ${kind} track after MatrixRTC reconnection`,
e,
),
);
}
}
} else {
for (const p of publications) {
if (p.track?.isUpstreamPaused === false) {
const kind = p.track.kind;
logger.log(
`Pausing ${kind} track (uncertain MatrixRTC connection)`,
);
p.track
.pauseUpstream()
.catch((e) =>
logger.error(
`Failed to pause ${kind} track after entering uncertain MatrixRTC connection`,
e,
),
);
}
}
}
});
}
}

View File

@@ -361,10 +361,7 @@ export type UserMediaViewModel =
* Some participant's user media.
*/
abstract class BaseUserMediaViewModel extends BaseMediaViewModel {
/**
* Whether the participant is speaking.
*/
public readonly speaking$ = this.scope.behavior(
private readonly _speaking$ = this.scope.behavior(
this.participant$.pipe(
switchMap((p) =>
p
@@ -376,15 +373,27 @@ abstract class BaseUserMediaViewModel extends BaseMediaViewModel {
),
),
);
/**
* Whether the participant is speaking.
*/
// Getter backed by a private field so that subclasses can override it
public get speaking$(): Behavior<boolean> {
return this._speaking$;
}
/**
* Whether this participant is sending audio (i.e. is unmuted on their side).
*/
public readonly audioEnabled$: Behavior<boolean>;
private readonly _videoEnabled$: Behavior<boolean>;
/**
* Whether this participant is sending video.
*/
public readonly videoEnabled$: Behavior<boolean>;
// Getter backed by a private field so that subclasses can override it
public get videoEnabled$(): Behavior<boolean> {
return this._videoEnabled$;
}
private readonly _cropVideo$ = new BehaviorSubject(true);
/**
@@ -421,7 +430,7 @@ abstract class BaseUserMediaViewModel extends BaseMediaViewModel {
this.audioEnabled$ = this.scope.behavior(
media$.pipe(map((m) => m?.microphoneTrack?.isMuted === false)),
);
this.videoEnabled$ = this.scope.behavior(
this._videoEnabled$ = this.scope.behavior(
media$.pipe(map((m) => m?.cameraTrack?.isMuted === false)),
);
}
@@ -572,6 +581,12 @@ export class LocalUserMediaViewModel extends BaseUserMediaViewModel {
* A remote participant's user media.
*/
export class RemoteUserMediaViewModel extends BaseUserMediaViewModel {
// This private field is used to override the value from the superclass
private __speaking$: Behavior<boolean>;
public get speaking$(): Behavior<boolean> {
return this.__speaking$;
}
private readonly locallyMutedToggle$ = new Subject<void>();
private readonly localVolumeAdjustment$ = new Subject<number>();
private readonly localVolumeCommit$ = new Subject<void>();
@@ -611,6 +626,12 @@ export class RemoteUserMediaViewModel extends BaseUserMediaViewModel {
),
);
// This private field is used to override the value from the superclass
private __videoEnabled$: Behavior<boolean>;
public get videoEnabled$(): Behavior<boolean> {
return this.__videoEnabled$;
}
/**
* Whether this participant's audio is disabled.
*/
@@ -624,6 +645,7 @@ export class RemoteUserMediaViewModel extends BaseUserMediaViewModel {
participant$: Observable<RemoteParticipant | undefined>,
encryptionSystem: EncryptionSystem,
livekitRoom: LivekitRoom,
private readonly pretendToBeDisconnected$: Behavior<boolean>,
displayname$: Behavior<string>,
handRaised$: Behavior<Date | null>,
reaction$: Behavior<ReactionOption | null>,
@@ -639,11 +661,33 @@ export class RemoteUserMediaViewModel extends BaseUserMediaViewModel {
reaction$,
);
this.__speaking$ = this.scope.behavior(
pretendToBeDisconnected$.pipe(
switchMap((disconnected) =>
disconnected ? of(false) : super.speaking$,
),
),
);
this.__videoEnabled$ = this.scope.behavior(
pretendToBeDisconnected$.pipe(
switchMap((disconnected) =>
disconnected ? of(false) : super.videoEnabled$,
),
),
);
// Sync the local volume with LiveKit
combineLatest([
participant$,
this.localVolume$.pipe(this.scope.bind()),
]).subscribe(([p, volume]) => p && p.setVolume(volume));
// The local volume, taking into account whether we're supposed to pretend
// that the audio stream is disconnected (since we don't necessarily want
// that to modify the UI state).
this.pretendToBeDisconnected$.pipe(
switchMap((disconnected) => (disconnected ? of(0) : this.localVolume$)),
this.scope.bind(),
),
]).subscribe(([p, volume]) => p?.setVolume(volume));
}
public toggleLocallyMuted(): void {
@@ -683,12 +727,20 @@ export class RemoteUserMediaViewModel extends BaseUserMediaViewModel {
* Some participant's screen share media.
*/
export class ScreenShareViewModel extends BaseMediaViewModel {
/**
* Whether this screen share's video should be displayed.
*/
public readonly videoEnabled$ = this.scope.behavior(
this.pretendToBeDisconnected$.pipe(map((disconnected) => !disconnected)),
);
public constructor(
id: string,
member: RoomMember | undefined,
participant$: Observable<LocalParticipant | RemoteParticipant>,
encryptionSystem: EncryptionSystem,
livekitRoom: LivekitRoom,
private readonly pretendToBeDisconnected$: Behavior<boolean>,
displayname$: Behavior<string>,
public readonly local: boolean,
) {

View File

@@ -54,6 +54,7 @@ interface SpotlightItemBaseProps {
targetWidth: number;
targetHeight: number;
video: TrackReferenceOrPlaceholder | undefined;
videoEnabled: boolean;
member: RoomMember | undefined;
unencryptedWarning: boolean;
encryptionStatus: EncryptionStatus;
@@ -63,7 +64,6 @@ interface SpotlightItemBaseProps {
}
interface SpotlightUserMediaItemBaseProps extends SpotlightItemBaseProps {
videoEnabled: boolean;
videoFit: "contain" | "cover";
}
@@ -90,12 +90,10 @@ const SpotlightUserMediaItem: FC<SpotlightUserMediaItemProps> = ({
vm,
...props
}) => {
const videoEnabled = useBehavior(vm.videoEnabled$);
const cropVideo = useBehavior(vm.cropVideo$);
const baseProps: SpotlightUserMediaItemBaseProps &
RefAttributes<HTMLDivElement> = {
videoEnabled,
videoFit: cropVideo ? "cover" : "contain",
...props,
};
@@ -135,6 +133,7 @@ const SpotlightItem: FC<SpotlightItemProps> = ({
const ref = useMergedRefs(ourRef, theirRef);
const displayName = useBehavior(vm.displayName$);
const video = useBehavior(vm.video$);
const videoEnabled = useBehavior(vm.videoEnabled$);
const unencryptedWarning = useBehavior(vm.unencryptedWarning$);
const encryptionStatus = useBehavior(vm.encryptionStatus$);
@@ -160,6 +159,7 @@ const SpotlightItem: FC<SpotlightItemProps> = ({
targetWidth,
targetHeight,
video,
videoEnabled,
member: vm.member,
unencryptedWarning,
displayName,
@@ -169,7 +169,7 @@ const SpotlightItem: FC<SpotlightItemProps> = ({
};
return vm instanceof ScreenShareViewModel ? (
<MediaView videoEnabled videoFit="contain" mirror={false} {...baseProps} />
<MediaView videoFit="contain" mirror={false} {...baseProps} />
) : (
<SpotlightUserMediaItem vm={vm} {...baseProps} />
);

View File

@@ -0,0 +1,24 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { test } from "vitest";
import { withTestScheduler } from "./test";
import { pauseWhen } from "./observable";
test("pauseWhen", () => {
withTestScheduler(({ behavior, expectObservable }) => {
const inputMarbles = " abcdefgh-i-jk-";
const pauseMarbles = " n-y--n-yn-y--n";
const outputMarbles = "abc--fgh-i---k";
expectObservable(
behavior(inputMarbles).pipe(
pauseWhen(behavior(pauseMarbles, { y: true, n: false })),
),
).toBe(outputMarbles);
});
});

View File

@@ -7,16 +7,23 @@ Please see LICENSE in the repository root for full details.
import {
type Observable,
audit,
combineLatest,
concat,
defer,
filter,
finalize,
map,
of,
scan,
startWith,
takeWhile,
tap,
withLatestFrom,
} from "rxjs";
import { type Behavior } from "../state/Behavior";
const nothing = Symbol("nothing");
/**
@@ -86,3 +93,27 @@ export function getValue<T>(state$: Observable<T>): T {
if (value === nothing) throw new Error("Not a state Observable");
return value;
}
/**
* Creates an Observable that has a value of true whenever all its inputs are
* true.
*/
export function and$(...inputs: Observable<boolean>[]): Observable<boolean> {
return combineLatest(inputs, (...flags) => flags.every((flag) => flag));
}
/**
* RxJS operator that pauses all changes in the input value whenever a Behavior
* is true. When the Behavior returns to being false, the most recently
* suppressed change is emitted as the most recent value.
*/
export function pauseWhen<T>(pause$: Behavior<boolean>) {
return (value$: Observable<T>): Observable<T> =>
value$.pipe(
withLatestFrom(pause$),
audit(([, pause]) =>
pause ? pause$.pipe(filter((pause) => !pause)) : of(null),
),
map(([value]) => value),
);
}

View File

@@ -0,0 +1,96 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { completeAuthorizationCodeGrant, generateOidcAuthorizationUrl } from "matrix-js-sdk/src/oidc/authorize";
import { OidcClientConfig } from "matrix-js-sdk/src/matrix";
import { secureRandomString } from "matrix-js-sdk/src/randomstring";
import { type IdTokenClaims } from "oidc-client-ts";
import { getOidcCallbackUrl } from "./callbackUrl";
import { OidcClientError } from "./error";
/**
* TODO: Mostly yoinked from element-web, so move to SDK if possible
*
* Start OIDC authorization code flow
* Generates auth params, stores them in session storage and
* Navigates to configured authorization endpoint
* @param delegatedAuthConfig from discovery
* @param clientId this client's id as registered with configured issuer
* @param homeserverUrl target homeserver
* @returns Promise that resolves after we have navigated to auth endpoint
*/
export async function startOidcLogin(
delegatedAuthConfig: OidcClientConfig,
clientId: string,
homeserverUrl: string,
isRegistration?: boolean,
): Promise<void> {
const redirectUri = getOidcCallbackUrl().href;
const nonce = secureRandomString(10);
const prompt = isRegistration ? "create" : undefined;
const authorizationUrl = await generateOidcAuthorizationUrl({
metadata: delegatedAuthConfig,
redirectUri,
clientId,
homeserverUrl,
identityServerUrl: undefined,
nonce,
prompt,
});
window.location.href = authorizationUrl;
}
// TODO: Mostly yoinked from element-web, so move to SDK if possible
type CompleteOidcLoginResponse = {
// url of the homeserver selected during login
homeserverUrl: string;
// accessToken gained from OIDC token issuer
accessToken: string;
// refreshToken gained from OIDC token issuer, when falsy token cannot be refreshed
refreshToken?: string;
// idToken gained from OIDC token issuer
idToken: string;
// this client's id as registered with the OIDC issuer
clientId: string;
// issuer used during authentication
issuer: string;
// claims of the given access token; used during token refresh to validate new tokens
idTokenClaims: IdTokenClaims;
};
/**
* TODO: Mostly yoinked from element-web, so move to SDK if possible
*
* Attempt to complete authorization code flow to get an access token
* @param queryParams the query-parameters extracted from the real query-string of the starting URI.
* @returns Promise that resolves with a CompleteOidcLoginResponse when login was successful
* @throws When we failed to get a valid access token
*/
export async function completeOidcLogin(queryParams: URLSearchParams): Promise<CompleteOidcLoginResponse> {
const code = queryParams.get("code");
const state = queryParams.get("state");
if (!code || !state) {
throw new Error(OidcClientError.InvalidQueryParameters);
}
const { homeserverUrl, tokenResponse, idTokenClaims, oidcClientSettings } =
await completeAuthorizationCodeGrant(code, state);
return {
homeserverUrl,
accessToken: tokenResponse.access_token,
refreshToken: tokenResponse.refresh_token,
idToken: tokenResponse.id_token,
clientId: oidcClientSettings.clientId,
issuer: oidcClientSettings.issuer,
idTokenClaims,
};
}

View File

@@ -0,0 +1,14 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* The URL to return to after a successful OIDC authentication
*/
export function getOidcCallbackUrl(): URL {
// TODO: save the path somewhere
return new URL("after-login", window.location.origin);
}

View File

@@ -0,0 +1,30 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { MatrixClient, MatrixError, OidcClientConfig } from "matrix-js-sdk";
import { Config } from "../../config/Config";
import { HomeserverMisconfigError } from "../../RichError";
export async function getAuthMetadata(): Promise<OidcClientConfig | null> {
const baseUrl = Config.defaultHomeserverUrl(); // TODO: Make this configurable
if (!baseUrl) {
throw new Error("No homeserver URL configured");
}
const tempClient = new MatrixClient({ baseUrl });
try {
return await tempClient.getAuthMetadata();
} catch (e) {
if (e instanceof MatrixError && e.httpStatus === 404 && e.errcode === "M_UNRECOGNIZED") {
// 404 M_UNRECOGNIZED means the server does not support OIDC
return null;
} else {
throw new HomeserverMisconfigError(e instanceof Error ? e.message : undefined);
}
}
}

16
src/utils/oidc/error.ts Normal file
View File

@@ -0,0 +1,16 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* TODO: Yoinked from element-web, so move to SDK if possible
*
* Errors thrown by EC during OIDC native flow authentication.
* Intended to be logged, not read by users.
*/
export enum OidcClientError {
InvalidQueryParameters = "Invalid query parameters for OIDC native login. `code` and `state` are required.",
}

View File

@@ -0,0 +1,21 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
// TODO: File yoinked from element-web, so move to SDK if possible
import { type OidcClientConfig } from "matrix-js-sdk/src/matrix";
/**
* Check the create prompt is supported by the OP, if so, we can do a registration flow
* https://openid.net/specs/openid-connect-prompt-create-1_0.html
* @param delegatedAuthConfig config as returned from discovery
* @returns whether user registration is supported
*/
export function isUserRegistrationSupported(delegatedAuthConfig: OidcClientConfig): boolean {
const supportedPrompts = delegatedAuthConfig.prompt_values_supported;
return Array.isArray(supportedPrompts) && supportedPrompts?.includes("create");
}

View File

@@ -0,0 +1,62 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { logger } from "matrix-js-sdk/src/logger";
import { registerOidcClient, type OidcClientConfig } from "matrix-js-sdk";
import { ConfigOptions } from "../../config/ConfigOptions";
import { Config } from "../../config/Config";
import { getOidcCallbackUrl } from "./callbackUrl";
/**
* TODO: Mostly yoinked from element-web, so move to SDK if possible
*
* Get the statically configured clientId for the issuer
* @param issuer delegated auth OIDC issuer
* @param staticOidcClients static client config from config.json
* @returns clientId if found, otherwise undefined
*/
function getStaticOidcClientId(
issuer: string,
staticOidcClients?: ConfigOptions["oidc_static_clients"],
): string | undefined {
// static_oidc_clients are configured with a trailing slash
const issuerWithTrailingSlash = issuer.endsWith("/") ? issuer : issuer + "/";
return staticOidcClients?.[issuerWithTrailingSlash]?.client_id;
}
/**
* TODO: Mostly yoinked from element-web, so move to SDK if possible
*
* Get the statically configured clientId for an OIDC OP
* @param delegatedAuthConfig Auth config from OP
* @returns resolves with clientId
* @throws if no clientId is found
*/
export async function getOidcClientId(
delegatedAuthConfig: OidcClientConfig,
): Promise<string> {
const config = Config.get();
const staticClientId = getStaticOidcClientId(delegatedAuthConfig.issuer, config.oidc_static_clients);
if (staticClientId) {
logger.debug(`Using static clientId for issuer ${delegatedAuthConfig.issuer}`);
return staticClientId;
}
return await registerOidcClient(
delegatedAuthConfig,
{
clientName: config.oidc_metadata?.client_name ?? "Element Call",
clientUri: config.oidc_metadata?.client_uri ?? window.location.origin,
logoUri: config.oidc_metadata?.logo_uri,
redirectUris: [getOidcCallbackUrl().href],
applicationType: "web",
contacts: config.oidc_metadata?.contacts,
tosUri: config.oidc_metadata?.tos_uri,
policyUri: config.oidc_metadata?.policy_uri,
},
);
}

View File

@@ -14,8 +14,13 @@ import { BehaviorSubject, of } from "rxjs";
import { vitest } from "vitest";
import { type RelationsContainer } from "matrix-js-sdk/lib/models/relations-container";
import EventEmitter from "events";
import {
type RoomMember,
type MatrixClient,
type Room,
SyncState,
} from "matrix-js-sdk";
import type { RoomMember, MatrixClient } from "matrix-js-sdk";
import { E2eeType } from "../e2ee/e2eeType";
import { CallViewModel } from "../state/CallViewModel";
import {
@@ -34,10 +39,11 @@ import { type RaisedHandInfo, type ReactionInfo } from "../reactions";
export function getBasicRTCSession(
members: RoomMember[],
initialRemoteRtcMemberships: CallMembership[] = [aliceRtcMember],
initialRtcMemberships: CallMembership[] = [localRtcMember, aliceRtcMember],
): {
rtcSession: MockRTCSession;
remoteRtcMemberships$: BehaviorSubject<CallMembership[]>;
matrixRoom: Room;
rtcMemberships$: BehaviorSubject<CallMembership[]>;
} {
const matrixRoomId = "!myRoomId:example.com";
const matrixRoomMembers = new Map(members.map((p) => [p.userId, p]));
@@ -51,6 +57,7 @@ export function getBasicRTCSession(
client: {
getUserId: () => localRtcMember.sender,
getDeviceId: () => localRtcMember.deviceId,
getSyncState: () => SyncState.Syncing,
sendEvent: vitest.fn().mockResolvedValue({ event_id: "$fake:event" }),
redactEvent: vitest.fn().mockResolvedValue({ event_id: "$fake:event" }),
decryptEventIfNeeded: vitest.fn().mockResolvedValue(undefined),
@@ -91,52 +98,53 @@ export function getBasicRTCSession(
),
});
const remoteRtcMemberships$ = new BehaviorSubject<CallMembership[]>(
initialRemoteRtcMemberships,
const rtcMemberships$ = new BehaviorSubject<CallMembership[]>(
initialRtcMemberships,
);
const rtcSession = new MockRTCSession(
matrixRoom,
localRtcMember,
).withMemberships(remoteRtcMemberships$);
const rtcSession = new MockRTCSession(matrixRoom).withMemberships(
rtcMemberships$,
);
return {
rtcSession,
remoteRtcMemberships$,
matrixRoom,
rtcMemberships$,
};
}
/**
* Construct a basic CallViewModel to test components that make use of it.
* @param members
* @param initialRemoteRtcMemberships
* @param initialRtcMemberships
* @returns
*/
export function getBasicCallViewModelEnvironment(
members: RoomMember[],
initialRemoteRtcMemberships: CallMembership[] = [aliceRtcMember],
initialRtcMemberships: CallMembership[] = [localRtcMember, aliceRtcMember],
): {
vm: CallViewModel;
remoteRtcMemberships$: BehaviorSubject<CallMembership[]>;
rtcMemberships$: BehaviorSubject<CallMembership[]>;
rtcSession: MockRTCSession;
handRaisedSubject$: BehaviorSubject<Record<string, RaisedHandInfo>>;
reactionsSubject$: BehaviorSubject<Record<string, ReactionInfo>>;
} {
const { rtcSession, remoteRtcMemberships$ } = getBasicRTCSession(
const { rtcSession, matrixRoom, rtcMemberships$ } = getBasicRTCSession(
members,
initialRemoteRtcMemberships,
initialRtcMemberships,
);
const handRaisedSubject$ = new BehaviorSubject({});
const reactionsSubject$ = new BehaviorSubject({});
const remoteParticipants$ = of([aliceParticipant]);
const liveKitRoom = mockLivekitRoom(
const livekitRoom = mockLivekitRoom(
{ localParticipant },
{ remoteParticipants$ },
);
const vm = new CallViewModel(
rtcSession as unknown as MatrixRTCSession,
liveKitRoom,
matrixRoom,
livekitRoom,
mockMediaDevices({}),
{
encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
@@ -147,7 +155,7 @@ export function getBasicCallViewModelEnvironment(
);
return {
vm,
remoteRtcMemberships$,
rtcMemberships$,
rtcSession,
handRaisedSubject$: handRaisedSubject$,
reactionsSubject$: reactionsSubject$,

View File

@@ -19,8 +19,11 @@ import {
type Focus,
MatrixRTCSessionEvent,
type MatrixRTCSessionEventHandlerMap,
MembershipManagerEvent,
type SessionMembershipData,
Status,
} from "matrix-js-sdk/lib/matrixrtc";
import { type MembershipManagerEventHandlerMap } from "matrix-js-sdk/lib/matrixrtc/IMembershipManager";
import {
type LocalParticipant,
type LocalTrackPublication,
@@ -233,6 +236,7 @@ export function mockLocalParticipant(
): LocalParticipant {
return {
isLocal: true,
trackPublications: new Map(),
getTrackPublication: () =>
({}) as Partial<LocalTrackPublication> as LocalTrackPublication,
...mockEmitter(),
@@ -295,6 +299,7 @@ export async function withRemoteMedia(
kind: E2eeType.PER_PARTICIPANT,
},
mockLivekitRoom({}, { remoteParticipants$: of([remoteParticipant]) }),
constant(false),
constant(roomMember.rawDisplayName ?? "nodisplayname"),
constant(null),
constant(null),
@@ -316,8 +321,10 @@ export function mockConfig(config: Partial<ResolvedConfigOptions> = {}): void {
}
export class MockRTCSession extends TypedEventEmitter<
MatrixRTCSessionEvent | RoomAndToDeviceEvents,
MatrixRTCSessionEventHandlerMap & RoomAndToDeviceEventsHandlerMap
MatrixRTCSessionEvent | RoomAndToDeviceEvents | MembershipManagerEvent,
MatrixRTCSessionEventHandlerMap &
RoomAndToDeviceEventsHandlerMap &
MembershipManagerEventHandlerMap
> {
public readonly statistics = {
counters: {},
@@ -327,7 +334,6 @@ export class MockRTCSession extends TypedEventEmitter<
public constructor(
public readonly room: Room,
private localMembership: CallMembership,
public memberships: CallMembership[] = [],
) {
super();
@@ -343,14 +349,37 @@ export class MockRTCSession extends TypedEventEmitter<
): MockRTCSession {
rtcMembers$.subscribe((m) => {
const old = this.memberships;
// always prepend the local participant
const updated = [this.localMembership, ...(m as CallMembership[])];
this.memberships = updated;
this.emit(MatrixRTCSessionEvent.MembershipsChanged, old, updated);
this.memberships = m as CallMembership[];
this.emit(
MatrixRTCSessionEvent.MembershipsChanged,
old,
this.memberships,
);
});
return this;
}
private _membershipStatus = Status.Connected;
public get membershipStatus(): Status {
return this._membershipStatus;
}
public set membershipStatus(value: Status) {
const prev = this._membershipStatus;
this._membershipStatus = value;
if (value !== prev)
this.emit(MembershipManagerEvent.StatusChanged, prev, value);
}
private _probablyLeft = false;
public get probablyLeft(): boolean {
return this._probablyLeft;
}
public set probablyLeft(value: boolean) {
const prev = this._probablyLeft;
this._probablyLeft = value;
if (value !== prev) this.emit(MembershipManagerEvent.ProbablyLeft, value);
}
}
export const mockTrack = (identity: string): TrackReference =>

View File

@@ -120,8 +120,8 @@ export const widget = ((): WidgetHelpers | null => {
const sendState = [
userId, // Legacy call membership events
`_${userId}_${deviceId}`, // Session membership events
`${userId}_${deviceId}`, // The above with no leading underscore, for room versions whose auth rules allow it
`_${userId}_${deviceId}_m.call`, // Session membership events
`${userId}_${deviceId}_m.call`, // The above with no leading underscore, for room versions whose auth rules allow it
].map((stateKey) => ({
eventType: EventType.GroupCallMemberPrefix,
stateKey,

View File

@@ -1,7 +1,15 @@
/*
Copyright 2025 New Vector Ltd.
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { defineConfig, mergeConfig } from "vite";
import fullConfig from "./vite.config";
import generateFile from "vite-plugin-generate-file";
import fullConfig from "./vite.config";
const base = "./";
// Config for embedded deployments (possibly hosted under a non-root path)

View File

@@ -5,7 +5,12 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { defineConfig, loadEnv, searchForWorkspaceRoot } from "vite";
import {
loadEnv,
searchForWorkspaceRoot,
type ConfigEnv,
type UserConfig,
} from "vite";
import svgrPlugin from "vite-plugin-svgr";
import { createHtmlPlugin } from "vite-plugin-html";
import { codecovVitePlugin } from "@codecov/vite-plugin";
@@ -15,10 +20,14 @@ import { realpathSync } from "fs";
import * as fs from "node:fs";
// https://vitejs.dev/config/
export default defineConfig(({ mode, packageType }) => {
// Modified type helper from defineConfig to allow for packageType (see defineConfig from vite)
export default ({
mode,
packageType,
}: ConfigEnv & { packageType?: "full" | "embedded" }): UserConfig => {
const env = loadEnv(mode, process.cwd());
// Environment variables with the VITE_ prefix are accessible at runtime.
// So, we set this to allow for build/package specific behaviour.
// So, we set this to allow for build/package specific behavior.
// In future we might be able to do what is needed via code splitting at
// build time.
process.env.VITE_PACKAGE = packageType ?? "full";
@@ -93,7 +102,7 @@ export default defineConfig(({ mode, packageType }) => {
sourcemap: true,
rollupOptions: {
output: {
assetFileNames: ({ originalFileNames }) => {
assetFileNames: ({ originalFileNames }): string => {
if (originalFileNames) {
for (const name of originalFileNames) {
// Custom asset name for locales to include the locale code in the filename
@@ -143,4 +152,4 @@ export default defineConfig(({ mode, packageType }) => {
exclude: ["@matrix-org/matrix-sdk-crypto-wasm"],
},
};
});
};

View File

@@ -1,5 +1,6 @@
import { defineConfig, mergeConfig } from "vitest/config";
import viteConfig from "./vite.config.js";
import viteConfig from "./vite.config";
export default defineConfig((configEnv) =>
mergeConfig(

View File

@@ -10046,8 +10046,8 @@ __metadata:
linkType: hard
"livekit-client@npm:^2.13.0":
version: 2.15.4
resolution: "livekit-client@npm:2.15.4"
version: 2.15.6
resolution: "livekit-client@npm:2.15.6"
dependencies:
"@livekit/mutex": "npm:1.1.1"
"@livekit/protocol": "npm:1.39.3"
@@ -10060,7 +10060,7 @@ __metadata:
webrtc-adapter: "npm:^9.0.1"
peerDependencies:
"@types/dom-mediacapture-record": ^1
checksum: 10c0/f12a3b604aed8e075791c60a75c9eea5467e5a0bab48c2eb23216ec357e89a50199274121e0330b61f946d0be9556058824ce1b832003e722786e54ba33017a6
checksum: 10c0/f1ab6cdf2b85647036e9de906734c1394dac497da0bd879a29d0c587c437ada262021478fcef24df99b3489a39d97fe67ab33de0785ed0a63335da2fef577192
languageName: node
linkType: hard
@@ -10279,8 +10279,8 @@ __metadata:
linkType: hard
"matrix-js-sdk@github:matrix-org/matrix-js-sdk#head=develop":
version: 37.12.0
resolution: "matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk.git#commit=c4e1e0723e3674725f8ed9cda5be7a7efe304e29"
version: 37.13.0
resolution: "matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk.git#commit=3a33c658bbcb8ce8791ec066db899f2571f5c52f"
dependencies:
"@babel/runtime": "npm:^7.12.5"
"@matrix-org/matrix-sdk-crypto-wasm": "npm:^15.1.0"
@@ -10296,7 +10296,7 @@ __metadata:
sdp-transform: "npm:^2.14.1"
unhomoglyph: "npm:^1.0.6"
uuid: "npm:11"
checksum: 10c0/e8f97b90ded9e46bdd803dde9861e178c76741999c224cbc869ad38695f300702405e5c365e270663b90e2a317b646bbf2b9e9bd7d51ffce36e6d2ca3ab29ed9
checksum: 10c0/1db0d39cfbe4f1c69c8acda0ea7580a4819fc47a7d4bff057382e33e72d9a610f8c03043a6c00bc647dfdc2815aa643c69d25022fb759342a92b77e1841524f1
languageName: node
linkType: hard