diff --git a/README.md b/README.md index 510b7c76..8ca7fa96 100644 --- a/README.md +++ b/README.md @@ -143,7 +143,8 @@ via the `org.matrix.msc4143.rtc_foci` key, e.g.: where the format for MatrixRTC using LiveKit backend is defined in [MSC4195](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md). In the example above Matrix clients do discover a focus of type `livekit` which -points them to a Matrix LiveKit JWT Auth Service via `livekit_service_url`. +points them to a [MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service) +via `livekit_service_url`. ### Backend Selection @@ -154,9 +155,9 @@ points them to a Matrix LiveKit JWT Auth Service via `livekit_service_url`. the **first participant who joined the call** defines via the `foci_preferred` key in their `org.matrix.msc3401.call.member` which actual MatrixRTC backend will be used for this call. -- During the actual call join flow, the **LiveKit JWT Auth Service** provides - the client with the **LiveKit SFU WebSocket URL** and an **access JWT token** - in order to exchange media via WebRTC. +- During the actual call join flow, the **[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service)** + provides the client with the **LiveKit SFU WebSocket URL** and an + **access JWT token** in order to exchange media via WebRTC. The example below illustrates how backend selection works across **Matrix federation**, using the setup from sites A, B, and C. It demonstrates backend @@ -208,7 +209,7 @@ A docker compose file `dev-backend-docker-compose.yml` is provided to start the whole stack of components which is required for a local development environment: - Minimum Synapse Setup (servername: `synapse.m.localhost`) -- LiveKit Authorization Service (Note requires Federation API and hence a TLS reverse proxy) +- MatrixRTC Authorization Service (Note requires Federation API and hence a TLS reverse proxy) - Minimum LiveKit SFU Setup using dev defaults for config - Redis db for completeness - Minimum `localhost` Certificate Authority (CA) for Transport Layer Security (TLS) diff --git a/docs/Federated_Setup.drawio.png b/docs/Federated_Setup.drawio.png index 6f0a4967..9813df37 100644 Binary files a/docs/Federated_Setup.drawio.png and b/docs/Federated_Setup.drawio.png differ diff --git a/docs/MSC4195_setup.drawio.png b/docs/MSC4195_setup.drawio.png index e9c7c8a7..80e47fe2 100644 Binary files a/docs/MSC4195_setup.drawio.png and b/docs/MSC4195_setup.drawio.png differ diff --git a/docs/element_call_standalone.drawio.png b/docs/element_call_standalone.drawio.png index 1e105ef4..ab541fbd 100644 Binary files a/docs/element_call_standalone.drawio.png and b/docs/element_call_standalone.drawio.png differ diff --git a/docs/element_call_widget.drawio.png b/docs/element_call_widget.drawio.png index 72a4e1de..61a36e42 100644 Binary files a/docs/element_call_widget.drawio.png and b/docs/element_call_widget.drawio.png differ diff --git a/docs/self-hosting.md b/docs/self-hosting.md index d7a1fbdf..4a663c3f 100644 --- a/docs/self-hosting.md +++ b/docs/self-hosting.md @@ -64,7 +64,7 @@ rc_delayed_event_mgmt: ``` As a prerequisite for the -[Matrix LiveKit JWT auth service](https://github.com/element-hq/lk-jwt-service) +[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service) make sure that your Synapse server has either a `federation` or `openid` [listener configured](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#listeners). @@ -77,7 +77,7 @@ required for each site deployment. As depicted above in the `example.com` site deployment, Element Call requires a [Livekit SFU](https://github.com/livekit/livekit) alongside a -[Matrix Livekit JWT auth service](https://github.com/element-hq/lk-jwt-service) +[MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service) to implement [MSC4195: MatrixRTC using LiveKit backend](https://github.com/hughns/matrix-spec-proposals/blob/hughns/matrixrtc-livekit/proposals/4195-matrixrtc-livekit.md). @@ -89,7 +89,7 @@ the example above, this results in: | Service | Endpoint | Example | | -------- | ------- | ------- | | [Livekit SFU](https://github.com/livekit/livekit) WebSocket signalling connection | `/livekit/sfu` | `matrix-rtc.example.com/livekit/sfu` | -| [Matrix Livekit JWT auth service](https://github.com/element-hq/lk-jwt-service) | `/livekit/jwt` | `matrix-rtc.example.com/livekit/jwt` | +| [MatrixRTC Authorization Service](https://github.com/element-hq/lk-jwt-service) | `/livekit/jwt` | `matrix-rtc.example.com/livekit/jwt` | Using Nginx, you can achieve this by: @@ -102,7 +102,7 @@ server { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; - # JWT Service running at port 8080 + # MatrixRTC Authorization Service running at port 8080 proxy_pass http://localhost:8080/; } diff --git a/locales/cs/app.json b/locales/cs/app.json index f988c336..f307bf6b 100644 --- a/locales/cs/app.json +++ b/locales/cs/app.json @@ -82,7 +82,7 @@ "error": { "call_is_not_supported": "Volání není podporováno", "call_not_found": "Volání nebylo nalezeno", - "call_not_found_description": "<0>Zdá se, že tento odkaz nepatří k žádnému existujícímu volání. Zkontrolujte, zda máte správný odkaz, nebo <1>vytvořte nový.", + "call_not_found_description": "<0>Zdá se, že tento odkaz nepatří k žádnému existujícímu hovoru. Zkontrolujte, zda máte správný odkaz, nebo<2> vytvořte nový.", "connection_lost": "Spojení ztraceno", "connection_lost_description": "Hovor byl přerušen.", "e2ee_unsupported": "Nekompatibilní prohlížeč", @@ -105,6 +105,11 @@ "knock_reject_heading": "Přístup odepřen", "reason": "Důvod" }, + "handset": { + "overlay_back_button": "Zpět do režimu reproduktoru", + "overlay_description": "Funguje pouze při používání aplikace", + "overlay_title": "Režim sluchátka" + }, "hangup_button_label": "Ukončit hovor", "header_label": "Domov Element Call", "header_participants_label": "Účastníci", @@ -177,6 +182,8 @@ "change_device_button": "Změnit zvukové zařízení", "default": "Výchozí", "default_named": "Výchozí <2> ({{name}}) ", + "handset": "Sluchátko", + "loudspeaker": "Reproduktor", "microphone": "Mikrofon", "microphone_numbered": "Mikrofon {{n}}", "speaker": "Reproduktor", diff --git a/locales/da/app.json b/locales/da/app.json index 5142ba08..7708551f 100644 --- a/locales/da/app.json +++ b/locales/da/app.json @@ -94,6 +94,8 @@ "matrix_rtc_focus_missing": "Serveren er ikke konfigureret til at arbejde med {{brand}}{{domain}}. Kontakt venligst din serveradministrator (domæne:{{domain}}, fejlkode: {{ errorCode }}).", "open_elsewhere": "Åbnet i en anden fane", "open_elsewhere_description": "{{brand}} er blevet åbnet i en anden fane. Hvis det ikke lyder rigtigt, kan du prøve at genindlæse siden.", + "room_creation_restricted": "Kunne ikke oprette opkald", + "room_creation_restricted_description": "Oprettelse af opkald er muligvis begrænset til autoriserede brugere. Prøv igen senere, eller kontakt din serveradministrator, hvis problemet fortsætter.", "unexpected_ec_error": "Der opstod en uventet fejl (<0>Fejlkode: <1> {{ errorCode }}). Kontakt venligst din serveradministrator." }, "group_call_loader": { @@ -105,6 +107,11 @@ "knock_reject_heading": "Adgang nægtet", "reason": "Årsag: {{reason}}" }, + "handset": { + "overlay_back_button": "Tilbage til højttalertilstand", + "overlay_description": "Virker kun, når du bruger appen", + "overlay_title": "Telefon-højtaler" + }, "hangup_button_label": "Afslut opkald", "header_label": "Element Ring hjem", "header_participants_label": "Deltagere", @@ -176,6 +183,7 @@ "change_device_button": "Skift lydenhed", "default": "Standard", "default_named": "Standard <2>({{name}})", + "handset": "Telefon", "loudspeaker": "Højttaler", "microphone": "Mikrofon", "microphone_numbered": "Mikrofon {{n}}", diff --git a/locales/de/app.json b/locales/de/app.json index fd6d2017..bb6328e7 100644 --- a/locales/de/app.json +++ b/locales/de/app.json @@ -55,6 +55,7 @@ "profile": "Profil", "reaction": "Reaktion", "reactions": "Reaktionen", + "reconnecting": "Verbindung wird wiederhergestellt...", "settings": "Einstellungen", "unencrypted": "Nicht verschlüsselt", "username": "Benutzername", diff --git a/locales/en/app.json b/locales/en/app.json index d375b629..007e372a 100644 --- a/locales/en/app.json +++ b/locales/en/app.json @@ -55,6 +55,7 @@ "profile": "Profile", "reaction": "Reaction", "reactions": "Reactions", + "reconnecting": "Reconnecting…", "settings": "Settings", "unencrypted": "Not encrypted", "username": "Username", diff --git a/locales/sv/app.json b/locales/sv/app.json index f8424225..bf0b742f 100644 --- a/locales/sv/app.json +++ b/locales/sv/app.json @@ -82,7 +82,7 @@ "error": { "call_is_not_supported": "Call stöds inte", "call_not_found": "Samtal hittades inte", - "call_not_found_description": "<0>Den länken verkar inte tillhöra något befintligt samtal. Kontrollera att du har rätt länk, eller <1>skapa en ny.", + "call_not_found_description": "<0>Den länken verkar inte tillhöra något befintligt samtal. Kontrollera att du har rätt länk, eller <2>skapa en ny.", "connection_lost": "Anslutning förlorad", "connection_lost_description": "Du kopplades bort från samtalet.", "e2ee_unsupported": "Inkompatibel webbläsare", @@ -94,6 +94,8 @@ "matrix_rtc_focus_missing": "Servern är inte konfigurerad för att fungera med {{brand}}. Vänligen kontakta serveradministratören (Domän: {{domain}}, Felkod: {{ errorCode }}).", "open_elsewhere": "Öppnades i en annan flik", "open_elsewhere_description": "{{brand}} har öppnats i en annan flik. Om det inte låter rätt, pröva att ladda om sidan.", + "room_creation_restricted": "Misslyckades att skapa samtal", + "room_creation_restricted_description": "Samtalsskapande kan vara begränsat till endast behöriga användare. Försök igen senare eller kontakta serveradministratören om problemet kvarstår.", "unexpected_ec_error": "Ett oväntat fel inträffade (<0>Felkod: <1>{{ errorCode }}). Kontakta din serveradministratör." }, "group_call_loader": { @@ -105,6 +107,11 @@ "knock_reject_heading": "Inte tillåten att gå med", "reason": "Anledning" }, + "handset": { + "overlay_back_button": "Tillbaka till högtalarläge", + "overlay_description": "Fungerar bara när appen används", + "overlay_title": "Telefonläge" + }, "hangup_button_label": "Avsluta samtal", "header_label": "Element Call Hem", "header_participants_label": "Deltagare", @@ -176,6 +183,8 @@ "change_device_button": "Byt ljudenhet", "default": "Förval", "default_named": "Förval <2>({{name}})", + "handset": "Telefonlur", + "loudspeaker": "Högtalare", "microphone": "Mikrofon", "microphone_numbered": "Mikrofon {{n}}", "speaker": "Högtalare", diff --git a/src/App.tsx b/src/App.tsx index 6d7d1e1e..b87f587c 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -24,7 +24,6 @@ import { RegisterPage } from "./auth/RegisterPage"; import { RoomPage } from "./room/RoomPage"; import { ClientProvider } from "./ClientContext"; import { ErrorPage, LoadingPage } from "./FullScreenView"; -import { DisconnectedBanner } from "./DisconnectedBanner"; import { Initializer } from "./initializer"; import { widget } from "./widget"; import { useTheme } from "./useTheme"; @@ -86,7 +85,6 @@ export const App: FC = ({ vm }) => { } > - } /> } /> diff --git a/src/AppBar.tsx b/src/AppBar.tsx index e70bb50d..aaa7565e 100644 --- a/src/AppBar.tsx +++ b/src/AppBar.tsx @@ -61,7 +61,11 @@ export const AppBar: FC = ({ children }) => { style={{ display: hidden ? "none" : "block" }} className={styles.bar} > -
+
diff --git a/src/Header.tsx b/src/Header.tsx index 577410f8..cffc3402 100644 --- a/src/Header.tsx +++ b/src/Header.tsx @@ -17,27 +17,38 @@ import Logo from "./icons/Logo.svg?react"; import { Avatar, Size } from "./Avatar"; import { EncryptionLock } from "./room/EncryptionLock"; import { useMediaQuery } from "./useMediaQuery"; +import { DisconnectedBanner } from "./DisconnectedBanner"; interface HeaderProps extends HTMLAttributes { ref?: Ref; children: ReactNode; className?: string; + /** + * Whether the header should display an informational banner whenever the + * client is disconnected from the homeserver. + * @default true + */ + disconnectedBanner?: boolean; } export const Header: FC = ({ ref, children, className, + disconnectedBanner = true, ...rest }) => { return ( -
- {children} -
+ <> +
+ {children} +
+ {disconnectedBanner && } + ); }; diff --git a/src/Toast.tsx b/src/Toast.tsx index ada5b29c..105572c8 100644 --- a/src/Toast.tsx +++ b/src/Toast.tsx @@ -45,6 +45,12 @@ interface Props { * A supporting icon to display within the toast. */ Icon?: ComponentType>; + /** + * Whether the toast should be portaled into the root of the document (rather + * than rendered in-place within the component tree). + * @default true + */ + portal?: boolean; } /** @@ -56,6 +62,7 @@ export const Toast: FC = ({ autoDismiss, children, Icon, + portal = true, }) => { const onOpenChange = useCallback( (open: boolean) => { @@ -71,29 +78,33 @@ export const Toast: FC = ({ } }, [open, autoDismiss, onDismiss]); + const content = ( + <> + + + + + + {children} + + + {Icon && } + + + + ); + return ( - - - - - - - {children} - - - {Icon && } - - - + {portal ? {content} : content} ); }; diff --git a/src/button/Button.tsx b/src/button/Button.tsx index 4caa9c4f..c11c92dd 100644 --- a/src/button/Button.tsx +++ b/src/button/Button.tsx @@ -16,7 +16,6 @@ import { EndCallIcon, ShareScreenSolidIcon, SettingsSolidIcon, - SwitchCameraSolidIcon, } from "@vector-im/compound-design-tokens/assets/web/icons"; import styles from "./Button.module.css"; @@ -67,23 +66,6 @@ export const VideoButton: FC = ({ muted, ...props }) => { ); }; -export const SwitchCameraButton: FC> = ( - props, -) => { - const { t } = useTranslation(); - - return ( - - - - ); -}; - interface ShareScreenButtonProps extends ComponentPropsWithoutRef<"button"> { enabled: boolean; } diff --git a/src/grid/TileWrapper.tsx b/src/grid/TileWrapper.tsx index 9e58fd7c..1bed08da 100644 --- a/src/grid/TileWrapper.tsx +++ b/src/grid/TileWrapper.tsx @@ -61,7 +61,12 @@ const TileWrapper_ = memo( useDrag((state) => onDrag?.current!(id, state), { target: ref, filterTaps: true, - preventScroll: true, + // Previous designs, which allowed tiles to be dragged and dropped around + // the scrolling grid, required us to set preventScroll to true here. But + // our designs no longer call for this, and meanwhile there's a bug in + // use-gesture that causes filterTaps + preventScroll to break buttons + // within tiles (like the 'switch camera' button) on mobile. + // https://github.com/pmndrs/use-gesture/issues/593 }); return ( diff --git a/src/room/CallEventAudioRenderer.test.tsx b/src/room/CallEventAudioRenderer.test.tsx index 281bbafd..1c515175 100644 --- a/src/room/CallEventAudioRenderer.test.tsx +++ b/src/room/CallEventAudioRenderer.test.tsx @@ -31,6 +31,7 @@ import { aliceRtcMember, bobRtcMember, local, + localRtcMember, } from "../utils/test-fixtures"; vitest.mock("../useAudioContext"); @@ -66,7 +67,7 @@ beforeEach(() => { * a noise every time. */ test("plays one sound when entering a call", () => { - const { vm, remoteRtcMemberships$ } = getBasicCallViewModelEnvironment([ + const { vm, rtcMemberships$ } = getBasicCallViewModelEnvironment([ local, alice, ]); @@ -74,56 +75,58 @@ test("plays one sound when entering a call", () => { // Joining a call usually means remote participants are added later. act(() => { - remoteRtcMemberships$.next([aliceRtcMember, bobRtcMember]); + rtcMemberships$.next([localRtcMember, aliceRtcMember, bobRtcMember]); }); expect(playSound).toHaveBeenCalledOnce(); }); test("plays a sound when a user joins", () => { - const { vm, remoteRtcMemberships$ } = getBasicCallViewModelEnvironment([ + const { vm, rtcMemberships$ } = getBasicCallViewModelEnvironment([ local, alice, ]); render(); act(() => { - remoteRtcMemberships$.next([aliceRtcMember, bobRtcMember]); + rtcMemberships$.next([localRtcMember, aliceRtcMember, bobRtcMember]); }); // Play a sound when joining a call. expect(playSound).toBeCalledWith("join"); }); test("plays a sound when a user leaves", () => { - const { vm, remoteRtcMemberships$ } = getBasicCallViewModelEnvironment([ + const { vm, rtcMemberships$ } = getBasicCallViewModelEnvironment([ local, alice, ]); render(); act(() => { - remoteRtcMemberships$.next([]); + rtcMemberships$.next([localRtcMember]); }); expect(playSound).toBeCalledWith("left"); }); test("plays no sound when the participant list is more than the maximum size", () => { - const mockRtcMemberships: CallMembership[] = []; + const mockRtcMemberships: CallMembership[] = [localRtcMember]; for (let i = 0; i < MAX_PARTICIPANT_COUNT_FOR_SOUND; i++) { mockRtcMemberships.push( mockRtcMembership(`@user${i}:example.org`, `DEVICE${i}`), ); } - const { vm, remoteRtcMemberships$ } = getBasicCallViewModelEnvironment( + const { vm, rtcMemberships$ } = getBasicCallViewModelEnvironment( [local, alice], mockRtcMemberships, ); render(); expect(playSound).not.toBeCalled(); + // Remove the last membership in the array to test the leaving sound + // (The array has length MAX_PARTICIPANT_COUNT_FOR_SOUND + 1) act(() => { - remoteRtcMemberships$.next( - mockRtcMemberships.slice(0, MAX_PARTICIPANT_COUNT_FOR_SOUND - 1), + rtcMemberships$.next( + mockRtcMemberships.slice(0, MAX_PARTICIPANT_COUNT_FOR_SOUND), ); }); expect(playSound).toBeCalledWith("left"); diff --git a/src/room/GroupCallView.test.tsx b/src/room/GroupCallView.test.tsx index 4eb32af0..12dfdf61 100644 --- a/src/room/GroupCallView.test.tsx +++ b/src/room/GroupCallView.test.tsx @@ -137,11 +137,9 @@ function createGroupCallView( getJoinRule: () => JoinRule.Invite, } as Partial as RoomState, }); - const rtcSession = new MockRTCSession( - room, - localRtcMember, - [], - ).withMemberships(constant([])); + const rtcSession = new MockRTCSession(room, []).withMemberships( + constant([localRtcMember]), + ); rtcSession.joined = joined; const muteState = { audio: { enabled: false }, diff --git a/src/room/GroupCallView.tsx b/src/room/GroupCallView.tsx index 76352523..ea57bd10 100644 --- a/src/room/GroupCallView.tsx +++ b/src/room/GroupCallView.tsx @@ -452,6 +452,7 @@ export const GroupCallView: FC = ({ client={client} matrixInfo={matrixInfo} rtcSession={rtcSession as MatrixRTCSession} + matrixRoom={room} participantCount={participantCount} onLeave={onLeave} header={header} diff --git a/src/room/InCallView.module.css b/src/room/InCallView.module.css index 24dfbe5c..41d48db1 100644 --- a/src/room/InCallView.module.css +++ b/src/room/InCallView.module.css @@ -115,7 +115,6 @@ Please see LICENSE in the repository root for full details. @media (max-width: 340px) { .invite, - .switchCamera, .shareScreen { display: none; } diff --git a/src/room/InCallView.test.tsx b/src/room/InCallView.test.tsx index b88aaad7..ec057e94 100644 --- a/src/room/InCallView.test.tsx +++ b/src/room/InCallView.test.tsx @@ -175,6 +175,7 @@ function createInCallView(): RenderResult & { kind: E2eeType.NONE, }, }} + matrixRoom={room} livekitRoom={livekitRoom} participantCount={0} onLeave={function (): void { diff --git a/src/room/InCallView.tsx b/src/room/InCallView.tsx index 5aa270d2..4e3229a5 100644 --- a/src/room/InCallView.tsx +++ b/src/room/InCallView.tsx @@ -7,8 +7,8 @@ Please see LICENSE in the repository root for full details. import { RoomContext, useLocalParticipant } from "@livekit/components-react"; import { IconButton, Text, Tooltip } from "@vector-im/compound-web"; -import { ConnectionState, type Room } from "livekit-client"; -import { type MatrixClient } from "matrix-js-sdk"; +import { ConnectionState, type Room as LivekitRoom } from "livekit-client"; +import { type MatrixClient, type Room as MatrixRoom } from "matrix-js-sdk"; import { type FC, type PointerEvent, @@ -44,7 +44,6 @@ import { ShareScreenButton, SettingsButton, ReactionToggleButton, - SwitchCameraButton, } from "../button"; import { Header, LeftNav, RightNav, RoomHeaderInfo } from "../Header"; import { type HeaderStyle, useUrlParams } from "../UrlParams"; @@ -94,7 +93,6 @@ import { useReactionsSender, } from "../reactions/useReactionsSender"; import { ReactionsAudioRenderer } from "./ReactionAudioRenderer"; -import { useSwitchCamera } from "./useSwitchCamera"; import { ReactionsOverlay } from "./ReactionsOverlay"; import { CallEventAudioRenderer } from "./CallEventAudioRenderer"; import { @@ -113,6 +111,7 @@ import { useMediaDevices } from "../MediaDevicesContext.ts"; import { EarpieceOverlay } from "./EarpieceOverlay.tsx"; import { useAppBarHidden, useAppBarSecondaryButton } from "../AppBar.tsx"; import { useBehavior } from "../useBehavior.ts"; +import { Toast } from "../Toast.tsx"; const canScreenshare = "getDisplayMedia" in (navigator.mediaDevices ?? {}); @@ -166,6 +165,7 @@ export const ActiveCall: FC = (props) => { const reactionsReader = new ReactionsReader(props.rtcSession); const vm = new CallViewModel( props.rtcSession, + props.matrixRoom, livekitRoom, mediaDevices, { @@ -184,6 +184,7 @@ export const ActiveCall: FC = (props) => { } }, [ props.rtcSession, + props.matrixRoom, livekitRoom, mediaDevices, props.e2eeSystem, @@ -212,7 +213,8 @@ export interface InCallViewProps { vm: CallViewModel; matrixInfo: MatrixInfo; rtcSession: MatrixRTCSession; - livekitRoom: Room; + matrixRoom: MatrixRoom; + livekitRoom: LivekitRoom; muteStates: MuteStates; participantCount: number; /** Function to call when the user explicitly ends the call */ @@ -228,6 +230,7 @@ export const InCallView: FC = ({ vm, matrixInfo, rtcSession, + matrixRoom, livekitRoom, muteStates, participantCount, @@ -272,7 +275,7 @@ export const InCallView: FC = ({ const [useExperimentalToDeviceTransport] = useSetting( useExperimentalToDeviceTransportSetting, ); - const encryptionSystem = useRoomEncryptionSystem(rtcSession.room.roomId); + const encryptionSystem = useRoomEncryptionSystem(matrixRoom.roomId); const memberships = useMatrixRTCSessionMemberships(rtcSession); const showToDeviceEncryption = useMemo( @@ -309,6 +312,7 @@ export const InCallView: FC = ({ () => void toggleRaisedHand(), ); + const reconnecting = useBehavior(vm.reconnecting$); const windowMode = useBehavior(vm.windowMode$); const layout = useBehavior(vm.layout$); const tileStoreGeneration = useBehavior(vm.tileStoreGeneration$); @@ -318,7 +322,6 @@ export const InCallView: FC = ({ const showFooter = useBehavior(vm.showFooter$); const earpieceMode = useBehavior(vm.earpieceMode$); const audioOutputSwitcher = useBehavior(vm.audioOutputSwitcher$); - const switchCamera = useSwitchCamera(vm.localVideo$); useSubscription(vm.autoLeaveWhenOthersLeft$, onLeave); // Ideally we could detect taps by listening for click events and checking @@ -502,7 +505,11 @@ export const InCallView: FC = ({ break; case "standard": header = ( -
+
= ({ }; const rageshakeRequestModalProps = useRageshakeRequestModal( - rtcSession.room.roomId, + matrixRoom.roomId, ); const toggleScreensharing = useCallback(() => { @@ -676,15 +683,6 @@ export const InCallView: FC = ({ data-testid="incall_videomute" />, ); - if (switchCamera !== null) - buttons.push( - , - ); if (canScreenshare && !hideScreensharing) { buttons.push( = ({ ); + // The reconnecting toast cannot be dismissed + const onDismissReconnectingToast = useCallback(() => {}, []); + return (
= ({ {renderContent()} + + {t("common.reconnecting")} + @@ -800,7 +808,7 @@ export const InCallView: FC = ({ = ({ }, [devices, videoInputId, videoTrack]); useTrackProcessorSync(videoTrack); - const showSwitchCamera = useShowSwitchCamera( - useObservable( - (inputs$) => inputs$.pipe(map(([video]) => video)), - [videoTrack], - ), - ); // TODO: Unify this component with InCallView, so we can get slick joining // animations and don't have to feel bad about reusing its CSS @@ -257,9 +248,6 @@ export const LobbyView: FC = ({ onClick={onVideoPress} disabled={muteStates.video.setEnabled === null} /> - {showSwitchCamera && ( - - )} {!confineToRoom && }
diff --git a/src/room/__snapshots__/InCallView.test.tsx.snap b/src/room/__snapshots__/InCallView.test.tsx.snap index b45a30ad..7d6ab966 100644 --- a/src/room/__snapshots__/InCallView.test.tsx.snap +++ b/src/room/__snapshots__/InCallView.test.tsx.snap @@ -256,7 +256,7 @@ exports[`InCallView > rendering > renders 1`] = ` > - } - side="left" - align="start" - > - {menu} - + primaryButton ?? ( + + + + } + side="left" + align="start" + > + {menu} + + ) } raisedHandTime={handRaised ?? undefined} currentReaction={reaction ?? undefined} @@ -208,6 +213,8 @@ const LocalUserMediaTile: FC = ({ const { t } = useTranslation(); const mirror = useBehavior(vm.mirror$); const alwaysShow = useBehavior(vm.alwaysShow$); + const switchCamera = useBehavior(vm.switchCamera$); + const latestAlwaysShow = useLatest(alwaysShow); const onSelectAlwaysShow = useCallback( (e: Event) => { @@ -223,6 +230,17 @@ const LocalUserMediaTile: FC = ({ vm={vm} locallyMuted={false} mirror={mirror} + primaryButton={ + switchCamera === null ? undefined : ( + + ) + } menuStart={ button:active { - background: var(--cpd-color-bg-action-primary-pressed) !important; + background: var(--cpd-color-bg-action-primary-pressed); } .fg > button[data-state="open"] { diff --git a/src/tile/SpotlightTile.test.tsx b/src/tile/SpotlightTile.test.tsx index 5165cfa1..9493b29a 100644 --- a/src/tile/SpotlightTile.test.tsx +++ b/src/tile/SpotlightTile.test.tsx @@ -12,6 +12,8 @@ import userEvent from "@testing-library/user-event"; import { SpotlightTile } from "./SpotlightTile"; import { + mockLocalParticipant, + mockMediaDevices, mockRtcMembership, withLocalMedia, withRemoteMedia, @@ -39,6 +41,8 @@ test("SpotlightTile is accessible", async () => { rawDisplayName: "Bob", getMxcAvatarUrl: () => "mxc://dlskf", }, + mockLocalParticipant({}), + mockMediaDevices({}), async (vm2) => { const user = userEvent.setup(); const toggleExpanded = vi.fn(); diff --git a/src/tile/SpotlightTile.tsx b/src/tile/SpotlightTile.tsx index 8bc45a81..abdd27f6 100644 --- a/src/tile/SpotlightTile.tsx +++ b/src/tile/SpotlightTile.tsx @@ -54,6 +54,7 @@ interface SpotlightItemBaseProps { targetWidth: number; targetHeight: number; video: TrackReferenceOrPlaceholder | undefined; + videoEnabled: boolean; member: RoomMember | undefined; unencryptedWarning: boolean; encryptionStatus: EncryptionStatus; @@ -63,7 +64,6 @@ interface SpotlightItemBaseProps { } interface SpotlightUserMediaItemBaseProps extends SpotlightItemBaseProps { - videoEnabled: boolean; videoFit: "contain" | "cover"; } @@ -90,12 +90,10 @@ const SpotlightUserMediaItem: FC = ({ vm, ...props }) => { - const videoEnabled = useBehavior(vm.videoEnabled$); const cropVideo = useBehavior(vm.cropVideo$); const baseProps: SpotlightUserMediaItemBaseProps & RefAttributes = { - videoEnabled, videoFit: cropVideo ? "cover" : "contain", ...props, }; @@ -135,6 +133,7 @@ const SpotlightItem: FC = ({ const ref = useMergedRefs(ourRef, theirRef); const displayName = useBehavior(vm.displayName$); const video = useBehavior(vm.video$); + const videoEnabled = useBehavior(vm.videoEnabled$); const unencryptedWarning = useBehavior(vm.unencryptedWarning$); const encryptionStatus = useBehavior(vm.encryptionStatus$); @@ -160,6 +159,7 @@ const SpotlightItem: FC = ({ targetWidth, targetHeight, video, + videoEnabled, member: vm.member, unencryptedWarning, displayName, @@ -169,7 +169,7 @@ const SpotlightItem: FC = ({ }; return vm instanceof ScreenShareViewModel ? ( - + ) : ( ); diff --git a/src/utils/observable.test.ts b/src/utils/observable.test.ts new file mode 100644 index 00000000..5f488fb1 --- /dev/null +++ b/src/utils/observable.test.ts @@ -0,0 +1,24 @@ +/* +Copyright 2025 New Vector Ltd. + +SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial +Please see LICENSE in the repository root for full details. +*/ + +import { test } from "vitest"; + +import { withTestScheduler } from "./test"; +import { pauseWhen } from "./observable"; + +test("pauseWhen", () => { + withTestScheduler(({ behavior, expectObservable }) => { + const inputMarbles = " abcdefgh-i-jk-"; + const pauseMarbles = " n-y--n-yn-y--n"; + const outputMarbles = "abc--fgh-i---k"; + expectObservable( + behavior(inputMarbles).pipe( + pauseWhen(behavior(pauseMarbles, { y: true, n: false })), + ), + ).toBe(outputMarbles); + }); +}); diff --git a/src/utils/observable.ts b/src/utils/observable.ts index 22f7c455..74acfaf2 100644 --- a/src/utils/observable.ts +++ b/src/utils/observable.ts @@ -7,16 +7,23 @@ Please see LICENSE in the repository root for full details. import { type Observable, + audit, + combineLatest, concat, defer, + filter, finalize, map, + of, scan, startWith, takeWhile, tap, + withLatestFrom, } from "rxjs"; +import { type Behavior } from "../state/Behavior"; + const nothing = Symbol("nothing"); /** @@ -86,3 +93,27 @@ export function getValue(state$: Observable): T { if (value === nothing) throw new Error("Not a state Observable"); return value; } + +/** + * Creates an Observable that has a value of true whenever all its inputs are + * true. + */ +export function and$(...inputs: Observable[]): Observable { + return combineLatest(inputs, (...flags) => flags.every((flag) => flag)); +} + +/** + * RxJS operator that pauses all changes in the input value whenever a Behavior + * is true. When the Behavior returns to being false, the most recently + * suppressed change is emitted as the most recent value. + */ +export function pauseWhen(pause$: Behavior) { + return (value$: Observable): Observable => + value$.pipe( + withLatestFrom(pause$), + audit(([, pause]) => + pause ? pause$.pipe(filter((pause) => !pause)) : of(null), + ), + map(([value]) => value), + ); +} diff --git a/src/utils/test-viewmodel.ts b/src/utils/test-viewmodel.ts index 4781bf3d..e5558ae2 100644 --- a/src/utils/test-viewmodel.ts +++ b/src/utils/test-viewmodel.ts @@ -14,8 +14,13 @@ import { BehaviorSubject, of } from "rxjs"; import { vitest } from "vitest"; import { type RelationsContainer } from "matrix-js-sdk/lib/models/relations-container"; import EventEmitter from "events"; +import { + type RoomMember, + type MatrixClient, + type Room, + SyncState, +} from "matrix-js-sdk"; -import type { RoomMember, MatrixClient } from "matrix-js-sdk"; import { E2eeType } from "../e2ee/e2eeType"; import { CallViewModel } from "../state/CallViewModel"; import { @@ -34,10 +39,11 @@ import { type RaisedHandInfo, type ReactionInfo } from "../reactions"; export function getBasicRTCSession( members: RoomMember[], - initialRemoteRtcMemberships: CallMembership[] = [aliceRtcMember], + initialRtcMemberships: CallMembership[] = [localRtcMember, aliceRtcMember], ): { rtcSession: MockRTCSession; - remoteRtcMemberships$: BehaviorSubject; + matrixRoom: Room; + rtcMemberships$: BehaviorSubject; } { const matrixRoomId = "!myRoomId:example.com"; const matrixRoomMembers = new Map(members.map((p) => [p.userId, p])); @@ -51,6 +57,7 @@ export function getBasicRTCSession( client: { getUserId: () => localRtcMember.sender, getDeviceId: () => localRtcMember.deviceId, + getSyncState: () => SyncState.Syncing, sendEvent: vitest.fn().mockResolvedValue({ event_id: "$fake:event" }), redactEvent: vitest.fn().mockResolvedValue({ event_id: "$fake:event" }), decryptEventIfNeeded: vitest.fn().mockResolvedValue(undefined), @@ -91,52 +98,53 @@ export function getBasicRTCSession( ), }); - const remoteRtcMemberships$ = new BehaviorSubject( - initialRemoteRtcMemberships, + const rtcMemberships$ = new BehaviorSubject( + initialRtcMemberships, ); - const rtcSession = new MockRTCSession( - matrixRoom, - localRtcMember, - ).withMemberships(remoteRtcMemberships$); + const rtcSession = new MockRTCSession(matrixRoom).withMemberships( + rtcMemberships$, + ); return { rtcSession, - remoteRtcMemberships$, + matrixRoom, + rtcMemberships$, }; } /** * Construct a basic CallViewModel to test components that make use of it. * @param members - * @param initialRemoteRtcMemberships + * @param initialRtcMemberships * @returns */ export function getBasicCallViewModelEnvironment( members: RoomMember[], - initialRemoteRtcMemberships: CallMembership[] = [aliceRtcMember], + initialRtcMemberships: CallMembership[] = [localRtcMember, aliceRtcMember], ): { vm: CallViewModel; - remoteRtcMemberships$: BehaviorSubject; + rtcMemberships$: BehaviorSubject; rtcSession: MockRTCSession; handRaisedSubject$: BehaviorSubject>; reactionsSubject$: BehaviorSubject>; } { - const { rtcSession, remoteRtcMemberships$ } = getBasicRTCSession( + const { rtcSession, matrixRoom, rtcMemberships$ } = getBasicRTCSession( members, - initialRemoteRtcMemberships, + initialRtcMemberships, ); const handRaisedSubject$ = new BehaviorSubject({}); const reactionsSubject$ = new BehaviorSubject({}); const remoteParticipants$ = of([aliceParticipant]); - const liveKitRoom = mockLivekitRoom( + const livekitRoom = mockLivekitRoom( { localParticipant }, { remoteParticipants$ }, ); const vm = new CallViewModel( rtcSession as unknown as MatrixRTCSession, - liveKitRoom, + matrixRoom, + livekitRoom, mockMediaDevices({}), { encryptionSystem: { kind: E2eeType.PER_PARTICIPANT }, @@ -147,7 +155,7 @@ export function getBasicCallViewModelEnvironment( ); return { vm, - remoteRtcMemberships$, + rtcMemberships$, rtcSession, handRaisedSubject$: handRaisedSubject$, reactionsSubject$: reactionsSubject$, diff --git a/src/utils/test.ts b/src/utils/test.ts index ce964ee8..3e47f4f6 100644 --- a/src/utils/test.ts +++ b/src/utils/test.ts @@ -19,8 +19,11 @@ import { type Focus, MatrixRTCSessionEvent, type MatrixRTCSessionEventHandlerMap, + MembershipManagerEvent, type SessionMembershipData, + Status, } from "matrix-js-sdk/lib/matrixrtc"; +import { type MembershipManagerEventHandlerMap } from "matrix-js-sdk/lib/matrixrtc/IMembershipManager"; import { type LocalParticipant, type LocalTrackPublication, @@ -233,6 +236,7 @@ export function mockLocalParticipant( ): LocalParticipant { return { isLocal: true, + trackPublications: new Map(), getTrackPublication: () => ({}) as Partial as LocalTrackPublication, ...mockEmitter(), @@ -243,9 +247,10 @@ export function mockLocalParticipant( export async function withLocalMedia( localRtcMember: CallMembership, roomMember: Partial, + localParticipant: LocalParticipant, + mediaDevices: MediaDevices, continuation: (vm: LocalUserMediaViewModel) => void | Promise, ): Promise { - const localParticipant = mockLocalParticipant({}); const vm = new LocalUserMediaViewModel( "local", mockMatrixRoomMember(localRtcMember, roomMember), @@ -254,6 +259,7 @@ export async function withLocalMedia( kind: E2eeType.PER_PARTICIPANT, }, mockLivekitRoom({ localParticipant }), + mediaDevices, constant(roomMember.rawDisplayName ?? "nodisplayname"), constant(null), constant(null), @@ -293,6 +299,7 @@ export async function withRemoteMedia( kind: E2eeType.PER_PARTICIPANT, }, mockLivekitRoom({}, { remoteParticipants$: of([remoteParticipant]) }), + constant(false), constant(roomMember.rawDisplayName ?? "nodisplayname"), constant(null), constant(null), @@ -314,8 +321,10 @@ export function mockConfig(config: Partial = {}): void { } export class MockRTCSession extends TypedEventEmitter< - MatrixRTCSessionEvent | RoomAndToDeviceEvents, - MatrixRTCSessionEventHandlerMap & RoomAndToDeviceEventsHandlerMap + MatrixRTCSessionEvent | RoomAndToDeviceEvents | MembershipManagerEvent, + MatrixRTCSessionEventHandlerMap & + RoomAndToDeviceEventsHandlerMap & + MembershipManagerEventHandlerMap > { public readonly statistics = { counters: {}, @@ -325,7 +334,6 @@ export class MockRTCSession extends TypedEventEmitter< public constructor( public readonly room: Room, - private localMembership: CallMembership, public memberships: CallMembership[] = [], ) { super(); @@ -341,14 +349,37 @@ export class MockRTCSession extends TypedEventEmitter< ): MockRTCSession { rtcMembers$.subscribe((m) => { const old = this.memberships; - // always prepend the local participant - const updated = [this.localMembership, ...(m as CallMembership[])]; - this.memberships = updated; - this.emit(MatrixRTCSessionEvent.MembershipsChanged, old, updated); + this.memberships = m as CallMembership[]; + this.emit( + MatrixRTCSessionEvent.MembershipsChanged, + old, + this.memberships, + ); }); return this; } + + private _membershipStatus = Status.Connected; + public get membershipStatus(): Status { + return this._membershipStatus; + } + public set membershipStatus(value: Status) { + const prev = this._membershipStatus; + this._membershipStatus = value; + if (value !== prev) + this.emit(MembershipManagerEvent.StatusChanged, prev, value); + } + + private _probablyLeft = false; + public get probablyLeft(): boolean { + return this._probablyLeft; + } + public set probablyLeft(value: boolean) { + const prev = this._probablyLeft; + this._probablyLeft = value; + if (value !== prev) this.emit(MembershipManagerEvent.ProbablyLeft, value); + } } export const mockTrack = (identity: string): TrackReference => diff --git a/src/widget.ts b/src/widget.ts index a122f70d..61bf3850 100644 --- a/src/widget.ts +++ b/src/widget.ts @@ -120,8 +120,8 @@ export const widget = ((): WidgetHelpers | null => { const sendState = [ userId, // Legacy call membership events - `_${userId}_${deviceId}`, // Session membership events - `${userId}_${deviceId}`, // The above with no leading underscore, for room versions whose auth rules allow it + `_${userId}_${deviceId}_m.call`, // Session membership events + `${userId}_${deviceId}_m.call`, // The above with no leading underscore, for room versions whose auth rules allow it ].map((stateKey) => ({ eventType: EventType.GroupCallMemberPrefix, stateKey, diff --git a/yarn.lock b/yarn.lock index 128e39e5..94f8feaf 100644 --- a/yarn.lock +++ b/yarn.lock @@ -10046,8 +10046,8 @@ __metadata: linkType: hard "livekit-client@npm:^2.13.0": - version: 2.15.5 - resolution: "livekit-client@npm:2.15.5" + version: 2.15.6 + resolution: "livekit-client@npm:2.15.6" dependencies: "@livekit/mutex": "npm:1.1.1" "@livekit/protocol": "npm:1.39.3" @@ -10060,7 +10060,7 @@ __metadata: webrtc-adapter: "npm:^9.0.1" peerDependencies: "@types/dom-mediacapture-record": ^1 - checksum: 10c0/52a70bdd39d802737ed7c25ae5d06daf9921156c4fc74f918009e86204430b2d200b66c55cefab949be4e5411cbc4d25eac92976f62f96b7226057a5b0706baa + checksum: 10c0/f1ab6cdf2b85647036e9de906734c1394dac497da0bd879a29d0c587c437ada262021478fcef24df99b3489a39d97fe67ab33de0785ed0a63335da2fef577192 languageName: node linkType: hard @@ -10279,8 +10279,8 @@ __metadata: linkType: hard "matrix-js-sdk@github:matrix-org/matrix-js-sdk#head=develop": - version: 37.12.0 - resolution: "matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk.git#commit=c4e1e0723e3674725f8ed9cda5be7a7efe304e29" + version: 37.13.0 + resolution: "matrix-js-sdk@https://github.com/matrix-org/matrix-js-sdk.git#commit=3a33c658bbcb8ce8791ec066db899f2571f5c52f" dependencies: "@babel/runtime": "npm:^7.12.5" "@matrix-org/matrix-sdk-crypto-wasm": "npm:^15.1.0" @@ -10296,7 +10296,7 @@ __metadata: sdp-transform: "npm:^2.14.1" unhomoglyph: "npm:^1.0.6" uuid: "npm:11" - checksum: 10c0/e8f97b90ded9e46bdd803dde9861e178c76741999c224cbc869ad38695f300702405e5c365e270663b90e2a317b646bbf2b9e9bd7d51ffce36e6d2ca3ab29ed9 + checksum: 10c0/1db0d39cfbe4f1c69c8acda0ea7580a4819fc47a7d4bff057382e33e72d9a610f8c03043a6c00bc647dfdc2815aa643c69d25022fb759342a92b77e1841524f1 languageName: node linkType: hard