Add the MatrixRTC SDK package, its harness and a failing smoke test

Set up `sdk/` as a package shaped exactly like `component/`: its own
package.json with peer dependencies, the pnpm workspace marker so a host can
install it as a git dependency, a build tsconfig for type emission, and a vite
lib config with an externals list that `pnpm lint:externals` now checks for
both library builds.

The package holds only the interface for now. `createRtcSession` returns an
empty object; `RtcSession`, `RtcMember`, `MemberMedia` and `TransportMetadata`
are the contract Element Call's `CallViewModel` will be rebuilt on (see
sdk-plan.md). The shared primitives are re-exported from `src` until the first
implementation slice moves them.

`sdk/dev` is the smallest consumer of the SDK: a plain page that logs in, joins
a room and renders one tile per member, served by `pnpm dev:sdk` on port 3002.
Playwright starts it as a third web server, and `playwright/sdk/smoke.spec.ts`
has two browsers join one session. It fails on the status line with the SDK's
own error until there is an implementation behind the interface.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

sdk/README.md

@@ -0,0 +1,74 @@
+# MatrixRTC SDK (EXPERIMENTAL)
+
+`@element-hq/matrixrtc-sdk` is the call model under Element Call, on its own:
+MatrixRTC memberships and transports, LiveKit connections, publishing, E2EE keys and
+the media of every member, as observables. It has no UI. Element Call's own
+`CallViewModel` is meant to become one consumer of it; the design is in
+[`sdk-plan.md`](../sdk-plan.md).
+
+**Status:** interface only. `createRtcSession` returns an object that does nothing.
+The development harness and its e2e test exist so the implementation can be built
+against them.
+
+## Using it
+
+```ts
+import {
+  constant,
+  createRtcSession,
+  E2eeType,
+  MatrixRTCMode,
+  ObservableScope,
+} from "@element-hq/matrixrtc-sdk";
+
+const scope = new ObservableScope();
+const session = createRtcSession(
+  scope,
+  client, // a matrix-js-sdk MatrixClient, logged in and syncing
+  room, // the matrix-js-sdk Room to hold the session in
+  {
+    microphoneEnabled$: constant(true),
+    cameraEnabled$: constant(true),
+    audioInputDeviceId$: constant(undefined),
+    videoInputDeviceId$: constant(undefined),
+    videoProcessor$: constant(undefined),
+  },
+  {
+    encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
+    matrixRTCMode: MatrixRTCMode.Compatibility,
+  },
+);
+session.join();
+
+session.remoteMembers$.subscribe((members) => {
+  // each member has displayName$, media$ and more; a media track is rendered
+  // by handing it a <video> or <audio> element: track.attach(element)
+});
+
+// later
+session.leave();
+scope.end();
+```
+
+[`dev/main.ts`](./dev/main.ts) is this example as a working page.
+
+## Development
+
+Everything runs from the repository root. This directory is a pnpm project of its
+own only so that a host can install it as a git dependency (see
+`pnpm-workspace.yaml`).
+
+```sh
+pnpm dev:sdk              # the harness, https://localhost:3002
+pnpm build:sdk            # dist/matrixrtc-sdk.js and dist/types
+pnpm test:playwright playwright/sdk   # needs `pnpm backend`
+```
+
+## Installing
+
+```sh
+pnpm add github:element-hq/element-call#<ref>&path:/sdk
+```
+
+The host supplies `matrix-js-sdk`, `livekit-client` and `rxjs`; the build leaves
+them external.

sdk/dev/index.html

@@ -0,0 +1,56 @@
+<!doctype html>
+<!--
+Copyright 2026 Element Creations Ltd.
+
+SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
+Please see LICENSE in the repository root for full details.
+-->
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>MatrixRTC SDK harness</title>
+    <style>
+      body {
+        margin: 1rem;
+        font-family: system-ui, sans-serif;
+      }
+      form {
+        display: flex;
+        gap: 0.5rem;
+        flex-wrap: wrap;
+      }
+      #members {
+        display: grid;
+        grid-template-columns: repeat(auto-fill, minmax(240px, 1fr));
+        gap: 0.5rem;
+      }
+      #members video {
+        width: 100%;
+        aspect-ratio: 4 / 3;
+        background: #222;
+      }
+      #members h2 {
+        font-size: 1rem;
+        margin: 0.25rem 0;
+      }
+    </style>
+  </head>
+  <body>
+    <form>
+      <input
+        name="homeserver"
+        placeholder="homeserver"
+        value="https://synapse.m.localhost"
+      />
+      <input name="username" placeholder="username" />
+      <input name="password" type="password" placeholder="password" />
+      <input name="room" placeholder="room id or alias" />
+      <button type="submit">Start</button>
+      <button type="button" id="leave" hidden>Leave</button>
+    </form>
+    <p id="status" data-testid="status"></p>
+    <main id="members"></main>
+    <script type="module" src="./main.ts"></script>
+  </body>
+</html>

sdk/dev/main.ts

@@ -0,0 +1,157 @@
+/*
+Copyright 2026 Element Creations Ltd.
+
+SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
+Please see LICENSE in the repository root for full details.
+*/
+
+/**
+ * The smallest consumer of the SDK: log in, join a room, show every member's
+ * camera and play every remote member's microphone, leave. What a host writes,
+ * and nothing a host would not.
+ */
+
+import { logger } from "matrix-js-sdk/lib/logger";
+import { combineLatest, type Observable, of, switchMap } from "rxjs";
+import {
+  constant,
+  createRtcSession,
+  E2eeType,
+  MatrixRTCMode,
+  type MediaTrack,
+  ObservableScope,
+  type RtcMember,
+  type RtcSession,
+} from "@element-hq/matrixrtc-sdk";
+
+import { createSession } from "./session";
+
+const form = document.querySelector("form")!;
+const status = document.getElementById("status")!;
+const members = document.getElementById("members")!;
+const leaveButton = document.getElementById("leave") as HTMLButtonElement;
+
+// So that a test, or a bookmark, can fill the form from the URL
+const params = new URLSearchParams(location.search);
+for (const input of form.querySelectorAll("input"))
+  input.value = params.get(input.name) ?? input.value;
+
+form.addEventListener("submit", (event) => {
+  event.preventDefault();
+  const fields = new FormData(form);
+  const field = (name: string): string => fields.get(name) as string;
+  void start(
+    field("homeserver"),
+    field("username"),
+    field("password"),
+    field("room"),
+  );
+});
+
+async function start(
+  homeserver: string,
+  username: string,
+  password: string,
+  roomIdOrAlias: string,
+): Promise<void> {
+  try {
+    status.textContent = "Logging in";
+    const client = await createSession(homeserver, username, password);
+    const room = await client.joinRoom(roomIdOrAlias);
+
+    const scope = new ObservableScope();
+    const session = createRtcSession(
+      scope,
+      client,
+      room,
+      {
+        microphoneEnabled$: constant(true),
+        cameraEnabled$: constant(true),
+        audioInputDeviceId$: constant(undefined),
+        videoInputDeviceId$: constant(undefined),
+        videoProcessor$: constant(undefined),
+      },
+      {
+        encryptionSystem: { kind: E2eeType.PER_PARTICIPANT },
+        matrixRTCMode: MatrixRTCMode.Compatibility,
+      },
+    );
+    showMembers(scope, session);
+    session.join();
+    status.textContent = "Joined";
+
+    leaveButton.hidden = false;
+    leaveButton.onclick = (): void => {
+      session.leave();
+      scope.end();
+      members.replaceChildren();
+      leaveButton.hidden = true;
+      status.textContent = "Left";
+    };
+  } catch (e) {
+    status.textContent = `Error: ${e}`;
+    logger.error(e);
+  }
+}
+
+function showMembers(scope: ObservableScope, session: RtcSession): void {
+  const tiles = new Map<string, HTMLElement>();
+  combineLatest([session.localMember$, session.remoteMembers$])
+    .pipe(scope.bind())
+    .subscribe(([local, remote]) => {
+      const current = local === null ? remote : [local, ...remote];
+      for (const [id, tile] of tiles)
+        if (!current.some((m) => m.id === id)) {
+          tile.remove();
+          tiles.delete(id);
+        }
+      for (const member of current)
+        if (!tiles.has(member.id)) {
+          const tile = memberTile(scope, member);
+          tiles.set(member.id, tile);
+          members.append(tile);
+        }
+    });
+}
+
+function memberTile(scope: ObservableScope, member: RtcMember): HTMLElement {
+  const tile = document.createElement("section");
+  tile.dataset.testid = "member";
+  tile.dataset.userId = member.userId;
+
+  const name = tile.appendChild(document.createElement("h2"));
+  member.displayName$.pipe(scope.bind()).subscribe((n) => {
+    name.textContent = n;
+  });
+
+  const camera$ = member.media$.pipe(
+    switchMap((media) => media?.camera$ ?? of(undefined)),
+  );
+  render(scope, camera$, tile.appendChild(document.createElement("video")));
+  // Our own microphone would only echo
+  if (!member.local) {
+    const microphone$ = member.media$.pipe(
+      switchMap((media) => media?.microphone$ ?? of(undefined)),
+    );
+    render(
+      scope,
+      microphone$,
+      tile.appendChild(document.createElement("audio")),
+    );
+  }
+  return tile;
+}
+
+function render(
+  scope: ObservableScope,
+  track$: Observable<MediaTrack | undefined>,
+  element: HTMLMediaElement,
+): void {
+  let attached: MediaTrack | undefined;
+  track$.pipe(scope.bind()).subscribe((track) => {
+    attached?.detach(element);
+    attached = track;
+    track?.attach(element);
+  });
+  scope.onEnd(() => attached?.detach(element));
+}

sdk/dev/session.ts

@@ -0,0 +1,49 @@
+/*
+Copyright 2026 Element Creations Ltd.
+
+SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
+Please see LICENSE in the repository root for full details.
+*/
+
+import {
+  ClientEvent,
+  createClient,
+  type MatrixClient,
+  MemoryStore,
+  SyncState,
+} from "matrix-js-sdk";
+
+/** Logs in with a password and returns a client that has finished its first sync. */
+export async function createSession(
+  homeserver: string,
+  username: string,
+  password: string,
+): Promise<MatrixClient> {
+  const login = await createClient({ baseUrl: homeserver }).login(
+    "m.login.password",
+    { identifier: { type: "m.id.user", user: username }, password },
+  );
+
+  const client = createClient({
+    baseUrl: homeserver,
+    accessToken: login.access_token,
+    userId: login.user_id,
+    deviceId: login.device_id,
+    store: new MemoryStore(),
+    useAuthorizationHeader: true,
+    fallbackICEServerAllowed: true,
+  });
+
+  await client.initRustCrypto({ useIndexedDB: false });
+  await client.startClient();
+  await new Promise<void>((resolve) => {
+    const onSync = (state: SyncState): void => {
+      if (state !== SyncState.Prepared && state !== SyncState.Syncing) return;
+      client.off(ClientEvent.Sync, onSync);
+      resolve();
+    };
+    client.on(ClientEvent.Sync, onSync);
+  });
+
+  return client;
+}

sdk/index.ts

@@ -0,0 +1,271 @@
+/*
+Copyright 2026 Element Creations Ltd.
+
+SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
+Please see LICENSE in the repository root for full details.
+*/
+
+/**
+ * EXPERIMENTAL
+ *
+ * MatrixRTC sessions with LiveKit media, without Element Call's UI: the call
+ * model Element Call's own view model is built on, for hosts that want to
+ * build a different one.
+ *
+ * Only the interface exists so far. `createRtcSession` returns an object that
+ * does nothing, so that the development harness in `sdk/dev` and its e2e test
+ * can be written against the contract before the implementation lands behind
+ * it. The design and the migration from Element Call's `CallViewModel` are in
+ * `sdk-plan.md` at the repository root.
+ */
+
+import { type MatrixClient, type Room } from "matrix-js-sdk";
+import {
+  type CallMembership,
+  type RTCCallIntent,
+  type RTCNotificationType,
+  type Transport,
+} from "matrix-js-sdk/lib/matrixrtc";
+import { type Track, type TrackProcessor } from "livekit-client";
+import { type Observable } from "rxjs";
+
+import { type Behavior } from "../src/state/Behavior";
+import { type ObservableScope } from "../src/state/ObservableScope";
+import { type EncryptionSystem } from "../src/e2ee/sharedKeyManagement";
+import { type MatrixRTCMode } from "../src/config/ConfigOptions";
+
+// Shared with Element Call. They live in `src` until the SDK has an
+// implementation to move them with; a consumer gets them from here either way.
+export { type Behavior, constant } from "../src/state/Behavior";
+export { ObservableScope } from "../src/state/ObservableScope";
+export { E2eeType } from "../src/e2ee/e2eeType";
+export { type EncryptionSystem } from "../src/e2ee/sharedKeyManagement";
+export { MatrixRTCMode } from "../src/config/ConfigOptions";
+
+// ---------------------------------------------------------------------------
+// Session
+
+export interface RtcSessionOptions {
+  encryptionSystem: EncryptionSystem;
+  /** Resolved by the host; the SDK reads neither config.json nor settings. */
+  matrixRTCMode: MatrixRTCMode;
+  /**
+   * MSC4075 notification sent with the join. Parameters of the MatrixRTC join
+   * itself, so they are here even though they are named after calls; reacting
+   * to a notification (ringing, timeouts, declines) is the application's job.
+   */
+  sendNotificationType?: RTCNotificationType;
+  callIntent?: RTCCallIntent;
+}
+
+/** What the local member publishes. */
+export interface LocalMediaInputs {
+  microphoneEnabled$: Behavior<boolean>;
+  cameraEnabled$: Behavior<boolean>;
+  audioInputDeviceId$: Behavior<string | undefined>;
+  videoInputDeviceId$: Behavior<string | undefined>;
+  /** Background blur and the like. */
+  videoProcessor$: Behavior<TrackProcessor<Track.Kind.Video> | undefined>;
+}
+
+export type SessionConnectionStatus =
+  | "waitingForTransport"
+  | "connecting"
+  | "connected"
+  | "reconnecting"
+  | "disconnected";
+
+export class RtcSessionError extends Error {
+  public constructor(message: string, options?: ErrorOptions) {
+    super(message, options);
+    this.name = "RtcSessionError";
+  }
+}
+
+export interface RtcSession {
+  join(): void;
+  leave(): void;
+  /** Collapsed view of the local member's state machine. */
+  status$: Behavior<SessionConnectionStatus>;
+  connected$: Behavior<boolean>;
+  reconnecting$: Behavior<boolean>;
+  /** A transport, Matrix or connection error that stops the session. */
+  fatalError$: Behavior<RtcSessionError | null>;
+
+  localMember$: Behavior<LocalRtcMember | null>;
+  remoteMembers$: Behavior<RemoteRtcMember[]>;
+  /** `remoteMembers.length`, plus one for the local member once it exists. */
+  memberCount$: Behavior<number>;
+
+  /**
+   * Whether the session has grown large enough that MatrixRTC has stopped
+   * rotating the media encryption key.
+   */
+  keyRotationSuppressed$: Behavior<boolean>;
+
+  /** Transports the session currently holds a live connection to. */
+  connectedTransports$: Behavior<TransportMetadata[]>;
+}
+
+// ---------------------------------------------------------------------------
+// Transports
+
+/**
+ * One transport advertised in a membership. Transport independent: `type` and
+ * `id` are all the SDK needs; `raw` and `resolved$` are there for a
+ * backend-specific developer panel and for connection diagnostics.
+ */
+export interface TransportMetadata {
+  /** `"livekit"` today. */
+  type: string;
+  /** Stable key, unique per transport in the session. For LiveKit, the service url. */
+  id: string;
+  /** The transport object as it appears in the membership. */
+  raw: Transport;
+  /**
+   * What the backend had to fetch before it could connect. Undefined until the
+   * connection has resolved it, and again after the connection stops.
+   */
+  resolved$: Behavior<ResolvedTransport | undefined>;
+}
+
+export type ResolvedTransport =
+  | {
+      type: "livekit";
+      /** The SFU websocket url, as opposed to the JWT service url in `raw`. */
+      url: string;
+      /** A secret: fit for a developer panel, never for a log line. */
+      token: string;
+      roomAlias: string;
+      identity: string;
+    }
+  | { type: string; [key: string]: unknown };
+
+// ---------------------------------------------------------------------------
+// Members
+
+export interface RtcMember {
+  local: boolean;
+  /** `${userId}:${deviceId}` before sticky events, a uuid in Matrix 2.0 mode. The backend identity. */
+  id: string;
+  userId: string;
+  deviceId: string;
+  membership$: Behavior<CallMembership>;
+  displayName$: Behavior<string>;
+  avatarUrl$: Behavior<string | undefined>;
+  /** Which transport this member is on; undefined when the membership has none. */
+  transport$: Behavior<TransportMetadata | undefined>;
+  /**
+   * Null while the member has a transport but no media has arrived on it yet
+   * ("waiting for media").
+   */
+  media$: Behavior<MemberMedia | null>;
+}
+
+export interface RemoteRtcMember extends RtcMember {
+  local: false;
+}
+
+export interface LocalRtcMember extends RtcMember {
+  local: true;
+  media$: Behavior<LocalMemberMedia | null>;
+  sharingScreen$: Behavior<boolean>;
+  /** Null when the platform cannot share a screen. */
+  toggleScreenSharing: (() => void) | null;
+  screenShareError$: Behavior<Error | null>;
+  dismissScreenShareError(): void;
+}
+
+// ---------------------------------------------------------------------------
+// Media
+
+export type MediaSource =
+  | "microphone"
+  | "camera"
+  | "screenShare"
+  | "screenShareAudio";
+
+export type MediaStreamStats =
+  | RTCInboundRtpStreamStats
+  | RTCOutboundRtpStreamStats
+  | undefined;
+
+/** One published track of a member. */
+export interface MediaTrack {
+  source: MediaSource;
+  kind: "audio" | "video";
+  /** Stable for the life of the track. */
+  id: string;
+  muted$: Behavior<boolean>;
+  /** False when the SFU reports the track as unencrypted. */
+  encrypted$: Behavior<boolean>;
+  /** Polled while subscribed. */
+  stats$: Behavior<MediaStreamStats>;
+  /**
+   * Rendering. The view hands its `<video>` or `<audio>` element over; the SDK
+   * sets its stream and, for video, registers the size and on-screen observers
+   * that pick a simulcast layer and pause the subscription while the element is
+   * hidden. `attach` is idempotent per element; `detach` has to be called
+   * before the element leaves the DOM so those observers are released.
+   */
+  attach(element: HTMLMediaElement): void;
+  detach(element: HTMLMediaElement): void;
+}
+
+export interface AudioMediaTrack extends MediaTrack {
+  kind: "audio";
+  /** Route playback through Web Audio, for earpiece pan and gain. Undefined resets. */
+  setAudioContext(ctx: AudioContext | undefined, plugins?: AudioNode[]): void;
+  setVolume(volume: number): void;
+}
+
+export interface VideoMediaTrack extends MediaTrack {
+  kind: "video";
+  /** Undefined for remote tracks. */
+  facingMode$?: Behavior<"user" | "environment" | undefined>;
+}
+
+export type EncryptionError = "MissingKey" | "InvalidKey";
+
+/**
+ * The media of one member, backed by a LiveKit participant inside the SDK.
+ * There is no identity field: the LiveKit identity is the member's `id`.
+ */
+export interface MemberMedia {
+  local: boolean;
+  speaking$: Behavior<boolean>;
+  screenShareEnabled$: Behavior<boolean>;
+
+  microphone$: Behavior<AudioMediaTrack | undefined>;
+  camera$: Behavior<VideoMediaTrack | undefined>;
+  screenShare$: Behavior<VideoMediaTrack | undefined>;
+  screenShareAudio$: Behavior<AudioMediaTrack | undefined>;
+
+  /** Emits when the SFU reports a key problem for this member. */
+  encryptionError$: Observable<EncryptionError>;
+}
+
+export interface LocalMemberMedia extends MemberMedia {
+  local: true;
+}
+
+// ---------------------------------------------------------------------------
+// Factory
+
+/**
+ * Takes the whole `MatrixClient` rather than a slice of it, and finds the
+ * MatrixRTC session itself. The js-sdk is the MatrixRTC implementation today;
+ * when the rust-rtc crate replaces it, the SDK has to bridge the crate to the
+ * js-sdk client, and only the SDK knows what that bridge needs. Holding the
+ * client keeps that change inside the SDK.
+ */
+export function createRtcSession(
+  scope: ObservableScope,
+  client: MatrixClient,
+  room: Room,
+  localMedia: LocalMediaInputs,
+  options: RtcSessionOptions,
+): RtcSession {
+  // The interface ships ahead of the implementation; see the module comment.
+  return {} as RtcSession;
+}

sdk/package.json

@@ -0,0 +1,37 @@
+{
+  "name": "@element-hq/matrixrtc-sdk",
+  "version": "0.0.0",
+  "description": "MatrixRTC sessions with LiveKit media, without Element Call's UI. Consumed straight from the repository as a git dependency (github:element-hq/element-call#<ref>&path:/sdk): the host's package manager runs `prepare`, which builds `dist/`.",
+  "license": "SEE LICENSE IN ../README.md",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/element-hq/element-call",
+    "directory": "sdk"
+  },
+  "type": "module",
+  "devEngines": {
+    "packageManager": {
+      "name": "pnpm"
+    }
+  },
+  "files": [
+    "dist"
+  ],
+  "main": "./dist/matrixrtc-sdk.js",
+  "module": "./dist/matrixrtc-sdk.js",
+  "types": "./dist/types/sdk/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/types/sdk/index.d.ts",
+      "default": "./dist/matrixrtc-sdk.js"
+    }
+  },
+  "scripts": {
+    "prepare": "cd .. && pnpm install --frozen-lockfile && pnpm build:sdk"
+  },
+  "peerDependencies": {
+    "livekit-client": "^2.18.1",
+    "matrix-js-sdk": "*",
+    "rxjs": "^7.8.1"
+  }
+}

sdk/pnpm-workspace.yaml

@@ -0,0 +1,14 @@
+# Makes `sdk/` a pnpm project of its own rather than a directory inside the
+# repository's workspace. That matters when a host installs the SDK straight
+# from this repository (`github:element-hq/element-call#…&path:/sdk`): pnpm
+# prepares such a dependency by running `pnpm install` in this directory, and
+# only a project root gets its `prepare` script (see package.json) run, which is
+# what builds `dist/`. Inside the repository's own workspace the install would
+# silently target the repository instead and build nothing.
+#
+# Consequence for development: pnpm commands run from within this directory see
+# this project, not the repository; run them from the repository root.
+
+# Nothing to install here: the peers in package.json are the host's, and the
+# build runs against the repository's own node_modules (see `prepare`).
+autoInstallPeers: false

sdk/tsconfig.build.json

@@ -0,0 +1,18 @@
+{
+  // Declaration output for the SDK package (`pnpm build:sdk:types`). The root
+  // tsconfig only type-checks; this one emits `.d.ts` files, and nothing else,
+  // for everything the entry point reaches. The layout under `dist/types`
+  // mirrors the repository (`sdk/index.d.ts`, `src/…`), which is what
+  // `package.json` points its `types` at.
+  "extends": "../tsconfig.json",
+  "compilerOptions": {
+    "noEmit": false,
+    "emitDeclarationOnly": true,
+    "declaration": true,
+    "declarationMap": false,
+    "rootDir": "..",
+    "outDir": "./dist/types"
+  },
+  "include": ["./index.ts", "../src/@types/*.d.ts"],
+  "exclude": []
+}