Port of edf1feae from johannes/rtc-remote-publish onto the SDK split,
where
the MatrixRTC logic lives in sdk/ rather than src/state/CallViewModel/.
Where the original lands:
- getLocalTransport's remote fallback: the LiveKit backend's new
findRemotePublishTransport (sdk/src/media-backend/livekit/
RemotePublishTransport.ts), which tries the candidates from the
js-sdk's
getRemoteLivekitPublishCandidates oldest first and keeps the first
whose
homeserver issues a token with the publish grant.
discoverLocalTransport
asks the backend for it after the host's fallback, with a snapshot of
the
session's memberships from RTCParticipation, and throws
MatrixRTCTransportMissingError where that finds nothing too. Transport
discovery stays backend neutral: the MSC4195 token check is the
backend's.
- LocalMember's "no delegation on a remote SFU" and the custom
transport's
server_name: LivekitBackend.prepareLocalTransport locates the
transport on
the homeserver it names (serverNameOfTransport) and skips the
delegation
probe where that is not our own.
- SessionBehaviors' membership.getTransportServerName: the same helper
in
LivekitBackend.mediaFor.
- decodeSFUJWTPayload, SFUJWTPayload.video.canPublish and the exported
isCsApiUnavailable: sdk/src/media-backend/livekit/auth/.
- DeveloperSettingsTab, the widget capabilities, the locale string and
the
js-sdk pin (9405287c, 44.0.0): applied as is. The error boundary
snapshots follow the string.
Tests: the LocalTransport cases go to RemotePublishTransport.test.ts and
LocalTransport.test.ts, the LocalMember remote case and the
SessionBehaviors
server-name case to LivekitBackend.test.ts, plus the memberships
snapshot in
RTCParticipation.test.ts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The SDK MediaView renders a plain video element, so LiveKit's
data-lk-source attribute is gone. Expose data-source from the SDK track
and point the screen-share spec at it.
The SDK replaced `src/livekit` and the remote-member modules before that PR
landed on `main`, so its `src/livekit/auth/*` and `remoteMembers/ConnectionMap.ts`
have no consumer here: the same code lives under `sdk/src/media-backend/livekit`.
The imports of them that `main` added to the view model, the developer
settings tab and the test utils go with them.
The bumped js-sdk reports the legacy identity as rtcBackendIdentity for every membership, so a device that rejoins after an improper leave, which has two memberships until the stale one expires, produced two tiles with the same key and crashed the call view. The membership id tells them apart; the audio renderer keys by it too.
ploy previews run pr-deploy.yaml from main, which downloads
build-output-sdk,
so a renamed artifact can never be found from a pull request. Keep the
package
name and point build:sdk:production and build:sdk:development at the
CallViewModel bundle the preview is meant to carry.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MediaBackendPlan.md and MemberMediaPlan.md described changes that have
landed. SdkMigration.md keeps only what moved from src/ into sdk/, with
the
paths as they are today. README.md gains the onboarding notes on the
three
API files, and its example uses createRTCSlot, which replaced
createMatrixRTCClient.
Unit tests for the pieces that had none: which transport the local
member advertises (override, homeserver, fallback, none), what the js-sdk
join is called with, how a publish request becomes LiveKit capture and
publish options, and the room options the connection factory builds from
the initial requests and the media quality settings, including what a
resolved connection reports while it is up. The backend's `mediaFor` gains
the cases from Element Call's member matching tests: a member on another
transport, a member changing transport, and a member without one.
The publisher is driven through a real LocalParticipant whose device and
signalling calls are faked, so that LiveKit's own pending-publication
bookkeeping is part of what is tested: the cases ported from Element
Call's Publisher tests (one permission prompt for both devices, nothing
created while muted, the upstream held back until the member publishes,
a mute that lands while the track is still being created) plus the
publish, unpublish and destroy paths.
The local media runs on a fake connection map: publishing before and
after the connection exists, republishing on a new connection, mutes
remembered across connections, the first publish error kept, and the
output device applied to the current room and the ones to come.
The participation runs on a fake media backend: the join options, the
state machine through connecting, connected, reconnecting, failed and
left, publishing only while joined, the delayed leave delegation, the
member ids per mode, the key replay and the data channel's sender check.
The loop that drives `scope.reconcile` was marked as running until the
promise of its last run had settled, which is a few microtasks after the
loop itself had checked for new values. A value arriving in that window
found the loop "running", was left to it, and was never processed: a
connection appearing at that moment would never get its publisher.
The flag is now cleared the moment the loop ends, so such a value starts
a new loop. The regression test drives the window microtask by microtask.
How a LiveKit transport is authenticated with by shape and role, the server name on connections and the options that now take a Transport.
(cherry picked from commit 26d9e1b0ae1e1f1ba8c3689bd50ba2e34aa0e820)
compound-web 10.4.0 treats a PortalRoot whose root is null as one whose root
is not available yet, so floating-ui never mounts the portal and every tooltip
label, and with it every icon button's accessible name, is missing. The
component defaulted portalRoot to null and wrapped the call unconditionally.
Wrap the call only when the host supplied a root, keeping null as the
not-yet-available case Compound defines, and say so on the prop. Cover it in
CallPortalRoot.test.tsx. Drop the git allowBuilds entry for compound-web now
that it is a release again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The SDK now takes `publish` requests and audio output control directly
in the main options object, removing the separate `LocalMediaInputs`
type. Audio output device selection moves from a reactive behavior to an
imperative method on `MatrixRTCClient`, giving hosts direct control over
when devices switch.
The session speaks MatrixRTC and sees media through MediaBackend in
sdk/src/backend/api.ts. Everything LiveKit lives in
sdk/src/backend/livekit/,
the only place in the SDK that imports livekit-client; the import
boundary rule
now enforces that outside sdk/src/backend/ the LiveKit packages appear
in type
imports only.
The seam sits above the connection manager: one connection per transport
url
is the LiveKit multi-SFU strategy, not a MatrixRTC concept, so a
cascading SFU
or full mesh replaces the folder, not the session. A backend prepares
the local
transport the session discovered (the token exchange and the delegation
probe,
reported as canDelegateDelayedLeave), hands the delayed leave over per
delay
id, gives the local member one publish switch plus publish/unpublish,
builds a
remote member's media from its membership and registers that
membership's
transport for as long as the member lives, and reports its connections
for the
transport registry, which keys on the raw transport serialised with
sorted
keys. The backend's context carries no Matrix client (a backend picks
what it
calls in its own options), no membership list, and no js-sdk session:
media
keys arrive as a stream replayed on subscribe, and MatrixKeyProvider
subscribes
to it. The MatrixRTC mode no longer reaches LiveKit; it becomes a
tokenEndpoint
choice made where the default backend is built.
Two behaviour changes ride along. Publishing is one boolean, joined and
homeserver reachable, where before a sync hiccup after leave() resumed
the
upstream of tracks that were meant to stay silent. And
ResolvedConnection keeps
one ResolvedTransport object per fetch, so a state change no longer
reads as a
change of what was fetched.
MatrixRTCClientOptions gains backend?: MediaBackendFactory, defaulting
to
createLivekitBackend built from the client options.
authenticateWithTransport
takes tokenEndpoint instead of matrixRTCMode.
media, track, source, publish request, active and attach/detach, between
participant and transport, and the participant row names the adapter as
the only reader.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Chromium's fake microphone never registers as a speaker at the SFU, and Firefox's steady tone does so in about two runs out of three, with or without the browser's audio processing. The test stays, out of the gate, until the dev SFU gets a lower speaker threshold or the harness a louder fake device.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
membership$ is typed as RTCMembership, the subset of the js-sdk CallMembership that hosts read. UnknownCallError is UnknownRTCError throughout, not only in the export. applicationData reaches the membership whole, since the js-sdk takes it next to the intent. The plan records the three slices as implemented.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One way to put media on the session: publish(request) resolves with the local track once LiveKit has it, and unpublish(id) removes it. A request names a source with its device, processor and capture settings, so CaptureSettings on the client options goes, as do setMicrophoneEnabled and setCameraEnabled on the client and the four screen share fields on the local member; sharing is tracks$ having a screenShare track, and a capture failure rejects the promise. LocalMediaInputs becomes the list of initial requests, which also become the room defaults so that one permission prompt covers microphone and camera. The publisher keeps the requests across a reconnection and republishes what is not muted. The harness gains a Share screen button.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Our own tracks are LocalAudioMediaTrack and LocalVideoMediaTrack, with setEnabled and setDevice, and on video facingMode$, switchFacingMode and setProcessor. switchCamera on the local media goes, as do the device and processor behaviors in LocalMediaInputs: a host calls the track when its setting changes instead of the SDK watching behaviors it does not own. The input device ids and the processor stay as initial values for the tracks created at the join. setMicrophoneEnabled and setCameraEnabled stay for the case of no track yet.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Unit tests for tracks$ and trackBySource$, an end-to-end check of isActive$ through a mute cycle (skipped on Chromium, whose fake microphone the SFU never hears), the architecture document brought in line with the track list, and the changed behaviour recorded in the plan.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Update the public API to export the primitives and operators the call
model needs, and change `LocalMediaInputs` from observable flags to
immediate booleans with imperative setters. Add session timings, media
quality and capture settings as configuration options, and allow the
host to override the transport discovery with explicit URLs.
Set up `sdk/` as a package shaped exactly like `component/`: its own
package.json with peer dependencies, the pnpm workspace marker so a host can
install it as a git dependency, a build tsconfig for type emission, and a vite
lib config with an externals list that `pnpm lint:externals` now checks for
both library builds.
The package holds only the interface for now. `createRtcSession` returns an
empty object; `RtcSession`, `RtcMember`, `MemberMedia` and `TransportMetadata`
are the contract Element Call's `CallViewModel` will be rebuilt on (see
sdk-plan.md). The shared primitives are re-exported from `src` until the first
implementation slice moves them.
`sdk/dev` is the smallest consumer of the SDK: a plain page that logs in, joins
a room and renders one tile per member, served by `pnpm dev:sdk` on port 3002.
Playwright starts it as a third web server, and `playwright/sdk/smoke.spec.ts`
has two browsers join one session. It fails on the status line with the SDK's
own error until there is an implementation behind the interface.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The `sdk/` directory is about to become the MatrixRTC SDK package, so the
demo that lives there — one bundle of Element Call's CallViewModel for a plain
page or widget — moves aside, unchanged. It keeps building under a script and
vite config of its own, `pnpm build:sdk-target-based-on-call-view-model`, into
its own `dist` rather than the repository root's, and no longer copies the
developer's config.json out of `public`. Its README says which script to run
and where the output lands.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Compound's Tooltip floats its bubble through floating-ui's FloatingPortal,
into the body: outside the element the component build confines its
stylesheet to, so a label tooltip shows as bare text at all times. An
enclosing FloatingPortal rooted at the container makes the nested portals
land inside it instead.
The call goes in an isolating wrapper there, standing in for the standalone
page's #root, so that its footer layer no longer paints above the modals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`downloadMedia` exists because a widget's client cannot fetch media: it
has no access token, and its only route to a file is MSC4039 over the
widget API. A component host hands over a client that does hold
credentials, so Element Call fetches media with it and the callback was
never needed there. It comes off `ElementCallHostBridge`, leaving a host
with only what it must understand, and stays on the internal HostBridge
for the widget.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The theme is state — what Element Call should look like right now — and
so belongs beside `language` as a prop, not on the imperative handle
(where it was a request, `setTheme`, because the internal host bridge
speaks the widget API and a widget's host sends theme changes as
requests) and not in the configuration (where `config.theme` only ever
set the starting theme).
The `theme` prop feeds the same channel the rest of Element Call listens
to for a host's theme, replayed so that whatever subscribes after the
host has set it still hears the current one. Changing it re-themes the
container and nothing else; unlike the language, it is per component.
`setTheme` and `config.theme` are gone, and the harness gets a theme
picker in place of its per-pane buttons.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Timo K.