The controls and the pipeline each asked the SDK on their own whether
background processing is supported, so one could offer what the other
refused. One function answers both, and it asks the browser, kept to
desktops until the menu says what the effects cost elsewhere.
Ported from spec/background-effects/2-pipeline (#4282): f78e9c05 and
87638325, the tests on the lobby and call footers as they are.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A camera track made without the processor lets the room see the camera
until the processor is built. So the processor travels in the publish
request: the lobby opens the camera with it, the join publishes with it,
the room's capture defaults follow the requests made since the join, and
a camera turned off before the processor was chosen is replaced on turning
it on rather than unmuted, so that the new track starts with it.
Ported from spec/background-effects/2-pipeline (#4282): the publish half
of a91dab8f. The replacement lives in the mute switch's track supply,
which the call view model shares with the lobby.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Blur runs through one pipeline, shared by the pre-join preview and the
call: attached the first time an effect is wanted and switched in place
after, never detached. The logic is BackgroundEffects in src/state;
BackgroundEffectsContext only builds it, from the app's view model, so
the standalone app and the component hand the same instance to every
camera. Disabled drops the blur radius, so no effect costs no
segmentation. The SDK gains fromLivekitProcessor beside the adapter,
now toLivekitProcessor.
Ported from spec/background-effects/2-pipeline (#4282): a91dab8f with
its review tidy-ups 445e010c, bbc73d07, 51729bcf, f52d2b79, 813f1e6f
and 584bbaa0. Left out as already solved here: OneStepPipeline (the SDK
runs a processor's steps in turn) and the LiveKit-track sync helpers
(the lobby's camera is an SDK track, synced by followProcessor).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
join
The lobby captured its own LiveKit tracks and the join captured again:
two permission prompts, a grey tile while the camera restarted, and the
background pipeline built twice. The lobby now opens the devices on the
SDK's `LocalMedia`, follows the mute switches as they change (they start
off until the devices are enumerated), the device choice and the
processor through controls shared with the call view model, and the call
joins with that media, so the tile carries the very track the lobby
previewed.
`useBehavior` subscribes without `useSyncExternalStore`, which left a
view without a re-render for an emission React had been told about,
about one time in three, when its store had switched. The LiveKit
preview hook and the processor sync helpers go, with
`@livekit/components-react`.
Background effects run under the dev server: the node polyfill no longer
rewrites MediaPipe's WASM loader. Taken from fkwp's 2fad4e08 on
spec/background-effects/2-pipeline (#4282).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A lobby captures the microphone and the camera on the browser's devices
alone, with `createLocalMedia`, so that any backend can take them over:
`LocalMedia` hands out the same `LocalAudioMediaTrack` and
`LocalVideoMediaTrack` a call renders, with mute, device and facing
switches and the host's `VideoProcessor` run on the preview. Given to
`join` as `publish`, the tracks are moved rather than shared: the
publisher takes them over as it starts publishing, their lobby objects
are finished, and the LiveKit backend adopts the device tracks as its
own and the processor with its running output, so the join captures
nothing and initialises nothing again.
Also here, found on the way: a track already on the requested device is
left alone (LiveKit would restart it, always on Firefox, which echoes no
device id in `getConstraints()`), identical captures in flight are
coalesced (React's strict mode mounts twice), and every operation on a
processor takes its turn, since a stopped track tears its processor down
without waiting for the next track's init.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Storybook's react-docgen plugin parses .ts files with Babel's jsx plugin
enabled, so the angle-bracket assertion `<EncryptionSystem>useMemo(...)`
reads as an opening JSX tag and the Storybook build fails with
"Unexpected token". Passing the type as useMemo's generic is equivalent
and parses either way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Localazy project is about to hold the native iOS and Android call
screens' strings in a file of their own. Without a condition, the download
would copy that file into locales/, and the upload on every push to main
would send a stale copy back, overwriting edits made in Localazy.
Naming app.json in the upload patterns as well means a copy that arrives
another way, such as a local download from an older checkout, is never sent
either.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
By keying the connections by the transport's resolved auth flow. This is important because connecting to the same SFU twice with the same participant ID will cause a 'duplicate participant ID' error that stops one of the connections.
The second connection only happens when the first one actually failed
on the WebSocket. connect() rejects after its socket has failed and the
engine is closed, so the retry never overlaps the failing socket.
This removes the notion of an auth flow fallback for now. We will no longer try the CS API when authenticating with transports that include both a `url` and `livekit_service_url`.
