* Update ghcr.io/element-hq/element-web:develop Docker digest to e063472
* Fix tests
* Dismiss further toasts
* Close even more toasts
* Yet more toast fixes
* Toast fixes all over
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
- A remembered effect this browser can't run is cleared rather than kept,
as the menu already showed no effect in force there.
- A camera turned off and on again with an effect chosen gets the same
pipeline as its track arrives, neither rebuilt nor switched again.
- End to end, on Chromium and Firefox: the effect chosen before a call is
still chosen, and worn, in the next call after leaving; and a second
tab still shuts the first down with the existing "Opened in another
tab" error while an effect runs.
- Switching camera and sharing a screen are checked by hand: they need
two cameras and a screen, which the fake devices don't give yet.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The e2e check held back the model download at the join and required
one, since the pipeline used to be built again for the call's track.
The join now takes the lobby's running pipeline over, so no download
happens, and the check says so.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- The standalone app at five window sizes, from a large desktop to a
phone-shaped 360x640, on Chromium and Firefox: the camera menu stays
inside the window, which is the call area there, and the last effect can
still be scrolled to and chosen.
- As a component in a host page, and as a widget in Element Web: a
picture chosen reaches the preview, so the pipeline's model and
WebAssembly load from those builds, and in a small container the last
effect is reached and chosen.
- Compound portals the menu to the page and takes no container for it, so
there it has none of the component's scoped styles, as the microphone
menu has none.
- The colour checks the three share live in playwright/utils.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- A host in the call records every frame of every video track it receives
from the moment the track arrives. A guest with a picture chosen joins,
or turns the camera on in the call from off at joining or from off in
the call; none of the guest's frames may be closer to the camera image
than to the picture. A guest with no effect fails it.
- The preview's track stopping destroys the shared pipeline, so the call's
track builds it again. That build is held back as long as the slower
route takes, and publishing still waits for it; each check that holds
the model back also checks that it did.
- The first build survives the preview camera turned off and on while it
runs, which is what running builds one at a time is for.
- The camera's colour is taken once the preview shows it; taken earlier it
was black, which let frames of the room pass as the picture. The camera
checks wait for the call itself, as the lobby has a camera button of
its own.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The desktop rule from #4059 goes: now that the menu says where effects
run slowly, using them there is the user's choice. A phone on the fast
route held 99% of its frame rate and was refused; a desktop on the slow
one held 73% and was allowed.
- Asks @livekit/track-processors directly; the wrapper module goes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The first effect chosen in a session shows a spinner on its own tile,
in place of the tick, until a frame carrying it is drawn. Later changes
show nothing, having nothing to prepare.
- Ended by the frame, not the promises, which resolve while the segmenter
is still being built. The transformer reports the first frame that went
through the segmenter; one passed through untouched doesn't count.
- The pipeline publishes the wait with its state, and both footer view
models take it as an input.
- The lobby tests' provider mock gains the observable hook, built once: a
fresh observable per render rebuilt the footer on every render.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Where the browser has no MediaStreamTrackProcessor, effects draw each
frame through a canvas: slower, and the first build stalls the page.
They stay offered and choosable there, with a notice under the grid
that says so and describes the group.
- Decided in the view model, beside whether effects can run at all, so
the lobby and the call give the same answer.
- The notice stays within the menu's width however long the sentence,
which the story checks.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The camera menu's blur toggle becomes a Background effects section
under the cameras: no effect, blur and the two shipped backgrounds as
one radio choice of tiles, three across. It scrolls with the device
list, so it stays reachable in a short call, and choosing keeps the
menu open. The lobby gets it too, as both render the same footer.
- The footer's view model gives the options, the notice and the actions;
the footer only names them.
- Each tile is Compound's MenuItem as a menuitemradio, styled as a tile.
Compound makes it an item of the dropdown, so the arrow keys walk the
tiles with the device rows, and takes its tap in the phone drawer. A
tile that can't be chosen has no select handler, so the keyboard passes
over it.
- Where the browser can't run effects the tiles are shown, disabled and
explained by a notice the group is described by. No effect stays
choosable and shows as in force. The notice is in the scrolling list,
so a short call doesn't push the menu out of it.
- Section headings are raised, or the positioned tiles scroll over the
one they sit under; raised within the list only, so the menu's frame
stays over them. Tiles clip with overflow: clip; with hidden they are
scroll containers, which focus scrolls into view without their border.
- End to end on Chromium and Firefox: a shipped background chosen in the
lobby shows in the preview, and None after Blur draws nothing, counted
in WebGL draws (Blur: about 1400 a second on Chromium, 2300 on
Firefox).
- The Quick Audio Menu's test of the camera menu keeps its name and now
finds blur among the effects rather than as a toggle.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- A catalogue of what can be chosen: no effect, blur, and two shipped
backgrounds, the app's own gradients flattened onto its canvas colour.
Their ids are what a saved choice stores, so they are named for the
pictures.
- One setting for the choice, typed as the ids an effect can have and
defaulting to blur for anyone who had it on. A stored value we no
longer ship, or one that isn't even a string, reads as no effect: it
comes from storage another build or a hand may have written.
- The pipeline follows it, switching in place to a picture as readily as
to blur, and the camera menu's blur toggle reads and writes it.
- So does Settings' blur checkbox: a picture reads as blur off, and
checking it replaces the picture with blur. The old on/off setting is
only read, to carry blur over. First tests for the Settings modal,
which fail with the checkbox on the old setting.
- The transformer's own init loads a picture chosen before it was built,
as the base init does and it had not: the first effect always is, and
the picture was drawn black.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@livekit/track-processors skips a frame on
`isDisabled ?? backgroundDisabled`, and `isDisabled` is never
undefined, so a disabled effect that keeps its blur radius is still
segmented and drawn on every frame. Still the case in 0.8.2. The
override stays until that is fixed upstream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SDK now builds a processor chosen while the camera was off when the
camera turns on, so the call view model no longer unpublishes the
camera and publishes a new one. TrackSupply.replace, the record of
which camera took which processor, and followProcessor's applied
callback go with it.
Answers robintown's review on #4282 ("what's wrong with attaching
processors to already published camera tracks?").
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LiveKit stops a camera's track when it is turned off, and the SDK
refused a processor on a stopped track, so an effect chosen while the
camera was off was dropped.
LiveKit already restarts a track's processor before it hands the new
track to the sender. So the SDK now takes a processor on a camera that
is off and builds it in that restart: the first frame the room sees is
processed, and the track is unmuted rather than replaced. An adapter
let go before it was built leaves the processor alone, since the app
shares one pipeline across camera tracks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The controls and the pipeline each asked the SDK on their own whether
background processing is supported, so one could offer what the other
refused. One function answers both, and it asks the browser, kept to
desktops until the menu says what the effects cost elsewhere.
Ported from spec/background-effects/2-pipeline (#4282): f78e9c05 and
87638325, the tests on the lobby and call footers as they are.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A camera track made without the processor lets the room see the camera
until the processor is built. So the processor travels in the publish
request: the lobby opens the camera with it, the join publishes with it,
the room's capture defaults follow the requests made since the join, and
a camera turned off before the processor was chosen is replaced on turning
it on rather than unmuted, so that the new track starts with it.
Ported from spec/background-effects/2-pipeline (#4282): the publish half
of a91dab8f. The replacement lives in the mute switch's track supply,
which the call view model shares with the lobby.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Blur runs through one pipeline, shared by the pre-join preview and the
call: attached the first time an effect is wanted and switched in place
after, never detached. The logic is BackgroundEffects in src/state;
BackgroundEffectsContext only builds it, from the app's view model, so
the standalone app and the component hand the same instance to every
camera. Disabled drops the blur radius, so no effect costs no
segmentation. The SDK gains fromLivekitProcessor beside the adapter,
now toLivekitProcessor.
Ported from spec/background-effects/2-pipeline (#4282): a91dab8f with
its review tidy-ups 445e010c, bbc73d07, 51729bcf, f52d2b79, 813f1e6f
and 584bbaa0. Left out as already solved here: OneStepPipeline (the SDK
runs a processor's steps in turn) and the LiveKit-track sync helpers
(the lobby's camera is an SDK track, synced by followProcessor).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
join
The lobby captured its own LiveKit tracks and the join captured again:
two permission prompts, a grey tile while the camera restarted, and the
background pipeline built twice. The lobby now opens the devices on the
SDK's `LocalMedia`, follows the mute switches as they change (they start
off until the devices are enumerated), the device choice and the
processor through controls shared with the call view model, and the call
joins with that media, so the tile carries the very track the lobby
previewed.
`useBehavior` subscribes without `useSyncExternalStore`, which left a
view without a re-render for an emission React had been told about,
about one time in three, when its store had switched. The LiveKit
preview hook and the processor sync helpers go, with
`@livekit/components-react`.
Background effects run under the dev server: the node polyfill no longer
rewrites MediaPipe's WASM loader. Taken from fkwp's 2fad4e08 on
spec/background-effects/2-pipeline (#4282).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A lobby captures the microphone and the camera on the browser's devices
alone, with `createLocalMedia`, so that any backend can take them over:
`LocalMedia` hands out the same `LocalAudioMediaTrack` and
`LocalVideoMediaTrack` a call renders, with mute, device and facing
switches and the host's `VideoProcessor` run on the preview. Given to
`join` as `publish`, the tracks are moved rather than shared: the
publisher takes them over as it starts publishing, their lobby objects
are finished, and the LiveKit backend adopts the device tracks as its
own and the processor with its running output, so the join captures
nothing and initialises nothing again.
Also here, found on the way: a track already on the requested device is
left alone (LiveKit would restart it, always on Firefox, which echoes no
device id in `getConstraints()`), identical captures in flight are
coalesced (React's strict mode mounts twice), and every operation on a
processor takes its turn, since a stopped track tears its processor down
without waiting for the next track's init.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Storybook's react-docgen plugin parses .ts files with Babel's jsx plugin
enabled, so the angle-bracket assertion `<EncryptionSystem>useMemo(...)`
reads as an opening JSX tag and the Storybook build fails with
"Unexpected token". Passing the type as useMemo's generic is equivalent
and parses either way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Localazy project is about to hold the native iOS and Android call
screens' strings in a file of their own. Without a condition, the download
would copy that file into locales/, and the upload on every push to main
would send a stale copy back, overwriting edits made in Localazy.
Naming app.json in the upload patterns as well means a copy that arrives
another way, such as a local download from an older checkout, is never sent
either.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Johannes Marbach