- A segmenter that can't be built, with the model unreachable or a GPU
its delegate refuses, took the camera down with the pipeline: the SDK
fails the track it was attaching to, and the lobby muted camera and
microphone, on every load while the choice was kept.
- One is now built on trial when an effect is first wanted, before the
pipeline is attached to anything, on the CPU where the GPU refuses.
Where it can't be built, the options are disabled with the existing
notice for the rest of the session. The choice is kept: a browser
update may fix what failed.
- The view model takes that verdict from the pipeline's state rather
than
asking the SDK, so both say the same, and it no longer builds a WebGL
context of its own to ask.
- Until the trial is over, the lobby opens no camera and the call makes
no
camera track, so neither starts without the effect.
Rebased onto main, where the Publisher is gone and the call view model
publishes the camera through the SDK:
- CallViewModel takes videoProcessorPreparing$. While it is true, the
camera's mute handler waits before making a track, and the join leaves
the camera out of its requests (initialPublishRequests); the view
model
publishes it once it is known whether the pipeline builds. The
Publisher's test became CallViewModel tests.
Cherry-picked from spec/background-effects/12-unbuildable-pipeline onto
toger5/background-effect/camera-preview, where the camera menu's preview
takes the camera from the footer's caller rather than from a track
synced
on the background-effects state. The footer factories take the
pipeline's
state again, as this commit needs its verdict and not only the settling
flag; the synced camera and the preview's wrapper type are not brought
along.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- A cross on the tile's top-right corner removes an added background,
critical-coloured at rest and named by its tooltip, as design draws it.
The one in force offers none, since taking it away would leave nothing
chosen; the view model refuses it too.
- The cross sits beside the item rather than in it, which would take the
press first and choose the tile, and isn't a control of its own, so the
menu keeps one keyboard model: Delete or Backspace on the tile does the
same, and the tile says so with aria-keyshortcuts.
- Removed from IndexedDB and its picture released. End to end on
Chromium and Firefox, a background removed by Delete is still gone
after a reload.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Five of one's own at most, so the grid is whole: at the limit the add
tile goes, and it comes back once one is removed. The store holds the
limit too.
- An added image is refused if it isn't an image, is animated, or can't
be decoded; reduced to 1920px on its longest edge; and laid on an opaque
ground, since a transparent pixel would be a hole in the background.
- Animation is read from the file, as a still and an animated WebP share
a type. Chromium has no selected track until the decoder's tracks are
ready; without waiting for them an animated GIF got through.
- A refusal says why, in place of the notice, and is scrolled into view.
It can be dismissed, is gone once the menu closes, and the next shows
even if it reads the same. What was in force stays.
- End to end on Chromium and Firefox, from what the device then keeps: an
animated GIF is refused with its message, a 4000x3000 image is kept at
1920x1440, and a half transparent one is kept opaque. All three fail
with the preparation bypassed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- An add tile after the pictures opens the file picker, and the image
chosen joins the backgrounds on offer without being put on; it is in
force once chosen, like a shipped one. The tile is a command rather
than a choice, so a menu item beside the radios, and it isn't offered
where effects can't run.
- Kept in IndexedDB on this device and never sent anywhere, behind a
small interface, so the unit tests use an in-memory store; the
IndexedDB store itself is tested against fake-indexeddb, a new dev
dependency.
- The picker takes the focus, which the menu reads as a click elsewhere,
so the menu holds itself open until a file is chosen or the picker is
cancelled.
- A chosen image the device no longer keeps, as after its storage is
cleared, falls back to no effect once the device has said what it
keeps.
- Switching is checked between any two kinds of effect. End to end on
Chromium and Firefox: an added image shows in the preview once chosen;
once it is in force, no request carries the file or the copy kept,
which a deliberate upload fails; chosen, it is in force again after a
reload, gone once storage is cleared, and can be added again.
- Kept as given here; what can be added is checked in the commit after.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The desktop rule from #4059 goes: now that the menu says where effects
run slowly, using them there is the user's choice. A phone on the fast
route held 99% of its frame rate and was refused; a desktop on the slow
one held 73% and was allowed.
- Asks @livekit/track-processors directly; the wrapper module goes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The first effect chosen in a session shows a spinner on its own tile,
in place of the tick, until a frame carrying it is drawn. Later changes
show nothing, having nothing to prepare.
- Ended by the frame, not the promises, which resolve while the segmenter
is still being built. The transformer reports the first frame that went
through the segmenter; one passed through untouched doesn't count.
- The pipeline publishes the wait with its state, and both footer view
models take it as an input.
- The lobby tests' provider mock gains the observable hook, built once: a
fresh observable per render rebuilt the footer on every render.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Where the browser has no MediaStreamTrackProcessor, effects draw each
frame through a canvas: slower, and the first build stalls the page.
They stay offered and choosable there, with a notice under the grid
that says so and describes the group.
- Decided in the view model, beside whether effects can run at all, so
the lobby and the call give the same answer.
- The notice stays within the menu's width however long the sentence,
which the story checks.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The camera menu's blur toggle becomes a Background effects section
under the cameras: no effect, blur and the two shipped backgrounds as
one radio choice of tiles, three across. It scrolls with the device
list, so it stays reachable in a short call, and choosing keeps the
menu open. The lobby gets it too, as both render the same footer.
- The footer's view model gives the options, the notice and the actions;
the footer only names them.
- Each tile is Compound's MenuItem as a menuitemradio, styled as a tile.
Compound makes it an item of the dropdown, so the arrow keys walk the
tiles with the device rows, and takes its tap in the phone drawer. A
tile that can't be chosen has no select handler, so the keyboard passes
over it.
- Where the browser can't run effects the tiles are shown, disabled and
explained by a notice the group is described by. No effect stays
choosable and shows as in force. The notice is in the scrolling list,
so a short call doesn't push the menu out of it.
- Section headings are raised, or the positioned tiles scroll over the
one they sit under; raised within the list only, so the menu's frame
stays over them. Tiles clip with overflow: clip; with hidden they are
scroll containers, which focus scrolls into view without their border.
- End to end on Chromium and Firefox: a shipped background chosen in the
lobby shows in the preview, and None after Blur draws nothing, counted
in WebGL draws (Blur: about 1400 a second on Chromium, 2300 on
Firefox).
- The Quick Audio Menu's test of the camera menu keeps its name and now
finds blur among the effects rather than as a toggle.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- A catalogue of what can be chosen: no effect, blur, and two shipped
backgrounds, the app's own gradients flattened onto its canvas colour.
Their ids are what a saved choice stores, so they are named for the
pictures.
- One setting for the choice, typed as the ids an effect can have and
defaulting to blur for anyone who had it on. A stored value we no
longer ship, or one that isn't even a string, reads as no effect: it
comes from storage another build or a hand may have written.
- The pipeline follows it, switching in place to a picture as readily as
to blur, and the camera menu's blur toggle reads and writes it.
- So does Settings' blur checkbox: a picture reads as blur off, and
checking it replaces the picture with blur. The old on/off setting is
only read, to carry blur over. First tests for the Settings modal,
which fail with the checkbox on the old setting.
- The transformer's own init loads a picture chosen before it was built,
as the base init does and it had not: the first effect always is, and
the picture was drawn black.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The controls and the pipeline each asked the SDK on their own whether
background processing is supported, so one could offer what the other
refused. One function answers both, and it asks the browser, kept to
desktops until the menu says what the effects cost elsewhere.
Ported from spec/background-effects/2-pipeline (#4282): f78e9c05 and
87638325, the tests on the lobby and call footers as they are.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Give the microphone level its own observable
* Add a microphone level meter
* Show speakers and microphones in the quick audio menu
* Wire speaker selection through the call footer
* Add stories for the meter and the device menu
* Add end-to-end specs for the quick audio menu
* Keep the footer while a menu opened from it is open
* Use the shared audio capture stub in the lobby test
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The view models reached for getUrlParams() — and so window.location —
from deep inside the call path: CallViewModel, MediaDevices, Publisher,
LocalMember and the footer view model. An embedded Element Call has no URL
of its own, so these values have to arrive as arguments instead.
Add the relevant options to CallViewModelOptions, to the MediaDevices and
Publisher constructors, to createLocalMembership$ and enterRTCSession, and
to createCallFooterViewModel. The remaining React consumers read the
context added in the previous commit. AppViewModel now takes its audio
output options too, moving that URL read out to main.tsx, where the app
shell can act as the adapter.
The new CallViewModelOptions fields are optional, defaulting to what the
URL parameters resolve to outside widget mode; the MediaDevices and
Publisher arguments are required, so that every construction site has to
be explicit.
useTheme.test.ts mocked the UrlParams module with a factory, so it needed
updating to mock the hook rather than getUrlParams.
No functional change.
On mobile, the ringing status indicator is supposed to display in the header rather than on a tile. The exact layout differs between Android and iOS. To get it right I had to refactor AppBar to use CSS grid templates.
(Also, I changed my mind about the exact ringing data I needed out of CallViewModel - sorry. A little move of the ringtone audio renderer into its own component was necessary to accommodate that.)