Compact AGENTS.md and split hand-off out into a skill

Update ghcr.io/element-hq/element-web:develop Docker digest to 8dc3767 (#4447 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-11-13 01:12:29 +00:00 · 2026-11-11 16:33:46 +01:00 · 2026-11-11 14:47:22 +00:00 · 2026-11-11 15:37:31 +01:00 · 2026-11-11 15:29:03 +01:00 · 2026-11-11 13:53:30 +01:00
78 changed files with 1180 additions and 537 deletions
@@ -0,0 +1,35 @@
+---
+name: handoff
+description: Implement the first cut of a task and stop for a direction check before the quality pass.
+argument-hint: <the task>
+disable-model-invocation: true
+---
+
+Task: $ARGUMENTS
+
+Stop after the first implementation and hand it back before the quality pass.
+
+## Before the hand-off
+
+- Run the narrowest check that rules out an obviously broken hand-off: the test
+  file for the touched code, `tsc` on its own, a single story. Not the full
+  `pnpm lint`, not the whole suite.
+- If a wider check is needed before feedback can be useful, keep it to the touched
+  code and say why it was needed.
+- Leave wider refactors, extra tests and documentation polish out. They are
+  follow-up work, not part of the first hand-off.
+
+## The hand-off
+
+Stop and ask whether the direction is right. The message says:
+
+- what was built and where, in a few lines,
+- which check ran and what it returned, verbatim if it failed,
+- what was left out on purpose,
+- the decisions that could have gone another way.
+
+Do not continue past this point on your own.
+
+## After the user confirms
+
+The skill ends here. Continue with the quality pass as usual.
@@ -14,7 +14,7 @@ permissions:
 jobs:
  prevent-blocked:
    name: Prevent blocked
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    permissions:
      pull-requests: read
    steps:
@@ -17,7 +17,7 @@ env:
 jobs:
  build_and_deploy:
    name: Build & publish docker
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    permissions:
      contents: write # required to upload release asset
      packages: write # needed for publishing packages to GHCR
@@ -29,7 +29,7 @@ on:
 jobs:
  build:
    name: Build Element Call
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    steps:
      - name: Checkout code
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
@@ -25,7 +25,7 @@ jobs:
    # Deploy main branch to call.element.dev after build completes
    if: github.ref == 'refs/heads/main'
    needs: build_full_element_call
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    steps:
      - name: Deploy to call.element.dev
        uses: actions/github-script@d7906e4ad0b1822421a7e6a35d5ca353c962f410 # v6
@@ -92,7 +92,7 @@ jobs:
  build_storybook:
    name: Build Storybook
    if: contains(github.event.pull_request.labels.*.name, 'storybook build')
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    steps:
      - name: Checkout code
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
@@ -13,7 +13,7 @@ permissions:

 jobs:
  pr-changelog-label:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    steps:
      - uses: yogevbd/enforce-label-action@a3c219da6b8fa73f6ba62b68ff09c469b3a1c024 # 2.2.2
        with:
@@ -13,7 +13,7 @@ jobs:
    name: Review
    # Only run on labelling and skip forks.
    if: github.event.label.name == 'Claude-Review' && github.event.pull_request.head.repo.full_name == github.repository
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    steps:
      - name: Checkout code
        uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
@@ -32,7 +32,7 @@ on:

 jobs:
  deploy:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    permissions:
      deployments: write
    environment: Netlify
@@ -10,7 +10,7 @@ on:
 jobs:
  lint:
    name: Lint, format & type check
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    steps:
      - name: Checkout code
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
@@ -17,7 +17,7 @@ jobs:
      github.event.workflow_run.conclusion == 'success' &&
      github.event.workflow_run.event == 'pull_request' &&
      github.event.workflow_run.head_repository.full_name == github.repository
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    outputs:
      pr_number: ${{ steps.prdetails.outputs.pr_id }}
      pr_data_json: ${{ steps.prdetails.outputs.data }}
@@ -17,7 +17,7 @@ permissions: {}
 jobs:
  versioning:
    name: Versioning
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    outputs:
      DRY_RUN: ${{ steps.dry_run.outputs.DRY_RUN }}
      PREFIXED_VERSION: ${{ steps.prefixed_version.outputs.PREFIXED_VERSION }}
@@ -59,7 +59,7 @@ jobs:
  publish_npm:
    needs: versioning
    name: Publish NPM
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    outputs:
      ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
    permissions:
@@ -111,7 +111,7 @@ jobs:
    needs: [versioning, publish_npm]
    if: ${{ needs.versioning.outputs.DRY_RUN == 'false' }}
    name: Update release notes
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    # Shared with publish-embedded-packages.yaml, which appends to the same release: two
    # appends at once would each write back the body without the other's section
    concurrency:
@@ -14,7 +14,7 @@ on:
 jobs:
  versioning:
    name: Versioning
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    outputs:
      DRY_RUN: ${{ steps.dry_run.outputs.DRY_RUN }}
      PREFIXED_VERSION: ${{ steps.prefixed_version.outputs.PREFIXED_VERSION }}
@@ -76,7 +76,7 @@ jobs:
    needs: [build_element_call, versioning]
    if: always()
    name: Publish tarball
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    permissions:
      contents: write # required to upload release asset
    steps:
@@ -107,7 +107,7 @@ jobs:
    needs: [build_element_call, versioning]
    if: always()
    name: Publish NPM
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    outputs:
      ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
    permissions:
@@ -152,7 +152,7 @@ jobs:
    needs: [build_element_call, versioning]
    if: always()
    name: Publish Android AAR
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    outputs:
      ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
    permissions:
@@ -212,7 +212,7 @@ jobs:
    needs: [build_element_call, versioning]
    if: always()
    name: Publish SwiftPM Library
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    outputs:
      ARTIFACT_VERSION: ${{ steps.artifact_version.outputs.ARTIFACT_VERSION }}
    permissions:
@@ -282,7 +282,7 @@ jobs:
    needs: [versioning, publish_npm, publish_android, publish_ios]
    if: always()
    name: Update release notes
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    # Shared with publish-component.yaml, which appends to the same release: two appends
    # at once would each write back the body without the other's section
    concurrency:
@@ -23,7 +23,7 @@ jobs:
    needs: build_element_call
    if: always()
    name: Publish tarball
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    permissions:
      contents: write # required to upload release asset
      packages: write
@@ -68,7 +68,7 @@ jobs:
  add_docker_release_note:
    needs: publish_docker
    name: Add docker release note
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    steps:
      - name: Add release note
        uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2
@@ -25,7 +25,7 @@ on:
 jobs:
  vitest:
    name: Run unit tests
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    steps:
      - name: Checkout code
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
@@ -71,10 +71,27 @@ jobs:
          flags: unittests
          fail_ci_if_error: true
  playwright:
-    name: Run end-to-end tests
+    name: Run end-to-end tests (${{ matrix.part }})
    if: ${{ !inputs.disable_e2e_tests }}
    timeout-minutes: 60
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
+    # We manually shard the tests across runners. Playwright's own shards cut
+    # the list of tests into equal counts in file order, which would leave the
+    # widget suite, the slowest tests by far, together at the end.
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - part: chromium-1
+            args: --project=chromium --shard=1/2
+          - part: chromium-2
+            args: --project=chromium --shard=2/2
+          - part: widget-1
+            args: --project=widget --shard=1/2
+          - part: widget-2
+            args: --project=widget --shard=2/2
+          - part: firefox-mobile
+            args: --project=firefox --project=mobile
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
        with:
@@ -107,17 +124,64 @@ jobs:
        run: pnpm playwright install --with-deps
      - name: Run backend components
        run: |
+          # Make sure the directory for the embedded build exists as Docker might
+          # create it with wrong access rights otherwise.
+          mkdir -p embedded/web/dist
          docker compose -f docker-compose-dev.yml -f docker-compose-playwright.yml pull
          docker compose -f docker-compose-dev.yml -f docker-compose-playwright.yml up -d
          docker ps
      - name: Run Playwright tests
        env:
          USE_DOCKER: 1
-        run: pnpm exec playwright test
+          ARGS: ${{ matrix.args }}
+        # Unquoted on purpose: the part's arguments are several words
+        run: pnpm exec playwright test $ARGS
      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        if: ${{ !cancelled() }}
+        with:
+          name: blob-report-${{ matrix.part }}
+          path: blob-report
+          if-no-files-found: error
+          # Only read by the merge below.
+          retention-days: 1
+
+  e2e_report:
+    # Combines the blobs into a single HTML report and blocks PRs on failure.
+    name: Run end-to-end tests
+    needs: playwright
+    if: ${{ always() && !inputs.disable_e2e_tests }}
+    runs-on: ubuntu-24.04
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+        with:
+          # Inside a called workflow, github.repository is the caller so we need to make sure to still check out the EC repo here.
+          repository: ${{ inputs.matrix-js-sdk-sha && 'element-hq/element-call' || github.repository }}
+          persist-credentials: false
+      - name: Enable Corepack
+        run: corepack enable
+      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
+        with:
+          cache: "pnpm"
+          node-version-file: ".node-version"
+      - name: Install dependencies
+        run: scripts/layered.sh --ignore-pnpmfile
+        env:
+          PR_HEAD_REPO_OWNER: ${{ github.event.pull_request.head.repo.owner.login }}
+          JS_SDK_GITHUB_BASE_REF: ${{ inputs.matrix-js-sdk-sha }}
+      - name: Download the parts' reports
+        uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
+        with:
+          pattern: blob-report-*
+          path: all-blob-reports
+          merge-multiple: true
+      - name: Merge into one HTML report
+        run: pnpm exec playwright merge-reports --reporter html ./all-blob-reports
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        with:
          name: html-report
          path: playwright-report
          if-no-files-found: error
          retention-days: 4
+      - name: Fail on a failed part
+        if: ${{ needs.playwright.result != 'success' }}
+        run: exit 1
@@ -7,7 +7,7 @@ on:

 jobs:
  download:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    permissions:
      contents: read

@@ -8,7 +8,7 @@ on:

 jobs:
  upload:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    permissions:
      contents: read

@@ -10,7 +10,7 @@ permissions: {}
 jobs:
  zizmor:
    name: Run zizmor 🌈
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-24.04
    permissions:
      security-events: write # Required for upload-sarif (used by zizmor-action) to upload SARIF files.
    steps:
@@ -44,3 +44,7 @@ storybook-static

 # Nested checkout made by scripts/layered.sh
 /matrix-js-sdk/
+
+# Claude
+/.claude/*
+!/.claude/skills/
@@ -16,19 +16,35 @@ host's page. It is the MatrixRTC reference implementation.
  `@media`, global `i18next` or the `widget` global — take it from a provider.
 - The change works standalone, as a widget and as a component. Say what you checked.
 - Every gate below is green before you push.
- Hand off after the first implementation, before the quality pass. Then commit on
-  the user's word where the setup allows it, or hand them the message.

 ## Read before you

 | …                                | …                                                          |
 | -------------------------------- | ---------------------------------------------------------- |
-| start a task of any size         | [docs/agents/workflow.md](docs/agents/workflow.md)         |
 | touch call logic or a view model | [docs/agents/architecture.md](docs/agents/architecture.md) |
 | write any code                   | [docs/agents/code-style.md](docs/agents/code-style.md)     |
 | write a test, story or e2e spec  | [docs/agents/testing.md](docs/agents/testing.md)           |
 | open a PR                        | [CONTRIBUTING.md](CONTRIBUTING.md)                         |
-| implement from a feature spec    | [FEATURES_SPEC/AGENTS.md](FEATURES_SPEC/AGENTS.md)         |
+
+## Scratch files live in `agent-workspace/`
+
+Git-ignored. One kebab-case subfolder per task, matching the branch topic:
+`agent-workspace/<slug>/`.
+
+| File                        | Purpose                                                                          |
+| --------------------------- | -------------------------------------------------------------------------------- |
+| `plan.md`                   | The concrete approach — files, symbols, edit order — once direction is confirmed |
+| `commit-msg.txt`            | `git commit -F agent-workspace/<slug>/commit-msg.txt`                            |
+| `pr-body.md`                | `gh pr create --body-file agent-workspace/<slug>/pr-body.md`                     |
+| `implementation-summary.md` | What was built, decisions, trade-offs                                            |
+| `NN-prompt.md`              | Raw prompt text, numbered, when worth keeping                                    |
+
+- Nothing here is durable. The folder is ignored, so anything worth keeping is
+  promoted into the PR body or a doc before the task ends.
+- Never write scratch files to the repo root. Stray `load_test_summary.md` and
+  `config.json_` files are what this folder prevents.
+- Start `pr-body.md` from `.github/PULL_REQUEST_TEMPLATE.md` and fill every
+  section.

 ## Gates

@@ -42,6 +58,11 @@ pnpm i18n:check
 `knip` fails on dead code; a file deliberately inert ahead of its consumer goes in
 `knip.ts` `ignoreFiles` with a reason.

+Ready means every gate is green and the change is covered at the layers
+[testing.md](docs/agents/testing.md) asks for. Read the diff against
+[code-style.md](docs/agents/code-style.md) first. Re-run the whole checklist after
+any fix.
+
 ## A good PR

 - One slice, under ~400 changed lines, green on its own, linked to its issue.
@@ -221,119 +221,68 @@ See also:

 - [Developing with linked packages](./docs/linking.md)

-#### Element Call as a component (experimental)
+#### Element Call as a component

-Element Call can also be embedded directly into another React application
-rather than being loaded in an iframe as a widget. `pnpm build:component`
-builds it as a library into `component/dist` (the bundle, its stylesheet and
-type declarations), and
+Element Call can be rendered inside another React app instead of an iframe.
+The host supplies its own Matrix client, and the call stays inside the element
+it is mounted in.

-```sh
-pnpm dev:component
-```
-
-serves a harness on port 3001 that stands in for such an application: it signs
-in twice against the development backend and shows two calls side by side, in
-resizable boxes, with page furniture of its own around them. Use it to see how
-Element Call behaves when it does not own the page — the size it is given,
-whether it stays inside its container, and what it says to its host, which is
-logged along the bottom. The harness is served with the same development
-certificate as the app, so unless the development CA is trusted, the browser
-needs a certificate exception for `https://localhost:3001` as well (see the
-note under [Backend](#backend)). It reads the same `public/config.json` as
-`pnpm dev` if one exists, and runs with Element Call's defaults otherwise.
-
-The call lays itself out for the size of the element it is mounted in, not the
-window: a host that shrinks the container to a corner of its page gets the
-picture-in-picture layout, just as a host that shrank the whole iframe used to.
-The breakpoints in the stylesheets the component uses are
-`@container element-call` queries against its root element for the same reason;
-for the standalone app the root is the page, so they mean what the media queries
-they replaced did. (The standalone-only views, such as the home and login pages,
-still use plain media queries, since the component never shows them.)
-
-The component's stylesheet is confined to the element it is mounted in: the
-build rewrites every selector so that it matches only Element Call's root or
-what is inside it, with `html`, `body` and `:root` standing for that root (see
-`component/build/scopeStylesToRoot.ts`). A host's own page keeps its styles,
-and Element Call brings its own fonts and design tokens along.
-
-The component speaks every language the app does. English is bundled in; the
-other locales are split into chunks the host's bundler loads the first time
-they are needed. It starts in the browser's language, and follows the host's
-own language setting through the `language` prop (`supportedLanguages` lists
-the tags it accepts, and anything else falls back to its base language or to
-English). The `theme` prop works the same way and takes the same values as the
-widget's `theme` URL parameter: `light`, `dark`, `light-high-contrast` or
-`dark-high-contrast`. Both can change while a call is running without
-disturbing it.
-
-A host must call and await `initializeElementCall(config)` once before
-rendering the component: it loads the `Intl` polyfills, applies the
-deployment-wide `config.json`-style configuration and sets up translations.
-The component itself takes the host's `client` and the `roomId` to call in, an
-`intent` saying what the user asked for (which decides whether to show the
-lobby, ring, and so on), an optional `config` overriding what the intent
-implies, and an optional `hostBridge` through which Element Call tells the host
-that the user has joined or hung up, that it wants to stay on screen, and so
-on. The host makes its own requests (`join`, `hangUp`, `setDeviceMute`) through
-the handle exposed on `ref`. The full API is documented in the type declarations
-(`component/index.tsx` and `component/host.ts`).
-
-A few things differ from the widget on purpose: the component draws a solid
-background rather than a gradient unless told otherwise, never offers to edit
-the user's profile (the account is the host's), scopes its keyboard shortcuts to
-its own root element so that several instances can share a page, and only shows
-its own post-call and error screens when the host has not supplied a `close()`
-callback; with one, it asks the host to unmount it instead. The
-[global JS controls](./docs/controls.md) on `window` are unchanged and remain
-page-wide, so with several instances on one page they apply to all of them.
-
-Each release publishes the package to npm, with the same version as the release:
+##### Install

 ```sh
 npm install @element-hq/element-call-component
 ```

-To use an unreleased commit, a host can instead install it as a git dependency
-on the `component` directory of this repository,
+Releases publish the package with the same version as Element Call. For an
+unreleased commit, depend on this repository directly:

 ```json
 "@element-hq/element-call-component": "github:element-hq/element-call#main&path:/component"
 ```

-whose `prepare` script runs the build on install. That build needs pnpm (via
-Corepack) on the host's machine, runs a full `pnpm install` of this repository,
-since it inherits the `--max-old-space-size` setting of
-the app build; the host's pnpm also has to allow it to run at all
-(`allowBuilds` in its `pnpm-workspace.yaml`). Note that `component/` is a pnpm
-project of its own for this reason, so pnpm commands run from inside that
-directory target it rather than the repository; run them from the repository
-root. The host imports the component from
-`@element-hq/element-call-component` and the stylesheet from
-`@element-hq/element-call-component/style.css`, and has to provide `react`,
-`react-dom`, `matrix-js-sdk`, `@vector-im/compound-web` and
-`@vector-im/compound-design-tokens` itself, since the bundle leaves them
-external.
-Compound's stylesheet are the host's to load too: the component shares
-the host's copy of Compound, (required for example for tooltips)
+This builds on install, so the host needs pnpm (via Corepack) and must allow
+the build (`allowBuilds` in its `pnpm-workspace.yaml`).

-The component is large, and a host will usually load it lazily, only once a
-call is shown. Everything a host needs in order to talk about a call before
-then is also exported
-from `@element-hq/element-call-component/api` (a small entry point that does not
-load the component).
+##### Use

- props and the types they are made of
- the handle
- the host bridge
- `UserIntent`
- `HeaderStyle`
- `BackgroundStyle` enums
- `configurationForIntent`, the defaults each intent implies
-  Import from it wherever a value such as
-  `BackgroundStyle.Solid` is needed on a path that must stay light; the main
-  entry point re-exports all of it too.
+1. Provide the externals yourself: `react`, `react-dom`, `matrix-js-sdk`,
+   `@vector-im/compound-web`, `@vector-im/compound-design-tokens`, and
+   Compound's stylesheet.
+2. Import `@element-hq/element-call-component/style.css`.
+3. Await `initializeElementCall(config)` once before rendering.
+4. Render the component with `client`, `roomId` and an `intent`; optionally
+   pass `config`, `hostBridge`, `language` and `theme` (both can change
+   mid-call). Control it through the `ref` handle (`join`, `hangUp`,
+   `setDeviceMute`).
+
+Load the component lazily. Types, enums and `configurationForIntent` are
+available without it from `@element-hq/element-call-component/api`. The full
+API is in `component/index.tsx`, `component/api.ts` and `component/host.ts`.
+
+##### How it differs from the widget
+
+- It lays out for its container, not the window: a small container gets the
+  picture-in-picture layout.
+- Its styles and keyboard shortcuts are scoped to its root, so the host page is
+  untouched and several instances can share a page. The
+  [global JS controls](./docs/controls.md) stay page-wide.
+- It draws a solid background by default and never offers profile editing.
+- If the host bridge implements `close()`, it asks to be unmounted instead of
+  showing its own post-call and error screens.
+- Sounds, images and the MediaPipe runtime are separate files the host's
+  bundler copies (webpack 5, Vite, Rollup); MediaPipe is fetched only when a
+  background effect is first used.
+
+##### Develop
+
+- `pnpm build:component` builds the library into `component/dist`.
+- `pnpm dev:component` serves a test host on `https://localhost:3001`: two
+  calls side by side in resizable boxes, with host messages logged at the
+  bottom. It uses the development backend, `public/config.json` if present,
+  and needs the same certificate exception as `pnpm dev` (see
+  [Backend](#backend)).
+- Run pnpm commands from the repository root; `component/` is its own pnpm
+  project.

 ### Backend

@@ -19,7 +19,12 @@ services:
      - ./backend/playwright_homeserver-othersite.yaml:/data/cfg/homeserver.yaml:Z
  element-web:
    # Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
-    image: ghcr.io/element-hq/element-web:develop@sha256:e063472b6bdf6d0bfa1d43dfa4ef7effee1efd2c9684daee665d1bedfe746a47
+    image: ghcr.io/element-hq/element-web:develop@sha256:72791f0b2541208f8f7edd9c6b5a7eaf88accd8ed1f0c4db1bc62bfe18eb90fe
  element-web-1:
    # Pin to a SHA so that upstream cannot break our tests. Renovate handles regular updates.
-    image: ghcr.io/element-hq/element-web:develop@sha256:e063472b6bdf6d0bfa1d43dfa4ef7effee1efd2c9684daee665d1bedfe746a47
+    image: ghcr.io/element-hq/element-web:develop@sha256:72791f0b2541208f8f7edd9c6b5a7eaf88accd8ed1f0c4db1bc62bfe18eb90fe
+    volumes:
+      # The other site opens the embedded package under test in place of the
+      # copy its image bundles (the first site opens the dev server's app.
+      # Built by the Playwright run (see build.setup.ts).
+      - ./embedded/web/dist:/app/widgets/element-call:ro,Z
@@ -14,7 +14,11 @@ This folder contains documentation for setup, usage, and development of Element

 Entry point: [AGENTS.md](../AGENTS.md).

- [Agent workflow](./agents/workflow.md)
 - [Architecture](./agents/architecture.md)
 - [Code style](./agents/code-style.md)
 - [Testing](./agents/testing.md)
+
+Opt-in skills in [`.claude/skills/`](../.claude/skills/):
+
+- `/handoff <task>` — get the first implementation back for a direction check
+  before lint, the full suite and polish.
@@ -34,6 +34,10 @@ translations. The browser is Playwright's, so a fresh clone needs

 ## End-to-end

+The Element Web instance `HOST1` opens the app the dev server serves. The instance
+`HOST2` (the other site) opens the embedded package as a host ships it. This package
+is built by the `build-embedded` setup project and mounted in `docker-compose-playwright.yml`.
+
 - `playwright/*.spec.ts` — standalone, on Chromium and Firefox.
 - `playwright/widget/` — widget mode, where cross-context bugs surface.
 - `playwright/component/` — the component in a host page, via the harness on port
@@ -1,50 +0,0 @@
-# Agent workflow
-
-## Scratch files live in `agent-workspace/`
-
-Git-ignored. One kebab-case subfolder per task, matching the branch topic:
-`agent-workspace/<slug>/`.
-
-| File                        | Purpose                                                                          |
-| --------------------------- | -------------------------------------------------------------------------------- |
-| `plan.md`                   | The concrete approach — files, symbols, edit order — once direction is confirmed |
-| `commit-msg.txt`            | `git commit -F agent-workspace/<slug>/commit-msg.txt`                            |
-| `pr-body.md`                | `gh pr create --body-file agent-workspace/<slug>/pr-body.md`                     |
-| `implementation-summary.md` | What was built, decisions, trade-offs                                            |
-| `NN-prompt.md`              | Raw prompt text, numbered, when worth keeping                                    |
-
- Nothing here is durable. The folder is ignored, so anything worth keeping is
-  promoted into the PR body, a doc, or a feature spec before the task ends.
- Never write scratch files to the repo root. Stray `load_test_summary.md` and
-  `config.json_` files are what this folder prevents.
- Start `pr-body.md` from `.github/PULL_REQUEST_TEMPLATE.md` and fill every
-  section.
- Commit subjects are plain imperative English. No conventional-commits prefixes.
- A `plan.md` and a feature spec sit at different altitudes; neither replaces the
-  other. A spec in `FEATURES_SPEC/` is durable and deliberately abstract —
-  behaviour, decisions, acceptance criteria — so the feature can be rebuilt against
-  a `main` nobody has seen yet. A plan is one slice of it landing on today's
-  `main`: the paths, symbols and edit order the spec's decisions and criteria must
-  not name. `FEATURES_SPEC/AGENTS.md` wins wherever the two genuinely overlap.
-
-## Hand off before the quality pass
-
- Implement the change, run the narrowest check that rules out an obviously broken
-  handoff, then stop and ask whether the direction is right.
- Full `pnpm lint`, the whole suite, coverage and benchmarks come after the
-  direction is confirmed.
- Wider refactors, extra tests and documentation polish are follow-up work, not
-  part of the first handoff.
- If a check is needed before feedback, keep it to the touched code and say why.
-
-## Commit and PR readiness
-
- Commit once the user confirms direction, or asks for one. Not before. Where the
-  setup has no git identity, a sandbox included, write the message to
-  `commit-msg.txt` and hand it over instead.
- Ready means every gate in [AGENTS.md](../../AGENTS.md#gates) is green and the
-  change is covered at the layers [testing.md](./testing.md) asks for. Read the diff
-  against [code-style.md](./code-style.md) first.
- Re-run the whole checklist after any fix. Only on green, either way.
- Once a human has started reviewing, fix forward — never force-push a
-  regeneration over a review in progress.
@@ -107,7 +107,7 @@
    "livekit-client": "^2.18.1",
    "lodash-es": "^4.17.21",
    "loglevel": "^1.9.1",
-    "matrix-js-sdk": "github:matrix-org/matrix-js-sdk#a9ef09fbf2f1644c8663d0aeaf6a921cd16e7b01",
+    "matrix-js-sdk": "github:matrix-org/matrix-js-sdk#dc8610e796c92553c658300a4882dde961a0678a",
    "matrix-widget-api": "^1.18.0",
    "node-stdlib-browser": "^1.3.1",
    "normalize.css": "^8.0.1",
@@ -26,6 +26,19 @@ process.env.NODE_EXTRA_CA_CERTS = join(
  "backend/dev_tls_local-ca.crt",
 );

+const desktopChrome = {
+  ...devices["Desktop Chrome"],
+  permissions: ["clipboard-write", "clipboard-read", "microphone", "camera"],
+  ignoreHTTPSErrors: true,
+  launchOptions: {
+    args: [
+      "--use-fake-ui-for-media-stream",
+      "--use-fake-device-for-media-stream",
+      "--mute-audio",
+    ],
+  },
+};
+
 /**
 * See https://playwright.dev/docs/test-configuration.
 */
@@ -43,11 +56,11 @@ export default defineConfig({
   * Reporter to use. See https://playwright.dev/docs/test-reporters
   *
   * On CI, `list` prints a line per test as it finishes, which keeps the job
-   * log moving; `github` annotates failures on the PR; `html` is uploaded as
-   * an artifact and must never try to open a browser there.
+   * log moving; `github` annotates failures on the PR; the `blob` format is
+   * uploaded by sharded CI runs and combined into a HTML report by the workflow.
   */
  reporter: process.env.CI
-    ? [["list"], ["github"], ["html", { open: "never" }]]
+    ? [["list"], ["github"], ["blob"]]
    : [["list"], ["html"]],
  /* Stop a broken run early on CI instead of waiting out every retry. */
  maxFailures: process.env.CI ? 5 : undefined,
@@ -63,29 +76,28 @@ export default defineConfig({
  },
  /* Configure projects for major browsers */
  projects: [
+    {
+      // Builds the embedded package into the directory the other site's
+      // Element Web container serves it from (see docker-compose-playwright.yml).
+      name: "build-embedded",
+      testMatch: "**/widget/embedded/build.setup.ts",
+    },
    {
      name: "chromium",
-      testIgnore: "**/mobile/**",
-      use: {
-        ...devices["Desktop Chrome"],
-        permissions: [
-          "clipboard-write",
-          "clipboard-read",
-          "microphone",
-          "camera",
-        ],
-        ignoreHTTPSErrors: true,
-        launchOptions: {
-          args: [
-            "--use-fake-ui-for-media-stream",
-            "--use-fake-device-for-media-stream",
-            "--mute-audio",
-          ],
-        },
-      },
+      testIgnore: ["**/mobile/**", "**/widget/**"],
+      use: desktopChrome,
+    },
+    {
+      // Element Call as a widget in Element Web, on Chromium. This uses its
+      // own project so that it can be sharded on CI.
+      name: "widget",
+      dependencies: ["build-embedded"],
+      testMatch: "**/widget/**/*.@(spec|test).ts",
+      use: desktopChrome,
    },
    {
      name: "firefox",
+      dependencies: ["build-embedded"],
      testIgnore: "**/mobile/**",
      use: {
        ...devices["Desktop Firefox"],
@@ -211,6 +211,46 @@ test.describe("background effects section", () => {
  });
 });

+test.describe("a pipeline that fails to build", () => {
+  test("offers no effects for the session, and keeps the choice", async ({
+    page,
+  }) => {
+    test.setTimeout(120_000);
+    await page.goto("/");
+    await SpaHelpers.createCall(page, "Failing user", "Pipeline fails");
+    const preview = page.locator("video").first();
+    const camera = await cameraColour(preview);
+    await page.route("**/*.tflite*", async (route) => route.abort());
+
+    await page.getByRole("button", { name: "Camera", exact: true }).click();
+    const tile = page.getByRole("menuitemradio", { name: "Background 1" });
+    await tile.click();
+    await expect(
+      page.getByText("Background effects are not supported on this platform."),
+    ).toBeVisible({ timeout: 30_000 });
+    await expect(tile).toBeDisabled();
+    await page.keyboard.press("Escape");
+    // The camera and microphone carry on without it.
+    expect(distance(await cameraColour(preview), camera)).toBeLessThan(30);
+    await expect(page.getByTestId("incall_mute")).toHaveAttribute(
+      "aria-checked",
+      "true",
+    );
+
+    await page.unroute("**/*.tflite*");
+    await page.reload();
+    await expect(page.getByTestId("lobby_joinCall")).toBeVisible();
+    await page.getByRole("button", { name: "Camera", exact: true }).click();
+    await expect(tile).toHaveAttribute("aria-checked", "true");
+    const picture = await averageColour(tile.locator("img"));
+    await expect
+      .poll(async () => distance(await averageColour(preview), picture), {
+        timeout: 60_000,
+      })
+      .toBeLessThan(distance(camera, picture) / 2);
+  });
+});
+
 test.describe("the chosen effect", () => {
  test("chosen effect survives leaving and rejoining", async ({ page }) => {
    test.setTimeout(120_000);
@@ -127,13 +127,14 @@ export const widgetTest = test.extend<MyFixtures>({
   * Provide a way to add additional users within a test.
   * The returned user will be registered on the default homeserver, the name will be made unique by appending a timestamp.
   */
-  addUser: async ({ browser }, use) => {
+  addUser: async ({ browser }, use, testInfo) => {
    await use(
      async (
        username: string,
        host: string = HOST1,
      ): Promise<UserBaseFixture> => {
-        const uniqueSuffix = Date.now();
+        // Tests in parallel workers can reach this in the same millisecond.
+        const uniqueSuffix = `${Date.now()}_${testInfo.parallelIndex}`;
        const { page, clientHandle, mxId } = await TestHelpers.registerUser(
          browser,
          `${username.toLowerCase()}_${uniqueSuffix}`,
@@ -0,0 +1,20 @@
+/*
+Copyright 2026 Element Creations Ltd.
+
+SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
+Please see LICENSE in the repository root for full details.
+*/
+
+import { test as setup } from "@playwright/test";
+import { execFileSync } from "node:child_process";
+import { mkdirSync } from "node:fs";
+
+const EMBEDDED_DIST = "embedded/web/dist";
+
+setup("build the embedded package", () => {
+  setup.setTimeout(10 * 60 * 1000);
+  mkdirSync(EMBEDDED_DIST, { recursive: true });
+  execFileSync("pnpm", ["build:embedded", "--outDir", EMBEDDED_DIST], {
+    stdio: "inherit",
+  });
+});
@@ -0,0 +1,56 @@
+/*
+Copyright 2026 Element Creations Ltd.
+
+SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
+Please see LICENSE in the repository root for full details.
+*/
+
+import { expect, type Route } from "@playwright/test";
+
+import { widgetTest } from "../../fixtures/widget-user.ts";
+import { HOST2, TestHelpers } from "../test-helpers.ts";
+
+/** Where the other site's Element Web serves the embedded package from */
+const CONFIG_JSON = "**/widgets/element-call/config.json";
+
+const HOST_ANSWERS: Record<string, (route: Route) => Promise<void>> = {
+  "a 404": async (route) => route.fulfill({ status: 404 }),
+  "a failed fetch": async (route) => route.abort("failed"),
+  "an HTML page": async (route) =>
+    route.fulfill({
+      status: 200,
+      contentType: "text/html",
+      body: "<!doctype html><title>Element</title>",
+    }),
+};
+
+widgetTest.skip(
+  ({ browserName }) => browserName === "firefox",
+  "The widget suite does not run on Firefox, see the other specs here",
+);
+
+for (const [answer, respond] of Object.entries(HOST_ANSWERS)) {
+  widgetTest(
+    `reaches the lobby when config.json gets ${answer}`,
+    async ({ addUser }) => {
+      const { page } = await addUser("Brooks", HOST2);
+      await TestHelpers.createRoom("Config Room", page);
+
+      let answered = 0;
+      await page.route(CONFIG_JSON, async (route) => {
+        answered++;
+        await respond(route);
+      });
+
+      await TestHelpers.startCallInCurrentRoom(page);
+
+      await expect(
+        page
+          .locator('iframe[title="Element Call"]')
+          .contentFrame()
+          .getByTestId("lobby_joinCall"),
+      ).toBeVisible({ timeout: 15000 });
+      expect(answered).toBeGreaterThan(0);
+    },
+  );
+}
@@ -144,7 +144,11 @@ export class TestHelpers {

    await this.dismissStartupToasts(page);

-    await TestHelpers.setDevToolElementCallDevUrl(page);
+    // The other site's Element Web opens the embedded package it serves itself
+    // (see docker-compose-playwright.yml), not the dev server's app.
+    if (host !== HOST2) {
+      await TestHelpers.setDevToolElementCallDevUrl(page);
+    }

    const clientHandle = await page.evaluateHandle(() =>
      window.mxMatrixClientPeg.get(),
@@ -191,8 +191,8 @@ importers:
        specifier: ^1.9.1
        version: 1.9.2
      matrix-js-sdk:
-        specifier: github:matrix-org/matrix-js-sdk#a9ef09fbf2f1644c8663d0aeaf6a921cd16e7b01
-        version: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/a9ef09fbf2f1644c8663d0aeaf6a921cd16e7b01
+        specifier: github:matrix-org/matrix-js-sdk#dc8610e796c92553c658300a4882dde961a0678a
+        version: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/dc8610e796c92553c658300a4882dde961a0678a
      matrix-widget-api:
        specifier: ^1.18.0
        version: 1.20.0
@@ -4734,8 +4734,8 @@ packages:
  matrix-events-sdk@0.0.1:
    resolution: {integrity: sha512-1QEOsXO+bhyCroIe2/A5OwaxHvBm7EsSQ46DEDn8RBIfQwN5HWBpFvyWWR4QY0KHPPnnJdI99wgRiAl7Ad5qaA==}

-  matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/a9ef09fbf2f1644c8663d0aeaf6a921cd16e7b01:
-    resolution: {gitHosted: true, integrity: sha512-ZAhn/zB44MgrXqUzi9ffSFcmZY3+2nx7bEgfEJR7chYDFyT/XmlmfCJx7Ar4rOTAYZJRVnMxrEwiEHGDapUi0Q==, tarball: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/a9ef09fbf2f1644c8663d0aeaf6a921cd16e7b01}
+  matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/dc8610e796c92553c658300a4882dde961a0678a:
+    resolution: {gitHosted: true, integrity: sha512-Ldt/VkCzBtre6m1cn0FqgrzRV0GrgWIoARgQUQO58ef2Edu2ERFCINPzp/psFS5RwoN3RIy9Rm4oPVc1ATkEWA==, tarball: https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/dc8610e796c92553c658300a4882dde961a0678a}
    version: 45.0.0
    engines: {node: '>=22.0.0'}

@@ -4946,8 +4946,8 @@ packages:
    resolution: {integrity: sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw==}
    engines: {node: '>=10'}

-  p-retry@8.0.1:
-    resolution: {integrity: sha512-NAigyu8hfvJe7MsS18mnc4is0MZtau3QMdBklaJKK23oBQb6occO3UPM3vHmTckW8KhfyjMZaUOqdTFEYliHgg==}
+  p-retry@9.0.0:
+    resolution: {integrity: sha512-HuEats8gYn/UUYa5uyveyQGv75mQmwdFtuMgnZlS12dj/7bdnJSO6keaTjgcZUhvkANTj5EO3lACUmV3xmgj0Q==}
    engines: {node: '>=22'}

  p-try@2.2.0:
@@ -5318,6 +5318,10 @@ packages:
  randomfill@1.0.4:
    resolution: {integrity: sha512-87lcbR8+MhcWcUiQ+9e+Rwx8MyR2P7qnt15ynUlbm3TU/fjbgz4GsvfSUDTemtCCtVCqb4ZcEFlyPNTh9bBTLw==}

+  re2js@2.9.1:
+    resolution: {integrity: sha512-JIT3bEEUZTOR2YS1walNEIfH4cF/cQeZcBUFq7ldqO6lR6g0X5CBq/S8rsa9omzZcsNf8qc36jCkWpKKtncrZg==}
+    engines: {node: '>=20.0.0'}
+
  react-docgen-typescript@2.4.0:
    resolution: {integrity: sha512-ZtAp5XTO5HRzQctjPU0ybY0RRCQO19X/8fxn3w7y2VVTUbGHDKULPTL4ky3vB05euSgG5NpALhEhDPvQ56wvXg==}
    peerDependencies:
@@ -10409,7 +10413,7 @@ snapshots:

  matrix-events-sdk@0.0.1: {}

-  matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/a9ef09fbf2f1644c8663d0aeaf6a921cd16e7b01:
+  matrix-js-sdk@https://codeload.github.com/matrix-org/matrix-js-sdk/tar.gz/dc8610e796c92553c658300a4882dde961a0678a:
    dependencies:
      '@babel/runtime': 8.0.8
      '@matrix-org/matrix-sdk-crypto-wasm': 18.9.0
@@ -10419,7 +10423,8 @@ snapshots:
      loglevel: 1.9.2
      matrix-events-sdk: 0.0.1
      matrix-widget-api: 1.20.0
-      p-retry: 8.0.1
+      p-retry: 9.0.0
+      re2js: 2.9.1
      sdp-transform: 3.0.0
      unhomoglyph: 1.0.6

@@ -10741,7 +10746,7 @@ snapshots:
    dependencies:
      p-limit: 3.1.0

-  p-retry@8.0.1:
+  p-retry@9.0.0:
    dependencies:
      is-network-error: 1.3.2

@@ -11165,6 +11170,8 @@ snapshots:
      randombytes: 2.1.0
      safe-buffer: 5.2.1

+  re2js@2.9.1: {}
+
  react-docgen-typescript@2.4.0(typescript@5.9.3):
    dependencies:
      typescript: 5.9.3
@@ -162,6 +162,7 @@ describe("createLocalMedia", () => {
    media.release("camera");
    expect(trackBehind(camera).stop).toHaveBeenCalled();
    expect(media.tracks$.value).toEqual([microphone]);
+    expect(media.requests.map((r) => r.source)).toEqual(["microphone"]);

    media.stop();
    expect(trackBehind(microphone).stop).toHaveBeenCalled();
@@ -107,6 +107,7 @@ export function createLocalMedia(
    if (source !== "microphone" && source !== "camera") return;
    capturedTracks.get(source)?.stop();
    capturedTracks.delete(source);
+    requests.delete(source);
    publishCapturedTracks();
  };

@@ -184,6 +184,7 @@ export function createRTCParticipation(
          encryptMedia: encryptionSystem.kind !== E2eeType.NONE,
          matrixRTCMode,
          sendNotificationType: options.sendNotificationType,
+          callIntent: options.callIntent,
          applicationData: options.applicationData,
          timings,
          delayedLeave,
@@ -12,6 +12,7 @@ Please see LICENSE in the repository root for full details.
 */

 import {
+  type RTCCallIntent,
  type RTCNotificationType,
  type Transport,
 } from "matrix-js-sdk/lib/matrixrtc";
@@ -103,11 +104,14 @@ export interface RTCParticipationOptions {
   * to a notification (ringing, timeouts, declines) is the application's job.
   */
  sendNotificationType?: RTCNotificationType;
+  /**
+   * The kind of call the local member intends to take part in: `"m.audio"`
+   * or `"m.video"`. Carried by the membership as a field of its own.
+   */
+  callIntent?: RTCCallIntent;
  /**
   * Whatever the application wants to say about itself in the membership,
-   * under namespaced keys. Opaque to the SDK, apart from `m.call.intent`
-   * (`"audio"` or `"video"`), which the membership format carries as a
-   * field of its own.
+   * under namespaced keys. Opaque to the SDK.
   */
  applicationData?: Record<string, unknown>;
  /** Timings the host has configured; the defaults otherwise. */
@@ -287,7 +291,7 @@ export interface RTCMember {
  memberId: string;
  /** Which transport this member is on; undefined when the membership has none. */
  transport$: Behavior<TransportMetadata | undefined>;
-  /** What the member's application says about itself, e.g. `m.call.intent`. */
+  /** What the member's application says about itself, e.g. `intent`. */
  applicationData$: Behavior<Record<string, unknown>>;

  displayName$: Behavior<string>;
@@ -60,7 +60,7 @@ describe("joinJsSdkSession", () => {
    expect(s.join).toHaveBeenCalledWith(ownMemberMock, [transportWithAlias], {
      notificationType: "ring",
      callIntent: undefined,
-      applicationData: {},
+      applicationData: undefined,
      manageMediaKeys: true,
      delayedLeaveEventRestartMs: 300_000,
      delayedLeaveEventDelayMs: 3_600_000,
@@ -95,13 +95,14 @@ describe("joinJsSdkSession", () => {
    );
  });

-  it("lifts the call intent out of the application data and sends sticky events in Matrix 2.0 mode", () => {
+  it("passes the call intent and application data through and sends sticky events in Matrix 2.0 mode", () => {
    const s = session();

    joinJsSdkSession("!room:example.org", s, ownMemberMock, transport, {
      encryptMedia: true,
      matrixRTCMode: MatrixRTCMode.Matrix_2_0,
-      applicationData: { "m.call.intent": "audio", custom: 1 },
+      callIntent: "m.audio",
+      applicationData: { custom: 1 },
      timings: defaultSessionTimings,
      delayedLeave: defaultSessionTimings.delayedLeave,
    });
@@ -110,7 +111,7 @@ describe("joinJsSdkSession", () => {
      ownMemberMock,
      [transportWithAlias],
      expect.objectContaining({
-        callIntent: "audio",
+        callIntent: "m.audio",
        applicationData: { custom: 1 },
        unstableSendStickyEvents: true,
      }),
@@ -124,7 +125,8 @@ describe("joinJsSdkSession", () => {
    joinJsSdkSession("!room:example.org", s, ownMemberMock, transport, {
      encryptMedia: true,
      matrixRTCMode: MatrixRTCMode.Matrix_2_0,
-      applicationData: { "m.call.intent": "audio", custom: 1 },
+      callIntent: "m.audio",
+      applicationData: { custom: 1 },
      timings: defaultSessionTimings,
      delayedLeave: defaultSessionTimings.delayedLeave,
    });
@@ -133,7 +135,7 @@ describe("joinJsSdkSession", () => {
      ownMemberMock,
      [transport],
      expect.objectContaining({
-        callIntent: "audio",
+        callIntent: "m.audio",
        applicationData: { custom: 1 },
        unstableSendStickyEvents: true,
      }),
@@ -23,6 +23,7 @@ interface Options {
  encryptMedia: boolean;
  matrixRTCMode: MatrixRTCMode;
  sendNotificationType?: RTCNotificationType;
+  callIntent?: RTCCallIntent;
  applicationData?: Record<string, unknown>;
  timings: SessionTimings;
  /** The delayed leave timings to use: delegated to the SFU or not. */
@@ -43,15 +44,12 @@ export function joinJsSdkSession(
    encryptMedia,
    matrixRTCMode,
    sendNotificationType,
+    callIntent,
    applicationData,
    timings,
    delayedLeave,
  }: Options,
 ): void {
-  // The membership format carries the intent as a field of its own
-  const { "m.call.intent": intent, ...rest } = applicationData ?? {};
-  const callIntent =
-    typeof intent === "string" ? (intent as RTCCallIntent) : undefined;
  // Give up on the network as soon as either the sync has been down for the
  // grace period or the delayed leave has probably been sent, whichever is
  // sooner
@@ -78,7 +76,7 @@ export function joinJsSdkSession(
    {
      notificationType: sendNotificationType,
      callIntent,
-      applicationData: rest,
+      applicationData,
      manageMediaKeys: encryptMedia,
      delayedLeaveEventRestartMs: delayedLeave.restart_ms,
      delayedLeaveEventDelayMs: delayedLeave.delay_ms,
@@ -18,6 +18,8 @@ Please see LICENSE in the repository root for full details.
 */

 import {
+  RTC_CALL_INTENT_AUDIO,
+  RTC_CALL_INTENT_VIDEO,
  type RTCCallIntent,
  type RTCNotificationType,
 } from "matrix-js-sdk/lib/matrixrtc";
@@ -179,40 +181,40 @@ export function configurationForIntent(intent: UserIntent): UrlConfiguration {
  switch (intent) {
    case UserIntent.StartNewCall:
      preset.skipLobby = false;
-      preset.callIntent = "video";
+      preset.callIntent = RTC_CALL_INTENT_VIDEO;
      break;
    case UserIntent.JoinExistingCall:
      // On desktop this will be overridden based on which button was used to join the call
      preset.skipLobby = false;
-      preset.callIntent = "video";
+      preset.callIntent = RTC_CALL_INTENT_VIDEO;
      break;
    case UserIntent.StartNewCallVoice:
      preset.skipLobby = false;
-      preset.callIntent = "audio";
+      preset.callIntent = RTC_CALL_INTENT_AUDIO;
      break;
    case UserIntent.JoinExistingCallVoice:
      // On desktop this will be overridden based on which button was used to join the call
      preset.skipLobby = false;
-      preset.callIntent = "audio";
+      preset.callIntent = RTC_CALL_INTENT_AUDIO;
      break;
    case UserIntent.StartNewCallDMVoice:
-      preset.callIntent = "audio";
+      preset.callIntent = RTC_CALL_INTENT_AUDIO;
    // Fall through
    case UserIntent.StartNewCallDM:
      preset.skipLobby = true;
      preset.sendNotificationType = "ring";
      preset.autoLeaveWhenOthersLeft = true;
      preset.waitForCallPickup = true;
-      preset.callIntent = preset.callIntent ?? "video";
+      preset.callIntent = preset.callIntent ?? RTC_CALL_INTENT_VIDEO;
      break;
    case UserIntent.JoinExistingCallDMVoice:
-      preset.callIntent = "audio";
+      preset.callIntent = RTC_CALL_INTENT_AUDIO;
    // Fall through
    case UserIntent.JoinExistingCallDM:
      // On desktop this will be overridden based on which button was used to join the call
      preset.skipLobby = true;
      preset.autoLeaveWhenOthersLeft = true;
-      preset.callIntent = preset.callIntent ?? "video";
+      preset.callIntent = preset.callIntent ?? RTC_CALL_INTENT_VIDEO;
      break;
    // Non widget usecase defaults
    default:
@@ -21,6 +21,10 @@ import {
  UserIntent,
 } from "../src/UrlParams";
 import { mockConfig } from "./utils/test";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  RTC_CALL_INTENT_VIDEO,
+} from "matrix-js-sdk/lib/matrixrtc";

 const ROOM_NAME = "roomNameHere";
 const ROOM_ID = "!d45f138fsd";
@@ -275,7 +279,7 @@ describe("UrlParams", () => {
      ).toMatchObject({
        ...startNewCallDefaults("desktop"),
        skipLobby: false,
-        callIntent: "video",
+        callIntent: RTC_CALL_INTENT_VIDEO,
      });
    });

@@ -322,7 +326,7 @@ describe("UrlParams", () => {
      ).toMatchObject({
        ...startNewCallDefaults("desktop"),
        skipLobby: false,
-        callIntent: "audio",
+        callIntent: RTC_CALL_INTENT_AUDIO,
      });
    });

@@ -333,7 +337,7 @@ describe("UrlParams", () => {
        ),
      ).toMatchObject({
        ...joinExistingCallDefaults("desktop"),
-        callIntent: "audio",
+        callIntent: RTC_CALL_INTENT_AUDIO,
      });
    });

@@ -348,7 +352,7 @@ describe("UrlParams", () => {
        sendNotificationType: "ring",
        autoLeaveWhenOthersLeft: true,
        waitForCallPickup: true,
-        callIntent: "audio",
+        callIntent: RTC_CALL_INTENT_AUDIO,
      });
    });

@@ -363,7 +367,7 @@ describe("UrlParams", () => {
        skipLobby: true,
        autoLeaveWhenOthersLeft: true,
        waitForCallPickup: false,
-        callIntent: "video",
+        callIntent: RTC_CALL_INTENT_VIDEO,
      });
    });

@@ -377,7 +381,7 @@ describe("UrlParams", () => {
        skipLobby: true,
        autoLeaveWhenOthersLeft: true,
        waitForCallPickup: false,
-        callIntent: "audio",
+        callIntent: RTC_CALL_INTENT_AUDIO,
      });
    });
  });
@@ -16,7 +16,8 @@ import {
  mockMediaDevices,
  mockVideoTrack,
 } from "../utils/test";
-import { constant } from "../state/Behavior";
+import { type Behavior, constant } from "../state/Behavior";
+import { type BackgroundEffectsState } from "../state/BackgroundEffects";
 import type { CallViewModel } from "../state/CallViewModel/CallViewModel";
 import type { Layout } from "../state/layout-types";
 import type { SpotlightTileViewModel } from "../state/TileViewModel";
@@ -112,6 +113,16 @@ const pipLayout: Layout = {
  spotlight: {} as SpotlightTileViewModel,
 };

+const noPipeline = constant<BackgroundEffectsState>({
+  supported: false,
+  processor: undefined,
+});
+
+/** The provider's verdict, which the SDK's check stands for here. */
+function pipelineAsTheSdkSays(): Behavior<BackgroundEffectsState> {
+  return constant({ supported: sdkSupportMock(), processor: undefined });
+}
+
 const twoMicsAndOneCamMediaDevices = mockMediaDevices({
  audioInput: {
    available$: constant(
@@ -143,7 +154,7 @@ describe("createCallFooterViewModel", () => {
        buildMinimalCallViewModel(gridLayout),
        mockMuteStates(),
        twoMicsAndOneCamMediaDevices,
-        constant(false),
+        noPipeline,
        /* reactionIdentifier */ undefined,
        { showControls: true, header: HeaderStyle.Standard },
      );
@@ -186,7 +197,7 @@ describe("createCallFooterViewModel", () => {
            select: vi.fn(),
          },
        }),
-        constant(false),
+        noPipeline,
        /* reactionIdentifier */ undefined,
        { showControls: true, header: HeaderStyle.Standard },
      );
@@ -205,7 +216,7 @@ describe("createCallFooterViewModel", () => {
        buildMinimalCallViewModel(layout),
        mockMuteStates(),
        twoMicsAndOneCamMediaDevices,
-        constant(false),
+        noPipeline,
        /* reactionIdentifier */ undefined,
        { showControls: true, header: HeaderStyle.Standard },
      );
@@ -228,7 +239,7 @@ describe("createCallFooterViewModel", () => {
        buildMinimalCallViewModel(gridLayout),
        mockMuteStates(),
        twoMicsAndOneCamMediaDevices,
-        constant(false),
+        noPipeline,
        /* reactionIdentifier */ undefined,
        { showControls: true, header: HeaderStyle.Standard },
      );
@@ -274,7 +285,7 @@ describe("createCallFooterViewModel", () => {
        testScope(),
        mockMuteStates(),
        twoMicsAndOneCamMediaDevices,
-        constant(false),
+        pipelineAsTheSdkSays(),
        /* camera$ */ constant(null),
        /* openSettings */ undefined,
        /* hangup */ undefined,
@@ -360,22 +371,52 @@ describe("createCallFooterViewModel", () => {

    it("passes on the wait for the first effect", () => {
      platformMock.mockReturnValue("desktop");
-      const settling$ = new BehaviorSubject(true);
+      const pipeline$ = new BehaviorSubject<BackgroundEffectsState>({
+        supported: true,
+        processor: undefined,
+        wait: "settling",
+      });
      const vm = createLobbyFooterViewModel(
        testScope(),
        mockMuteStates(),
        twoMicsAndOneCamMediaDevices,
-        settling$,
+        pipeline$,
        /* camera$ */ constant(null),
        /* openSettings */ undefined,
        /* hangup */ undefined,
        /* showLogo */ false,
      );
      expect(vm.backgroundEffectSettling$.value).toBe(true);
-      settling$.next(false);
+      pipeline$.next({ ...pipeline$.value, wait: undefined });
      expect(vm.backgroundEffectSettling$.value).toBe(false);
    });

+    it("offers nothing once the pipeline fails to build, keeping the choice", () => {
+      platformMock.mockReturnValue("desktop");
+      backgroundEffectSetting.setValue("blur");
+      const pipeline$ = new BehaviorSubject<BackgroundEffectsState>({
+        supported: true,
+        processor: undefined,
+      });
+      const vm = createLobbyFooterViewModel(
+        testScope(),
+        mockMuteStates(),
+        twoMicsAndOneCamMediaDevices,
+        pipeline$,
+        /* camera$ */ constant(null),
+        /* openSettings */ undefined,
+        /* hangup */ undefined,
+        /* showLogo */ false,
+      );
+      expect(vm.backgroundEffect$.value).toBe("blur");
+
+      pipeline$.next({ supported: false, processor: undefined });
+      expect(vm.selectBackgroundEffect$.value).toBeUndefined();
+      expect(vm.addBackgroundImage$.value).toBeUndefined();
+      expect(vm.backgroundEffect$.value).toBe("none");
+      expect(backgroundEffectSetting.getValue()).toBe("blur");
+    });
+
    it("adding is unavailable at the limit", () => {
      sdkSupportMock.mockReturnValue(true);
      const added$ = addedBackgrounds.added$ as BehaviorSubject<
@@ -439,7 +480,7 @@ describe("createCallFooterViewModel", () => {
        testScope(),
        mockMuteStates(),
        twoMicsAndOneCamMediaDevices,
-        constant(false),
+        noPipeline,
        camera$,
        /* openSettings */ undefined,
        /* hangup */ undefined,
@@ -459,7 +500,7 @@ describe("createCallFooterViewModel", () => {
        { ...buildMinimalCallViewModel(gridLayout), camera$: constant(track) },
        mockMuteStates(),
        twoMicsAndOneCamMediaDevices,
-        constant(false),
+        noPipeline,
        /* reactionIdentifier */ undefined,
        { showControls: true, header: HeaderStyle.Standard },
      );
@@ -479,7 +520,7 @@ describe("createCallFooterViewModel", () => {
          buildMinimalCallViewModel(gridLayout),
          mockMuteStates(),
          twoMicsAndOneCamMediaDevices,
-          constant(false),
+          pipelineAsTheSdkSays(),
          /* reactionIdentifier */ undefined,
          { showControls: true, header: HeaderStyle.Standard },
        );
@@ -16,10 +16,7 @@ import {
 } from "rxjs";
 import { supportsAudioOutputSelection } from "livekit-client";
 import { type LocalVideoMediaTrack } from "@element-hq/matrixrtc-sdk";
-import {
-  supportsBackgroundProcessors,
-  supportsModernBackgroundProcessors,
-} from "@livekit/track-processors";
+import { supportsModernBackgroundProcessors } from "@livekit/track-processors";
 import { logger } from "matrix-js-sdk/lib/logger";

 import {
@@ -35,6 +32,7 @@ import {
 } from "../tracks/backgroundImages";

 import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
+import { type BackgroundEffectsState } from "../state/BackgroundEffects";
 import { type MenuOptions } from "./MediaMuteAndSwitchButton";
 import { type MediaDevices } from "../state/MediaDevices";
 import {
@@ -89,7 +87,7 @@ function buildDeviceBehaviors(
  mediaDevices: MediaDevices,
  /** return empty arrays for  audioOptions and videoOptions*/
  disableSwitcher$: Behavior<boolean>,
-  backgroundEffectSettling$: Observable<boolean>,
+  backgroundEffectsState$: Observable<BackgroundEffectsState>,
  /** The camera as the pipeline has it, for the menu's preview. */
  camera$: Observable<LocalVideoMediaTrack | null | undefined>,
 ): Pick<
@@ -128,10 +126,14 @@ function buildDeviceBehaviors(
      ),
    );

-  const supported = supportsBackgroundProcessors();
+  // Also false once the pipeline has failed to build this session.
+  const supported$ = backgroundEffectsState$.pipe(
+    map(({ supported }) => supported === true),
+    distinctUntilChanged(),
+  );
  const slow = !supportsModernBackgroundProcessors();
-  const offered$ = disableSwitcher$.pipe(
-    map((switcherDisabled) => !switcherDisabled && supported),
+  const offered$ = combineLatest([disableSwitcher$, supported$]).pipe(
+    map(([switcherDisabled, supported]) => !switcherDisabled && supported),
  );
  // A new object for each refusal, so the same one twice is shown twice.
  const refusal$ = new BehaviorSubject<BackgroundImageRefusal | undefined>(
@@ -162,8 +164,10 @@ function buildDeviceBehaviors(
    ),
    selectVideoButtonOption$: constant(mediaDevices.videoInput.select),
    backgroundEffect$: scope.behavior(
-      backgroundEffectSetting.value$.pipe(
-        map((raw) => (supported ? serializeEffect(parseEffect(raw)) : "none")),
+      combineLatest([backgroundEffectSetting.value$, supported$]).pipe(
+        map(([raw, supported]) =>
+          supported ? serializeEffect(parseEffect(raw)) : "none",
+        ),
      ),
    ),
    selectBackgroundEffect$: scope.behavior(
@@ -191,7 +195,10 @@ function buildDeviceBehaviors(
      ),
    ),
    backgroundEffectSettling$: scope.behavior(
-      backgroundEffectSettling$.pipe(distinctUntilChanged()),
+      backgroundEffectsState$.pipe(
+        map(({ wait }) => wait !== undefined),
+        distinctUntilChanged(),
+      ),
      false,
    ),
    // Kept and offered, not put on: that waits for the user to choose it.
@@ -253,8 +260,8 @@ function buildDeviceBehaviors(
 * @param callModel - The root CallViewModel; provides layout, grid mode, reactions, etc.
 * @param muteStates - Audio and video mute state + toggles.
 * @param mediaDevices - Available and selected input devices.
- * @param backgroundEffectSettling$ - Whether the first effect chosen is still
- *   being prepared.
+ * @param backgroundEffectsState$ - The background pipeline's state: whether
+ *   effects run here, and the wait for the first.
 * @param reactionIdentifier - The local user's reaction identifier string, or
 *   undefined when reactions are not supported (hides the reaction button).
 * @param options - `showControls`: whether the call controls should be shown.
@@ -265,7 +272,7 @@ export function createCallFooterViewModel(
  callModel: CallViewModel,
  muteStates: MuteStates,
  mediaDevices: MediaDevices,
-  backgroundEffectSettling$: Observable<boolean>,
+  backgroundEffectsState$: Observable<BackgroundEffectsState>,
  reactionIdentifier: string | undefined,
  options: { showControls: boolean; header: HeaderStyle },
 ): ViewModel<FooterSnapshot> {
@@ -284,7 +291,7 @@ export function createCallFooterViewModel(
      scope,
      mediaDevices,
      disableDeviceSwitcher$,
-      backgroundEffectSettling$,
+      backgroundEffectsState$,
      callModel.camera$,
    ),
    // candidat to move into the FooterViewModel
@@ -348,8 +355,8 @@ export function createCallFooterViewModel(
 * @param scope - ObservableScope that bounds the lifetime of derived behaviors.
 * @param muteStates - Audio and video mute state + toggles.
 * @param mediaDevices - Available and selected input devices.
- * @param backgroundEffectSettling$ - Whether the first effect chosen is still
- *   being prepared.
+ * @param backgroundEffectsState$ - The background pipeline's state: whether
+ *   effects run here, and the wait for the first.
 * @param camera$ - The lobby's camera, for the camera menu's preview.
 * @param openSettings - Callback to open the settings modal, or undefined.
 * @param hangup - Callback to leave/cancel, or undefined (hides the button).
@@ -359,7 +366,7 @@ export function createLobbyFooterViewModel(
  scope: ObservableScope,
  muteStates: MuteStates,
  mediaDevices: MediaDevices,
-  backgroundEffectSettling$: Observable<boolean>,
+  backgroundEffectsState$: Observable<BackgroundEffectsState>,
  camera$: Behavior<LocalVideoMediaTrack | null>,
  openSettings: (() => void) | undefined,
  hangup: (() => void) | undefined,
@@ -408,7 +415,7 @@ export function createLobbyFooterViewModel(
      scope,
      mediaDevices,
      constant(false),
-      backgroundEffectSettling$,
+      backgroundEffectsState$,
      camera$,
    ),
  };
@@ -108,6 +108,22 @@ describe("Config.initWith", () => {
    expect(Config.get().media_quality).toEqual(DEFAULT_CONFIG.media_quality);
  });

+  it("falls back to the defaults when config.json is not JSON", async () => {
+    vi.spyOn(globalThis, "fetch").mockResolvedValue(
+      new Response("<!doctype html><title>Element</title>", {
+        status: 200,
+        headers: { "content-type": "text/html" },
+      }),
+    );
+    const warnSpy = vi.spyOn(logger, "warn").mockImplementation(() => {});
+    Reflect.set(Config, "internalInstance", undefined);
+
+    await expect(Config.init()).resolves.toBeUndefined();
+
+    expect(Config.get().media_quality).toEqual(DEFAULT_CONFIG.media_quality);
+    expect(warnSpy).toHaveBeenCalledTimes(1);
+  });
+
  it("replaces a config initialized earlier", () => {
    Config.initWith({ ssla: "https://first.invalid/ssla" });
    Config.initWith({ ssla: "https://second.invalid/ssla" });
@@ -156,5 +156,10 @@ async function downloadConfig(fetchTarget: string): Promise<ConfigOptions> {
    return {};
  }

-  return response.json();
+  try {
+    return await response.json();
+  } catch (e) {
+    logger.warn(`Config at ${fetchTarget} is not JSON, using defaults`, e);
+    return {};
+  }
 }
@@ -7,30 +7,30 @@ Please see LICENSE in the repository root for full details.

 import { type RelationType } from "matrix-js-sdk";

-import catSoundOgg from "../sound/reactions/cat.ogg?url";
-import catSoundMp3 from "../sound/reactions/cat.mp3?url";
-import clapSoundOgg from "../sound/reactions/clap.ogg?url";
-import clapSoundMp3 from "../sound/reactions/clap.mp3?url";
-import cricketsSoundOgg from "../sound/reactions/crickets.ogg?url";
-import cricketsSoundMp3 from "../sound/reactions/crickets.mp3?url";
-import dogSoundOgg from "../sound/reactions/dog.ogg?url";
-import dogSoundMp3 from "../sound/reactions/dog.mp3?url";
-import genericSoundOgg from "../sound/reactions/generic.ogg?url";
-import genericSoundMp3 from "../sound/reactions/generic.mp3?url";
-import lightbulbSoundOgg from "../sound/reactions/lightbulb.ogg?url";
-import lightbulbSoundMp3 from "../sound/reactions/lightbulb.mp3?url";
-import partySoundOgg from "../sound/reactions/party.ogg?url";
-import partySoundMp3 from "../sound/reactions/party.mp3?url";
-import deerSoundOgg from "../sound/reactions/deer.ogg?url";
-import deerSoundMp3 from "../sound/reactions/deer.mp3?url";
-import rockSoundOgg from "../sound/reactions/rock.ogg?url";
-import rockSoundMp3 from "../sound/reactions/rock.mp3?url";
-import waveSoundOgg from "../sound/reactions/wave.ogg?url";
-import waveSoundMp3 from "../sound/reactions/wave.mp3?url";
-import baduntssSoundOgg from "../sound/reactions/baduntss.ogg?url";
-import baduntssSoundMp3 from "../sound/reactions/baduntss.mp3?url";
-import bellSoundOgg from "../sound/reactions/bell.ogg?url";
-import bellSoundMp3 from "../sound/reactions/bell.mp3?url";
+import catSoundOgg from "../sound/reactions/cat.ogg?url&no-inline";
+import catSoundMp3 from "../sound/reactions/cat.mp3?url&no-inline";
+import clapSoundOgg from "../sound/reactions/clap.ogg?url&no-inline";
+import clapSoundMp3 from "../sound/reactions/clap.mp3?url&no-inline";
+import cricketsSoundOgg from "../sound/reactions/crickets.ogg?url&no-inline";
+import cricketsSoundMp3 from "../sound/reactions/crickets.mp3?url&no-inline";
+import dogSoundOgg from "../sound/reactions/dog.ogg?url&no-inline";
+import dogSoundMp3 from "../sound/reactions/dog.mp3?url&no-inline";
+import genericSoundOgg from "../sound/reactions/generic.ogg?url&no-inline";
+import genericSoundMp3 from "../sound/reactions/generic.mp3?url&no-inline";
+import lightbulbSoundOgg from "../sound/reactions/lightbulb.ogg?url&no-inline";
+import lightbulbSoundMp3 from "../sound/reactions/lightbulb.mp3?url&no-inline";
+import partySoundOgg from "../sound/reactions/party.ogg?url&no-inline";
+import partySoundMp3 from "../sound/reactions/party.mp3?url&no-inline";
+import deerSoundOgg from "../sound/reactions/deer.ogg?url&no-inline";
+import deerSoundMp3 from "../sound/reactions/deer.mp3?url&no-inline";
+import rockSoundOgg from "../sound/reactions/rock.ogg?url&no-inline";
+import rockSoundMp3 from "../sound/reactions/rock.mp3?url&no-inline";
+import waveSoundOgg from "../sound/reactions/wave.ogg?url&no-inline";
+import waveSoundMp3 from "../sound/reactions/wave.mp3?url&no-inline";
+import baduntssSoundOgg from "../sound/reactions/baduntss.ogg?url&no-inline";
+import baduntssSoundMp3 from "../sound/reactions/baduntss.mp3?url&no-inline";
+import bellSoundOgg from "../sound/reactions/bell.ogg?url&no-inline";
+import bellSoundMp3 from "../sound/reactions/bell.mp3?url&no-inline";

 export const ElementCallReactionEventType = "io.element.call.reaction";

@@ -8,18 +8,18 @@ Please see LICENSE in the repository root for full details.
 import { type ReactNode, useEffect } from "react";

 import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
-import joinCallSoundMp3 from "../sound/join_call.mp3";
-import joinCallSoundOgg from "../sound/join_call.ogg";
-import leftCallSoundMp3 from "../sound/left_call.mp3";
-import leftCallSoundOgg from "../sound/left_call.ogg";
-import handSoundOgg from "../sound/raise_hand.ogg";
-import handSoundMp3 from "../sound/raise_hand.mp3";
-import screenShareStartedOgg from "../sound/screen_share_started.ogg";
-import screenShareStartedMp3 from "../sound/screen_share_started.mp3";
-import declineMp3 from "../sound/call_declined.mp3?url";
-import declineOgg from "../sound/call_declined.ogg?url";
-import timeoutMp3 from "../sound/call_timeout.mp3?url";
-import timeoutOgg from "../sound/call_timeout.ogg?url";
+import joinCallSoundMp3 from "../sound/join_call.mp3?url&no-inline";
+import joinCallSoundOgg from "../sound/join_call.ogg?url&no-inline";
+import leftCallSoundMp3 from "../sound/left_call.mp3?url&no-inline";
+import leftCallSoundOgg from "../sound/left_call.ogg?url&no-inline";
+import handSoundOgg from "../sound/raise_hand.ogg?url&no-inline";
+import handSoundMp3 from "../sound/raise_hand.mp3?url&no-inline";
+import screenShareStartedOgg from "../sound/screen_share_started.ogg?url&no-inline";
+import screenShareStartedMp3 from "../sound/screen_share_started.mp3?url&no-inline";
+import declineMp3 from "../sound/call_declined.mp3?url&no-inline";
+import declineOgg from "../sound/call_declined.ogg?url&no-inline";
+import timeoutMp3 from "../sound/call_timeout.mp3?url&no-inline";
+import timeoutOgg from "../sound/call_timeout.ogg?url&no-inline";
 import { useAudioContext } from "../useAudioContext";
 import { prefetchSounds } from "../soundUtils";
 import { useLatest } from "../useLatest";
@@ -75,7 +75,6 @@ import {
  type RTCSlotOptions,
  constant,
 } from "@element-hq/matrixrtc-sdk";
-import { map } from "rxjs";
 import {
  advancedCamera,
  advancedScreenShare,
@@ -181,7 +180,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
              props.muteStates,
              mediaDevices,
              capture,
-              backgroundEffectsState$.value.processor,
+              backgroundEffectsState$.value,
            ),
    });

@@ -195,9 +194,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
        ...callViewModelOptionsFromParams(urlParams),
        encryptionSystem: props.e2eeSystem,
        capture,
-        videoProcessor$: scope.behavior(
-          backgroundEffectsState$.pipe(map((state) => state.processor)),
-        ),
+        backgroundEffectsState$,
        hostBridge,
        autoLeaveWhenOthersLeft,
        waitForCallPickup: waitForCallPickup && sendNotificationType === "ring",
@@ -241,7 +238,7 @@ export const ActiveCall: FC<ActiveCallProps> = (props) => {
      vm,
      props.muteStates,
      mediaDevices,
-      backgroundEffectsState$.pipe(map(({ settling }) => settling ?? false)),
+      backgroundEffectsState$,
      `${props.client.getUserId()}:${props.client.getDeviceId()}`,
      { showControls: urlParams.showControls, header: urlParams.header },
    );
@@ -825,9 +822,7 @@ export function rtcParticipationOptions(
  const session = config.matrix_rtc_session;
  return {
    sendNotificationType: urlParams.sendNotificationType,
-    applicationData: urlParams.callIntent
-      ? { "m.call.intent": urlParams.callIntent }
-      : undefined,
+    callIntent: urlParams.callIntent,
    timings: {
      syncDisconnectGracePeriodMs: config.sync_disconnect_grace_period_ms,
      networkErrorRetryMs: session.network_error_retry_ms,
@@ -16,7 +16,6 @@ import {
  type LocalMedia,
  type LocalVideoMediaTrack,
 } from "@element-hq/matrixrtc-sdk";
-import { map } from "rxjs";

 import inCallStyles from "./InCallView.module.css";
 import styles from "./LobbyView.module.css";
@@ -134,9 +133,7 @@ export const LobbyView: FC<Props> = ({
        muteStates,
        devices,
        captureSettings(),
-        mediaScope.behavior(
-          backgroundEffectsState$.pipe(map((state) => state.processor)),
-        ),
+        backgroundEffectsState$,
      ),
    );
    return (): void => {
@@ -165,7 +162,7 @@ export const LobbyView: FC<Props> = ({
        footerScope,
        muteStates,
        devices,
-        backgroundEffectsState$.pipe(map(({ settling }) => settling ?? false)),
+        backgroundEffectsState$,
        mediaVm.videoTrack$,
        openSettings,
        hangup,
@@ -8,6 +8,7 @@ Please see LICENSE in the repository root for full details.
 import { expect, type MockedFunction, test, vi } from "vitest";
 import { act, render } from "@testing-library/react";
 import { BehaviorSubject } from "rxjs";
+import { RTC_CALL_INTENT_AUDIO } from "matrix-js-sdk/lib/matrixrtc";

 import { useAudioContext } from "../useAudioContext";
 import { createRingingMedia } from "../state/media/RingingMediaViewModel";
@@ -39,7 +40,7 @@ test("ringtone plays on loop while ringing", () => {
    userId: alice.userId,
    displayName$: constant("Alice"),
    mxcAvatarUrl$: constant(undefined),
-    intent: "audio",
+    intent: RTC_CALL_INTENT_AUDIO,
    pickupState$,
  });

@@ -12,8 +12,8 @@ import { type RingingMediaViewModel } from "../state/media/RingingMediaViewModel
 import { useBehavior } from "../useBehavior";
 import { useInitial } from "../useInitial";
 import { prefetchSounds } from "../soundUtils";
-import ringtoneMp3 from "../sound/ringtone.mp3?url";
-import ringtoneOgg from "../sound/ringtone.ogg?url";
+import ringtoneMp3 from "../sound/ringtone.mp3?url&no-inline";
+import ringtoneOgg from "../sound/ringtone.ogg?url&no-inline";
 import { type UseAudioContext, useAudioContext } from "../useAudioContext";
 import { useLatest } from "../useLatest";

@@ -34,23 +34,6 @@ describe("prefetchSounds", () => {
    expect(bytesOf(sounds.beep)).toEqual([1, 2, 3]);
  });

-  it("decodes base64 data: URLs without fetching them", async () => {
-    // "OggS" — the bytes 0x4f 0x67 0x67 0x53
-    const sounds = await prefetchSounds({
-      beep: { ogg: "data:audio/ogg;base64,T2dnUw==" },
-    });
-    expect(fetchSpy).not.toHaveBeenCalled();
-    expect(bytesOf(sounds.beep)).toEqual([0x4f, 0x67, 0x67, 0x53]);
-  });
-
-  it("leaves out a sound with a percent-encoded data: URL", async () => {
-    const sounds = await prefetchSounds({
-      beep: { ogg: "data:audio/ogg,Ogg%53" },
-    });
-    expect(fetchSpy).not.toHaveBeenCalled();
-    expect(sounds).toEqual({});
-  });
-
  it("leaves out a sound whose response is not okay", async () => {
    fetchSpy.mockResolvedValue(new Response(null, { status: 404 }));
    const sounds = await prefetchSounds({
@@ -60,18 +43,16 @@ describe("prefetchSounds", () => {
  });

  it("leaves out a sound that fails to load, keeping the others", async () => {
-    fetchSpy.mockRejectedValue(new TypeError("Failed to fetch"));
+    // Fetched in the order given
+    fetchSpy
+      .mockRejectedValueOnce(new TypeError("Failed to fetch"))
+      .mockResolvedValueOnce(
+        new Response(new Uint8Array([1, 2, 3]), { status: 200 }),
+      );
    const sounds = await prefetchSounds({
      beep: { ogg: "https://example.org/beep.ogg" },
-      boop: { ogg: "data:audio/ogg;base64,T2dnUw==" },
+      boop: { ogg: "https://example.org/boop.ogg" },
    });
    expect(Object.keys(sounds)).toEqual(["boop"]);
  });
-
-  it("leaves out a sound with a malformed data: URL", async () => {
-    const sounds = await prefetchSounds({
-      beep: { ogg: "data:audio/ogg;base64" },
-    });
-    expect(sounds).toEqual({});
-  });
 });
@@ -32,38 +32,12 @@ function getPreferredAudioFormat(): "ogg" | "mp3" {

 const preferredFormat = getPreferredAudioFormat();

-/**
- * Decode a `data:` URL into the bytes it carries.
- *
- * The component build inlines the sound files as `data:` URLs. Those could be
- * `fetch`ed like any other URL, but that fetch is subject to the host page's
- * Content Security Policy, and `connect-src` lists rarely allow `data:` (a `*`
- * source does not cover it). Reading the bytes out of the URL directly needs no
- * network access, so no policy can stop it.
- * @param url The `data:` URL to decode.
- * @returns The decoded bytes.
- */
-function decodeDataUrl(url: string): ArrayBuffer {
-  const separator = url.indexOf(",");
-  if (!url.startsWith("data:") || separator === -1)
-    throw new Error("Not a valid data: URL");
-  const header = url.slice("data:".length, separator);
-  // Vite only percent-encodes SVGs; every other inlined asset is base64.
-  if (!header.split(";").includes("base64"))
-    throw new Error("Only base64 data: URLs are supported");
-  // `decoded` is a binary string: one char per byte
-  const decoded = atob(url.slice(separator + 1));
-  const bytes = Uint8Array.from(decoded, (c) => c.charCodeAt(0));
-  return bytes.buffer;
-}
-
 /**
 * Load the bytes of a single sound file.
- * @param url Where the sound is: an ordinary URL to fetch, or a `data:` URL.
+ * @param url Where the sound is.
 * @returns The bytes, or null if the sound could not be loaded.
 */
 async function loadSound(url: string): Promise<ArrayBuffer | null> {
-  if (url.startsWith("data:")) return decodeDataUrl(url);
  const response = await fetch(url);
  if (isFailure(response)) return null;
  return await response.arrayBuffer();
@@ -7,7 +7,11 @@ Please see LICENSE in the repository root for full details.

 import { it, vi, expect, beforeEach, afterEach, describe } from "vitest";
 import { firstValueFrom, of, Subject, take, toArray } from "rxjs";
-import { type RTCCallIntent } from "matrix-js-sdk/lib/matrixrtc";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  RTC_CALL_INTENT_VIDEO,
+  type RTCCallIntent,
+} from "matrix-js-sdk/lib/matrixrtc";

 import { AndroidControlledAudioOutput } from "./AndroidControlledAudioOutput.ts";
 import type { Controls, OutputDevice } from "../controls";
@@ -117,7 +121,7 @@ describe("Default selection", () => {
    const controlledAudioOutput = new AndroidControlledAudioOutput(
      of(BASE_DEVICE_LIST),
      testScope,
-      "video",
+      RTC_CALL_INTENT_VIDEO,
      mockControls,
    );

@@ -142,7 +146,7 @@ describe("Default selection", () => {
    const controlledAudioOutput = new AndroidControlledAudioOutput(
      of(BASE_DEVICE_LIST),
      testScope,
-      "audio",
+      RTC_CALL_INTENT_AUDIO,
      mockControls,
    );

@@ -163,7 +167,7 @@ describe("Default selection", () => {
    });
  });

-  ["audio", "video"].forEach((callIntent) => {
+  [RTC_CALL_INTENT_AUDIO, RTC_CALL_INTENT_VIDEO].forEach((callIntent) => {
    it(`Default to BT headset for ${callIntent} calls if present`, async () => {
      const controlledAudioOutput = new AndroidControlledAudioOutput(
        of(BT_HEADSET_BASE_DEVICE_LIST),
@@ -190,7 +194,7 @@ describe("Default selection", () => {
    });
  });

-  ["audio", "video"].forEach((callIntent) => {
+  [RTC_CALL_INTENT_AUDIO, RTC_CALL_INTENT_VIDEO].forEach((callIntent) => {
    it(`Default to wired headset for ${callIntent} calls if present`, async () => {
      const controlledAudioOutput = new AndroidControlledAudioOutput(
        of(WIRED_HEADSET_BASE_DEVICE_LIST),
@@ -300,7 +304,7 @@ describe("Test select a device", () => {
          b: BT_HEADSET_BASE_DEVICE_LIST,
        }),
        testScope,
-        "audio",
+        RTC_CALL_INTENT_AUDIO,
        mockControls,
      );

@@ -340,7 +344,7 @@ describe("Test select a device", () => {
          c: BASE_DEVICE_LIST,
        }),
        testScope,
-        "audio",
+        RTC_CALL_INTENT_AUDIO,
        mockControls,
      );

@@ -393,7 +397,7 @@ describe("Available device changes", () => {
  });

  it("When a BT headset is added, control should switch to use it", () => {
-    createAudioControlledOutput("video");
+    createAudioControlledOutput(RTC_CALL_INTENT_VIDEO);

    // Emit the base device list, the speaker should be selected
    availableSource$.next(BASE_DEVICE_LIST);
@@ -419,7 +423,9 @@ describe("Available device changes", () => {

  // Android does not set `isExternalHeadset` to true for wired headphones, so we can't test this case.'
  it.skip("When a wired headset is added, control should switch to use it", async () => {
-    const controlledAudioOutput = createAudioControlledOutput("video");
+    const controlledAudioOutput = createAudioControlledOutput(
+      RTC_CALL_INTENT_VIDEO,
+    );

    // Emit the base device list, the speaker should be selected
    availableSource$.next(BASE_DEVICE_LIST);
@@ -446,7 +452,7 @@ describe("Available device changes", () => {
  });

  it("When the active bt headset is removed on audio call, control should switch to earpiece", () => {
-    createAudioControlledOutput("audio");
+    createAudioControlledOutput(RTC_CALL_INTENT_AUDIO);

    // Emit the BT headset device list, the BT headset should be selected
    availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
@@ -472,7 +478,7 @@ describe("Available device changes", () => {
  });

  it("When the active bt headset is removed on video call, control should switch to speaker", () => {
-    createAudioControlledOutput("video");
+    createAudioControlledOutput(RTC_CALL_INTENT_VIDEO);

    availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);

@@ -497,7 +503,7 @@ describe("Available device changes", () => {
  });

  it("Do not repeatidly set the same device", () => {
-    createAudioControlledOutput("video");
+    createAudioControlledOutput(RTC_CALL_INTENT_VIDEO);

    availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
    availableSource$.next(BT_HEADSET_BASE_DEVICE_LIST);
@@ -22,7 +22,10 @@ import {
  type SelectedAudioOutputDevice,
 } from "./MediaDevices.ts";
 import type { ObservableScope } from "./ObservableScope.ts";
-import type { RTCCallIntent } from "matrix-js-sdk/lib/matrixrtc";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  type RTCCallIntent,
+} from "matrix-js-sdk/lib/matrixrtc";
 import { type Controls, type OutputDevice } from "../controls.ts";
 import { type Behavior } from "./Behavior.ts";

@@ -102,7 +105,7 @@ export class AndroidControlledAudioOutput implements MediaDevice<
   * @constructor
   * @param controlledDevices$ - The list of available output devices coming from the hosting application, ordered by preference order (most preferred first).
   * @param scope - The ObservableScope to create the Behaviors in.
-   * @param initialIntent - The initial call intent (e.g. "audio" or "video") that can be used to determine the default audio routing (e.g. default to earpiece for audio calls and speaker for video calls).
+   * @param initialIntent - The initial call intent (e.g. "m.audio" or "m.video") that can be used to determine the default audio routing (e.g. default to earpiece for audio calls and speaker for video calls).
   * @param controls - The controls provided by the hosting application to control the audio routing and notify of user actions.
   */
  public constructor(
@@ -316,7 +319,7 @@ export class AndroidControlledAudioOutput implements MediaDevice<
    this.logger.debug(
      `Android routing logic intent: ${this.initialIntent} finding best default...`,
    );
-    if (this.initialIntent === "audio") {
+    if (this.initialIntent === RTC_CALL_INTENT_AUDIO) {
      const systemProposed = available[0];
      // If no headset is connected, android will route to the speaker by default,
      // but for a voice call we want to route to the earpiece instead,
@@ -6,7 +6,7 @@ Please see LICENSE in the repository root for full details.
 */

 import { beforeEach, describe, expect, it, vi } from "vitest";
-import { BehaviorSubject, distinctUntilChanged, map } from "rxjs";
+import { BehaviorSubject, distinctUntilChanged, map, of } from "rxjs";
 import { type BackgroundProcessorWrapper } from "@livekit/track-processors";

 import {
@@ -47,23 +47,28 @@ function fakePipeline(): {
  };
 }

-/** One letter per state: idle, waiting for a frame, attached. */
+/** One letter per state: idle, preparing, waiting for a frame, attached. */
 function letter(state: BackgroundEffectsState): string {
+  if (state.supported === false) return "x";
+  if (state.wait === "preparing") return "p";
  if (state.processor === undefined) return "i";
-  return state.settling ? "w" : "a";
+  return state.wait === "settling" ? "w" : "a";
 }

 describe("the pipeline's state", () => {
  function testState({
    effect,
+    answer = "-y",
    firstFrame = "",
    expected,
  }: {
    effect: string;
+    /** When the trial build answers, from being asked. */
+    answer?: string;
    firstFrame?: string;
    expected: string;
  }): void {
-    withTestScheduler(({ behavior, schedule, expectObservable }) => {
+    withTestScheduler(({ behavior, cold, schedule, expectObservable }) => {
      const transformer = {
        onFirstFrame: undefined as (() => void) | undefined,
      };
@@ -72,6 +77,7 @@ describe("the pipeline's state", () => {
        effect$: behavior(effect, { n: "none", b: "blur" }),
        setEffect: vi.fn(),
        added$: new BehaviorSubject<AddedBackground[] | undefined>([]),
+        canSegment: () => cold(answer, { y: true, n: false }),
        pipeline: fakePipeline().pipeline,
        transformer,
      });
@@ -84,15 +90,21 @@ describe("the pipeline's state", () => {

  it("defaults to no effect", () => testState({ effect: "n", expected: "i" }));

+  it("attaches only once it is known the pipeline builds", () =>
+    testState({ effect: "nb", answer: "--y", expected: "ip-w" }));
+
+  it("offers none where the pipeline fails to build", () =>
+    testState({ effect: "nb", answer: "-n", expected: "ipx" }));
+
  it("waits from the first effect until a frame carries it", () =>
    testState({
-      effect: "    nb-n-b",
-      firstFrame: "  --f",
-      expected: "  iwa",
+      effect: "    nb---n-b",
+      firstFrame: "---f",
+      expected: "  ipwa",
    }));

  it("attaches on first use and stays attached", () =>
-    testState({ effect: "nb-n", firstFrame: "--f", expected: "iwa" }));
+    testState({ effect: "nb--n", firstFrame: "---f", expected: "ipwa" }));
 });

 describe("background effects", () => {
@@ -109,6 +121,7 @@ describe("background effects", () => {
      effect$,
      setEffect,
      added$,
+      canSegment: () => of(true),
      pipeline: fake.pipeline,
      transformer: { onFirstFrame: undefined },
      ...options,
@@ -118,8 +131,6 @@ describe("background effects", () => {
    effect$.next(raw);
    await flushPromises();
  };
-  const blur = async (on: boolean): Promise<void> =>
-    choose(on ? "blur" : "none");

  beforeEach(() => {
    effect$ = new BehaviorSubject("none");
@@ -128,6 +139,17 @@ describe("background effects", () => {
    fake = fakePipeline();
  });

+  it("offers none for the session where the pipeline fails to build", async () => {
+    const effects = build({ canSegment: () => of(false) });
+    await choose("blur");
+
+    expect(effects.state$.value.supported).toBe(false);
+    expect(effects.state$.value.processor).toBeUndefined();
+    expect(effects.state$.value.wait).toBeUndefined();
+    // A browser update may fix it, so the next session asks again.
+    expect(effect$.value).toBe("blur");
+  });
+
  it("puts a shipped background on as that picture", async () => {
    build();
    await choose(`image:${shippedBackgrounds[0].id}`);
@@ -141,10 +163,10 @@ describe("background effects", () => {

  it("switches in place rather than reattaching", async () => {
    const effects = build();
-    await blur(true);
+    await choose("blur");
    const pipeline = effects.state$.value.processor;
-    await blur(false);
-    await blur(true);
+    await choose("none");
+    await choose("blur");

    expect(effects.state$.value.processor).toBe(pipeline);
    expect(fake.switches).toEqual([
@@ -157,18 +179,18 @@ describe("background effects", () => {
  it("keeps switching after a switch fails", async () => {
    vi.mocked(fake.pipeline.switchTo).mockRejectedValueOnce(new Error("lost"));
    build();
-    await blur(true);
-    await blur(false);
+    await choose("blur");
+    await choose("none");
    expect(fake.switches).toEqual([{ mode: "disabled" }]);
  });

  it("switches one at a time, skipping those overtaken", async () => {
    const finished = fake.holdNext();
    build();
-    await blur(true);
-    await blur(false);
-    await blur(true);
-    await blur(false);
+    await choose("blur");
+    await choose("none");
+    await choose("blur");
+    await choose("none");
    expect(fake.switches).toHaveLength(1);

    finished();
@@ -8,9 +8,15 @@ Please see LICENSE in the repository root for full details.
 import {
  combineLatest,
  distinctUntilChanged,
+  first,
+  from,
  map,
  Observable,
+  type ObservableInput,
+  of,
  scan,
+  startWith,
+  switchMap,
 } from "rxjs";
 import {
  type BackgroundProcessorWrapper,
@@ -38,8 +44,12 @@ const blurRadius = 15;
 export interface BackgroundEffectsState {
  supported: boolean | undefined;
  processor: undefined | VideoProcessor;
-  /** From the first effect chosen until a frame carrying it is drawn. */
-  settling?: boolean;
+  /**
+   * The effect is not ready:
+   * preparing, asking whether the pipeline can be built.
+   * settling, once it is attached until a frame carrying the effect
+   */
+  wait?: "preparing" | "settling";
 }

 export interface BackgroundEffectsOptions {
@@ -51,6 +61,8 @@ export interface BackgroundEffectsOptions {
  setEffect: (id: EffectId) => void;
  /** The backgrounds the device keeps, undefined until it has said. */
  added$: Behavior<AddedBackground[] | undefined>;
+  /** Builds a segmenter on trial, and answers whether it could. */
+  canSegment: () => ObservableInput<boolean>;
  /** The background processor pipeline to be switched between effects. */
  pipeline: BackgroundProcessorWrapper;
  /** Tells, once, that a frame carrying an effect has been drawn. */
@@ -70,6 +82,7 @@ export function createBackgroundEffects(
    effect$,
    setEffect,
    added$,
+    canSegment,
    pipeline,
    transformer,
  }: BackgroundEffectsOptions,
@@ -77,9 +90,11 @@ export function createBackgroundEffects(
  // Wrapped once, so the call and the preview see one processor throughout.
  const processor = fromLivekitProcessor(pipeline);
  const choice$ = effect$.pipe(map(parseEffect));
-  const wanted$ = choice$.pipe(
-    map((effect) => effect.kind !== "none"),
-    distinctUntilChanged(),
+  const wanted$ = scope.behavior(
+    choice$.pipe(
+      map((effect) => effect.kind !== "none"),
+      distinctUntilChanged(),
+    ),
  );

  combineLatest([choice$, added$])
@@ -95,6 +110,18 @@ export function createBackgroundEffects(
        setEffect("none");
    });

+  // Asked once an effect is first wanted. A browser update may fix what
+  // fails, so the choice is kept for the next session rather than cleared.
+  const buildable$ = scope.behavior<boolean | undefined>(
+    supported
+      ? wanted$.pipe(
+          first(Boolean),
+          switchMap(() => from(canSegment())),
+          startWith(undefined),
+        )
+      : of(undefined),
+  );
+
  const drewAFrame$ = scope.behavior(
    new Observable<boolean>((subscriber) => {
      subscriber.next(false);
@@ -105,18 +132,25 @@ export function createBackgroundEffects(
    }),
  );

+  // In one step, so that nothing sees the pipeline neither preparing nor
+  // attached.
  const state$ = scope.behavior(
-    combineLatest([wanted$, drewAFrame$]).pipe(
-      scan<[boolean, boolean], BackgroundEffectsState>(
-        (previous, [wanted, drewAFrame]) => {
+    combineLatest([wanted$, buildable$, drewAFrame$]).pipe(
+      scan<[boolean, boolean | undefined, boolean], BackgroundEffectsState>(
+        (previous, [wanted, buildable, drewAFrame]) => {
          // Attached the first time an effect is wanted and never detached
          // after, so someone who never turns one on pays for none of it.
          const enable =
-            previous.processor !== undefined || (supported && wanted);
+            previous.processor !== undefined || (wanted && buildable === true);
+          const preparing = supported && wanted && buildable === undefined;
          return {
-            supported,
+            supported: supported && buildable !== false,
            processor: enable ? processor : undefined,
-            settling: enable && !drewAFrame,
+            wait: preparing
+              ? "preparing"
+              : enable && !drewAFrame
+                ? "settling"
+                : undefined,
          };
        },
        { supported, processor: undefined },
@@ -12,7 +12,11 @@ import {
  MatrixEvent,
  type Room,
 } from "matrix-js-sdk";
-import { type RTCCallIntent } from "matrix-js-sdk/lib/matrixrtc";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  RTC_CALL_INTENT_VIDEO,
+  type RTCCallIntent,
+} from "matrix-js-sdk/lib/matrixrtc";
 import { map, mergeMap, NEVER, type Observable, startWith } from "rxjs";

 import { withTestScheduler } from "../../utils/test";
@@ -33,10 +37,12 @@ function mockRingEvent(
  eventId: string,
  lifetimeMs: number | undefined,
  sender = local.userId,
+  legacyIntent?: string,
 ): CallNotificationWrapper {
  return {
    event_id: eventId,
    ...(lifetimeMs === undefined ? {} : { lifetime: lifetimeMs }),
+    ...(legacyIntent === undefined ? {} : { "m.call.intent": legacyIntent }),
    notification_type: "ring",
    sender,
  } as unknown as CallNotificationWrapper;
@@ -128,7 +134,25 @@ test("ring attempt times out after nobody joins", () => {
    });

    expectObservable(summarizeRingAttempts$(ringAttempts$)).toBe("-a 29ms A", {
-      a: { intent: "audio", recipient: alice.userId },
+      a: { intent: RTC_CALL_INTENT_AUDIO, recipient: alice.userId },
+      A: { outcome: "timeout" },
+    });
+  });
+});
+
+test("ring attempt maps the legacy intent on the notification to its stable value", () => {
+  withTestScheduler(({ scope, expectObservable, hot }) => {
+    const { ringAttempts$ } = createCallNotificationLifecycle$({
+      scope,
+      ...defaultProps,
+      memberUserIds$: constant<string[]>([]),
+      sentCallNotification$: hot("-a", {
+        a: mockRingEvent("$notif1", 30, local.userId, "video"),
+      }),
+    });
+
+    expectObservable(summarizeRingAttempts$(ringAttempts$)).toBe("-a 29ms A", {
+      a: { intent: RTC_CALL_INTENT_VIDEO, recipient: alice.userId },
      A: { outcome: "timeout" },
    });
  });
@@ -148,7 +172,7 @@ test("ring attempt is accepted once recipient joins", () => {
    });

    expectObservable(summarizeRingAttempts$(ringAttempts$)).toBe("-aA", {
-      a: { intent: "audio", recipient: alice.userId },
+      a: { intent: RTC_CALL_INTENT_AUDIO, recipient: alice.userId },
      A: { outcome: "accept" },
    });
  });
@@ -166,7 +190,7 @@ test("ring attempt is immediately accepted if recipient is already joined", () =
    });

    expectObservable(summarizeRingAttempts$(ringAttempts$)).toBe("-(aA)", {
-      a: { intent: "audio", recipient: alice.userId },
+      a: { intent: RTC_CALL_INTENT_AUDIO, recipient: alice.userId },
      A: { outcome: "accept" },
    });
  });
@@ -201,7 +225,7 @@ test("ring attempt can be declined", () => {
    });

    expectObservable(summarizeRingAttempts$(ringAttempts$)).toBe("-aA", {
-      a: { intent: "audio", recipient: alice.userId },
+      a: { intent: RTC_CALL_INTENT_AUDIO, recipient: alice.userId },
      A: { outcome: "decline" },
    });
  });
@@ -236,7 +260,7 @@ test("ring attempt times out if recipient declines too late", () => {
    });

    expectObservable(summarizeRingAttempts$(ringAttempts$)).toBe("-a 29ms A", {
-      a: { intent: "audio", recipient: alice.userId },
+      a: { intent: RTC_CALL_INTENT_AUDIO, recipient: alice.userId },
      A: { outcome: "timeout" },
    });
  });
@@ -271,7 +295,7 @@ test("decline event relating to wrong event is ignored (times out)", () => {
    });

    expectObservable(summarizeRingAttempts$(ringAttempts$)).toBe("-a 29ms A", {
-      a: { intent: "audio", recipient: alice.userId },
+      a: { intent: RTC_CALL_INTENT_AUDIO, recipient: alice.userId },
      A: { outcome: "timeout" },
    });
  });
@@ -306,7 +330,7 @@ test("decline event from wrong sender is ignored (times out)", () => {
    });

    expectObservable(summarizeRingAttempts$(ringAttempts$)).toBe("-a 29ms A", {
-      a: { intent: "audio", recipient: alice.userId },
+      a: { intent: RTC_CALL_INTENT_AUDIO, recipient: alice.userId },
      A: { outcome: "timeout" },
    });
  });
@@ -9,7 +9,9 @@ import {
  type IRTCNotificationContent,
  type MatrixRTCSession,
  MatrixRTCSessionEvent,
+  RTC_CALL_INTENT_AUDIO,
  type RTCCallIntent,
+  toStableCallIntent,
 } from "matrix-js-sdk/lib/matrixrtc";
 import {
  filter,
@@ -155,7 +157,10 @@ export function createCallNotificationLifecycle$({
        );

        return of({
-          intent: notificationEvent["m.call.intent"] ?? "audio",
+          // The notification still carries the legacy intent values
+          intent:
+            toStableCallIntent(notificationEvent["m.call.intent"]) ??
+            RTC_CALL_INTENT_AUDIO,
          recipient,
          outcome$: race(timeout$, accept$, decline$).pipe(
            take(1),
@@ -21,6 +21,7 @@ import {
  type CallMembership,
  type IRTCNotificationContent,
  MatrixRTCSessionEvent,
+  RTC_CALL_INTENT_AUDIO,
 } from "matrix-js-sdk/lib/matrixrtc";
 import { deepCompare } from "matrix-js-sdk/lib/utils";

@@ -70,9 +71,11 @@ import {
 } from "./CallViewModel.ts";
 import { type Handler, MuteState } from "../MuteStates.ts";
 import {
+  type LocalVideoMediaTrack,
  type RTCParticipation,
  type VideoProcessor,
 } from "@element-hq/matrixrtc-sdk";
+import { type BackgroundEffectsState } from "../BackgroundEffects";

 initializeWidget();

@@ -1557,7 +1560,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
            b: expect.objectContaining({
              type: "ringing",
              userId: alice.userId,
-              intent: "audio",
+              intent: RTC_CALL_INTENT_AUDIO,
            }),
          });
          // Layout should show placeholder media for the participant we're
@@ -1603,7 +1606,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
            b: expect.objectContaining({
              type: "ringing",
              userId: alice.userId,
-              intent: "audio",
+              intent: RTC_CALL_INTENT_AUDIO,
            }),
          });
          // Layout should show placeholder media for the participant we're
@@ -1692,28 +1695,113 @@ describe("a camera with a video processor", () => {
        muteStates,
        mockMediaDevices({}),
        {},
-        processor,
+        { supported: true, processor },
      ).find((request) => request.source === "camera"),
    ).toMatchObject({ processor });
  });

+  it("leaves the camera out of the join while the processor is prepared", () => {
+    const muteStates = mockMuteStates();
+    vi.spyOn(muteStates.video.enabled$, "value", "get").mockReturnValue(true);
+    const sources = (wait?: "preparing"): string[] =>
+      initialPublishRequests(
+        muteStates,
+        mockMediaDevices({}),
+        {},
+        {
+          supported: true,
+          processor: undefined,
+          wait,
+        },
+      ).map((request) => request.source);
+    expect(sources("preparing")).not.toContain("camera");
+    expect(sources()).toContain("camera");
+  });
+
+  it("waits to make a camera track until it is known the processor builds", async () => {
+    const setHandler = vi.spyOn(MuteState.prototype, "setHandler");
+    onTestFinished(() => setHandler.mockRestore());
+    const effects$ = new BehaviorSubject<BackgroundEffectsState>({
+      supported: true,
+      processor: undefined,
+      wait: "preparing",
+    });
+    let participation: RTCParticipation | undefined;
+    withCallViewModel(
+      {},
+      (_vm, _session, { rtcParticipation }) => {
+        participation = rtcParticipation;
+      },
+      { backgroundEffectsState$: effects$ },
+    );
+    const camera = participation!.localMember$.value?.tracks$.value?.find(
+      (track) => track.source === "camera",
+    ) as LocalVideoMediaTrack;
+    const turnedOn = cameraHandler(setHandler)(true);
+    await flushPromises();
+    expect(camera.setEnabled).not.toHaveBeenCalledWith(true);
+
+    effects$.next({ supported: true, processor });
+    await turnedOn;
+    expect(camera.setEnabled).toHaveBeenCalledWith(true);
+    expect(participation!.publish).not.toHaveBeenCalled();
+  });
+
+  it("turns the camera on only once its processor is on the track", async () => {
+    const setHandler = vi.spyOn(MuteState.prototype, "setHandler");
+    onTestFinished(() => setHandler.mockRestore());
+    const effects$ = new BehaviorSubject<BackgroundEffectsState>({
+      supported: true,
+      processor: undefined,
+      wait: "preparing",
+    });
+    let participation: RTCParticipation | undefined;
+    withCallViewModel(
+      {},
+      (_vm, _session, { rtcParticipation }) => {
+        participation = rtcParticipation;
+      },
+      { backgroundEffectsState$: effects$ },
+    );
+    const camera = participation!.localMember$.value?.tracks$.value?.find(
+      (track) => track.source === "camera",
+    ) as LocalVideoMediaTrack;
+    // Attaching takes its time, as the pipeline is built on the track
+    let attached!: () => void;
+    vi.mocked(camera.setProcessor).mockReturnValueOnce(
+      new Promise<void>((resolve) => {
+        attached = resolve;
+      }),
+    );
+    const turnedOn = cameraHandler(setHandler)(true);
+    effects$.next({ supported: true, processor });
+    await flushPromises();
+    expect(camera.setProcessor).toHaveBeenCalledWith(processor);
+    expect(camera.setEnabled).not.toHaveBeenCalledWith(true);
+
+    attached();
+    await turnedOn;
+    expect(camera.setEnabled).toHaveBeenCalledWith(true);
+  });
+
  // The SDK builds the processor on the camera's restart, before the room
  // sees a frame, so the track is unmuted rather than published anew
  it("turns on a camera turned off before the processor was chosen", async () => {
    const setHandler = vi.spyOn(MuteState.prototype, "setHandler");
    onTestFinished(() => setHandler.mockRestore());
-    const videoProcessor$ = new BehaviorSubject<VideoProcessor | undefined>(
-      undefined,
-    );
+    const effects$ = new BehaviorSubject<BackgroundEffectsState>({
+      supported: true,
+      processor: undefined,
+    });
    let participation: RTCParticipation | undefined;
    withCallViewModel(
      {},
      (_vm, _session, { rtcParticipation }) => {
        participation = rtcParticipation;
      },
-      { videoProcessor$ },
+      { backgroundEffectsState$: effects$ },
    );
-    videoProcessor$.next(processor);
+    effects$.next({ supported: true, processor });
    await cameraHandler(setHandler)(true);
    await flushPromises();

@@ -37,6 +37,7 @@ import {
  combineLatest,
  distinctUntilChanged,
  filter,
+  firstValueFrom,
  fromEvent,
  map,
  merge,
@@ -77,6 +78,7 @@ import {
  type ReactionInfo,
  type ReactionOption,
 } from "../../reactions";
+import { type BackgroundEffectsState } from "../BackgroundEffects";
 import { shallowEquals as arrayShallowEquals } from "../../utils/array";
 import { type MediaDevices } from "../MediaDevices";
 import { type MuteStates } from "../MuteStates";
@@ -170,13 +172,16 @@ export function initialPublishRequests(
  muteStates: MuteStates,
  mediaDevices: MediaDevices,
  capture: CaptureSettings,
-  processor?: VideoProcessor,
+  effects?: BackgroundEffectsState,
 ): PublishRequest[] {
  const requests: PublishRequest[] = [];
  if (muteStates.audio.enabled$.value)
    requests.push(publishRequest("microphone", mediaDevices, capture));
-  if (muteStates.video.enabled$.value)
-    requests.push(publishRequest("camera", mediaDevices, capture, processor));
+  // Left out while the processor is prepared; the call publishes it after
+  if (muteStates.video.enabled$.value && effects?.wait !== "preparing")
+    requests.push(
+      publishRequest("camera", mediaDevices, capture, effects?.processor),
+    );
  return requests;
 }

@@ -187,8 +192,8 @@ export interface CallViewModelOptions {
   * defaults otherwise.
   */
  capture?: CaptureSettings;
-  /** Background blur and the like, applied to the camera as it changes. */
-  videoProcessor$?: Behavior<VideoProcessor | undefined>;
+  /** The background effects state (preparing, supported...) including the processor itself. */
+  backgroundEffectsState$?: Behavior<BackgroundEffectsState>;
  /**
   * The application hosting Element Call, which can ask it to hang up and wants
   * to know when the user joins or leaves. Defaults to no host.
@@ -490,7 +495,21 @@ export function createCallViewModel$(
    sentCallNotification$ = of(null),
  } = options;

-  const { capture = {}, videoProcessor$ = constant(undefined) } = options;
+  const {
+    capture = {},
+    backgroundEffectsState$ = constant<BackgroundEffectsState>({
+      supported: false,
+      processor: undefined,
+    }),
+  } = options;
+  const videoProcessor$ = scope.behavior(
+    backgroundEffectsState$.pipe(map(({ processor }) => processor)),
+  );
+  // Resolves once it is known whether the processor builds
+  const processorKnown = async (): Promise<unknown> =>
+    firstValueFrom(
+      backgroundEffectsState$.pipe(filter(({ wait }) => wait !== "preparing")),
+    );
  const localTracks$ = scope.behavior<
    (LocalAudioMediaTrack | LocalVideoMediaTrack)[]
  >(
@@ -514,6 +533,19 @@ export function createCallViewModel$(
    "screenShare",
  ) as Behavior<LocalVideoMediaTrack | undefined>;

+  const processorAttached = followProcessor(
+    scope,
+    camera$,
+    videoProcessor$,
+    logger,
+  );
+
+  // A camera turned on waits for its processor to be known, then to be on
+  // the track: LiveKit restarts the track with whatever processor it has then.
+  const cameraReady = async (): Promise<void> => {
+    await processorKnown();
+    await processorAttached();
+  };
  const supply = (source: "microphone" | "camera"): TrackSupply => ({
    publish: async () =>
      rtcParticipation.publish(
@@ -524,6 +556,7 @@ export function createCallViewModel$(
          source === "camera" ? videoProcessor$.value : undefined,
        ),
      ),
+    ready: source === "camera" ? cameraReady : undefined,
  });
  followMuteState(
    scope,
@@ -550,6 +583,29 @@ export function createCallViewModel$(
  );
  followDevice(scope, camera$, mediaDevices.videoInput, "camera", logger);

+  // A camera wanted at the join was left out of it while the processor was
+  // prepared (see initialPublishRequests), and is published once that is known
+  if (
+    backgroundEffectsState$.value.wait === "preparing" &&
+    muteStates.video.enabled$.value
+  )
+    processorKnown()
+      .then(async () => {
+        if (camera$.value !== undefined || !muteStates.video.enabled$.value)
+          return;
+        await rtcParticipation.publish(
+          publishRequest(
+            "camera",
+            mediaDevices,
+            capture,
+            videoProcessor$.value,
+          ),
+        );
+      })
+      .catch((e) => {
+        logger.error("Failed to publish the camera", e);
+      });
+
  // The output device follows the selection onto the client the same way. A
  // host that routes audio itself picks the output outside the browser, so the
  // client is left alone
@@ -563,8 +619,6 @@ export function createCallViewModel$(
        });
      });

-  followProcessor(scope, camera$, videoProcessor$, logger);
-
  const members$ = scope.behavior<(LocalRTCMember | RemoteRTCMember)[]>(
    combineLatest(
      [rtcParticipation.localMember$, rtcParticipation.remoteMembers$],
@@ -21,6 +21,10 @@ import {
  EARPIECE_CONFIG_ID,
  IOSControlledAudioOutput,
 } from "./IOSControlledAudioOutput";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  RTC_CALL_INTENT_VIDEO,
+} from "matrix-js-sdk/lib/matrixrtc";

 // `vi.mock` calls are hoisted above all imports, so the static imports below
 // already see these mocks. Force the iOS platform so that the virtual earpiece
@@ -81,7 +85,7 @@ describe("Default selection", () => {
    const output = new IOSControlledAudioOutput(
      constant(false),
      testScope,
-      "audio",
+      RTC_CALL_INTENT_AUDIO,
    );
    const selected = latestSelection(output);

@@ -100,7 +104,7 @@ describe("Default selection", () => {
    const output = new IOSControlledAudioOutput(
      constant(false),
      testScope,
-      "video",
+      RTC_CALL_INTENT_VIDEO,
    );
    const selected = latestSelection(output);

@@ -113,7 +117,7 @@ describe("Default selection", () => {
    const output = new IOSControlledAudioOutput(
      constant(false),
      testScope,
-      "audio",
+      RTC_CALL_INTENT_AUDIO,
    );
    const selected = latestSelection(output);

@@ -130,7 +134,7 @@ describe("Explicit selection", () => {
    const output = new IOSControlledAudioOutput(
      constant(false),
      testScope,
-      "audio",
+      RTC_CALL_INTENT_AUDIO,
    );
    const selected = latestSelection(output);

@@ -150,7 +154,7 @@ describe("Explicit selection", () => {
    const output = new IOSControlledAudioOutput(
      constant(false),
      testScope,
-      "audio",
+      RTC_CALL_INTENT_AUDIO,
    );
    const selected = latestSelection(output);

@@ -8,7 +8,10 @@ Please see LICENSE in the repository root for full details.
 import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
 import { combineLatest, merge, startWith, Subject, tap } from "rxjs";

-import type { RTCCallIntent } from "matrix-js-sdk/lib/matrixrtc";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  type RTCCallIntent,
+} from "matrix-js-sdk/lib/matrixrtc";
 import {
  availableOutputDevices$ as controlledAvailableOutputDevices$,
  outputDevice$ as controlledOutputSelection$,
@@ -111,7 +114,7 @@ export class IOSControlledAudioOutput implements MediaDevice<
   * Chooses the default output device when no explicit selection (from the user
   * or the hosting application) has been made yet.
   *
-   * For voice calls (`initialIntent === "audio"`) we want to start on the
+   * For voice calls (`initialIntent === "m.audio"`) we want to start on the
   * earpiece rather than the speaker, like a regular phone call. We only
   * override when the device that would otherwise be the default is the
   * speaker: if the host already routed to a headset (e.g. Bluetooth) — which
@@ -122,7 +125,7 @@ export class IOSControlledAudioOutput implements MediaDevice<
    available: Map<string, AudioOutputDeviceLabel>,
  ): string | undefined {
    const firstId = available.keys().next().value;
-    if (this.initialIntent === "audio") {
+    if (this.initialIntent === RTC_CALL_INTENT_AUDIO) {
      const firstLabel =
        firstId !== undefined ? available.get(firstId) : undefined;
      if (firstLabel?.type === "speaker") {
@@ -21,6 +21,7 @@ import { type MediaDevices } from "./MediaDevices";
 import { MuteStates } from "./MuteStates";
 import { nullHostBridge } from "../HostBridge";
 import { flushPromises, mockMediaDevices, testScope } from "../utils/test";
+import { type BackgroundEffectsState } from "./BackgroundEffects";

 describe("LobbyMediaViewModel", () => {
  it("opens what starts enabled in one capture and follows the devices", async () => {
@@ -135,7 +136,7 @@ function setup(
    muteStates,
    mediaDevices,
    {},
-    constant(processor),
+    constant<BackgroundEffectsState>({ supported: true, processor }),
  );
  return { media, vm, muteStates, mediaDevices };
 }
@@ -13,7 +13,6 @@ import {
  type ObservableScope,
  type PublishRequest,
  trackBySource$,
-  type VideoProcessor,
 } from "@element-hq/matrixrtc-sdk";
 import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
 import { combineLatest, debounceTime, map } from "rxjs";
@@ -25,6 +24,7 @@ import {
 import { type MediaDevices } from "./MediaDevices";
 import { type MuteStates } from "./MuteStates";
 import { followDevice, followProcessor } from "./localMediaControls";
+import { type BackgroundEffectsState } from "./BackgroundEffects";

 export interface LobbyMediaViewModel {
  /** The camera, once open, for the preview. */
@@ -42,9 +42,13 @@ export function createLobbyMediaViewModel(
  muteStates: MuteStates,
  mediaDevices: MediaDevices,
  capture: CaptureSettings,
-  videoProcessor$: Behavior<VideoProcessor | undefined>,
+  /** No camera is opened while its pipeline is prepared: it would show the room without the processor. */
+  backgroundEffectsState$: Behavior<BackgroundEffectsState>,
 ): LobbyMediaViewModel {
  const logger = rootLogger.getChild("[LobbyMedia]");
+  const videoProcessor$ = scope.behavior(
+    backgroundEffectsState$.pipe(map(({ processor }) => processor)),
+  );
  const microphone$ = trackBySource$(
    scope,
    media.tracks$,
@@ -64,13 +68,19 @@ export function createLobbyMediaViewModel(
    muteStates.video.enabled$,
    microphone$,
    camera$,
+    backgroundEffectsState$,
  ])
    .pipe(debounceTime(0), scope.bind())
-    .subscribe(([audio, video, microphone, camera]) => {
+    .subscribe(([audio, video, microphone, camera, { wait }]) => {
      const open: PublishRequest[] = [];
      if (audio && microphone === undefined && !opening.has("microphone"))
        open.push(publishRequest("microphone", mediaDevices, capture));
-      if (video && camera === undefined && !opening.has("camera"))
+      if (
+        video &&
+        camera === undefined &&
+        !opening.has("camera") &&
+        wait !== "preparing"
+      )
        open.push(
          publishRequest(
            "camera",
@@ -35,6 +35,10 @@ import { AudioOutput, MediaDevices } from "./MediaDevices";
 import { AndroidControlledAudioOutput } from "./AndroidControlledAudioOutput";
 import { IOSControlledAudioOutput } from "./IOSControlledAudioOutput";
 import { ObservableScope } from "./ObservableScope";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  RTC_CALL_INTENT_VIDEO,
+} from "matrix-js-sdk/lib/matrixrtc";

 // Which audio output implementation is used is decided by what the app hosting
 // Element Call told it, rather than being discovered from the environment.
@@ -52,7 +56,7 @@ describe("MediaDevices audio output", () => {

    const devices = new MediaDevices(new ObservableScope(), {
      controlledAudioDevices: true,
-      callIntent: "audio",
+      callIntent: RTC_CALL_INTENT_AUDIO,
    });

    expect(devices.audioOutput).toBeInstanceOf(AndroidControlledAudioOutput);
@@ -63,7 +67,7 @@ describe("MediaDevices audio output", () => {

    const devices = new MediaDevices(new ObservableScope(), {
      controlledAudioDevices: true,
-      callIntent: "video",
+      callIntent: RTC_CALL_INTENT_VIDEO,
    });

    expect(devices.audioOutput).toBeInstanceOf(IOSControlledAudioOutput);
@@ -6,7 +6,11 @@ Please see LICENSE in the repository root for full details.
 */

 import { test, expect } from "vitest";
-import { type RTCCallIntent } from "matrix-js-sdk/lib/matrixrtc";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  RTC_CALL_INTENT_VIDEO,
+  type RTCCallIntent,
+} from "matrix-js-sdk/lib/matrixrtc";

 import { calculateInitialMuteState } from "./initialMuteState";

@@ -14,10 +18,10 @@ test.each<{
  callIntent: RTCCallIntent;
  allowJoinUnmutedViaIntent: boolean;
 }>([
-  { callIntent: "audio", allowJoinUnmutedViaIntent: false },
-  { callIntent: "audio", allowJoinUnmutedViaIntent: true },
-  { callIntent: "video", allowJoinUnmutedViaIntent: false },
-  { callIntent: "video", allowJoinUnmutedViaIntent: true },
+  { callIntent: RTC_CALL_INTENT_AUDIO, allowJoinUnmutedViaIntent: false },
+  { callIntent: RTC_CALL_INTENT_AUDIO, allowJoinUnmutedViaIntent: true },
+  { callIntent: RTC_CALL_INTENT_VIDEO, allowJoinUnmutedViaIntent: false },
+  { callIntent: RTC_CALL_INTENT_VIDEO, allowJoinUnmutedViaIntent: true },
  { callIntent: "unknown", allowJoinUnmutedViaIntent: false },
  { callIntent: "unknown", allowJoinUnmutedViaIntent: true },
 ])(
@@ -29,15 +33,15 @@ test.each<{
      allowJoinUnmutedViaIntent,
    );
    expect(audioEnabled).toBe(true);
-    expect(videoEnabled).toBe(callIntent !== "audio");
+    expect(videoEnabled).toBe(callIntent !== RTC_CALL_INTENT_AUDIO);
  },
 );

 test.each<{
  callIntent: RTCCallIntent;
 }>([
-  { callIntent: "audio" },
-  { callIntent: "video" },
+  { callIntent: RTC_CALL_INTENT_AUDIO },
+  { callIntent: RTC_CALL_INTENT_VIDEO },
  { callIntent: "unknown" },
 ])(
  "Should always mute on start if skipping lobby and the host does not vouch for the intent (callIntent: $callIntent)",
@@ -55,8 +59,8 @@ test.each<{
 test.each<{
  callIntent: RTCCallIntent;
 }>([
-  { callIntent: "audio" },
-  { callIntent: "video" },
+  { callIntent: RTC_CALL_INTENT_AUDIO },
+  { callIntent: RTC_CALL_INTENT_VIDEO },
  { callIntent: "unknown" },
 ])(
  "Can start unmuted if skipping lobby and the host vouches for the intent (callIntent: $callIntent)",
@@ -67,6 +71,6 @@ test.each<{
      true,
    );
    expect(audioEnabled).toBe(true);
-    expect(videoEnabled).toBe(callIntent !== "audio");
+    expect(videoEnabled).toBe(callIntent !== RTC_CALL_INTENT_AUDIO);
  },
 );
@@ -6,7 +6,10 @@ Please see LICENSE in the repository root for full details.
 */

 import { logger } from "matrix-js-sdk/lib/logger";
-import { type RTCCallIntent } from "matrix-js-sdk/lib/matrixrtc";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  type RTCCallIntent,
+} from "matrix-js-sdk/lib/matrixrtc";

 /**
 * Calculates the initial mute state for media devices based on configuration.
@@ -40,10 +43,10 @@ export function calculateInitialMuteState(

  // A host that vouches for the intent is a trusted environment, so it allows
  // unmuted by default. Same for when showing a lobby, as users can adjust
-  // their settings there. Additionally, if the call intent is "audio", we
+  // their settings there. Additionally, if the call intent is "m.audio", we
  // disable video by default.
  return {
    audioEnabled: true,
-    videoEnabled: callIntent != "audio",
+    videoEnabled: callIntent != RTC_CALL_INTENT_AUDIO,
  };
 }
@@ -21,6 +21,8 @@ import { type Handler } from "./MuteStates";
 /** How a mute switch gets a track where there is none. */
 export interface TrackSupply {
  publish: () => Promise<unknown>;
+  /** Awaited before a switch turned on acts: a camera waits for its processor to be known. */
+  ready?: () => Promise<unknown>;
 }

 /**
@@ -45,6 +47,7 @@ export function followMuteState(
  let desired = muteState.enabled$.value;
  muteState.setHandler(async (enabled) => {
    desired = enabled;
+    if (enabled) await supply.ready?.();
    const track = track$.value;
    if (track !== undefined) {
      desired = await track.setEnabled(enabled);
@@ -94,18 +97,23 @@ export function followDevice(
 /**
 * Attached once the camera track exists rather than carried in the publish
 * request, so that a new track and a change of processor take the same path.
+ * Answers with a wait for the latest attachment to have gone through: a
+ * camera turned on before then would restart without its processor.
 */
 export function followProcessor(
  scope: ObservableScope,
  camera$: Behavior<LocalVideoMediaTrack | undefined>,
  videoProcessor$: Behavior<VideoProcessor | undefined>,
  logger: Logger,
-): void {
+): () => Promise<void> {
+  let attached = Promise.resolve();
  combineLatest([camera$, videoProcessor$])
    .pipe(scope.bind())
    .subscribe(([camera, processor]) => {
-      camera?.setProcessor(processor).catch((e) => {
-        logger.error("Failed to set the video processor", e);
-      });
+      attached =
+        camera?.setProcessor(processor).catch((e) => {
+          logger.error("Failed to set the video processor", e);
+        }) ?? Promise.resolve();
    });
+  return async () => attached;
 }
@@ -8,7 +8,10 @@ Please see LICENSE in the repository root for full details.
 import { test, expect } from "vitest";
 import { act, render, screen } from "@testing-library/react";
 import { axe } from "vitest-axe";
-import { type MatrixRTCSession } from "matrix-js-sdk/lib/matrixrtc";
+import {
+  RTC_CALL_INTENT_AUDIO,
+  type MatrixRTCSession,
+} from "matrix-js-sdk/lib/matrixrtc";
 import { BehaviorSubject } from "rxjs";

 import { GridTile } from "./GridTile";
@@ -122,7 +125,7 @@ test("GridTile displays ringing media", async () => {
  const vm = createRingingMedia({
    pickupState$,
    id: "test",
-    intent: "audio",
+    intent: RTC_CALL_INTENT_AUDIO,
    userId: "@alice:example.org",
    displayName$: constant("Alice"),
    mxcAvatarUrl$: constant(undefined),
@@ -12,6 +12,7 @@ import {
  EndCallIcon,
 } from "@vector-im/compound-design-tokens/assets/web/icons";
 import { useTranslation } from "react-i18next";
+import { RTC_CALL_INTENT_VIDEO } from "matrix-js-sdk/lib/matrixrtc";

 import { type RingingMediaViewModel } from "../state/media/RingingMediaViewModel";
 import { useBehavior } from "../useBehavior";
@@ -25,7 +26,7 @@ export const RingingStatus: FC<Props> = ({ vm }) => {
  const pickupState = useBehavior(vm.pickupState$);
  const Icon =
    pickupState === "ringing"
-      ? vm.intent === "video"
+      ? vm.intent === RTC_CALL_INTENT_VIDEO
        ? VideoCallSolidIcon
        : VoiceCallSolidIcon
      : EndCallIcon;
@@ -11,6 +11,7 @@ import { axe } from "vitest-axe";
 import userEvent from "@testing-library/user-event";
 import { TooltipProvider } from "@vector-im/compound-web";
 import { BehaviorSubject } from "rxjs";
+import { RTC_CALL_INTENT_AUDIO } from "matrix-js-sdk/lib/matrixrtc";

 import { SpotlightTile } from "./SpotlightTile";
 import {
@@ -176,7 +177,7 @@ test("SpotlightTile displays ringing media", async () => {
  const vm = createRingingMedia({
    pickupState$,
    id: "test",
-    intent: "audio",
+    intent: RTC_CALL_INTENT_AUDIO,
    userId: "@alice:example.org",
    displayName$: constant("Alice"),
    mxcAvatarUrl$: constant(undefined),
@@ -13,13 +13,20 @@ import {
 } from "@livekit/track-processors";
 import { ImageSegmenter } from "@mediapipe/tasks-vision";

-import { BackgroundEffectTransformer } from "./BackgroundEffectTransformer";
+import {
+  BackgroundEffectTransformer,
+  canSegment,
+} from "./BackgroundEffectTransformer";

 const frame = {} as VideoFrame;
 const controller = {} as TransformStreamDefaultController<VideoFrame>;
+const stats = { processingTimeMs: 1, segmentationTimeMs: 1, filterTimeMs: 0 };

 describe("BackgroundEffectTransformer", () => {
-  afterEach(() => vi.restoreAllMocks());
+  afterEach(() => {
+    vi.restoreAllMocks();
+    vi.unstubAllGlobals();
+  });

  it("loads a background chosen before it was built", async () => {
    vi.spyOn(VideoTransformer.prototype, "init").mockResolvedValue();
@@ -41,10 +48,29 @@ describe("BackgroundEffectTransformer", () => {
    expect(load).toHaveBeenCalledWith("/background.jpg");
  });

+  it("runs MediaPipe from its own bundle", async () => {
+    vi.spyOn(VideoTransformer.prototype, "init").mockResolvedValue();
+    const create = vi
+      .spyOn(ImageSegmenter, "createFromOptions")
+      .mockResolvedValue({} as ImageSegmenter);
+
+    await new BackgroundEffectTransformer({ backgroundDisabled: true }).init(
+      {} as VideoTransformerInitOptions,
+    );
+
+    const [fileset, options] = create.mock.calls[0];
+    expect(fileset.wasmLoaderPath).toMatch(/vision_wasm_internal\.js/);
+    expect(fileset.wasmBinaryPath).toMatch(/vision_wasm_internal\.wasm/);
+    expect(options.baseOptions?.modelAssetPath).toMatch(
+      /selfie_segmenter\.tflite/,
+    );
+  });
+
  it("reports the first frame carrying an effect, once", async () => {
+    // The library reports every frame it draws.
    vi.spyOn(BackgroundTransformer.prototype, "transform").mockImplementation(
      async function (this: BackgroundTransformer) {
-        this.isFirstFrame = false;
+        this.options.onFrameProcessed?.(stats);
        return Promise.resolve();
      },
    );
@@ -67,6 +93,37 @@ describe("BackgroundEffectTransformer", () => {
    expect(transformer.onFirstFrame).not.toHaveBeenCalled();
  });

+  it("keeps reporting drawn frames to whoever asked", async () => {
+    vi.spyOn(BackgroundTransformer.prototype, "transform").mockImplementation(
+      async function (this: BackgroundTransformer) {
+        this.options.onFrameProcessed?.(stats);
+        return Promise.resolve();
+      },
+    );
+    const onFrameProcessed = vi.fn();
+    const transformer = new BackgroundEffectTransformer({ onFrameProcessed });
+
+    await transformer.transform(frame, controller);
+    expect(onFrameProcessed).toHaveBeenCalledWith(stats);
+  });
+
+  it("never sends the first frame on untouched", async () => {
+    // The library copies its first frame through untouched, ahead of the
+    // drawn one, unless told the first frame is past
+    const seen: boolean[] = [];
+    vi.spyOn(BackgroundTransformer.prototype, "transform").mockImplementation(
+      async function (this: BackgroundTransformer) {
+        seen.push(this.isFirstFrame);
+        return Promise.resolve();
+      },
+    );
+    const transformer = new BackgroundEffectTransformer({});
+    expect(transformer.isFirstFrame).toBe(true);
+
+    await transformer.transform(frame, controller);
+    expect(seen).toEqual([false]);
+  });
+
  it("passes frames on untouched once disabled, even after blurring", async () => {
    const transformer = new BackgroundEffectTransformer({});
    await transformer.update({ blurRadius: 15, backgroundDisabled: false });
@@ -78,4 +135,33 @@ describe("BackgroundEffectTransformer", () => {
    expect(transformer.options.blurRadius).toBeUndefined();
    expect(transformer.options.imagePath).toBeUndefined();
  });
+
+  it("segments on the CPU where the GPU refuses", async () => {
+    vi.spyOn(VideoTransformer.prototype, "init").mockResolvedValue();
+    const build = vi
+      .spyOn(ImageSegmenter, "createFromOptions")
+      .mockRejectedValueOnce(new Error("blocklisted"))
+      .mockResolvedValue({} as ImageSegmenter);
+
+    await new BackgroundEffectTransformer({}).init(
+      {} as VideoTransformerInitOptions,
+    );
+    expect(build.mock.calls.map(([, o]) => o.baseOptions?.delegate)).toEqual([
+      "GPU",
+      "CPU",
+    ]);
+  });
+
+  it("says whether a segmenter can be built at all", async () => {
+    vi.stubGlobal("OffscreenCanvas", class {});
+    const close = vi.fn();
+    const build = vi
+      .spyOn(ImageSegmenter, "createFromOptions")
+      .mockResolvedValue({ close } as unknown as ImageSegmenter);
+    expect(await canSegment()).toBe(true);
+    expect(close).toHaveBeenCalled();
+
+    build.mockRejectedValue(new Error("no model"));
+    expect(await canSegment()).toBe(false);
+  });
 });
@@ -11,11 +11,18 @@ import {
  VideoTransformer,
  type VideoTransformerInitOptions,
  type BackgroundOptions,
+  type SegmenterOptions,
 } from "@livekit/track-processors";
 import { logger } from "matrix-js-sdk/lib/logger";
 import { ImageSegmenter } from "@mediapipe/tasks-vision";

-import modelAssetPath from "../mediapipe/imageSegmenter/selfie_segmenter.tflite?url";
+// Only the SIMD build of MediaPipe's runtime: every browser Element Call
+// supports has WebAssembly SIMD. As files rather than data URLs, so that a
+// host's bundler serves them as assets of its own, and a browser fetches them
+// only when an effect is first switched on.
+import wasmLoaderPath from "@mediapipe/tasks-vision/vision_wasm_internal.js?url&no-inline";
+import wasmBinaryPath from "@mediapipe/tasks-vision/vision_wasm_internal.wasm?url&no-inline";
+import modelAssetPath from "../mediapipe/imageSegmenter/selfie_segmenter.tflite?url&no-inline";

 interface WasmFileset {
  /** The path to the Wasm loader script. */
@@ -24,26 +31,7 @@ interface WasmFileset {
  wasmBinaryPath: string;
 }

-// The MediaPipe package, by default, ships some alternative versions of the
-// WASM files which avoid SIMD for compatibility with older browsers. But SIMD
-// in WASM is actually fine by our support policy, so we include just the SIMD
-// versions.
-// It's really not ideal that we have to reference these internal files from
-// MediaPipe and depend on node_modules having this specific structure. It's
-// easy to see this breaking if our dependencies changed and MediaPipe were
-// no longer hoisted, or if we switched to another dependency loader such as
-// yarn PnP.
-// https://github.com/google-ai-edge/mediapipe/issues/5961
-const wasmFileset: WasmFileset = {
-  wasmLoaderPath: new URL(
-    "../../node_modules/@mediapipe/tasks-vision/wasm/vision_wasm_internal.js",
-    import.meta.url,
-  ).href,
-  wasmBinaryPath: new URL(
-    "../../node_modules/@mediapipe/tasks-vision/wasm/vision_wasm_internal.wasm",
-    import.meta.url,
-  ).href,
-};
+const wasmFileset: WasmFileset = { wasmLoaderPath, wasmBinaryPath };

 /**
 * Track processor that applies effects such as blurring to a user's background.
@@ -53,18 +41,41 @@ const wasmFileset: WasmFileset = {
 * resource fetched from the public internet.
 */
 export class BackgroundEffectTransformer extends BackgroundTransformer {
-  /** Called once, when the first frame carrying an effect has been drawn. */
+  /**
+   * Called once, when the first frame carrying an effect has been drawn.
+   * This is used for drewAFrame$ to show the correct loading state.
+   */
  public onFirstFrame: (() => void) | undefined;
+  private drewAFrame = false;
+
+  public constructor(opts: BackgroundOptions) {
+    super(opts);
+    // Called by the library for a frame it drew, never for one it passed on
+    // untouched.
+    this.options.onFrameProcessed = (stats): void => {
+      opts.onFrameProcessed?.(stats);
+      if (this.drewAFrame) return;
+      this.drewAFrame = true;
+      this.onFirstFrame?.();
+    };
+  }

  public override async transform(
    frame: VideoFrame,
    controller: TransformStreamDefaultController<VideoFrame>,
  ): Promise<void> {
-    const priming = this.isFirstFrame;
+    // From the LK docs:
+    // > Render a copy of the first frame is rendered to the screen as soon as possible to act
+    // > as a less jarring initial state than a solid color while the synchronous work below
+    // > (segmentation + frame rendering) occurs.
+    // We do not want this copy (it leaks unblurred media).
+    // So we always set `isFirstFrame` to false -> never show an unprocessed frame.
+    // It does not have any impact other than setting the unprocessed frame while loading the processor.
+    //
+    // To not show the blank frame, we have our own mechanism for hiding the loading overlay:
+    // `onFirstFrame` is what the EC UI will listen to and only then show the video feed.
+    this.isFirstFrame = false;
    await super.transform(frame, controller);
-    // Cleared only once a frame has been through the segmenter; a frame
-    // passed through untouched leaves it set.
-    if (priming && !this.isFirstFrame) this.onFirstFrame?.();
  }

  /**
@@ -92,17 +103,10 @@ export class BackgroundEffectTransformer extends BackgroundTransformer {
      inputElement: inputVideo,
    });

-    this.imageSegmenter = await ImageSegmenter.createFromOptions(wasmFileset, {
-      baseOptions: {
-        modelAssetPath,
-        delegate: "GPU",
-        ...this.options.segmenterOptions,
-      },
-      canvas: this.canvas,
-      runningMode: "VIDEO",
-      outputCategoryMask: true,
-      outputConfidenceMasks: false,
-    });
+    this.imageSegmenter = await createSegmenter(
+      this.canvas,
+      this.options.segmenterOptions,
+    );

    // BackgroundTransformer's own init applies these, and this one replaces it.
    if (this.options.imagePath) {
@@ -116,3 +120,38 @@ export class BackgroundEffectTransformer extends BackgroundTransformer {
    this.gl?.setBackgroundDisabled(this.options.backgroundDisabled ?? false);
  }
 }
+
+/**
+ * Whether a segmenter can be built here, asked once before any camera depends
+ * on one: a pipeline that fails to build takes the camera down with it.
+ */
+export async function canSegment(): Promise<boolean> {
+  try {
+    (await createSegmenter(new OffscreenCanvas(1, 1))).close();
+    return true;
+  } catch (e) {
+    logger.warn("Background effects cannot run here", e);
+    return false;
+  }
+}
+
+/** On the GPU, or on the CPU where the GPU refuses, as a blocklisted one does. */
+async function createSegmenter(
+  canvas: HTMLCanvasElement | OffscreenCanvas | undefined,
+  segmenterOptions?: SegmenterOptions,
+): Promise<ImageSegmenter> {
+  const build = async (delegate: "GPU" | "CPU"): Promise<ImageSegmenter> =>
+    ImageSegmenter.createFromOptions(wasmFileset, {
+      baseOptions: { modelAssetPath, delegate, ...segmenterOptions },
+      canvas,
+      runningMode: "VIDEO",
+      outputCategoryMask: true,
+      outputConfidenceMasks: false,
+    });
+  try {
+    return await build("GPU");
+  } catch (e) {
+    logger.warn("Segmenting on the CPU, as the GPU refused", e);
+    return build("CPU");
+  }
+}
@@ -56,7 +56,8 @@ vi.mock("./backgroundImages", async () => {
  };
 });
 vi.mock("./BackgroundEffectTransformer", () => ({
-  BackgroundEffectTransformer: vi.fn(),
+  BackgroundEffectTransformer: vi.fn(function () {}),
+  canSegment: async (): Promise<boolean> => Promise.resolve(true),
 }));

 describe("BackgroundEffectsProvider", () => {
@@ -12,7 +12,10 @@ import {
 import { createContext, type FC, type JSX, use } from "react";

 import { backgroundEffect as backgroundEffectSetting } from "../settings/settings";
-import { BackgroundEffectTransformer } from "./BackgroundEffectTransformer";
+import {
+  BackgroundEffectTransformer,
+  canSegment,
+} from "./BackgroundEffectTransformer";
 import { addedBackgrounds } from "./backgroundImages";
 import { type Behavior } from "../state/Behavior";
 import { type ObservableScope } from "../state/ObservableScope";
@@ -56,6 +59,7 @@ export function createAppBackgroundEffects(
    effect$: backgroundEffectSetting.value$,
    setEffect: backgroundEffectSetting.setValue,
    added$: addedBackgrounds.added$,
+    canSegment,
    pipeline: new BackgroundProcessorWrapper(transformer, "background-effect"),
    transformer,
  });
@@ -5,8 +5,8 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
 Please see LICENSE in the repository root for full details.
 */

-import arc from "../graphics/background-arc.jpg?url";
-import glow from "../graphics/background-glow.jpg?url";
+import arc from "../graphics/background-arc.jpg?url&no-inline";
+import glow from "../graphics/background-glow.jpg?url&no-inline";

 export interface ShippedBackground {
  /** Stored in the user's settings when chosen, so it must not change. */
@@ -217,7 +217,7 @@ export function getBasicCallViewModelEnvironment(
    vm,
    muteStates,
    mediaDevices,
-    constant(false),
+    constant({ supported: false, processor: undefined }),
    "reactionId",
    { showControls: true, header: HeaderStyle.Standard },
  );
@@ -6,6 +6,7 @@ Please see LICENSE in the repository root for full details.
 */

 import { defineConfig } from "vite";
+import { posix } from "node:path";

 import { vitePluginsConfig } from "./vite.config.ts";
 import { scopeStylesToRoot } from "./component/build/scopeStylesToRoot.ts";
@@ -34,6 +35,25 @@ export default defineConfig(({ mode }) => {
    // whatever is in `public` — including the developer's own config.json, which
    // is not in the repository — into the output we would publish.
    publicDir: false,
+    // A library does not know where it will be served from, so every file it
+    // emits beside itself — the sounds, the pictures, MediaPipe's runtime — is
+    // referenced relative to the module's own URL. (A library build inlines
+    // each asset as a data URL unless its import says `no-inline`, so the
+    // imports do.)
+    base: "./",
+    experimental: {
+      // Relative to the chunk that references it, wherever that chunk lands,
+      // and spelled `./assets/…` rather than `assets/…`: a host's bundler reads
+      // a bare name as a package to look up.
+      renderBuiltUrl: (filename, { hostId, hostType }) => {
+        if (hostType !== "js") return { relative: true };
+        const relative = posix.relative(posix.dirname(hostId), filename);
+        const specifier = relative.startsWith(".") ? relative : `./${relative}`;
+        return {
+          runtime: `new URL(${JSON.stringify(specifier)}, import.meta.url).href`,
+        };
+      },
+    },
    // A host's document is not ours to style: everything in the stylesheet is
    // confined to the element Element Call is mounted in
    css: { postcss: { plugins: [scopeStylesToRoot()] } },
@@ -64,6 +84,15 @@ export default defineConfig(({ mode }) => {
        cssFileName: "element-call",
      },
      rollupOptions: {
+        output: {
+          // Hashed, as the app's are, so that a file a host serves as it is
+          // stays correct across versions of the package. Not the stylesheet,
+          // which the package exports by name (see `cssFileName`).
+          assetFileNames: ({ names }) =>
+            names.some((name) => name.endsWith(".css"))
+              ? "[name][extname]"
+              : "assets/[name]-[hash][extname]",
+        },
        // The host already has these, and a second copy of any of them does not
        // merely bloat the bundle: React would hold two sets of hooks, the
        // Matrix client would run two sync loops, and a second Compound would
Author	SHA1	Message	Date
Johannes Marbach	d5f72f464d	Compact AGENTS.md and split hand-off out into a skill	2026-11-11 16:33:46 +01:00
renovate[bot]	96949a57da	Update ghcr.io/element-hq/element-web:develop Docker digest to 8dc3767 (#4447 ) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2026-11-11 14:47:22 +00:00
Johannes Marbach	2e05ef77a0	Merge pull request #4441 from element-hq/johannes/flakes Try to fix flaky background tests	2026-11-11 15:37:31 +01:00
Johannes Marbach	73826ec4f9	Reformat	2026-11-11 15:29:03 +01:00
Johannes Marbach	1135c02e6f	Merge branch 'main' into johannes/flakes	2026-11-11 13:53:30 +01:00
Johannes MarbachandTimo	9dc425b5f8	Apply batched suggestions from code review Improve comments Co-authored-by: Timo <16718859+toger5@users.noreply.github.com>	2026-11-11 13:52:05 +01:00
Johannes Marbach	c1b94b62dd	Merge pull request #4434 from element-hq/johannes/intent Stabilise m.call.intent	2026-11-11 11:02:22 +01:00
Johannes Marbach	fa6efd9dad	Merge branch 'main' into johannes/intent	2026-11-11 10:45:51 +01:00
Johannes Marbach	6906404a64	Pin ubuntu image to 24.04 because 26.04 breaks Playwright, renovate should update it	2026-11-11 10:20:44 +01:00
Johannes Marbach	98e8343d0b	Stabilise m.call.intent	2026-11-11 10:10:33 +01:00
Timo	88659d6498	Merge pull request #4444 from element-hq/toger5/better-component-readme Shorten component README	2026-11-10 16:56:40 +01:00
Timo K.	704cbd65a6	Shorten component readme - At the end there were not that many unique facts in there. It now only lists the relevant bits without much text around it.	2026-11-10 14:31:12 +01:00
Timo	5f59283b4a	Merge pull request #4428 from element-hq/toger5/component-assets-as-files Ship the component's assets as files beside the module	2026-11-10 14:18:30 +01:00
Johannes Marbach	07dc08cd1c	Merge pull request #4440 from element-hq/johannes/embedded Make one site use the embedded build in playwright tests	2026-11-10 07:46:51 +01:00
Johannes Marbach	60d6029184	Delint	2026-11-10 07:38:23 +01:00
Johannes Marbach	f41bc2589a	Merge branch 'main' into johannes/embedded	2026-11-10 07:36:15 +01:00
Johannes Marbach	ceeb26b271	Remove test.slow()	2026-11-10 07:30:57 +01:00
Johannes Marbach	e822f967c1	Merge pull request #4442 from element-hq/johannes/shard Shard playwright tests	2026-11-10 07:22:10 +01:00
renovate[bot]	ec46b04b71	Update ghcr.io/element-hq/element-web:develop Docker digest to 37d88b4 (#4443 ) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2026-11-10 03:24:30 +00:00
Johannes Marbach	4f3dc9af75	Shard playwright tests	2026-11-09 13:00:52 +01:00
Johannes Marbach	cfd8312e46	Try to fix flaky background tests	2026-11-09 12:20:00 +01:00
renovate[bot]	38edc47f64	Update ghcr.io/element-hq/element-web:develop Docker digest to 0c39f7b (#4439 ) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2026-11-09 09:29:45 +00:00
Johannes Marbach	ab4f0f0f77	Make one site use the embedded build in playwright tests	2026-11-09 10:05:04 +01:00
renovate[bot]	9411a56afc	Update ghcr.io/element-hq/element-web:develop Docker digest to f30be05 (#4438 ) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2026-11-08 01:46:48 +00:00
Timo K.andClaude Fable 5.1	a5374791e4	Resolve asset references from the chunk that holds them The runtime path was `./` plus the asset's path from the package root, which is right for every chunk the build emits today, all of which sit at that root, and wrong for one emitted under a directory. Vite names the referencing chunk, so the path is now computed relative to it. The sound loader kept a branch for sounds inlined as data URLs, with a comment saying the component build produces them. No build does any more, and no host would inline a sound of this size, so the branch and its tests go. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-11-07 16:06:05 +01:00
renovate[bot]	a0601876d8	Update ghcr.io/element-hq/element-web:develop Docker digest to eada111 (#4435 ) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2026-11-07 01:07:59 +00:00
fkwp	eef86fc06e	Merge pull request #4292 from element-hq/spec/background-effects/12-unbuildable-pipeline Offer no effects for the session where the pipeline cannot be built	2026-11-06 18:27:12 +01:00
Timo K.	cd5c367ce9	fix race in processor setup vs camera start	2026-11-06 16:54:42 +01:00
Timo K.andClaude Fable 5.1	190902eec0	Test the session where the pipeline cannot be built - The effects state's timeline through preparing, settling and attached, and that a failed trial build withholds the effects for the session while keeping the choice. - The footer offers nothing once the pipeline fails to build, and passes on the wait as the state's single wait field. - The call leaves the camera out of the join while the pipeline is prepared, publishes it once that is known, and turns on a camera turned off before the processor was chosen; the lobby opens no camera meanwhile. Both view models are driven by one state subject. - The transformer's trial build answers whether a segmenter can be made. - End to end, a browser whose pipeline cannot be built sees the effects withheld. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-11-06 16:54:42 +01:00
Timo K.andClaude Opus 5.5	88f748ec21	Offer no effects for the session where the pipeline cannot be built - A segmenter that can't be built, with the model unreachable or a GPU its delegate refuses, took the camera down with the pipeline: the SDK fails the track it was attaching to, and the lobby muted camera and microphone, on every load while the choice was kept. - One is now built on trial when an effect is first wanted, before the pipeline is attached to anything, on the CPU where the GPU refuses. Where it can't be built, the options are disabled with the existing notice for the rest of the session. The choice is kept: a browser update may fix what failed. - The view model takes that verdict from the pipeline's state rather than asking the SDK, so both say the same, and it no longer builds a WebGL context of its own to ask. - Until the trial is over, the lobby opens no camera and the call makes no camera track, so neither starts without the effect. Rebased onto main, where the Publisher is gone and the call view model publishes the camera through the SDK: - CallViewModel takes videoProcessorPreparing$. While it is true, the camera's mute handler waits before making a track, and the join leaves the camera out of its requests (initialPublishRequests); the view model publishes it once it is known whether the pipeline builds. The Publisher's test became CallViewModel tests. Cherry-picked from spec/background-effects/12-unbuildable-pipeline onto toger5/background-effect/camera-preview, where the camera menu's preview takes the camera from the footer's caller rather than from a track synced on the background-effects state. The footer factories take the pipeline's state again, as this commit needs its verdict and not only the settling flag; the synced camera and the preview's wrapper type are not brought along. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-11-06 16:54:42 +01:00
Timo	9639effd28	Merge pull request #4423 from element-hq/toger5/background-effect/camera-preview Camera Preview in video mute button context menu	2026-11-06 15:01:06 +00:00
Timo K.andClaude Fable 5.1	797a86815b	Ship the component's assets as files beside the module The component module was 18 MB, of which 15 MB was MediaPipe's runtime as a base64 data URL, with the segmentation model, every sound and the background pictures inlined the same way: a library build inlines every asset unless its import says `no-inline`. Every host downloaded all of it before the first call, whether or not a background effect was ever switched on, and a host with a Content Security Policy without `script-src data:` could not run effects at all, since MediaPipe injects its loader as a script tag. The imports now say `no-inline`, and the build renders each reference as `new URL("./assets/…", import.meta.url)`, which webpack 5, Vite and Rollup follow as an asset of the host's own; the `./` is spelled out because a bare `assets/…` is read as a package name. The module is 2.3 MB. The runtime is fetched when an effect is first switched on and cached on its own. MediaPipe's files come through the package's `exports` rather than a path into node_modules. MediaPipe's JavaScript (160 kB) stays in the module: LiveKit's track processors import it statically, so a dynamic import moves nothing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-11-06 14:39:25 +01:00