Skip to content

feat(frontend): add Browser Live to Custom Agent chats - #4719

Merged
WillemJiang merged 2 commits into
bytedance:mainfrom
AnnaSuSu:feat/custom-agent-browser-live
Aug 7, 2026
Merged

feat(frontend): add Browser Live to Custom Agent chats#4719
WillemJiang merged 2 commits into
bytedance:mainfrom
AnnaSuSu:feat/custom-agent-browser-live

Conversation

@AnnaSuSu

@AnnaSuSu AnnaSuSu commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Closes #4716

Why

Custom Agents can already use the browser tool group, but their chat route does not mount the Browser Live provider or render its entry point. Users can therefore see browser-produced output without being able to inspect or take over the live per-conversation browser session for navigation, sign-in, CAPTCHA, or other human steps.

What changed

  • Custom Agent chats now mount the shared chat providers, including BrowserViewProvider.
  • Browser Live is available for an existing Custom Agent thread when browser control is globally available and the agent's tool_groups are unrestricted or include browser.
  • Explicit non-browser tool-group allowlists, mock chats, and new threads keep Browser Live hidden. E2E coverage pins each of those boundaries and verifies that the visible trigger opens the URL controls.

Surface area

  • Frontend UI — page / component / setting / interaction under frontend/
  • Backend API — endpoint / SSE event / request-response shape under backend/app
  • Agents / LangGraph — agent node, graph wiring, langgraph.json, or prompt change
  • Sandboxdocker/ or sandboxed execution
  • Skills — change under skills/
  • Dependencies — new/upgraded entry in backend/pyproject.toml or frontend/package.json (say what it buys us)
  • Default behavior change — changes existing behavior without the user opting in (default model, default setting, data shape)
  • Docs / tests / CI only — no runtime behavior change

Screenshots / Recording

No new visual design is introduced: Custom Agent chats reuse the existing workspace Browser Live trigger and panel. The production-build E2E test opens the real trigger and asserts that the Browser Live URL controls appear.

Validation

Frontend, on the final state of this branch:

  • pnpm format — clean
  • pnpm check — ESLint and TypeScript clean
  • pnpm test — 125 test files, 987 tests, 0 failures
  • BETTER_AUTH_SECRET=local-dev-secret pnpm build — succeeded
  • PLAYWRIGHT_SKIP_WEB_SERVER=1 PLAYWRIGHT_BASE_URL=http://127.0.0.1:3017 pnpm test:e2e -- tests/e2e/agent-chat.spec.ts tests/e2e/browser-feature.spec.ts — 13 passed against the production build

The new Browser Live cases were run before the implementation: the explicit-browser and unrestricted-tool-group cases failed because the trigger was absent, while the three hidden-state cases passed. All five passed after the implementation.

Backend code is untouched, so backend suites were not run.

AI assistance

Tool(s) used: Codex

How you used it: Used Codex to inspect the current Custom Agent and Browser Live call paths, implement the shared provider and capability/tool-group gates, add E2E coverage, run red/green and broader frontend validation, and prepare this PR. I reviewed every line and the final diff.

  • I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

Copilot AI lite review requested due to automatic review settings August 7, 2026 01:58
@github-actions github-actions Bot added area:docs Documentation and Markdown only area:frontend Next.js frontend under frontend/ risk:medium Medium risk: regular code changes size/M PR changes 100-300 lines labels Aug 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables the existing “Browser Live” UI in Custom Agent chat threads by mounting the shared chat provider stack (including BrowserViewProvider) and gating visibility based on the global browser-control feature flag plus the agent’s tool_groups configuration.

Changes:

  • Mount shared chat providers for Custom Agent chat routes so Browser Live can function.
  • Add Browser Live gating in Custom Agent chat UI: requires an existing thread, non-mock chat, browser control enabled, and agent tool groups either unrestricted or including browser.
  • Add E2E coverage for Browser Live visibility boundaries in Custom Agent chats and update mock agent typing to support tool_groups.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
README.md Documents Browser Live availability for Custom Agent chats.
frontend/tests/e2e/utils/mock-api.ts Extends MockAgent to support tool_groups in E2E fixtures.
frontend/tests/e2e/agent-chat.spec.ts Adds E2E scenarios asserting when Browser Live should show/hide for Custom Agent threads.
frontend/src/app/workspace/agents/[agent_name]/chats/[thread_id]/page.tsx Implements Browser Live gating and renders BrowserTrigger + passes browserEnabled to ChatBox.
frontend/src/app/workspace/agents/[agent_name]/chats/[thread_id]/layout.tsx Switches to shared ChatProviders so Browser Live provider is mounted.
frontend/AGENTS.md Updates module guidance to reflect new gating behavior for Custom Agent chats.

Comment thread README.md Outdated
Comment thread frontend/tests/e2e/agent-chat.spec.ts
Copilot AI review requested due to automatic review settings August 7, 2026 02:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (1)

frontend/src/app/workspace/agents/[agent_name]/chats/[thread_id]/layout.tsx:3

  • layout.tsx is marked as a client component even though it only renders ChatProviders (a client component) and doesn't use client-only APIs itself. Keeping the layout as a server component (by removing the top-level "use client") preserves the server/client boundary and avoids needlessly forcing this route segment to be client-rendered if the page composition changes later.
"use client";

import { ChatProviders } from "@/components/workspace/chats/chat-providers";

@WillemJiang WillemJiang added this to the 2.1.0 milestone Aug 7, 2026
@WillemJiang
WillemJiang merged commit e5c62ca into bytedance:main Aug 7, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:docs Documentation and Markdown only area:frontend Next.js frontend under frontend/ risk:medium Medium risk: regular code changes size/M PR changes 100-300 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] Support Browser Live in Custom Agent chats when browser tools are enabled

3 participants