-
Notifications
You must be signed in to change notification settings - Fork 690
Add Kevin AI support chat to the studio footer #6159
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
AnkhHeart
wants to merge
20
commits into
master
Choose a base branch
from
ankh/ai-chat-v3
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,749
−14
Open
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
ed7d5a7
feat(support): add Kevin AI support chat to the studio footer
AnkhHeart 4e37e02
feat(v2): point Kevin support at /v2 and expose OBS tools to the agent
AnkhHeart b4669e6
fix(support): render the knowledge-source link footer
AnkhHeart e8ff05e
style(v2): let Prettier own the local protocol.ts copy
AnkhHeart db6e6d7
chore(v2): sync protocol contract to tool version 2
AnkhHeart c118717
feat(v2): update tool protocol version to 3 with poll management enha…
AnkhHeart b0b72b7
Add the agent's support tools: mic filters, health, diagnostics, ticket
AnkhHeart 9dd71b3
Show the interaction quota in support chat, and offer Ultra at the limit
AnkhHeart 3bcec49
Redesign Kevin support chat to match Figma: reposition footer icon, r…
AnkhHeart 2943ad5
refactor: remove unused sendTrigger method from StreamAvatarApiService
AnkhHeart 9217ec1
Address PR review: a11y, safe links, and error/log handling in Kevin …
AnkhHeart 5d5ca87
feat(tool): add watch_replay
AnkhHeart 9ab0472
feat(bubble): add kevin bubble when window is not visible for approvals
AnkhHeart 2e16a08
fix(pr): copilot review fixes
AnkhHeart 50ff66b
chore(protocol): remove outdated comments and documentation
AnkhHeart b6b5086
fix(pr): comments implemented
AnkhHeart 54509dc
feat(analytics): track generic events
AnkhHeart a876a24
fix(kevin-support): reset error state on run start
AnkhHeart a737087
fix(bubble): show a badge when a browser view overlaps.
AnkhHeart 4b43e0f
fix(kevin-support): enforce maximum message length in input field
AnkhHeart File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| @import '../../styles/index'; | ||
|
|
||
| /** | ||
| * A callout above the footer's Kevin icon, pointing down at it. | ||
| * | ||
| * `fixed`, not `absolute`: the footer clips on both axes (`overflow-y: hidden` | ||
| * on `.footer`, `overflow-x: auto` on it and on `.footer--left`), so a child | ||
| * positioned above the bar disappears. Fixed escapes ancestor overflow, and no | ||
| * ancestor sets a transform that would turn it back into a containing block. | ||
| * The component sets left/bottom from the icon's measured rect. | ||
| * | ||
| * Kept in the React tree rather than portalled to document.body: the theme is a | ||
| * class on the main window's root div, and everything below reads its CSS vars. | ||
| * Portalling out renders the card unstyled and light against a dark app. | ||
| * | ||
| * Deliberately the same fills, border and radius as the inline card in | ||
| * KevinSupport.m.less, so the two read as one component in two places. | ||
| */ | ||
| .bubble { | ||
| position: fixed; | ||
| width: 320px; | ||
| padding: 12px 14px; | ||
| background-color: var(--section-alt); | ||
| border: 1px solid var(--border); | ||
| border-radius: 12px; | ||
| box-shadow: 0 4px 12px rgba(0, 0, 0, 0.4); | ||
| color: var(--title); | ||
| font-size: 14px; | ||
| line-height: 1.5; | ||
| // Above the footer and the OBS display, below a real modal (.ant-modal-wrap | ||
| // is 1003 in Main.m.less). | ||
| z-index: 1002; | ||
| cursor: default; | ||
|
|
||
| // The arrow, pointing down at the icon. Two triangles so the border reads as | ||
| // a continuous outline rather than stopping where the bubble ends. Nothing | ||
| // here may sit inside an overflow container -- both are outside the box, and | ||
| // an `overflow` on this element would clip them and add a stray scrollbar. | ||
| &::before, | ||
| &::after { | ||
| content: ''; | ||
| position: absolute; | ||
| top: 100%; | ||
| left: 18px; | ||
| width: 0; | ||
| height: 0; | ||
| border-style: solid; | ||
| } | ||
|
|
||
| &::before { | ||
| border-width: 8px 8px 0 8px; | ||
| border-color: var(--border) transparent transparent transparent; | ||
| } | ||
|
|
||
| &::after { | ||
| border-width: 7px 7px 0 7px; | ||
| border-color: var(--section-alt) transparent transparent transparent; | ||
| left: 19px; | ||
| } | ||
| } | ||
|
|
||
| /* Left accent marks a decision point, matching `.approval` in | ||
| KevinSupport.m.less. Stacked when one turn proposes more than one. */ | ||
| .approval { | ||
| padding-left: 10px; | ||
| border-left: 3px solid var(--warning); | ||
|
|
||
| & + & { | ||
| margin-top: 12px; | ||
| padding-top: 12px; | ||
| border-top: 1px solid var(--border); | ||
| } | ||
| } | ||
|
|
||
| .summary { | ||
| font-weight: 600; | ||
| margin-bottom: 4px; | ||
| } | ||
|
|
||
| .warning { | ||
| color: var(--warning); | ||
| font-size: 12px; | ||
| margin-bottom: 8px; | ||
| } | ||
|
|
||
| .actions { | ||
| display: flex; | ||
| flex-wrap: wrap; | ||
| gap: 8px; | ||
| margin-top: 10px; | ||
|
|
||
| button { | ||
| margin: 0; | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,159 @@ | ||
| import React, { useEffect, useLayoutEffect, useState } from 'react'; | ||
| import { Services } from 'components-react/service-provider'; | ||
| import { useVuex } from 'components-react/hooks'; | ||
| import { $t } from 'services/i18n'; | ||
| import { | ||
| browserViewRects, | ||
| isCoveredByBrowserView, | ||
| IViewRect, | ||
| } from 'components-react/shared/browser-view-rects'; | ||
| import styles from './KevinApprovalBubble.m.less'; | ||
|
|
||
| interface Props { | ||
| /** The footer icon this points at. Measured, never positioned against. */ | ||
| anchorRef: React.RefObject<HTMLElement>; | ||
| } | ||
|
|
||
| /** Must match `.bubble` in KevinApprovalBubble.m.less. */ | ||
| const BUBBLE_WIDTH = 320; | ||
| /** Must match the `bottom` offset applied below. */ | ||
| const ANCHOR_GAP = 10; | ||
| // ponytail: a fixed cap rather than the rendered height. The bubble grows with the | ||
| // number of pending approvals, but anything covering it is essentially always the | ||
| // whole page container, so an exact height buys nothing — and measuring the bubble | ||
| // to decide whether to draw the bubble flickers. | ||
| const BUBBLE_MAX_HEIGHT = 240; | ||
|
|
||
| /** | ||
| * The approval prompt, shown above the Kevin icon in the studio footer. | ||
| * | ||
| * An approval used to force the support window open and focus it, which is an | ||
| * ambush mid-stream: a window jumps in front of whatever the streamer was doing, | ||
| * for a decision that is usually a single "yes". This carries the whole decision | ||
| * instead, so they answer without leaving the editor. | ||
| * | ||
| * Positioned `fixed` from the icon's measured rect rather than absolutely inside | ||
| * the footer, because the footer scrolls: `.footer` sets `overflow-y: hidden` | ||
| * and `overflow-x: auto`, and `.footer--left` another `overflow-x: auto`. A | ||
| * child positioned above the bar is clipped by both, which is invisible rather | ||
| * than merely misplaced. Fixed escapes that; it stays in the React tree so it | ||
| * keeps inheriting the theme class's CSS variables. | ||
| * | ||
| * It duplicates the card in KevinSupport.tsx rather than sharing one, because | ||
| * the two differ in everything but the three buttons: that one is a turn in a | ||
| * conversation, this one is a floating callout with no room for an avatar or a | ||
| * lead-in. Both call the same `resolveApproval`, which is the part that matters. | ||
| * | ||
| * It draws nothing when an Electron BrowserView covers where it would land — | ||
| * plugin pages, the app store, Alertbox Library, a Browser editor element. Those | ||
| * are composited by the OS above the entire host page, so no z-index reaches over | ||
| * them and the bubble would render as the arrow alone. The fallback is the dot on | ||
| * the footer icon (StudioFooter.tsx), which is below every such view and always | ||
| * visible; clicking it opens the chat window, which is a real BrowserWindow and | ||
| * carries the same card. The alternative was blanking the plugin for as long as | ||
| * the approval sat unanswered, which is worse than one extra click. | ||
| */ | ||
| export default function KevinApprovalBubble({ anchorRef }: Props) { | ||
| const { KevinSupportService, WindowsService } = Services; | ||
|
|
||
| const { pendingApprovals, chatFocused } = useVuex(() => ({ | ||
| pendingApprovals: KevinSupportService.state.pendingApprovals, | ||
| // The state entry is deleted when the window closes, so `undefined` covers | ||
| // "closed" and `false` covers "open but behind something" in one read. | ||
| chatFocused: !!WindowsService.state['kevin-support']?.isFocused, | ||
| })); | ||
|
|
||
| // Nothing to add when the streamer is already looking at the chat — the card | ||
| // in there is the live surface, and two copies of one decision is worse than | ||
| // one in the wrong place. | ||
| const show = !chatFocused && pendingApprovals.length > 0; | ||
|
|
||
| const [anchor, setAnchor] = useState<DOMRect | null>(null); | ||
|
|
||
| useLayoutEffect(() => { | ||
| if (!show) return; | ||
| const measure = () => setAnchor(anchorRef.current?.getBoundingClientRect() ?? null); | ||
| measure(); | ||
| // ponytail: re-measured on resize only. The icon also shifts when the | ||
| // performance metrics beside it change width, or if the footer is scrolled | ||
| // horizontally — watch those with a ResizeObserver if it ever looks off. | ||
| window.addEventListener('resize', measure); | ||
| return () => window.removeEventListener('resize', measure); | ||
| }, [show, anchorRef]); | ||
|
|
||
| // This subscription is also what re-renders on navigation: moving to a plugin | ||
| // page fires no resize, so nothing else here would notice the page went native. | ||
| const [views, setViews] = useState<IViewRect[]>([]); | ||
|
|
||
| useEffect(() => { | ||
| const subscription = browserViewRects.subscribe(setViews); | ||
| return () => subscription.unsubscribe(); | ||
| }, []); | ||
|
|
||
| const occluded = | ||
| !!anchor && | ||
| isCoveredByBrowserView( | ||
| { | ||
| left: anchor.left, | ||
| top: anchor.top - ANCHOR_GAP - BUBBLE_MAX_HEIGHT, | ||
| width: BUBBLE_WIDTH, | ||
| height: BUBBLE_MAX_HEIGHT, | ||
| }, | ||
| views, | ||
| ); | ||
|
|
||
| if (!show || !anchor || occluded) return <></>; | ||
|
|
||
| return ( | ||
| // role="alert" rather than "alertdialog": the content is what should be | ||
| // announced, and an alertdialog would need a label naming a prompt whose | ||
| // whole text is already the summary below. | ||
| <div | ||
| className={styles.bubble} | ||
| role="alert" | ||
| style={{ left: anchor.left, bottom: window.innerHeight - anchor.top + ANCHOR_GAP }} | ||
| > | ||
| {pendingApprovals.map(approval => ( | ||
| <div key={approval.approvalId} className={styles.approval}> | ||
| <div className={styles.summary}>{approval.summary}</div> | ||
| {approval.risk === 'irreversible' && ( | ||
| <div className={styles.warning}>{$t('This cannot be undone.')}</div> | ||
| )} | ||
| {approval.risk === 'external' && ( | ||
| <div className={styles.warning}>{$t('This will be visible to your viewers.')}</div> | ||
| )} | ||
| <div className={styles.actions}> | ||
| <button | ||
| className="button button--action" | ||
| onClick={() => | ||
| KevinSupportService.actions.resolveApproval( | ||
| approval.approvalId, | ||
| 'approve', | ||
| 'footer', | ||
| ) | ||
| } | ||
| > | ||
| {$t('Allow once')} | ||
| </button> | ||
| <button | ||
| className="button button--default" | ||
| onClick={() => | ||
| KevinSupportService.actions.resolveApproval(approval.approvalId, 'always', 'footer') | ||
| } | ||
| > | ||
| {$t('Always allow')} | ||
| </button> | ||
| <button | ||
| className="button button--default" | ||
| onClick={() => | ||
| KevinSupportService.actions.resolveApproval(approval.approvalId, 'deny', 'footer') | ||
| } | ||
| > | ||
| {$t('Deny')} | ||
| </button> | ||
| </div> | ||
| </div> | ||
| ))} | ||
| </div> | ||
| ); | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.