[Hackathon] feat: Dashboard Visualizer — Presentable Results Boards from Workflow Data#5116
Open
EmilySun621 wants to merge 16 commits into
Open
[Hackathon] feat: Dashboard Visualizer — Presentable Results Boards from Workflow Data#5116EmilySun621 wants to merge 16 commits into
EmilySun621 wants to merge 16 commits into
Conversation
This bundles the feature work that built up on this branch:
- Custom agents: dashboard CRUD page and editor dialog (48px icon tile,
chip-style guardrails, model selector). Each custom agent now carries a
LiteLLM model_name (Opus 4.7 / Haiku 4.5) that is passed through to the
agent-service so different agents can use different models.
- Conversation history is scoped per (workflowId, agentId): switching
agent or workflow yields a different conversation list. localStorage
key: texera.workflowConversations.v1.{workflowId}.{agentId}.
- Time machine: workflow snapshot list, revert, and agent-tagged
checkpoints. New workflow-history-tool in agent-service backs the
"undo my last change" flow; amber gains a WorkflowSnapshotResource;
sql/updates/23.sql adds the snapshot table.
- Operator-aware custom-agent prompts: the system prompt now injects the
full operator catalog with a "prefer built-in operators over Python
UDFs" rule, sourced from WorkflowSystemMetadata at request time.
- LiteLLM: added the claude-opus-4.7 entry alongside claude-haiku-4.5
and gpt-5-mini in bin/litellm-config.yaml.
- Agent panel rewritten around the (conversation list / chat) two-view
model with subscription-managed list reloads and per-step persistence.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a community gallery under /dashboard/hub/workflow-hub where users browse, star, fork, and publish data science workflows. Backed by 15 seed entries and localStorage for stars/forks/views/publishes so the page is never empty. - List page: search, sort (trending/stars/forks/recent), category chips, featured grid, DAG-chain preview cards, agent badges. - Detail page: SVG DAG preview, stats panel, fork-to-my-workflows (uses WorkflowPersistService.duplicateWorkflow when a backend wid is attached, otherwise falls back to a local stub), star toggle, and an optional 'Agent Included' card. - Publish dialog: pulls the user's workflows via the persist service, derives operator chain from workflow content, writes a hub entry to localStorage. - Sidebar: 'Workflow Hub' link added to the Hub submenu.
Seed entries don't have a workflowId, so the previous code only incremented a localStorage counter and navigated to /dashboard/user/workflow without actually writing to the backend — the forked workflow never showed up in the Workflows page. Now the seed path calls WorkflowPersistService.createWorkflow with empty content named "[Fork] <title>", waits for the backend to return the new wid, and routes straight into the new workflow's workspace. The duplicate-workflow path for real-wid entries is unchanged.
…board Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…llback
- New WorkflowDataService: lists user workflows, parses operators from
workflow content, fetches latest-execution runtime stats per operator
via existing /executions REST endpoints.
- Widgets now record a WidgetSource ({manual} or {workflow, wid, opId?,
metric, scope}). Workflow-sourced widgets refresh on dashboard load.
- Add Widget modal now has two paths:
From a Workflow — pick workflow + widget type + metric (and operator
for Metric Card); supports metric/bar/hbar/donut/table.
Manual Entry — unchanged form for all 6 widget types.
- Seed dashboard is now empty with a clear CTA instead of hardcoded demo
data.
Note: Texera does not expose operator tuple data via REST (results live
only in the WebSocket cache during a workspace session). We render the
data that IS persisted and REST-queryable: per-operator tuple counts,
sizes, processing times, worker counts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Switch to Texera's light theme: white card surfaces, #f5f6f8 page bg,
light-mode chart colors, #1677ff accent matching the rest of the app.
Removed every dark-theme rule.
- Add Widget modal redesigned for multi-select:
- Checkbox gallery of all 6 widget types with inline config forms.
- Quick metric presets (Accuracy / F1 / Precision / Recall) pre-fill
the Metric Card.
- "Add N widgets" submits everything checked at once; modal stays
open with a confirmation badge so users can compose another batch.
- "Done" closes the modal.
- Dropped the operator runtime-stats path. Texera does not expose tuple
data via REST, and runtime stats (tuple counts, sizes, latencies)
aren't what researchers want on a results dashboard. Deleted the
workflow-data service and the WidgetSource discriminator.
- Removed seed data entirely. List page now shows a real empty state
with a Create Dashboard CTA. Service no longer auto-creates anything;
legacy seed-* dashboards from earlier iterations get purged on load.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add Widget is now a tabbed modal:
- Tab 1 "From Workflow" — pick a workflow that has been executed,
see what's available, pick a data point, pick a widget type, add.
Two data sources:
* Saved results from localStorage at texera.workflow.results.{wid}
(WorkflowResultsSnapshot shape) — picks up evaluation metrics
(accuracy, F1, …) and tabular outputs whenever the workspace
writes them. Allows Metric Card / Bar / Donut / Table widgets.
* Runtime stats from REST — output rows, input rows, output size,
processing time per operator. Always available after a run.
Renders as Metric Card.
- Tab 2 "Manual Input" — pick widget type, fill in data, add.
Five widget types: Metric / Bar / Donut / Table / Text.
Widgets added from a workflow carry a WidgetSource and display a
"From <workflow>" pill in the top-right of the widget frame.
Modal stays open after each add with a confirmation badge so users
can compose several widgets in one session; Done closes it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Result Panel already receives operator output via WebSocket during
execution. We tap the same stream — no new endpoint, no duplicate fetch.
- New DashboardResultCacheService subscribes to
WorkflowResultService.getResultUpdateStream() at app load. Each
update is snapshotted to localStorage:
key : texera.results.{wid}.{opId}
value : { columns: string[], rows: any[][], timestamp: string }
Caps at 100 rows per operator; LRU-evicts on QuotaExceededError. The
current wid is tracked from /dashboard/user/workflow/{wid} URLs.
Paginated operators are fetched via OperatorPaginationResultService
.selectPage(1, 100) (debounced per op) since pagination updates only
carry metadata, not rows.
- workflow-data.service.ts now reads per-operator keys and merges them
with any legacy texera.workflow.results.{wid} bundle. Auto-detects
metric shapes:
* Single-row table → every numeric column becomes a metric
(catches evaluation outputs like {accuracy, f1, precision}).
* 2-column ≤20-row name|value table → each row becomes a metric
(catches aggregations like {model: "RF", score: 0.94}).
Rows are always exposed for Table / Bar / Donut widgets.
- AppComponent injects the cache service so it instantiates at boot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… Results
Two bugs in the Add Widget modal:
1. The workflow dropdown was sized by nz-select's default min-width
(~120px), truncating names to "[Example] Da...". Made it stretch to
the full modal width and ellipsize at the actual edge.
2. Data cards were single-select via a separate "Widget type" step,
which felt unresponsive — users had to click twice to enable Add.
Switched to multi-select with toggle-on-click:
- Each card has a checkmark in the top-right when picked.
- "Add N Widget(s)" creates one widget per selected card in one go;
metric/stat cards become Metric Cards, row cards become Tables.
- Modal stays open afterward with confirmation; selection clears so
users can compose another batch.
Also restructured the sections:
- Results — actual run data — gets a primary blue-tinted card with a
prominent header at the top.
- Runtime stats moved into a collapsed section (a count pill + caret).
Expand only when needed; most users don't care about input row
counts and processing time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The empty-grid had `span[nz-icon] { font-size: 56px }` as a descendant
selector, which cascaded onto the button's "+" icon too — that's what
made it render at 56px on its own line above the "Add Widget" label.
Scoped that rule to a new `.empty-grid-icon` class on the dashboard
icon only, and gave `.empty-cta` explicit inline-flex so the icon and
text render side-by-side.
Also gated the CTA button (and its "click + Add Widget" prompt) behind
mode === 'edit'. In view mode the empty state shows just an icon and
"No widgets yet." — no button to click since adding widgets requires
edit mode.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the Result Panel receives output from a visualization operator (Scatterplot, ConfusionMatrix, etc.), the row contains an html-content column with a self-contained Plotly HTML document. We were storing it in localStorage but then showing the raw HTML source in a Table widget. - New "html" widget type. Renders via <iframe [srcdoc]> with sandbox="allow-scripts" + bypassSecurityTrustHtml so inline Plotly scripts actually execute. Mirrors what VisualizationFrameContentComponent does in the workspace. - The Add Widget modal now scans cached operator outputs for cells that look like an HTML document (starts with <!doctype/<html/<script/ <svg, or contains script/plotly markers) and surfaces each as its own HTML data point alongside the regular metrics/rows. Default widget type for those points is "html" with a chart-area icon. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause: safeHtmlContent was a getter that ran sanitizer.bypassSecurityTrustHtml(...) on every change-detection cycle, returning a fresh SafeHtml each time. Angular saw a "new" srcdoc value and rebound it; the browser reloaded the iframe; Plotly redrew. Hence the flicker. - Cache the SafeHtml in a private field via an @input setter. We only recompute when the underlying htmlContent string actually changes. - ChangeDetectionStrategy.OnPush on the widget component so CD only fires when its input reference actually changes. - trackBy by widget id on the editor's widget *ngFor so a fresh widgets array reference doesn't destroy + recreate every iframe. - min-height 240px on .html-iframe so a still-loading chart doesn't shift layout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous attempt (cached SafeHtml + OnPush + trackBy) wasn't enough — the user still saw the Plotly chart appear/disappear/reappear. Switched to an imperative approach: - Drop [srcdoc] binding entirely. Get the iframe via @ViewChild and set element.srcdoc = htmlContent imperatively. Track the currently- applied content; only re-set when the html string actually changes. Angular's change detection now has no way to retrigger a srcdoc attribute write, no matter how many times the parent's array reference flips. - Memoize widgetStyle() in the editor — keyed on widget id + layout signature — so [ngStyle] receives a referentially-stable object when nothing has moved. Prevents Angular from re-applying inline styles on every CD cycle, which in turn keeps the widget-cell from retriggering CSS transitions that could resize the iframe and force Plotly to redraw. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the 12-column grid in favor of pixel-perfect x/y/width/height
layouts. Several problems with the grid:
- Newly-added widgets, dragged in grid units with a re-assigned mouse
origin on every snap, gave a "glued to the cursor" feel — the widget
appeared to chase the mouse after Add.
- Snapping made the resize handle feel sluggish.
- CSS transitions on the cell's width/height meant a resize redrew
Plotly continuously.
New layout:
- WidgetLayout is { x, y, width, height } in pixels.
- nextLayout() places new widgets at x=16, y=(max bottom + 16). They
always land below the existing stack — never under the cursor.
- Drag captures (mouseStart, widgetStart) on mousedown and applies
newX = widgetStart + (mouseNow - mouseStart). Standard pattern; no
origin re-assignment, no jitter.
- Resize handle: same pattern on bottom-right; clamped to MIN 160x120.
- CSS transitions on the cell's geometry removed.
- Edit-mode-only drag/resize via mode check at the top of startMove
/ startResize.
- Persists on every layout update through localStorage.
- Legacy {x,y,w,h} dashboards migrate to pixels on load.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Resize handle is now an always-visible diagonal grip in the bottom-right corner of every widget when in Edit mode. Drawn with two CSS pseudo-element stripes (no icon font). Hover and active drag turn it blue. - Blue selection border on the widget frame no longer appears on hover; it only shows on .widget-cell.dragging, which is toggled via isWidgetActive(w.id) when the user is actively moving or resizing the widget. - iframe in the HTML widget drops min-height: 240px and gets min-height: 0 plus height: 100%, so it shrinks freely with the widget. The flex column on .widget already handles the title + iframe split, so the iframe fills whatever vertical space the user resizes to. - Min widget size raised to 200x150 per spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…L rendering Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
engine
ddl-change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
😤 The Problem
You just ran a 15-operator ML pipeline. Your PI asks: "What did you find?"
Today's answer: open each operator's result panel one by one, take screenshots, paste into PowerPoint, lose all interactivity. 30 minutes of manual work for a 5-minute meeting.
✨ The Solution
One dashboard. Real data. Zero screenshots.
🚀 What We Built
📋 Dashboard Gallery
Create, manage, and organize multiple dashboards — one per project, one per experiment, one per meeting.
➕ Add Widget — Two Paths
🔗 Path 1: From Workflow (the magic path)
✏️ Path 2: Manual Input
Quick presets for common metrics:
Click → enter number → done. Also supports custom bar charts, donut charts, tables, and text notes.
🧩 7 Widget Types