The About page, a reader-first dataset route, and ltool follow-ons - #164
Merged
Conversation
`dashboards/index.jsx` is the static bundle's written landing page — the
bundler looks for it by name and compiles it in its own pass, deliberately
with no Malloy and no DuckDB. Lint knew nothing about that and applied the
orphaned-component rule to every .jsx it found:
✗ index.jsx
component "index.jsx" has no matching "index.malloy" dashboard
There is no fix an author can make. The file has no query BY DESIGN, so an
`index.malloy` beside it would mean inventing a dashboard nobody wants — and
publish gates on lint, so a repo that bundled perfectly could not be published
at all. malloyyo-babynames is one.
Nothing else disagreed: gatherDashboards and the server's GitHub ingestion both
walk .malloy files only, so index.jsx was never sent, never stored, and never
reached the server. The failure was entirely in the CLI, before the request.
Exempt it, but do not leave it unchecked — `bundle` compiles it, so a syntax
error should surface at lint time with the file named rather than later. An
index.malloy that DOES exist is unaffected: that is a dashboard named "index"
whose component is this file, and the existing pair lints normally.
Test fixtures leave eslint's scope in the same change. v2-landing-broken holds
an intentionally unparseable landing page so the parse error can be asserted;
linting fixtures as source turns a fixture's whole purpose into a build failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: lloyd tabb <lloyd@malloydata.org>
`dashboards/index.jsx` was a written introduction that only `bundle` could see.
`publish` never uploaded it, the GitHub refresh never fetched it, and the dev
server ignored it — so the page an author wrote to explain their data's scope
and origins was invisible on the two surfaces where a reader arrives cold.
Make it a real artifact, first in the list, everywhere:
- discover/gather find it (name "index", title "About", no query), so the dev
server serves it at `/` and publish uploads the component as the source.
- the GitHub refresh fetches it, so a pulled repo gets it too.
- it leads every listing. That ordering IS the front door: the switcher and
the home page both link to a dataset's FIRST dashboard, so putting About at
the head is what makes "switch datasets, land on the introduction" true —
no new route, and /datasets/<ref> stays the config page it already was.
A page with no query needed three things to work. Given introspection is
skipped rather than attempted, because compiling the model to look for a query
that does not exist reported its absence as a "model error" printed over the
author's prose. The static site's CSS variable names are aliased onto the frame
theme, because the same component is authored against the bundle's site.css and
rendered with invisible borders anywhere else. And `dashboards` is now a runtime
prop, so a page can link to its siblings on all three surfaces instead of only
where the bundler injected the list.
Those sibling links go through the navigate bridge. A relative href inside the
sandboxed frame navigates the FRAME to its own origin and 404s, and a top-level
href would need allow-top-navigation, which the sandbox deliberately withholds
— so the click is intercepted and the parent is handed a name, exactly as drill
already does. The listener installs once (mount() runs again on every
client-side navigation) and reads the sibling list per click, not at mount,
because the host's injection order is not guaranteed.
Also fixed, both found while building this: an artifact's name comes from its
`## artifact { name= }` tag, not its filename, so guarding the About page on
"is there an index.malloy" would let a tag that names another file "index"
publish two artifacts sharing one name — there is no unique (model_id, name),
and getDashboard's unordered `.limit(1)` would pick between them arbitrarily.
The guard is on resolved names, and the real dashboard wins. And the "view
source" link matched on basename alone, so a dashboard named "index" linked to
the repo-root index.malloy, a different file that happens to share the name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: lloyd tabb <lloyd@malloydata.org>
lloydtabb
changed the title
`/datasets/<ref>` was the config page — model version, files, GitHub settings.
That is the operator's view, and it was the first thing anyone following a
dataset link saw. Config moves to `/datasets/<ref>/config`, and the root becomes
a redirect to whatever the dataset actually offers:
1. its first dashboard — which is the About page when the repo ships one,
since that already leads the list;
2. AI Q&A, when questions have been asked;
3. ltool with the first source selected, when there is neither.
The third tier is the one worth stating: a dataset with no dashboards and no
questions is EMPTY, not broken, and the useful thing to hand someone then is a
query surface pointed at its data. ltool with no source is a blank picker — the
same dead end the config page was — so the source is what makes it land
somewhere.
The chain is a pure function so it can be tested without a database; the page
only does the I/O. It is a server component, so the redirect happens before
anything renders, and it stays DuckDB-free like every page.
Also add a "Query" item to the dashboard nav, beside AI Q&A — the same kind of
thing, a way into the data that isn't a dashboard someone built in advance. It
passes the dataset ID (the route param may be a name) and the first source, so
ltool opens on a `run: <source> ->` starter.
Three links moved with the route: the nav's config item, the home page's gear,
and the home page's "+N more" under a question list — that one now goes to the
Q&A page, which is what it always meant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: lloyd tabb <lloyd@malloydata.org>
Opening ltool on a dataset required a `source` to get an editable query. Without one the sidebar was empty, nothing was selected, and the editor rendered "Select a query from the sidebar" — with no query to select and no way to write one. That is exactly a freshly loaded dataset, and it is now reachable by design: the last tier of the /datasets/<ref> landing chain sends a dataset with no dashboards and no questions here, and a model that declares no sources arrives with no `source` at all. Seed the scratch query on a `dataset` deep link too, with or without a source. With one it keeps the `run: <source> ->` starter and expands the schema; without, it opens blank and titled "New query", schema collapsed because there is no source to show one for. Nothing is minted client-side: the scratch carries id/slug null, and /api/run mints the slug when the query is actually run — so the editor is writable immediately and the query becomes a real, shareable thing only once it has produced a result. Verified: a run with no baseSlug returns a fresh slug. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
lloydtabb
changed the title
On the home page the dataset name was plain text, so the only ways in were a dashboard chip, a question, or the config gear. It is now a link to `/datasets/<name>`, which is not a page but the landing redirect — About, else the first dashboard, else Q&A, else ltool on the first source. One link is correct for every dataset, and the home page needs to know nothing about which tier applies. The dataset switcher goes the same way. It had reimplemented the first two tiers client-side from /api/dashboards, which was already a second copy of the decision and had drifted the moment ltool became the third tier: a dataset with no dashboards and no questions landed on an empty Q&A page. Pointing it at /datasets/<name> deletes the duplicate, the /api/dashboards fetch it existed to feed, and the drift — for one redirect hop. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
Three things that all came down to the same gap: a source is only meaningful alongside the dataset that defines it. The source filter listed every dataset's sources as one flat run. It was in API order — dataset order — but with nothing marking where one dataset ended and the next began there was no way to see that, so it read as arbitrary. It also deduped on the bare source name, which silently dropped a source when two datasets both defined one called "orders" and left the survivor attributed to whichever the API listed first. Sources now carry their dataset, dedupe on dataset+source, and render grouped under the dataset name. Picking a source nobody has asked anything of filtered the sidebar down to nothing and left the editor on "Select a query from the sidebar" — the same dead end an empty dataset had. It now opens the new-query workflow, the same scratch a deep link gets, so the answer to "no questions here yet" is the thing you came to do rather than an empty list. The home page's source affordances become the same `<> Query` action as the dashboard nav's, sharing one icon rather than three hand-drawn copies. And ltool links carry the dataset NAME, not its uuid — `?dataset=babynames`. That needed the two web-query paths to resolve by ref rather than by id, which they should have done anyway: every other route takes either, and resolving only by id made a readable link fail in the ugliest way available, with Postgres refusing to cast the name to uuid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
Switching source opened a fresh query only when that source had no history. But picking a source is stating what you want to ask about, and the previous source's query left sitting in the editor answers a different question than the sidebar is now showing — so the "has questions" case was the one where the editor and the list disagreed. Now any source pick opens its starter. The filtered history is still right there to click when the answer already exists. "All sources" stays the exception: that clears the filter rather than choosing a subject, so it leaves the editor alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
With one group per dataset the list is long — 2600px in a 320px box on a server with eight datasets — and it opened at the top every time. Picking a sibling source in your own dataset, the common move, meant scrolling to find yourself first. It now opens centred on the current source, so its dataset's other sources are the ones on screen. Scrolls the list box only, never the page. That needed the picker to know WHICH source is current, which it could not work out alone: the filter value is a bare source name and names are not unique across datasets — two can each define "orders" — so it would have marked both and scrolled to whichever came first. It now takes the current dataset as well. That ref may be an id or a name and both genuinely occur — a replayed history item carries the recorded uuid, a deep link carries the readable name — so the match accepts either. Comparing against the id alone matched nothing at all once links started carrying names, which is how this turned up. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
/api/sources returned the dataset's name in a field called `model`. It has been
that way since the file's first commit, when a dataset and "a Malloy model" were
the same idea. They are not any more — `malloy_models` is a different table
holding the versioned model rows FOR a dataset — so the name pointed at the
wrong thing, and reading `s.model` to build a dataset link looked like a bug
every time. It is `dataset` now, and its four consumers with it.
A dataset id must not reach a URL or the screen. Two places still leaked one:
- ltool's drill built `/datasets/<id>/dashboard/<name>` from the selected
item's recorded dataset_id, so drilling out of a replayed history row put a
uuid in the address bar. It resolves the name through the source list first;
MalloyResultView takes a `datasetRef` and says in its type that an id is not
something to show anyone.
- SourceOption's `datasetId` is now documented as internal-only, beside the
`dataset` name that links are built from.
The GitHub webhook URL on the config page keeps its id, deliberately. That URL
is registered with GitHub and has to keep resolving; a name would tie a stored
external registration to a mutable field. It is an integration endpoint someone
copies once, not a link anyone navigates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: lloyd tabb <lloyd@malloydata.org>
lloydtabb
changed the title
/api/sources returned a flat list of sources with the dataset's five fields copied onto every row — 44 rows for eight datasets — and both callers began by regrouping it back into datasets. The endpoint returned the inverse of what anyone wanted, and that duplication was the only reason a dataset id had to ride along at all: it was the join key holding the flattened rows together. It is dataset-first now: each dataset once, with the sources it offers. 44 rows became 8, the home page's groupByDataset is gone, and the switcher no longer dedupes a list that no longer repeats. With the shape fixed, the id had nothing left to do, so it is gone from the wire entirely — along with the joins it existed for. A dataset is identified by NAME everywhere a browser can see: /api/sources, /api/dashboards, /api/favorited-queries, /api/history and the ltool share endpoint all say `dataset`, and the front page joins the three of them on it. Names are unique per server (findByDatasetRef), so nothing is lost. /api/run's body field follows: it takes `dataset`, a ref, which is what it has actually accepted since the web-query paths moved to findByDatasetRef. Calling the field datasetId while passing it a name was the misnomer one layer down. ownerEmail goes too. It was admin-gated, so this is not a leak being closed — it was simply carried through two types and the grouping and never rendered. Still by id, deliberately: the GitHub webhook URL on the config page. That is registered with GitHub and has to keep resolving, so it must not depend on a mutable field. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
**The About page's "view source" linked to the wrong file.** An earlier fix preferred a `dashboards/`-prefixed match but still fell back to the first match anywhere — and the About page exists precisely BECAUSE there is no `dashboards/index.malloy`, so it landed on the repo-root `index.malloy` every time: the model's MCP entry point, a different file that happens to share the basename. That root file is now excluded outright, so the link is correctly absent rather than wrong. Its real source is a .jsx and is not a model file at all. Pinned by a test, this being the second attempt. **Duplicate React keys in the schema source picker.** Dropping the dedupe-by- source-name (so two datasets can each declare "orders") left SchemaPanel's picker keyed on the bare name while ltool's was updated. It now keys on dataset+source like the other one. **A GitHub refresh could invent an About page the CLI never would.** The guard moved from "is there a dashboards/index.malloy" to "did anything resolve to the name index", and each misses what the other catches: a `dashboards/index.malloy` tagged `name="overview"` publishes as `overview`, so no row is called `index` and the old guard passed — inserting a second, queryless artifact from the same component, which then sorted first and became the dataset's front door. Both guards now apply, which is what the CLI has always effectively done. **The landing chain could redirect to an empty Q&A page.** `hasQuestions` accepted any history row carrying a question; the Q&A page lists only executed runs with no error and a slug. A failed MCP query records exactly such a row, so a dataset whose only activity was one would land on "no questions" — the dead end the chain exists to avoid — and never reach ltool. The predicate now mirrors what the page lists. **Two datasets could merge under one name.** `datasets_name_ready_unique` is partial on `status = 'ready'` while the catalogue lists everything not-failed, so a creation stuck in `modeling` can share a name with the live dataset. Now that the name is the key every caller joins and renders on, that merged a card, duplicated a React key, and hid one dataset. The catalogue keeps the ready one, and findByDatasetRef's name branch orders ready-first rather than taking an arbitrary row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
The previous commit added a ready-first `orderBy` to findByDatasetRef's name branch, to stop a stuck `modeling` namesake being resolved instead of the live dataset. That row can't reach the ordering: visibleDatasetWhere already restricts to `status = 'ready'`, and the unique index on `name` is partial on exactly that status, so at most one row matches. The comment asserted otherwise, which was the worse half of it. The collision is real on the catalogue endpoint, which builds its own WHERE (everything not-failed) — that dedupe stays. The comment here now says why the two differ. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
SourceOption's doc said the schema panel shows one dataset's sources and so had no use for the dataset name. It hasn't for a while: ltool hands the same cross-dataset array to its filter AND to the schema panel, which is exactly why both had to start keying rows on dataset+source. Comment-only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: lloyd tabb <lloyd@malloydata.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nine commits, one thread: the page an author writes to introduce their data should be the first thing a reader sees, and every route and label should agree with that.
1–2. The About page becomes a first-class dashboard
dashboards/index.jsxis the static bundle's written landing page (bundle.ts:465) — plain React, no Malloy, no query, by design. Two problems:It broke
publish.lintapplied the orphaned-component rule to every.jsx, and no fix was available to an author, so a repo that bundled perfectly could not be published. (The server was never involved —gatherDashboardsandgithub-refreshwalk.malloyfiles; the publish died at the CLI's own lint gate.) Exempted now, but still parsed with the sameesbuild.transformthe bundler uses.It existed on one surface out of three. Now discover/gather find it, the GitHub refresh fetches it, and it leads every listing.
Three things a queryless page needed: given introspection is skipped, not attempted (compiling to look for a query that doesn't exist printed a
model errorover the author's prose); the static site's CSS variables are aliased onto the frame theme (otherwise transparent cards, invisible borders); anddashboardsis now a runtime prop (only the bundler injected the sibling list).Sibling links route through the navigate bridge — a relative href inside the sandboxed frame navigates the frame to its own origin and 404s. The listener installs once (
mount()re-runs on every client-side navigation) and reads the list per click, since injection order differs between hosts.3–4.
/datasets/<ref>lands on the dataset, not its configConfig moved to
/datasets/<ref>/config; the root redirects down a chain:Tier 3 made a latent dead end reachable, so it's fixed too: ltool needed a
sourceto give you an editable query, and without one you got "Select a query from the sidebar" with no query to select. Adatasetdeep link now seeds a writable scratch either way. Nothing is minted client-side —/api/runmints on run.5–7. Getting into the data
<> Queryon the dashboard nav and on home-page sources, one shared icon.orderslost one. Now keyed on dataset+source.8–9. Naming and ids
/api/sourcesreturned the dataset's name in a field calledmodel— since the file's first commit, when a dataset and "a Malloy model" were the same idea. They aren't:malloy_modelsis a different table holding a dataset's versioned model rows. Renamed todataset, with its four consumers.Not renamed: MCP's
model_refis the same misnomer but on the wire (list_sourceskeys,describe_source/queryparams) — a breaking protocol change that needs its own deprecation. MCP'smodelparameter is unrelated and correctly named: it's the calling LLM, for run attribution.Dataset ids stay out of URLs. All ltool and dataset links carry the name; that required the two web-query paths to resolve by ref rather than by id, which every other route already did. ltool's drill resolved the recorded uuid to a name rather than putting it in the address bar.
Deliberate exception: the GitHub webhook URL keeps its id — it's registered with GitHub and has to keep resolving, so it must not depend on a mutable field.
Bugs found along the way
index— artifact names come from## artifact { name= }, not filenames, andmalloy_artifactshas no unique(model_id, name)whilegetDashboarduses an unordered.limit(1). Guard is on resolved names.indexto the repo-rootindex.malloy.Verification
bash scripts/preflight.sh— 11/11 green at every commit. 24 new cases acrossabout-page.test.ts,about.test.tsanddataset-landing.test.ts.Exercised on the real
malloyyo-babynamesrepo across bundle / dev / hosted, and the routing against six real datasets in a production fork:The ltool changes are UI and have no test harness in this repo; they were verified in the browser — grouped picker, centred open on
stadiums, new-query on every source switch, and a run posted with"datasetId":"babynames"returning a fresh slug.Not included
POST /api/datasets/<ref>/model/githubrequires a UUID and 500s on a dataset name — now the last route that insists on one. Pre-existing, hit while testing, out of scope.🤖 Generated with Claude Code