Perceived-latency P1: ETag/304, deferred tab fetches, lazy widgets#28014
Open
harshach wants to merge 7 commits into
Open
Perceived-latency P1: ETag/304, deferred tab fetches, lazy widgets#28014harshach wants to merge 7 commits into
harshach wants to merge 7 commits into
Conversation
Server: ETagResponseFilter already emits ETag on entity GETs. Extended it to also (a) emit Cache-Control: must-revalidate, private and (b) short-circuit to 304 Not Modified with empty body when the request's If-None-Match matches the computed ETag (RFC 7232 weak comparison). Client: new attachEtagInterceptor() in rest/etagInterceptor.ts holds an LRU cache of (etag, response body) keyed by the canonical URL+params. On request, sends If-None-Match if a cached etag exists. On 304, returns the cached body as if it were a 200 — caller sees a normal Axios success with no awareness that no bytes crossed the wire. Cap of 200 entries keeps the cache bounded to ~10 MB worst case. clearEtagCache() is wired into AuthProvider.onLogoutHandler so a freshly-authenticated user can't pick up another principal's cached body via 304. Wins on revisits: zero body bytes on the wire, skip JSON parse, skip render. Server still computes the body (we'd need a cheap version-stamp lookup to truly skip the work — design doc tracks it as a follow-up). P1.1 of .context/perceived-latency-design.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Most users never click into the Queries tab on a Table detail page, but TableDetailsPageV1 was firing getQueriesList() unconditionally on every page load to populate the "Queries (N)" badge. That's one wasted server round-trip per Table view for the ~95% of users who skip the tab. New useDeferredTabData hook gates a fetch on first tab activation — the badge populates when the user actually clicks into Queries, and re-arms when they navigate to a different Table FQN. Kept getTestCaseFailureCount eager: it drives the global red-alert badge in the page chrome, so deferring would mean a freshly-landed user could miss a critical "this dataset has failing tests" indicator. P1.2 of .context/perceived-latency-design.md. The "parallelize the serial chain" half of the original P1.2 was a no-op on inspection — the existing useEffects already run in parallel within their dep tracks; the chain is forced by data dependency (queryCount and DQ counts both need tableDetails.id), not by ordering choice. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MyDataPage's grid renders every widget eagerly, and each widget runs its own data-fetch effect on mount — so eagerly mounting all of them on first paint pays for multiple below-fold network requests the user may never scroll to. New DeferredWidget wraps each grid item. It uses react-intersection-observer (already a dep) to defer rendering the child tree until the wrapper enters the viewport, with a 200px root-margin look-ahead so a normal scroll never reveals a placeholder. Once revealed, the widget stays mounted — no remount on scroll-out. Users with very tall screens see no behavior change (every widget is already in view on initial paint, so all mount immediately). Users on typical viewports save 2-4 widget worth of network and JS-render cost on the critical path. P1.3 of .context/perceived-latency-design.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
backend
safe to test
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements three “perceived latency” optimizations across the Java service and React UI: conditional GETs via ETag/304 with a client-side cache, deferring rarely-used tab data fetches on Table details, and deferring below-fold widget mounting on My Data.
Changes:
- Backend: add
Cache-Controland return304 Not Modifiedon matchingIf-None-Matchfor entity GETs. - UI REST client: add an Axios interceptor that sends
If-None-Matchand serves cached bodies on 304. - UI rendering/fetching: defer Queries-tab count fetch until tab activation; lazy-mount MyData widgets via
IntersectionObserver.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| openmetadata-ui/src/main/resources/ui/src/rest/index.ts | Attaches the ETag/304 Axios interceptor to the shared REST client. |
| openmetadata-ui/src/main/resources/ui/src/rest/etagInterceptor.ts | Implements LRU caching of (etag, body) and 304→200 response translation. |
| openmetadata-ui/src/main/resources/ui/src/pages/TableDetailsPageV1/TableDetailsPageV1.tsx | Stops eager query-count fetch and wires in deferred Queries-tab fetching. |
| openmetadata-ui/src/main/resources/ui/src/hooks/useDeferredTabData.ts | New hook to fire a fetcher on first tab activation with reset semantics. |
| openmetadata-ui/src/main/resources/ui/src/pages/MyDataPage/MyDataPage.component.tsx | Wraps widgets in a new DeferredWidget to avoid below-fold mounting on initial paint. |
| openmetadata-ui/src/main/resources/ui/src/components/common/DeferredWidget/DeferredWidget.component.tsx | New viewport-gated wrapper using react-intersection-observer. |
| openmetadata-ui/src/main/resources/ui/src/components/Auth/AuthProviders/AuthProvider.tsx | Clears the ETag client cache on logout to avoid cross-principal reuse. |
| openmetadata-service/src/main/java/org/openmetadata/service/resources/filters/ETagResponseFilter.java | Adds Cache-Control and 304 short-circuit behavior for entity GET responses. |
Comment on lines
+66
to
+79
| const [hasBeenVisible, setHasBeenVisible] = useState(false); | ||
|
|
||
| const { ref, inView } = useInView({ | ||
| rootMargin, | ||
| threshold, | ||
| // Fire only the first crossing into view — once revealed, the widget mounts and the | ||
| // observer detaches. Re-scrolling above and back doesn't re-trigger because the child | ||
| // tree stays mounted (we drive that via {@link hasBeenVisible}). | ||
| triggerOnce: true, | ||
| }); | ||
|
|
||
| if (inView && !hasBeenVisible) { | ||
| setHasBeenVisible(true); | ||
| } |
Comment on lines
+94
to
+109
| /** | ||
| * Wire ETag handling into the axios client. Idempotent — calling twice is harmless because | ||
| * each call uses a fresh interceptor pair (callers that re-init axios should clear the cache | ||
| * via {@link clearEtagCache}). | ||
| */ | ||
| export function attachEtagInterceptor(client: AxiosInstance): void { | ||
| // Treat 304 as a success status so axios delivers it through the response interceptor | ||
| // instead of the error path. Without this, our 304-handling code would have to live in | ||
| // the error interceptor and intercepts on every error path. | ||
| const previousValidate = client.defaults.validateStatus; | ||
| client.defaults.validateStatus = (status: number) => | ||
| status === 304 || | ||
| (previousValidate | ||
| ? previousValidate(status) | ||
| : status >= 200 && status < 300); | ||
|
|
Comment on lines
+59
to
+61
| // re-fire the fetch. We deliberately depend only on the tab id so we fire exactly once | ||
| // per activation window. | ||
| }, [activeTab, tabKey]); |
Comment on lines
+37
to
+61
| export function useDeferredTabData( | ||
| tabKey: string, | ||
| activeTab: string | undefined, | ||
| fetcher: () => void | Promise<void>, | ||
| resetDeps: ReadonlyArray<unknown> = [] | ||
| ): void { | ||
| const fetchedRef = useRef(false); | ||
|
|
||
| // Reset the once-flag when any reset dep changes — typically when the user navigates to | ||
| // a different entity, even if the tab id is the same. The empty-deps default never | ||
| // re-arms; useful for ambient hooks that genuinely fire once. | ||
| useEffect(() => { | ||
| fetchedRef.current = false; | ||
| }, resetDeps); | ||
|
|
||
| useEffect(() => { | ||
| if (activeTab !== tabKey || fetchedRef.current) { | ||
| return; | ||
| } | ||
| fetchedRef.current = true; | ||
| void fetcher(); | ||
| // The fetcher closure changes on every render in most callers — depending on it would | ||
| // re-fire the fetch. We deliberately depend only on the tab id so we fire exactly once | ||
| // per activation window. | ||
| }, [activeTab, tabKey]); |
Comment on lines
+789
to
+804
| // P1.2: getTestCaseFailureCount drives the global red-alert badge in the page chrome, | ||
| // so it must run as soon as tableDetails resolves — deferring would mean the user could | ||
| // miss a critical "this dataset has failing tests" indicator on first paint. | ||
| useEffect(() => { | ||
| if (tableDetails) { | ||
| fetchQueryCount(); | ||
| getTestCaseFailureCount(); | ||
| } | ||
| }, [tableDetails?.fullyQualifiedName]); | ||
|
|
||
| // P1.2: queryCount only drives the "Queries (N)" tab badge — most users never click that | ||
| // tab, so eagerly fetching it on every page load wasted a server round-trip per view. | ||
| // Defer until the user actually activates the Queries tab (or any of its column-scoped | ||
| // sub-tabs); the badge then populates on first activation. | ||
| useDeferredTabData(EntityTabs.TABLE_QUERIES, activeTab, fetchQueryCount, [ | ||
| tableDetails?.fullyQualifiedName, | ||
| ]); |
Comment on lines
+68
to
+85
| const { ref, inView } = useInView({ | ||
| rootMargin, | ||
| threshold, | ||
| // Fire only the first crossing into view — once revealed, the widget mounts and the | ||
| // observer detaches. Re-scrolling above and back doesn't re-trigger because the child | ||
| // tree stays mounted (we drive that via {@link hasBeenVisible}). | ||
| triggerOnce: true, | ||
| }); | ||
|
|
||
| if (inView && !hasBeenVisible) { | ||
| setHasBeenVisible(true); | ||
| } | ||
|
|
||
| return ( | ||
| <div className={className} ref={ref}> | ||
| {hasBeenVisible ? children : placeholder ?? null} | ||
| </div> | ||
| ); |
Contributor
🔴 Playwright Results — 158 failure(s), 17 flaky✅ 3910 passed · ❌ 158 failed · 🟡 17 flaky · ⏭️ 86 skipped
Genuine Failures (failed on all attempts)❌
|
Six review-driven fixes spanning all three P1 commits:
DeferredWidget (lazy widget loader):
- Move state update out of render — was calling `setHasBeenVisible(true)`
in the component body guarded by `!hasBeenVisible`, which is a React
anti-pattern (works, but trips warnings and triggers extra render
cycles). Now drives state via `useInView`'s `onChange` callback so
the update only fires once on the IntersectionObserver event.
- Add `fallbackInView: true` so children render eagerly in
environments where IntersectionObserver is unavailable
(SSR, very old browsers, some Jest setups).
- Add `initialInView` opt-out prop so callers (e.g. test wrappers,
above-the-fold widgets) can skip the observer entirely.
etagInterceptor (304 client-side cache):
- Make `attachEtagInterceptor` properly idempotent. The previous "called
twice is harmless" claim was wrong — each call stacked another
interceptor pair and re-wrapped `validateStatus`. Now guarded by a
symbol marker on the AxiosInstance so re-invocation
(HMR, test bootstrap re-runs) is a true no-op.
- Deep-clone cached body on read (304 hit path) via `structuredClone`.
The cache stored a shared reference; consumers that mutated the entity
(edit handlers, UI-local state mixing) would leak those mutations
back into the cache and the next 304 would serve the mutated copy.
useDeferredTabData (per-tab gated fetch):
- When `resetDeps` change while the gated tab is already the active
tab, fire the fetcher immediately rather than waiting for a tab
toggle the user has no reason to make. Without this, navigating from
table A → table B while staying on Queries left the badge showing
A's count until the user clicked elsewhere and back.
- Latest-fetcher captured via a ref so the reset effect always calls
the up-to-date closure without re-firing on every render.
TableDetailsPageV1:
- Reset `queryCount` to 0 when `tableDetails?.fullyQualifiedName`
changes. The badge previously kept showing the previous table's
count until the deferred fetch (which also re-arms via the fix
above) resolved for the new entity.
Tested locally: `yarn build` green, eslint + prettier clean on all
four touched files, tsc clean (the one pre-existing tsc error at
TableDetailsPageV1:746 is unrelated — `findColumnByEntityLink` typing
on a `string | undefined`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P1.3 wrapped landing-page widgets in DeferredWidget so below-fold widgets
mount only after entering the viewport. Existing tests assumed eager mount
and broke:
Jest (MyDataPage.test.tsx): jsdom's IntersectionObserver mock is a no-op
jest.fn() that never fires, so `useInView` keeps `inView` false forever
and children never render. The 4 `findByText('KnowledgePanel.*')`
assertions in the test couldn't find widgets the wrapper was holding
back. Fix: mock `DeferredWidget` to render children directly — same
pattern as the existing LimitWrapper / DataInsightProvider mocks.
Playwright (CustomizeLandingPage.spec.ts): real browser, real IO — but
the 1280x720 CI viewport leaves widgets like `KnowledgePanel.KPI` below
the fold. Without a scroll, those widgets are never observed, never
mount, and `toBeVisible()` resolves to false. Fix: call
`scrollIntoViewIfNeeded()` before each `toBeVisible` assertion in
`checkAllDefaultWidgets`, plus the two inline assertions in
"Add, Remove and Reset widget" and "Widget drag and drop reordering".
Existing `not.toBeVisible()` checks stay as-is — a placeholder div without
children is correctly "not visible", so the deferred-mount behaviour
matches the assertion intent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment on lines
+130
to
+178
| const entry = etagCache.get(buildKey(config)); | ||
| if (!entry) { | ||
| return config; | ||
| } | ||
|
|
||
| // Axios 1.x always populates config.headers with AxiosHeaders instance, but be | ||
| // defensive in case an upstream interceptor swapped it for a plain object. | ||
| if (config.headers instanceof AxiosHeaders) { | ||
| config.headers.set('If-None-Match', entry.etag); | ||
| } else if (config.headers) { | ||
| (config.headers as Record<string, string>)['If-None-Match'] = entry.etag; | ||
| } else { | ||
| config.headers = new AxiosHeaders({ 'If-None-Match': entry.etag }); | ||
| } | ||
|
|
||
| return config; | ||
| }); | ||
|
|
||
| client.interceptors.response.use((response) => { | ||
| const method = (response.config.method ?? 'get').toLowerCase(); | ||
| if (method !== 'get') { | ||
| return response; | ||
| } | ||
|
|
||
| const key = buildKey(response.config); | ||
|
|
||
| if (response.status === 304) { | ||
| const entry = etagCache.get(key); | ||
| if (entry) { | ||
| touch(key, entry); | ||
|
|
||
| // Deep-clone the cached body before handing it back. Consumers (UI components, | ||
| // utilities, edit handlers) routinely mutate the entity object they receive — adding | ||
| // local UI state, normalising fields, stripping properties — and a shared reference | ||
| // would let those mutations leak back into the cache. The next 304 would then return | ||
| // the mutated copy and cross-page bugs become very hard to track. structuredClone is | ||
| // available in all supported browsers (Chrome 98+, Firefox 94+, Safari 15.4+). | ||
| return { | ||
| ...response, | ||
| status: 200, | ||
| statusText: 'OK (from ETag cache)', | ||
| data: structuredClone(entry.data), | ||
| }; | ||
| } | ||
|
|
||
| // 304 without a cached body shouldn't happen in normal flow — a stale interceptor | ||
| // attaching If-None-Match for a key we no longer hold. Bubble through; the caller | ||
| // sees 304 and decides. Better than fabricating a fake 200. | ||
| return response; |
Comment on lines
+107
to
+123
| export function attachEtagInterceptor(client: AxiosInstance): void { | ||
| const marker = client as unknown as Record<symbol, boolean>; | ||
| if (marker[ETAG_INTERCEPTOR_INSTALLED]) { | ||
| return; | ||
| } | ||
| marker[ETAG_INTERCEPTOR_INSTALLED] = true; | ||
|
|
||
| // Treat 304 as a success status so axios delivers it through the response interceptor | ||
| // instead of the error path. Without this, our 304-handling code would have to live in | ||
| // the error interceptor and intercepts on every error path. | ||
| const previousValidate = client.defaults.validateStatus; | ||
| client.defaults.validateStatus = (status: number) => | ||
| status === 304 || | ||
| (previousValidate | ||
| ? previousValidate(status) | ||
| : status >= 200 && status < 300); | ||
|
|
Comment on lines
+95
to
+100
| // When IntersectionObserver is unavailable (SSR, very old browsers, some test | ||
| // environments) treat the wrapper as "in view" so children render eagerly rather than | ||
| // staying invisible forever. The repo's Jest setup installs a no-op IO mock that never | ||
| // fires — combined with `initialInView` above, tests get sane defaults. | ||
| fallbackInView: true, | ||
| onChange: handleChange, |
Comment on lines
+35
to
+60
| * <p>{@code Cache-Control: must-revalidate, private}: clients (browsers, our Axios interceptor) | ||
| * may keep the body but must revalidate via {@code If-None-Match} before reusing it; private | ||
| * keeps it out of any shared/proxy cache so per-user data doesn't leak. | ||
| */ | ||
| @Provider | ||
| public class ETagResponseFilter implements ContainerResponseFilter { | ||
|
|
||
| private static final String CACHE_CONTROL_VALUE = "must-revalidate, private"; | ||
|
|
||
| @Override | ||
| public void filter( | ||
| ContainerRequestContext requestContext, ContainerResponseContext responseContext) { | ||
| try (var ignored = RequestLatencyContext.phase("etagGeneration")) { | ||
| if ("GET".equals(requestContext.getMethod()) | ||
| && responseContext.getStatus() == Response.Status.OK.getStatusCode() | ||
| && responseContext.getEntity() instanceof EntityInterface entity) { | ||
| if (!"GET".equals(requestContext.getMethod()) | ||
| || responseContext.getStatus() != Response.Status.OK.getStatusCode() | ||
| || !(responseContext.getEntity() instanceof EntityInterface entity)) { | ||
| return; | ||
| } | ||
|
|
||
| String etag = EntityETag.generateETag(entity); | ||
| if (etag == null) { | ||
| return; | ||
| } | ||
| responseContext.getHeaders().putSingle(HttpHeaders.ETAG, etag); | ||
| responseContext.getHeaders().putSingle(HttpHeaders.CACHE_CONTROL, CACHE_CONTROL_VALUE); | ||
|
|
`FailedTestCaseSampleData.test.tsx` (and any other test that mounts a
core-components component) fails with:
TypeError: Cannot read properties of null (reading 'useContext')
at openmetadata-ui-core-components/.../node_modules/react/cjs/react.development.js:1618
at ... openmetadata-ui-core-components/.../dist/SnackbarContent.cjs.js
The error is the classic dual-React-instance — the second React (with a
null hooks dispatcher) lives at
`openmetadata-ui-core-components/src/main/resources/ui/node_modules/react/`,
which the core-components package installs for its own dev/test. When the
consumer (`openmetadata-ui`) loads the CJS bundle from `dist/*.cjs.js`,
Node's `require('react')` resolution walks up from the bundle file and
finds the core-components-local React first, not the consumer's copy.
Add explicit `moduleNameMapper` entries so every `require('react')` and
`require('react-dom')` (and submodules) resolves to the consumer's
`<rootDir>/node_modules/react[-dom]`. Single React instance, shared hooks
dispatcher, no more null-useContext crash.
Verified: failing test now passes, and a sampling of 22 unrelated suites
(304 tests across Loader, Lineage, IncidentManager, AlertsListing, etc.)
all stay green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Jest test CoverageUI tests summary
|
DeferredWidget kept its children inside an unkeyed wrapper with no
data-testid and no min-height. On the home page, below-fold widgets
(KnowledgePanel.KPI etc.) never entered the viewport in a 1280x720
Playwright browser, so:
- The wrapper div with `ref` was 0-height and never crossed the
IntersectionObserver threshold.
- `page.getByTestId('KnowledgePanel.KPI')` matched nothing because
the testid lives on the *child* widget that hadn't mounted yet.
- `scrollIntoViewIfNeeded()` from my previous fix attempt waited
forever for that non-existent element — hanging every shard for
27 minutes before the test timeout fired.
All 6 Playwright shards were failing as a result. The earlier
scroll-based mitigation was strictly worse than the original code: it
turned a 60s timeout into a 27-minute hang.
Reverting the DeferredWidget integration on MyDataPage. The component
itself, its only consumer, the Jest mock, and the playwright workarounds
are all removed. P1.1 (ETag/304) and P1.2 (defer Queries-tab) stand —
those are clean wins. Lazy widget mounting can come back later with
proper testid-on-wrapper + min-height semantics so tests can find the
slot before its contents mount.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code Review ✅ Approved 4 resolved / 4 findingsImplements ETag-based 304 caching, deferred tab fetches, and lazy-loading widgets to optimize perceived latency. Resolved issues include stale ETag cache references, state updates during render, and improper widget lazy-loading logic. ✅ 4 resolved✅ Bug: setState called during render in DeferredWidget
✅ Edge Case: ETag cache stores response data references, risks stale mutations
✅ Edge Case: useDeferredTabData eslint-disable needed for intentional dep omission
✅ Performance: DeferredWidget wraps ALL widgets including above-fold ones
OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
Comment on lines
+181
to
+185
| if (response.status === 200) { | ||
| const etag = readEtagHeader(response); | ||
| if (etag && response.data !== undefined) { | ||
| touch(key, { etag, data: response.data }); | ||
| } |
Comment on lines
+42
to
+60
| private static final String CACHE_CONTROL_VALUE = "must-revalidate, private"; | ||
|
|
||
| @Override | ||
| public void filter( | ||
| ContainerRequestContext requestContext, ContainerResponseContext responseContext) { | ||
| try (var ignored = RequestLatencyContext.phase("etagGeneration")) { | ||
| if ("GET".equals(requestContext.getMethod()) | ||
| && responseContext.getStatus() == Response.Status.OK.getStatusCode() | ||
| && responseContext.getEntity() instanceof EntityInterface entity) { | ||
| if (!"GET".equals(requestContext.getMethod()) | ||
| || responseContext.getStatus() != Response.Status.OK.getStatusCode() | ||
| || !(responseContext.getEntity() instanceof EntityInterface entity)) { | ||
| return; | ||
| } | ||
|
|
||
| String etag = EntityETag.generateETag(entity); | ||
| if (etag == null) { | ||
| return; | ||
| } | ||
| responseContext.getHeaders().putSingle(HttpHeaders.ETAG, etag); | ||
| responseContext.getHeaders().putSingle(HttpHeaders.CACHE_CONTROL, CACHE_CONTROL_VALUE); | ||
|
|
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes:
I shipped the three Phase-1 perceived-latency wins from
.context/perceived-latency-design.mdbecause the cache-perf work showed the cache was masking a too-slow uncached floor — these changes lower that floor on the most-trafficked pages so latency is acceptable whetherCACHE_PROVIDER=redisorCACHE_PROVIDER=none. Server emitsETag+Cache-Controlon entity GETs and short-circuits to304 Not ModifiedonIf-None-Match, paired with a client Axios interceptor that LRU-caches(etag, body)per URL so revisits skip the body bytes + JSON parse + render. NewuseDeferredTabDatahook gates the Queries-tab fetch onTableDetailsPageV1until the tab is activated (95%+ of users never click it, eliminating one wasted round-trip per page view). New<DeferredWidget>wrapsMyDataPage's grid items so below-fold widgets defer their data-fetch effects until scrolled into view, usingreact-intersection-observerwith a 200 px look-ahead.Type of change:
High-level design:
Cache-independence — every change must improve performance with
CACHE_PROVIDER=none, not justredis. The cache is a multiplier, not the floor.P1.1 — ETag / If-None-Match (
ETagResponseFilter.java,etagInterceptor.ts)Server already emitted
ETagon entity GETs (EntityETag.generateETagfromversion+updatedAt). Extended the existingETagResponseFilterto:Cache-Control: must-revalidate, privateGETwithIf-None-Matchmatching the computed ETag (RFC 7232 weak comparison): override status to304, drop the body. Headers preserved.Client-side, a new
attachEtagInterceptoris wired inrest/index.ts(beforeAuthProvider's interceptors so it sits closest to the wire):If-None-Matchon GETs that have a cached ETag for the URL+params304translates to a synthetic200with the cached body — callers see a normal Axios successclearEtagCache()is called fromAuthProvider.onLogoutHandlerso a freshly-authenticated user can't pick up another principal's body via304Wins are network bytes + client render, not server CPU (server still computes the body — a cheap version-stamp lookup that skips entity hydration is design-doc-tracked as P3).
P1.2 — Defer Queries-tab fetch (
useDeferredTabData.ts,TableDetailsPageV1.tsx)fetchQueryCountdrives the "Queries (N)" tab badge. Most users never click that tab. The hook fires the fetcher on first tab activation and re-arms when the entity FQN changes (so navigating across Tables doesn't reuse stale counts).getTestCaseFailureCountis intentionally kept eager because it drives the global red-alert badge in the page chrome — deferring it would risk users missing a critical "this dataset has failing tests" indicator on first paint.Original P1.2 also called for "parallelize the serial chain"; on inspection the existing
useEffects already run in parallel within their dependency tracks — the chain is forced by data dependency ontableDetails.id, not ordering. Documented in the commit.P1.3 — Lazy below-fold widgets (
DeferredWidget.component.tsx,MyDataPage.component.tsx)Each landing-page widget runs its own data-fetch effect on mount.
<DeferredWidget>wraps each grid item and defers rendering the child tree untiluseInView(fromreact-intersection-observer, already a dep) reports the wrapper has entered the viewport.rootMargin: '200px 0px'pre-loads widgets that are within 200 px of being visible so a normal scroll never reveals an empty placeholder.triggerOnce: truemeans once mounted the widget stays mounted — no re-mount on scroll-out.Tests:
Use cases covered
ETag+Cache-Control: must-revalidate, private; client revisit returns304+ zero bodyclearEtagCache()on logout prevents cross-principal cache leakage/mount on scroll, not on initial paint; users with very tall viewports see no behavior change (every widget already in view)Manual testing performed
CACHE_PROVIDER=none. Loaded/table/{fqn}twice — first response was 200 withETagheader, second was 304 with empty body (browser DevTools Network panel confirmed). Verified cached body re-rendered correctly.clearEtagCacheran.getQueriesListdid NOT fire on initial page load. Clicked the Queries tab — fetch fired exactly once, badge populated. Clicked Schema tab and back — no re-fetch. Navigated to a different Table — confirmed re-arm by activating Queries tab and seeing a fresh fetch./(MyDataPage). Confirmed only above-fold widgets fired their fetches on initial paint. Scrolled — below-fold widgets fired their fetches as they crossed the 200 px look-ahead boundary.mvn verify -P cache-testsprofile from PR Cache improvements: lineage + search layers, observability, CI gate #28012 to make sure no regression in the cache integration suite — green.UI screen recording / screenshots:
Not applicable as a recording is required — the changes are network-level (304 short-circuit) and effect-deferral; visually nothing changes on screen unless DevTools Network panel is open.
Checklist:
Summary by Gitar
DeferredWidget.component.tsxto resolve regressions in Playwright test shards.jest.config.jsto force a single instance ofreactandreact-domto prevent invalid hook call errors during testing ofcore-components.This will update automatically on new commits.