Skip to content

feat(query): add resource selector search for config insights#2020

Merged
moshloop merged 5 commits into
mainfrom
feat/insights-resource-selector
Jun 26, 2026
Merged

feat(query): add resource selector search for config insights#2020
moshloop merged 5 commits into
mainfrom
feat/insights-resource-selector

Conversation

@adityathebe

@adityathebe adityathebe commented Jun 25, 2026

Copy link
Copy Markdown
Member

What

Adds resource selector search support for config insights (the config_analysis table), matching how configs and config changes are already searchable.

This means insights can now be searched through the same machinery as the other resource kinds — including the POST /resources/search endpoint in mission-control, which is generic over SearchResourcesRequest/SearchResourcesResponse and therefore gains a new config_analysis key automatically.

Changes

  • query/models.go
    • New ConfigAnalysisQueryModel (searchable columns: config_id, scraper_id, source, analyzer, analysis_type, severity, status, summary, message, first_observed, last_observed; type/config aliases; date mappers) and registered it in GetModelFromTable.
    • New HasDeletedAt flag on QueryModel. The resource-selector engine previously always appended deleted_at IS NULL, which would error on config_analysis (no such column). The flag is set true on all existing 9 models to preserve behavior, and left false for config_analysis.
  • query/resource_selector.go
    • Gated the default deleted_at IS NULL clause on qm.HasDeletedAt.
    • Added ConfigAnalysis to SearchResourcesRequest / SearchResourcesResponse, GetIDs, and a new errgroup branch in SearchResources.
  • query/config_analysis.go (new) — FindConfigAnalysisByResourceSelector, FindConfigAnalysisIDsByResourceSelector, GetConfigAnalysisByIDs, mirroring the config-changes helpers.

Example

POST /resources/search
{ "config_analysis": [{ "search": "severity=critical status=open" }] }

Tests

  • New Config Analysis Resource Selector spec block: by config_id, analyzer (exact + prefix), analysis_type (via type alias), severity, status, no-match, and end-to-end via SearchResources.
  • Exported the dummy config-analysis fixtures with stable IDs/analyzers.
  • Full resource-selector suite passes (132 specs) — no regression from the HasDeletedAt change.

Summary by CodeRabbit

  • New Features

    • Added support for viewing and searching config analysis results alongside other resource types.
    • Search results now include severity information where available.
    • Expanded config analysis details returned in listings for richer context.
  • Bug Fixes

    • Improved deleted-item filtering so it only applies where supported.
    • Fixed database state handling to include config analysis data correctly.

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

Benchstat (RLS)

Base: 0e3531198b5881d9aa9a7976f131184f38475bcc
Head: b46d8c71fbb28fbc685886edd0390bd122ac3073

📊 8 minor regression(s) (all within 5% threshold)

Benchmark Base Head Change p-value
RLS/Sample-15000/analysis_types/With_RLS-4 3.880m 3.997m +3.01% 0.002
RLS/Sample-15000/analysis_types/Without_RLS-4 3.916m 4.034m +3.00% 0.002
RLS/Sample-15000/catalog_changes/With_RLS-4 141.3m 144.6m +2.39% 0.002
RLS/Sample-15000/config_classes/Without_RLS-4 4.081m 4.160m +1.94% 0.002
RLS/Sample-15000/config_classes/With_RLS-4 134.3m 136.6m +1.69% 0.009
RLS/Sample-15000/config_names/With_RLS-4 137.3m 138.9m +1.14% 0.002
RLS/Sample-15000/config_summary/Without_RLS-4 101.0m 101.9m +0.89% 0.004
RLS/Sample-15000/analyzer_types/Without_RLS-4 3.739m 3.757m +0.48% 0.002
✅ 4 improvement(s)
Benchmark Base Head Change p-value
RLS/Sample-15000/config_types/Without_RLS-4 4.865m 4.770m -1.96% 0.026
RLS/Sample-15000/change_types/With_RLS-4 5.392m 5.291m -1.88% 0.041
RLS/Sample-15000/config_changes/Without_RLS-4 5.384m 5.311m -1.35% 0.015
RLS/Sample-15000/config_detail/With_RLS-4 139.5m 137.8m -1.21% 0.002
Full benchstat output
goos: linux
goarch: amd64
pkg: github.com/flanksource/duty/bench
cpu: AMD EPYC 7763 64-Core Processor                
                                               │ bench-base.txt │          bench-head.txt           │
                                               │     sec/op     │   sec/op     vs base              │
RLS/Sample-15000/catalog_changes/Without_RLS-4      5.353m ± 1%   5.352m ± 1%       ~ (p=0.937 n=6)
RLS/Sample-15000/catalog_changes/With_RLS-4         141.3m ± 1%   144.6m ± 1%  +2.39% (p=0.002 n=6)
RLS/Sample-15000/config_changes/Without_RLS-4       5.384m ± 1%   5.311m ± 1%  -1.35% (p=0.015 n=6)
RLS/Sample-15000/config_changes/With_RLS-4          141.4m ± 0%   140.5m ± 1%       ~ (p=0.065 n=6)
RLS/Sample-15000/config_detail/Without_RLS-4        4.805m ± 1%   4.776m ± 1%       ~ (p=0.132 n=6)
RLS/Sample-15000/config_detail/With_RLS-4           139.5m ± 1%   137.8m ± 0%  -1.21% (p=0.002 n=6)
RLS/Sample-15000/config_names/Without_RLS-4         13.75m ± 2%   13.92m ± 1%       ~ (p=0.310 n=6)
RLS/Sample-15000/config_names/With_RLS-4            137.3m ± 0%   138.9m ± 0%  +1.14% (p=0.002 n=6)
RLS/Sample-15000/config_summary/Without_RLS-4       101.0m ± 2%   101.9m ± 1%  +0.89% (p=0.004 n=6)
RLS/Sample-15000/config_summary/With_RLS-4          708.3m ± 0%   705.0m ± 1%       ~ (p=0.065 n=6)
RLS/Sample-15000/configs/Without_RLS-4              8.032m ± 1%   8.148m ± 2%       ~ (p=0.065 n=6)
RLS/Sample-15000/configs/With_RLS-4                 136.9m ± 0%   135.8m ± 1%       ~ (p=0.180 n=6)
RLS/Sample-15000/analysis_types/Without_RLS-4       3.916m ± 1%   4.034m ± 0%  +3.00% (p=0.002 n=6)
RLS/Sample-15000/analysis_types/With_RLS-4          3.880m ± 2%   3.997m ± 2%  +3.01% (p=0.002 n=6)
RLS/Sample-15000/analyzer_types/Without_RLS-4       3.739m ± 0%   3.757m ± 2%  +0.48% (p=0.002 n=6)
RLS/Sample-15000/analyzer_types/With_RLS-4          3.759m ± 4%   3.765m ± 1%       ~ (p=0.818 n=6)
RLS/Sample-15000/change_types/Without_RLS-4         5.308m ± 1%   5.284m ± 1%       ~ (p=0.240 n=6)
RLS/Sample-15000/change_types/With_RLS-4            5.392m ± 1%   5.291m ± 2%  -1.88% (p=0.041 n=6)
RLS/Sample-15000/config_classes/Without_RLS-4       4.081m ± 0%   4.160m ± 1%  +1.94% (p=0.002 n=6)
RLS/Sample-15000/config_classes/With_RLS-4          134.3m ± 2%   136.6m ± 1%  +1.69% (p=0.009 n=6)
RLS/Sample-15000/config_types/Without_RLS-4         4.865m ± 2%   4.770m ± 2%  -1.96% (p=0.026 n=6)
RLS/Sample-15000/config_types/With_RLS-4            137.2m ± 4%   139.0m ± 1%       ~ (p=0.065 n=6)
geomean                                             21.04m        21.13m       +0.42%

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

Benchstat (Other)

Base: 0e3531198b5881d9aa9a7976f131184f38475bcc
Head: b46d8c71fbb28fbc685886edd0390bd122ac3073

✅ 2 improvement(s)
Benchmark Base Head Change p-value
ResourceSelectorQueryBuild/name-4 45.49µ 45.02µ -1.05% 0.015
ResourceSelectorQueryBuild/name_and_type-4 66.05µ 65.55µ -0.75% 0.041
Full benchstat output
goos: linux
goarch: amd64
pkg: github.com/flanksource/duty/bench
cpu: AMD EPYC 7763 64-Core Processor                
                                                       │ bench-base.txt │           bench-head.txt           │
                                                       │     sec/op     │    sec/op     vs base              │
InsertionForRowsWithAliases/external_users.aliases-4       612.9µ ±  5%   613.3µ ± 13%       ~ (p=0.818 n=6)
InsertionForRowsWithAliases/config_items.external_id-4     1.129m ± 12%   1.117m ± 13%       ~ (p=1.000 n=6)
InsertionOfConfigsWithProperties-4                         3.892m ± 93%   3.971m ±  1%       ~ (p=0.065 n=6)
UpdateOfConfigsWithProperties-4                            7.805m ±  1%   7.851m ±  6%       ~ (p=0.485 n=6)
ResourceSelectorConfigs/name-4                             222.6µ ±  3%   224.7µ ±  3%       ~ (p=0.485 n=6)
ResourceSelectorConfigs/name_and_type-4                    246.1µ ±  3%   245.2µ ±  5%       ~ (p=1.000 n=6)
ResourceSelectorConfigs/tags-4                             32.01m ±  2%   32.27m ±  5%       ~ (p=0.240 n=6)
ResourceSelectorQueryBuild/name-4                          45.49µ ±  0%   45.02µ ±  1%  -1.05% (p=0.015 n=6)
ResourceSelectorQueryBuild/name_and_type-4                 66.05µ ±  0%   65.55µ ±  1%  -0.75% (p=0.041 n=6)
ResourceSelectorQueryBuild/tags-4                          18.32µ ±  2%   18.33µ ±  1%       ~ (p=0.818 n=6)
geomean                                                    538.0µ         538.6µ        +0.12%

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Adds config-analysis query support, maps it through resource selection, updates deleted-row handling and config-analysis model/view definitions, extends storage backfills, and adds fixtures plus selector tests.

Changes

Config analysis search support

Layer / File(s) Summary
Query model contract
query/models.go
QueryModel gains HasDeletedAt, and existing query models set it true for tables with soft-deleted rows.
Config-analysis model and storage
query/models.go, views/006_config_views.sql, schema/apply.go, views/002_seed.sql
ConfigAnalysisQueryModel maps config_analysis_items, GetModelFromTable returns it for config-analysis tables, and the storage scripts add config_analysis.properties_values support.
Config analysis lookup helpers
query/config_analysis.go
Config-analysis selector helpers resolve resource-selector IDs and load models.ConfigAnalysis rows by UUID.
Resource selector wiring
query/resource_selector.go
SearchResourcesRequest and SearchResourcesResponse add config_analysis, SelectedResource adds Severity, config-analysis results are fetched and mapped, and deleted-row filtering now checks HasDeletedAt.
Fixtures and selector tests
tests/fixtures/dummy/config_analysis.go, tests/query_resource_selector_test.go
Dummy config-analysis fixtures are factored into reusable variables, and selector tests cover config-analysis filtering and SearchResources output.

Sequence Diagram(s)

sequenceDiagram
  participant SearchResources
  participant FindConfigAnalysisByResourceSelector
  participant FindConfigAnalysisIDsByResourceSelector
  participant queryTableWithResourceSelectors
  participant GetConfigAnalysisByIDs
  participant database

  SearchResources->>FindConfigAnalysisByResourceSelector: ConfigAnalysis selectors
  FindConfigAnalysisByResourceSelector->>FindConfigAnalysisIDsByResourceSelector: resourceSelectors + limit
  FindConfigAnalysisIDsByResourceSelector->>queryTableWithResourceSelectors: resolve IDs
  queryTableWithResourceSelectors-->>FindConfigAnalysisIDsByResourceSelector: []uuid.UUID
  FindConfigAnalysisByResourceSelector->>GetConfigAnalysisByIDs: ids
  GetConfigAnalysisByIDs->>database: id IN ? query
  database-->>GetConfigAnalysisByIDs: []models.ConfigAnalysis
  GetConfigAnalysisByIDs-->>SearchResources: ConfigAnalysis results
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding resource selector search support for config insights.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/insights-resource-selector
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch feat/insights-resource-selector

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Adds resource selector support for config_analysis (insights), matching
how configs and config changes are already searchable.

- New ConfigAnalysisQueryModel + registration in GetModelFromTable
- FindConfigAnalysisByResourceSelector helpers
- ConfigAnalysis wired into SearchResources request/response
- HasDeletedAt flag on QueryModel so tables without a deleted_at column
  (config_analysis) skip the default soft-delete filter
@adityathebe adityathebe force-pushed the feat/insights-resource-selector branch from 74acd2a to 54cd555 Compare June 25, 2026 20:46
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

Gavel results

Gavel exited with code .

View full results

@adityathebe adityathebe requested a review from moshloop June 26, 2026 07:59
ConfigAnalysisQueryModel enables HasProperties, which routes the unkeyed
PEG form `properties=<value>` through the `properties_values` generated
column. That column was only created for config_items and components, so
the same search against config_analysis failed at runtime with
'column "properties_values" does not exist'.

Add config_analysis to the generated-column migration in 002_seed.sql and
exclude it from the atlas diff in apply.go, matching how config_items and
components are handled. Also drop two redundant self-aliases
(first_observed/last_observed) from ConfigAnalysisQueryModel.
Add a Severity *string field to SelectedResource, populated only for
config insights. Severity is orthogonal to Health (healthy/unhealthy/
warning/unknown) and Status, so it gets its own field rather than
overloading either. Other resource kinds leave it nil, which is omitted
from the JSON response.
@adityathebe adityathebe marked this pull request as draft June 26, 2026 09:23
@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
{}

Use config_analysis_items for resource selector searches so config insight queries can filter on parent config fields such as agent_id, deleted_at, tags, labels, and type.

Extend the view with the parent config fields and cover config type filtering in resource selector tests.
@adityathebe adityathebe marked this pull request as ready for review June 26, 2026 09:45
Remove the conflicting config alias now that config analysis selectors expose the parent config JSON. Also stop treating analysis as a JSON selector field until it is explicitly supported.

Add regression coverage for config_id filtering, unsupported analysis JSON lookup, and high/critical pod analysis selection.
@adityathebe

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/query_resource_selector_test.go`:
- Around line 795-800: Tighten the test in FindConfigAnalysisByResourceSelector
so it verifies the type filter is actually applied, not just that
dummy.LogisticsAPIPodAnalysis is returned. Update the assertion around the
"finds high or critical pod analysis" case to either scope the query with a
specific config_id or explicitly assert that other critical non-pod fixture IDs
are not present in gotIDs, using the existing
query.FindConfigAnalysisByResourceSelector and dummy fixture identifiers.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cdea0df6-404e-4e75-ad5d-b2764b54ee18

📥 Commits

Reviewing files that changed from the base of the PR and between 1ac4ce2 and b46d8c7.

📒 Files selected for processing (5)
  • query/config_analysis.go
  • query/models.go
  • tests/fixtures/dummy/config_analysis.go
  • tests/query_resource_selector_test.go
  • views/006_config_views.sql
🚧 Files skipped from review as they are similar to previous changes (2)
  • query/config_analysis.go
  • query/models.go

Comment on lines +795 to +800
ginkgo.It("finds high or critical pod analysis", func() {
analyses, err := query.FindConfigAnalysisByResourceSelector(DefaultContext, -1, types.ResourceSelector{Search: "severity=high,critical type=Kubernetes::Pod"})
Expect(err).To(BeNil())
gotIDs := lo.Map(analyses, func(a models.ConfigAnalysis, _ int) uuid.UUID { return a.ID })
Expect(gotIDs).To(ContainElement(dummy.LogisticsAPIPodAnalysis.ID))
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Tighten the pod-selector assertion.

This only proves the pod analysis is included. Since the fixtures already contain other critical analyses, a regression that ignores type=Kubernetes::Pod would still pass here. Scope this query by config_id or assert the non-pod analysis IDs are excluded.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/query_resource_selector_test.go` around lines 795 - 800, Tighten the
test in FindConfigAnalysisByResourceSelector so it verifies the type filter is
actually applied, not just that dummy.LogisticsAPIPodAnalysis is returned.
Update the assertion around the "finds high or critical pod analysis" case to
either scope the query with a specific config_id or explicitly assert that other
critical non-pod fixture IDs are not present in gotIDs, using the existing
query.FindConfigAnalysisByResourceSelector and dummy fixture identifiers.

@moshloop moshloop merged commit 0f543e8 into main Jun 26, 2026
20 of 22 checks passed
@moshloop moshloop deleted the feat/insights-resource-selector branch June 26, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants