Skip to content

perf(context): skip unused CEL env funcs#2014

Closed
adityathebe wants to merge 1 commit into
mainfrom
test/cel-env-func-benchmark
Closed

perf(context): skip unused CEL env funcs#2014
adityathebe wants to merge 1 commit into
mainfrom
test/cel-env-func-benchmark

Conversation

@adityathebe

@adityathebe adityathebe commented Jun 23, 2026

Copy link
Copy Markdown
Member

RunTemplate built every registered CEL env function before evaluating templates, even for simple CEL expressions that did not call any registered functions.

This caused avoidable allocation churn on cache-hit CEL evaluations because DB/catalog/gitops function declarations and closures were rebuilt every call.

Only append registered CEL env options when the CEL expression calls the corresponding function. Add a focused benchmark covering cache-hit CEL evaluation with different registered function counts.

Summary by CodeRabbit

  • Performance

    • Improved template evaluation efficiency by loading only referenced CEL environment functions instead of all registered functions, reducing overhead in scenarios with many available functions.
  • Tests

    • Added comprehensive test coverage for CEL function reference detection and template evaluation optimization.

RunTemplate constructed every registered CEL env function before evaluating any template, even for simple CEL expressions that did not call those functions. This created avoidable closure and cel.Function allocation churn on cache-hit evaluations.

Only append registered CEL env options when the expression calls the corresponding function, while preserving support for legacy registry keys that end in Cel. Add a benchmark for cache-hit CEL evaluation with varying registered function counts.
@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

RunTemplate is changed to register only the CEL environment functions referenced in a template expression, rather than all registered functions. A new appendReferencedCelEnvFuncs helper scans the expression for identifier-boundary matches followed by (. Tests verify the detection logic and selective construction behavior; a benchmark measures cache-hit overhead across 0, 18, and 64 registered functions.

Changes

Selective CEL env function registration

Layer / File(s) Summary
appendReferencedCelEnvFuncs helper and RunTemplate change
context/template.go
Adds strings import. Changes RunTemplate to call appendReferencedCelEnvFuncs instead of unconditionally appending all CelEnvFuncs. Implements unexported helpers that locate identifier-boundary matches in t.Expression, skip CEL-style whitespace, and confirm a trailing ( to decide which functions to register.
Unit tests for selective registration and expression scanning
context/template_test.go
Adds sync/atomic and CEL package imports. TestRunTemplateOnlyBuildsReferencedCelEnvFuncs overrides CelEnvFuncs with atomic-counter factories and asserts only the referenced factory is constructed. TestCelExpressionCalls validates celExpressionCalls across whitespace, prefix, suffix, and qualified-name cases.
Cache-hit benchmark with varying registered function counts
bench/template_cel_env_bench_test.go
BenchmarkRunTemplateCELCacheHitRegisteredEnvFuncs warms the CEL program cache, then measures RunTemplateBool allocations for 0, 18, and 64 globally registered CEL env functions. installBenchmarkCelEnvFuncs temporarily replaces global CelEnvFuncs/TemplateFuncs and restores them via b.Cleanup. benchmarkCelEnvFunc constructs individually named CEL function definitions used by the benchmark.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 9.09% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'perf(context): skip unused CEL env funcs' directly aligns with the main objective: optimizing CEL template evaluation by avoiding initialization of unreferenced functions, as documented in the objectives.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/cel-env-func-benchmark
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch test/cel-env-func-benchmark

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@adityathebe adityathebe marked this pull request as draft June 23, 2026 18:10
@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown

Benchstat (Other)

Base: fe0592b12ec8a287e14c7576c5cd27116c680eae
Head: cb5ced1ccb3981173222d3ac261d52350b44b924

📊 1 minor regression(s) (all within 5% threshold)

Benchmark Base Head Change p-value
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=00-4 1.997µ 2.031µ +1.73% 0.002
✅ 7 improvement(s)
Benchmark Base Head Change p-value
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=64-4 19.493Ki 1.000Ki -94.87% 0.002
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=64-4 481.00 26.00 -94.59% 0.002
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=64-4 17.940µ 2.042µ -88.62% 0.002
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=18-4 6.414Ki 1.000Ki -84.41% 0.002
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=18-4 158.00 26.00 -83.54% 0.002
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=18-4 6.827µ 2.035µ -70.20% 0.002
ResourceSelectorQueryBuild/name_and_type-4 62.70µ 62.24µ -0.73% 0.009
Full benchstat output
goos: linux
goarch: amd64
pkg: github.com/flanksource/duty/bench
cpu: AMD EPYC 7763 64-Core Processor                
                                                         │ bench-base.txt │           bench-head.txt            │
                                                         │     sec/op     │    sec/op     vs base               │
InsertionForRowsWithAliases/external_users.aliases-4         575.6µ ±  7%   578.0µ ± 11%        ~ (p=0.240 n=6)
InsertionForRowsWithAliases/config_items.external_id-4       1.082m ± 11%   1.084m ± 11%        ~ (p=1.000 n=6)
InsertionOfConfigsWithProperties-4                           3.687m ±  2%   3.668m ±  1%        ~ (p=0.180 n=6)
UpdateOfConfigsWithProperties-4                              7.346m ±  0%   7.364m ±  4%        ~ (p=0.589 n=6)
ResourceSelectorConfigs/name-4                               219.9µ ±  1%   218.6µ ±  3%        ~ (p=0.699 n=6)
ResourceSelectorConfigs/name_and_type-4                      233.1µ ±  6%   233.1µ ±  6%        ~ (p=0.589 n=6)
ResourceSelectorConfigs/tags-4                               28.91m ±  3%   28.48m ±  3%        ~ (p=0.699 n=6)
ResourceSelectorQueryBuild/name-4                            42.86µ ±  1%   42.44µ ±  1%        ~ (p=0.093 n=6)
ResourceSelectorQueryBuild/name_and_type-4                   62.70µ ±  1%   62.24µ ±  1%   -0.73% (p=0.009 n=6)
ResourceSelectorQueryBuild/tags-4                            17.02µ ±  1%   17.00µ ±  0%        ~ (p=0.368 n=6)
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=00-4     1.997µ ±  0%   2.031µ ±  1%   +1.73% (p=0.002 n=6)
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=18-4     6.827µ ±  1%   2.035µ ±  0%  -70.20% (p=0.002 n=6)
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=64-4    17.940µ ±  0%   2.042µ ±  2%  -88.62% (p=0.002 n=6)
geomean                                                      184.1µ         141.7µ        -23.03%

                                                         │ bench-base.txt │            bench-head.txt             │
                                                         │      B/op      │     B/op      vs base                 │
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=00-4     1.000Ki ± 0%   1.000Ki ± 0%        ~ (p=1.000 n=6) ¹
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=18-4     6.414Ki ± 0%   1.000Ki ± 0%  -84.41% (p=0.002 n=6)
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=64-4    19.493Ki ± 0%   1.000Ki ± 0%  -94.87% (p=0.002 n=6)
geomean                                                      5.000Ki        1.000Ki       -80.00%
¹ all samples are equal

                                                         │ bench-base.txt │           bench-head.txt            │
                                                         │   allocs/op    │ allocs/op   vs base                 │
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=00-4       26.00 ± 0%   26.00 ± 0%        ~ (p=1.000 n=6) ¹
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=18-4      158.00 ± 0%   26.00 ± 0%  -83.54% (p=0.002 n=6)
RunTemplateCELCacheHitRegisteredEnvFuncs/registered=64-4      481.00 ± 0%   26.00 ± 0%  -94.59% (p=0.002 n=6)
geomean                                                        125.5        26.00       -79.28%
¹ all samples are equal

@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown

Benchstat (RLS)

Base: fe0592b12ec8a287e14c7576c5cd27116c680eae
Head: cb5ced1ccb3981173222d3ac261d52350b44b924

📊 2 minor regression(s) (all within 5% threshold)

Benchmark Base Head Change p-value
RLS/Sample-15000/config_names/With_RLS-4 140.9m 142.9m +1.42% 0.002
RLS/Sample-15000/catalog_changes/With_RLS-4 140.8m 142.1m +0.97% 0.002
✅ 7 improvement(s)
Benchmark Base Head Change p-value
RLS/Sample-15000/config_changes/Without_RLS-4 5.860m 5.530m -5.64% 0.002
RLS/Sample-15000/config_types/Without_RLS-4 5.241m 5.002m -4.56% 0.041
RLS/Sample-15000/analyzer_types/With_RLS-4 4.003m 3.824m -4.46% 0.002
RLS/Sample-15000/analysis_types/With_RLS-4 4.120m 3.994m -3.08% 0.015
RLS/Sample-15000/config_changes/With_RLS-4 142.9m 140.4m -1.80% 0.009
RLS/Sample-15000/config_classes/With_RLS-4 138.0m 135.6m -1.75% 0.002
RLS/Sample-15000/configs/With_RLS-4 137.8m 137.2m -0.50% 0.009
Full benchstat output
goos: linux
goarch: amd64
pkg: github.com/flanksource/duty/bench
cpu: AMD EPYC 7763 64-Core Processor                
                                               │ bench-base.txt │          bench-head.txt           │
                                               │     sec/op     │   sec/op     vs base              │
RLS/Sample-15000/catalog_changes/Without_RLS-4     5.608m ±  4%   5.479m ± 2%       ~ (p=0.132 n=6)
RLS/Sample-15000/catalog_changes/With_RLS-4        140.8m ±  0%   142.1m ± 0%  +0.97% (p=0.002 n=6)
RLS/Sample-15000/config_changes/Without_RLS-4      5.860m ±  3%   5.530m ± 2%  -5.64% (p=0.002 n=6)
RLS/Sample-15000/config_changes/With_RLS-4         142.9m ±  1%   140.4m ± 1%  -1.80% (p=0.009 n=6)
RLS/Sample-15000/config_detail/Without_RLS-4       5.014m ± 10%   5.150m ± 5%       ~ (p=0.485 n=6)
RLS/Sample-15000/config_detail/With_RLS-4          137.2m ±  1%   137.9m ± 0%       ~ (p=0.180 n=6)
RLS/Sample-15000/config_names/Without_RLS-4        14.66m ±  1%   14.40m ± 3%       ~ (p=0.132 n=6)
RLS/Sample-15000/config_names/With_RLS-4           140.9m ±  1%   142.9m ± 1%  +1.42% (p=0.002 n=6)
RLS/Sample-15000/config_summary/Without_RLS-4      102.0m ±  9%   102.5m ± 1%       ~ (p=0.394 n=6)
RLS/Sample-15000/config_summary/With_RLS-4         711.2m ±  1%   712.1m ± 1%       ~ (p=0.818 n=6)
RLS/Sample-15000/configs/Without_RLS-4             9.246m ± 11%   9.179m ± 7%       ~ (p=0.394 n=6)
RLS/Sample-15000/configs/With_RLS-4                137.8m ±  3%   137.2m ± 1%  -0.50% (p=0.009 n=6)
RLS/Sample-15000/analysis_types/Without_RLS-4      3.995m ±  3%   3.973m ± 3%       ~ (p=0.240 n=6)
RLS/Sample-15000/analysis_types/With_RLS-4         4.120m ±  3%   3.994m ± 2%  -3.08% (p=0.015 n=6)
RLS/Sample-15000/analyzer_types/Without_RLS-4      3.853m ±  5%   3.785m ± 1%       ~ (p=0.310 n=6)
RLS/Sample-15000/analyzer_types/With_RLS-4         4.003m ±  2%   3.824m ± 2%  -4.46% (p=0.002 n=6)
RLS/Sample-15000/change_types/Without_RLS-4        5.759m ±  4%   5.516m ± 6%       ~ (p=0.093 n=6)
RLS/Sample-15000/change_types/With_RLS-4           5.881m ±  6%   5.630m ± 4%       ~ (p=0.132 n=6)
RLS/Sample-15000/config_classes/Without_RLS-4      4.276m ± 19%   4.215m ± 3%       ~ (p=0.818 n=6)
RLS/Sample-15000/config_classes/With_RLS-4         138.0m ±  0%   135.6m ± 1%  -1.75% (p=0.002 n=6)
RLS/Sample-15000/config_types/Without_RLS-4        5.241m ±  5%   5.002m ± 6%  -4.56% (p=0.041 n=6)
RLS/Sample-15000/config_types/With_RLS-4           137.3m ±  1%   135.7m ± 2%       ~ (p=0.065 n=6)
geomean                                            21.93m         21.58m       -1.56%

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@bench/template_cel_env_bench_test.go`:
- Around line 41-54: The benchmark test uses b.Fatal for assertions instead of
gomega expectations, which violates the coding guidelines. Replace the b.Fatal
error checks and the b.Fatal boolean check in the benchmark loop with gomega
assertions by first creating a gomega instance with gomega.NewWithT(b), then use
Expect to verify the error is nil and the result ok is true. Specifically,
replace the two b.Fatal calls related to the ctx.RunTemplateBool function calls
with gomega Expect assertions that check for nil errors and that ok evaluates to
true.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 60adbd86-1fbc-48a3-a163-78893cb2293c

📥 Commits

Reviewing files that changed from the base of the PR and between fe0592b and cb5ced1.

📒 Files selected for processing (3)
  • bench/template_cel_env_bench_test.go
  • context/template.go
  • context/template_test.go

Comment on lines +41 to +54
if _, err := ctx.RunTemplateBool(tmpl, env); err != nil {
b.Fatal(err)
}

b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
ok, err := ctx.RunTemplateBool(tmpl, env)
if err != nil {
b.Fatal(err)
}
if !ok {
b.Fatal("expected expression to evaluate true")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

# Locate the benchmark test file and check its structure
find . -name "template_cel_env_bench_test.go" -o -name "*_test.go" -path "*/bench/*" 2>/dev/null

Repository: flanksource/duty

Length of output: 323


🏁 Script executed:

# Read the benchmark test file
cat -n bench/template_cel_env_bench_test.go

Repository: flanksource/duty

Length of output: 3651


🏁 Script executed:

# Check the imports and lines around 41-54 in detail
sed -n '1,80p' bench/template_cel_env_bench_test.go | cat -n

Repository: flanksource/duty

Length of output: 3149


Use gomega assertions in this _test.go benchmark.

This benchmark uses assertion-style b.Fatal checks instead of gomega expectations. Please switch to gomega.NewWithT(b) and Expect(...) for these checks.

Proposed patch
 import (
 	"fmt"
 	"testing"
 	"time"
 
 	dutyctx "github.com/flanksource/duty/context"
 	"github.com/flanksource/gomplate/v3"
 	"github.com/google/cel-go/cel"
 	celtypes "github.com/google/cel-go/common/types"
 	"github.com/google/cel-go/common/types/ref"
+	. "github.com/onsi/gomega"
 )
@@
 	for _, registeredFuncs := range []int{0, 18, 64} {
 		b.Run(fmt.Sprintf("registered=%02d", registeredFuncs), func(b *testing.B) {
+			g := NewWithT(b)
 			installBenchmarkCelEnvFuncs(b, registeredFuncs)
@@
-			if _, err := ctx.RunTemplateBool(tmpl, env); err != nil {
-				b.Fatal(err)
-			}
+			_, err := ctx.RunTemplateBool(tmpl, env)
+			g.Expect(err).NotTo(HaveOccurred())
@@
 			b.ReportAllocs()
 			b.ResetTimer()
 			for i := 0; i < b.N; i++ {
-				ok, err := ctx.RunTemplateBool(tmpl, env)
-				if err != nil {
-					b.Fatal(err)
-				}
-				if !ok {
-					b.Fatal("expected expression to evaluate true")
-				}
+				ok, err := ctx.RunTemplateBool(tmpl, env)
+				g.Expect(err).NotTo(HaveOccurred())
+				g.Expect(ok).To(BeTrue())
 			}
 		})
 	}
 }

Per coding guidelines, **/*_test.go: "Always use github.com/onsi/gomega package for assertions in test files" and "use the gomega.NewWithT(t) approach for assertions."

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if _, err := ctx.RunTemplateBool(tmpl, env); err != nil {
b.Fatal(err)
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
ok, err := ctx.RunTemplateBool(tmpl, env)
if err != nil {
b.Fatal(err)
}
if !ok {
b.Fatal("expected expression to evaluate true")
}
_, err := ctx.RunTemplateBool(tmpl, env)
g.Expect(err).NotTo(HaveOccurred())
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
ok, err := ctx.RunTemplateBool(tmpl, env)
g.Expect(err).NotTo(HaveOccurred())
g.Expect(ok).To(BeTrue())
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@bench/template_cel_env_bench_test.go` around lines 41 - 54, The benchmark
test uses b.Fatal for assertions instead of gomega expectations, which violates
the coding guidelines. Replace the b.Fatal error checks and the b.Fatal boolean
check in the benchmark loop with gomega assertions by first creating a gomega
instance with gomega.NewWithT(b), then use Expect to verify the error is nil and
the result ok is true. Specifically, replace the two b.Fatal calls related to
the ctx.RunTemplateBool function calls with gomega Expect assertions that check
for nil errors and that ok evaluates to true.

Source: Coding guidelines

@github-actions

Copy link
Copy Markdown

Gavel results

Gavel exited with code .

View full results

@adityathebe adityathebe deleted the test/cel-env-func-benchmark branch June 24, 2026 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant