Kafka Connect: Add Trivy CVE scan to CI by rmoff · Pull Request #15430 · apache/iceberg

rmoff · 2026-02-24T14:11:35Z

Summary

Adds a Trivy vulnerability scan to the Kafka Connect CI workflow
Scans the bundled jars for CRITICAL/HIGH CVEs
On push events (main, version branches, RC tags), uploads SARIF results to GitHub's Security tab
On PRs, outputs scan results to CI logs for developer visibility
Fails the job, but since the job is not a required status check, it will not block merging PRs.

Context

Discussion on dev@ mailing list: https://lists.apache.org/thread/kbf98950pzstzgon92st7mh9vrbv5yhb

Confluent Marketplace requires a Trivy scan before listing connectors. This has previously caught CVEs that needed patching (e.g. #14985). Running the scan in CI catches vulnerabilities during development and — critically — on RC tags before the release vote starts, when fixes can still be applied.

This is independent of #15212 (adding the KC artifact to the release process) and can land in either order.

kevinjqliu · 2026-02-26T05:28:29Z

i think its better to do this in the docker image publishing step, similar to what Superset is doing

rmoff · 2026-02-26T09:33:14Z

@kevinjqliu I'm not sure I follow. How's this relate to publishing docker images? This is to identify CVEs in the Kafka Connect connector itself. thanks.

kevinjqliu · 2026-02-26T20:11:02Z

Looking at how Trivy is used in Superset, its scanning the docker image only. In this PR, we're unpacking the jars to scan. I think it makes more sense to build a kafka connect image and then use Trivy to scan the image.

Just my preference

rymurr · 2026-02-27T11:41:17Z

Looking at how Trivy is used in Superset, its scanning the docker image only. In this PR, we're unpacking the jars to scan. I think it makes more sense to build a kafka connect image and then use Trivy to scan the image.

Just my preference

Not really clear why we would add the infra to build a docker image just to use trivy. Seems like a direct scan is more parsimonious

rymurr

LGTM, I just want to see if we can avoid 2x build w/o over complicating the CI job

kevinjqliu · 2026-02-27T19:47:44Z

Not really clear why we would add the infra to build a docker image just to use trivy. Seems like a direct scan is more parsimonious

Just looking at general patterns from the apache repos. https://grep.app/search?f.repo.pattern=apache%2F&q=uses%3A+aquasecurity%2Ftrivy-action
All the use cases I can find are using trivy for scanning images. (using image-ref: )

The only instance of scan-type: 'fs' i found has it disabled https://github.com/apache/plc4x/blob/eb41533bfab101acb87b9acdaf81c70d2e2fa286/.github/workflows/sast.yaml#L34-L47

From the docs, it seems like Filesystem scan and Container image scan are similar in that they both scan for Vulnerabilities, Misconfigurations, Secrets, and Licenses.

I think it would be helpful here if you can run the change on your fork repo and see if the fs trivy scan catches the currently open CVE for kafka connect (#15440)

rmoff · 2026-03-02T11:41:15Z

I think it would be helpful here if you can run the change on your fork repo and see if the fs trivy scan catches the currently open CVE for kafka connect (#15440)

That's what's shown here under "Reproducing" section.

rymurr

The changes themselves look good to me. I am just confused/questioning the github actions config...

My assumption is we want:

results to be uploaded regardless of pass/fail states of steps above
the job to fail if it detects errors
the 2nd scan to run regardless of if the other failed
I think whats happening now is the first always looks like it fails, the second always looks like it passes and the entire job always looks like it passes.

It might be worth trying this a few times in your own fork to make sure this actaully has the edge cases we expect.

kevinjqliu

Thanks for the PR! I think this is a great addition, however I'm concerned about supply chain risk due to the recent (second) compromise of trivy's infra.

See

I see we are using aquasecurity/trivy-action@e368e328979b113139d6f9068e03accaed98a518 . Glad to see we're using the commit hash.

I believe this is still ongoing; we should wait until theres resolution to proceed with this PR.

EDIT: looks like trivy was just pulled from the list of allowed actions by asf-infra apache/infrastructure-actions#548
https://infra.apache.org/blog/trivy_security_incident.html

lhotari · 2026-03-23T19:36:53Z

EDIT: looks like trivy was just pulled from the list of allowed actions by asf-infra apache/infrastructure-actions#548
https://infra.apache.org/blog/trivy_security_incident.html

@kevinjqliu PR to add it: apache/infrastructure-actions#573

kevinjqliu · 2026-04-01T21:19:19Z

apache/infrastructure-actions#582 is merged
we can try using lhotari/sandboxed-trivy-action, which is a fork running trivy in a locked-down docker container (https://github.com/lhotari/sandboxed-trivy-action)

ASF Infra has already allowlisted it

rmoff · 2026-04-07T16:30:41Z

apache/infrastructure-actions#582 is merged we can try using lhotari/sandboxed-trivy-action, which is a fork running trivy in a locked-down docker container (https://github.com/lhotari/sandboxed-trivy-action)

ASF Infra has already allowlisted it

Thanks @kevinjqliu.
I've raised a PR to add rootfs support to the action so that we can use it: lhotari/sandboxed-trivy-action#1
edit: now merged, ~~waiting on apache/infrastructure-actions#711~~
edit edit: apache/infrastructure-actions#718 now merged, so we can use f01374b6cc3bf7264ab238293e94f6db7ada6dd0 # v1.0.2 of the trivy action which includes rootfs.

rmoff · 2026-04-15T16:59:18Z

OK, I think this PR is ready for folk to take another look please.
It uses the locked-down Trivy action (h/t @lhotari), which has been allowlisted by ASF.

How it works:

Adds a new Kafka Connect CVE Scan workflow, separate from the existing Kafka Connect CI tests
Builds distZip, unpacks it, and scans the JARs using rootfs mode
Flag and fail: exit-code 1 fails the job when CVEs are found. Since this job is not a required check, it won't block merging.
On push to main/release branches: uploads SARIF to the GitHub Security tab
On PRs: prints findings to the CI log (SARIF upload skipped — GitHub only accepts results from default/protected branches)

Here are the execution paths, expected behaviours, and test evidence:

#	Event	CVEs present?	CVE Scan job	SARIF upload	Test run
1	`pull_request`	No	Green (exit 0)	Skipped	apache #25001192662
2	`pull_request`	Yes	Red (exit 1)	Skipped	apache #25001846149
3	`push`	No	Green (exit 0)	Uploads empty SARIF	fork #24563271199 ¹
4	`push`	Yes	Red (exit 1)	Uploads SARIF	fork #24562836884 ¹

¹ Push-event workflows only trigger when present on the default branch, so scenarios 3 & 4 reference runs from an earlier iteration where Trivy was a step in the test job. The scan configuration and SARIF upload logic are unchanged.

N.B. To test "no CVEs present" condition, known CVEs were temporarily suppressed via .trivyignore.

rymurr · 2026-04-17T08:51:31Z

+    # Behaviour:
+    #   - Flag, don't block: the scan step uses exit-code 1 so it
+    #     "fails" when CVEs are found, but continue-on-error keeps
+    #     the overall job green. GitHub Actions shows the step with


I don't think this is true. Perhaps a hallucination.

My suggestion would be to invert the flow.

remove continue-on-error and exit-code from the first run.

change the if statement to always() && matrix.jvm==21

This job will fail adn show 'red' in the UI but its a non-blocking check and the PR can still be merged. the always clause ensures that the results are always uploaded and printed regardless of the state of the scan.

I think this has the desired ux: the user knows they have a CVE and why but it isn't blocking for the PR (eg if the patched version hasn';t been released yet)

Makes sense, thanks @rymurr. Sounds like a good option.
Out of interest where can I see which checks will block a merge and which won't?

Re. exit-code, don't we still want this to be 1, otherwise the step won't fail (it defaults to 0) and show the job as red?

OK, I've made this change now, and updated the above detail & testing to reflect it.
@rymurr please could you take another look?

Add a new workflow (kafka-connect-cve-scan.yml) that scans bundled Kafka Connect jars for known CVEs using Trivy in rootfs mode. The scan runs as a separate, non-required check so that CVE findings are visible without blocking merges. On push to main/release branches, results are uploaded as SARIF to the GitHub Security tab. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

old review

Add a new workflow (kafka-connect-cve-scan.yml) that scans bundled Kafka Connect jars for known CVEs using Trivy in rootfs mode. The scan runs as a separate, non-required check so that CVE findings are visible without blocking merges. On push to main/release branches, results are uploaded as SARIF to the GitHub Security tab. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

rymurr · 2026-05-11T16:15:54Z

This LGTM. I think its a good start. We can tweak to see how much people pay attention to this alert and how easy they are to imporve

kevinjqliu · 2026-05-11T17:06:08Z

looks like this is blocking CI for main branch, i opened #16287 as a followup. Please take a look!

github-actions Bot added the INFRA label Feb 24, 2026

rmoff marked this pull request as draft February 24, 2026 14:12

rmoff marked this pull request as ready for review February 24, 2026 14:36

rymurr reviewed Feb 27, 2026

View reviewed changes

Comment thread .github/workflows/kafka-connect-ci.yml Outdated

Comment thread .github/workflows/kafka-connect-ci.yml Outdated

rmoff requested a review from rymurr March 17, 2026 09:44

rymurr reviewed Mar 20, 2026

View reviewed changes

Comment thread .github/workflows/kafka-connect-ci.yml Outdated

kevinjqliu previously requested changes Mar 20, 2026

View reviewed changes

rmoff mentioned this pull request Apr 7, 2026

Add rootfs scan type support lhotari/sandboxed-trivy-action#1

Merged

lhotari mentioned this pull request Apr 14, 2026

Add lhotari/sandboxed-trivy-action v1.0.2 to approved actions apache/infrastructure-actions#711

Closed

rmoff force-pushed the trivy-cve-scan-kafka-connect branch from 7f169af to 9d21bb3 Compare April 15, 2026 16:56

rmoff force-pushed the trivy-cve-scan-kafka-connect branch 2 times, most recently from ed906dd to 0264d75 Compare April 15, 2026 17:34

rmoff requested review from kevinjqliu and rymurr April 15, 2026 18:10

rymurr reviewed Apr 17, 2026

View reviewed changes

rmoff force-pushed the trivy-cve-scan-kafka-connect branch 2 times, most recently from 6d19226 to d1b2ca2 Compare April 17, 2026 13:15

rmoff requested a review from rymurr April 17, 2026 13:27

rmoff force-pushed the trivy-cve-scan-kafka-connect branch from 216abed to e3a2a5d Compare April 27, 2026 14:00

rmoff marked this pull request as draft April 27, 2026 14:02

rmoff force-pushed the trivy-cve-scan-kafka-connect branch 3 times, most recently from cf65721 to 8856129 Compare April 27, 2026 14:44

rmoff force-pushed the trivy-cve-scan-kafka-connect branch from 8856129 to 507443e Compare April 27, 2026 14:45

rmoff marked this pull request as ready for review April 27, 2026 15:49

rymurr approved these changes Apr 28, 2026

View reviewed changes

rmoff force-pushed the trivy-cve-scan-kafka-connect branch from 507443e to 7e1308a Compare May 11, 2026 10:43

rmoff force-pushed the trivy-cve-scan-kafka-connect branch from 7e1308a to da8cc4c Compare May 11, 2026 10:53

rmoff force-pushed the trivy-cve-scan-kafka-connect branch from da8cc4c to 1239c13 Compare May 11, 2026 11:35

rmoff force-pushed the trivy-cve-scan-kafka-connect branch from 1239c13 to ba1fccc Compare May 11, 2026 13:31

rymurr approved these changes May 11, 2026

View reviewed changes

rymurr merged commit 20d5b31 into apache:main May 11, 2026
37 of 39 checks passed

rmoff mentioned this pull request May 11, 2026

Kafka Connect: Multiple HIGH netty CVEs in transitive netty dependencies #16286

Closed

kevinjqliu mentioned this pull request May 11, 2026

CI: Make CVE scan blocking on PRs, informational on main #16287

Open

rmoff mentioned this pull request May 11, 2026

Kafka Connect: CVE-2026-5598 in org.bouncycastle:bcprov-jdk18on #16288

Closed

This was referenced May 11, 2026

INFRA: Expand Trivy CVE scan to all bundle and runtime modules #16291

Merged

Build: Simplify CI workflow path filters to avoid per-workflow maintenance #16302

Draft

Conversation

rmoff commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

Uh oh!

kevinjqliu commented Feb 26, 2026

Uh oh!

rmoff commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevinjqliu commented Feb 26, 2026

Uh oh!

rymurr commented Feb 27, 2026

Uh oh!

rymurr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kevinjqliu commented Feb 27, 2026

Uh oh!

rmoff commented Mar 2, 2026

Uh oh!

rymurr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kevinjqliu left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lhotari commented Mar 23, 2026

Uh oh!

kevinjqliu commented Apr 1, 2026

Uh oh!

rmoff commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rmoff commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rymurr Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

rmoff Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

rmoff Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

rymurr commented May 11, 2026

Uh oh!

Uh oh!

kevinjqliu commented May 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rmoff commented Feb 24, 2026 •

edited

Loading

rmoff commented Feb 26, 2026 •

edited

Loading

kevinjqliu left a comment •

edited

Loading

rmoff commented Apr 7, 2026 •

edited

Loading

rmoff commented Apr 15, 2026 •

edited

Loading