Skip to content

(fix/refactor): Remove unnecessary calls to Notification APIs (IDETECT-4992)#1708

Merged
shantyk merged 24 commits into11.4.zfrom
dev/shanty/IDETECT-4992_notification_apis_removal
Apr 17, 2026
Merged

(fix/refactor): Remove unnecessary calls to Notification APIs (IDETECT-4992)#1708
shantyk merged 24 commits into11.4.zfrom
dev/shanty/IDETECT-4992_notification_apis_removal

Conversation

@shantyk
Copy link
Copy Markdown
Contributor

@shantyk shantyk commented Apr 2, 2026

Goal: Remove or replace expensive notification API usage in Detect (as per request from perflab) without changing existing behaviour.

Related blackbuck-common PR that must be merged and released first: blackducksoftware/blackduck-common#475. This will be a major version bump as the removal of the now unused "get latest user notification timestamp" API is a breaking change.

Summary of changes:

1. Safe removal of calls to notifications API in places where we do not need it:
► detector and signature (uses BOM status polling in all cases)
► binary (uses BOM status polling, except when server version is too old for multipart upload)
► impact analysis (currently no mechanism for checking the completion status. See HUB-25142)
► iac (never used notification based waiting, noop)

2. For remaining notification use cases, reduce API calls by simply using upload start time as the polling window start.

Previously, two API calls were made for each code location upload: one to fetch the current user and another to fetch the user’s notifications. The timestamp of the most recent notification becomes a given code location's NotificationTaskRange.startDate.
All ranges were eventually combined to derive the earliest polling start date.
This happened before Detect even checked whether waiting for results was required or whether BOM status polling was available instead.

New behavior:

Now we have codeLocationsUploadStartTime, recorded at code location upload initiation which eliminates per-upload API calls (and potential for race conditions from server delays, caching, or persistence lags, etc) and is a tighter start for the notification window than querying the most recent notification date (which will always be earlier or equal to upload start so we are certain to count only notifications from this Detect run).

If and when needed, notifications APIs are used for polling as they were before, only the starting timestamp has changed.
3. Removal of waitAtScanLevel boolean which simply represented if BD sever version exists and it is >=2023.1.1. 2023.1.x has reached end of support.
4. Docs: release note + updating wait property to indicate iac and impact analysis tools are not applicable.

Note on where we still use notification APIs:

In the case of a binary scan against a BD version < 2024.7.0, we use pre-SCASS binary scan with legacy upload mechanism (no multiparty upload possible). In this specific case, we mustWaitAtBomSummaryLevel for completion if detect.wait.for.results is set to TRUE. Though we will no longer make API calls to determine most recent notification, we will still eventually poll the notifications endpoint. In this case, we will see in the logs:

DEBUG [main] --- Using legacy binary scan upload method. This can be slow, consider upgrading to a newer version of Black Duck SCA to enable multipart uploading if possible.
INFO  [main] --- Preparing to upload binary scan file: /Users/shanty/blackduck/example-source/binaries/ifm.tar
DEBUG [main] --- Will use old notifications based waiting for the following code locations: [ifm.tar/shanty-testing/testing binary]
...
DEBUG [main] --- Will use old notifications based waiting for the following code locations: [ifm.tar/shanty-testing/testing binary]
DEBUG [main] --- Notifications after Tue Apr 14 04:47:10 MDT 2026 will be considered.
DEBUG [main] --- Expected notification count 1
DEBUG [main] --- Expected code locations:
DEBUG [main] ---   Code Location -> ifm.tar/shanty-testing/testing binary
DEBUG [main] --- 
DEBUG [main] --- At least one code location has been found, now looking for notifications.
DEBUG [main] --- There were 2 notifications found.
INFO  [main] --- All code locations have not been added to the BOM yet...
INFO  [main] --- Try #1 for task code location (elapsed: 00:00:00.000)...not done yet, waiting 60 seconds and trying again...
DEBUG [main] --- At least one code location has been found, now looking for notifications.
DEBUG [main] --- There were 3 notifications found.
INFO  [main] --- Found ifm.tar/shanty-testing/testing binary code location (1 of 1).

shantyk added 2 commits April 1, 2026 11:55
…not check their completion status via notifications or bom status. This is existing behaviour, setting wait to true was a noop.
@shantyk shantyk changed the base branch from master to 11.4.z April 2, 2026 08:10
…utNotificationTaskRange. Prevents call to notifications API for which the results were never used downstream, CodeLocationCreationData's NotificationTaskRange is null.
@shantyk shantyk force-pushed the dev/shanty/IDETECT-4992_notification_apis_removal branch from 8477775 to 9036a70 Compare April 6, 2026 20:58
@shantyk shantyk force-pushed the dev/shanty/IDETECT-4992_notification_apis_removal branch from 9036a70 to 5f4efc0 Compare April 6, 2026 23:39
@shantyk shantyk self-assigned this Apr 13, 2026
@shantyk shantyk changed the title Notification APIs: remove unnecessary calls (fix/tests/refactor): Remove calls to Notification APIs (IDETECT-4992) Apr 13, 2026
@shantyk shantyk force-pushed the dev/shanty/IDETECT-4992_notification_apis_removal branch from 4b1c584 to 784a008 Compare April 14, 2026 07:27
@shantyk shantyk changed the title (fix/tests/refactor): Remove calls to Notification APIs (IDETECT-4992) (fix/tests/refactor): Remove unnecessary calls to Notification APIs (IDETECT-4992) Apr 14, 2026
@shantyk shantyk force-pushed the dev/shanty/IDETECT-4992_notification_apis_removal branch from 76512a3 to 74c9aa5 Compare April 14, 2026 11:27
@shantyk shantyk force-pushed the dev/shanty/IDETECT-4992_notification_apis_removal branch from 32756b4 to 08a210c Compare April 14, 2026 11:41
@shantyk shantyk changed the title (fix/tests/refactor): Remove unnecessary calls to Notification APIs (IDETECT-4992) (fix/refactor): Remove unnecessary calls to Notification APIs (IDETECT-4992) Apr 14, 2026
@shantyk shantyk marked this pull request as ready for review April 14, 2026 11:56
@shantyk shantyk force-pushed the dev/shanty/IDETECT-4992_notification_apis_removal branch from 08a210c to 01329b3 Compare April 14, 2026 12:04
failedScans.add(output.getCodeLocationName());
handleNoScanStatusFile(scassScan, scanOutputLocation);
return;
}
Copy link
Copy Markdown
Contributor Author

@shantyk shantyk Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just moved to make try/catch block tighter around different exceptions expected, instead of one big umbrella try/catch.

}

public void uploadBdioEntries(BlackDuckRunData blackDuckRunData, UUID bdScanId) throws IntegrationException, IOException {
public void uploadBdioEntriesForRapidMode(BlackDuckRunData blackDuckRunData, UUID bdScanId) throws IntegrationException, IOException {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for changing these names, massive w! 👍

@shantyk shantyk merged commit c30a8d4 into 11.4.z Apr 17, 2026
@shantyk shantyk deleted the dev/shanty/IDETECT-4992_notification_apis_removal branch April 17, 2026 05:30
shantyk added a commit that referenced this pull request Apr 24, 2026
* poc for correlated scanning endpoint

* fix hardcoding and other bugs

* adjust accept header

* disable binary scans

* no version is okay in component detection

* Release 11.4.0-SIGQA1-dterry.IDETECT-4817-correlated-scans

* Using the next snapshot post release 11.4.0-SIGQA2-dterry.IDETECT-4817-correlated-scans-SNAPSHOT

* pass correlation decisions to signature scans

* properly check package manager scans

* standardize package manager checks

* Release 11.4.0-SIGQA2-dterry.IDETECT-4817-correlated-scans

* Using the next snapshot post release 11.4.0-SIGQA3-dterry.IDETECT-4817-correlated-scans-SNAPSHOT

* new server properties section

* boolean logging

* Update tomlj library to latest version

* Resolve the antlr conflict with other dependency

* Upgrade antlr version

* Upgrade antlr version

* add test

* Update NpmCliParser.java

use actualName

* Release 11.4.0-SIGQA3-dterry.IDETECT-4817-correlated-scans

* Using the next snapshot post release 11.4.0-SIGQA4-dterry.IDETECT-4817-correlated-scans-SNAPSHOT

* added release note

* Release 11.4.0-SIGQA9

* Using the next snapshot post release 11.4.0-SIGQA10-SNAPSHOT

* safety addition for stateless and finalize mime type

* Migrate Bitbake to use graphviz library

* Release 11.4.0-SIGQA10-devm.IDETECT-5058

* add server properties to status.json

* Using the next snapshot post release 11.4.0-SIGQA11-devm.IDETECT-5058-SNAPSHOT

* Update currentreleasenotes.md

* Release 11.4.0-SIGQA4-dterry.IDETECT-4817-correlated-scans

* Using the next snapshot post release 11.4.0-SIGQA5-dterry.IDETECT-4817-correlated-scans-SNAPSHOT

* Update commonproblems.md

* Update DetectProperties.java

* Update currentreleasenotes.md

remove this change imported from another PR (Will get covered under that PR)

* Update commonproblems.md

* code review updates

* further code review improvements

* fix output to show only actual values

* Update commonproblems.md

fix typo

* Release 11.4.0-SIGQA10

* Using the next snapshot post release 11.4.0-SIGQA11-SNAPSHOT

* Release 11.4.0-SIGQA11-devm.IDETECT-5058

* Release 11.4.0-SIGQA12-devm.IDETECT-5058

* Release 11.4.0-SIGQA11-devm.IDETECT-5058

* Using the next snapshot post release 11.4.0-SIGQA12-devm.IDETECT-5058-SNAPSHOT

* Check if correlation scanning was enabled for fallback scenario

* Add doc changes for previous tickets

* Address review comment

* Address review comment

* feat(bazel): faster discovery path for HTTP-archive family repositories in Bzlmod projects (Bazel 7.1+) (#1709)

* Update currentreleasenotes.md

* Update currentreleasenotes.md

* Update commonproblems.md

* Release 11.4.0-SIGQA12

* Using the next snapshot post release 11.4.0-SIGQA13-SNAPSHOT

* Add support for .slnx files in NuGet Solution Native Inspector (#1688)

* Add support for .slnx files in NuGet Solution Native Inspector

* Release notes

* WIP chnages

* Remove comments and whitespace in diff

* rev bouncy castle and plexutil libs

* (fix/refactor): Remove unnecessary calls to Notification APIs (IDETECT-4992) (#1708)

* Notification APIs: remove unnecessary calls

* Don't wait at BOM level for IAC or impact analysis scans since we cannot check their completion status via notifications or bom status. This is existing behaviour, setting wait to true was a noop.

* Refactor Impact Analysis upload operation to createCodeLocationsWithoutNotificationTaskRange. Prevents call to notifications API for which the results were never used downstream, CodeLocationCreationData's NotificationTaskRange is null.

* Latest IntelligentModeStepRunner changes

* Pkg mngr and signature updates

* Testing for regressions

* Remove unused impact analysis related methods

* Remove unused ImpactAnalysisToolResult class

* Deprecate shouldWaitAtScanLevel since it simply checks if a server version exists and is at least 2023.1.1 which has reached end of service.

* Update bd-common version with soon to be released lib version

* Propogate removal of shouldWaitAtScanLevel to signature scanner

* Remove prescass pkg mngr BDIO code location upload from being considered a waitable (via notifications).

* Minor updates

* Rename back to uploadBdioFiles

* Clean up comments

* Revert changes to waitable signature scanner code loctaion

* Remove comments in signaturescansteprunner

* Add missed method update for binary upload

* Remove NotificationTaskRange param from WaitableCodeLocationData constructor as it is no longer used.

* Fix whitespace in diff

* Fix method rename issue after rebase

* Fix regression wrt correlated scanning. Correlated scan count calculations are tightly coupled with the legacy idea of a waitable code location.

* Bump bd-common version 68.0.0

* Release 11.4.0-SIGQA13

* Using the next snapshot post release 11.4.0-SIGQA14-SNAPSHOT

* Release 11.4.0-SIGQA14

* Using the next snapshot post release 11.4.0-SIGQA15-SNAPSHOT

* Release 11.5.0-SIGQA2-shanty.merge_11.4.z_to_main

* Using the next snapshot post release 11.5.0-SIGQA3-shanty.merge_11.4.z_to_main-SNAPSHOT

---------

Co-authored-by: dterrybd <dterry@blackduck.com>
Co-authored-by: blackduck-serv-builder <serv-builder@blackducksoftware.com>
Co-authored-by: dterrybd <103214400+dterrybd@users.noreply.github.com>
Co-authored-by: devmehtabd <devmehta@blackduck.com>
Co-authored-by: cpottsbd <36172712+cpottsbd@users.noreply.github.com>
Co-authored-by: Dev Mehta <128397570+devmehtabd@users.noreply.github.com>
Co-authored-by: Samrat Mukherjee <samratmuk@blackduck.com>
shantyk added a commit that referenced this pull request May 7, 2026
* poc for correlated scanning endpoint

* fix hardcoding and other bugs

* adjust accept header

* disable binary scans

* no version is okay in component detection

* Release 11.4.0-SIGQA1-dterry.IDETECT-4817-correlated-scans

* Using the next snapshot post release 11.4.0-SIGQA2-dterry.IDETECT-4817-correlated-scans-SNAPSHOT

* pass correlation decisions to signature scans

* properly check package manager scans

* standardize package manager checks

* Release 11.4.0-SIGQA2-dterry.IDETECT-4817-correlated-scans

* Using the next snapshot post release 11.4.0-SIGQA3-dterry.IDETECT-4817-correlated-scans-SNAPSHOT

* new server properties section

* boolean logging

* Update tomlj library to latest version

* Resolve the antlr conflict with other dependency

* Upgrade antlr version

* Upgrade antlr version

* add test

* Update NpmCliParser.java

use actualName

* Release 11.4.0-SIGQA3-dterry.IDETECT-4817-correlated-scans

* Using the next snapshot post release 11.4.0-SIGQA4-dterry.IDETECT-4817-correlated-scans-SNAPSHOT

* added release note

* Release 11.4.0-SIGQA9

* Using the next snapshot post release 11.4.0-SIGQA10-SNAPSHOT

* safety addition for stateless and finalize mime type

* Migrate Bitbake to use graphviz library

* Release 11.4.0-SIGQA10-devm.IDETECT-5058

* add server properties to status.json

* Using the next snapshot post release 11.4.0-SIGQA11-devm.IDETECT-5058-SNAPSHOT

* Update currentreleasenotes.md

* Release 11.4.0-SIGQA4-dterry.IDETECT-4817-correlated-scans

* Using the next snapshot post release 11.4.0-SIGQA5-dterry.IDETECT-4817-correlated-scans-SNAPSHOT

* Update commonproblems.md

* Update DetectProperties.java

* Update currentreleasenotes.md

remove this change imported from another PR (Will get covered under that PR)

* Update commonproblems.md

* code review updates

* further code review improvements

* fix output to show only actual values

* Update commonproblems.md

fix typo

* Release 11.4.0-SIGQA10

* Using the next snapshot post release 11.4.0-SIGQA11-SNAPSHOT

* Release 11.4.0-SIGQA11-devm.IDETECT-5058

* Release 11.4.0-SIGQA12-devm.IDETECT-5058

* Release 11.4.0-SIGQA11-devm.IDETECT-5058

* Using the next snapshot post release 11.4.0-SIGQA12-devm.IDETECT-5058-SNAPSHOT

* update docs to describe correlated server property

* Check if correlation scanning was enabled for fallback scenario

* Add doc changes for previous tickets

* Address review comment

* Address review comment

* feat(bazel): faster discovery path for HTTP-archive family repositories in Bzlmod projects (Bazel 7.1+) (#1709)

* Update currentreleasenotes.md

* Update currentreleasenotes.md

* Update commonproblems.md

* Release 11.4.0-SIGQA12

* Using the next snapshot post release 11.4.0-SIGQA13-SNAPSHOT

* Add support for .slnx files in NuGet Solution Native Inspector (#1688)

* Add support for .slnx files in NuGet Solution Native Inspector

* Release notes

* WIP chnages

* Remove comments and whitespace in diff

* rev bouncy castle and plexutil libs

* (fix/refactor): Remove unnecessary calls to Notification APIs (IDETECT-4992) (#1708)

* Notification APIs: remove unnecessary calls

* Don't wait at BOM level for IAC or impact analysis scans since we cannot check their completion status via notifications or bom status. This is existing behaviour, setting wait to true was a noop.

* Refactor Impact Analysis upload operation to createCodeLocationsWithoutNotificationTaskRange. Prevents call to notifications API for which the results were never used downstream, CodeLocationCreationData's NotificationTaskRange is null.

* Latest IntelligentModeStepRunner changes

* Pkg mngr and signature updates

* Testing for regressions

* Remove unused impact analysis related methods

* Remove unused ImpactAnalysisToolResult class

* Deprecate shouldWaitAtScanLevel since it simply checks if a server version exists and is at least 2023.1.1 which has reached end of service.

* Update bd-common version with soon to be released lib version

* Propogate removal of shouldWaitAtScanLevel to signature scanner

* Remove prescass pkg mngr BDIO code location upload from being considered a waitable (via notifications).

* Minor updates

* Rename back to uploadBdioFiles

* Clean up comments

* Revert changes to waitable signature scanner code loctaion

* Remove comments in signaturescansteprunner

* Add missed method update for binary upload

* Remove NotificationTaskRange param from WaitableCodeLocationData constructor as it is no longer used.

* Fix whitespace in diff

* Fix method rename issue after rebase

* Fix regression wrt correlated scanning. Correlated scan count calculations are tightly coupled with the legacy idea of a waitable code location.

* Bump bd-common version 68.0.0

* Release 11.4.0-SIGQA13

* Using the next snapshot post release 11.4.0-SIGQA14-SNAPSHOT

* Release 11.4.0-SIGQA14

* Using the next snapshot post release 11.4.0-SIGQA15-SNAPSHOT

* rework release note

* fix(bazel) - Change info logs to debug logs  (#1734)

* Release 11.4.0-SIGQA15

* Using the next snapshot post release 11.4.0-SIGQA16-SNAPSHOT

* Support for customizing the output directory for Quack Patch (#1713)

* Support for customizing quack patch output directory

* updated release note for the new flag

* Release 11.4.0-SIGQA10-IDETECT-5066

* Using the next snapshot post release 11.4.0-SIGQA11-IDETECT-5066-SNAPSHOT

* fix

* updated docs

* Release 11.4.0-SIGQA14-IDETECT-5066

* Using the next snapshot post release 11.4.0-SIGQA15-IDETECT-5066-SNAPSHOT

* fix invalid path case

* Release 11.4.0-SIGQA15-IDETECT-5066

* Using the next snapshot post release 11.4.0-SIGQA16-IDETECT-5066-SNAPSHOT

* updated release note as suggested

---------

Co-authored-by: blackduck-serv-builder <serv-builder@blackducksoftware.com>

* Release 11.4.0-SIGQA16

* Using the next snapshot post release 11.4.0-SIGQA17-SNAPSHOT

* Warn Java 8 End of Support in logs (#1737)

* Update currentreleasenotes.md

Updates and omissions

* fix compatibility error with java 8 (#1742)

* Release 11.4.0-SIGQA17

* Using the next snapshot post release 11.4.0-SIGQA18-SNAPSHOT

* Remove logic to not create project version in case of empty detector bdio (#1747)

* Release 11.4.0-SIGQA18

* Using the next snapshot post release 11.4.0-SIGQA19-SNAPSHOT

* Increase signatur escan code location count by the number of successful code location paths (#1748)

* Release 11.4.0-SIGQA19

* Using the next snapshot post release 11.4.0-SIGQA20-SNAPSHOT

* Send quack patch custom output path to nuget inspector invocation (#1749)

* Release 11.4.0-SIGQA20

* Using the next snapshot post release 11.4.0-SIGQA21-SNAPSHOT

* Release 11.4.0

* Using the next snapshot post release 11.4.1-SNAPSHOT

* Fix release notes for 11.4.1 release (#1750)

* Update topics.ditamap (#1751)

* Release 11.4.1

* Using the next snapshot post release 11.4.2-SNAPSHOT

* Empty current release notes for 11.5.0

---------

Co-authored-by: dterrybd <dterry@blackduck.com>
Co-authored-by: blackduck-serv-builder <serv-builder@blackducksoftware.com>
Co-authored-by: dterrybd <103214400+dterrybd@users.noreply.github.com>
Co-authored-by: devmehtabd <devmehta@blackduck.com>
Co-authored-by: cpottsbd <36172712+cpottsbd@users.noreply.github.com>
Co-authored-by: Dev Mehta <128397570+devmehtabd@users.noreply.github.com>
Co-authored-by: Samrat Mukherjee <samratmuk@blackduck.com>
Co-authored-by: Aditya Byreddy <37792774+sig-abyreddy@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants