Skip to content

refactor: convert the analyst1 app to the splunk soar sdk - #3

Merged
athreya-splunk merged 32 commits into
splunk-soar-connectors:mainfrom
analyst-one:refactor/sdk-conversion
Aug 3, 2026
Merged

athreya-splunk merged 32 commits into
splunk-soar-connectors:mainfrom
analyst-one:refactor/sdk-conversion

Conversation

@mvforgione

@mvforgione mvforgione commented Jul 10, 2026 •

Copy link
Copy Markdown
Contributor

Features

  • Convert the app from the classic BaseConnector format to the Splunk SOAR SDK (splunk-soar-sdk 3.25.3). The existing appid is preserved, so version 1.3.0 installs as an upgrade over the classic 1.2.1 app. Requires Splunk SOAR 7.0 or later.
  • All 13 classic actions are reimplemented with behavior preserved: test connectivity, lookup domain, lookup email, lookup hash, lookup string, lookup ip, lookup ipv6, lookup url, lookup mutex, lookup http request, upload evidence file, check evidence status, get evidence.
  • 8 new actions for parity with the Analyst1 Cortex XSOAR integration: batch check, get indicator by id, get actor by id, get malware by id, get sensors, get sensor taskings, get sensor config, get sensor diff.
  • Custom result views for every action: table views for list results (batch check, evidence, sensors, taskings, diff) and detail views for single entities (indicators, actors, malware, sensor config, evidence upload and status). Entity IDs link back to the configured Analyst1 server (indicators, actors, malware, sensors, evidence files).
  • Unit test suite (145 tests) covering actions, output models, view rendering, HTML escaping, and the manifest.

Manual Documentation

  • I have verified that manual documentation has been updated where appropriate

manual_readme_content.md documents both authentication methods (basic auth for the 1_0 API and OAuth2 client credentials for the 1_1 API).

Other information

  • Local action runs are supported via the SDK CLI (python src/app.py ...), and the package builds with soarapps package build.
  • Release notes for this version are in release_notes/unreleased.md.
  • The app id static checks (app_package_name and valid_app_name_and_guid) depend on the registry mappings in Add Analyst1 app id to registry mappings phantomcyber/dev-cicd-tools#233, which registers this app id. Those two checks will fail on this PR until that one is merged.

mvforgione added 29 commits July 7, 2026 18:21
Port the SDK app from the old repo's feature/sdk-conversion branch
(sdk_app/, commit 1045422) into the fork using the standard SDK
connector layout (src/, templates/, pyproject.toml, uv.lock).

- remove classic scaffold (analyst1_connector.py, analyst1.json,
  analyst1_consts.py, root __init__.py)
- rename logos to logo_analyst1.svg / logo_analyst1_dark.svg
- set manifest identity: name Analyst1, publisher Analyst1,
  appid preserved (8ee822f1-a285-44f4-b8e7-303245811a32),
  app_version 2.0.0, min_phantom_version 7.0.0, package_name
  phantom_analyst1, license Copyright (c) Analyst1
- bump splunk-soar-sdk 3.2.3 -> >=3.25.3 with mechanical API fixes:
  asset.auth_state replaces actions_manager.auth_state and
  vault.get_attachment replaces vault.get_vault_file_info
drop call-publish-template.yml, call-push-template.yml and
setup-from-template.yml so the workflow set matches current sdk-format
connectors (virustotalv3, splunk), which carry only call-publish.yml
set target-version to py313 to match requires-python (mirrors the
splunk sdk connector); delete src/views.py, which nothing imports —
app.py registers its own view handler
output of splunk's dev-cicd-tools and lint hooks: copyright headers and
year bump, generated readme docs and notice file, pyupgrade rewrites for
py313, djlint template formatting, and an unreleased release note for
the sdk conversion
- model the classic nested action_result.data.* datapath contract for the
  nine lookup actions as typed ActionOutput sub-models, replacing the
  raw_data json-blob flattening
- preserve classic runtime decorations: base_url, actor links, and friendly
  enrichment result names; keep enrichmentResults result as a json string
  (documented delta, the sdk cannot type an arbitrary object)
- restore classic summary keys: lookup summary.id, upload summary.uuid,
  evidence-status summary.message/evidence_id, and get-evidence paging keys
- restore the classic not-found lookup shape: no data records, empty summary
- surface get_evidence's evidence type filter as the classic type parameter
  and restore classic page semantics (absent means multi-page, must be >= 1)
- make upload_evidence_file's source_id required again and skip falsy form
  fields like classic
- emit get_evidence records as untyped passthrough rows like classic
- render the custom view from the typed models instead of re-parsed raw_data
…rity

- default a missing enrichment result type to an empty friendly name like
  classic instead of raising KeyError
- default the sort keys the display template uses (name on originatingIps,
  subjects and fileNames elements; value on ports elements) so Jinja's sort
  filter cannot raise on elements missing the key
- fail the lookup ip action on an invalid IP address like classic instead of
  silently proceeding as type ip
check a batch of indicator values against GET /batchCheck (type
auto-detected), parsing the {"results": [...]} envelope. enforce the
6000-char url-length cap on the normalized csv input. extract the 1_1
links object->list normalization into a module-level _links_to_list and
add the shared EnumDto/AkaDto output models.
fetch indicators, actors, and malware families by their analyst1
numeric id. indicator-by-id strips hash-id type suffixes (14131-md5)
and reuses the lookup output contract, custom view, and runtime
decorations via an extracted _decorate_indicator helper. actor and
malware map the spec resources with {id, name, classification} triples
for country/sponsor/primaryMotivation and category/stage.
add get sensors (paginated like get evidence, with the live _links ->
links row rename), get sensor taskings and get sensor diff (extended
200s timeout for parity with the xsoar integration), and get sensor
config (raw text config delivered to the vault). a sensor with no
finalized config version answers the diff with http 500 "no finalized
versions found", now recognized via a structured status attribute on
Analyst1APIError and treated as an empty result rather than a failure.
live verification against the partner instance shows both the 1_1 (oauth)
and 1_0 (basic) apis serialize links as a rel-keyed object on
/indicator/match, /indicator/{id}, and /actor/{id}; the list-of-{rel,href}
form exists only in the classic manifest contract and xsoar's fixtures.
the coercion to list form is unchanged and still required to make the
declared links.* datapaths resolve.
…tors entry

the contributors injection rides on the sdk's app_meta_info setattr
propagation (no first-class mechanism in sdk 3.25.3); add a round-trip
test so an sdk upgrade that drops the path fails loudly.
svg logos serve correctly but the soar 8.5 app header falls back to the
default splunk logo instead of rendering them; the classic app and other
working sdk apps on the instance ship png. restore the classic
analyst1.png / analyst1_dark.png assets and point the app metadata and the
lookup view at them.
the analyst1 auth tier answers bad basic-auth credentials with a
text/html tomcat 401 page; surface its readable text in the action
error message instead of the raw html blob, restoring classic 1.2.1
_process_html_response behavior (minus its brace-escaping, which only
guarded classic phantom's format pipeline). applies to both
_make_request and get_sensor_config_text; non-html bodies still pass
through verbatim. declares beautifulsoup4 as a direct dependency
(already bundled as a transitive of splunk-soar-sdk).
…metadata

replace splunk inc. with analyst1 in the apache 2.0 headers of all
source, test, and template files, the license appendix boilerplate,
and the pyproject license field. per splunk's LEGAL.md, contributors
retain copyright on contributions while licensing them under apache
2.0. normalize the license string to years-then-name so the copyright
hook maintains it. regenerate NOTICE so its header picks up the new
license string. leave the generated readme footer and the
splunk-soar-sdk attribution block in NOTICE, which correctly credit
splunk.
add custom contains (analyst1 indicator id, analyst1 actor id,
analyst1 malware id, analyst1 sensor id) to the new actions' id
parameters and output datapaths so the soar ui offers contextual
pivots between them, following the evidence upload key precedent.
batch check values gets broad indicator contains (ip, domain, url,
hash, email) and lookup ipv6 gains ipv6; id-bearing sub-models
subclass the shared ClassifiedIdName so campaigns, targets, cves,
akas and other plain references stay untagged. classic lookup
parameters are unchanged (verified manifest-identical). manifest
guard tests pin the contains contract.
marketing-supplied 3:1 svg lockups (light and dark theme variants)
replace the interim classic pngs, whose 7.4:1 strip rendered compressed
in the playbook picker and whose png format failed splunk's svg-only
logo check. the svgs pass the light_and_dark_theme_logos static test.
every dark-theme logo across splunk-soar-connectors is monochrome white
(surveyed 16 incl. slack, fortinet, sentinelone, check point). recolor
the gold fills white and drop the black shield underlay so the falcon
renders as a transparent knockout.
the contributor wire names (contributorOrg, iso country/region codes,
and the misspelled contibutorConsent) were defined in both the classic
and sdk field maps but never exposed as action parameters, so no
version of the app ever sent them; the api spec documents no such
fields on the evidence upload endpoint.
per analyst1 product direction, 2.x is reserved for the upcoming
analyst1 v2 api; the sdk conversion ships as 1.3.0. release notes now
state the splunk soar 7.0 minimum explicitly since the version number
alone no longer signals the platform floor.
replaces the letterboxed dark variant with marketing's full-bleed
1560x204 export so the artwork renders larger in the soar ui.
the openapi spec describes http 204 from /evidence/uploadStatus/{uuid} as
"an error occurred processing the evidence. no evidence record available."
previously the empty 204 body collapsed to the same null message/id as a
200 with a null id (ingest still processing), so a permanently failed
ingest looked like it was processing forever. the client now surfaces the
status code for this endpoint and the action reports "evidence processing
failed (no evidence record available)" on 204. the 204 semantics are
spec-only (not yet observed live); confirmation requested from analyst1
engineering. 200 and 404 behavior is unchanged.
get_evidence_upload_status bypasses _make_request to expose the http
status code, but dropped _make_request's refresh-and-retry-once on a
server-rejected oauth token. check evidence status is a polled endpoint,
so playbook loops routinely cross token-expiry windows; without the
retry, a stale token failed the action until the local expiry passed.
the method now clears the cached token, forces a refresh, and retries
exactly once, matching _make_request.
- fix widgets-pane blank render: template license headers become jinja
  comments so the widget div is the first parsed node of the response
  (the pane injects via $(data)[0] and silently drops anything else)
- fix stored xss: remove |safe on api-sourced description and enrichment
  text; preserve line breaks with white-space: pre-line instead
- fix widget logo: stop overriding the platform-supplied themed
  title_logo paths (404 + missing dark logo) and drop title text that
  painted over the centered logo
- redesign the indicator view: verdict badges, sectioned layout, shared
  macros and styles (a1_macros.html), theme-safe colors, noopener links
- add contextual views for the ten actions that rendered blank default
  tables: batch check, actor/malware by id, evidence upload/status/list,
  sensors, sensor taskings/config/diff (tabular views for lists, detail
  views for single entities)
- deep-link entities to the asset's analyst1 portal: indicators, actors,
  malware, sensors (/sensors/{id}), evidence (/files/{id}); link bases
  derive from api self-links or the asset base_url injected at action
  time, never hardcoded, with an http(s) scheme guard
- unwrap live classified {name/date} objects in evidence rows and show
  the reported date column
- keep monospace blocks theme-colored (the platform styles bare pre
  near-black on both themes)
- add a view rendering test suite covering the injection contract,
  escaping regressions, empty states, and portal links
@github-actions

Copy link
Copy Markdown

Thank you for your submission! We will take a look as soon as we can.

@mvforgione

Copy link
Copy Markdown
Contributor Author

Following up now that phantomcyber/dev-cicd-tools#233 has merged: pre-commit is passing. Two notes on the remaining check failures, which both appear to be on the CI side rather than in the app:

  • compile: the app builds successfully, but the job could not reach the internal test instance 10.1.64.115 (connection refused on all attempts). The previous run on July 10 installed this same code successfully on all three test instances, including that one.
  • semantic-release-preview: the dry run attempts a push to the PR head repository, which is our fork, so the GitHub Actions token is denied. It looks like this new check may not currently be able to run on fork PRs.

Happy to retrigger or adjust anything on our side if needed.

@athreya-splunk

Copy link
Copy Markdown
Contributor

Hi @mvforgione - I had a quick look at the Pr and I would like to highlight following issues that we might want to resolve before we can merge the PR. Here's the generated summary -

1. Validate path parameters before building URLs

     User input is inserted directly into API paths:
      - src/app.py:381 uses clean_id in /indicator/{clean_id}
      - src/app.py:489 uses uuid in /evidence/uploadStatus/{uuid}

     Please validate these values or URL-encode them with quote(value, safe="") before using them in the path. 

  2. Reject blank lookup values before calling the API

     Several lookup params default to empty strings:
      - src/app.py:1060-1134
      - Used by _do_indicator_lookup at src/app.py:1177

     Please strip and reject blank values locally with ActionFailure before making the Analyst1 API request.

  3. Add OAuth 401 retry for sensor config

     get_sensor_config_text makes a direct request for raw text:
      - src/app.py:423-436

     But it does not retry on OAuth 401 like get_evidence_upload_status does:
      - src/app.py:502-506

     Please add the same clear-token, refresh-token, retry-once behavior here.
     

address first review round on the upstream pr:

- url-encode user-supplied indicator id and evidence uuid before
  inserting them into api paths
- strip lookup values and fail locally on blank input before any
  api call, matching the classic app's required-value semantics
- give get_sensor_config_text the same oauth 401 clear-refresh-retry
  behavior as evidence upload status polling
@mvforgione

Copy link
Copy Markdown
Contributor Author

Thanks for the review @athreya-splunk. All three items are addressed in 46b5d93:

  1. Path parameters: the indicator id and evidence uuid are now URL-encoded with quote(value, safe="") before being inserted into the request path.
  2. Blank lookups: lookup values are stripped, and blank input now fails locally with ActionFailure before any API call. This also matches the classic app, which marked these parameters as required.
  3. Sensor config auth: get_sensor_config_text now has the same clear-token, refresh, and retry-once behavior on 401 as the evidence upload status call.

Each fix has test coverage (encoded-path assertions, blank rejection with no outbound request, and single-retry behavior on 401). Happy to adjust anything further.

One note on the CI run for this push: every job now fails at the checkout step with "Refusing to check out fork pull request code from a 'pull_request_target' workflow", which looks like a recent actions/checkout default change affecting fork PRs org-wide rather than anything in this branch. The previous run on July 20 checked out and ran normally.

existing assets must trust the api server certificate chain or
explicitly disable verification, matching the platform security
baseline for connector asset configuration
@athreya-splunk

Copy link
Copy Markdown
Contributor

@mvforgione thanks for fixing. Could you also please take a look at this PR and add them to this sdkfied code?
#4
It contains minor patches based on internal VULN tickets.
Everything else looks good to me otherwise.

@mvforgione

Copy link
Copy Markdown
Contributor Author

Thanks @athreya-splunk. Both security items from #4 are covered on this branch:

  1. Certificate verification defaults to enabled as of 500e53f (pushed yesterday), with the same guidance in the release notes: existing assets must trust the API server certificate chain or explicitly disable verification.
  2. The credential fields (client secret and password) are already password fields in the SDK asset configuration, as they were in the published 1.2.1 app, so no rotation or re-entry is needed on upgrade.

The LICENSE, NOTICE, and dependency metadata changes in #4 are specific to the classic format; the SDK build generates those artifacts.

One sequencing note: #4 and this PR modify the same files, so whichever merges second will need a rebase. If it is simplest, this PR can carry the security changes and #4 can be closed, but happy to rebase either way.

@athreya-splunk
athreya-splunk merged commit 202b40c into splunk-soar-connectors:main Aug 3, 2026
13 of 14 checks passed
@splunk-soar-semantic-release

Copy link
Copy Markdown

🎉 This PR is included in version 1.0.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@athreya-splunk

Copy link
Copy Markdown
Contributor

I will create an internal request to release this app in splunkbase. I'll update here once it's released.

@mvforgione

Copy link
Copy Markdown
Contributor Author

@athreya-splunk one flag before the Splunkbase release request goes out: this release was tagged 1.0.0, but Splunkbase already has Analyst1 at 1.2.1 under this app id, released before the app moved to GitHub. A 1.0.0 release would register as a downgrade and break the upgrade path for existing installs. The PR set the version to 1.3.0 for that reason; semantic-release recomputed it to 1.0.0 because the repo has no baseline tag from the pre-GitHub releases.

Could the release go out as 1.3.0? One option is to baseline the repo with a 1.2.1 tag on the pre-merge history and re-run the release so the commits compute to 1.3.0. Happy to help however is useful.

@athreya-splunk

Copy link
Copy Markdown
Contributor

@athreya-splunk one flag before the Splunkbase release request goes out: this release was tagged 1.0.0, but Splunkbase already has Analyst1 at 1.2.1 under this app id, released before the app moved to GitHub. A 1.0.0 release would register as a downgrade and break the upgrade path for existing installs. The PR set the version to 1.3.0 for that reason; semantic-release recomputed it to 1.0.0 because the repo has no baseline tag from the pre-GitHub releases.

Could the release go out as 1.3.0? One option is to baseline the repo with a 1.2.1 tag on the pre-merge history and re-run the release so the commits compute to 1.3.0. Happy to help however is useful.

@mvforgione - I thought this was the first release of the app. Missed the description. I didn't see any past releases of this app. Could you please share splunkbase link so that I can take a look? I only see a splunk app https://splunkbase.splunk.com/app/6816 and not SOAR integration app.

@mvforgione

Copy link
Copy Markdown
Contributor Author

You are right, and I should correct my earlier comment. This connector was never published on Splunkbase, so there is no link to share. App 6816 is a different product, our add-on for Splunk Enterprise (TA-Analyst1). Analyst1 distributed the SOAR connector directly to customers, and 1.2.1 is the version they are running today.

The 1.3.0 request is about those existing installs rather than a Splunkbase listing. Customer SOAR instances have the classic app installed under this same app id, which we kept so the SDK app upgrades those installs in place. Publishing as 1.0.0 would appear as a downgrade to them. The generated release notes for this release also state that version 1.3.0 replaces the classic Analyst1 app (v1.2.1), so the 1.0.0 tag does not match its own notes.

Would it be possible to release as 1.3.0? One approach is to tag the pre-merge commit as 1.2.1 so semantic-release computes 1.3.0 from the merged commits. Happy to help however is useful.

@athreya-splunk

athreya-splunk commented Aug 4, 2026 •

Copy link
Copy Markdown
Contributor

I’ll need to check internally whether we can publish the app as v1.3.0.
Given that Splunkbase already has an Analyst1 app, I don’t think we’ll be able to publish another app with the exact same name. We may need to rename this listing to something like “Analyst1 for Splunk SOAR” and release it as a new Splunkbase app. Would that work for you if the release version is v1.0.0 for the new app with the new name?
Since you mentioned that customers already have a directly distributed connector installed with this same app ID, we would also need to update the app UUID along with the app name. The customers can then move to this new app and slowly move away from the custom app that was shared earlier.

@mvforgione

mvforgione commented Aug 4, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks, the rename works for us. "Analyst1 for Splunk SOAR" would follow the convention other connectors use when the vendor already has a Splunk Enterprise app, for example GreyNoise for SOAR, TrackMe for Splunk SOAR, Dataminr Pulse for Splunk SOAR, and Censys for Splunk SOAR. Happy to use that, or a different name if you prefer one.

On the app UUID, could you confirm the upgrade behavior for us? Our understanding is that if the UUID stays the same, a customer running the classic app gets an in-place upgrade with their assets and playbooks intact, and that if the UUID changes they get a parallel install instead, leaving the old app on the instance and requiring assets to be recreated with their credentials and playbooks repointed. If that is right, keeping the UUID would spare existing customers that migration, and a rename alone should be enough to resolve the listing name.

For what it is worth, Recorded Future appears to have taken that route: the connector was renamed from "Recorded Future" to "Recorded Future For Splunk SOAR" in 4.1.0 and kept appid 6efe0e1b-76ac-4ffd-8fa0-ac58fd6efd77 from its earlier versions.

The version request follows from the same answer. If the UUID stays, the published version would need to be above 1.2.1 so existing installs see an upgrade. If the UUID has to change, then 1.0.0 is correct and I will drop the request.

How would you like the rename delivered? I can open a follow-up PR against main is that is easiest.

@athreya-splunk

Copy link
Copy Markdown
Contributor

@mvforgione -

I think there are two separate questions here.

Keeping the same UUID while renaming the app is one decision, and Recorded Future is a useful precedent for that. It was renamed to a SOAR-specific app name while retaining the same app UUID.

The version request is different. Our connector release flow is automated through semantic-release: the workflow computes the next version from the repository’s release/tag history and then updates the app metadata during publish. Since the earlier Analyst1 builds were distributed outside this GitHub/Splunkbase release flow, I’d prefer not to manually add release artifacts, tags, or metadata only to make the first public release come out as 1.0.3.

Given that this is a conversion to the SOAR SDK,if we keep the same UUID and treat this as the next version of the existing customer-distributed connector, then this should be a major version release, e.g. 2.0.0.

My preference is to treat this as a clean new Splunkbase SOAR app: update the app UUID and app name in phantomcyber/dev-cicd-tools, splunk-soar-connectors/.github, and this repo, then let the automated release flow publish the first public version from that new app lineage. Existing customers would need to create a new asset and repoint playbooks, but I think that is cleaner long term and avoids mixing private distribution history with the public Splunkbase release history.

@mvforgione

Copy link
Copy Markdown
Contributor Author

Thanks for laying out both options. Would it be possible to keep the same UUID and release as 2.0.0, the version you indicated for that path?

Our concern with the new UUID is the existing installs. Every customer running the current connector would have to install the new app alongside the old one, recreate each asset and re-enter its credentials, and repoint each Analyst1 action block in their visual playbooks. Keeping the UUID would make it an ordinary upgrade, with assets and playbooks left in place.

On the release-history point, this would not require backfilling anything from the private distribution. No tags, artifacts, or metadata behind the existing 1.0.0 tag. The follow-up PR would just need to carry a breaking change commit so the flow computes 2.0.0 from that tag, which seems reasonable given the SDK conversion and the move to a 7.0 minimum.

Happy to open the follow-up PR against main with the rename whenever you decide. I think the appid_to_name entry in dev-cicd-tools would need the new name as well, since the app name check compares against it, and I can open that PR too if it helps.

@athreya-splunk

Copy link
Copy Markdown
Contributor

Hey @mvforgione - could you please create a PR renaming the app to Analyst1 for Splunk SOAR? We need 2 PRs, phantomcyber/dev-cicd-tools, splunk-soar-connectors/.github

I've created a PR to bump version to 2.0.0 #5. I'll merge once we rename the app.

@mvforgione

Copy link
Copy Markdown
Contributor Author

Both registry PRs are open, and the rename is #6.

phantomcyber/dev-cicd-tools#256 renames the entry, and splunk-soar-connectors/.github#620 adds it. That file was missing this app id, so it is an addition there rather than an edit. appid_to_package_name.json needs no change, since the package name comes from the pyproject project name and stays phantom_analyst1.

#6 will show a red static-tests until #256 merges, because check_valid_app_name_and_guid reads the registry from dev-cicd-tools main at run time. A re-run after #256 merges should go green without a rebase or a new commit.

One question on sequencing. Should the rename commit carry the breaking change marker so it publishes 2.0.0 on merge, or stay non-releasing so #5 does the bump? I ask because the build and enqueue-publish jobs do not look gated on semantic-release producing a new version, and I was not sure whether a non-releasing merge would re-enqueue 1.0.0. It is a chore: commit right now and I can amend it either way.

#6 also touches release_notes/unreleased.md, since the release notes hook requires content there, so it will conflict with #5. Glad to drop my line and keep yours.

@athreya-splunk

Copy link
Copy Markdown
Contributor

@mvforgione - https://splunkbase.splunk.com/app/9495 app released to splunkbase with the available versions starting from 2.0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants