Skip to content

Add transect update monitoring flow - #174

Merged
LOCEANlloydizard merged 4 commits into
echostack-org:mainfrom
LOCEANlloydizard:monitor-transect-updates
Aug 14, 2026
Merged

Add transect update monitoring flow#174
LOCEANlloydizard merged 4 commits into
echostack-org:mainfrom
LOCEANlloydizard:monitor-transect-updates

Conversation

@LOCEANlloydizard

@LOCEANlloydizard LOCEANlloydizard commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

EDIT

following the workflow discussed in #163, this PR adds event-based monitoring of transect start/end updates, and builds toward the event-driven processing described in #159.

we add:

  • a generic file/directory watcher using watchdog
  • a raw-file watcher that emits a Prefect event when matching .raw files arrive/update
  • a transect-specific watcher that emits a Prefect event when the transect CSV is updated
  • a transect_update flow triggered by this event, including:
    • ignoring incomplete/open transects
    • detection of new/updated transect segments using a snapshot of the previous CSV
    • identification of existing Sv files that overlap each updated transect time range

The Sv lookup currently uses Sv_files.csv from the existing raw2Sv workflow. This can later be replaced by the database-backed lookup as that functionality is integrated (#141)

A small companion example showing how to configure and launch the watchers is included in the corresponding echodataflow-recipes PR: echostack-org/echodataflow-recipes#5

Refs: #163, #159

@leewujung

Copy link
Copy Markdown
Member

Oh sorry @LOCEANlloydizard I turned on the auto RTD build, but until #170 is merged that would continue to fail. 😅

@LOCEANlloydizard

Copy link
Copy Markdown
Contributor Author

oh cheers! yes i saw the fails and then saw the #170.. it all make sense now! 😅

@LOCEANlloydizard

Copy link
Copy Markdown
Contributor Author

@leewujung i updated this PR to also include the generic raw-directory monitoring discussed in #159, together with the transect monitoring from #163! I think its good for review, cheers!

@LOCEANlloydizard LOCEANlloydizard moved this to In Review in Echopype 2026 Aug 14, 2026
@LOCEANlloydizard LOCEANlloydizard linked an issue Aug 14, 2026 that may be closed by this pull request
2 tasks
@LOCEANlloydizard
LOCEANlloydizard merged commit b2bae07 into echostack-org:main Aug 14, 2026
1 check failed
@github-project-automation github-project-automation Bot moved this from In Review to Done in Echopype 2026 Aug 14, 2026
@LOCEANlloydizard
LOCEANlloydizard deleted the monitor-transect-updates branch August 14, 2026 22:36
This was referenced Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Add flow to monitor transect start/end info updates

2 participants