Skip to content

fix: db error in ListTransactions - #1081

Merged
pawellewandowski98 merged 2 commits into
mainfrom
fix-db-error-in-list-transactions
Oct 28, 2026
Merged

pawellewandowski98 merged 2 commits into
mainfrom
fix-db-error-in-list-transactions

Conversation

@pawellewandowski98

Copy link
Copy Markdown
Collaborator

Load labels and tags by row id to fix "stack depth limit exceeded" on Postgres

What Changed

  • Added loadLabels (pkg/internal/storage/repo/transactions.go). FindTransactions uses it instead of GORM's Preload of the many2many Labels.
  • Added loadTags (pkg/internal/storage/repo/outputs.go). ListAndCountOutputs (when IncludeTags is set) and FindInputsAndOutputsWithBaskets use it instead of Preload("Tags").
  • Both helpers join the link table to bsv_labels / bsv_tags and filter on the single id column: transaction_id IN (...) or output_id IN (...).
  • IDs are bound in batches of relationBatchSize (5,000), well under the bind-parameter limits of SQLite (32,766) and Postgres (65,535).
  • Preload's soft-delete filters are kept: soft-deleted links, and links to soft-deleted labels or tags, are still left out.
  • In ListAndCountOutputs, tags are loaded after the paged query, only for the rows on that page.

Why It Was Necessary

  • On Postgres, ListTransactions failed with ERROR: stack depth limit exceeded once a user had a few thousand transactions.
  • Label and Tag are keyed by (name, user_id), so Preload loads them with a row-value list, WHERE (name, user_id) IN ((..),(..),...), one tuple per distinct label or tag. Postgres fails to evaluate that list somewhere between 5,000 and 8,000 tuples.
  • Wallets that label each transaction, or tag each output, with its own ID reach that quickly. We hit it at ~5.5k transactions on a BCS-Lite stack whose token wallet labels every action with its operation ID.
  • The same pattern sat in three read paths:
    • FindTransactions, used by ListTransactions, abort and abort-abandoned;
    • ListAndCountOutputs, used by ListOutputs with tags and by balance;
    • FindInputsAndOutputsWithBaskets, used by ListActions.

Testing Performed

  • Added TestFindTransactions_ManyDistinctLabels (repo/transactions_find_labels_test.go):
    • Setup: 9,000 transactions, each with its own label plus a shared one, one soft-deleted link, and one link to a soft-deleted label.
    • Checks that every transaction comes back with exactly its live labels.
  • Added TestOutputs_ManyDistinctTags (repo/outputs_tags_test.go):
    • Setup: one transaction with 9,000 outputs, each with its own tag plus a shared one, with the same soft-delete cases.
    • Covers both ListAndCountOutputs with tags and FindInputsAndOutputsWithBaskets.
  • Both tests fail on Postgres without the change and pass with it. They also pass on SQLite:
    • go test ./pkg/internal/storage/repo/ -run ManyDistinct
    • TEST_DB_MODE=postgres go test ./pkg/internal/storage/repo/ -run ManyDistinct

Impact / Risk

  • No API, schema or migration changes. The returned entities don't change shape.
  • Regression risk is low. Only how labels and tags are loaded changes, and the new tests cover the soft-delete cases.
  • Performance:
    • Labels and tags now take one extra query per 5,000 rows, instead of the Preload query.
    • For the page-sized results that ListOutputs and ListActions return, the cost is about the same.
    • Large result sets no longer fail.
  • Not changed:
    • The remaining Preload("Labels") calls load a single transaction, so they never build a large list.
    • The row-value (tx_id, vout) IN (...) list for no-send change outpoints stays small in practice.

@pawellewandowski98 pawellewandowski98 self-assigned this Oct 28, 2026
@github-actions github-actions Bot added bug-P3 Lowest rated bug, affects nearly none or low-impact size/L Large change (201-500 lines) labels Oct 28, 2026
dzolt-4chain
dzolt-4chain previously approved these changes Oct 28, 2026
kuba-4chain
kuba-4chain previously approved these changes Oct 28, 2026
@sonarqubecloud

Copy link
Copy Markdown

@pawellewandowski98
pawellewandowski98 enabled auto-merge (squash) October 28, 2026 13:27
@pawellewandowski98
pawellewandowski98 removed the request for review from mrz1836 October 28, 2026 13:28
@pawellewandowski98
pawellewandowski98 merged commit 90b9b2a into main Oct 28, 2026
49 checks passed
@pawellewandowski98
pawellewandowski98 deleted the fix-db-error-in-list-transactions branch October 28, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-P3 Lowest rated bug, affects nearly none or low-impact size/L Large change (201-500 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants