Skip to content

Support query filters matching list values in generic and mapped tables - #271

Open
pavelhoral wants to merge 5 commits into
WrenSecurity:mainfrom
pavelhoral:json-filter-support
Open

Support query filters matching list values in generic and mapped tables#271
pavelhoral wants to merge 5 commits into
WrenSecurity:mainfrom
pavelhoral:json-filter-support

Conversation

@pavelhoral

@pavelhoral pavelhoral commented May 24, 2026

Copy link
Copy Markdown
Member

PR adds support for query filters matching list values as defined in the QueryFilter contract (#167) and adds support for JSON based queries on mapped tables in PostgreSQL database.

  • Generic tables index each array item under two distinct property table rows. One for the exact JSON pointer (e.g. /foo/0) and the other for parent property pointer (e.g. /foo). This forced switching generic table queries from using JOINs to using subqueries with EXISTS (...) clause.
  • I did test for performance impact on the JOIN->EXISTS switch using PostgreSQL and both approaches seem to yield comparable results. I hope other DB engines will behave in a similar way.
  • Overall architecture of query rendering for mapped tables was significantly tuned (and kind of simplified). Now each column gets its own query filter visitor and there are separate methods and classes for working with JSON based columns and for other simple value columns.
  • Right now, the only database that got support for working with JSON columns in mapped tables is PostgreSQL. Nothing prevents other databases from getting the support as well. The basic JsonFieldFilterVisitor is using standard SQL/JSON functions which is AFAIK being supported by Oracle and DB2 (from the list of databases supported by Wren:IDM).

Data migration for preexisting installations:

  • Searching JSON arrays is possible only after objects stored in generic table format are updated (forcing cleanup and generation of values in the property table). There is currently no automatic migration. I am not sure it is worth the effort... we just might provide guidance for manual migration in release notes (e.g. what SQL queries can be used to unwrap existing arrays data).

This PR replaces #268.

@pavelhoral
pavelhoral force-pushed the json-filter-support branch from fb9d456 to c894474 Compare May 24, 2026 17:25
@pavelhoral
pavelhoral force-pushed the json-filter-support branch 5 times, most recently from fedba2c to 0b8d92d Compare May 25, 2026 12:21
@pavelhoral
pavelhoral force-pushed the json-filter-support branch from 0b8d92d to c37d643 Compare May 25, 2026 15:54
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
64.6% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant