Skip to content

Consolidated DeltaSolver and KeyValueStoreSolver into DefaultSolver #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tombonfert wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Consolidated DeltaSolver and KeyValueStoreSolver into DefaultSolver #46

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
02fa529
Consolidated DeltaSolver and KeyValueStoreSolver into DefaultSolver
tombonfert Jun 12, 2026
2f6a8db
updated comment
tombonfert Jun 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 27 additions & 34 deletions docs/impulse/docs/config/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Pydantic models. The canonical schema lives in
"table_prefix": "my_report"
},
"query_engine": {
"solver": "DeltaSolver",
"solver": "DefaultSolver",
"data_type": "RAW"
},
"container_filters": {
Expand Down Expand Up @@ -61,11 +61,12 @@ Maps the silver-layer input tables.
| `channels_uri` | `str` | Yes | Full Unity Catalog path. Time-series sample data. |
| `container_tags_table` | `str` | No | Full Unity Catalog path. Container EAV tags. |
| `channel_tags_table` | `str` | No | Full Unity Catalog path. Channel EAV tags. |
| `channel_mapping_table` | `str` | No | Full Unity Catalog path. Logical-to-physical channel alias table. Required when using `QueryBuilder.channel_with_alias()` (currently supported by `KeyValueStoreSolver`). In reporting mode the resolved alias-to-physical-channel mapping is materialized to the gold-layer [`channel_mapping_resolution_dimension`](../data_model/gold_layer_event_normalized.md#dimension-tables). |
| `unit_conversion_table` | `str` | No | Full Unity Catalog path. Per-unit-family conversion factors. When configured together with a `channel_mapping_table` whose rows carry `source_unit` / `target_unit` columns, aliased selectors auto-convert values from source to target unit during `solve()` (currently supported by `KeyValueStoreSolver`). |
| `channel_mapping_table` | `str` | No | Full Unity Catalog path. Logical-to-physical channel alias table. Required when using `QueryBuilder.channel_with_alias()`. In reporting mode the resolved alias-to-physical-channel mapping is materialized to the gold-layer [`channel_mapping_resolution_dimension`](../data_model/gold_layer_event_normalized.md#dimension-tables). |
| `unit_conversion_table` | `str` | No | Full Unity Catalog path. Per-unit-family conversion factors. When configured together with a `channel_mapping_table` whose rows carry `source_unit` / `target_unit` columns, aliased selectors auto-convert values from source to target unit during `solve()`. |

Tag tables are required for solvers that consume tag-based filters
(`DeltaSolver` with tag filters, `KeyValueStoreSolver`).
A `container_tags_table` is required to use tag-based container filters;
a `channel_tags_table` is required to select channels by tag rather than
by columns on `channel_metrics`.

---

Expand Down Expand Up @@ -132,13 +133,13 @@ Two independent filter families:

| Field | Type | Default | Description |
|-------------------------|----------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------|
| `solver` | `str` | `"KeyValueStoreSolver"` | One of `"DeltaSolver"`, `"KeyValueStoreSolver"`. `"KeyValueStoreSolver"` works either against a narrow EAV `container_tags` table or, when `source.container_tags_table` is omitted, against a wide-only data model where container attributes live directly on `container_metrics`. |
| `solver` | `str` | `"DefaultSolver"` | `"DefaultSolver"` adapts to the silver layer: it selects channels from a narrow EAV `channel_tags` table when `source.channel_tags_table` is set and otherwise from columns on `channel_metrics`; it filters containers via a narrow EAV `container_tags` table or, when `source.container_tags_table` is omitted, a wide-only `container_metrics`. `"DeltaSolver"` and `"KeyValueStoreSolver"` are **deprecated aliases** that resolve to `DefaultSolver`. |
| `data_type` | `str` | `"RLE"` | `"RLE"` (intervals `[tstart, tend)`) or `"RAW"` (raw timestamps; converted to RLE before aggregation). |
| `drop_implausible_data` | `bool` | `false` | When `true`, drops `channels` rows where `is_plausible = false`. Requires `data_type = "RAW"`; combining with `"RLE"` raises a validation error. |
| `batch_size` | `int` | `500` | Maximum number of selectors solved per batch. |
| `solver_config` | `SolverConfig` | `null` | Per-table column mappings, per-table equality filters, and project scoping. Set `project_id` to scope reads by project — it is applied to `container_tags` (if configured), `container_metrics`, and `channel_mapping` (if configured), so it works in both narrow EAV and wide-only data models. Omit it when you don't need project scoping. See [Solver column mappings and filters](#solver-column-mappings-and-filters). |

If `query_engine` is omitted, the default is `KeyValueStoreSolver` with
If `query_engine` is omitted, the default is `DefaultSolver` with
`data_type = "RLE"`.

---
Expand Down Expand Up @@ -167,15 +168,15 @@ Top-level fields on `SolverConfig`:

Per-table sections (each a `TableConfig`):

| Section | Used by | Typical mappings |
|--------------------|--------------------------------------|-------------------------------------------------------------------|
| `container_tags` | DeltaSolver, KeyValueStoreSolver | `entity_id → container_id`, custom EAV `key`/`value` columns |
| `container_metrics`| All solvers | Custom container_id column, custom timestamp columns |
| `channel_tags` | DeltaSolver | Tag key/value column renames |
| `channel_metrics` | All solvers | Custom channel_id column, custom value/timestamp columns |
| `channel_mapping` | KeyValueStoreSolver | Alias-table column renames; `priority` column; optional `join_keys` for non-default alias-resolution composite keys |
| `channels` | All solvers | RLE column renames (`tstart`/`tend`/`value`) |
| `unit_conversion` | KeyValueStoreSolver | Unit-conversion table column renames (`unit`, `group_id`, `conversion_factor`) |
| Section | When it applies | Typical mappings |
|--------------------|--------------------------------------------|-------------------------------------------------------------------|
| `container_tags` | when `container_tags_table` is configured | `entity_id → container_id`, custom EAV `key`/`value` columns |
| `container_metrics`| always | Custom container_id column, custom timestamp columns |
| `channel_tags` | when `channel_tags_table` is configured | Tag key/value column renames |
| `channel_metrics` | always | Custom channel_id column, custom value/timestamp columns |
| `channel_mapping` | when `channel_mapping_table` is configured | Alias-table column renames; `priority` column; optional `join_keys` for non-default alias-resolution composite keys |
| `channels` | always | RLE column renames (`tstart`/`tend`/`value`) |
| `unit_conversion` | when `unit_conversion_table` is configured | Unit-conversion table column renames (`unit`, `group_id`, `conversion_factor`) |

Internal column names that mappings can target:

Expand All @@ -199,28 +200,20 @@ Internal column names that mappings can target:
| `group_id` | Unit-family identifier on the `unit_conversion` table |
| `conversion_factor` | Per-unit factor on `unit_conversion`; also the per-channel factor name carried into the solve UDF |

:::note Per-solver feature support
:::note Feature support

`solver_config` in your JSON config is forwarded to **both**
`KeyValueStoreSolver` and `DeltaSolver` by the `Report` factory.
However, only the parts each solver supports are actually consumed:

- `KeyValueStoreSolver` uses all sections: per-table
`column_name_mapping`, per-table `filters`, and top-level
`project_id`.
- `DeltaSolver` uses only the per-table `column_name_mapping` entries
on `container_tags`, `container_metrics`, `channel_tags`,
`channel_metrics`, and `channels`. Per-table `filters`, top-level
`project_id`, and the `channel_mapping` section (alias resolution)
are **silently ignored** — the solver class does not read them.
`DefaultSolver` consumes every section of `solver_config`: per-table
`column_name_mapping`, per-table `filters`, top-level `project_id`, and the
`channel_mapping` / `unit_conversion` sections. Sections for tables you do
not configure (e.g. `channel_tags`, `channel_mapping`) are simply unused.

:::

### Example: KeyValueStoreSolver with renamed columns and per-table filters
### Example: DefaultSolver with renamed columns and per-table filters

```python
"query_engine": {
"solver": "KeyValueStoreSolver",
"solver": "DefaultSolver",
"solver_config": {
"project_id": "my_project",
"container_tags": {
Expand Down Expand Up @@ -263,7 +256,7 @@ conversion is a property of the alias, not of the channel. See
"unit_conversion_table": "my_catalog.silver.unit_conversion"
},
"query_engine": {
"solver": "KeyValueStoreSolver",
"solver": "DefaultSolver",
"solver_config": {
"unit_conversion": {
"column_name_mapping": {}
Expand All @@ -274,7 +267,7 @@ conversion is a property of the alias, not of the channel. See

### Alias-resolution join keys (optional)

`KeyValueStoreSolver.filter_aliased_channel_metrics` joins `channel_mapping`
`DefaultSolver.filter_aliased_channel_metrics` joins `channel_mapping`
to `channel_metrics` to resolve aliased selectors. The default composite key
is `(source_channel, channel_name) + (data_key, data_key)`. Override
`channel_mapping.join_keys` to change the arity or column choice — for
Expand Down Expand Up @@ -405,7 +398,7 @@ in `solver_config`, then reference the internal name in
```json
{
"query_engine": {
"solver": "KeyValueStoreSolver",
"solver": "DefaultSolver",
"solver_config": {
"container_metrics": {
"column_name_mapping": { "my_measurement_id": "container_id" }
Expand Down
23 changes: 11 additions & 12 deletions docs/impulse/docs/data_model/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,17 @@ title: Data Model
:::caution Prerequisite

The schema described on this page is the full silver-layer shape that
Impulse's [**default solvers**](../references/query_engine.md)
(`DeltaSolver`, `KeyValueStoreSolver`) recognise. **Landing your data in
this shape during ingest is the simplest and most maintainable path** —
see the [Ingestion guide](ingestion.md).

The default solvers also run on a **subset** of this model.
`KeyValueStoreSolver` needs only three of the five tables —
`container_metrics`, `channel_metrics`, and `channels` — and treats
`container_tags` and `channel_mapping` as optional add-ons. `DeltaSolver`
is the one that requires all five. See
[Which solver should I use?](../references/query_engine.md#which-solver-should-i-use)
for the decision rule.
Impulse's [**`DefaultSolver`**](../references/query_engine.md) recognises.
**Landing your data in this shape during ingest is the simplest and most
maintainable path** — see the [Ingestion guide](ingestion.md).

`DefaultSolver` also runs on a **subset** of this model: only
`container_metrics`, `channel_metrics`, and `channels` are required.
`container_tags` (EAV container filtering), `channel_tags` (EAV channel
selection), `channel_mapping` (aliases), and `unit_conversion` are optional
add-ons that the solver uses when configured. See the
[Query Engine](../references/query_engine.md#table-requirements) page for
the table-requirements matrix.

Advanced deployments with existing data layouts they cannot or do not want
to reshape can adapt further by passing a `SolverConfig` to remap column
Expand Down
14 changes: 5 additions & 9 deletions docs/impulse/docs/data_model/ingestion.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ ingesting your own data, four invariants matter most:
tstart, tend, value)`. Run-length encoded data, where identical consecutive values are collapsed into intervals to significantly reduce processing time during analysis.

An optional boolean `is_plausible` column lets the solver drop implausible
samples when configured to (`drop_implausible_data=True` on `DeltaSolver`).
samples when configured to (`drop_implausible_data=True` on `DefaultSolver`).

The remaining columns on `container_metrics` and `channel_metrics`
(timestamps, durations, mean/min/max, etc.) are *not* fixed by the engine —
Expand Down Expand Up @@ -199,14 +199,10 @@ Impulse's expectations — same set of tables and relationships — but the
[Solver column mappings and filters](../config/configuration.md#solver-column-mappings-and-filters)
for the full schema.

How it gets wired in depends on which solver you use:

- **`KeyValueStoreSolver`** and **`DeltaSolver`** — set
`query_engine.solver_config` in your report config. The `Report`
factory forwards it to both solvers. `KeyValueStoreSolver` consumes
every section (column mappings, per-table `filters`, `project_id`,
`channel_mapping`); `DeltaSolver` consumes only the per-table
`column_name_mapping` entries and silently ignores the rest.
Set `query_engine.solver_config` in your report config. `DefaultSolver`
consumes every section that applies to the tables you have configured —
column mappings, per-table `filters`, `project_id`, and the
`channel_mapping` / `unit_conversion` sections.

Trade-off either way: this gives you naming flexibility and per-table
scoping filters without writing code, but the underlying tables must
Expand Down
23 changes: 12 additions & 11 deletions docs/impulse/docs/data_model/silver_layer_schema.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,9 @@ The Silver layer stores measurement data in a normalized, tag-based model.
Time-series samples live in `channels`; metadata is split across tag
tables (EAV key-value pairs) and metric tables (pre-computed statistics).

The five tables and columns on this page describe the **typical
The tables and columns on this page describe the **typical
default-solver shape** — what Impulse's
[default solvers](../references/query_engine.md)
(`DeltaSolver`, `KeyValueStoreSolver`) expect when
[`DefaultSolver`](../references/query_engine.md) expects when
no [`SolverConfig`](../config/configuration.md#solver-column-mappings-and-filters)
overrides are applied. The framework hard-requires only a small subset of this shape:

Expand Down Expand Up @@ -122,7 +121,7 @@ when your physical column has a different name.
| `container_id`| Join key against `container_tags`, upsert key in incremental mode |
| `start_ts` | `ContainerEvent` event-fact `start_ts`; default `measurement_dimensions` entry |
| `stop_ts` | `ContainerEvent` event-fact `end_ts`; default `measurement_dimensions` entry |
| `project_id` | `KeyValueStoreSolver` project scoping (when `solver_config.project_id` is set) |
| `project_id` | `DefaultSolver` project scoping (when `solver_config.project_id` is set) |

Every other column on `container_metrics` is **pass-through**: it lands in
the gold `measurement_dimension` table verbatim if you list it in
Expand Down Expand Up @@ -198,7 +197,7 @@ when your physical column has a different name.
| `container_id`| Join key into `container_metrics` after tag filtering |
| `key` | EAV pivot key — driven by `query.havingTag(...)` and `container_filters.tag_filters` |
| `value` | EAV pivot value — driven by `query.havingTag(...)` and `container_filters.tag_filters` |
| `project_id` | `KeyValueStoreSolver` project scoping (when `solver_config.project_id` is set, and this table carries a project column) |
| `project_id` | `DefaultSolver` project scoping (when `solver_config.project_id` is set, and this table carries a project column) |
| `parent_id` | Optional per-table filter target (e.g. `solver_config.container_tags.filters = {"parent_id": ...}`) |

---
Expand Down Expand Up @@ -273,7 +272,9 @@ selectors look up signals by tag key (e.g.
Map any silver column to these via
[`solver_config.channel_tags.column_name_mapping`](../config/configuration.md#solver-column-mappings-and-filters)
when your physical column has a different name. Note: `channel_tags` is
read by `DeltaSolver` only.
read by `DefaultSolver` only when a `channel_tags_table` is configured
(EAV channel selection); otherwise channel selection runs against columns
on `channel_metrics`.

| Internal name | Referenced by |
|---------------|------------------------------------------------------------------------------|
Expand Down Expand Up @@ -346,7 +347,7 @@ during raw→RLE conversion.

## channel_mapping (optional)

Alias-resolution table used by `KeyValueStoreSolver` when selectors are
Alias-resolution table used by `DefaultSolver` when selectors are
created via `QueryBuilder.channel_with_alias()`. Each row maps a logical
channel name to one or more physical channels keyed by `project_id` /
`data_key`, with an optional `priority` tie-breaker.
Expand Down Expand Up @@ -385,7 +386,7 @@ so they agree on the unit pair per physical channel.
Map any silver column to these via
[`solver_config.channel_mapping.column_name_mapping`](../config/configuration.md#solver-column-mappings-and-filters)
when your physical column has a different name. Note: `channel_mapping`
is read by `KeyValueStoreSolver` only.
is read by `DefaultSolver` only when a `channel_mapping_table` is configured.

| Internal name | Referenced by |
|------------------|------------------------------------------------------------------------------------------------|
Expand All @@ -394,7 +395,7 @@ is read by `KeyValueStoreSolver` only.
| `data_key` | Default join target on the `channel_mapping`→`channel_metrics` join |
| `source_channel` | Alias-resolution source on the `channel_mapping`→`channel_metrics` join |
| `priority` | Tie-breaker when multiple physical channels match a logical alias |
| `project_id` | `KeyValueStoreSolver` project scoping (when `solver_config.project_id` is set) |
| `project_id` | `DefaultSolver` project scoping (when `solver_config.project_id` is set) |
| `source_unit` | Source unit fallback (when paired with a `unit_conversion_table`) |
| `target_unit` | Target unit for aliased reads (when paired with a `unit_conversion_table`) |

Expand All @@ -407,7 +408,7 @@ explicitly.

## unit_conversion (optional)

Per-unit-family conversion factors. Read by `KeyValueStoreSolver` at
Per-unit-family conversion factors. Read by `DefaultSolver` at
solve time when `source.unit_conversion_table` is configured and the
`channel_mapping` table carries `source_unit` / `target_unit` columns.

Expand All @@ -428,7 +429,7 @@ multiplies values by `source_factor / target_factor`. Missing rows or a
Map any silver column to these via
[`solver_config.unit_conversion.column_name_mapping`](../config/configuration.md#solver-column-mappings-and-filters)
when your physical column has a different name. Note: `unit_conversion`
is read by `KeyValueStoreSolver` only.
is read by `DefaultSolver` only when a `unit_conversion_table` is configured.

| Internal name | Referenced by |
|----------------------|--------------------------------------------------------------------------------------------|
Expand Down
Loading
Loading