Skip to content

milp-optimization.md

Latest commit

 

History

History
593 lines (430 loc) · 37.5 KB

milp-optimization.md

File metadata and controls

593 lines (430 loc) · 37.5 KB
Raw

HSEM MILP Optimization

The MILP solver (planner/milp_optimizer.py) finds the globally optimal battery charge/discharge schedule using scipy's HiGHS linear programming solver. It is the sole optimisation authority. The only accompanying candidates are no_action (diagnostic only) and passive (fail-closed PV-only fallback); the former heuristic candidate family is retired. scipy/numpy are declared in manifest.json's requirements, so Home Assistant installs them automatically on setup; the "unavailable" fallback below is a defensive path, not the expected steady state.

Pipeline

flowchart TD
    A[Engine passes slots with populated prices / PV / consumption]
    B[Identify future active slot indices past vs fixed-zero]
    C[Build per-slot data arrays: p_imp, p_exp, net_load, base_load, pv_avail]
    D{EV configs provided?}
    E[Rebuild net_load without pre-computed EV planned loads]
    F[Keep fixed EV planned load in net_load]
    G[Build objective vector c_obj: import cost, export revenue, cycle cost, terminal-SoC credit, SoC penalties, EV deadline penalties, EV pre-deadline benefit, EV charge-past-target benefit]
    H[Build equality constraints A_eq: energy balance per slot inc. EV charger load]
    I[Build inequality constraints A_ub: SoC recurrence soft bounds, mutual exclusion, cycle cost auxiliary m >= ec/ed, EV cumulative SOC, EV deadline target, EV post-deadline zero-charge, EV surplus-only]
    J[Build variable bounds: ec, ed capped; pv fixed to actual surplus; penalties >= 0]
    K[linprog method = highs, timeout = 2s]
    L{Solution found?}
    M[Decode solution: ec, ed, gi, ge, pv, m, penalties, EV charging]
    N[Write recommendations to output slots BatteriesChargeGrid, BatteriesChargeSolar, BatteriesDischargeMode, ForceBatteriesDischarge]
    N2[Write LP-derived energy flow fields: batteries_discharged_kwh, grid_import_kwh, grid_export_kwh — these are the source of truth]
    O[Compute penalty violation diagnostics]
    P[Return slots and diagnostics]
    Q[Return None: solver failed or no future slots]

    A --> B --> C --> D
    D -->|Yes| E --> G
    D -->|No| F --> G
    G --> H --> I --> J --> K --> L
    L -->|Yes| M --> N --> N2 --> O --> P
    L -->|No| Q
Loading

Variable layout

Base battery variables (8 × n slots)

For each slot t ∈ 0…n-1 the LP variable vector x contains eight decision variables:

Offset Variable Name Description Bounds
0 ec[t] ec_off Energy charged and stored in battery this slot (kWh) [0, max_charge_per_slot]
n ed[t] ed_off Energy discharged from battery this slot (kWh) [0, max_discharge_per_slot]
2n gi[t] gi_off Grid import this slot (kWh) [0, ∞)
3n ge[t] ge_off Grid export this slot (kWh) [0, ∞) — hard-capped to max_grid_export_power_kw × slot_hours when the export cap is configured (issue #726)
4n pv[t] pv_off PV surplus available in slot t (kWh) [pv_avail[t], pv_avail[t]] (fixed)
5n m[t] m_off Auxiliary variable ≥ max(ec[t], ed[t]) for cycle cost (kWh) [0, ∞)
6n s_max_pen[t] s_max_off SoC upper penalty — kWh by which state of charge exceeds usable_kwh [0, ∞)
7n s_min_pen[t] s_min_off SoC lower penalty — kWh by which state of charge drops below 0 [0, ∞)

The state of charge soc[t] is not an explicit variable — it is derived from the forward recurrence:

$$ soc[t] = soc[0] + \sum_{k=0}^{t} \bigl( ec[k] - ed[k] \bigr) $$

Penalty variables s_max_pen and s_min_pen prevent infeasibility when the initial SoC lies outside [0, usable_kwh]. Their objective coefficient is extremely high (max(p_imp) × 100), so they are only used when the initial state is physically out of bounds.

EV co-optimization extension

When one or more active EVs are provided, the variable vector expands to:

$$ \text{total variables} = 8n + n \cdot E + E $$

where E is the number of active EVs.

Offset Variable Name Description Bounds
8n + i·n evN_c[t] EV N DC-side charge per slot (kWh) [0, evN.max_charge_per_slot]
8n + E·n + i evN_pen EV N deadline target slack (kWh shortfall) [0, ∞)

The EV charger AC load entering the energy balance equation is evN_c[t] / charger_efficiency.

When an EV's charge_past_target flag is True (EV already at user-configured target SoC, allow_charge_past_target_soc enabled, SoC < 100 %):

  • The deadline constraint is suppressed (deadline_slot = None) — no grid import pressure
  • The surplus-only constraint is added (see Constraints below)
  • An avoided-future-import-cost benefit (future_value_per_kwh, issue #630) is added to the objective, falling back to a tiny fixed tiebreaker when no future price data is available (see Objective function below)

When an EV has a deadline and charge_past_target=False (normal mode):

  • Pre-deadline slots (t \u2264 D, issue #797): no direct per-kWh benefit on ev_c[t]; the deadline slack penalty alone already prices meeting the target far above any real import price, so charging competes only against its own real grid/PV opportunity cost. A tiny tiebreak cost (_EV_TARGET_ENERGY_TIEBREAK_COST = 1e-7/kWh) nudges the LP toward the smallest executable energy that clears the target-cap constraint.
  • Post-deadline slots (t > D): hard constraint ev_c[t] = 0 — charging is forbidden

Managed EV whole-amp lattice (issue #797)

A managed EV (not fixed_session_only) gets a solver-native semi-integer amp variable ev_{i}_amps[t] (HiGHS integrality type 3: zero, or an integer in [min_amp, rated_amp]), declared as the last blocks in build_milp_column_layout (planner/milp/_layout.py), after every physical and fuse block. An equality constraint per slot links it to the DC charge variable:

$$ ev\_c[t] = ev\_amps[t] \times \text{one\_amp\_dc\_kwh}[t] $$

so the solved allocation is always directly executable — _write_results.py publishes it verbatim, with no post-solve concentration/quantization pass. See planner/milp/_ev_amp_lattice.py and the Discharge permission and whole-amp lattice section of docs/planner-spec.md for the full constraint model, including the conditional ev_{i}_on binary that caps primary battery discharge while a permission-restricted EV charges.

Fuse constraint extension (issue #567)

When main_fuse_amps > 0, the variable vector expands further:

$$ \text{total variables} = 8n + n \cdot E + E + n $$

Offset Variable Name Description Bounds
after EV vars gi_pen[t] gi_pen_off Grid import fuse penalty — kWh exceeding the main fuse rating [0, ∞)

House-battery target extension (issue #1109)

The stage-2 solve of the opt-in house-battery target adds one column, battery_target_penalty (width 1, [0, ∞)), the shortfall in kWh against the target at slot $T$. It is declared only for that solve, so the stage-1 model is unchanged. The same solve also replaces the gi[t] bounds with the stage-1 import (fixed for $t \le T$, capped for $t &gt; T$) and raises the ge[t] lower bound to the stage-1 battery-origin export for $t \le T$ (issue #1203). See planner-spec.md.

The max grid import per slot is converted from amps to kWh/slot:

$$ \mathrm{max\_grid\_import} = \frac{\mathrm{amps} \times 230 \times \mathrm{phases}}{1000} \times \frac{\mathrm{interval\_minutes}}{60} $$

where phases is the electrical phase count (1 or 3, default 3). This assumes balanced load at 230 V phase-to-neutral per phase.

The penalty uses the same high coefficient as SoC penalties (max(p_imp) × 100), ensuring the solver only exceeds the fuse limit when physically unavoidable (e.g. house base load alone exceeds the rating). When main_fuse_amps is None or 0, no variables or constraints are added — behaviour is unchanged.

Objective function

$$ \begin{aligned} \mathrm{minimise} \quad \sum_{t} \delta_t \cdot \bigg[ &amp; p_{\mathrm{imp}}[t] \cdot gi[t] &amp;&amp; \text{grid import cost} \\ - &amp; p_{\mathrm{exp}}[t] \cdot ge[t] &amp;&amp; \text{export revenue} \\ + &amp; \alpha \cdot m[t] &amp;&amp; \text{battery cycle cost (depreciation)} \\ + &amp; p_{\mathrm{soc}} \cdot \bigl( \mathrm{s\_max\_pen}[t] + \mathrm{s\_min\_pen}[t] \bigr) &amp;&amp; \text{SoC soft-constraint penalties} \\ + &amp; p_{\mathrm{fuse}} \cdot \mathrm{gi\_pen}[t] &amp;&amp; \text{Main fuse grid-import penalty} \bigg] \\ {} + \sum_{t} \gamma \cdot \bigl( ed[t] - ec[t] \bigr) &amp;&amp; \text{terminal-SoC valuation (undiscounted)} \\ \end{aligned} $$

Plus EV deadline penalties (undiscounted — deadline is a hard commitment):

$$ \sum_{v=1}^{E} p_{\mathrm{ev\_pen}}^{(v)} \cdot \mathrm{ev\_pen}_v $$

Where:

Symbol Description
$\delta_t$ Time discount per slot: $\delta_t = r^{\Delta t}$ where $\Delta t$ is hours from now
$p_{\mathrm{imp}}[t]$ Grid import price (currency/kWh), sanitised to max(p_imp_raw[t], 0) (issue #655).
$p_{\mathrm{exp}}[t]$ Grid export price (currency/kWh). Before solving, p_exp is sanitised by clamping to min(p_exp, p_imp) to prevent an unbounded LP when p_exp > p_imp in any slot (issue #635). The legacy min_export_price clamp has been removed because the applier no longer physically blocks grid export below this price (issue #767). The export-revenue coefficient above actually uses p_exp_net[t] = p_exp[t] - export_fee_per_kwh (issue #925, default fee 0.0) — a fixed retailer margin/balancing-fee deduction so a positive-but-net-negative price is valued the same as a genuinely negative price. This netting is local to the objective; the battery-export floor mask (below) still uses the raw p_exp.
$\alpha$ Battery cycle cost per kWh: $\alpha = \frac{P \cdot L_{pct}/100}{2 \cdot N \cdot C_u}$

| $\gamma$ | Terminal-SoC end value $V$ (currency per DC kWh), from the engine: $\max(0, \min(0.9(\eta_{dis} \cdot peak - \alpha), night/\eta_{chg} + \alpha))$ over the last known day of prices (cost_helpers.terminal_end_value_from_last_day, issue #1138). One value for every slot, so an in-horizon cycle that leaves the end energy unchanged adds zero. | | $p_{\mathrm{soc}}$ | SoC penalty cost: $\max(p_{\mathrm{imp}}) \times 100$ | | $p_{\mathrm{fuse}}$ | Fuse penalty cost: $\max(p_{\mathrm{imp}}) \times 100$ (same magnitude as SoC) | | $p_{\mathrm{ev\pen}}^{(v)}$ | EV deadline penalty for EV v: $\max(p{\mathrm{imp}}) \cdot \max(\mathrm{energy\needed}, 1.0) \cdot 10$ | | $\beta{\mathrm{ev}}^{(v)}$ | EV charge-past-target benefit for EV v: future_value_per_kwh — avoided-future-import valuation (issue #630), or a $0.0001$ per kWh AC fallback tiebreaker when no future price data is available |

Grid-charge break-even

There is no fixed-schedule min_price_difference constraint in the MILP. The solver chooses grid charging only when its full-horizon objective improves. For a simple charge-now/discharge-later comparison, one stored kWh is profitable when:

$$ p_{discharge} \eta_{discharge} > \frac{p_{charge}}{\eta_{charge}} + \alpha $$

or equivalently:

$$ p_{discharge} > \frac{p_{charge}/\eta_{charge} + \alpha}{\eta_{discharge}} $$

The formula includes the actual AC energy needed to store one kWh, the AC energy delivered by one discharged kWh, and cycle wear. A raw price spread that only matches $\alpha$ is insufficient because it ignores both conversion losses. Terminal-SoC valuation and export opportunity cost remain part of the complete objective and can change the global result.

Plus EV pre-deadline benefit (undiscounted, per EV $v$ with deadline, slots $t \leq D_v$):

$$ -\sum_{v=1}^{E} \sum_{t=0}^{D_v} p_{\mathrm{ev\_pen}}^{(v)} \cdot \mathrm{ev\_c}_v[t] $$

This direct benefit on pre-deadline slots ensures the LP always prefers charging over paying the deadline penalty. Post-deadline slots ($t &gt; D_v$) have zero coefficient unless charge_past_target=True.

The pre-deadline benefit block and the charge-past-target benefit block are mutually exclusive by design. The LP construction guards the pre-deadline block with and not ev.charge_past_target, mirroring the post-deadline zero-charge constraint's guard. An EV in charge-past-target mode never receives the large penalty-driven benefit — only the future_value_per_kwh benefit (or its tiebreaker fallback) applies. This exclusion is enforced directly in the LP construction rather than relying on caller discipline in engine_core.py.

Plus EV charge-past-target benefit (discounted, per charge-past-target EV $v$):

$$ -\sum_{v \in \mathrm{past\_target}} \sum_{t} \delta_t \cdot \frac{\beta_{\mathrm{ev}}^{(v)}}{\eta_{\mathrm{charger}}^{(v)}} \cdot \mathrm{ev\_c}_v[t] $$

$\beta_{\mathrm{ev}}^{(v)}$ is EVConfig.future_value_per_kwh: the avoided cost of importing the same energy later, computed as confidence_factor × mean(import_price) over the next 24 hours (ev_future_charge_value_per_kwh in candidate_selector.py; the house battery's terminal SoC uses a similar avoided-cost value, $\gamma$). confidence_factor defaults to 0.9 and is configurable per EV (hsem_ev_past_target_confidence_factor / hsem_ev_second_past_target_confidence_factor) to discount for uncertainty in whether the EV will actually need the extra energy before its next charge.

Because $\beta_{\mathrm{ev}}^{(v)}$ is priced in the same currency units as $p_{\mathrm{imp}}$ and $p_{\mathrm{exp}}$, charge-past-target EV charging competes fairly against both house battery charging and grid export — whichever has the higher genuine avoided-cost value wins the surplus for that slot. When no future price data is available (future_value_per_kwh is None), the MILP falls back to a tiny fixed tiebreaker ($0.0001$ per kWh AC) so surplus PV still prefers the EV over being wastefully curtailed/exported at near-zero or negative prices.

Constraints

Equality constraint — energy balance per slot

For each slot $t$:

$$ gi[t] + pv[t] + ed[t] \cdot \eta_{\mathrm{dis}} = \operatorname{base\_load}[t] + \frac{ec[t]}{\eta_{\mathrm{chg}}} + ge[t] + \sum_{v=1}^{E} \frac{\operatorname{ev\_c}_v[t]}{\eta_{\mathrm{charger}}^{(v)}} $$

  • base_load[t] = $\max(\operatorname{net\_load}[t], 0)$ — demand the grid/battery must satisfy (kWh)
  • net_load[t] = avg_house_consumption[t] - solcast_pv_estimate[t] (when EV co-optimisation active)
  • pv_avail[t] = $\max(-\operatorname{net\_load}[t], 0)$ — PV surplus fixed to the pv[t] variable bounds
  • EV charger efficiency re-scales DC-side charge to AC grid/PV load

Inequality constraints

SoC upper bound (soft):

$$ \sum_{k=0}^{t} \bigl( ec[k] - ed[k] \bigr) - \mathrm{s\_max\_pen}[t] \leq C_u - soc_0 $$

SoC lower bound (soft):

$$ -\sum_{k=0}^{t} \bigl( ec[k] - ed[k] \bigr) - \mathrm{s\_min\_pen}[t] \leq soc_0 $$

Mutual exclusion — no simultaneous charge + discharge:

$$ \frac{ec[t]}{\mathrm{max\_charge}} + \frac{ed[t]}{\mathrm{max\_discharge}} \leq 1 $$

Cycle cost auxiliary — forcing $m[t] \geq ec[t]$ and $m[t] \geq ed[t]$:

$$ -m[t] + ec[t] \leq 0 $$

$$ -m[t] + ed[t] \leq 0 $$

EV cumulative SoC upper bound (per EV v):

$$ \sum_{k=0}^{t} \mathrm{ev\_c}_v[k] \leq \mathrm{capacity}_v - \mathrm{initial\_soc}_v $$

EV deadline target (soft, per EV v):

$$ \mathrm{initial\_soc}_v + \sum_{k=0}^{D_v} \mathrm{ev\_c}_v[k] + \mathrm{ev\_pen}_v \geq \mathrm{target}_v $$

EV post-deadline zero-charge (hard, per EV v with deadline and charge_past_target=False):

$$ \mathrm{ev\_c}_v[t] = 0 \quad \forall\, t > D_v $$

This hard constraint prevents any EV charging after the deadline unless charge_past_target=True (surplus-PV-only mode).

EV surplus-only constraint (per charge-past-target EV v, per slot t):

$$ \frac{\mathrm{ev\_c}_v[t]}{\eta_{\mathrm{charger}}^{(v)}} \leq \max\bigl(0,\; \mathrm{pv\_avail}[t] - \mathrm{base\_load}[t]\bigr) $$

This constraint ensures charge-past-target EVs only consume genuine PV surplus — never battery discharge or grid import. It is added for EVs where charge_past_target=True (EV already at user-configured target SoC but allow_charge_past_target_soc is enabled and SoC < 100 %). The bound is pro-rated by the remaining fraction of a partly elapsed live slot (issue #1012).

Shared battery-first budget (issue #775, #1015):

A per-slot surplus cap alone cannot stop a past-target EV from taking surplus the house battery would have stored while the battery refills from cheap grid — the EV would then draw from grid in all but name. And because ec[t] is the battery's total charge (grid- and PV-sourced energy share one column), a shared row $ec[t] + \sum_v \mathrm{ev\_c}v[t]/\eta{\mathrm{charger}}^{(v)} \le S[t]$ caps all battery charging at the surplus, so the battery could not grid-charge while such an EV was plugged in.

The production path solves the counterfactual in two stages (planner/milp/_past_target_reservation.py). Stage 1 solves without any charge-past-target EV and records $\mathrm{reserved}[t]$, the AC energy it spent on the battery and the other EVs. Stage 2 caps the past-target EVs at the PV stage 1 left unused:

$$ \sum_v \frac{\mathrm{ev\_c}_v[t]}{\eta_{\mathrm{charger}}^{(v)}} \leq \max\bigl(0,\; S_{\mathrm{full}}[t] - \mathrm{reserved}[t]\bigr) \cdot \mathrm{remaining\_fraction}[t] $$

The battery then needs no row and stays free to grid-charge. The second solve runs only while a charge-past-target EV is active. A direct solve_milp call without a reservation keeps the conservative shared row above. See planner-spec.md Battery-first for charge-past-target.

House-battery target (soft, stage 2 only, issue #1109):

$$ -\sum_{k \le T} (ec[k] - ed[k]) - \mathrm{bt\_pen} \leq E_0 - E_{target} $$

with $\mathrm{bt\_pen}$ priced at $P$ in the objective (undiscounted) and $gi[t]$ pinned to the stage-1 plan as variable bounds, and $ge[t]$ bounded from below by the stage-1 battery-origin export for $t \le T$ (issue #1203). With import fixed and the battery's own export kept, the only way to raise $soc[T]$ is to export less PV (planner/milp/_battery_target.py).

Main fuse grid import limit (soft):

For each slot $t$, when main_fuse_amps > 0:

$$ gi[t] - \mathrm{gi\_pen}[t] \leq \frac{\mathrm{amps} \times 230 \times \mathrm{phases}}{1000} \times \frac{\mathrm{interval\_minutes}}{60} $$

The penalty variable gi_pen[t] absorbs any excess at high cost (p_fuse), preventing infeasibility when house base load alone exceeds the fuse rating. When main_fuse_amps is None or 0, this constraint is not added.

Grid export power cap (hard, issue #726):

For each slot $t$, when max_grid_export_power_kw > 0:

$$ ge[t] \leq \mathrm{max\_grid\_export\_power\_kw} \times \mathrm{slot\_hours} $$

This is a hard bound on the ge[t] variable — unlike the fuse it needs no penalty variable because the cap is physically enforced by the inverter/DNO, so exceeding it is never required for feasibility. Battery export and PV export compete for the same cap through the energy-balance equality, so the LP naturally front-loads battery export into low-PV slots and tapers it as PV ramps; PV that cannot be exported at the cap is handled by the free curt[t] variable. When max_grid_export_power_kw is None or 0, ge[t] remains unbounded above (identical to previous behaviour).

Where $D_v$ is the deadline slot index for EV v.

Conditional battery-export reserve:

When excess export is enabled and the discharge buffer is positive, explicit battery-origin export bx[t] activates binary z_export[t]. Every slot in one contiguous forecast PV-surplus run uses the checkpoint derived from that run's final slot: immediately before the next distinct surplus run, or horizon end.

bx[t] <= max_discharge_per_slot * z_export[t]
SoC[checkpoint[t]] >= buffer_kwh - usable_kwh * (1 - z_export[t])

Aggregate export is source-constrained by direct PV plus battery AC export, so the reserve and no-export mode can suppress battery export without restricting ordinary PV export. Grouping changes only checkpoint preprocessing and adds no variables or rows beyond the per-slot source/reserve formulation.

Battery export forecast reserve (issue #807, Stage 1):

An independent, opt-in reserve (hsem_batteries_forecast_reserve_pct, 0–50 %, default 0 = disabled) adds one row per slot, active whenever forecast_reserve_kwh > 0:

SoC[t] >= forecast_reserve_kwh - usable_kwh * (1 - z_export[t])

Unlike the checkpoint reserve, this row is indexed by the same slot t, so it binds the SoC immediately after the exporting slot itself — a later PV or grid refill can never justify spending it first. forecast_reserve_kwh is the configured percentage above the hardware discharge floor, which is the MILP's inventory origin. The dynamic discharge floor is a separate per-slot lower bound on stored energy above that same origin (issue #1188), so the two never add up: the higher one binds. See docs/planner-spec.md § Battery export forecast reserve for the full derivation.

Battery export minimum price floor (issue #752):

For each slot $t$, when battery_export_min_price > 0 and the slot's raw p_exp[t] < battery_export_min_price (evaluated before the min_export_price and export-≤-import clamps):

$$ ed[t] \leq \frac{\mathrm{base\_load}[t]}{\eta_{\mathrm{dis}}} $$

This is the per-slot, soft-switch companion to the global no_export cap. Where no_export blocks battery export on every slot when excess_export_enabled = False, the floor blocks it only on slots where the user's explicit per-slot price guard is unsatisfied. The battery can still serve house load on the blocked slot — it just cannot intentionally export to the grid there. Above the floor the optimizer is free to decide whether exporting is worthwhile; reaching the threshold does not automatically trigger export.

Scope: the floor applies only to intentional battery-to-grid export (ForceBatteriesDischarge). It does not affect normal battery self-consumption, battery discharge for house load, direct PV export (ge is not capped — only ed is), or PV charging of the battery.

Evaluation on the raw p_exp is essential: the user's battery_export_min_price floor must be honoured even when the recommended_threshold (auto-calculated cycle wear) or the export_min_price battery-export floor are lower. Selecting the larger of the three floors would force every user to overwrite their own threshold when they want a stricter guard, defeating the purpose of the dedicated setting.

The non-MILP apply_excess_export path enforces the same floor by requiring export_price >= max(export_min_price, recommended_threshold, battery_export_min_price) for any slot it would otherwise label ForceBatteriesDischarge.

When battery_export_min_price = 0 (default), no slot is ever blocked by the floor — the behavior is entirely backward compatible with the pre-#752 PLP.

Price sanitisation

Before the LP is built, two sanitisation steps are applied to p_exp to prevent solver instability and maintain consistency with physical constraints:

1. Battery export floor (replaces min-export-price clamp)

Slots where p_exp < max(min_export_price, battery_export_min_price) do not have their export price clamped to 0. Instead, the battery discharge variable ed[t] is capped to base_load[t] / η_dis so the battery can serve house load but cannot intentionally export to the grid. This keeps surplus PV export possible (the applier no longer throttles the grid feed-in limit for non-negative prices, issue #767) while still preventing battery-to-grid discharge below the configured floor. Negative export prices are handled by the applier, which writes a physical watt limit to block all grid export when exporting costs money.

2. Export-≤-import clamp (issue #635)

p_exp is clamped to never exceed p_imp for the same slot:

$$ p_{\mathrm{exp}}[t] = \min\bigl(p_{\mathrm{exp}}[t],\; p_{\mathrm{imp}}[t]\bigr) $$

Without this, slots where p_exp > p_imp create an unbounded LP (HiGHS status=3). gi[t] and ge[t] are both [0, ∞) and linked only through the per-slot energy-balance equality, so the LP can drive both to infinity (import cheap, export expensive) while the terms cancel. A single such slot causes solve_milp() to return None for the entire horizon, silently falling back to weaker heuristic candidates.

This condition occurs in practice whenever negative import spot prices coincide with positive export tariffs (common in DK/DE/NL markets during high wind/solar hours), or when asymmetric import/export grid fees create an apparent export-price premium.

The clamp is economically correct and capping the achievable arbitrage spread removes the unbounded direction without changing any other optimisation behaviour.

3. Primary conversion loss: physical single-count accounting

The site balance uses ec[t] / charge_efficiency as AC charge draw and ed[t] * discharge_efficiency as AC discharge delivery. Consequently the gi[t] and ge[t] money coefficients price conversion loss exactly once. A separate (1-efficiency) coefficient would double-count it.

The retained conversion_loss_cost and discharge_loss_cost_destination_aware fields remain for schema compatibility and are always zero. The LP and authoritative scorer use the same single-count physical model.

Post-processing

After solving, the solution is decoded into slot recommendations and energy flow fields:

flowchart TD
    A[LP solution: ec, ed, gi, ge]
    B{ec > threshold AND ed > threshold?}
    C[Numerical tolerance conflict: resolve by net profit]
    D{Round-trip profitable?}
    E[Keep ec, zero ed]
    F[Zero both]
    G{ec > threshold?}
    H{PV surplus available?}
    I[BatteriesChargeSolar]
    J[BatteriesChargeGrid]
    K{ed > threshold?}
    L{Grid export > 0 AND price >= min_export_price?}
    M[ForceBatteriesDischarge]
    N[BatteriesDischargeMode]
    N2[Write LP energy flow fields to ALL future slots:
    batteries_discharged_kwh, grid_import_kwh, grid_export_kwh]
    O[Write EV charge decisions]
    P[Recompute estimated_net_consumption and estimated_cost per slot]
    Q[Compute penalty violation diagnostics]

    A --> B
    B -->|Yes| C --> D
    D -->|Yes| E --> G
    D -->|No| F --> G
    B -->|No| G
    G -->|Yes| H
    H -->|Yes| I --> N2
    H -->|No| J --> N2
    G -->|No| K
    K -->|Yes| L
    L -->|Yes| M --> N2
    L -->|No| N --> N2
    K -->|No| N2
    N2 --> O --> P --> Q
Loading

Energy flow fields written to slots (issue #637)

The MILP now writes all per-slot energy flow fields directly from the LP solution, making them the source of truth:

Field LP variable Description
batteries_discharged_kwh ed[t] Energy discharged from battery (kWh)
grid_import_kwh gi[t] Grid import (kWh)
grid_export_kwh ge[t] Grid export (kWh)

These are populated for every future slot (zero for idle slots). The SoC simulation (:func:~soc_simulation.simulate_soc) must be called with milp_prepopulated=True for MILP-sourced candidates so it preserves these values verbatim instead of re-deriving a different (greedy) allocation from the recommendation label and net demand.

EV charging fields written to slots

The raw CT-position flag applies only to current live-meter reconciliation. Before MILP construction, HSEM derives base_load_includes_ev independently per EV from the normalized house baseline: configured EV power telemetry is removed by HSEM history preprocessing, so that EV is not embedded even when the raw CT is upstream of the EVSE. Current-slot live injection additionally records which measured sessions it actually removed. The MILP net-load rebuild, writeout, and SoC simulation all use this same per-EV split; no site-wide any(...) shortcut is valid for two-EV accounting.

Field Source
ev_planned_load_kwh Sum of EV AC contributions not embedded in normalized house consumption
ev_accounted_load_kwh Sum of EV AC contributions still embedded in normalized house demand
ev_total_planned_load_kwh Total AC load (sum of planned + accounted)
ev_charger_calculated_power Target AC power (W) for primary EV
ev_second_charger_calculated_power Target AC power (W) for second EV

The pure-house load used for battery discharge is avg_house_consumption_kwh - ev_accounted_load_kwh; it is not clamped to zero. This lets the battery cover genuine house demand while an EV charges, subject to the existing permission and positive per-EV discharge ceiling, while EV demand itself remains grid/PV supplied.

Engine-level post-processing (after winner selection)

After the MILP (or baseline) winner is selected, the engine runs a final pass over all slots to ensure consistency:

  1. Power recomputation: ev_charger_calculated_power is recomputed from the actual per-slot EV AC load (ev_planned_load_kwh + ev_accounted_load_kwh). For the current (partially elapsed) slot the remaining time is used as the divisor. This ensures the power field always matches the load, even when the baseline candidate wins (bypassing the MILP's own power calculation).

  2. Minimum power floor: If the computed AC power is below charger_min_power_w (default 1380 W = 230 V × 6 A), the charger physically cannot start. The slot's EV fields are zeroed out:

    • ev_charger_calculated_power = 0
    • ev_planned_load_kwh = 0, ev_accounted_load_kwh = 0, ev_total_planned_load_kwh = 0
    • recommendation cleared if it was ev_smart_charging
    • estimated_net_consumption_kwh and estimated_cost_currency recomputed without EV load

  3. EV plan rebuild: When the MILP wins, the EVChargingPlan objects (used by the ev_optimal_charging_plan sensor) are rebuilt from the winning slots via rebuild_ev_plan_from_slots(). This ensures the sensor displays the MILP's actual decisions, not the EV planner's pre-MILP estimate.

Assumptions

  • Linear relaxation: Binary charge/discharge flags are relaxed to continuous because the mutual-exclusion constraint and per-slot power caps already prevent simultaneous charge + discharge in the optimal solution.
  • Deterministic inputs: All forecasts (prices, PV, load) are treated as known with certainty — no stochastic programming.
  • Cycle cost proxy: The m[t] = max(ec[t], ed[t]) formulation counts the larger of charge or discharge per slot, matching the 2× denominator in the cycle cost formula.
  • Time discount: The objective uses exponential discounting with time_discount_rate^hours_ahead to match the selector's discounted score.
  • Export price clamping: Negative export prices and prices below min_export_price are clamped to 0 before solving, reflecting physical inverter behaviour.
  • Terminal-SoC credit: Undiscounted in the objective, matching the cost function's terminal_soc_value.

Solver configuration

Parameter Value Rationale
Method highs scipy's HiGHS is the only supported LP method
Timeout 2.0 s Covers 192-slot (768+ variable) problems where preprocessing reaches 200-400 ms
pv[t] bounds (pv_avail[t], pv_avail[t]) Fixed — PV surplus is not chosen by the LP

Fallback

If scipy is unavailable, usable_kwh ≤ 0, or the solver fails (crash, timeout, or non-success status), solve_milp() returns None. The engine drops the MILP candidate and fails closed to the passive PV-only candidate; no_action remains diagnostic and cannot win. Since scipy/numpy are declared manifest.json requirements, Home Assistant installs them for every user on setup; scipy unavailable should now only occur if the environment's package installation itself failed, not as routine behavior. Plan origin is exposed as milp for a successful solve and rule_based for the defensive fallback.

Related