Feat: scheduler support for device groups

documentation/concepts/commitments.rst

@@ -0,0 +1,179 @@
+.. _commitments:
+
+Commitments
+===========
+
+Overview
+--------
+
+A **Commitment** is the economic abstraction FlexMeasures uses to express
+market positions and soft constraints (preferences) inside the scheduler.
+Commitments are converted to linear objective terms; all non-negotiable
+operational limits are modelled separately as Pyomo constraints.
+
+A commitment describes:
+
+- a **baseline quantity** over time (the contracted or preferred position), and
+- marginal prices for **upwards** and **downwards deviations** from that baseline.
+
+The scheduler converts all provided commitments into terms in the optimization
+objective function so that the solver *minimizes the total deviation cost*
+across the schedule horizon. Absolute physical limitations (for example generator or
+line capacities) are *not* modelled as commitments — those are enforced as
+Pyomo constraints.
+
+Key properties
+--------------
+
+Each Commitment has the following important attributes (high level):
+
+- ``name`` — a logical string identifier (e.g. ``"energy"``, ``"production peak"``).
+- ``device`` — optional: restricts the commitment to a single device; otherwise
+  it is an EMS/site-level commitment.
+- ``index`` — the DatetimeIndex (time grid) on which the series are defined.
+- ``quantity`` — the baseline Series (per slot or per group).
+- ``upwards_deviation_price`` — Series defining marginal cost/reward for upward deviations.
+- ``downwards_deviation_price`` — Series defining marginal cost/reward for downward deviations.
+- ``_type`` — grouping indicator: ``'each'`` or ``'any'`` (see Grouping below).
+
+Sign convention (flows vs stocks)
+--------------------------------
+
+- **Flow commitments** (e.g. power/energy flows):
+
+  - A *positive* baseline quantity denotes **consumption**.
+
+    - Actual > baseline → *upwards* deviation (more consumption).
+    - Actual < baseline → *downwards* deviation (less consumption).
+  - A *negative* baseline quantity denotes **production** (feed-in).
+
+    - Actual less negative (i.e. closer to zero) → *upwards* deviation (less production).
+    - Actual more negative → *downwards* deviation (more production).
+
+- **Stock commitments** (e.g. state of charge for storage):
+
+  - ``quantity`` is the target stock level; deviations above/below that target
+    are priced via the upwards/downwards price series.
+
+Soft vs hard semantics
+----------------------
+
+Commitments in FlexMeasures are **soft** by design: they represent economic
+penalties or rewards that the optimizer considers when building schedules.
+Hard operational constraints (such as physical power limits or strict device
+interlocks) are expressed separately as Pyomo constraints in the scheduling
+model. If a “hard” behaviour is required from a commitment, assign very large
+penalty prices, but prefer modelling non-negotiable limits as Pyomo constraints.
+
+Converting flex-context fields into commitments
+-----------------------------------------------
+
+Users may supply preferences and price fields in the ``flex-context``. The
+scheduler translates the relevant fields into one or more `Commitment` objects
+before calling the optimizer.
+
+Typical translations include:
+
+- tariffs (``consumption-price``, ``production-price``) → an ``"energy"`` FlowCommitment with zero baseline so net consumption/production is priced;
+- peak/excess limits (``site-peak-production``, ``site-peak-production-price``, etc.) → dedicated peak FlowCommitment(s);
+- storage-related fields (``soc-minima``, ``soc-minima-breach-price``, etc.) → StockCommitment(s).
+
+A short example
+---------------
+
+Below is a compact example showing how the scheduler conceptually creates an
+``"energy"`` flow commitment from a (per-slot) tariff:
+
+.. code-block:: python
+
+    from pandas import Series, date_range
+    from flexmeasures.data.models.planning import FlowCommitment
+
+    index = date_range(start="2025-01-01 00:00", periods=24, freq="H")
+    # zero baseline → the asset may consume or produce; deviations are priced.
+    baseline = Series(0.0, index=index)
+
+    # consumption and production tariffs (per kWh)
+    consumption_price = Series(0.20, index=index)  # 0.20 EUR/kWh for consumption
+    production_price = Series(-0.05, index=index)  # -0.05 EUR/kWh reward for production
+
+    energy_commitment = FlowCommitment(
+        name="energy",
+        index=index,
+        quantity=baseline,
+        upwards_deviation_price=consumption_price,
+        downwards_deviation_price=production_price,
+        _type="each"
+    )
+
+The scheduler sets up such commitments (site-level and device-level) and, together with any prior commitments, hands them to the linear optimizer.
+
+Examples (commitments commonly derived from flex-context)
+--------------------------------------------------------
+
+The examples below map the most common `flex-context` semantics to the
+commitments the scheduler constructs.
+
+1. **Energy (tariff)**
+
+   - *Fields used*: ``consumption-price``, ``production-price``.
+   - *Commitment*: Flow commitment named ``"energy"`` with zero baseline and
+     the two price series as upwards/downwards deviation prices.
+
+2. **Peak consumption**
+
+   - *Fields used*: ``site-peak-consumption`` (baseline) and ``site-peak-consumption-price`` (upwards-deviation price); the downwards price is set to ``0``.
+   - *Commitment*: Flow commitment named ``"consumption peak"``; positive baseline
+     values denote the prior consumption peak associated with sunk costs, and the upwards price penalises going beyond that baseline.
+
+3. **Peak production / peak feed-in**
+
+   - *Fields used*: ``site-peak-production`` (baseline) and ``site-peak-production-price`` (downwards-deviation price); the upwards price is set to ``0``.
+   - *Commitment*: Flow commitment named ``"production peak"``; negative baseline
+     values denote the prior production peak associated with sunk costs, and the downwards price penalises going beyond that baseline.
+
+4. **Consumption capacity**
+
+   - *Fields used*: ``site-consumption-capacity`` (baseline), and ``site-consumption-breach-price`` (upwards-deviation price); the downwards price is set to ``0``.
+   - *Commitment*: Flow commitment named ``"consumption breach"``; positive baseline
+     values denote the allowed consumption limit and the upwards price penalises going
+     beyond that limit.
+
+5. **Production capacity**
+
+   - *Fields used*: ``site-production-capacity`` (baseline) and ``site-production-breach-price`` (downwards-deviation price); the upwards price is set to ``0``.
+   - *Commitment*: Flow commitment named ``"production breach"``; negative baseline
+     values denote the allowed production limit and the downwards price penalises going
+     beyond that limit.
+
+6. **SOC minima / maxima (storage preferences)**
+
+   - *Fields used*: ``soc-minima``, ``soc-minima-breach-price``, ``soc-maxima`` and ``soc-maxima-breach-price``.
+   - *Commitment*: StockCommitment(s) that price deviations below minima or
+     above maxima. Hard storage capacities are set through ``soc-min`` and ``soc-max`` instead and are modelled as Pyomo constraints.
+
+7. **Power bands per device**
+
+   - *Fields used*: ``consumption-capacity`` and ``production-capacity`` (baselines), ``consumption-breach-price`` (upwards-deviation price, with 0 downwards) and ``production-breach-price`` (downwards-deviation price, with 0 upwards).
+   - *Commitment*: FlowCommitment with either baseline and corresponding prices.
+
+Grouping across time and devices
+--------------------------------
+
+- ``_type == 'each'``: penalise deviations per time slot (default for time series).
+- ``_type == 'any'``: treat the whole commitment horizon as one group (useful
+  for peak-style penalties where only the maximum over the window should be
+  counted).
+
+.. note::
+
+   Near-term feature: support for **grouping over devices** is planned and
+   documented here. When enabled, grouping over devices lets you express
+   soft constraints that aggregate deviations across a set of devices,
+   for example, an intermediate capacity constraint from a feeder shared by a group of devices (via **flow commitments**), or multiple power-to-heat devices that feed a shared thermal buffer (via **stock commitments**).
+
+
+Advanced: mathematical formulation
+----------------------------------
+
+For a compact formulation of how commitments enter the optimization problem, see :ref:`storage_device_scheduler`.

documentation/concepts/device_scheduler.rst

@@ -5,12 +5,13 @@ Storage device scheduler: Linear model
 
 Introduction
 --------------
-This generic storage device scheduler is able to handle an EMS with multiple devices, with various types of constraints on the EMS level and on the device level,
-and with multiple market commitments on the EMS level.
+This generic storage device scheduler is able to handle a site with multiple devices, with various types of constraints on the site level and on the device level,
+and with multiple market commitments on the site level.
 
 A typical example is a house with many devices. The commitments are assumed to be with regard to the flow of energy to the device (positive for consumption, negative for production). In practice, this generic scheduler is used in the **StorageScheduler** to schedule a storage device.
 
 The solver minimizes the costs of deviating from the commitments.
+For a more detailed explanation of commitments in FlexMeasures, see :ref:`commitments`.
 
 
 
@@ -45,9 +46,9 @@ Symbol                              Variable in the Code
 :math:`\epsilon(d,j)`                 efficiencies                                       Stock energy losses.
 :math:`P_{max}(d,j)`                  device_derivative_max                              Maximum flow of device :math:`d` during time period :math:`j`.
 :math:`P_{min}(d,j)`                  device_derivative_min                              Minimum flow of device :math:`d` during time period :math:`j`.
-:math:`P^{ems}_{min}(j)`              ems_derivative_min                                 Minimum flow of the EMS during time period :math:`j`.
-:math:`P^{ems}_{max}(j)`              ems_derivative_max                                 Maximum flow of the EMS during time period :math:`j`.
-:math:`Commitment(c,j)`               commitment_quantity                                Commitment c (at EMS level) over time step :math:`j`.
+:math:`P^{ems}_{min}(j)`              ems_derivative_min                                 Minimum flow of the site's grid connection point during time period :math:`j`.
+:math:`P^{ems}_{max}(j)`              ems_derivative_max                                 Maximum flow of the site's grid connection point during time period :math:`j`.
+:math:`Commitment(c,j)`               commitment_quantity                                Commitment c (at site level) over time step :math:`j`.
 :math:`M`                             M                                                  Large constant number, upper bound of :math:`Power_{up}(d,j)` and :math:`|Power_{down}(d,j)|`.
 :math:`D(d,j)`                        stock_delta                                        Explicit energy gain or loss of device :math:`d` during time period :math:`j`.
 ================================ ================================================ ==============================================================================================================  
@@ -58,8 +59,8 @@ Variables
 ================================ ================================================ ==============================================================================================================  
 Symbol                              Variable in the Code                           Description
 ================================ ================================================ ==============================================================================================================  
-:math:`\Delta_{up}(c,j)`              commitment_upwards_deviation                       Upwards deviation from the power commitment :math:`c` of the EMS during time period :math:`j`.
-:math:`\Delta_{down}(c,j)`            commitment_downwards_deviation                     Downwards deviation from the power commitment :math:`c` of the EMS during time period :math:`j`.
+:math:`\Delta_{up}(c,j)`              commitment_upwards_deviation                       Upwards deviation from the power commitment :math:`c` of the site during time period :math:`j`.
+:math:`\Delta_{down}(c,j)`            commitment_downwards_deviation                     Downwards deviation from the power commitment :math:`c` of the site during time period :math:`j`.
 :math:`\Delta Stock(d,j)`                           n/a                                  Change of stock of device :math:`d` at the end of time period :math:`j`.
 :math:`P_{up}(d,j)`                   device_power_up                                    Upwards power of device :math:`d` during time period :math:`j`.
 :math:`P_{down}(d,j)`                 device_power_down                                  Downwards power of device :math:`d` during time period :math:`j`.

documentation/index.rst

@@ -189,6 +189,7 @@ In :ref:`getting_started`, we have some helpful tips how to dive into this docum
     concepts/flexibility
     concepts/data-model
     concepts/security_auth
+    concepts/commitments
     concepts/device_scheduler
 
 

flexmeasures/data/models/planning/__init__.py

@@ -2,8 +2,9 @@
 
 from dataclasses import dataclass, field
 from datetime import datetime, timedelta
+from tabulate import tabulate
 from typing import Any, Dict, List, Type, Union
-
+from collections.abc import Iterable
 import pandas as pd
 from flask import current_app
 
@@ -280,6 +281,7 @@ class Commitment:
 
     name: str
     device: pd.Series = None
+    device_group: pd.Series = None
     index: pd.DatetimeIndex = field(repr=False, default=None)
     _type: str = field(repr=False, default="each")
     group: pd.Series = field(init=False)
@@ -339,10 +341,67 @@ def __post_init__(self):
             "downwards deviation price"
         )
         self.group = self.group.rename("group")
+        self._init_device_group()
+
+    def _init_device_group(self):
+        # EMS-level commitment
+        if self.device is None:
+            self.device_group = pd.Series({"EMS": 0}, name="device_group")
+            return
+
+        # Extract device universe
+        if isinstance(self.device, pd.Series):
+            devices = extract_devices(self.device)
+        else:
+            devices = [self.device]
+
+        devices = list(devices)
+
+        # Default: one group per device (backwards compatible)
+        if self.device_group is None:
+            self.device_group = pd.Series(
+                range(len(devices)), index=devices, name="device_group"
+            )
+        else:
+            # Validate custom grouping
+            missing = set(devices) - set(self.device_group.index)
+            if missing:
+                raise ValueError(
+                    f"device_group missing assignments for devices: {missing}"
+                )
+            self.device_group = self.device_group.loc[devices]
+            self.device_group.name = "device_group"
+
+    def pretty_print(self):
+        """
+        Pretty-print a list of FlowCommitment objects as tabulated pandas DataFrames.
+
+        For each FlowCommitment, a DataFrame indexed by time is created containing
+        the commitment name, device values, group index, quantity, and any available
+        upward or downward deviation prices. Each commitment is printed separately
+        in a readable table format, making this function suitable for debugging,
+        logging, and interactive inspection.
+        """
+        df = self.to_frame()
+        df = pd.DataFrame(index=df.device.index)
+
+        df["commitment"] = self.name
+        df["device"] = self.device
+        df["group"] = self.group
+        df["quantity"] = self.quantity
+
+        if hasattr(self, "upwards_deviation_price"):
+            df["up_price"] = self.upwards_deviation_price
+
+        if hasattr(self, "downwards_deviation_price"):
+            df["down_price"] = self.downwards_deviation_price
+
+        if not df.empty:
+            print(tabulate(df, headers=df.columns, tablefmt="fancy_grid"))
 
     def to_frame(self) -> pd.DataFrame:
         """Contains all info apart from the name."""
-        return pd.concat(
+        df = pd.concat(
             [
                 self.device,
                 self.quantity,
@@ -353,6 +412,13 @@ def to_frame(self) -> pd.DataFrame:
             ],
             axis=1,
         )
+        # map device → device_group
+        if self.device is not None:
+            df["device_group"] = map_device_to_group(self.device, self.device_group)
+        else:
+            df["device_group"] = 0
+
+        return df
 
 
 class FlowCommitment(Commitment):
@@ -374,3 +440,48 @@ class StockCommitment(Commitment):
 Scheduler.compute_schedule = deprecated(Scheduler.compute, "0.14")(
     Scheduler.compute_schedule
 )
+
+
+def extract_devices(device):
+    """
+    Return a flat list of unique device identifiers from:
+    - scalar device
+    - Series of scalars
+    - Series of iterables (e.g. [0, 1])
+    """
+    if device is None:
+        return []
+
+    if isinstance(device, pd.Series):
+        values = device.dropna().values
+    else:
+        values = [device]
+
+    devices = set()
+    for v in values:
+        if isinstance(v, Iterable) and not isinstance(v, (str, bytes)):
+            devices.update(v)
+        else:
+            devices.add(v)
+
+    return list(devices)
+
+
+def map_device_to_group(device_series, device_group_map):
+    """
+    Map device identifiers to device_group.
+
+    - scalar device → group label
+    - iterable of devices → group label (must be identical)
+    """
+
+    def resolve(v):
+        if isinstance(v, (list, tuple, set)):
+            groups = {device_group_map[d] for d in v}
+            if len(groups) != 1:
+                raise ValueError(f"Devices {v} map to multiple device groups: {groups}")
+            return groups.pop()
+        else:
+            return device_group_map[v]
+
+    return device_series.apply(resolve)