Transport improvements for 2025-W44

[khaeru]

#430 ← previous PR | following PR → #471

• iiasa/message_data#626
• Implement runs using ExogenousEmissionPrice.
• Add a SLURM batch script for running mix-models transport on UniCC.
  • Small adjustments and clean-ups to other code to work in this environment.
• New utility class .sdmx.StructureFactory.

New scenario labels

The PR extends the set of MESSAGEix-Transport scenario codes or labels. Previously, for each SSP, there were 3, for instance:

• SSP2 —a baseline.
• SSP2 tax —an arbitrary, exponentially increasing price, same in all regions.
• SSP2 exo price —using PRICE_EMISSION data drawn from base model scenarios.

These are now extended in the following 2 ways:

1. Materials-enabled scenarios with an M prefix, e.g. M SSP2, M SSP2 tax, etc. These activate the code in message_ix_models.model.transport.material that adds/modifies data for MESSAGEix-Materials integration.

2. Full set of … exo price scenarios.

   Instead of a single, handpicked source scenario for emission price trajectories, now every available file in data/transport/R12/price-emission/*.csv is selectable. The different price emission source scenarios are now identified by a short suffix like a1b2, that is a hash of the (unique) full scenario URL.

   So for instance SSP2 exo price is now SSP2 exo price 2e17, still using prices from "SSP_SSP2_v5.3.1/SSP2 - Low Emissions#2". Likewise:

   • LED-SSP2 exo price → LED-SSP2 exo price 1aa5, prices from "SSP_LED_v5.3.1/SSP2 - Very Low Emissions#2".
   • SSP1 exo price → SSP1 exo price 4c35, prices from "SSP_SSP1_v5.3.1/SSP1 - Low Emissions#2"
   • SSP3 exo price → SSP3 exo price c536, prices from "SSP_SSP3_v5.3.1/baseline_1000f#1"
   • SSP4 exo price → SSP4 exo price 853b, prices from "SSP_SSP4_v5.3.1/SSP4 - Low Overshoot#2"
   • SSP5 exo price → SSP5 exo price 6321, prices from "SSP_SSP5_v5.3.1/SSP5 - Low Overshoot#2"

   These additions are available for (a) all the SSPs and (b) all the DIGSY-… scenarios, since we are using the latter in CircEUlar runs for 2025-W48 #462 for CircEUlar.

In short, to run any of these scenarios: edit .github/workflows/transport.yaml as usual. Where one previously would have used the labels like SSP2, now one can also/instead use labels like:

• M SSP2 exo price 2e17: SSP2, materials enabled, exogenous price as mentioned above for this short hash.
• DIGSY-BEST-C exo price 2e17: DIGSY input data, SSP2, no materials (because no M prefix), same price.

As before, the IIASA_ECE_CL_TRANSPORT_SCENARIO(1.3.0).xml file (note the new version #) can be inspected/searched to check exactly which settings will be applied for a given label.

Other notes for usage

• CircEUlar runs for 2025-W48 #462 is based on this branch.
• As mentioned previously in Slack, not all of the visible base model scenarios have a 'complete' (distinct for all nodes and periods). For SSP2 in particular, the files that do are:
  • "SSP_SSP2_v5.3.1/SSP2 - Low Emissions#2" = 2e17
  • "SSP_SSP2_v5.3.1/SSP2 - Medium Emissions_a#2" = 4a33
  • "SSP_SSP2_v5.3.1/NPIREF_price_cap_5$_bkp#1" = 5cad
  • "SSP_SSP2_v5.3.1/INDC2030i_SSP2 - Low Emissions#1" = 3888
• If the criteria is relaxed, i.e. n=World prices are okay, there is a larger set of about 22 different inputs.
• For the PREMISE pathways project, any 2 of the above plus an SSP2 baseline could be used to give results with different policy responses.

How to review

• Note the CI checks all pass.

PR checklist

• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update doc/whatsnew.

[codecov]

Codecov Report

❌ Patch coverage is 97.75281% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.3%. Comparing base (bfeba06) to head (5cf4924).
⚠️ Report is 472 commits behind head on main.

Files with missing lines	Patch %	Lines
message_ix_models/report/plot.py	94.3%	5 Missing ⚠️
message_ix_models/cli.py	66.6%	2 Missing ⚠️
message_ix_models/model/transport/operator.py	33.3%	2 Missing ⚠️
message_ix_models/model/transport/plot.py	96.0%	2 Missing ⚠️
message_ix_models/model/workflow.py	80.0%	2 Missing ⚠️
message_ix_models/model/buildings/rc_afofi.py	0.0%	1 Missing ⚠️
message_ix_models/model/transport/check.py	97.8%	1 Missing ⚠️
message_ix_models/model/transport/report.py	97.0%	1 Missing ⚠️
message_ix_models/util/scenarioinfo.py	85.7%	1 Missing ⚠️
message_ix_models/util/sdmx.py	94.4%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (bfeba06) and HEAD (5cf4924). Click for more details.

HEAD has 529 uploads less than BASE

Flag	BASE (bfeba06)	HEAD (5cf4924)
	544	15

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #447     +/-   ##
=======================================
- Coverage   77.4%   70.3%   -7.1%     
=======================================
  Files        286     286             
  Lines      25460   23106   -2354     
=======================================
- Hits       19719   16265   -3454     
- Misses      5741    6841   +1100     

Files with missing lines	Coverage Δ
message_ix_models/__init__.py	100.0% <100.0%> (ø)
message_ix_models/model/emissions.py	98.7% <100.0%> (-0.4%)	⬇️
message_ix_models/model/transport/__init__.py	100.0% <100.0%> (ø)
message_ix_models/model/transport/build.py	98.3% <100.0%> (+1.9%)	⬆️
message_ix_models/model/transport/config.py	100.0% <100.0%> (+0.3%)	⬆️
message_ix_models/model/transport/data.py	98.9% <100.0%> (-0.6%)	⬇️
message_ix_models/model/transport/demand.py	100.0% <ø> (ø)
message_ix_models/model/transport/key.py	100.0% <100.0%> (ø)
message_ix_models/model/transport/material.py	100.0% <100.0%> (ø)
message_ix_models/model/transport/passenger.py	96.6% <100.0%> (ø)
... and 47 more

... and 17 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[khaeru]

The codecov/project failure/coverage drop is due to some .model.material tests that were stalling, leading to the GitHub Actions jobs that would hit the 6-hour time limit. I addressed this by skipping the tests, but this is a temporary mitigation, and will be addressed in a future PR. I've discussed this with @macflo8 and will file a separate issue with further details.