Skip to content

Replace period_type enum with ISO 8601 duration-driven period handling #136

@turban

Description

@turban

Problem

period_type is a predefined string enum ("hourly", "daily", "monthly", "yearly") that is used as a branching key in at least five separate modules. Adding any new granularity — bi-weekly (P2W), six-hourly (PT6H), dekadal (P10D) — requires touching core code in multiple places. A custom dataset with an unsupported period type will hit an "unsupported period_type" error at runtime.

The enum already has a parallel ISO 8601 representation: extents.temporal.resolution (e.g. P1D, P1M, PT1H) was introduced in time.py alongside _iso_step_to_approx_hours. Both fields exist in the codebase with no single source of truth.

Where period_type is used today

File Usage
shared/time.py datetime_to_period_string, normalize_period_string, numpy_datetime_to_period_string
data_manager/services/downloader.py Chunk size computation
ingestions/sync_engine.py _next_period_start, _default_target_end
ingestions/services.py Period normalization, default end computation
providers/availability.py Availability cutoff computation
data_accessor/services/accessor.py Coverage metadata
processing/resample.py _frequency_to_period_type, _previous_source_period_start
stac/services.py _period_step — converts period_type → ISO step for STAC
Dataset YAMLs era5_land.yaml (×2), chirps3.yaml, worldpop.yaml

What the handling actually does

Most of the branching is string formatting, not logic:

  • Period string truncation — decides how many characters to keep from an ISO datetime ("2020-01-01""2020-01" for monthly). Derivable from ISO duration precision.
  • Period arithmetic — advances one period for sync. Replaceable with dateutil.relativedelta or pandas.DateOffset driven by the parsed duration.
  • Chunk sizing — already handled by _iso_step_to_approx_hours; the period_type-based version is redundant.
  • Default end computation — "what is today for this period type?" Business logic, not a type system.
  • STAC step_period_step() converts period_type → ISO duration; eliminated once ISO is canonical.

Target state

  • extents.temporal.resolution (ISO 8601 duration) is the canonical field in all dataset YAMLs; period_type is removed
  • Period string truncation length is derived mechanically from duration precision (PT1H → 13 chars, P1D → 10, P1M → 7, P1Y → 4)
  • Period arithmetic uses dateutil.relativedelta / pandas.DateOffset driven by the parsed duration — no lookup table of named types
  • _period_step() in stac/services.py is deleted
  • Any dataset with a valid ISO 8601 duration in extents.temporal.resolution works without core changes

Short-term fix (unblocks #129)

Validate extents.temporal.resolution centrally in time.pyresolve_iso_period_step should verify the string is a well-formed ISO 8601 duration using _iso_step_to_approx_hours, logging a warning and returning None if invalid. This is the fix requested in the #129 review and does not touch period_type.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions