`SubspaceDiscrete` Refactor — Problem Statement

[AdrianSosic]

Overview

SubspaceDiscrete is the core class representing discrete search spaces in BayBE. Its current design has accumulated several structural problems that limit scalability, maintainability, and backend flexibility. This issue describes the problems motivating a comprehensive refactor.

Implementation: See the dev branch PR for the design and step-by-step implementation plan.

---

Problems

Five concrete problems motivate this refactor. Each is tracked as a sub-issue:

1. Unenforced invariants — Four independent attributes must remain mutually consistent, with no structural enforcement. → Unenforced search space invariants #794
2. Backend lock-in — Pandas is hardcoded; Polars support is partial and inconsistent. → Dataframe backend lock-in #795
3. Eager materialization — The full search space is materialized in RAM at construction time, even for millions/billions of candidates. → Eager search space materialization #796
4. Static encoded-columns hack — comp_rep[self.comp_rep.columns] is a fragile defensive anchor with no structural justification. → Static encoded-columns hack in transform #797
5. No mechanism to navigate large/infinite spaces — Recommenders have no way to request a tractable subset or construct candidates from an infinite space. → No mechanism to navigate large/infinite spaces #798

What is NOT a problem

The existing allow_recommending_* flags and toggle_discrete_candidates on Campaign are not problems. They provide legitimate trajectory/history-based control. Their implementation may change, but their semantics and placement on Campaign are correct.