Example solution to adapting HP bounds and defaults dynamically

[thijssnelleman]

#412

This is a proposed solution to this issue; Hyperparameters have properties that are not updated by changing the values. By adapting the updating functionality, it first tests if there already is a value present, and if so, verifies what needs to be recalculated.

1. Assumes that in the initialisation all values are set exactly once, otherwise the update will be triggered and may cause issues when properties are accessed that do not exist yet
2. Shares a lot of code with that found in the init function. Would be better if this would be settled in one function rather than spread over two
3. Needs to be adapted for each HP type. Need to find out if we can do this more generally (For example, what can be done in the parent Hyperparameter class? Not everything but maybe something like default value?)

Have written a partial pytest to verify the behaviour, seems to work just fine with this suggested solution

[eddiebergman]

Might be a lot easier to just reconstruct the whole hyperparameter, and then steal it's values. This means you don't have to duplicate validation, and you can use this method with minor adjustments on everywhere.

Read up on dataclasses. There is a method signature .replace() -> Self which will basically do this, but it returns a new instance.

Just take special note that we define a custom __init__ for dataclasses, which is not normally what you should do. This means you don't get all their benefits. They exist to be backwards compatbile with the public API

def __setattr__(self, name: str, value: Any):
    init_params: dict[str, Any] = # get init params
    if name not in init_params:
        raise ValueError("Can't set attribute {name}, must be one passed to init.") # Something better error message than this
    init_params[name] = value
    
    # This raises if invalid
    new_instance = self.__class__(**init_params)
    self.__dict__ = new_instance.__dict__ # Unsure if this works exactly

---

[thijssnelleman]

That is actually a good idea, although it could be inefficient in some cases. I went with a slightly different strategy; the setattr has now a different workflow if a value is not being set from init. The check is quite ugly, but works. Instead of using replace, I am just calling self.init again. Although current tests are basics, they are passing, wondering what @eddiebergman thinks of this solution.

[thijssnelleman]

@eddiebergman

That is actually a good idea, although it could be inefficient in some cases. I went with a slightly different strategy; the setattr has now a different workflow if a value is not being set from init. The check is quite ugly, but works. Instead of using replace, I am just calling self.init again. Although current tests are basics, they are passing, wondering what @eddiebergman thinks of this solution.

Does this solutions stray too far from your initial idea?

[thijssnelleman]

@mfeurer Reminder for this PR

[karibbov]

Summary:
This PR:

1. Prevents direct (assignment) attribute setting from anywhere except from within any function named __init__.
2. Then allows only the attributes that has the same name as an initialization parameter, by reinitializing from those attributes. This implies all the initialization parameters must be saved without modification at every HP.__init__ implementation. This is not as strict of a condition as it seems, since all HPs are dataclasses. However I think we should document it somewhere visible since we always have a custom __init__ function.

Suggestion

1. Mainly documetation (see comments)
2. Can we include an example test for all the other init parameters (attributes) being the same before and after an attribute is reset? This is to catch the case when some HP.__init__ saves a modified version of the init parameter as an attribute.

[Copilot]

Pull request overview

Implements a first-pass solution for dynamically updating hyperparameter bounds/defaults by re-initializing hyperparameter instances on attribute assignment, addressing ConfigSpace issue #412 (online updates/search space shrinking).

Changes:

• Add Hyperparameter.__setattr__ logic to restrict post-init mutations to __init__ parameters and to re-run __init__ when such attributes are updated.
• Standardize several “illegal settings” error messages to “Illegal value(s) for Hyperparameter …” and adjust tests accordingly.
• Add a new pytest covering runtime updates for integer/float/categorical/ordinal hyperparameters and verifying sampling stays in-bounds.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/ConfigSpace/hyperparameters/hyperparameter.py	Adds update-via-reinit behavior in __setattr__ and fixes a typo in an error message.
src/ConfigSpace/hyperparameters/uniform_integer.py	Updates illegal-bounds error message text.
src/ConfigSpace/hyperparameters/uniform_float.py	Updates illegal-bounds error message text.
src/ConfigSpace/hyperparameters/normal_integer.py	Updates error message text for invalid params and bounds.
src/ConfigSpace/hyperparameters/normal_float.py	Updates error message text for invalid params and bounds.
src/ConfigSpace/hyperparameters/beta_integer.py	Updates illegal-bounds error message text.
src/ConfigSpace/hyperparameters/beta_float.py	Updates illegal-bounds error message text.
test/test_hyperparameters.py	Updates error-message matching and adds a new test validating runtime hyperparameter updates.
docs/reference/hyperparameters.md	Documents the new update behavior and aligns an example snippet’s error message.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The new assertions rely on stochastic sampling (sample_configuration) without seeding the ConfigurationSpace RNG. This can lead to rare but real test flakiness (especially for the weighted categorical count comparisons). Seed the space (e.g., space.seed(…)) or pass a deterministic RNG where supported so the test outcome is reproducible.

[Copilot]

The test matches the full error string including an exact tuple of __init__ parameters. This is brittle (it will fail if the signature changes, parameter order changes, or if you later decide to omit 'self' from the message). Prefer matching a stable substring/regex (e.g., that the attribute name is not allowed) rather than the entire signature dump.

Suggested change

	match=re.escape(
	"Can only set parameters passed to self.__init__. 'size' is not one of: ('self', 'name', 'lower', 'upper', 'default_value', 'log', 'meta')",
	),
	):
	space["b"].size = 1_000_000 # Not an init parameter
	with pytest.raises(
	ValueError,
	match=re.escape(
	"Can only set parameters passed to self.__init__. 'non_existiting_attribute' is not one of: ('self', 'name', 'lower', 'upper', 'default_value', 'log', 'meta')",
	match=r"Can only set parameters passed to self\.__init__\..*'size' is not one of:",
	):
	space["b"].size = 1_000_000 # Not an init parameter
	with pytest.raises(
	ValueError,
	match=(
	r"Can only set parameters passed to self\.__init__\..*"
	r"'non_existiting_attribute' is not one of:"

[Copilot]

__setattr__ calls inspect.stack() on every attribute assignment to detect initialization. This is very expensive (it collects full stack frame info) and also brittle: any attribute assignment performed from a helper called inside __init__ (or from a dataclass __post_init__) won’t see the caller as __init__ and will incorrectly trigger the re-init path. Consider using an explicit internal flag (e.g., _is_initializing / _is_reinitializing) or a sentinel like _initialized in __dict__ instead of call-stack inspection.

[Copilot]

Re-initializing the existing instance via self.__init__(**init_params) is not transactional: many hyperparameter __init__ methods assign to attributes (e.g., lower/upper) before validation, so if validation raises, the object may be left partially mutated/inconsistent even though the setter raised. To avoid corrupting the instance, restore the previous state on failure (e.g., snapshot self.__dict__ and roll back in except) or build a new instance and swap state only after successful construction.

Suggested change

	self.__init__(**init_params) # Reinitialise
	previous_state = self.__dict__.copy()
	try:
	self.__init__(**init_params) # Reinitialise
	except Exception:
	self.__dict__.clear()
	self.__dict__.update(previous_state)
	raise

[Copilot]

init_params extraction via self.__init__.__code__.co_varnames[:co_argcount] includes 'self' and ignores keyword-only parameters (it doesn’t consider co_kwonlyargcount). This will make the allowlist/error message awkward and can prevent updating keyword-only constructor args in subclasses. Using inspect.signature(self.__init__).parameters (and filtering out 'self') is more robust; also the annotation tuple[str] should be tuple[str, ...] if you keep it as a tuple of names.

[Copilot]

The admonition title contains spaces but isn’t quoted. Elsewhere in the docs, titled admonitions use quotes (e.g., !!! tip "New in 1.1!"), and unquoted titles with spaces may not parse/render as intended depending on the Markdown extension configuration. Use the quoted title form (or omit the title entirely) for consistency and correct rendering.

Suggested change

	!!! tip Updating Hyperparameter properties
	!!! tip "Updating Hyperparameter properties"