First draft Continuous optimizer by StefanPSchmid · Pull Request #828 · emdgroup/baybe

StefanPSchmid · 2026-06-15T13:36:39Z

creating a new PR, this time on the baybe repo instead of my fork, so you can commit. Kept the previous changes with some slight modifications, and copy your comments here so we can have the discussions here. Code is changed only insignificantly (I know there are still mypy issues for example, I have some questions about your opinions first), I mainly have new thoughts/questions in the comments :)

Removed deprecated surrogate_model property and its warning.

StefanPSchmid · 2026-06-15T13:46:08Z

    )
    """The acquisition function. When omitted, a default is used."""

+    optimizer: OptimizerProtocol | None = field(


Comment from @AVHopp on other PR:
I think allowing None here is currently causing typing problems. I think the patter should actually just be optimizer: OptimizerProtocol which is then automatically being set in the derived classes. That is, the BotorchRecommender would set the default direclty without the check for whether or not this is None, and at some point, we might be able to give a reasonable default working for all kind of search spaces already here.

Comment from @AdrianSosic on other PR:
Was going in the same direction but more critically so:

Strictly speaking, this new argument is already a breaking change (but not saying we don't need it, just needs more refinement)

Agree with @AVHopp: None doesn't make sense – we always need an optimization procedure. Without it, the recommender is non-functional

This refactoring in fact opens up a larger debate: potentially, we can kick the BotorchRecommender altogether, and instead use the BayesianRecommender as a non-abstract class in the future. Reason: so far, the BotorchRecommender encapsulated the specific botorch routines in its body. With the refactoring, this is no longer the case and the part is modularized. That is, we move from inheritance to composition --> a BayesianRecommender takes over the role of the previous BotorchRecommender when it is constructed with the corresponding botorch routines as arguments.

@StefanPSchmid's new thoughts/questions:

I was orienting myself at the acquisition_function , which also can have None at this field and is only set later to a sensible default. I was thinking of doing the same here, i.e. in recommend there would be a new function setup_optimizer that sets a default (depending on the search space type) when the given optimizer is None. The problem from my POV currently is that I am still trying to make it work that when the user gives some arguments to BotorchRecommender for the Optimizer (e.g. sequential_continuous), that they are arguments of the Recommender, and not available at the setup stage (if I understood the code directly).

I think for that matter too, I would be for removing the BotorchRecommender, that would move the dispatching based on searchspace type into BayesianRecommender and that gets then optimized with a specific Optimizer. Then there would also not be this dychtomy where these arguments should be. One thing of which I am very unsure is how this would be handled with backwards compatibility, currently trying to get it to work even when old interface of BotorchRecommender is called (see point above).

So to summarize - I would remove the BotorchRecommender, put the dispatching logic into BayesianRecommender and implement a _setup_acquisition_function style function for the optimizer too. What do you think?

I think dropping the BotorchRecommender is the right thing to do 👍🏼 At least from what I can currently overlook. But we need to ensure backward compatibility here, so we need some deprecation for it.

Regarding the None discussion: I think it really depends on what we expect from the Optimizer object. I think your current idea is that you want to dispatch between a conti/disc/hybrid optimizer depending on the given search space, in which case you'd indeed need a delayed default. However, the alternative is that we simply require the optimizer to be able to handle all three kinds of search spaces. While neither of the simple optimizers fulfills it, we'll later have composite optimizers that do so. For example, the alternating optimizer can handle all spaces (with purely conti/disc just being special cases where the sequence stops immediately after the first subspace). And due to the recursive interface, composite optimizers also fulfill the required type, meaning we could set an appropriate composite recommender as default. Similarly, all hybrid optimizers trivially also support purely conti/disc spaces.

But this doesn't need to be decided now, we can follow up on it later when all puzzle pieces are in place.

Note that I left some thoughts about this in the old PR, explaining a bit the potential role of None (which might however not be accurate now that I see this discussion here 😆)

@AdrianSosic can you elaborate on why this is a breaking change already? Is it only because it is public? Because right now, it can still simply be ignored, right?

Let me see if I get @AdrianSosic's idea correctly and whether we can agree on it:

There are two kinds of optimizers: PureOptimizer which handle exactly one kind of space (probably not user facing) and CompositeOptimizers which combine PureOptimizers (or other CompositeOptimizers?) to handle all kinds of search spaces

We could then always have a CompositeOptimizer as default which knows how to handle all sorts of search spaces

If an optimization procedure is only intended to be used for e.g. discrete search spaces, then this is implemented as PureOptimizer. However, this will then be (automatically) be wrapped into a CompositeOptimizer

I have to say that I am not sure what I think of this idea. While I like it from a code design perspective, it makes me wonder what other implications this would have. For example, wouldn't we then somehow implicitly assume that every search space is hybrid if our recommendation procedure always needs to know how to handle hybrid spaces? Also, for a user wanting to only do some optimization in a discrete space, it might be weird if things are being wrapped into some other object that could then also be used for other spaces.

Currently, I'd thus prefer the delayed setting of the optimizer based on the search space as this seems simpler to me and follows the design of the acquisition function - which makes sense, given that the optimizer is something that is also closely related to it.

@AdrianSosic @StefanPSchmid what did we align on here? Iirc, I think it was "Let's move fast and having the None is fine", is that correct?

I thought we said GradientOptimizer as a default, that will then eventually get replaced by a compositeoptimizer that can handle all three search spaces. That's why I put the gradientOptimizer there for now

StefanPSchmid · 2026-06-15T15:25:32Z

Thread to discuss potential removal of BoTorch Recommender:

Considering allowing None for the Optimizer of the Recommender, I was orienting myself at the acquisition_function , which also can have None at this field and is only set later to a sensible default. I was thinking of doing the same here, i.e. in recommend there would be a new function setup_optimizer that sets a default (depending on the search space type) when the given optimizer is None. The problem from my POV currently is that I am still trying to make it work that when the user gives some arguments to BotorchRecommender for the Optimizer (e.g. sequential_continuous), that they are arguments of the Recommender, and not available at the setup stage (if I understood the code directly).

I think for that matter too, I would be for removing the BotorchRecommender, that would move the dispatching based on searchspace type into BayesianRecommender and that gets then optimized with a specific Optimizer. Then there would also not be this dychtomy where these arguments should be. One thing of which I am very unsure is how this would be handled with backwards compatibility, currently trying to get it to work even when old interface of BotorchRecommender is called (see point above).

So to summarize - I would remove the BotorchRecommender, put the dispatching logic into BayesianRecommender and implement a _setup_acquisition_function style function for the optimizer too. What do you think?

I think @AdrianSosic already stated that he'd support the removal of the BoTorchRecommender. However, my feeling is that this might heavily depend on how the Protocols/Interface change, so we should wait with this decision in my opinion.

AVHopp · 2026-06-22T09:22:37Z

+
+
+@runtime_checkable
+class OptimizerProtocol(Protocol):


Give this a Flag about the supported SearchSpaceType and add validation for this.

…elete Botorch recommender logic

StefanPSchmid · 2026-06-24T16:03:11Z

+                )
+
+        # TODO: Add option for automatic choice once the "settings" PR is merged,
+        #   which ships the necessary machinery


moved from continuous.py to here, since the optimizer should check if it is actually usable (same as checks below/above)

Makes in principle sense that this is the responsibility of the recommender. However, I am asking myself if there is an even better place for checking compatibility - is there an earlier place where we already have access to both the optimizer and the search space to make this check, both working with and without a campaign? If so, we could move this validation maybe up a bit more, but I am genuinly unsure if there is a more suitable position. For now, I'd be fine with keeping the validation here as well, because it might also be the case that trying to bring all of the validation somewhere else is too complex - in particular since the validation for this optimizer here also depends on the setting of the optimizer itself.
Opinions, also from @AdrianSosic on this point?

Yeah, the final point was my main motivation for keeping it there. We can see how it develops

StefanPSchmid · 2026-06-24T16:05:13Z

+    )
+    """The acquisition function optimizer."""
+
+    #TODO: Move fields to respective optimizers


fields moved from BotorchRecommender to BayesianRecommender for now, ultimate goal is to store them in the respective Optimizers where needed; preparation to remove BotorchRecommender (as discussed should be done in this PR, once we have decided on the BayesianRecommender looks)

Reasonable to just have them here for the moment imo

StefanPSchmid · 2026-06-24T16:05:47Z

                raise

+    @override
+    def _recommend_discrete(


dispatching logic moved from BotorchRecommender to BayesianRecommender to prepare for removal of BotorchRecommender as discussed

Just one question: Do we then still need the @OverRide annotations? I do not think that those internal functions specialized to the individual search space types are defined on the level of the PureRecommender, could you verify?

There are functions with that name in the PureRecommender, but they raise NonImplementedErrors

StefanPSchmid · 2026-06-24T16:06:28Z

@@ -0,0 +1,168 @@
+"""Continuous recommendation routines for BotorchRecommender."""


file moved from BotorchRec. level to BayesianRec. level to prepare for BotorchRec. removal

As I understand this as a pure "move", I didn't review this file - just FYI

StefanPSchmid · 2026-06-24T16:06:36Z

@@ -0,0 +1,142 @@
+"""Discrete recommendation routines for BotorchRecommender."""


file moved from BotorchRec. level to BayesianRec. level to prepare for BotorchRec. removal

As I understand this as a pure move, I didn't review this file - FYI.

StefanPSchmid · 2026-06-24T16:06:45Z

@@ -0,0 +1,252 @@
+"""Hybrid recommendation routines for BotorchRecommender."""


file moved from BotorchRec. level to BayesianRec. level to prepare for BotorchRec. removal

See my other comments.

AVHopp · 2026-06-26T06:49:05Z

@StefanPSchmid could it be the case that your pre-commit hook is not activated/configured incorrectly? I'm asking since both the Lint and the pre-commit checks are failing which should not happen, so would be great if you could check that.

AVHopp

Some intermediate comments

AVHopp · 2026-06-26T06:51:09Z

+    def __call__(
+        self,
+        batch_size: int,
+        acquisition_function: Optimand,


I appreciate the type alias but I think the name Optimand is very confusing, I would not know what is meant by that. I however also struggle to find a better alternative since the natural term (Objective) is already occupied. Do you have other ideas? Because for now, I have to say that I'd prefer the raw type hint over a name that is not clear.

After some brainstorming with @fabianliebig, how about Score? OptimizableScore?

Or maybe something like CandidateScorer?

AVHopp · 2026-06-26T06:52:19Z

+
+    n_restarts: int = field(validator=[instance_of(int), gt(0)], default=10)
+    """Number of times gradient-based optimization is restarted from different initial
+    points. **Does not affect purely discrete optimization**.


Suggested change

points. **Does not affect purely discrete optimization**.

points.

Since we now cleanly separate these two kinds of optimization, this comment is no longer required

Same applies to some other places here in the code and the docstrings

AVHopp · 2026-06-26T07:00:27Z

+                )
+
+        # TODO: Add option for automatic choice once the "settings" PR is merged,
+        #   which ships the necessary machinery


Makes in principle sense that this is the responsibility of the recommender. However, I am asking myself if there is an even better place for checking compatibility - is there an earlier place where we already have access to both the optimizer and the search space to make this check, both working with and without a campaign? If so, we could move this validation maybe up a bit more, but I am genuinly unsure if there is a more suitable position. For now, I'd be fine with keeping the validation here as well, because it might also be the case that trying to bring all of the validation somewhere else is too complex - in particular since the validation for this optimizer here also depends on the setting of the optimizer itself.
Opinions, also from @AdrianSosic on this point?

AVHopp · 2026-06-26T07:02:02Z

+    )
+    """The acquisition function optimizer."""
+
+    #TODO: Move fields to respective optimizers


Reasonable to just have them here for the moment imo

AVHopp · 2026-06-26T07:03:53Z

                raise

+    @override
+    def _recommend_discrete(


Just one question: Do we then still need the @OverRide annotations? I do not think that those internal functions specialized to the individual search space types are defined on the level of the PureRecommender, could you verify?

AVHopp · 2026-06-26T07:04:50Z

@@ -0,0 +1,168 @@
+"""Continuous recommendation routines for BotorchRecommender."""


As I understand this as a pure "move", I didn't review this file - just FYI

AVHopp · 2026-06-26T07:05:13Z

@@ -0,0 +1,142 @@
+"""Discrete recommendation routines for BotorchRecommender."""


As I understand this as a pure move, I didn't review this file - FYI.

AVHopp · 2026-06-26T07:05:25Z

@@ -0,0 +1,252 @@
+"""Hybrid recommendation routines for BotorchRecommender."""


See my other comments.

StefanPSchmid and others added 2 commits June 15, 2026 15:33

First draft Continuous optimizer

0de44da

Remove deprecated surrogate_model property that was there in 0.14

6cbdd39

Removed deprecated surrogate_model property and its warning.

StefanPSchmid commented Jun 16, 2026

View reviewed changes

StefanPSchmid requested a review from fabianliebig June 17, 2026 16:08

AVHopp reviewed Jun 22, 2026

View reviewed changes

Comment thread baybe/recommenders/pure/bayesian/botorch/optimizers/base.py Outdated

AVHopp reviewed Jun 22, 2026

View reviewed changes

OptimizerProtocol SearchSpaceType

5236c3b

AVHopp assigned StefanPSchmid Jun 23, 2026

StefanPSchmid added 3 commits June 24, 2026 17:53

Copy optimization logic files on BayesianRecommender logic, prepare d…

0eed2f9

…elete Botorch recommender logic

Resolve mypy issues

355ce33

Resolve mypy issues, prepare deletion of BotorchRec

9cb1d30

StefanPSchmid commented Jun 25, 2026

View reviewed changes

AVHopp reviewed Jun 26, 2026

View reviewed changes

StefanPSchmid added 5 commits June 26, 2026 19:24

Exchange BotorchRecommender with BayesianRecommender throughout repo

e3d2983

Linting and comment improvements

d34f290

Exchange BotorchRecommender with BayesianRecommender throughout repo

3c44679

Modify init files to import BayesianRecommenders

5d12775

Deprecate BotorchRecommender

ccad3c8

		@@ -0,0 +1,168 @@
		"""Continuous recommendation routines for BotorchRecommender."""

		@@ -0,0 +1,142 @@
		"""Discrete recommendation routines for BotorchRecommender."""

		@@ -0,0 +1,252 @@
		"""Hybrid recommendation routines for BotorchRecommender."""

	points. Does not affect purely discrete optimization.
	points.

Uh oh!

Conversation

StefanPSchmid commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AVHopp commented Jun 26, 2026

Uh oh!

AVHopp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

StefanPSchmid Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

StefanPSchmid commented Jun 15, 2026 •

edited

Loading

StefanPSchmid Jun 26, 2026 •

edited

Loading