Multi-fidelity surrogate models#721
Open
jpenn2023 wants to merge 49 commits into
Open
Conversation
jpenn2023
requested review from
AVHopp,
AdrianSosic and
Scienfitz
as code owners
jpenn2023
force-pushed
the
dev-mfbo-main-surrogate-models
branch
from
d062293 to
8ed0f2a
Compare
jpenn2023
commented
Feb 3, 2026
jpenn2023
commented
Feb 3, 2026
jpenn2023
commented
Feb 3, 2026
jpenn2023
commented
Feb 3, 2026
AdrianSosic
force-pushed
the
dev-mfbo-main-surrogate-models
branch
from
ccac1f8 to
178772a
Compare
jpenn2023
commented
Feb 5, 2026
AVHopp
reviewed
Feb 6, 2026
AVHopp
left a comment
AVHopp
left a comment
Collaborator
There was a problem hiding this comment.
Some comments on everything but the multi_fidelity.py file, as I guess that we will discuss this today in a bit more detail.
jpenn2023
force-pushed
the
dev-mfbo-main-surrogate-models
branch
from
2e64082 to
8e8547b
Compare
Collaborator
|
Hey @jpenn2023, I just wanted to have a look at your PR but then noticed that something went wrong with your rebase. I know you asked me about the weird diff shown on Github (and yes, that is generally still a problem), but it turns out that your diff is wrong for a different reason – namely because you somehow rebased your commits together with an entire bunch of other commits 😬 Can you quickly fix it and ping me once ready for review? |
jpenn2023
force-pushed
the
dev-mfbo-main-surrogate-models
branch
from
d96cf75 to
2c513b0
Compare
AdrianSosic
force-pushed
the
dev-mfbo-main-surrogate-models
branch
from
2c513b0 to
754d5da
Compare
AdrianSosic
force-pushed
the
dev-mfbo-main-surrogate-models
branch
from
754d5da to
563b86a
Compare
AdrianSosic
force-pushed
the
dev-mfbo-main-surrogate-models
branch
from
563b86a to
88db97f
Compare
Collaborator
|
Hey @jpenn2023, have a look, this should now give you a clean picture of your PR content, right? |
Collaborator
Author
Hi @AdrianSosic. Yes, this view looks right to me. |
Scienfitz
requested changes
Mar 11, 2026
Scienfitz
left a comment
Scienfitz
left a comment
Collaborator
There was a problem hiding this comment.
my biggest question is why the concepts of TL and MF are now deedply intertwined by using a single searchspace property for combining them isntead of two properties characterizing the searchspaces task / fidelity character
AVHopp
reviewed
Mar 13, 2026
AVHopp
left a comment
AVHopp
left a comment
Collaborator
There was a problem hiding this comment.
Some more thoughts from my end
Collaborator
|
@jpenn2023 it seems like the continuous search space example is currently failing which is why the doc building does not run (just FYI as reading the doc building error logs can be a bit annoying) |
AVHopp
requested changes
Mar 27, 2026
Collaborator
There was a problem hiding this comment.
I've found several comments that were marked as resolved without them actually being addressed. Please ONLY resolve comments when the changes are actually visible and have been pushed. Also, it would be great if you could give context when resolving a comment. For example, I asked an explicit question at one point and the comment was just being resolved without providing an answer and without changing the corresponding code.
EDIT: Whoopsie, this was on me. Ignore :)
Previously, it was able to have a search space with multiple fidelity/task parameters as well as having both types of parameters. It was decided not to support this, so validation and according tests were added.
Note that we cannot use a mean factory here since the corresponding BoTorch class for the surrogate does not support handing over a mean_module via `__init__`. The construction of the kernel is left to BoTorch internally, so there is also no need for a kernel factory.
Since we hand over to the BoTorch model in the case of numerical fidelities and to the core GP surrogate in the case of categorical fidelities, we need to perform the corresponding validations in the corresponding cases.
The reason for this generalization is that using the discrete fidelity parameter is intended to trigger the same behavior as the presence of a task parameter - use an index kernel. As both transfer learning and multi-fidelity cases are mutually exclusive, we can unify the logic for both cases in a single method.
…eters Extends `_BayBETaskKernelFactory` to accept fidelity parameter kinds alongside task parameters. Also generalizes validators from task-only to any index-type
As the factory now handles both task and categorical fidelity kernels, the previous name was misleading.
This follows the same pattern as the auto-dispatch of the acquisition function by changing the field to allow `None`. The correct model is then selected upon the first call to `get_surrogate` since this is when the search space is available
This generalizes the previous behavior which only used LOO for search spaces with a TaskParameter.
Previously, the fit context was only validate within the surrogates `_fit` method. As this is called only after caching the surrogate, it was possible to create broken surrogate caches
This was deleted by accident on a merge.
Previously, the default surrogate model was lazily chosen during the first use. This causes issues if the same recommender is used, but a search space requiring a diffferent surrogate model is used, as the surrogate model was not re-dispatched. This changes this behavior by memorizing the which surrogate model was auto-selected previously and only replace it when a new kind of search space requiring a differend default surrogate model is used.
Since the ICML kernel factory is only able to handle categorical fidelities, the check needed to be tightened
Previously, search spaces that only contained task or fidelity parameters were piped through until they reached BoTorch normalization. There, they caused an onimous error about the `indices` list being empty. This is now checked earlier and returns a more informative error message.
Variant A of removing the explicit surrogate class for Multi Fidelity models. This approach now dispatches to the corresponding surrogate class based from botorch on the model-class inside the `_fit` method. That is, we do not recreate the botorch model but simply use it.
This reverts commit 6167b93.
AVHopp
force-pushed
the
dev-mfbo-main-surrogate-models
branch
from
6167b93 to
dbe5a3d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding multi-fidelity properties for the SearchSpace class and multi-fidelity Gaussian process classes with the required fidelity kernels.