[Question] botorch.optim vs botorch.generation?

[zhenlan0426]

Not a bug but not a feature request either. Just trying to understand BoTorch but cannot find anything online or in the doc.

I see two separte classes botorch.optim and botorch.generation. In my mind, these are the same thing, given a acqusition, find the maximizer. In examples, I see people sometimes use optimize_acqusition and sometimes use gen_candidates_scipy. The API is kind of similar. So what is the difference and which one should I use? Thank you!

[esantorella]

optimize_acqf, which is from botorch.optim.optimize, is generally a good way to generate the candidates with the highest acquisition values. (Yep, the naming is confusing.) You can see an example of its usage here.

optimize_acqf usually uses gen_candidates_scipy under the hood. But if optimize_acqf is provided a function argument gen_candidates, it can use that function in place of gen_candidates_scipy. optimize_acqf validates inputs, and handles errors. It also finds the right optimization strategy for the problem; for example, if all of the features are fixed features, then there is no real optimization to be done. So optimize_acqf is an outer function, and gen_candidates_scipy is an inner function that does the computationally intensive work.

For most cases, users should use optimize_acqf and not worry about gen_candidates_scipy.

[zhenlan0426]

Thank you so much for the quick response. It is still not clear to me why we need optimize_acqf on top of gen_candidates_scipy. It seems to me that validating inputs, handling errors, and checking for fixed features is sinple enough that it can be part of gen_candidates_scipy. I will read through the source code for optimize_acqf to see what additional function it adds.

writing a custom replacement to gen_candidates_scipy. It makes sense to me as a stand-alone optimizer. Documentation would benefit if you show how it can be made as inputs to optimize_acqf.

[Balandat]

There is also some other logic in optimize_acqf that gen_candidates_scipy doesn't handle, such as optimizing one-shot acquisition functions such as the one-shot Knowledge Gradient. I guess at some level you're right that all this stuff could potentially be done in something like gen_candidates_scipy as well. But the current design modularizes things somewhat so that it's easy to plug switch out the core numerical optimizer without having to rewrite everything. E.g you may want to use a pytorch optimizer to optimize some of the MC acquisition function if you don't want to use sample-average approximation. Or you may want to use IPOPT or some other advanced nonlinear optimizers that are not available in scipy.

[esantorella]

I think our documentation could benefit from updating on this point, for example for optimization and on writing a custom replacement to gen_candidates_scipy.

[zhenlan0426]

And let me know if I should close this or you prefer to leave it open for documentation.

[zhenlan0426]

optimize_acqf uses gen_candidates_scipy under the hood