Replies: 2 comments 4 replies
-
Likely because it will work very well when it is feasible to enumerate discrete options and optimize the continuous parameters for each. It's worth checking out my new NeurIPS paper (earlier workshop version here: https://realworldml.github.io/files/cr/paper22.pdf), with many co-authors that contribute to BoTorch, which proposes a new approach for optimizing acquisition functions (AF) over discrete and mixed spaces and reviews many alternative AF optimization approaches (and different choices of surrogate models in the related work section). (The camera-ready will be out soon) Although the code is external to BoTorch at the moment (https://github.com/facebookresearch/bo_pr), I plan to put up a pull request to move key components upstream into BoTorch. |
Beta Was this translation helpful? Give feedback.
-
At a high level, the lack of certain approaches is likely due to limited bandwidth available on our side for implementing and comparing these approaches, and the approaches already implemented working well for our use cases. For one-hot encoding based approaches, Ax supports one-hot encoding of categoricals.
See models defined in https://github.com/pytorch/botorch/blob/main/botorch/models/contextual.py and https://github.com/pytorch/botorch/blob/main/botorch/models/contextual_multioutput.py for embedding based approaches. I believe @qingfeng10 has worked with these, but I don't know if we're actively using them anywhere. |
Beta Was this translation helpful? Give feedback.
-
BoTorch uses the CategoricalKernel and MixedSingleTaskGP model that handle categoricals. This method seems to be very accurate but this accuracy seems to come at a cost in computational efficiency. As far as I understood it, the way it works is running a separate optimization process for every potential combination of categoricals what is not viable when there's a lot of them or their cardinality is high.
I wonder why this approach was used in BoTorch and I'm also trying to understand tradeoffs between other existing approaches. Namely tradeoffs between:
Does anyone have a good intuition on that? Are there any sources worth checking out?
Beta Was this translation helpful? Give feedback.
All reactions