How to use acqf_discrete ?

[WooMOS]

Hi, I am a student using a botorch.

I use integer input and float output data. I have implemented BO based on the example code provided by default.

Like this

import torch
from botorch.models import SingleTaskGP
from botorch.fit import fit_gpytorch_model
from botorch.utils import standardize
from gpytorch.mlls import ExactMarginalLogLikelihood

train_X = torch.tensor([[150.,100.,100.,100.,100.,150.],
                        [150.,100.,0.,50.,100.,0.],
                        [50.,50.,150.,0.,150.,100.],
                        [150.,50.,0.,50.,150.,50.],
                        [0.,100.,50.,50.,50.,100.],
                        ])


train_Y = torch.tensor([[279.39],
                        [283.60],
                        [286.57],
                        [289.63],
                        [294.51],
                        ])

gp = SingleTaskGP(train_X, train_Y)
mll = ExactMarginalLogLikelihood(gp.likelihood, gp)
fit_gpytorch_model(mll)

I used integer values of 0, 50, 100, 150 as inputs and float values as outputs. Modeling proceeded as in the code above.

from botorch.optim import optimize_acqf

set_1 = torch.tensor([150, 150, 150, 150, 150, 150])
bounds = torch.stack([torch.zeros(6), set_1])
i=5
st=[]

while(i!=0):
  candidate, acq_value = optimize_acqf(EI, bounds=bounds, q=1, num_restarts=20, raw_samples=512)
  if(acq_value >= 0):
    i=i-1
    round_value = torch.round(candidate/50)*50
    st.append(round_value)

st=torch.stack(st, 1)
st=torch.squeeze(st)

I rounded the output value because I need to get a parameter set of 0, 50, 100, and 150. And this is where the problem arises.

The biggest problem is that the training data are output again. I did some searching to solve this problem and found the acqf_discrete function. But I'm not sure how to use this function. Please provide examples or information that can be used as a reference for using this function.

Additionally, I was wondering if there is a way to get multiple maximal parameter sets that don't overlap using acqf. For example, if the maximum point a is obtained through acqf, is there a way to get the next maximum point b and c?

Thanks for reading the beginner's article.

[Balandat]

acqf_discrete is meant to be used if you have a set of possible discrete inputs of relatively small cardinality. It looks like here you're using ints between 0 and 150, and so that is not the case here (151^6 ~ 1.2e13). While there can be issues with rounding to integer values if the range is relatively small, wouldn't be too worried about this in your case, so your approach of using optimize_acqf is reasonable.

However, it looks like you're generating a batch of candidates to be evaluated in general here - in order for this to make sense in an iterative fashion you'll need to tell the acquisition function about the points you've previously selected (using the X_pending input to the constructor - if you're using analytical EI you will have to switch that to the MC version qEI or better even, qNEI). if you don't want to do this manually, we built this into the interface if you pass in q=5 an set sequential=True (see the code of optimize_acqf how this is done under the hood).

Another thing to note is that the default priors we use on the models assume that the input data lives in the unit cube and that the observations are (more or less) zero mean with unit variance. With inputs/outputs of the scale you're using here, the model fits could end up being quite bad, so I recommend you normalize the inputs to the unit cube and standardize the outputs (you can use the following utilities for that:

botorch/botorch/utils/transforms.py

Line 44 in 3318c06

def standardize(Y: Tensor) -> Tensor:

,

botorch/botorch/utils/transforms.py

Line 67 in 3318c06

def normalize(X: Tensor, bounds: Tensor) -> Tensor:

)

[WooMOS]

Thank you for answer.

Since I only use the values 0, 50, 100, and 150 in train_X, I think there are 4,096 (4^6) cases (6 parameters in 4 cases). Isn't this a small cardinality?

fidelities = np.array(( (0, 50, 100, 150), 
                        (0, 50, 100, 150), 
                        (0, 50, 100, 150), 
                        (0, 50, 100, 150), 
                        (0, 50, 100, 150), 
                        (0, 50, 100, 150), ),dtype=object)

def meshgrid_discrete_fidelities(fidelities):
    return np.transpose(np.reshape(np.meshgrid(*fidelities),[np.size(fidelities,0),-1])).astype(np.float32)

train_X = torch.tensor([[150,100,100,100,100,150],
                        [150,100,0,50,100,0],
                        [50,50,150,0,150,100],
                        [150,50,0,50,150,50],
                        [0,100,50,50,50,100],
                        ], dtype=torch.float)


Y = torch.tensor([[279.39],
                  [283.60],
                  [286.57],
                  [289.63],
                  [294.51],
                  ], dtype=torch.float)

train_Y = standardize(Y)

gp = SingleTaskGP(train_X, train_Y)
mll = ExactMarginalLogLikelihood(gp.likelihood, gp)
fit_gpytorch_model(mll)

EI = ExpectedImprovement(gp, best_f=0.2)

candidate , acq_value = optimize_acqf_discrete(qEI,
                                                q = 1,
                                                choices = torch.tensor(meshgrid_discrete_fidelities(fidelities)),)

print(candidate)
>>>tensor([[  0., 100.,  50.,  50.,  50., 100.]])

I tried using acqf_discrete by referring to the code of another discussion, and the value of train_X was output again as before. My experiment only uses certain integers (0, 50, 100, 150), so I'm wondering if this case is not suitable for BO.

[Balandat]

Since I only use the values 0, 50, 100, and 150 in train_X, I think there are 4,096 (4^6) cases (6 parameters in 4 cases). Isn't this a small cardinality?

Sure, is this also the set of possibilities you want to optimize over? Then optimize_acf_discrete is the right thing to use.

I tried using acqf_discrete by referring to the code of another discussion, and the value of train_X was output again as before. My experiment only uses certain integers (0, 50, 100, 150), so I'm wondering if this case is not suitable for BO.

Are your observations deterministic (i.e. there is no observation noise)? If so SingleTaskGP may not be the right model since it estimates an observation noise level. If that is nonzero then it is not surprising that the acquisition function would recommend previously observed points (since those observations would indeed reduce model uncertainty). Try using FixedNoiseGP instead.

[WooMOS]

Thanks for the advice. I'll try again using the method you advised.