Plotting the acquisition values for knowledgegradient

[kentwar]

Hi,

The one_shot knowledge gradient implementation appears to work very well. I'm attempting to migrate my research over to BoTorch and as a sanity check would like to check the results of my old KG code with the new. To get an insight into how the KG is working I was thinking I could plot the values, but I'm hitting a few barriers.

Assuming my objective is defined over 1 input dimension [0,1]

def Fun(x): 
    x = x*(7.5-2.7)+2.7 #rescaling
    y= -(np.sin(x) - np.sin(10/3 *x))
    return(-y)

and I train the model:

dim = 1
init_points = 10
xtrain = torch.rand(init_points,dim)
yvals = [Fun(x) for x in xtrain]
ytrain = torch.tensor(yvals).reshape(init_points,dim)
model = SingleTaskGP(xtrain, ytrain)
mll = ExactMarginalLogLikelihood(model.likelihood, model)
fit_gpytorch_model(mll)

Can I get the KG valuation at some point X (candidate), given some randomly sampled points (rand_points) like this:

KG = qKnowledgeGradient(model, num_fantasies=512,)
rand_points = torch.random(512).reshape(512, -1)
test_X = torch.cat(candidate, rand_points)
KG_value = KG.forward(test_X)

If I plot this for a discretisation of my input domain the peaks do not always align with the optim_acq results, though I would expect this as I'm not enforcing they use the same random sampling to generate the values. Is there a way to enforce this? and/or am I abusing/misunderstanding the code here?

Thanks in advance.

[saitcakmak]

Hi @kentwar. You should check out the evaluate method of qKnowledgeGradient. qKG is very sensitive to the solutions of the inner optimization problems, for which you're passing torch.random here. This would undoubtedly lead to some plots that don't make much sense. When you use optimize_acqf, these get optimized along with the candidate point but that doesn't happen when plotting. For this exact reason, we implemented the evalute method, to which you only pass the candidate point. It will solve the inner optimization problems under the hood, leading to a much more reliable qKG estimate.

Side note: I see that you're using 512 fantasies. Particularly for low dimensional problems, you can get away with much fewer fantasies, such as 16 / 64. Just noting this since it will help a lot with runtime and memory use.

[saitcakmak]

In theory, the two should be equal. In practice, the two are produced based on the solution of two independent numerical optimization runs, which are setup somewhat differently. So, it is expected to have differences in the reported values, though the results should be in agreement for the most part.

[TimLK]

In theory, the two should be equal. In practice, the two are produced based on the solution of two independent numerical optimization runs, which are setup somewhat differently. So, it is expected to have differences in the reported values, though the results should be in agreement for the most part.

Thanks a lot for reply. I still want to know further why they are different in practice. From my understanding, if they’re both simulated using the same n_f, they should be equal in practice as well (by setting with torch.munal_seed()?).

[TimLK]

In theory, the two should be equal. In practice, the two are produced based on the solution of two independent numerical optimization runs, which are setup somewhat differently. So, it is expected to have differences in the reported values, though the results should be in agreement for the most part.

Thanks a lot for reply. I still want to know further why they are different in practice. From my understanding, if they’re both simulated using the same n_f, they should be equal in practice as well (by setting with torch.munal_seed()?).

I think made a mistake here. kg_value is computed by optimize_acqf() which has no relevance to qKG_proper.evaluate(), right?
but true_value is simulated by averaging maximized fantasies models (implemented by qKG_proper.evaluate()). This is the reason of difference, right?
However, how should I explicitly plot the figure like other acquisition function such as qEI in which the two values, ei_value and true_value are equal:
x_next_ei, ei_value=optimize_acqf( acq_function=qEI, bounds=bounds, q=1, ... )
true_value=qEI.forward(x_next_ei)

[Balandat]

I think made a mistake here. kg_value is computed by optimize_acqf() which has no relevance to qKG_proper.evaluate(), right?

They are related and as Said mentioned should be the same. But the values are obtained as the solution of very hard (non-convex) numerical optimization problems that in general we may not actually be able to find the true global solution for.

[TimLK]

Thank you! I think I finally got the point here.
So, it is actually difficult to plot the KG acquisition function value because the location of the optimal KG, say, x_next_kg, is possible to be different from the location of the optima that qKG_proper.evaluate() tells. (actually I found this happens occasionally)
I verify this in a very simple code here. I appreciate so much if you can look at it and provide any suggestions because I'm a very beginner of Bayesian Optimization. Thank you very much!
Single-KG.zip