Entropy Acquisition Functions with Approximate GPs

[acb-code]

I have a quick question about using the approximate GPs with the entropy acquisition functions.

I am trying to use a SingleTaskVariationalGP with GIBBON (qLowerBoundMaxValueEntropy) and MES (qMaxValueEntropy). I set up an optimization loop similar to the ones in the tutorials, and when I use a SingleTaskGP the loop will run successfully, but when I switch to the SingleTaskVariationalGP, it works correctly for the qNoisyExpectedImprovement acquisition function, but not for the GIBBON and MES acquisition functions. The error I am seeing is

AttributeError: 'SingleTaskVariationalGP' object has no attribute 'batch_shape'

I started looking into the source for SingleTaskGP vs SingleTaskVariationalGP, and I see they're subclassing from different starting points, with the batch_shape property being defined based in the SingleTaskGP maybe via BatchedMultiOutputGPyTorchModel, and I don't see an equivalent in the SingleTaskVariationalGP classes. I thought I would ask here before continuing to search for the issue.

Are qLowerBoundMaxValueEntropy and qMaxValueEntropy incompatible with the SingleTaskVariationalGP?

Here's the setup I'm using:
def initialize_model(train_x, train_obj, state_dict=None): """function to initialize the GP model""" model_obj = SingleTaskVariationalGP(train_x, train_obj, outcome_transform=Standardize(m=train_obj.shape[-1]), input_transform=Normalize(d=train_x.shape[-1])) mll = PredictiveLogLikelihood(model_obj.likelihood, model_obj.model, num_data=train_x.shape[-2]) return mll, model_obj

mll_gib, model_gib = initialize_model(train_x_gib, train_obj_gib)

fit_gpytorch_mll(mll_gib)

candidate_set = torch.rand(10000, bounds.size(1), device=bounds.device, dtype=bounds.dtype) qGIB = qLowerBoundMaxValueEntropy( model=model_gib, candidate_set=candidate_set, )

This is where the error is getting thrown - when the setup for qLowerBoundMaxValueEntropy is called. When I use the SingleTaskGP, I don't get the error. When I use the qNoisyExpectedImprovement with the SingleTaskVariationalGP I don't get this error.

Thanks!

[eytan]

cc @henrymoss

[Balandat]

We may just need to add the batch_shape property to SingleTaskVariationalGP. Do you have a full repro including some toy data that we could use for debugging?

[acb-code]

Yes, here's an example notebook with a reproduction of the error: https://github.com/acb-code/test_repo/blob/main/vargp_gibbon_bo_repro.ipynb

[henrymoss]

Interesting. I think this would have been an issue even before my IPA PR?

[Balandat]

With #1799 your notebook runs all the way, including SingleTaskVariationalGP

[acb-code]

Sounds great, working for me now with that property function added, Thanks!