What are the implications to utilize AnalyticExpectedUtilityOfBestOption when previous_winner is not specified?

[changkun]

Hi botorch authors, thanks for developing this amazing framework.

I come across to consider using AnalyticExpectedUtilityOfBestOption and have some questions on a proper usage of it's parameter previous_winner:

previous_winner (Optional[Tensor]) – Tensor representing the previous winner in the Y space.

In the tutorial, there is no specification on the previous_winner parameter:

            acqf = AnalyticExpectedUtilityOfBestOption(
                pref_model=pref_model, outcome_model=one_sample_outcome_model
            )

Assume the following function is called each time to generate the next optimal candidate:

• X is n by dim tensor of input data, includes all prior candidates used for optimization
• y is n by 2 tensor of all prior preferential information

def BOPE(X: torch.tensor, y: torch.tensor, dim: int):
    while True:
        try:
            bounds = torch.stack([torch.zeros(dim), torch.ones(dim)])
            gp = PairwiseGP(X, y, covar_module=ScaleKernel(
                MaternKernel(nu=2.5, lengthscale_prior=GammaPrior(3.0, 6.0)),
                outputscale_prior=GammaPrior(2.0, 0.15),
            ))
            mll = PairwiseLaplaceMarginalLogLikelihood(gp.likelihood, gp)
            fit_gpytorch_model(mll)

            candidates, acq_val = optimize_acqf(
                acq_function=AnalyticExpectedUtilityOfBestOption(
                    pref_model=gp),
                bounds=bounds,
                q=2,
                num_restarts=5,
                raw_samples=20,
            )
            break
        except:
            continue
    return candidates, acq_val

I am generally confused by the optional argument previous_winner of the AnalyticExpectedUtilityOfBestOption acquisition function.

If the argument is optional, does it mean we don't have to specify it for conducting BOPE optimization process? If it is somewhat required to specify in terms of preference optimization, what are the implications if this argument is not specified? Thank you in advance.

[ItsMrLin]

Hi @changkun, thanks for loving BoTorch!

You are correct that previous_winner is an optional argument. When using AnalyticExpectedUtilityOfBestOption (i.e., Analytic EUBO) for preference learning, it is meant to suggest a pair of candidates for pairwise comparisons.

However, in some cases, we may only want to keep one of the comparison candidate to be the winner from the previous comparison round (or really any manually specified candidate you'd like keep as one of the choices in the comparison). In this case, one can specify previous_winner and set q=1, and optimize_acqf will only need to optimize for 1 candidate instead of 2.

This would be useful for cases where optimizing for 2 candidates is too costly or we would like to make the comparison easier for the decision maker by keeping one of the candidates fixed or unchanged (e.g., the winner from last round).

[changkun]

Hi @ItsMrLin thanks for the response. I am still not entirely sure if I understood everything correctly. Let me state my understanding:

1. If we want to keep the comparison candidate from the previous comparison round, we should set q=1 and specify the previous_winner is set to the selected candidate.
2. Otherwise, in the posted implementation, the optimizer always evaluates all historical candidates and their preference choice, specifying q=2 and unspecifies the previous_winner is enough.

Does that somewhat match your explanation? Would you mind also pointing me to a few more materials on this topic? In this way, I can learn a bit more about the consequence of specifying and not specifying a previous winner.

[ItsMrLin]

Your understanding is correct.

In either case, the model/optimizer takes all historical candidates and their preference choices into consideration. We are always generating two candidates for pairwise comparisons using this acquisition function -- with a choice of whether we want one of them to be a manually specified candidate. If yes, you can specify previous_winner (maybe a better name is manually_selected_candidate) to be the one you wanna use with q=1; if you want the acquisition function to select both candidates, you can use q=2 without specifying previous_winner.

Winner from the previous round of comparison is just a common way to select such fixed candidate when q=1.