alpha - > variance of additional Gaussian measurement noise on the training observations

[muammar]

When you apply GaussianProcessRegressor from sklearn, there is an alpha parameter, that as explained in their documentation, "It can also be interpreted as the variance of additional Gaussian measurement noise on the training observations."

I want to make sure I am doing this correctly with Gpytorch. If I want to add alpha to the covariance matrix, the only place I see where it makes sense is in the forward function. However, the covar_module is returning a ScaleKernel and I am unsure the code below is correct:

import gpytorch
import torch


class ExactGPModel(gpytorch.models.ExactGP):
    def __init__(self, train_x, train_y, likelihood, alpha):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)

        self.alpha = alpha
        self.mean_module = gpytorch.means.ConstantMean()

        self.covar_module = gpytorch.kernels.ScaleKernel(
            gpytorch.kernels.RBFKernel() + gpytorch.kernels.LinearKernel()
        )

    def forward(self, x):
        mean_x = self.mean_module(x)

        covar_x = (
            self.covar_module(x)
            + torch.diag(torch.ones(len(x), device=x.device)) * self.alpha
        )
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)


# initialize likelihood and model
likelihood = gpytorch.likelihoods.GaussianLikelihood()

train_x = torch.from_numpy(np.stack(train.morgan_std_scaled.values))
train_y = torch.from_numpy(train[label_column].values)

alpha = 100
model = ExactGPModel(train_x, train_y, likelihood, alpha)

Is that right? Do I have to make extra modifications when I make a prediction?

Thanks.