[Feature Request] Gaussian Process Ordinal Regression (GPOR)

[j-adamczyk]

🚀 Feature Request

Support for ordinal regression (ordinal classification).

Motivation

Ordinal regression, or classification with ordered classes, naturally arises for a lot of problems where the target label are discrete preferences. This can be starts (X out of 5), ratings (X of out 10), age (ordered integer) etc. It requires a support for ordinal likelihood model to work.

Pitch

I came upon this problem when trying to reimplement the ordinal regression from GPflow (which fully supports it) to GPyTorch: https://gpflow.github.io/GPflow/develop/notebooks/advanced/ordinal_regression.html. The basic tutorial code is:

GPflow code is very simple:

# construct ordinal likelihood - bin_edges is the same as unique(Y) but centered
bin_edges = np.array(np.arange(np.unique(Y).size + 1), dtype=float)
bin_edges = bin_edges - bin_edges.mean()
likelihood = gpflow.likelihoods.Ordinal(bin_edges)

# build a model with this likelihood
m = gpflow.models.VGP(
    data=(X, Y), kernel=gpflow.kernels.Matern32(), likelihood=likelihood
)

# fit the model
opt = gpflow.optimizers.Scipy()
opt.minimize(m.training_loss, m.trainable_variables, options=dict(maxiter=100))

As far as I can tell, this is currently impossible to reimplement in GPyTorch, as it has no Ordinal likelihood analogue. But this should be really the only addition needed to make this work.