Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variational GP with derivatives and monotonic gp #2272

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

ankushaggarwal
Copy link
Contributor

I have implemented an Indexed version of VariationalStrategy that allows us to use derivative information. I have used it to create a monotonic GP with a composite likelihood. The new variational strategy is called VariationalStrategyIndexed, but it could be easily merged with the normal VariationalStrategy (I can do that if that is preffered).

I have also provided an example on how to use it. One minor issue though, as raised in a bug report #1554, the slicing raises a spurious error. To suppress it, we have to set gpytorch.linear_operator.settings.debug._default = False. This pull request is related to #1639 (comment)

@gpleiss
Copy link
Member

gpleiss commented May 26, 2023

Sorry for the slow reply @ankushaggarwal . I'm a little hesitant about adding this PR for a few reasons. 1) I'm not sure how re-usable VariationalStrategyIndex is, and I think it's also too specific to add to VariationStrategy proper. 2) I'm not sure this is the best/correct way to use derivative information with variational models. Derivative observations qualify as an inter-domain GP, and for variational GPs its best to factorize over the derivative and non-derivative observations (see https://arxiv.org/pdf/2003.01115.pdf for more details).

@ankushaggarwal
Copy link
Contributor Author

Sorry for the slow reply @ankushaggarwal . I'm a little hesitant about adding this PR for a few reasons. 1) I'm not sure how re-usable VariationalStrategyIndex is, and I think it's also too specific to add to VariationStrategy proper. 2) I'm not sure this is the best/correct way to use derivative information with variational models. Derivative observations qualify as an inter-domain GP, and for variational GPs its best to factorize over the derivative and non-derivative observations (see https://arxiv.org/pdf/2003.01115.pdf for more details).

Thank you @gpleiss for your response. No problem. Since we needed a monotonic GP which needs variational GP with Bernouille likelihood for derivatives (as pointed out by you in #1639 (comment)). The current implementation by me works for us in practice, but we would be very much interested in the variational GPs that handle derivatives through factorization over the derivative and non-derivative observations. So please let me know when this gets implemented in gpytorch, and we would love to use this more suitable approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants