GPyTorch support for ELFI #487
Description
Feature proposal
I believe it would be beneficial for ELFI to not only work on GPy as a backbone, but GPyTorch as well. My reasons are as follows:
- GPyTorch supports more involved Gaussian Process Regression modelling.
- It supports massively parallelized computations, for example on HPCs or GPUs.
- It makes its inner workings more transparent.
- It has more active support.
Work that has already been done
To understand ELFI, GPy, PyTorch, and GPyTorch better, I wrote a re-implementation of BOLFI within the ELFI framework that uses GPyTorch for its computations. You may find the implementation on my fork: https://github.com/YannickNoelStephanKuhn/elfi/tree/gpytorch-support
It consists of four files:
- elfi/methods/bo/botorch_acquisition.py re-implements the LCBSC acquisition function within BoTorch.
- elfi/methods/bo/gpytorch_bolfi_model.py re-implements the parabolic regression model of BOLFI within GPyTorch.
- elfi/methods/bo/gpytorch_regression.py wraps the Adam optimizer for Gaussian Process Regression.
- elfi/methods/inference/gpytorch_bolfi.py shows how the Gaussian Process hyperparameters may be tuned with Pyro.
Possible future avenues
The implementation of hyperparameter tuning with Pyro does not work as of now, since Pyro does not support unnormalized distributions. I could not normalize the Gaussian Process distribution or find a way to work around the Pyro limitation.
What happens now
I wanted to have this work freely available, in case someone finds it useful. It may fully integrate into ELFI, or it may be here in case someone else wants to combine the benefits of ELFI and GPyTorch on their own.