Kernel's hyper-parameters; initialization and setting bounds

Question

I think many other people like me might be interested in how they can use GPFlow for their special problems. The key is how GPFlow is customizable, and a good example would be very helpful.

In my case, I read and tried lots of comments in raised issues without any real success. Setting kernel model parameters is not straightforward (creating with default values, and then do it via the delete object method). Transform method is vague.

It would be really helpful if you could add an example showing. how one can initialize and set bounds of an anisotropic kernel model (length-scales values and bounds, variances, ...) and specially adding observations error (as an array-like alpha parameter)

Could you clarify what you mean by "setting kernel model parameters is not straightforward" (you can do, e.g., `kern = gpflow.kernels.Matern52(input_dim=1, lengthscales=0.3)` and later `kern.lengthscales = 5.0`)? What do you mean by "delete object method"? What do you mean by "transform method"? What do you mean by "adding observations error" - different noise for different data-points? — STJ, Apr 01 '19 at 15:00
not straightforward compare to sklearn.GP: `kern = rbf(length_scale=[a,b,c], length_scale_bounds=([a1,a2],[b1,b2],[c1,c2]) ` or `ConstantKernel(sill, tuple(sill_bound)` observations error ; Value added to the diagonal of the kernel matrix during fitting. Larger values correspond to increased noise level in the observations. — Mo Ko, Apr 02 '19 at 15:11

score 3 · Answer 1 · edited Apr 02 '19 at 18:35

If you just want to set a value, then you can do

model = gpflow.models.GPR(np.zeros((1, 1)),
                          np.zeros((1, 1)),
                          gpflow.kernels.RBF(1, lengthscales=0.2))

Alternatively

model = gpflow.models.GPR(np.zeros((1, 1)),
                          np.zeros((1, 1)),
                          gpflow.kernels.RBF(1))
model.kern.lengthscales = 0.2

If you want to change the transform, you either need to subclass the kernel, or you can also do

with gpflow.defer_build():
     model = gpflow.models.GPR(np.zeros((1, 1)),
                               np.zeros((1, 1)),
                               gpflow.kernels.RBF(1))
     transform = gpflow.transforms.Logistic(0.1, 1.))
     model.kern.lengthscales = gpflow.params.Parameter(0.3, transform=transform)
model.compile()

You need the defer_build to stop the graph being compiled before you've changed the transform. Using the approach above, the compilation of the tensorflow graph is delayed (until the explicit model.compile()) so is built with the intended bounding transform.

Using an array parameter for likelihood variance is outside the scope of gpflow. For what it's worth (and because it has been asked about before), that particular model is especially problematic as it is not clear how test points are defined.

score 1 · Answer 2 · answered Apr 01 '19 at 15:06

Setting kernel parameters can be done using the .assign() function, or through direct assignment. See the notebook https://github.com/GPflow/GPflow/blob/develop/doc/source/notebooks/understanding/tf_graphs_and_sessions.ipynb. You do not need to delete a parameter to assign a new value to it.

If you want to have per-datapoint noise, you will need to implement your own custom likelihood, which you can do by taking Gaussian likelihood in likelihoods.py as an example.

If by "bounds" you mean limiting the optimisation range for a parameter, you can use the Logistic transform. If you want to pass in a custom transformation for a parameter, you can pass a constructed Parameter object into constructors with a custom transform. Alternatively you can assign a newly created Parameter with a new transform to the model.

score 1 · Answer 3 · answered Apr 01 '19 at 16:05

Here is more information on how to access and change GPflow parameters: viewing, getting and settings parameters documentation.

Extra bit for @user1018464 answer about replacing transform in existing parameter: changing transformation is a bit tricky, you can't change transformation once a model was compiled in TensorFlow.

E.g.

likelihood = gpflow.likelihoods.Gaussian()
likelihood.variance.transform = gpflow.transforms.Logistic(1., 10.)
----
GPflowError: Parameter "Gaussian/variance" has already been compiled.

Instead you have to reset GPflow object:

likelihood = gpflow.likelihoods.Gaussian()  # All tensors compiled

likelihood.clear()
likelihood.variance.transform = gpflow.transforms.Logistic(2, 5)
likelihood.variance = 2.5
likelihood.compile()

Kernel's hyper-parameters; initialization and setting bounds

3 Answers3

Linked