GPClassification vs GP - Optimal Hyperparameters

[jcockayne]

Hi,

I am having a strange issue with optimizing hyperparameters in GP classification. Based on this I understand there are two ways to construct a GP classification model - the first using GPy.models.GPClassification and the second using GPy.core.GP with a RBF kernel, Bernoulli likelihood and an EP inference method specificed. I thought the two were equivalent, but I get different results when trying to optimize hyperparameters with these two methods:

import GPy
import numpy as np
np.random.seed(123)
X1 = np.random.normal(-1, 0.1, size=100)
X2 = np.random.normal(1, 0.1, size=100)
X = np.concatenate([X1, X2])[:,None]
y = np.concatenate([np.ones(100), np.zeros(100)])[:,None]

m0 = GPy.models.GPClassification(X, y)
for i in range(5):
    m0.optimize('bfgs', max_iters=100)

lik = GPy.likelihoods.Bernoulli()
k = GPy.kern.RBF(input_dim=1, variance=1, lengthscale=1)
inf_method = GPy.inference.latent_function_inference.expectation_propagation.EP()
m1 = GPy.core.GP(X=X, Y=y, kernel=k, likelihood=lik, inference_method=inf_method)
for i in range(5):
    m1.optimize('bfgs', max_iters=100)

print(m0)
print(m1)

Here m0 and m1 return different hyperparameter values. In a more substantial example I've seen the objective function values and optimal hyperparameters differ even more significantly.

Can someone help to explain the issue? I'm sure I'm just misunderstanding how GP and GPClassification are connected to each other but some help would be appreciated!