Python 在GPFlow中使用自定义内核进行分类时的Cholesky分解问题

我在GPFlow中构建了一个自定义Matern32内核，用于对一些昂贵的计算矩阵值进行操作，这些值包含我的数据点之间的“距离”：

将numpy作为np导入 导入tensorflow作为tf 导入gpflow dat_mat=np.load'example_matrix.npy' dat_diag=tf.constantnp.diagdat_mat，dtype=tf.float64 dat_mat=tf.constantdat_mat，dtype=tf.float64 def sub_matA，A2： 全球数据网 A=tf.castA，tf.int32 A2=tf.castA2，tf.int32 K_rows=tf.gather_nddat_mat，A K_行=tf.transposeK_行 K_mat=tf.聚集和K_行，A2 K_-mat=tf.transposeK_-mat 返回tf.castK_mat，tf.float64 类Matern32\u RBFgpflow.kernels.Kernel： 全球数据表，数据诊断表 定义初始自我： super.\uuuuu init\uuuuu input\u dim=1，active\u dims=[0] self.var=gpflow.Param1.0，transform=gpflow.transforms.Exp self.mag=gpflow.Param1.0，transform=gpflow.transforms.Exp @gpflow.params_as_张量 def Kself，A，A2=无： 如果A2为无： A2=A K_mat=子_matA，A2 z=np.sqrt3*K_mat*self.var 返回self.mag*1+z*tf.math.exp-z def Kdiagself，A： A=tf.castA，tf.int32 K_diag=tf.casttf.gather_nddat_diag，A，tf.float64 z=np.sqrt3*K_diag*self.var 返回self.mag*1+z*tf.math.exp-z 此自定义内核是为二进制分类任务实现的，示例数据文件随附：

Y=np.loadtxt'example_data.csv'，usecols=1 len_数据集=lenY 这些x值仅指我计算的距离矩阵中的行 X=np.ArrayRangeren_数据集 X=X.astypeint X=X。重塑-1,1 Y=Y。重塑-1,1 k=材料32\u径向基函数 数据集分割 从sklearn.model\u选择导入列车\u测试\u拆分 X_序列，X_测试，y_序列，y_测试=序列测试，X，y，随机状态=1 自定义配置=gpflow.settings.get\u设置 自定义配置.numerics.jitter\u级别=1e-3 opt=gpflow.train.ScipyOptimizer model=gpflow.models.VGPX_列，y_列，似然=gpflow.likelihoods.Bernoulli，kern=k 使用gpflow.settings.temp_settingscustom_配置： opt.emodel 运行上述命令将返回以下错误：

Traceback (most recent call last):
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cholesky decomposition was not successful. The input might not be valid.
[[{{node VGP-cb46397c-6/Cholesky}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "trial.py", line 64, in
opt.minimize(model)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/training/scipy_optimizer.py", line 89, in minimize
**kwargs)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/training/external_optimizer.py", line 175, in minimize
optimizer_kwargs=self.optimizer_kwargs)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/training/external_optimizer.py", line 436, in _minimize
result = scipy.optimize.minimize(*minimize_args, **minimize_kwargs)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/scipy/optimize/_minimize.py", line 601, in minimize
callback=callback, **options)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/scipy/optimize/lbfgsb.py", line 335, in _minimize_lbfgsb
f, g = func_and_grad(x)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/scipy/optimize/lbfgsb.py", line 285, in func_and_grad
f = fun(x, args)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 300, in function_wrapper
return function((wrapper_args + args))
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 63, in call
fg = self.fun(x, *args)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/training/external_optimizer.py", line 398, in loss_grad_func_wrapper
loss, gradient = loss_grad_func(x)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/training/external_optimizer.py", line 309, in eval_func
augmented_fetches, feed_dict=augmented_feed_dict)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cholesky decomposition was not successful. The input might not be valid.
[[node VGP-cb46397c-6/Cholesky (defined at /home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/models/vgp.py:108) ]]

Caused by op 'VGP-cb46397c-6/Cholesky', defined at:
File "trial.py", line 61, in
model = gpflow.models.VGP(X_train, y_train, likelihood=gpflow.likelihoods.Bernoulli(), kern=k)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/core/compilable.py", line 90, in init
self.build()
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/core/node.py", line 156, in build
self._build()
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/models/model.py", line 79, in _build
likelihood = self._build_likelihood()
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/decors.py", line 67, in tensor_mode_wrapper
result = method(obj, *args, **kwargs)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/models/vgp.py", line 108, in _build_likelihood
L = tf.cholesky(K)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/ops/gen_linalg_ops.py", line 767, in cholesky
"Cholesky", input=input, name=name)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
op_def=op_def)
File "/home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1801, in init
self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Cholesky decomposition was not successful. The input might not be valid.
[[node VGP-cb46397c-6/Cholesky (defined at /home/wjm41/.conda/envs/gpenv/lib/python3.6/site-packages/gpflow/models/vgp.py:108) ]]

我尝试过改变GPFlow设置中的抖动级别以及内核参数的初始值，但这并没有解决Cholesky分解问题。我的自定义内核可以很好地用于回归任务，所以我不确定问题出在哪里。我的数据真的很糟糕吗？示例数据是完整数据集的一小部分-有关脚本和数据，请参阅

---

我使用的是Python 3.6、GPFlow 1.3.0和TensorFlow 1.13.1。任何帮助都将不胜感激

我无法立即发现您的代码有问题。我的建议是调试代码，查看Cholesky失败前获得的K矩阵

TensorFlow中的调试很麻烦。我认为[1]中的建议很好，在TensorFlow图中的操作中打开一个IPython终端

---

[1] 幻灯片46之后。

感谢您的调试建议！我做了一些探索，发现我只是没有为内核选择合适的超参数范围来收敛——self.var的值要小得多，我不再遇到Cholesky分解问题。然而，VGP模型并不收敛于合理的答案——内核中的所有条目都是相同的，因此所有的y预测都是相同的，不管x是什么。将任务视为0和1之间的简单GP回归不会出现此问题。有什么建议吗？变分GPs的优化有时很有挑战性。我的一般建议是，首先将超参数固定为合理值，然后只学习变分参数，从而强制拟合数据。然后解放一切，训练。此外，该模型是否适合普通的Matern32内核，而不是您的实现？