Keras-witf tensorflow优化器

Keras-witf tensorflow优化器,tensorflow,keras,Tensorflow,Keras,我正在尝试使用Tensorflow后端优化器优化Keras模型。 我可以运行模型,计算损失,打印所有权重和偏差,但是当我尝试应用梯度时,模型崩溃了 PS:我使用的是tensorflow Cuda docker容器,主机系统是Ubuntu18.04.2 LTS 代码: 错误: 2019-03-12 21:59:20.737232: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.

我正在尝试使用Tensorflow后端优化器优化Keras模型。 我可以运行模型,计算损失,打印所有权重和偏差,但是当我尝试应用梯度时,模型崩溃了

PS:我使用的是tensorflow Cuda docker容器,主机系统是Ubuntu18.04.2 LTS

代码:

错误:

2019-03-12 21:59:20.737232: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737326: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737359: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737436: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at training_ops.cc:2933 : Not found: Resource localhost/dense_1/bias/RMSProp/N10tensorflow3VarE does not exist.
2019-03-12 21:59:20.737524: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737569: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737605: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737647: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at training_ops.cc:2933 : Not found: Resource localhost/dense_1/kernel/RMSProp/N10tensorflow3VarE does not exist.
2019-03-12 21:59:20.737896: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737944: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737983: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.738050: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at training_ops.cc:2933 : Not found: Resource localhost/dense/bias/RMSProp/N10tensorflow3VarE does not exist.
2019-03-12 21:59:20.738110: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.738149: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.738186: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.738231: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at training_ops.cc:2933 : Not found: Resource localhost/dense/kernel/RMSProp/N10tensorflow3VarE does not exist.

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InternalError: Invalid variable reference.
    [[{{node RMSProp/update_dense_1/bias/ResourceApplyRMSProp}} = ResourceApplyRMSProp[T=DT_FLOAT, use_locking=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dense_1/bias, dense_1/bias/RMSProp, dense_1/bias/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, gradients/dense_1/BiasAdd_grad/BiasAddGrad)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/project/test6.py", line 50, in <module>
     out_data, out_loss, out_train = sess.run([output, loss, _train], feed_dict={input: data.reshape(50, 1), Y_data_placeholder: Y_Data}) # Runs Error.
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Invalid variable reference.
 [[node RMSProp/update_dense_1/bias/ResourceApplyRMSProp (defined at /opt/project/test6.py:31)  = ResourceApplyRMSProp[T=DT_FLOAT, use_locking=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dense_1/bias, dense_1/bias/RMSProp, dense_1/bias/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, gradients/dense_1/BiasAdd_grad/BiasAddGrad)]]

Caused by op 'RMSProp/update_dense_1/bias/ResourceApplyRMSProp', defined at:
  File "/opt/project/test6.py", line 31, in <module>
    _train = trainer.apply_gradients(grads)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/optimizer.py", line 610, in apply_gradients
    update_ops.append(processor.update_op(self, grad))
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/optimizer.py", line 167, in update_op
    update_op = optimizer._resource_apply_dense(g, self._v)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/rmsprop.py", line 194, in _resource_apply_dense
    use_locking=self._use_locking)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/gen_training_ops.py", line 2073, in resource_apply_rms_prop
    use_locking=use_locking, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

InternalError (see above for traceback): Invalid variable reference.
 [[node RMSProp/update_dense_1/bias/ResourceApplyRMSProp (defined at /opt/project/test6.py:31)  = ResourceApplyRMSProp[T=DT_FLOAT, use_locking=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dense_1/bias, dense_1/bias/RMSProp, dense_1/bias/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, gradients/dense_1/BiasAdd_grad/BiasAddGrad)]]
2019-03-12 21:59:20.737232:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.737326:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.737359:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.737436:W tensorflow/core/framework/op_kernel.cc:1273]op_REQUIRES在训练_ops时失败。cc:2933:未找到:资源localhost/densite_1/bias/RMSProp/N10tensorflow3VarE不存在。
2019-03-12 21:59:20.737524:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.737569:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.737605:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.737647:W tensorflow/core/framework/op_kernel.cc:1273]op_REQUIRES在训练_ops时失败。cc:2933:未找到:资源localhost/denseu 1/kernel/RMSProp/N10tensorflow3VarE不存在。
2019-03-12 21:59:20.737896:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.737944:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.737983:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.738050:W tensorflow/core/framework/op_kernel.cc:1273]op_REQUIRES在训练_ops时失败。cc:2933:未找到:资源localhost/dense/bias/RMSProp/N10tensorflow3VarE不存在。
2019-03-12 21:59:20.738110:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.738149:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.738186:W tensorflow/core/framework/op_kernel.cc:1261]内部:变量引用无效。
2019-03-12 21:59:20.738231:W tensorflow/core/framework/opúu kernel.cc:1273]op戋u REQUIRES在培训戋u ops时失败。cc:2933:未找到:资源localhost/dense/kernel/RMSProp/N10tensorflow3VarE不存在。
回溯(最近一次呼叫最后一次):
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/client/session.py”,第1334行,在
返回fn(*args)
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/client/session.py”,第1319行,在
选项、提要、获取列表、目标列表、运行元数据)
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/client/session.py”,第1407行,位于调用会话运行中
运行(元数据)
tensorflow.python.framework.errors\u impl.InternalError:变量引用无效。
[{node RMSProp/update_densite_1/bias/resourceAppyrmsprop}}=resourceAppyrmsprop[T=DT_FLOAT,use_locking=false,_device=“/job:localhost/replica:0/task:0/device:GPU:0”](稠密_1/bias/RMSProp,稠密_1/bias/RMSProp,RMSProp/1,RMSProp/learning_比率,RMSProp/learning_比率,RMSProp/decause,RMSProp/deption,RMSProp/epsilon,梯度/sadd]
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“/opt/project/test6.py”,第50行,在
out_数据,out_丢失,out_列=sess.run([output,loss,_train],feed_dict={input:data.Reformate(50,1),Y_数据占位符:Y_data})运行错误。
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py”,第929行,正在运行
运行_元数据_ptr)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py”,第1152行,正在运行
feed_dict_tensor、options、run_元数据)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py”,第1328行,运行
运行(元数据)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py”,第1348行,在
提升类型(e)(节点定义、操作、消息)
tensorflow.python.framework.errors\u impl.InternalError:变量引用无效。
[[node RMSProp/update\u densite\u 1/bias/resourceAppyrmsprop(定义于/opt/project/test6.py:31)=resourceAppyrmsprop[T=DT\u FLOAT,use\u locking=false,_device=“/job:localhost/replica:0/task:0/device:GPU:0”](稠密_1/偏差,稠密_1/偏差/RMSProp,稠密_1/偏差/RMSProp_1,RMSProp/学习率,RMSProp/衰减,RMSProp/动量,RMSProp/ε,梯度/稠密_1/比阿萨德•梯度/比阿萨德•梯度)]]
由op“RMSProp/update\u densite\u 1/bias/ResourceApplyRMSProp”引起,定义于:
文件“/opt/project/test6.py”,第31行,在
_培训=培训师。应用梯度(梯度)
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/training/optimizer.py”,第610行,应用梯度
更新操作追加(处理器更新操作(self,grad))
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/training/optimizer.py”,第167行,在update\u op中
更新\u op=optimizer.\u资源\u应用\u密集(g,self.\u v)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/rmsprop.py”,第194行,在资源应用中
使用锁定=自锁。\使用锁定)
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/training/gen\u training\u ops.py”,第2073行,在resource\u apply\u rms\u prop中
使用锁定=使用锁定,名称=名称)
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/framework/op_def_library.py”,第787行,在“应用”和“操作”助手中
op_def=op_def)
文件“/usr/local/lib/python3.5/dist packages/tensorflow/python/util/deprecation.py”,第488行,在new_func中
返回函数(*args,**kwargs)
文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py”,第3行
2019-03-12 21:59:20.737232: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737326: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737359: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737436: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at training_ops.cc:2933 : Not found: Resource localhost/dense_1/bias/RMSProp/N10tensorflow3VarE does not exist.
2019-03-12 21:59:20.737524: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737569: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737605: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737647: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at training_ops.cc:2933 : Not found: Resource localhost/dense_1/kernel/RMSProp/N10tensorflow3VarE does not exist.
2019-03-12 21:59:20.737896: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737944: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.737983: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.738050: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at training_ops.cc:2933 : Not found: Resource localhost/dense/bias/RMSProp/N10tensorflow3VarE does not exist.
2019-03-12 21:59:20.738110: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.738149: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.738186: W tensorflow/core/framework/op_kernel.cc:1261] Internal: Invalid variable reference.
2019-03-12 21:59:20.738231: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at training_ops.cc:2933 : Not found: Resource localhost/dense/kernel/RMSProp/N10tensorflow3VarE does not exist.

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InternalError: Invalid variable reference.
    [[{{node RMSProp/update_dense_1/bias/ResourceApplyRMSProp}} = ResourceApplyRMSProp[T=DT_FLOAT, use_locking=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dense_1/bias, dense_1/bias/RMSProp, dense_1/bias/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, gradients/dense_1/BiasAdd_grad/BiasAddGrad)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/project/test6.py", line 50, in <module>
     out_data, out_loss, out_train = sess.run([output, loss, _train], feed_dict={input: data.reshape(50, 1), Y_data_placeholder: Y_Data}) # Runs Error.
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Invalid variable reference.
 [[node RMSProp/update_dense_1/bias/ResourceApplyRMSProp (defined at /opt/project/test6.py:31)  = ResourceApplyRMSProp[T=DT_FLOAT, use_locking=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dense_1/bias, dense_1/bias/RMSProp, dense_1/bias/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, gradients/dense_1/BiasAdd_grad/BiasAddGrad)]]

Caused by op 'RMSProp/update_dense_1/bias/ResourceApplyRMSProp', defined at:
  File "/opt/project/test6.py", line 31, in <module>
    _train = trainer.apply_gradients(grads)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/optimizer.py", line 610, in apply_gradients
    update_ops.append(processor.update_op(self, grad))
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/optimizer.py", line 167, in update_op
    update_op = optimizer._resource_apply_dense(g, self._v)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/rmsprop.py", line 194, in _resource_apply_dense
    use_locking=self._use_locking)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/gen_training_ops.py", line 2073, in resource_apply_rms_prop
    use_locking=use_locking, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

InternalError (see above for traceback): Invalid variable reference.
 [[node RMSProp/update_dense_1/bias/ResourceApplyRMSProp (defined at /opt/project/test6.py:31)  = ResourceApplyRMSProp[T=DT_FLOAT, use_locking=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dense_1/bias, dense_1/bias/RMSProp, dense_1/bias/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, gradients/dense_1/BiasAdd_grad/BiasAddGrad)]]
sess.run(tf.global_variables_initializer())