Tensorflow Keras:使用重量计算NCE损失
这是标准损耗函数的模型Tensorflow Keras:使用重量计算NCE损失,tensorflow,layer,keras-2,sampled-softmax,nce,Tensorflow,Layer,Keras 2,Sampled Softmax,Nce,这是标准损耗函数的模型 target = Input(shape=(1, ), dtype='int32') w_inputs = Input(shape=(1, ), dtype='int32') w_emb = Embedding(V, dim, embeddings_initializer='glorot_uniform',name='word_emb')(w_inputs) w_flat= Flatten()(w_emb) # context w1= Dense(input
target = Input(shape=(1, ), dtype='int32')
w_inputs = Input(shape=(1, ), dtype='int32')
w_emb = Embedding(V, dim, embeddings_initializer='glorot_uniform',name='word_emb')(w_inputs)
w_flat= Flatten()(w_emb)
# context
w1= Dense(input_dim=dim, units=V, activation='softmax') # because I want to use predicition on valid set)
w= w1(w_flat)
model = Model(inputs=[w_inputs], outputs=[w])
model.compile(loss='sparse_categorical_crossentropy', optimizer='sgd',metrics=['accuracy'])
它很好用。鉴于NCE损失在keras中不可用,我写了一个自定义损失
def model_loss(layer,labels, inputs, num_sampled, num_classes, num_true):
weights= K.transpose( layer.get_weights()[0])
biases = layer.get_weights()[1]
def loss(y_true, y_pred):
if K.learning_phase() == 1:
compute_loss = tf.nn.nce_loss(weights, biases, labels, inputs, num_sampled, num_classes, num_true,
partition_strategy="div")
else:
logits = tf.matmul(K.squeeze(inputs,axis=0), K.transpose(weights))
logits = tf.nn.bias_add(logits, biases)
labels_one_hot = tf.one_hot(labels, num_classes)
loss = tf.nn.sigmoid_cross_entropy_with_logits(
labels=labels_one_hot[:][0][:],
logits=logits)
compute_loss = tf.reduce_sum(loss, axis=1)
return compute_loss
return loss
并将最后一行更改为:
model.compile(loss=model_loss(w1,target, w_emb, num_sampled, num_classes, num_true), optimizer='sgd',metrics=['accuracy'])
顺便说一句,这是汇编的
在执行死刑时死亡
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-68-d3b3ef93b81b> in <module>
3 epochs=epochs, steps_per_epoch = seq_len,
4
----> 5 verbose=1, max_queue_size=15)
/opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
89 warnings.warn('Update your `' + object_name + '` call to the ' +
90 'Keras 2 API: ' + signature, stacklevel=2)
---> 91 return func(*args, **kwargs)
92 wrapper._original_function = func
93 return wrapper
/opt/conda/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
1416 use_multiprocessing=use_multiprocessing,
1417 shuffle=shuffle,
-> 1418 initial_epoch=initial_epoch)
1419
1420 @interfaces.legacy_generator_methods_support
/opt/conda/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
38
39 do_validation = bool(validation_data)
---> 40 model._make_train_function()
41 if do_validation:
42 model._make_test_function()
/opt/conda/lib/python3.6/site-packages/keras/engine/training.py in _make_train_function(self)
507 training_updates = self.optimizer.get_updates(
508 params=self._collected_trainable_weights,
--> 509 loss=self.total_loss)
510 updates = (self.updates +
511 training_updates +
/opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
89 warnings.warn('Update your `' + object_name + '` call to the ' +
90 'Keras 2 API: ' + signature, stacklevel=2)
---> 91 return func(*args, **kwargs)
92 wrapper._original_function = func
93 return wrapper
/opt/conda/lib/python3.6/site-packages/keras/optimizers.py in get_updates(self, loss, params)
182 @interfaces.legacy_get_updates_support
183 def get_updates(self, loss, params):
--> 184 grads = self.get_gradients(loss, params)
185 self.updates = [K.update_add(self.iterations, 1)]
186
/opt/conda/lib/python3.6/site-packages/keras/optimizers.py in get_gradients(self, loss, params)
89 grads = K.gradients(loss, params)
90 if None in grads:
---> 91 raise ValueError('An operation has `None` for gradient. '
92 'Please make sure that all of your ops have a '
93 'gradient defined (i.e. are differentiable). '
ValueError: An operation has `None` for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.
---------------------------------------------------------------------------
ValueError回溯(最近一次调用上次)
在里面
3个历元=历元,每个历元的步骤=顺序,
4.
---->5详细=1,最大队列大小=15)
/包装器中的opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py(*args,**kwargs)
89 warnings.warn('Update your`'+object\u name+'`调用+
90'Keras 2 API:'+签名,堆栈级别=2)
--->91返回函数(*args,**kwargs)
92包装器._原始函数=func
93返回包装器
/opt/conda/lib/python3.6/site-packages/keras/engine/training.py-in-fit\u生成器(self、生成器、每个历元的步骤、历元、冗余、回调、验证数据、验证步骤、类权重、最大队列大小、工人、使用多处理、无序、初始历元)
1416使用多处理=使用多处理,
1417洗牌=洗牌,
->1418初始_历元=初始_历元)
1419
1420@interfaces.legacy\u生成器\u方法\u支持
/opt/conda/lib/python3.6/site-packages/keras/engine/training\u generator.py in-fit\u generator(模型、生成器、每个历元的步骤、历元、详细信息、回调、验证数据、验证步骤、类权重、最大队列大小、工人、使用多处理、无序、初始历元)
38
39 do\U验证=bool(验证数据)
--->40型。_make_train_function()
41如果进行验证:
42型号.\u制造\u测试\u功能()
/opt/conda/lib/python3.6/site-packages/keras/engine/training.py in_make_train_函数(self)
507培训\u更新=self.optimizer.get\u更新(
508参数=自身收集的可训练重量,
-->509损失=自身总损失)
510更新=(自我更新)+
511培训课程更新+
/包装器中的opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py(*args,**kwargs)
89 warnings.warn('Update your`'+object\u name+'`调用+
90'Keras 2 API:'+签名,堆栈级别=2)
--->91返回函数(*args,**kwargs)
92包装器._原始函数=func
93返回包装器
/获取更新(self、loss、params)中的opt/conda/lib/python3.6/site-packages/keras/optimizers.py
182@interfaces.legacy\u获取更新\u支持
183 def get_更新(自我、丢失、参数):
-->184梯度=自获取梯度(损失、参数)
185 self.updates=[K.update\u add(self.iterations,1)]
186
/获取梯度中的opt/conda/lib/python3.6/site-packages/keras/optimizers.py(self、loss、params)
89梯度=K梯度(损失,参数)
90如果没有梯度:
--->91 raise VALUETERROR('一个操作的梯度为'None'
92“请确保您的所有老年退休金计划都有”
93'定义的梯度(即可微)。'
ValueError:操作的梯度为“无”。请确保所有操作都定义了梯度(即可微分)。没有梯度的常见操作:K.argmax、K.round、K.eval。
当然,问题是,层中的权重没有得到更新,因此是非梯度的。如果不制作自定义层,我怎么能做到这一点?我尝试过这种方法,但我放弃了使用层来测量val_acc之类的东西。如果没有层的API,在Keras中似乎无法做到这一点。您可以使用cust尝试这种解决方案om层:事实上,这不起作用。我修复了代码使其起作用-它确实运行了,但有一个层是有问题的。这个解决方案在这里工作得更好,感谢您的关注。