Python Keras-Tensorflow中的加权样本损失
我想建立一个损失,把每个样本的个体重量,不仅在训练期间有效。它还应该用于验证和测试集 到目前为止,我所尝试的:Python Keras-Tensorflow中的加权样本损失,python,tensorflow,keras,loss,Python,Tensorflow,Keras,Loss,我想建立一个损失,把每个样本的个体重量,不仅在训练期间有效。它还应该用于验证和测试集 到目前为止,我所尝试的: def MMSE2(targets, preds,sample_weight): #some calculations... return loss input_dim = Input(shape = (dim, ),name='rating_in') weights_tensor = Input(shape=
def MMSE2(targets, preds,sample_weight):
#some calculations...
return loss
input_dim = Input(shape = (dim, ),name='rating_in')
weights_tensor = Input(shape=(dim,),name='weights')
encoder,decoder = AddLayers(neurons,setup['AFunction'],
setup['BatchNorm'],setup['Dropout'],setup['Layers'],dim,setup['Noise'])
encoded = encoder(input_dim)
decoded = decoder(encoded)
autoencoder = Model([input_dim,weights_tensor], decoded)
autoencoder.add_loss(MMSE2(input_dim,decoded,weights_tensor))
autoencoder.compile(optimizer='adam')
history = autoencoder.fit(x=[helper.trainx,helper.trainy,helper.trainm],
validation_data= [helper.valx,helper.valy,helper.valm],
epochs = setup['Epochs'], batch_size = setup['BatchSize'])
它在没有验证数据的情况下工作
验证数据错误:
File "<ipython-input-11-fe466c688bcd>", line 3, in <module>
epochs = setup['Epochs'], batch_size = setup['BatchSize'])
File "C:\Users\Admin\Anaconda3\envs\tf21\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 819, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\Admin\Anaconda3\envs\tf21\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 235, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\Admin\Anaconda3\envs\tf21\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 614, in _process_training_inputs
distribution_strategy=distribution_strategy)
File "C:\Users\Admin\Anaconda3\envs\tf21\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 646, in _process_inputs
x, y, sample_weight=sample_weights)
File "C:\Users\Admin\Anaconda3\envs\tf21\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 2383, in _standardize_user_data
batch_size=batch_size)
File "C:\Users\Admin\Anaconda3\envs\tf21\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 2410, in _standardize_tensors
exception_prefix='input')
File "C:\Users\Admin\Anaconda3\envs\tf21\lib\site-packages\tensorflow_core\python\keras\engine\training_utils.py", line 539, in standardize_input_data
str(data)[:200] + '...')
ValueError: Error when checking model input:
the list of Numpy arrays that you are passing to your model is not the size the model expected.
Expected to see 2 array(s), for inputs ['rating_in', 'weights']
but instead got the following list of 1 arrays: [array([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 5....
文件“”,第3行,在
epochs=setup['epochs'],batch_size=setup['BatchSize'])
文件“C:\Users\Admin\Anaconda3\envs\tf21\lib\site packages\tensorflow\u core\python\keras\engine\training.py”,第819行
使用多处理=使用多处理)
文件“C:\Users\Admin\Anaconda3\envs\tf21\lib\site packages\tensorflow\u core\python\keras\engine\training\u v2.py”,第235行
使用多处理=使用多处理)
文件“C:\Users\Admin\Anaconda3\envs\tf21\lib\site packages\tensorflow\u core\python\keras\engine\training\u v2.py”,第614行,在流程培训输入中
分销策略=分销策略)
文件“C:\Users\Admin\Anaconda3\envs\tf21\lib\site packages\tensorflow\u core\python\keras\engine\training\u v2.py”,第646行,在进程输入中
x、 y,样本重量=样本重量)
文件“C:\Users\Admin\Anaconda3\envs\tf21\lib\site packages\tensorflow\u core\python\keras\engine\training.py”,第2383行,在用户数据中
批次大小=批次大小)
文件“C:\Users\Admin\Anaconda3\envs\tf21\lib\site packages\tensorflow\u core\python\keras\engine\training.py”,第2410行,在张量中
异常(前缀为“输入”)
标准化输入数据中的文件“C:\Users\Admin\Anaconda3\envs\tf21\lib\site packages\tensorflow\u core\python\keras\engine\training\u utils.py”第539行
str(数据)[:200]+''.')
ValueError:检查模型输入时出错:
传递给模型的Numpy数组列表的大小不是模型所期望的大小。
预计将看到2个数组,用于输入['rating_in','weights']
但是得到了以下1个数组的列表:[array([[0,0,0.,…,0,0,0.]),
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 5....
我已经从.fit()方法切换到了低级API方法,您可以在循环中记录培训内容
然后改变损失函数
def MMSE(targets,mask,preds):
num_rating = math.reduce_sum(mask) #count ratings
loss = targets-preds
loss = mask * loss
loss = math.square(loss)
loss = math.reduce_sum(loss)
loss = loss / num_rating
#in a single line
#loss = math.reduce_sum(math.square(mask*(preds - targets))) / num_rating
return loss
我已经从.fit()方法切换到了低级API方法,您可以在循环中记录训练 然后改变损失函数
def MMSE(targets,mask,preds):
num_rating = math.reduce_sum(mask) #count ratings
loss = targets-preds
loss = mask * loss
loss = math.square(loss)
loss = math.reduce_sum(loss)
loss = loss / num_rating
#in a single line
#loss = math.reduce_sum(math.square(mask*(preds - targets))) / num_rating
return loss