TensorFlow损失函数在第一个历元后归零
我正试图实现一个判别损失函数,例如基于本文的图像分割:(此链接仅供读者参考;我不希望任何人阅读它来帮助我!) 我的问题:一旦我从一个简单的损失函数转移到一个更复杂的损失函数(如您在所附的代码片段中所看到的),损失函数在第一个历元之后就会归零。我检查了重量,几乎所有的重量都在-300左右。它们并不完全相同,但彼此非常接近(仅在小数位上有所不同) 实现鉴别损失功能的相关代码:TensorFlow损失函数在第一个历元后归零,tensorflow,deep-learning,keras,image-segmentation,loss,Tensorflow,Deep Learning,Keras,Image Segmentation,Loss,我正试图实现一个判别损失函数,例如基于本文的图像分割:(此链接仅供读者参考;我不希望任何人阅读它来帮助我!) 我的问题:一旦我从一个简单的损失函数转移到一个更复杂的损失函数(如您在所附的代码片段中所看到的),损失函数在第一个历元之后就会归零。我检查了重量,几乎所有的重量都在-300左右。它们并不完全相同,但彼此非常接近(仅在小数位上有所不同) 实现鉴别损失功能的相关代码: def regDLF(y_true, y_pred): global alpha global beta
def regDLF(y_true, y_pred):
global alpha
global beta
global gamma
global delta_v
global delta_d
global image_height
global image_width
global nDim
y_true = tf.reshape(y_true, [image_height*image_width])
X = tf.reshape(y_pred, [image_height*image_width, nDim])
uniqueLabels, uniqueInd = tf.unique(y_true)
numUnique = tf.size(uniqueLabels)
Sigma = tf.unsorted_segment_sum(X, uniqueInd, numUnique)
ones_Sigma = tf.ones((tf.shape(X)[0], 1))
ones_Sigma = tf.unsorted_segment_sum(ones_Sigma,uniqueInd, numUnique)
mu = tf.divide(Sigma, ones_Sigma)
Lreg = tf.reduce_mean(tf.norm(mu, axis = 1))
T = tf.norm(tf.subtract(tf.gather(mu, uniqueInd), X), axis = 1)
T = tf.divide(T, Lreg)
T = tf.subtract(T, delta_v)
T = tf.clip_by_value(T, 0, T)
T = tf.square(T)
ones_Sigma = tf.ones_like(uniqueInd, dtype = tf.float32)
ones_Sigma = tf.unsorted_segment_sum(ones_Sigma,uniqueInd, numUnique)
clusterSigma = tf.unsorted_segment_sum(T, uniqueInd, numUnique)
clusterSigma = tf.divide(clusterSigma, ones_Sigma)
Lvar = tf.reduce_mean(clusterSigma, axis = 0)
mu_interleaved_rep = tf.tile(mu, [numUnique, 1])
mu_band_rep = tf.tile(mu, [1, numUnique])
mu_band_rep = tf.reshape(mu_band_rep, (numUnique*numUnique, nDim))
mu_diff = tf.subtract(mu_band_rep, mu_interleaved_rep)
mu_diff = tf.norm(mu_diff, axis = 1)
mu_diff = tf.divide(mu_diff, Lreg)
mu_diff = tf.subtract(2*delta_d, mu_diff)
mu_diff = tf.clip_by_value(mu_diff, 0, mu_diff)
mu_diff = tf.square(mu_diff)
numUniqueF = tf.cast(numUnique, tf.float32)
Ldist = tf.reduce_mean(mu_diff)
L = alpha * Lvar + beta * Ldist + gamma * Lreg
return L
def tf_norm(inputs, axis=1, epsilon=1e-7, name='safe_norm'):
squared_norm = tf.reduce_sum(tf.square(inputs), axis=axis, keep_dims=True)
safe_norm = tf.sqrt(squared_norm+epsilon)
return tf.identity(safe_norm, name=name)
问题:我知道不阅读论文就很难理解代码的作用,但我有几个问题:
非常感谢您的时间和帮助 在您的
Ldist
计算中,您使用tf.tile
和tf.reformate
以以下方式找到不同聚类均值之间的距离(假设我们有三个聚类):
mu_1-mu_1mu_2-mu_1
mu_3-mu_1
mu_1-mu_2
mu_2-mu_2
mu_3-mu_2
mu_1-mu_3
mu_2-mu_3
mu_3-mu_3 问题是距离向量包含零向量,然后执行范数运算
tf.norm
由于在向量长度上执行除法,因此数值不稳定。结果是梯度要么为零
要么为inf
。看这个
解决方法是以这样的方式删除这些零向量。我认为您的问题受到tf.norm的影响,这是不安全的(导致向量中的某个地方出现零,因此在其梯度中出现nan)。 最好用此自定义函数替换tf.norm:
def regDLF(y_true, y_pred):
global alpha
global beta
global gamma
global delta_v
global delta_d
global image_height
global image_width
global nDim
y_true = tf.reshape(y_true, [image_height*image_width])
X = tf.reshape(y_pred, [image_height*image_width, nDim])
uniqueLabels, uniqueInd = tf.unique(y_true)
numUnique = tf.size(uniqueLabels)
Sigma = tf.unsorted_segment_sum(X, uniqueInd, numUnique)
ones_Sigma = tf.ones((tf.shape(X)[0], 1))
ones_Sigma = tf.unsorted_segment_sum(ones_Sigma,uniqueInd, numUnique)
mu = tf.divide(Sigma, ones_Sigma)
Lreg = tf.reduce_mean(tf.norm(mu, axis = 1))
T = tf.norm(tf.subtract(tf.gather(mu, uniqueInd), X), axis = 1)
T = tf.divide(T, Lreg)
T = tf.subtract(T, delta_v)
T = tf.clip_by_value(T, 0, T)
T = tf.square(T)
ones_Sigma = tf.ones_like(uniqueInd, dtype = tf.float32)
ones_Sigma = tf.unsorted_segment_sum(ones_Sigma,uniqueInd, numUnique)
clusterSigma = tf.unsorted_segment_sum(T, uniqueInd, numUnique)
clusterSigma = tf.divide(clusterSigma, ones_Sigma)
Lvar = tf.reduce_mean(clusterSigma, axis = 0)
mu_interleaved_rep = tf.tile(mu, [numUnique, 1])
mu_band_rep = tf.tile(mu, [1, numUnique])
mu_band_rep = tf.reshape(mu_band_rep, (numUnique*numUnique, nDim))
mu_diff = tf.subtract(mu_band_rep, mu_interleaved_rep)
mu_diff = tf.norm(mu_diff, axis = 1)
mu_diff = tf.divide(mu_diff, Lreg)
mu_diff = tf.subtract(2*delta_d, mu_diff)
mu_diff = tf.clip_by_value(mu_diff, 0, mu_diff)
mu_diff = tf.square(mu_diff)
numUniqueF = tf.cast(numUnique, tf.float32)
Ldist = tf.reduce_mean(mu_diff)
L = alpha * Lvar + beta * Ldist + gamma * Lreg
return L
def tf_norm(inputs, axis=1, epsilon=1e-7, name='safe_norm'):
squared_norm = tf.reduce_sum(tf.square(inputs), axis=axis, keep_dims=True)
safe_norm = tf.sqrt(squared_norm+epsilon)
return tf.identity(safe_norm, name=name)
你的损失似乎是由三个条款组成的。为什么不改变这三个术语的权重,看看哪一个是有问题的?