Neural network 如何在反向传播期间更改网络的当前输出的NO梯度？
我正试着编写一个从千层面/Theano中（方程式11）反演梯度法的例子。基本上，我试图做的是确保网络的输出在某些指定的范围内，在本例中为[1，-1] 我一直在看给出的例子，这对我很有帮助，但在这一点上我被卡住了。我认为执行此操作的最佳位置是渐变计算方法，因此我复制并尝试在应用更新之前编辑渐变 这就是我目前所拥有的Neural network 如何在反向传播期间更改网络的当前输出的NO梯度？,neural-network,theano,backpropagation,lasagne,Neural Network,Theano,Backpropagation,Lasagne,我正试着编写一个从千层面/Theano中（方程式11）反演梯度法的例子。基本上，我试图做的是确保网络的输出在某些指定的范围内，在本例中为[1，-1] 我一直在看给出的例子，这对我很有帮助，但在这一点上我被卡住了。我认为执行此操作的最佳位置是渐变计算方法，因此我复制并尝试在应用更新之前编辑渐变 这就是我目前所拥有的 def rmspropWithInvert(loss_or_grads, params, p, learning_rate=1.0, rho=0.9, epsilon=1e-6):
def rmspropWithInvert(loss_or_grads, params, p, learning_rate=1.0, rho=0.9, epsilon=1e-6):
clip = 2.0
grads = lasagne.updates.get_or_compute_grads(loss_or_grads, params)
# grads = theano.gradient.grad_clip(grads, -clip, clip)
grads_ = []
for grad in grads:
grads_.append(theano.gradient.grad_clip(grad, -clip, clip) )
grads = grads_
a, p_ = T.scalars('a', 'p_')
z_lazy = ifelse(T.gt(a,0.0), (1.0-p_)/(2.0), (p_-(-1.0))/(2.0))
f_lazyifelse = theano.function([a,p_], z_lazy,
mode=theano.Mode(linker='vm'))
# compute the parameter vector to invert the gradients by
ps = theano.shared(
np.zeros((3, 1), dtype=theano.config.floatX),
broadcastable=(False, True))
for i in range(3):
ps[i] = f_lazyifelse(grads[-1][i], p[i])
# Apply vector through computed gradients
grads2=[]
for grad in grads.reverse():
grads2.append(theano.mul(ps, grad))
ps = grad
grads = grads2.reverse()
print "Grad Update: " + str(grads[0])
updates = OrderedDict()
# Using theano constant to prevent upcasting of float32
one = T.constant(1)
for param, grad in zip(params, grads):
value = param.get_value(borrow=True)
accu = theano.shared(np.zeros(value.shape, dtype=value.dtype),
broadcastable=param.broadcastable)
accu_new = rho * accu + (one - rho) * grad ** 2
updates[accu] = accu_new
updates[param] = param - (learning_rate * grad /
T.sqrt(accu_new + epsilon))
return updates