Python 千层面的学习率衰减
我使用nesterov动量来更新卷积神经网络的权重。我在用千层面做CNN。如何实现每个时代的学习率衰减Python 千层面的学习率衰减,python,theano,deep-learning,lasagne,Python,Theano,Deep Learning,Lasagne,我使用nesterov动量来更新卷积神经网络的权重。我在用千层面做CNN。如何实现每个时代的学习率衰减 prediction = lasagne.layers.get_output(network) loss = lasagne.objectives.categorical_crossentropy(prediction, target_var) params = lasagne.layers.get_all_params(network, trainable=True) lasagne.upd
prediction = lasagne.layers.get_output(network)
loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)
params = lasagne.layers.get_all_params(network, trainable=True)
lasagne.updates.nesterov_momentum(loss, params, learning_rate, momentum=0.9)
train_fn = theano.function([input_var_1, input_var_2, target_var], loss, updates=updates)
我想你遗漏了什么:
prediction = lasagne.layers.get_output(network)
loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)
params = lasagne.layers.get_all_params(network, trainable=True)
updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate, momentum=0.9)
train_fn = theano.function([input_var_1, input_var_2, target_var], loss, updates=updates)
我想你遗漏了什么:
prediction = lasagne.layers.get_output(network)
loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)
params = lasagne.layers.get_all_params(network, trainable=True)
updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate, momentum=0.9)
train_fn = theano.function([input_var_1, input_var_2, target_var], loss, updates=updates)
将您的学习率定义为图形元素
...
learning_rate = T.scalar(name='learning_rate')
updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate, momentum=0.9)
...
将您的学习率节点添加到培训中
train_fn = theano.function([input_var_1, input_var_2, target_var, learning_rate], loss, updates=updates)
现在,您可以调整学习速率,例如:
base_lr = 1e-4
lr_decay = 0.8
for epoch in range(epochs):
lr = base_lr * (lr_decay ** epoch)
loss, updates = train_fn(input_1, input_2, target, lr)
将您的学习率定义为图形元素
...
learning_rate = T.scalar(name='learning_rate')
updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate, momentum=0.9)
...
将您的学习率节点添加到培训中
train_fn = theano.function([input_var_1, input_var_2, target_var, learning_rate], loss, updates=updates)
现在,您可以调整学习速率,例如:
base_lr = 1e-4
lr_decay = 0.8
for epoch in range(epochs):
lr = base_lr * (lr_decay ** epoch)
loss, updates = train_fn(input_1, input_2, target, lr)
你能像Keras那样使用adam吗?如果是这样的话,你可以使用adam或one,如果它的变体…你能像Keras一样使用adam吗?如果是这样,你可以使用adam或one,如果它的变体。。。