Python 在keras中使用delta规则
我试图构建一个线性单层感知器(即没有隐藏层,所有输入连接到所有输出,线性激活函数),并使用增量规则一次训练一个数据点,但我没有得到我期望的结果。我使用均方误差作为我的损失函数,其导数应该会导致重量更新,这只是学习_rate*error(*2),但不知何故,结果看起来与我的手动计算非常不同。我错过了什么Python 在keras中使用delta规则,python,machine-learning,keras,gradient-descent,perceptron,Python,Machine Learning,Keras,Gradient Descent,Perceptron,我试图构建一个线性单层感知器(即没有隐藏层,所有输入连接到所有输出,线性激活函数),并使用增量规则一次训练一个数据点,但我没有得到我期望的结果。我使用均方误差作为我的损失函数,其导数应该会导致重量更新,这只是学习_rate*error(*2),但不知何故,结果看起来与我的手动计算非常不同。我错过了什么 import numpy as np from keras.models import Sequential from keras.optimizers import SGD from keras
import numpy as np
from keras.models import Sequential
from keras.optimizers import SGD
from keras.layers import Dense
features = np.array([[1,0,1],[0,1,1]])
features = np.tile(features, (500,1))
labels = np.array([[1,0],[0,1]])
labels = np.tile(labels, (500,1))
network = Sequential()
network.add(Dense(2, input_dim = 3, init = "zero", activation = "linear"))
network.compile(loss = "mse", optimizer = SGD(lr = 0.01))
network.fit(features, labels, nb_epoch = 1, batch_size = 1, shuffle = False)
network.get_weights()
# [[ 0.59687883, -0.39686254],
# [-0.39689422, 0.59687883],
# [ 0.19998412, 0.20001581]],
# manually
weights = np.array([[0.0,0.0],[0.0,0.0],[0.0,0.0]])
for i in range(500):
summed_out1 = weights[0,0] + weights[2,0]
summed_out2 = weights[0,1] + weights[2,1]
change_out1 = 0.01 * (1.0 - summed_out1)
change_out2 = 0.01 * (0.0 - summed_out2)
weights[0,0] += change_out1
weights[2,0] += change_out1
weights[0,1] += change_out2
weights[2,1] += change_out2
#
summed_out1 = weights[1,0] + weights[2,0]
summed_out2 = weights[1,1] + weights[2,1]
change_out1 = 0.01 * (0.0 - summed_out1)
change_out2 = 0.01 * (1.0 - summed_out2)
weights[1,0] += change_out1
weights[2,0] += change_out1
weights[1,1] += change_out2
weights[2,1] += change_out2
weights
# [[ 0.66346388, -0.33011442],
# [-0.33014677, 0.66346388],
# [ 0.33331711, 0.33334946]]
我发现了问题。默认情况下,密集层包含偏差-一旦更改,网络将显示所需的行为