Python 随机输入实现的梯度下降法
我试图在数据集上实现梯度下降。尽管我什么都试过了,但还是没能成功。所以,我创建了一个测试用例。我正在随机数据上尝试我的代码,并尝试调试 更具体地说,我正在做的是,我正在为这些向量生成0-1和随机标签之间的随机向量。并尝试过拟合训练数据 然而,我的权重向量在每次迭代中变得越来越大。然后,我有无穷大。所以,我实际上什么都没学到。这是我的密码:Python 随机输入实现的梯度下降法,python,machine-learning,gradient-descent,Python,Machine Learning,Gradient Descent,我试图在数据集上实现梯度下降。尽管我什么都试过了,但还是没能成功。所以,我创建了一个测试用例。我正在随机数据上尝试我的代码,并尝试调试 更具体地说,我正在做的是,我正在为这些向量生成0-1和随机标签之间的随机向量。并尝试过拟合训练数据 然而,我的权重向量在每次迭代中变得越来越大。然后,我有无穷大。所以,我实际上什么都没学到。这是我的密码: import numpy as np import random def getRandomVector(n): return np.random.u
import numpy as np
import random
def getRandomVector(n):
return np.random.uniform(0,1,n)
def getVectors(m, n):
return [getRandomVector(n) for i in range(n)]
def getLabels(n):
return [random.choice([-1,1]) for i in range(n)]
def GDLearn(vectors, labels):
maxIterations = 100
stepSize = 0.01
w = np.zeros(len(vectors[0])+1)
for i in range(maxIterations):
deltaw = np.zeros(len(vectors[0])+1)
for i in range(len(vectors)):
temp = np.append(vectors[i], -1)
deltaw += ( labels[i] - np.dot(w, temp) ) * temp
w = w + ( stepSize * (-1 * deltaw) )
return w
vectors = getVectors(100, 30)
labels = getLabels(100)
w = GDLearn(vectors, labels)
print w
我使用LMS的损失函数。因此,在所有迭代中,我的更新如下:
其中w^i是第i个权重向量,R是步长,E(w^i)是损失函数
这是我的损失函数。(LMS)
这是我如何推导损失函数的
,
现在,我的问题是:
maxIterations
和stepSize
参数。仍然不起作用。
PS2:这是我在这里提问的最好方式。对不起,问题太具体了。但这让我发疯了。我真的很想了解这个问题。您的代码有几个错误:
- 在
方法中,实际上没有使用输入变量GetVectors()
李>m
- 在
方法中,您有一个双循环,但在两个循环中使用相同的变量GDLearn()
。(我想逻辑仍然正确,但令人困惑)i
- 预测错误(
)的符号错误李>标签[i]-np.dot(w,temp)
- 步长很重要。如果我使用0.01作为步长,那么每次迭代的成本都会增加。将其更改为0.001解决了问题李>
cost at 0 = 100.0
cost at 1 = 99.4114482617
cost at 2 = 98.8476022685
cost at 3 = 98.2977744556
cost at 4 = 97.7612851154
cost at 5 = 97.2377571222
cost at 6 = 96.7268325883
cost at 7 = 96.2281642899
cost at 8 = 95.7414151147
cost at 9 = 95.2662577529
cost at 10 = 94.8023744037
......
cost at 90 = 77.367904046
cost at 91 = 77.2744249433
cost at 92 = 77.1823702888
cost at 93 = 77.0917090883
cost at 94 = 77.0024111475
cost at 95 = 76.9144470493
cost at 96 = 76.8277881325
cost at 97 = 76.7424064707
cost at 98 = 76.6582748518
cost at 99 = 76.5753667579
[ 0.16232142 -0.2425511 0.35740632 0.22548442 0.03963853 0.19595213
0.20080207 -0.3921798 -0.0238925 0.13097533 -0.1148932 -0.10077534
0.00307595 -0.30111942 -0.17924479 -0.03838637 -0.23938181 0.1384443
0.22929163 -0.0132466 0.03325976 -0.31489526 0.17468025 0.01351012
-0.25926117 0.09444201 0.07637793 -0.05940019 0.20961315 0.08491858
0.07438357]
您的代码有明显的错误。我很快会在帖子中回复。我在某处丢失了一个负号。问题解决了。谢谢你
cost at 0 = 100.0
cost at 1 = 99.4114482617
cost at 2 = 98.8476022685
cost at 3 = 98.2977744556
cost at 4 = 97.7612851154
cost at 5 = 97.2377571222
cost at 6 = 96.7268325883
cost at 7 = 96.2281642899
cost at 8 = 95.7414151147
cost at 9 = 95.2662577529
cost at 10 = 94.8023744037
......
cost at 90 = 77.367904046
cost at 91 = 77.2744249433
cost at 92 = 77.1823702888
cost at 93 = 77.0917090883
cost at 94 = 77.0024111475
cost at 95 = 76.9144470493
cost at 96 = 76.8277881325
cost at 97 = 76.7424064707
cost at 98 = 76.6582748518
cost at 99 = 76.5753667579
[ 0.16232142 -0.2425511 0.35740632 0.22548442 0.03963853 0.19595213
0.20080207 -0.3921798 -0.0238925 0.13097533 -0.1148932 -0.10077534
0.00307595 -0.30111942 -0.17924479 -0.03838637 -0.23938181 0.1384443
0.22929163 -0.0132466 0.03325976 -0.31489526 0.17468025 0.01351012
-0.25926117 0.09444201 0.07637793 -0.05940019 0.20961315 0.08491858
0.07438357]