Python 为Softmax回归计算pytorch中偏差b的梯度
我正在学习Pytork材料,我遇到了一个问题,当使用Pytork的autograd moudule进行反向传播时,我能够得到权重的梯度W,但不能得到SOFTMAX再压缩的偏差b 代码如下所示 问题是,我得到了b.grad=None,我期望的不是None值 我们如何修复这个错误 我们将不胜感激Python 为Softmax回归计算pytorch中偏差b的梯度,python,machine-learning,deep-learning,pytorch,softmax,Python,Machine Learning,Deep Learning,Pytorch,Softmax,我正在学习Pytork材料,我遇到了一个问题,当使用Pytork的autograd moudule进行反向传播时,我能够得到权重的梯度W,但不能得到SOFTMAX再压缩的偏差b 代码如下所示 问题是,我得到了b.grad=None,我期望的不是None值 我们如何修复这个错误 我们将不胜感激 import torch from torch.autograd import Variable dtype = torch.FloatTensor dtype2 = torch.LongTensor
import torch
from torch.autograd import Variable
dtype = torch.FloatTensor
dtype2 = torch.LongTensor
X_data = 10 * torch.rand(2, 300).type(dtype)
y_labels = torch.zeros(300).type(dtype2)
y_labels[0:100] = 1
y_labels[100:200] = 2
y_labels = y_labels.type(dtype2)
print(y_labels)
X = Variable(X_data, requires_grad = False)
y = Variable(y_labels, requires_grad = False)
N = X.shape[1]
decay = 0.01
W = Variable(torch.randn(3, 2).type(dtype), requires_grad = True)
b = Variable(torch.zeros(3, 1).type(dtype), requires_grad = True)
XT = X.t()
WT = W.t()
ones = Variable(torch.ones(1,N).type(dtype), requires_grad = False)
# print("XT.shape = {}".format(XT.shape)) # 300,2
# print("WT.shape = {}".format(WT.shape)) # 2,3
# print("b.shape = {}".format(b.shape)) # 3,1
# scores
z = torch.mm(XT, WT) + torch.mm(b,ones).t() # 300,3
# Get exponentials and probabilities
exp_z = torch.exp(z - torch.max(z))
probs = exp_z / torch.sum(exp_z) # 300,3
# loss
correct_probs = probs.data[0:N,y.data.type(torch.LongTensor)] # 300,
logprobs = torch.log(correct_probs)
loss_data = -1/N * torch.sum(logprobs)
# regularization
loss_reg = 0.5 * decay * torch.sum(W**2)
loss = loss_data + loss_reg
# gradients
loss.backward()
print("W = {}".format(W))
print("b = {}".format(b))
print("W.grad = {}".format(W.grad))
print("b.grad = {}".format(b.grad)) # this is None