Deep learning Pytorch运行时错误:“;主机“U softmax”;未针对'实施;torch.cuda.LongTensor&x27;
我正在使用Pytork训练模型。但我在计算交叉熵损失时遇到了运行时错误Deep learning Pytorch运行时错误:“;主机“U softmax”;未针对'实施;torch.cuda.LongTensor&x27;,deep-learning,pytorch,Deep Learning,Pytorch,我正在使用Pytork训练模型。但我在计算交叉熵损失时遇到了运行时错误 Traceback (most recent call last): File "deparser.py", line 402, in <module> d.train() File "deparser.py", line 331, in train total, correct, avgloss = self.train_util() File "deparser.py", line
Traceback (most recent call last):
File "deparser.py", line 402, in <module>
d.train()
File "deparser.py", line 331, in train
total, correct, avgloss = self.train_util()
File "deparser.py", line 362, in train_util
loss = self.step(X_train, Y_train, correct, total)
File "deparser.py", line 214, in step
loss = nn.CrossEntropyLoss()(out.long(), y)
File "/home/summer2018/TF/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/summer2018/TF/lib/python3.5/site-packages/torch/nn/modules/loss.py", line 862, in forward
ignore_index=self.ignore_index, reduction=self.reduction)
File "/home/summer2018/TF/lib/python3.5/site-packages/torch/nn/functional.py", line 1550, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
File "/home/summer2018/TF/lib/python3.5/site-packages/torch/nn/functional.py", line 975, in log_softmax
return input.log_softmax(dim)
RuntimeError: "host_softmax" not implemented for 'torch.cuda.LongTensor'
此函数step()由以下函数调用:
def train_util(self):
total = 0
correct = 0
avgloss = 0
for i in range(self.step_num_per_epoch):
X_train, Y_train = self.trainloader()
self.optimizer.zero_grad()
if torch.cuda.is_available():
self.cuda()
for i in range(len(X_train)):
X_train[i] = Variable(torch.from_numpy(X_train[i]))
X_train[i].requires_grad = False
X_train[i] = X_train[i].cuda()
Y_train = torch.from_numpy(Y_train)
Y_train.requires_grad = False
Y_train = Y_train.cuda()
loss = self.step(X_train, Y_train, correct, total)
avgloss+=float(loss)*Y_train.size(0)
self.optimizer.step()
if i%100==99:
print('STEP %d, Loss: %.4f, Acc: %.4f'%(i+1,loss,correct/total))
return total, correct, avgloss/self.data_len
输入数据X\u train,Y\u train=self.trainloader()
一开始就是numpy数组
这是一个数据示例:
>>> X_train, Y_train = d.trainloader()
>>> X_train[0].dtype
dtype('int64')
>>> X_train[1].dtype
dtype('int64')
>>> X_train[2].dtype
dtype('int64')
>>> Y_train.dtype
dtype('float32')
>>> X_train[0]
array([[ 0, 6, 0, ..., 0, 0, 0],
[ 0, 1944, 8168, ..., 0, 0, 0],
[ 0, 815, 317, ..., 0, 0, 0],
...,
[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 23, 6, ..., 0, 0, 0],
[ 0, 0, 297, ..., 0, 0, 0]])
>>> X_train[1]
array([ 6, 7, 8, 21, 2, 34, 3, 4, 19, 14, 15, 2, 13, 3, 11, 22, 4,
13, 34, 10, 13, 3, 48, 18, 16, 19, 16, 17, 48, 3, 3, 13])
>>> X_train[2]
array([ 4, 5, 8, 36, 2, 33, 5, 3, 17, 16, 11, 0, 9, 3, 10, 20, 1,
14, 33, 25, 19, 1, 46, 17, 14, 24, 15, 15, 51, 2, 1, 14])
>>> Y_train
array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1.],
[0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
...,
[0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]],
dtype=float32)
尝试所有可能的组合:
案例1:loss=nn.CrossEntropyLoss()(out,y)
我得到:
RuntimeError:预期对象的类型为torch.cuda.LongTensor,但找到参数#2'target'的类型为torch.cuda.FloatTensor
案例二:loss=nn.CrossEntropyLoss()(out.long(),y)
如上所述 案例3:
loss=nn.CrossEntropyLoss()(out.float(),y)
我得到:
RuntimeError:预期对象的类型为torch.cuda.LongTensor,但找到参数#2'target'的类型为torch.cuda.FloatTensor
案例4:loss=nn.CrossEntropyLoss()(out,y.long())
我得到:
RuntimeError:at/pytorch/aten/src/THCUNN/generic/ClassNLLCriterion.cu:15不支持多目标
案例5:
loss=nn.CrossEntropyLoss()(out.long(),y.long())
我得到:
运行时错误:“host_softmax”未为“torch.cuda.LongTensor”实现
案例6:
loss=nn.CrossEntropyLoss()(out.float(),y.long())
我得到:
RuntimeError:at/pytorch/aten/src/THCUNN/generic/ClassNLLCriterion.cu:15不支持多目标
案例7:
loss=nn.CrossEntropyLoss()(out,y.float())
我得到:
RuntimeError:预期对象的类型为torch.cuda.LongTensor,但找到参数#2'target'的类型为torch.cuda.FloatTensor
案例8:
loss=nn.CrossEntropyLoss()(out.long(),y.float())
我得到:
运行时错误:“host_softmax”未为“torch.cuda.LongTensor”实现
案例9:
loss=nn.CrossEntropyLoss()(out.float(),y.float())
我得到:
运行时错误:预期对象的类型为torch.cuda.LongTensor,但找到参数#2“target”的类型为torch.cuda.FloatTensor
我知道问题出在哪里
y
应该在torch.int64
dtype中,不使用一个热编码。
而CrossEntropyLoss()
将使用一个hot自动编码它(out是预测的概率分布,就像一个hot格式一样)
它现在可以运行了 在我的例子中,这是因为我翻转了目标
和logits
,而且logits显然不是torch。int64
引发了错误。因此,您使用长张量计算交叉熵。您能否共享导致此问题的特定数据样本?你是怎么把它转换成浮动的?什么是y
?您好!谢谢你的评论!我在上面的描述中添加了数据格式。下面的注释是我如何尝试将其转换为float的:案例1:loss=nn.CrossEntropyLoss()(out,y)
I get:RuntimeError:torch.cuda.LongTensor类型的预期对象,但为参数#2'target'找到了torch.cuda.FloatTensor类型
案例2:loss=nn.CrossEntropyLoss()(out.long(),y)
如上所述哦!我知道问题在哪里<代码>y
应为torch.int64数据类型,无需一次热编码。CrossEntropyLoss()将使用一个hot自动编码它(而out
是预测的概率分布,就像一个hot格式一样)。它现在可以运行了!谢谢你的帮助!再澄清一点:输入:(N,C)其中C=目标类的数量:(N)其中每个值都是0≤目标[i]≤C−1.
>>> X_train, Y_train = d.trainloader()
>>> X_train[0].dtype
dtype('int64')
>>> X_train[1].dtype
dtype('int64')
>>> X_train[2].dtype
dtype('int64')
>>> Y_train.dtype
dtype('float32')
>>> X_train[0]
array([[ 0, 6, 0, ..., 0, 0, 0],
[ 0, 1944, 8168, ..., 0, 0, 0],
[ 0, 815, 317, ..., 0, 0, 0],
...,
[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 23, 6, ..., 0, 0, 0],
[ 0, 0, 297, ..., 0, 0, 0]])
>>> X_train[1]
array([ 6, 7, 8, 21, 2, 34, 3, 4, 19, 14, 15, 2, 13, 3, 11, 22, 4,
13, 34, 10, 13, 3, 48, 18, 16, 19, 16, 17, 48, 3, 3, 13])
>>> X_train[2]
array([ 4, 5, 8, 36, 2, 33, 5, 3, 17, 16, 11, 0, 9, 3, 10, 20, 1,
14, 33, 25, 19, 1, 46, 17, 14, 24, 15, 15, 51, 2, 1, 14])
>>> Y_train
array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1.],
[0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
...,
[0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]],
dtype=float32)