Pytorch 找不到有效的cuDNN算法来运行卷积

Pytorch 找不到有效的cuDNN算法来运行卷积,pytorch,Pytorch,我在尝试运行前馈torch.nn.Conv2d时收到此消息,得到以下stacktrace: --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-26-04bd4a00565d> in <module&

我在尝试运行前馈torch.nn.Conv2d时收到此消息,得到以下stacktrace:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-26-04bd4a00565d> in <module>
      3 
      4 # call training function
----> 5 losses = train(D, G, n_epochs=n_epochs)

<ipython-input-24-b539315e0aa0> in train(D, G, n_epochs, print_every)
     46                 real_images = real_images.cuda()
     47 
---> 48             D_real = D(real_images)
     49             d_real_loss = real_loss(D_real, True) # smoothing label 1 => 0.9
     50 

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

<ipython-input-14-bf68e57c25ff> in forward(self, x)
     48         """
     49 
---> 50         x = self.leaky_relu(self.conv1(x))
     51         x = self.leaky_relu(self.conv2(x))
     52         x = self.leaky_relu(self.conv3(x))

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/container.py in forward(self, input)
     98     def forward(self, input):
     99         for module in self:
--> 100             input = module(input)
    101         return input
    102 

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/conv.py in forward(self, input)
    347 
    348     def forward(self, input):
--> 349         return self._conv_forward(input, self.weight)
    350 
    351 class Conv3d(_ConvNd):

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight)
    344                             _pair(0), self.dilation, self.groups)
    345         return F.conv2d(input, weight, self.bias, self.stride,
--> 346                         self.padding, self.dilation, self.groups)
    347 
    348     def forward(self, input):

RuntimeError: Unable to find a valid cuDNN algorithm to run convolution
我使用的是Python 3.7、Pytorch 1.5,GPU是Nvidia GeForce GTX 770,运行在Ubuntu 18.04.2上。我在任何地方都没有找到那个错误消息。它响了吗


提前非常感谢。

问题是您正在使用torch.nn.Module进行前馈,但您正在返回功能模块
F.conv2d()
。将返回代码更改为
nn.Conv2d()


这可能会帮助您更多-

我在测试不同EC2节点机器的推理速度时遇到了这个错误。当我翻阅日志时,我发现:

(pid=20839) /home/ubuntu/src/skai-ml/venv/lib/python3.7/site-packages/torch/cuda/__init__.py:87: UserWarning: 
(pid=20839)     Found GPU0 GRID K520 which is of cuda capability 3.0.
(pid=20839)     PyTorch no longer supports this GPU because it is too old.
(pid=20839)     The minimum cuda capability that we support is 3.5.
经验教训:不要对PyTorch模型使用
g2.XX
实例类型
g3.XX
p
系列工作正常。

根据tensorflow对类似问题的回答,可能是因为达到了VRAM内存限制(从错误消息来看,这是非常不直观的)


对于我的PyTorch模型培训案例,减少批量有帮助。您可以试试这个,或者减小型号以减少VRAM的消耗。

同样的事情也发生在我身上,这并不是一个非常直观的信息
(pid=20839) /home/ubuntu/src/skai-ml/venv/lib/python3.7/site-packages/torch/cuda/__init__.py:87: UserWarning: 
(pid=20839)     Found GPU0 GRID K520 which is of cuda capability 3.0.
(pid=20839)     PyTorch no longer supports this GPU because it is too old.
(pid=20839)     The minimum cuda capability that we support is 3.5.