Python torch Adamoptimizer在优化器中产生cuda错误。步骤()
使用3 Torch线性层添加自定义损耗函数后 我收到一个cuda错误Python torch Adamoptimizer在优化器中产生cuda错误。步骤(),python,pytorch,Python,Pytorch,使用3 Torch线性层添加自定义损耗函数后 我收到一个cuda错误 class KLDLoss(nn.Module): def __init__(self, reduction='sum'): super(KLDLoss, self).__init__() self.reduction = reduction def forward(self, mean, logvar): # KLD loss kld_loss = -0.5 * torc
class KLDLoss(nn.Module):
def __init__(self, reduction='sum'):
super(KLDLoss, self).__init__()
self.reduction = reduction
def forward(self, mean, logvar):
# KLD loss
kld_loss = -0.5 * torch.sum(1 + logvar - mean.pow(2) - logvar.exp(), 1)
# Size average
if self.reduction == 'mean':
kld_loss = torch.mean(kld_loss)
elif self.reduction == 'sum':
kld_loss = torch.sum(kld_loss)
return kld_loss
class Latent_Classifier(nn.Module):
def __init__(self):
super(Latent_Classifier, self).__init__()
layers = []
layers += [nn.Linear(128, 750)]
layers += [nn.Linear(750, 750)]
layers += [nn.Linear(750, 1)]
self.seq = nn.Sequential(*layers)
def forward(self, latent_z):
x = self.seq(latent_z)
return -torch.mean(torch.log(x)) - torch.mean(torch.log(1 - x))
KLDLoss没有错误,但在optimizer.step()中的某个训练阶段后,潜在分类器有错误
我的潜在分类器代码中是否存在错误
优化器是AdamOptimizer,参数是0.0002 lr,(0.5,0.999)betas,根据我的经验,这些类型的CUDA错误可能由两种原因引起:
- 尝试访问嵌入层中的越界索引
- 尝试执行无效操作,如日志为零或负值
所以我的猜测是:您试图在[0,1]间隔之外的某个对象上使用KLDiv[(不包括0和1)。在输出层中添加一个sigmoid激活,问题应该得到解决
您可以在CPU上运行代码,您将收到一条更有意义的错误消息
105 denom = (max_exp_avg_sq.sqrt() / math.sqrt(bias_correction2)).add_(group['eps'])
106 else:
--> 107 denom = (exp_avg_sq.sqrt() / math.sqrt(bias_correction2)).add_(group['eps'])
108
109 step_size = group['lr'] / bias_correction1
RuntimeError: CUDA error: device-side assert triggered