Pytorch OPTORCH:目标函数返回的值不能转换为浮点值
正如你在上面所看到的,我正在尝试运行Optuna试验,为我的CNN模型寻找最佳的超参数Pytorch OPTORCH:目标函数返回的值不能转换为浮点值,pytorch,cnn,optuna,pytorch-ignite,Pytorch,Cnn,Optuna,Pytorch Ignite,正如你在上面所看到的,我正在尝试运行Optuna试验,为我的CNN模型寻找最佳的超参数 def autotune(trial): cfg= { 'device' : "cuda" if torch.cuda.is_available() else "cpu", # 'train_batch_size' : 64, # 'test_batch_size' : 1000, # 'n
def autotune(trial):
cfg= { 'device' : "cuda" if torch.cuda.is_available() else "cpu",
# 'train_batch_size' : 64,
# 'test_batch_size' : 1000,
# 'n_epochs' : 1,
# 'seed' : 0,
# 'log_interval' : 100,
# 'save_model' : False,
# 'dropout_rate' : trial.suggest_uniform('dropout_rate',0,1.0),
'lr' : trial.suggest_loguniform('lr', 1e-3, 1e-2),
'momentum' : trial.suggest_uniform('momentum', 0.4, 0.99),
'optimizer': trial.suggest_categorical('optimizer',[torch.optim.Adam,torch.optim.SGD, torch.optim.RMSprop, torch.optim.$
'activation': F.tanh}
optimizer = cfg['optimizer'](model.parameters(), lr=cfg['lr'])
#optimizer = torch.optim.Adam(model.parameters(),lr=0.001
但是,当我运行上面的代码来优化并找到我的最佳参数时,出现了如下错误,似乎试验失败了,尽管我仍然得到了历元损失和值。请告知,谢谢
# Train the model
# use small epoch for large dataset
# An epoch is 1 run through all the training data
# losses = [] # use this array for plotting losses
for _ in range(epochs):
# using data_loader
for i, (data, labels) in enumerate(trainloader):
# Forward and get a prediction
# x is the training data which is X_train
if name.lower() == "rnn":
model.hidden = (torch.zeros(1,1,model.hidden_sz),
torch.zeros(1,1,model.hidden_sz))
y_pred = model.forward(data)
# compute loss/error by comparing predicted out vs acutal labels
loss = criterion(y_pred, labels)
#losses.append(loss)
if i%10==0: # print out loss at every 10 epoch
print(f'epoch {i} and loss is: {loss}')
#Backpropagation
optimizer.zero_grad()
loss.backward()
optimizer.step()
study = optuna.create_study(sampler=optuna.samplers.TPESampler(), direction='minimize',pruner=optuna.pruners.SuccessiveHalvingPrune$
study.optimize(autotune, n_trials=1)
[W 2020-11-11 13:59:48000]试用0失败,因为无法将目标函数返回的值转换为float。返回值为:无
回溯(最近一次呼叫最后一次):
文件“自动调谐2”,第481行,在
n_实例、n_特征、分数=运行_分析()
运行分析中第350行的文件“autotune2”
打印(研究最佳参数)
文件“/home/shar/anaconda3/lib/python3.7/site packages/optuna/study.py”,第67行,最佳参数
返回self.best_.params
文件“/home/shar/anaconda3/lib/python3.7/site packages/optuna/study.py”,第92行,在best_试用版中
返回副本。深度副本(自存储。获取最佳试用(自学习id))
文件“/home/shar/anaconda3/lib/python3.7/site packages/optuna/storages/_in_memory.py”,第287行,在get_best_试用版中
raise VALUE ERROR(“尚未完成任何试验”)
ValueError:尚未完成任何测试。
引发此异常是因为您研究的目标函数必须返回浮点值
在您的情况下,问题在于这一行:
[W 2020-11-11 13:59:48,000] Trial 0 failed, because the returned value from the objective function cannot be cast to float. Returned value is: None
Traceback (most recent call last):
File "autotune2", line 481, in <module>
n_instances, n_features, scores = run_analysis()
File "autotune2", line 350, in run_analysis
print(study.best_params)
File "/home/shar/anaconda3/lib/python3.7/site-packages/optuna/study.py", line 67, in best_params
return self.best_trial.params
File "/home/shar/anaconda3/lib/python3.7/site-packages/optuna/study.py", line 92, in best_trial
return copy.deepcopy(self._storage.get_best_trial(self._study_id))
File "/home/shar/anaconda3/lib/python3.7/site-packages/optuna/storages/_in_memory.py", line 287, in get_best_trial
raise ValueError("No trials are completed yet.")
ValueError: No trials are completed yet.
您之前定义的自动调谐功能不返回值,无法用于优化
如何修复
对于超参数搜索,autotune函数必须返回经过一些训练后可以得到的度量值,如损失或交叉熵
对代码的快速修复可能如下所示:
study.optimize(autotune, n_trials=1)
optunarepo中有一个很好的例子,它使用pythoch回调来检索准确性(但是如果需要,可以很容易地更改为使用RMSE)。它还使用多个实验,并对超参数取中值
def autotune():
cfg= { 'device' : "cuda" if torch.cuda.is_available() else "cpu"
...etc...
}
best_loss = 1e100; # or larger
# Train the model
for _ in range(epochs):
for i, (data, labels) in enumerate(trainloader):
... (train the model) ...
# compute loss/error by comparing predicted out vs actual labels
loss = criterion(y_pred, labels)
best_loss = min(loss,best_loss)
return best_loss