Python 取消勾选保存的pytorch模型抛出属性错误:Can';t获取属性';净';on<;模块'__主&';尽管添加了类定义内联
我正试图在烧瓶应用程序中提供pytorch模型。这段代码在我之前在jupyter笔记本上运行时是有效的,但现在我在虚拟环境中运行这段代码,显然它无法获取属性“Net”,即使类定义就在那里。所有其他类似的问题都告诉我在同一个脚本中添加已保存模型的类定义。但它仍然不起作用。torch版本是1.0.1(保存的模型和virtualenv都经过培训) 我做错了什么? 这是我的密码Python 取消勾选保存的pytorch模型抛出属性错误:Can';t获取属性';净';on<;模块'__主&';尽管添加了类定义内联,python,pickle,pytorch,Python,Pickle,Pytorch,我正试图在烧瓶应用程序中提供pytorch模型。这段代码在我之前在jupyter笔记本上运行时是有效的,但现在我在虚拟环境中运行这段代码,显然它无法获取属性“Net”,即使类定义就在那里。所有其他类似的问题都告诉我在同一个脚本中添加已保存模型的类定义。但它仍然不起作用。torch版本是1.0.1(保存的模型和virtualenv都经过培训) 我做错了什么? 这是我的密码 import os import numpy as np from flask import Flask, request,
import os
import numpy as np
from flask import Flask, request, jsonify
import requests
import torch
from torch import nn
from torch.nn import functional as F
MODEL_URL = 'https://storage.googleapis.com/judy-pytorch-model/classifier.pt'
r = requests.get(MODEL_URL)
file = open("model.pth", "wb")
file.write(r.content)
file.close()
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.fc1 = nn.Linear(input_size, hidden_size)
self.fc2 = nn.Linear(hidden_size, hidden_size)
self.fc3 = nn.Linear(hidden_size, output_size)
def forward(self, x):
x = torch.sigmoid(self.fc1(x))
x = torch.sigmoid(self.fc2(x))
x = self.fc3(x)
return F.log_softmax(x, dim=-1)
model = torch.load('model.pth')
app = Flask(__name__)
@app.route("/")
def hello():
return "Binary classification example\n"
@app.route('/predict', methods=['GET'])
def predict():
x_data = request.args['x_data']
x_data = x_data.split()
x_data = list(map(float, x_data))
sample = np.array(x_data)
sample_tensor = torch.from_numpy(sample).float()
out = model(sample_tensor)
_, predicted = torch.max(out.data, -1)
if predicted.item() == 0:
pred_class = "Has no liver damage - ", predicted.item()
elif predicted.item() == 1:
pred_class = "Has liver damage - ", predicted.item()
return jsonify(pred_class)
以下是完整的回溯:
Traceback (most recent call last):
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/bin/flask", line 10, in <module>
sys.exit(main())
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/flask/cli.py", line 894, in main
cli.main(args=args, prog_name=name)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/flask/cli.py", line 557, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/flask/cli.py", line 767, in run_command
app = DispatchingApp(info.load_app, use_eager_loading=eager_loading)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/flask/cli.py", line 293, in __init__
self._load_unlocked()
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/flask/cli.py", line 317, in _load_unlocked
self._app = rv = self.loader()
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/flask/cli.py", line 372, in load_app
app = locate_app(self, import_name, name)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/flask/cli.py", line 235, in locate_app
__import__(module_name)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/app.py", line 34, in <module>
model = torch.load('model.pth')
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/torch/serialization.py", line 368, in load
return _load(f, map_location, pickle_module)
File "/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/lib/python3.6/site-packages/torch/serialization.py", line 542, in _load
result = unpickler.load()
AttributeError: Can't get attribute 'Net' on <module '__main__' from '/Users/judyraj/Judy/pytorch-deployment/flask_app/liver_disease_finder/bin/flask'>
回溯(最近一次呼叫最后一次):
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/bin/flask”,第10行,在
sys.exit(main())
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/flask/cli.py”,第894行,主文件
cli.main(args=args,prog_name=name)
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/flask/cli.py”,第557行,主文件
返回超级(烧瓶组,自身)。主(*args,**kwargs)
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/click/core.py”,第717行,主视图
rv=自调用(ctx)
文件“/Users/judyraj/Judy/pytorch deployment/flask_app/liver_disease_finder/lib/python3.6/site packages/click/core.py”,第1137行,在调用中
返回进程结果(sub_ctx.command.invoke(sub_ctx))
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/click/core.py”,第956行,调用中
返回ctx.invoke(self.callback,**ctx.params)
文件“/Users/judyraj/Judy/pytorch deployment/flask_app/liver_disease_finder/lib/python3.6/site packages/click/core.py”,第555行,在调用中
返回回调(*args,**kwargs)
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/click/decorators.py”,第64行,在新函数中
返回ctx.invoke(f,obj,*args,**kwargs)
文件“/Users/judyraj/Judy/pytorch deployment/flask_app/liver_disease_finder/lib/python3.6/site packages/click/core.py”,第555行,在调用中
返回回调(*args,**kwargs)
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/flask/cli.py”,第767行,在run_命令中
app=DispatchingApp(info.load\u app,使用\u eager\u loading=eager\u loading)
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/flask/cli.py”,第293行,在__
自身。_加载_解锁()
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/flask/cli.py”,第317行,在“加载”中
self.\u app=rv=self.loader()
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/flask/cli.py”,第372行,加载_app
应用程序=定位应用程序(自我、导入名称、名称)
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/flask/cli.py”,第235行,在locate_app中
__导入(模块名称)
文件“/Users/judyraj/Judy/pytorch部署/flask_app/app.py”,第34行,在
模型=火炬荷载('model.pth'))
文件“/Users/judyraj/Judy/pytorch部署/flask_app/liver_disease_finder/lib/python3.6/site packages/torch/serialization.py”,第368行,已加载
返回加载(f,映射位置,pickle模块)
文件“/Users/judyraj/Judy/pytorch deployment/flask_app/liver_disease_finder/lib/python3.6/site packages/torch/serialization.py”,第542行,装入
结果=unpickler.load()
AttributeError:无法在上获取属性“Net”
不能解决我的问题。我不想改变我坚持模型的方式。在虚拟环境之外,torch.save()对我来说很好。我不介意将类定义添加到脚本中。尽管如此,我仍在试图找出导致错误的原因。(这是部分答案)
我认为torch.save(model,'model.pt')
在命令提示符下不起作用,或者当一个模型从一个脚本保存为“\uuuuu main”
并从另一个脚本加载时也不起作用
原因是torch必须自动加载用于保存文件的模块,并且它从\uuuuu name\uuuu
获取模块名称
现在来看部分内容:目前还不清楚如何解决这个问题,尤其是当您的组合中有VirtualNV时
感谢您从这个方向开始对话。首先,我初始化了一个空模型,然后加载了保存的模型,出于某种原因,这解决了问题。这可能不是一个非常流行的答案,但是,我发现,
dill
包在使代码正常工作方面非常一致。对我来说,我甚至没有尝试加载一个模型,我尝试解包一个自定义对象,这有助于我的东西,但它找不到它,因为某种原因。我不知道为什么,但根据我的经验,莳萝似乎是更好的酸洗选择:
# - path to files
path = Path(path2dataset).expanduser()
path2file_data_prep = Path(path2file_data_prep).expanduser()
# - create dag dataprep obj
print(f'path to data set {path=}')
dag_prep = SplitDagDataPreparation(path)
# - save data prep splits object
print(f'saving to {path2file_data_prep=}')
torch.save({'data_prep': dag_prep}, path2file_data_prep, pickle_module=dill)
# - load the data prep splits object to test it loads correctly
db = torch.load(path2file_data_prep, pickle_module=dill)
db['data_prep']
print(db)
return path2file_data_prep
可能的重复与此无关。torch.save()在virtualenv之外对我来说很好。我只是想弄明白如何修正这个错误。我不想更改模型持久性的方式。您是如何保存模型的?你保存了整个模型还是只保存了它的
state_dict
?如果我只使用state_dict,它会起作用。我试图理解为什么pickle会在添加类定义的情况下抛出属性错误。你是如何运行你的应用程序的?您可以在代码中添加一行打印(\uuuuu name\uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu)
吗?我猜在保存pickle时,脚本的\uuuu name\uuuuuuu
与\uuuu main\uuuuuuu
相同,但在使用flask运行脚本时,情况有所不同,导致属性查找错误。