Python 3.x fastai:使用预拆分数据集评估表格预测模型
给定一个用于训练和测试的预分割数据集,我想知道如何相应地应用fastai中的预测来访问MAE和RMSE值 以下示例来自fastai,并使用sklearn中的train_test_分割稍微修改Python 3.x fastai:使用预拆分数据集评估表格预测模型,python-3.x,pandas,evaluation,fast-ai,Python 3.x,Pandas,Evaluation,Fast Ai,给定一个用于训练和测试的预分割数据集,我想知道如何相应地应用fastai中的预测来访问MAE和RMSE值 以下示例来自fastai,并使用sklearn中的train_test_分割稍微修改 import numpy as np from sklearn.model_selection import train_test_split from fastai.tabular.all import * import pandas as pd path = untar_data(URLs.ADULT
import numpy as np
from sklearn.model_selection import train_test_split
from fastai.tabular.all import *
import pandas as pd
path = untar_data(URLs.ADULT_SAMPLE)
df = pd.read_csv(path/'adult.csv')
train, test = train_test_split(df, test_size=0.20, random_state=42)
cat_names = ['workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race']
cont_names = ['age', 'fnlwgt', 'education-num']
procs = [Categorify, FillMissing, Normalize]
dls = TabularDataLoaders.from_df(train, path, procs=procs, cat_names=cat_names, cont_names=cont_names,
y_names="salary")
learn = tabular_learner(dls)
learn.fit_one_cycle(5)
epoch train_loss valid_loss time
0 0.378432 0.356029 00:05
1 0.369692 0.358837 00:05
2 0.355757 0.348524 00:05
3 0.342714 0.348011 00:05
4 0.334072 0.346690 00:05
learn.unfreeze()
learn.fit_one_cycle(10, max_lr=slice(10e-4, 10e-3))
epoch train_loss valid_loss time
0 0.343953 0.350457 00:05
1 0.349379 0.353308 00:04
2 0.360508 0.352564 00:04
3 0.338458 0.351742 00:05
4 0.334585 0.352128 00:05
5 0.342312 0.351003 00:04
6 0.329152 0.350455 00:05
7 0.334460 0.351833 00:05
8 0.328608 0.351415 00:05
9 0.333205 0.352079 00:04
现在,我如何将学习模型应用到测试集来计算度量?下面这样的情况对我不起作用:
learn.predict(test)
这里我得到了以下错误:AttributeError:'DataFrame'对象没有属性'to_frame'
提前谢谢你的帮助 我最后为每个预测都编写了一个简单的for循环 当然,这远不是有效的,但解决了我的问题。如果您对克服慢for循环有任何改进建议,请随时在下面发表评论
predicted = []
real = []
for elem in range(0,len(test),1):
row, clas, probs = learn.predict(test.iloc[elem])
predicted.append(row["salary"].iloc[-1])
real.append(test["salary"].iloc[elem])
我还想分享我在fastai论坛上得到的关于这个问题的答案,因为它很有帮助:“您应该使用文档中显示的test_dl方法,然后通过后期处理将概率转化为您的类:”