Python 3.x 在Python 3.6.3中尝试删除数据帧中的变量时出错

Python 3.x 在Python 3.6.3中尝试删除数据帧中的变量时出错,python-3.x,pandas,compiler-errors,Python 3.x,Pandas,Compiler Errors,我有一个数据库,包含322098个观察值和3868个变量。为了测试脚本,我生成了一个子样本,其中包含50个观察值和3868个变量。当我在子示例中运行脚本时,它工作得非常好。但是,当我尝试使用完整的数据库(322098备注)运行时,从dataframe中删除贸易变量会产生错误 以下是脚本: ## Load External DataSet mydata = pd.read_csv ('C:\\Users\\Inspiron\\Desktop\\policies.csv', sep = ',',

我有一个数据库,包含322098个观察值和3868个变量。为了测试脚本,我生成了一个子样本,其中包含50个观察值和3868个变量。当我在子示例中运行脚本时,它工作得非常好。但是,当我尝试使用完整的数据库(322098备注)运行时,从dataframe中删除贸易变量会产生错误

以下是脚本:

## Load External DataSet

mydata = pd.read_csv ('C:\\Users\\Inspiron\\Desktop\\policies.csv', sep = ',', na_values = '.')

## Normalized Data

mydata ['normalized'] = (mydata ['trade'] - mydata ['trade'].min ())/(mydata ['trade'].max () - mydata ['trade'].min ())

## Descriptive Statistics for a Single Variable

mydata ['normalized'].describe ()

## Drop Columns

mydata = mydata.drop (['trade'], axis = 1)
错误如下:

Traceback (most recent call last):
File "C:\Users\Inspiron\OneDrive\academic\articles\2018\non-discriminatory\script-dofile\mfn.py", line 31, in <module>
mydata = mydata.drop (['trade'], axis = 1)
File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 2530, in drop
obj = obj._drop_axis(labels, axis, level=level, errors=errors)
File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 2563, in _drop_axis
dropped = self.reindex(**{axis_name: new_axis})
File "C:\Python36\lib\site-packages\pandas\util\_decorators.py", line 127, in wrapper
return func(*args, **kwargs)
File "C:\Python36\lib\site-packages\pandas\core\frame.py", line 2935, in reindex
return super(DataFrame, self).reindex(**kwargs)
File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 3004, in reindex
self._consolidate_inplace()
File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 3677, in _consolidate_inplace
self._protect_consolidate(f)
File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 3666, in _protect_consolidate
result = f()
File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 3675, in f
self._data = self._data.consolidate()
File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 3826, in consolidate
bm._consolidate_inplace()
File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 3831, in _consolidate_inplace
self.blocks = tuple(_consolidate(self.blocks))
File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 4853, in _consolidate
_can_consolidate=_can_consolidate)
File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 4876, in _merge_blocks
new_values = new_values[argsort]
MemoryError
回溯(最近一次呼叫最后一次):
文件“C:\Users\Inspiron\OneDrive\academic\articles\2018\non-discriminative\script dofile\mfn.py”,第31行,在
mydata=mydata.drop(['trade'],axis=1)
文件“C:\Python36\lib\site packages\pandas\core\generic.py”,第2530行,在下拉列表中
obj=obj.\u下降\u轴(标签、轴、级别=级别、错误=错误)
文件“C:\Python36\lib\site packages\pandas\core\generic.py”,第2563行,在放置轴中
drop=self.reindex(**{axis\u name:new\u axis})
文件“C:\Python36\lib\site packages\pandas\util\\u decorators.py”,第127行,在包装器中
返回函数(*args,**kwargs)
reindex中的文件“C:\Python36\lib\site packages\pandas\core\frame.py”,第2935行
返回超级(数据帧,自).reindex(**kwargs)
reindex中的文件“C:\Python36\lib\site packages\pandas\core\generic.py”,第3004行
self.\u巩固\u就位()
文件“C:\Python36\lib\site packages\pandas\core\generic.py”,第3677行,在\u consolid\u inplace中
自我保护巩固(f)
文件“C:\Python36\lib\site packages\pandas\core\generic.py”,第3666行,在\u protect\u consolidate中
结果=f()
文件“C:\Python36\lib\site packages\pandas\core\generic.py”,第3675行,f
self.\u data=self.\u data.consolidate()
文件“C:\Python36\lib\site packages\pandas\core\internals.py”,第3826行,合并
bm._巩固_到位()
文件“C:\Python36\lib\site packages\pandas\core\internals.py”,第3831行,在\u consolid\u inplace中
self.blocks=元组(_合并(self.blocks))
文件“C:\Python36\lib\site packages\pandas\core\internals.py”,第4853行,位于\u
_可以合并=\u可以合并)
文件“C:\Python36\lib\site packages\pandas\core\internals.py”,第4876行,在\u merge\u块中
新值=新值[argsort]
记忆者

有人能帮我吗?

我会尝试使用
sklearn.preprocessing.MinMaxScaler来代替:

from sklearn.preprocessing import MinMaxScaler

mms = MinMaxScaler()

# use `.to_frame()` to prevent `ValueError: Expected 2D array, got 1D array instead:`    
mydata['normalized'] = mms.fit_transform(mydata.pop('trade').to_frame())

直接规范化交易变量,而不是从交易变量创建规范化变量

按照以下命令操作:

## Normalized Data

mydata ['trade'] = (mydata ['trade'] - mydata ['trade'].min ())/(mydata ['trade'].max () - mydata ['trade'].min ())