Python 熊猫通过读取csv方法读取大文件时。IOError:[Errno 13]权限被拒绝
Env:Python 熊猫通过读取csv方法读取大文件时。IOError:[Errno 13]权限被拒绝,python,pandas,Python,Pandas,Env: Package Version ----------------------------- --------- backports.functools-lru-cache 1.5 beautifulsoup4 4.6.0 certifi 2018.4.16 chardet 3.0.4 cycler
Package Version
----------------------------- ---------
backports.functools-lru-cache 1.5
beautifulsoup4 4.6.0
certifi 2018.4.16
chardet 3.0.4
cycler 0.10.0
Django 1.11.14
et-xmlfile 1.0.1
future 0.16.0
googlemaps 2.5.1
idna 2.6
jdcal 1.4
jenkinsapi 0.3.6
Jinja2 2.10
kiwisolver 1.0.1
lxml 4.2.1
MarkupSafe 1.0
matplotlib 2.2.2
numpy 1.14.3
openpyxl 2.5.3
pandas 0.23.0
pip 10.0.1
psycopg2 2.7.5
pymongo 3.7.0
pyparsing 2.2.0
python-dateutil 2.7.3
pytz 2018.4
PyYAML 3.12
requests 2.18.4
scipy 1.1.0
seaborn 0.8.1
selenium 3.12.0
setuptools 18.2
six 1.11.0
urllib3 1.22
web.py 0.40.dev1
wheel 0.31.1
xmldiff 1.1.1
操作系统:Windows 10
Python 2.7
说明:
Package Version
----------------------------- ---------
backports.functools-lru-cache 1.5
beautifulsoup4 4.6.0
certifi 2018.4.16
chardet 3.0.4
cycler 0.10.0
Django 1.11.14
et-xmlfile 1.0.1
future 0.16.0
googlemaps 2.5.1
idna 2.6
jdcal 1.4
jenkinsapi 0.3.6
Jinja2 2.10
kiwisolver 1.0.1
lxml 4.2.1
MarkupSafe 1.0
matplotlib 2.2.2
numpy 1.14.3
openpyxl 2.5.3
pandas 0.23.0
pip 10.0.1
psycopg2 2.7.5
pymongo 3.7.0
pyparsing 2.2.0
python-dateutil 2.7.3
pytz 2018.4
PyYAML 3.12
requests 2.18.4
scipy 1.1.0
seaborn 0.8.1
selenium 3.12.0
setuptools 18.2
six 1.11.0
urllib3 1.22
web.py 0.40.dev1
wheel 0.31.1
xmldiff 1.1.1
当我通过方法read\u csv
读取一个大型csv文件test.csv(1.15G)时,python抛出一些异常
代码如下:
file =os.path.join(DATA_PATH, "test.csv")
test_chunks = pd.read_csv(file,iterator=True, engine="python",error_bad_lines=False, sep=',')
test_chunk = test_chunks.get_chunk(5)
解释器抛出一些异常:
Traceback (most recent call last):
File "D:/work_code/QA_tools/autogencases/utils/csvReader.py", line 149, in <module>
error_bad_lines=False, sep=',')
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 678, in parser_f
return _read(filepath_or_buffer, kwds)
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 440, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 787, in __init__
self._make_engine(self.engine)
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 1024, in _make_engine
self._engine = klass(self.f, **self.options)
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 2089, in __init__
self.columns, self.num_original_columns = self._infer_columns()
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 2359, in _infer_columns
line = self._buffered_line()
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 2530, in _buffered_line
return self._next_line()
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 2635, in _next_line
orig_line = self._next_iter_line(row_num=self.pos + 1)
File "C:\Python27\lib\site-packages\pandas\io\parsers.py", line 2695, in _next_iter_line
return next(self.data)
IOError: [Errno 13] Permission denied
回溯(最近一次呼叫最后一次):
文件“D:/work\u code/QA\u tools/autogencases/utils/csvReader.py”,第149行,在
错误_bad_line=False,sep=',')
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第678行,在解析器中
返回读取(文件路径或缓冲区,kwds)
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第440行,已读
parser=TextFileReader(文件路径或缓冲区,**kwds)
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第787行,在\uuu init中__
自制发动机(自制发动机)
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第1024行,在\u make\u引擎中
self.\u引擎=klass(self.f,**self.options)
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第2089行,在\uuu init中__
self.columns,self.num\u original\u columns=self.\u expert\u columns()
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第2359行,在\u expert\u列中
行=自。_缓冲线()
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第2530行,在缓冲行中
返回自我。_下一行()
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第2635行,下一行
原始行=自我。下一行(行数=自我位置+1)
文件“C:\Python27\lib\site packages\pandas\io\parsers.py”,第2695行,下一行
返回下一个(self.data)
IOError:[Errno 13]权限被拒绝
但是我检查test.csv的权限是否正常,以及它的父路径的权限是否正常。同时,同一文件夹中的另一个csv文件获得了相同的权限,可以正确读取,唯一的区别是它只有135M。有人可以帮助我吗?您尝试过将该文件移到其他地方吗?你确定运行Python的用户可以使用权限吗?我的眼睛呆呆地看着没有格式的文本墙…@AshishAcharya谢谢你的回答。这给了我一些想法。我发现它是用Excel后台打开的。当我关闭Excel时,问题已经解决了。啊,是的,好的ol'Excel锁:)