Python 在django上进行多处理时“;django.db.utils.OperationalError:磁盘I/O错误“;
我使用多处理功能爬行并将信息保存到django.db函数。 它一开始工作正常,但后来出现了错误。 在桌面上启动stockPriceUpdate()时,不会出现任何错误。但当我在我的笔记本电脑上启动时,它一开始工作正常,但后来,它的速度变慢,并产生如下错误 股票是我的django模型 麦可德Python 在django上进行多处理时“;django.db.utils.OperationalError:磁盘I/O错误“;,python,django,Python,Django,我使用多处理功能爬行并将信息保存到django.db函数。 它一开始工作正常,但后来出现了错误。 在桌面上启动stockPriceUpdate()时,不会出现任何错误。但当我在我的笔记本电脑上启动时,它一开始工作正常,但后来,它的速度变慢,并产生如下错误 股票是我的django模型 麦可德 def stockPriceCrawling(stockTitle): url = 'http://asp1.krx.co.kr/servlet/krx.asp.XMLSiseEng?cod
def stockPriceCrawling(stockTitle):
url = 'http://asp1.krx.co.kr/servlet/krx.asp.XMLSiseEng?code=' + str(stockTitle)
html = requests.get(url).content
soup = BeautifulSoup(html, 'html.parser')
stockinfo = soup.select('TBL_StockInfo')[0]
price = stockinfo['curjuka'].replace(',','')
highestPrice = stockinfo['highjuka'].replace(',','')
lowestPrice = stockinfo['lowjuka'].replace(',','')
tradeVolume = stockinfo['volume'].replace(',','')
tradeValue = stockinfo['money'].replace(',','')
print("info : ",' ', price," ",highestPrice," ",lowestPrice," ",tradeVolume," ",tradeValue)
db.connections.close_all()
stock= Stock.objects.get(code=stockTitle)
stock.price = price
stock.highestPrice = highestPrice
stock.lowestPrice = lowestPrice
stock.tradeVolume = tradeVolume
stock.tradeValue = tradeValue
stock.save()
def stockPriceUpdate(process=32):
# start = time.time()
stocks = Stock.objects.all()
stockTitles=[]
for stock in stocks:
stockTitles.append(str(stock.code))
pool = Pool(32)
r=pool.map(stockPriceCrawling, stockTitles)
pool.close()
pool.join()
错误
回溯(最近一次呼叫最后一次):
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/backends/utils.py”,第84行,在
返回self.cursor.execute(sql,params)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/backends/sqlite3/base.py”,第413行,在execute中
返回Database.Cursor.execute(self、query、params)
sqlite3.0错误:磁盘I/O错误
上述异常是以下异常的直接原因:
回溯(最近一次呼叫最后一次):
worker中的文件“/usr/local/lib/python3.7/multiprocessing/pool.py”,第121行
结果=(True,func(*args,**kwds))
mapstar中的文件“/usr/local/lib/python3.7/multiprocessing/pool.py”,第44行
返回列表(映射(*args))
文件“price_paser.py”,第84行,股票价格爬网
stock.save()
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/models/base.py”,第754行,保存
强制更新=强制更新,更新字段=更新字段)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/models/base.py”,第792行,在save_base中
强制更新,使用,更新字段,
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/models/base.py”,第873行,在保存表格中
强制更新)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/models/base.py”,第926行,更新
返回已筛选的。\u更新(值)>0
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/models/query.py”,第803行,在更新中
返回query.get_编译器(self.db)。执行_sql(游标)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/models/sql/compiler.py”,第1522行,在execute_sql中
cursor=super()。执行sql(结果类型)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/models/sql/compiler.py”,第1156行,在execute_sql中
cursor.execute(sql,params)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/backends/utils.py”,第98行,执行
return super().execute(sql,params)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/backends/utils.py”,第66行,在execute中
返回self.\u使用包装器执行(sql,params,many=False,executor=self.\u execute)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/backends/utils.py”,第75行,使用包装器执行
返回执行器(sql、参数、多个、上下文)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/backends/utils.py”,第84行,在
返回self.cursor.execute(sql,params)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/utils.py”,第90行,在退出时__
使用exc_值的_回溯(回溯)提高dj_exc_值
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/backends/utils.py”,第84行,在
返回self.cursor.execute(sql,params)
文件“/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site packages/django/db/backends/sqlite3/base.py”,第413行,在execute中
返回Database.Cursor.execute(self、query、params)
django.db.utils.OperationalError:磁盘I/O错误
"""
上述异常是以下异常的直接原因:
回溯(最近一次呼叫最后一次):
文件“price_paser.py”,第166行,在
股票价格更新()
stockPriceUpdate中第97行的文件“price_paser.py”
r=pool.map(股价爬行、股票标题)
文件“/usr/local/lib/python3.7/multiprocessing/pool.py”,第268行,在地图中
返回self.\u map\u async(func、iterable、mapstar、chunksize).get()
get中第657行的文件“/usr/local/lib/python3.7/multiprocessing/pool.py”
提升自我价值
django.db.utils.OperationalError:磁盘I/O错误
Traceback (most recent call last):
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: disk I/O error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/usr/local/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "price_paser.py", line 84, in stockPriceCrawling
stock.save()
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/models/base.py", line 754, in save
force_update=force_update, update_fields=update_fields)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/models/base.py", line 792, in save_base
force_update, using, update_fields,
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/models/base.py", line 873, in _save_table
forced_update)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/models/base.py", line 926, in _do_update
return filtered._update(values) > 0
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/models/query.py", line 803, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1522, in execute_sql
cursor = super().execute_sql(result_type)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
cursor.execute(sql, params)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/backends/utils.py", line 98, in execute
return super().execute(sql, params)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/mnt/c/Users/DAUN/Desktop/swpp/swpp/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: disk I/O error
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "price_paser.py", line 166, in <module>
stockPriceUpdate()
File "price_paser.py", line 97, in stockPriceUpdate
r=pool.map(stockPriceCrawling, stockTitles)
File "/usr/local/lib/python3.7/multiprocessing/pool.py", line 268, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/local/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
django.db.utils.OperationalError: disk I/O error