Python 对等pymongo重置连接
我必须从mongodb获取一些文档,并将其设置为memcache。这是密码Python 对等pymongo重置连接,python,mongodb,azure,tcp,pymongo,Python,Mongodb,Azure,Tcp,Pymongo,我必须从mongodb获取一些文档,并将其设置为memcache。这是密码 import memcache from pymongo import MongoClient db = mongo_client.job_db.JobParsedData jobs = db.find().sort("JobId", 1) def set_to_memcache_raw(jobs): print("Setting raw message to memcache") count = 0
import memcache
from pymongo import MongoClient
db = mongo_client.job_db.JobParsedData
jobs = db.find().sort("JobId", 1)
def set_to_memcache_raw(jobs):
print("Setting raw message to memcache")
count = 0
for item in jobs:
job_id = item.get('JobId')
job_details = item.get('JobDetails')
if job_id.strip():
count += 1
memcache_obj.set(job_id, job_details, time=72000)
if count % 1000 == 0:
print("Inserted {} keys in memcache".format(count))
if count >= 1000000:
break
但是,经过奇数次迭代后,代码会抛出此错误-
Traceback (most recent call last):
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/pool.py", line 450, in receive_message
self.sock, operation, request_id, self.max_message_size)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/network.py", line 137, in receive_message
header = _receive_data_on_socket(sock, 16)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/network.py", line 164, in _receive_data_on_socket
chunk = sock.recv(length)
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "memcache-poc.py", line 56, in <module>
elapsed = time.time() - t0
File "memcache-poc.py", line 52, in main
jobs = db.find(query)
File "memcache-poc.py", line 17, in set_to_memcache_raw
print("Setting raw message to memcache")
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/cursor.py", line 1114, in next
if len(self.__data) or self._refresh():
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/cursor.py", line 1056, in _refresh
self.__max_await_time_ms))
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/cursor.py", line 873, in __send_message
**kwargs)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/mongo_client.py", line 905, in _send_message_with_response
exhaust)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/mongo_client.py", line 916, in _reset_on_error
return func(*args, **kwargs)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/server.py", line 136, in send_message_with_response
response_data = sock_info.receive_message(1, request_id)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/pool.py", line 452, in receive_message
self._raise_connection_failure(error)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/pool.py", line 550, in _raise_connection_failure
_raise_connection_failure(self.address, error)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/pool.py", line 211, in _raise_connection_failure
raise AutoReconnect(msg)
pymongo.errors.AutoReconnect: xxx.xxx.xxx.xxx:27017: [Errno 104] Connection reset by peer
在这种情况下,在for循环上使用try-cacth块是否有帮助根据
pymongo
api文档中关于问题的以下信息,需要手动处理异常
异常pymongo.errors.AutoReconnect(消息='',错误=无)
当与数据库的连接丢失并且将尝试自动重新连接时引发
为了自动重新连接,您必须处理此异常,认识到导致此异常的操作不一定成功。未来的操作将尝试打开与数据库的新连接(并将继续引发此异常,直到第一次成功连接)
我搜索了该问题的一些现有解决方案,请参见下文
AutoReconnect
异常AutoReconnect
异常希望能有帮助。自己尝试一下也比问问题好。你能写下,你得到异常的确切位置吗?如果您要在每个循环中插入多个项,为什么不创建一个要插入的项列表,并尝试立即插入它们(这与解决上述错误无关,只是一个建议)代码在“for item in jobs:”行抛出异常。此外,memcache不支持一次设置多个键,因此我必须在每个循环中在memcache中写入单个项。@AnuragSharma so线程的答案可能对您有所帮助。
sysctl net.ipv4.tcp_keepalive_time
7200