Python 为什么在线程中使用此变量后访问它会失败?
我正在尝试使用谷歌的Python API进行云日志记录(以下是他们的和他们的) 我还尝试使用Python线程(特别是用于加速列表处理的线程)。问题是,在线程池中使用Python 为什么在线程中使用此变量后访问它会失败?,python,multithreading,google-cloud-platform,Python,Multithreading,Google Cloud Platform,我正在尝试使用谷歌的Python API进行云日志记录(以下是他们的和他们的) 我还尝试使用Python线程(特别是用于加速列表处理的线程)。问题是,在线程池中使用记录器之后,如果我在main函数中再次使用它,它将失败(或无限期挂起) 示例代码 下面是一些对我来说失败的示例代码: 从google.cloud导入日志 来自多处理导入池 从线程导入锁 通用记录器=‘某物’ log\u client=logging.client() logger=log\u client.logger(通用记录器)
记录器之后,如果我在main
函数中再次使用它,它将失败(或无限期挂起)
示例代码
下面是一些对我来说失败的示例代码:
从google.cloud导入日志
来自多处理导入池
从线程导入锁
通用记录器=‘某物’
log\u client=logging.client()
logger=log\u client.logger(通用记录器)
记录器_lock=lock()
def main()->无:
全局记录器
logger.log_结构({
“msg”:“这很有效”,
})
项目=[1,2,3,4,5]
将池(2)作为p:
p、 映射(测试线程、项目)
logger.log_结构({
“msg”:“此操作失败”,
})
def测试_线程(num)->无:
全局记录器
#做一些紧张的工作。。。
#然后记录
记录器_lock.acquire()
logger.log_struct({“number”:num})#此操作成功
记录器锁定释放()
如果uuuu name uuuuuu='\uuuuuuu main\uuuuuuu':
main()
对于上面的代码,一旦它尝试记录失败,它就不会记录,而是无限期地挂起。然后我必须^C
,产生以下输出:
E0503 18:54:34.153646870 786847 ssl_transport_security.cc:510] Corruption detected.
E0503 18:54:34.154107944 786847 ssl_transport_security.cc:486] error:1e000065:Cipher functions:OPENSSL_internal:BAD_DECRYPT
E0503 18:54:34.154475815 786847 ssl_transport_security.cc:486] error:1000008b:SSL routines:OPENSSL_internal:DECRYPTION_FAILED_OR_BAD_RECORD_MAC
E0503 18:54:34.154670516 786847 secure_endpoint.cc:208] Decryption error: TSI_DATA_CORRUPTED
E0503 18:54:34.154883721 786847 ssl_transport_security.cc:537] SSL_write failed with error SSL_ERROR_SSL.
^C
Process ForkPoolWorker-2:
Process ForkPoolWorker-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
File "so.py", line 38, in test_thread
logger_lock.acquire()
KeyboardInterrupt
Traceback (most recent call last):
File "so.py", line 44, in <module>
main()
File "so.py", line 25, in main
p.map(test_thread, items)
File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib/python3.8/multiprocessing/pool.py", line 765, in get
self.wait(timeout)
File "/usr/lib/python3.8/multiprocessing/pool.py", line 762, in wait
self._event.wait(timeout)
File "/usr/lib/python3.8/threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
File "/usr/lib/python3.8/threading.py", line 302, in wait
waiter.acquire()
KeyboardInterrupt
E0503 18:54:34.153646870 786847 ssl_传输_安全。cc:510]检测到损坏。
E0503 18:54:34.154107944 786847 ssl_传输_安全性。cc:486]错误:1e000065:密码函数:OPENSSL_内部:错误\u解密
E0503 18:54:34.154475815 786847 ssl\u传输\u安全。cc:486]错误:100008b:ssl例程:OPENSSL\u内部:解密\u失败\u或\u坏记录\u MAC
E0503 18:54:34.154670516 786847安全_端点。cc:208]解密错误:TSI_数据_损坏
E0503 18:54:34.154883721 786847 ssl_传输_安全。cc:537]ssl_写入失败,错误为ssl_错误_ssl。
^C
工艺工人-2:
进程1-1:
回溯(最近一次呼叫最后一次):
文件“/usr/lib/python3.8/multiprocessing/process.py”,第315行,在引导程序中
self.run()
文件“/usr/lib/python3.8/multiprocessing/process.py”,第108行,正在运行
自我目标(*自我参数,**自我参数)
worker中的文件“/usr/lib/python3.8/multiprocessing/pool.py”,第125行
结果=(True,func(*args,**kwds))
文件“/usr/lib/python3.8/multiprocessing/pool.py”,第48行,在mapstar中
返回列表(映射(*args))
测试线程中第38行的文件“so.py”
记录器_lock.acquire()
键盘中断
回溯(最近一次呼叫最后一次):
文件“so.py”,第44行,在
main()
文件“so.py”,第25行,主
p、 映射(测试线程、项目)
文件“/usr/lib/python3.8/multiprocessing/pool.py”,第364行,在地图中
返回self.\u map\u async(func、iterable、mapstar、chunksize).get()
get中第765行的文件“/usr/lib/python3.8/multiprocessing/pool.py”
self.wait(超时)
文件“/usr/lib/python3.8/multiprocessing/pool.py”,第762行,正在等待
self.\u事件等待(超时)
文件“/usr/lib/python3.8/threading.py”,第558行,正在等待
信号=自身。第二次等待(超时)
文件“/usr/lib/python3.8/threading.py”,第302行,正在等待
服务员
键盘中断
我真正的错误
我遇到问题的实际代码太长,无法放在这里,但是,将其打印错误放在这里可能会有所帮助(因为它看起来确实略有不同):
E0503 18:36:42.763982000 769264 ssl_传输_安全。cc:510]检测到损坏。
E0503 18:36:42.764023851 769264 ssl_传输_安全性。cc:486]错误:1e000065:密码函数:OPENSSL_内部:错误\u解密
E0503 18:36:42.764032243 769264 ssl_传输_安全。cc:486]错误:100008b:ssl例程:OPENSSL_内部:解密_失败_或_坏记录_MAC
E0503 18:36:42.764038246 769264安全_端点。cc:208]解密错误:TSI_数据_损坏
E0503 18:36:42.764096712 769264 ssl_传输_安全。cc:537]ssl_写入失败,错误为ssl_错误_ssl。
回溯(最近一次呼叫最后一次):
文件“/usr/local/lib/python3.8/dist packages/google/api\u core/grpc\u helpers.py”,第73行,错误\u重新映射\u可调用
返回可调用函数(*args,**kwargs)
文件“/usr/local/lib/python3.8/dist packages/grpc/_channel.py”,第946行,在调用中__
返回\u结束\u一元\u响应\u阻塞(状态、调用、错误、无)
文件“/usr/local/lib/python3.8/dist packages/grpc/_channel.py”,第849行,一元响应中
raise\u InactiveRpcError(状态)
grpc.\u信道。\u不活动覆盖PCERROR:
回溯(最近一次呼叫最后一次):
文件“/opt/scraper”,第331行,在
main()
文件“/opt/scraper”,主菜单第162行
logger.log_结构({
文件“/usr/local/lib/python3.8/dist packages/google/cloud/logging\u v2/logger.py”,第181行,在log\u结构中
self.\u do\u日志(客户端、结构条目、信息,**kw)
文件“/usr/local/lib/python3.8/dist packages/google/cloud/logging\u v2/logger.py”,第134行,在日志中
client.logging\u api.write\u条目([api\u repr])
文件“/usr/local/lib/python3.8/dist packages/google/cloud/logging\u v2/\u gapic.py”,第149行,在write\u条目中
self.\u gapic\u api.write\u log\u条目(请求=请求)
文件“/usr/local/lib/python3.8/dist packages/google/cloud/logging_v2/services/logging_service_v2/client.py”,第591行,在write_log_条目中
响应=rpc(请求,重试=重试,超时=超时,元数据=元数据,)
文件“/usr/local/lib/python3.8/dist packages/google/api_core/gapic_v1/method.py”,第145行,在调用中__
返回包装函数(*args,**kwargs)
文件“/usr/local/lib/python3.8/dist packages/google/api\u core/retry.py”,第281行,在retry\u wrapped\u func中
返回重试\u目标(
文件“/usr/local/lib/python3.8/dist packages/google/api\u core/retry.py”,第184行,在retry\u目标中
返回目标()
文件“/usr/local/lib/python3.8/dist packages/google/api\u core/grpc\u helpers.py”,第75行,错误\u重新映射\u可调用
六、从(例外情况。从grpc错误(exc),exc)
文件“”,第3行,从
google.api_core.exceptions.Unknown:未删除任何流
E0503 18:36:42.763982000 769264 ssl_transport_security.cc:510] Corruption detected.
E0503 18:36:42.764023851 769264 ssl_transport_security.cc:486] error:1e000065:Cipher functions:OPENSSL_internal:BAD_DECRYPT
E0503 18:36:42.764032243 769264 ssl_transport_security.cc:486] error:1000008b:SSL routines:OPENSSL_internal:DECRYPTION_FAILED_OR_BAD_RECORD_MAC
E0503 18:36:42.764038246 769264 secure_endpoint.cc:208] Decryption error: TSI_DATA_CORRUPTED
E0503 18:36:42.764096712 769264 ssl_transport_security.cc:537] SSL_write failed with error SSL_ERROR_SSL.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/google/api_core/grpc_helpers.py", line 73, in error_remapped_callable
return callable_(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/usr/local/lib/python3.8/dist-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "Stream removed"
debug_error_string = "{"created":"@1620067002.764071153","description":"Error received from peer ipv4:172.217.11.202:443","file":"src/core/lib/surface/call.cc","file_line":1067,"grpc_message":"Stream removed","grpc_status":2}"
>
Traceback (most recent call last):
File "/opt/scraper", line 331, in <module>
main()
File "/opt/scraper", line 162, in main
logger.log_struct({
File "/usr/local/lib/python3.8/dist-packages/google/cloud/logging_v2/logger.py", line 181, in log_struct
self._do_log(client, StructEntry, info, **kw)
File "/usr/local/lib/python3.8/dist-packages/google/cloud/logging_v2/logger.py", line 134, in _do_log
client.logging_api.write_entries([api_repr])
File "/usr/local/lib/python3.8/dist-packages/google/cloud/logging_v2/_gapic.py", line 149, in write_entries
self._gapic_api.write_log_entries(request=request)
File "/usr/local/lib/python3.8/dist-packages/google/cloud/logging_v2/services/logging_service_v2/client.py", line 591, in write_log_entries
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "/usr/local/lib/python3.8/dist-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/google/api_core/retry.py", line 281, in retry_wrapped_func
return retry_target(
File "/usr/local/lib/python3.8/dist-packages/google/api_core/retry.py", line 184, in retry_target
return target()
File "/usr/local/lib/python3.8/dist-packages/google/api_core/grpc_helpers.py", line 75, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.Unknown: None Stream removed