Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/325.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 线程池执行器的concurrent.futures.Future有时挂起_Python_Python 3.x_Threadpoolexecutor_Concurrent.futures - Fatal编程技术网

Python 线程池执行器的concurrent.futures.Future有时挂起

Python 线程池执行器的concurrent.futures.Future有时挂起,python,python-3.x,threadpoolexecutor,concurrent.futures,Python,Python 3.x,Threadpoolexecutor,Concurrent.futures,我面临一个问题,来自ThreadPoolExecutor的concurrent.futures.Future有时会挂起,我使用python 3.7,下面是我的代码: import pandas as pd import concurrent.futures import requests import time import sys from urllib3.util.retry import Retry import os from decimal import localcontext, D

我面临一个问题,来自ThreadPoolExecutor的concurrent.futures.Future有时会挂起,我使用python 3.7,下面是我的代码:

import pandas as pd
import concurrent.futures
import requests
import time
import sys
from urllib3.util.retry import Retry
import os
from decimal import localcontext, Decimal, ROUND_HALF_UP
from datetime import datetime,timezone ,timedelta
import urllib3
import base64

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

class Test:
    out = []
    CONNECTIONS = 200

    f1 = open('D:/NewProxeyUserPass.txt', 'r')
    Proxies= f1.readlines()
    f1.close()

    f2 = open('D:/UserAgents.txt', 'r')
    UserAgents= f2.readlines()
    f2.close()

    MAX_RETRIES = 3

    ExcptionCount= int
    ExcptionCount=0

    BASE_URI = 'https://h.com/session.php'

    def load_url(n):
        Proxy_Port_User_Pass=Test.Proxies[n].split(":")
        useragent=Test.UserAgents[n]
        Proxy_Port_User_Pass[0].strip()
        ProxyIP =Proxy_Port_User_Pass[0].strip()
        ProxyPort =Proxy_Port_User_Pass[1].strip()
        username = Proxy_Port_User_Pass[2].strip()
        password =Proxy_Port_User_Pass[3].strip()
        usrPass = username+":"+password
        b64Val = base64.b64encode(usrPass.encode()).decode()
        Timestamp =int((datetime.utcnow() - datetime(1970,1,1)).total_seconds())
        headers = {
            "user-agent": useragent.strip(),
            "Connection":"close"
            }

        proxies = {
           "http"  :"http://" + username + ":" + password + "@"  +  ProxyIP + ":" + ProxyPort,
           "https" :"https://" + username + ":" + password + "@"  +  ProxyIP + ":" +ProxyPort
           }
        JsonSession = "{'device':'id'}"
        retries = Retry(total=3,
                       backoff_factor=0.1)
        adapter = requests.adapters.HTTPAdapter(max_retries=retries)
        with requests.Session() as session:
           session.mount('https://', adapter)
           session.keep_alive = False
           session.trust_env=False
           ans = session.post(Test.BASE_URI,
                              data=JsonSession,
                              headers=headers,
                              params={"timestamp":Timestamp},
                              proxies=proxies,
                              verify=False)
           return ans.content

    def DoWork():
        with concurrent.futures.ThreadPoolExecutor(max_workers=Test.CONNECTIONS) as executor:
            future_to_url = (executor.submit(Test.load_url, n) for n in range(0,150))
            count=int
            count=1
            ExcptionCount = int
            ExcptionCount = 0
            time1 = time.time()
            for future in concurrent.futures.as_completed(future_to_url):
                try:
                    print( str(count)+" - "+str(future.result())+"\n")
                    count = count +1
                except Exception as exc:
                        ExcptionCount=ExcptionCount+1
                        print("Exception count : "+ str(ExcptionCount) + " - "+str(exc))

        time2 = time.time()
        print(f'Took {time2-time1:.2f} s'+"\n Total Exceptions Count : "+ str(ExcptionCount))

    pass

Test.DoWork()

我找到了这些 和 但我不知道如何在我的代码中使用它,
那么如何防止ThreadPoolExecutor的concurrent.futures.Future挂起,如果你能给我工作代码,谢谢你的帮助

如果我用很少的数字运行for循环,例如150将不会挂起,但如果我以15000(我需要以大量运行它)这样的大数字运行它,将挂起,有什么建议吗?那么,是否有任何方法可以为ThreadPoolExecutor中的每个工作线程设置一个超时来防止挂起?有什么帮助吗?如果我以较小的数字运行for循环,例如150,则不会挂起,但如果我以较大的数字运行它,例如15000(我需要以较大的数字运行),则会挂起,有什么建议吗?那么有没有办法为ThreadPoolExecutor中的每个工作线程设置一个超时来防止挂起?有什么帮助吗?