如何使用python proxybroker方法从本地文件夹获取和旋转代理

如何使用python proxybroker方法从本地文件夹获取和旋转代理,python,proxy,proxies,proxybroker,Python,Proxy,Proxies,Proxybroker,我尝试使用一个脚本从在线源抓取和旋转公共代理(来自defirent网站的免费代理列表),我使用proxybroker工具(python)直到现在一切都很好,但我正在寻找方法如何从我的语言环境机器中的文本或csv文件中绘制我的私有代理列表,并每x次旋转一次。 服侍 运行本地代理服务器,将传入请求分发到已找到的具有高匿名性的HTTP(S)代理池: $ proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl Hig

我尝试使用一个脚本从在线源抓取和旋转公共代理(来自defirent网站的免费代理列表),我使用proxybroker工具(python)直到现在一切都很好,但我正在寻找方法如何从我的语言环境机器中的文本或csv文件中绘制我的私有代理列表,并每x次旋转一次。 服侍

运行本地代理服务器,将传入请求分发到已找到的具有高匿名性的HTTP(S)代理池:

$ proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High
对外部代理的传入请求。”“”

import asyncio
import aiohttp

from proxybroker import Broker


async def get_pages(urls, proxy_url):
    tasks = [fetch(url, proxy_url) for url in urls]
    for task in asyncio.as_completed(tasks):
        url, content = await task
        print('Done! url: %s; content: %.100s' % (url, content))


async def fetch(url, proxy_url):
    resp = None
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url, proxy=proxy_url) as response:
                resp = await response.read()
    except (aiohttp.errors.ClientOSError, aiohttp.errors.ClientResponseError,
            aiohttp.errors.ServerDisconnectedError) as e:
        print('Error. url: %s; error: %r' % (url, e))
    finally:
        return (url, resp)


def main():
    host, port = '127.0.0.1', 8888  # by default

    loop = asyncio.get_event_loop()

    types = [('HTTP', 'High'), 'HTTPS', 'CONNECT:80']
    codes = [200, 301, 302]

    broker = Broker(max_tries=1, loop=loop)

    # Broker.serve() also supports all arguments that are accepted
    # Broker.find() method: data, countries, post, strict, dnsbl.
    broker.serve(host=host, port=port, types=types, limit=10, max_tries=3,
                 prefer_connect=True, min_req_proxy=5, max_error_rate=0.5,
                 max_resp_time=8, http_allowed_codes=codes, backlog=100)

    urls = ['http://httpbin.org/get', 'https://httpbin.org/get',
            'http://httpbin.org/redirect/1', 'http://httpbin.org/status/404']

    proxy_url = 'http://%s:%d' % (host, port)
    loop.run_until_complete(get_pages(urls, proxy_url))

    broker.stop()


if __name__ == '__main__':
    main()