在python中每秒发送1000多个请求
我试图同时向服务器发送请求,然后使用以下代码记录平均延迟:在python中每秒发送1000多个请求,python,python-multithreading,grequests,Python,Python Multithreading,Grequests,我试图同时向服务器发送请求,然后使用以下代码记录平均延迟: import Queue import time import threading import urllib2 data = "{"image_1":"abc/xyz.jpg"}" headers = {.....} def get_url(q, url): num = 1 sum = 0 while num <= 200: start = time.time() req
import Queue
import time
import threading
import urllib2
data = "{"image_1":"abc/xyz.jpg"}"
headers = {.....}
def get_url(q, url):
num = 1
sum = 0
while num <= 200:
start = time.time()
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
end = time.time()
print end - start
num = num + 1
q.put(response.read())
sum = sum + (end - start)
print sum
theurls = ["http://example.com/example"]
q = Queue.Queue()
for u in theurls:
t = threading.Thread(target = get_url, args = (q, u))
t.daemon = True
t.start()
while True:
s = q.get()
print s
导入队列
导入时间
导入线程
导入urllib2
data=“{”image_1:“abc/xyz.jpg”}”
标题={…}
def get_url(q,url):
num=1
总和=0
虽然num文档不是很好,但是源代码很好。阅读来源!:
:
:
返回一个新的分部对象,该对象在调用时的行为类似于使用位置参数args和关键字参数关键字调用的func。如果为调用提供了更多参数,则这些参数将附加到参数。如果提供了其他关键字参数,它们将扩展和覆盖关键字
基本上是调用grequests.get
调用AsyncRequest(“get”,其他参数转到这里**)
。它的文件说:
会话已在前面定义:
from requests import Session
这里有什么见解吗伙计们……也许你可以看看类似于python内置asyncio之上的异步框架?这里或网上有一些关于使用aiohttp发出多个(几乎同时)请求的好帖子(如或),请查看存储库中的一些示例。4年后,它可能与您无关,但此代码不会每秒发送200个连接。我打开了200个线程来完成这项工作
__all__ = (
'map', 'imap',
'get', 'options', 'head', 'post', 'put', 'patch', 'delete', 'request'
)
# Shortcuts for creating AsyncRequest with appropriate HTTP method
get = partial(AsyncRequest, 'GET')
options = partial(AsyncRequest, 'OPTIONS')
head = partial(AsyncRequest, 'HEAD')
post = partial(AsyncRequest, 'POST')
put = partial(AsyncRequest, 'PUT')
patch = partial(AsyncRequest, 'PATCH')
delete = partial(AsyncRequest, 'DELETE')
from functools import partial
""" Asynchronous request.
Accept same parameters as ``Session.request`` and some additional:
:param session: Session which will do request
:param callback: Callback called on response.
Same as passing ``hooks={'response': callback}``
"""
from requests import Session