Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/281.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python:如何发送多个HTTP请求并接收响应?_Python_Json_Http_Python Requests_Grequests - Fatal编程技术网

Python:如何发送多个HTTP请求并接收响应?

Python:如何发送多个HTTP请求并接收响应?,python,json,http,python-requests,grequests,Python,Json,Http,Python Requests,Grequests,如何以最快的方式发送1000个请求? 我知道您可以使用grequests发送多个请求: urls = [ 'sample.url/1', 'sample.url/2', ... ] request = (grequests.get(u) for u in urls) print grequests.map(request) 但回报不是内容。 我需要的是获取json数据,例如: request = (grequests.get(u) for u in urls) cont

如何以最快的方式发送1000个请求? 我知道您可以使用
grequests
发送多个请求:

urls = [
    'sample.url/1',
    'sample.url/2',
    ...
]
request = (grequests.get(u) for u in urls)
print grequests.map(request)
但回报不是内容。 我需要的是获取json数据,例如:

request = (grequests.get(u) for u in urls)
content = grequests.json(request)
result = grequests.map(request)
json = [r.json() for r in result]

返回的项目不是内容,但它们确实包含内容。您可以按如下方式获取所有内容:

result = grequests.map(request)
content = '\n'.join(r.content for r in result) # raw content
text = '\n'.join(r.text for r in result)       # decoded content
您可以像这样解析json:

request = (grequests.get(u) for u in urls)
content = grequests.json(request)
result = grequests.map(request)
json = [r.json() for r in result]
示例程序:

import grequests
import pprint

urls = [
    'http://httpbin.org/user-agent',
    'http://httpbin.org/headers',
    'http://httpbin.org/ip',
]

requests = (grequests.get(u) for u in urls)
responses = grequests.map(requests)

json = [response.json() for response in responses]
pprint.pprint(json)

text = '\n'.join(response.text for response in responses)
print(text)