Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/282.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Scrapy splash 飞溅冻结与“的;超时客户端:IPv4Address“;_Scrapy Splash - Fatal编程技术网

Scrapy splash 飞溅冻结与“的;超时客户端:IPv4Address“;

Scrapy splash 飞溅冻结与“的;超时客户端:IPv4Address“;,scrapy-splash,Scrapy Splash,我正在运行scrapy splash从一个网站上抓取数据 定期(随机)冻结下一个日志: [36msplash-service_1 |[0m 2020-07-16 08:49:35.119333 [-] "172.31.0.4" - - [16/Jul/2020:08:49:34 +0000] "POST /execute HTTP/1.1" 200 266018 "-" "Mozilla/5.0 (Windows

我正在运行
scrapy splash
从一个网站上抓取数据

定期(随机)冻结下一个日志:

[36msplash-service_1        |[0m 2020-07-16 08:49:35.119333 [-] "172.31.0.4" - - [16/Jul/2020:08:49:34 +0000] "POST /execute HTTP/1.1" 200 266018 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36"
[36msplash-service_1        |[0m 2020-07-16 08:50:10.012973 [-] Timing out client: IPv4Address(type='TCP', host='172.31.0.4', port=51970)
[36msplash-service_1        |[0m 2020-07-16 08:50:10.858080 [-] Timing out client: IPv4Address(type='TCP', host='172.31.0.4', port=51978)
[36msplash-service_1        |[0m 2020-07-16 08:50:16.873014 [-] Timing out client: IPv4Address(type='TCP', host='172.31.0.4', port=51974)
[36msplash-service_1        |[0m 2020-07-16 08:50:17.547947 [-] Timing out client: IPv4Address(type='TCP', host='172.31.0.4', port=51966)
[36msplash-service_1        |[0m 2020-07-16 08:50:18.037436 [-] Timing out client: IPv4Address(type='TCP', host='172.31.0.4', port=51976)
[36msplash-service_1        |[0m 2020-07-16 08:50:29.064655 [-] Timing out client: IPv4Address(type='TCP', host='172.31.0.4', port=51932)
[36msplash-service_1        |[0m 2020-07-16 08:50:35.119997 [-] Timing out client: IPv4Address(type='TCP', host='172.31.0.4', port=51968)
我怎么才能知道原因呢?为什么会卡住

p.S我使用
args={“lua\u source”:self.lua\u script\u navigate,“timeout”:60000}

请参阅参数的
timeout

超时:浮动:可选

渲染的超时(秒)(默认为30)

默认情况下,超时允许的最大值为90秒。要覆盖它,请使用--max timeout命令行选项启动Splash。 例如,这里的Splash配置为允许最多5次超时 会议记录:

如果未使用
--max timeout
启动splash,则即使在
args
中设置了更高的超时,lua_脚本也会在30秒后中止。请参阅参数的
timeout

超时:浮动:可选

渲染的超时(秒)(默认为30)

默认情况下,超时允许的最大值为90秒。要覆盖它,请使用--max timeout命令行选项启动Splash。 例如,这里的Splash配置为允许最多5次超时 会议记录:

如果您没有使用
--max timeout
启动splash,则即使在
args
中设置了更高的超时,lua_脚本也会在30秒后中止

$ docker run -it -p 8050:8050 scrapinghub/splash --max-timeout 300