Python Scrapy Splash服务器未通过docker运行启动

Python Scrapy Splash服务器未通过docker运行启动,python,docker,scrapy,Python,Docker,Scrapy,按照Alexe的指示安装docker、scrapyjs和splash 然后,运行docker run-p 8050:8050 scrapinghub/splash,我得到以下输出: 2016-05-08 17:17:45+0000 [-] Log opened. 2016-05-08 17:17:45.978866 [-] Splash version: 2.1 2016-05-08 17:17:45.979553 [-] Qt 5.5.1, PyQt 5.5.1, WebKit 538.1,

按照Alexe的指示安装docker、scrapyjs和splash

然后,运行
docker run-p 8050:8050 scrapinghub/splash
,我得到以下输出:

2016-05-08 17:17:45+0000 [-] Log opened.
2016-05-08 17:17:45.978866 [-] Splash version: 2.1
2016-05-08 17:17:45.979553 [-] Qt 5.5.1, PyQt 5.5.1, WebKit 538.1, sip 4.17, Twisted 16.1.1, Lua 5.2
2016-05-08 17:17:45.980138 [-] Python 3.4.3 (default, Oct 14 2015, 20:28:29) [GCC 4.8.4]
2016-05-08 17:17:45.980401 [-] Open files limit: 1048576
2016-05-08 17:17:45.981020 [-] Can't bump open files limit
2016-05-08 17:17:46.086232 [-] Xvfb is started: ['Xvfb', ':1', '-screen', '0', '1024x768x24']
2016-05-08 17:17:46.161902 [-] proxy profiles support is enabled, proxy profiles path: /etc/splash/proxy-profiles
2016-05-08 17:17:46.260357 [-] verbosity=1
2016-05-08 17:17:46.260607 [-] slots=50
2016-05-08 17:17:46.261170 [-] argument_cache_max_entries=500
2016-05-08 17:17:46.262476 [-] Web UI: enabled, Lua: enabled (sandbox: enabled)
2016-05-08 17:17:46.264565 [-] Site starting on 8050
2016-05-08 17:17:46.265203 [-] Starting factory <twisted.web.server.Site object at 0x7f270ec81e10>
2016-05-08 17:17:45+0000[-]日志打开。
2016-05-08 17:17:45.978866[-]初始版本:2.1
2016-05-08 17:17:45.979553[-]Qt 5.5.1,PyQt 5.5.1,WebKit 538.1,sip 4.17,Twisted 16.1.1,Lua 5.2
2016-05-08 17:17:45.980138[-]Python 3.4.3(默认值,2015年10月14日,20:28:29)[GCC 4.8.4]
2016-05-08 17:17:45.980401[-]打开文件限制:1048576
2016-05-08 17:17:45.981020[-]无法碰撞打开的文件限制
2016-05-08 17:17:46.086232[-]Xvfb启动:[“Xvfb”、“:1”、“-屏幕”、“0”、“1024x768x24”]
2016-05-08 17:17:46.161902[-]代理配置文件支持已启用,代理配置文件路径:/etc/splash/proxy profiles
2016-05-08 17:17:46.260357[-]冗长=1
2016-05-08 17:17:46.260607[-]槽=50
2016-05-08 17:17:46.261170[-]参数\u缓存\u最大\u条目=500
2016-05-08 17:17:46.262476[-]Web UI:已启用,Lua:已启用(沙盒:已启用)
2016-05-08 17:17:46.264565[-]从8050开始的站点
2016-05-08 17:17:46.265203[-]启动工厂
它挂在那里。我试着根据说明进行故障排除(重新安装docker,验证vm是否正在运行,重新生成证书,设置env),但仍然一无所获

当运行spider时,如果没有
中的
meta=
参数,就产生了scrapy.Request“callwithin
start\u requests”,spider运行良好(除了动态字段没有被捕获),因此我不确定这是docker还是splash的问题


提前感谢。

使用Docker运行Splash时,控制台一开始确实会“挂在那里”。Splash正在等待通过其一个端点的请求

$ sudo docker run -p 8050:8050 scrapinghub/splash
2016-05-09 10:21:42+0000 [-] Log opened.
2016-05-09 10:21:42.773541 [-] Splash version: 2.1
2016-05-09 10:21:42.774298 [-] Qt 5.5.1, PyQt 5.5.1, WebKit 538.1, sip 4.17, Twisted 16.1.1, Lua 5.2
2016-05-09 10:21:42.774453 [-] Python 3.4.3 (default, Oct 14 2015, 20:28:29) [GCC 4.8.4]
2016-05-09 10:21:42.774632 [-] Open files limit: 1048576
2016-05-09 10:21:42.774842 [-] Can't bump open files limit
2016-05-09 10:21:42.879868 [-] Xvfb is started: ['Xvfb', ':1', '-screen', '0', '1024x768x24']
2016-05-09 10:21:43.072351 [-] proxy profiles support is enabled, proxy profiles path: /etc/splash/proxy-profiles
2016-05-09 10:21:43.214478 [-] verbosity=1
2016-05-09 10:21:43.214617 [-] slots=50
2016-05-09 10:21:43.214703 [-] argument_cache_max_entries=500
2016-05-09 10:21:43.215195 [-] Web UI: enabled, Lua: enabled (sandbox: enabled)
2016-05-09 10:21:43.217494 [-] Site starting on 8050
2016-05-09 10:21:43.217635 [-] Starting factory <twisted.web.server.Site object at 0x7f529d0fee48>
$sudo docker run-p 8050:8050 scrapinghub/splash
2016-05-09 10:21:42+0000[-]日志已打开。
2016-05-09 10:21:42.773541[-]初始版本:2.1
2016-05-09 10:21:42.774298[-]Qt 5.5.1,PyQt 5.5.1,WebKit 538.1,sip 4.17,Twisted 16.1.1,Lua 5.2
2016-05-09 10:21:42.774453[-]Python 3.4.3(默认值,2015年10月14日,20:28:29)[GCC 4.8.4]
2016-05-09 10:21:42.774632[-]打开文件限制:1048576
2016-05-09 10:21:42.774842[-]无法碰撞打开的文件限制
2016-05-09 10:21:42.879868[-]Xvfb启动:['Xvfb',':1','屏幕','0','1024x768x24']
2016-05-09 10:21:43.072351[-]代理配置文件支持已启用,代理配置文件路径:/etc/splash/proxy profiles
2016-05-09 10:21:43.214478[-]冗长=1
2016-05-09 10:21:43.214617[-]槽=50
2016-05-09 10:21:43.214703[-]参数缓存最大项=500
2016-05-09 10:21:43.215195[-]Web UI:已启用,Lua:已启用(沙盒:已启用)
2016-05-09 10:21:43.217494[-]地点从8050开始
2016-05-09 10:21:43.217635[-]启动工厂
要测试Splash是否正确运行,请在上尝试其web UI

您应该看到如下内容:


然后您可以尝试输入一些URL并再次单击“渲染我!”

谢谢-我已经做到了这一点,但当我转到localhost时,它说无法访问localhost拒绝连接。有什么建议吗?正如我所说,如果我能看到
settings.py
settings.py
与Splash服务器无关,可能会更容易。我不知道是什么原因导致8050端口上本地主机的连接被拒绝。您在该端口上运行了其他程序吗?没有,我运行了
lsof-i | grep LISTEN
,但8050未被列为正在使用。由于我在mac上,我们的系统管理员说我可能需要双重暴露端口(在vm和我的机器上)。。有什么建议吗?抱歉打扰了,我注意到你在scrapinghub工作。