Python 禁用重试中间件
我在settings.py中对该行进行了注释,但它仍处于启用状态Python 禁用重试中间件,python,web-scraping,scrapy,Python,Web Scraping,Scrapy,我在settings.py中对该行进行了注释,但它仍处于启用状态 DOWNLOADER_MIDDLEWARES = { #'scrapy.downloadermiddlewares.retry.RetryMiddleware': 90, 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 110, } 在程序开始时,它加载了很多我没有启用的中间件 2019-08-09 10:43:3
DOWNLOADER_MIDDLEWARES = {
#'scrapy.downloadermiddlewares.retry.RetryMiddleware': 90,
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 110,
}
在程序开始时,它加载了很多我没有启用的中间件
2019-08-09 10:43:37 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
我错过什么了吗?有没有办法禁用它?根据文档,
下载程序\u中间件
。在后者中,默认情况下启用选项scrapy.downloadermiddleware.httpproxy.RetryMiddleware
所以要么写
DOWNLOADER_MIDDLEWARES = {
'scrapy.downloadermiddlewares.retry.RetryMiddleware': None,
# ^^^
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 110,
}
或者查看DOWNLOADER\u middleware\u BASE
。看