Python 刮擦问题
我正在尝试使用Scrapy运行刮板,以前我可以使用此代码,但现在我遇到了一个奇怪的错误Python 刮擦问题,python,scrapy,Python,Scrapy,我正在尝试使用Scrapy运行刮板,以前我可以使用此代码,但现在我遇到了一个奇怪的错误 _rules =(Rule(LinkExtractor(restrict_xpaths=(xpath_str)), follow=True, callback='parse_url'),) def parse_url(self, response): print response.url ... 基本上,当我运行它时,我得到的是: Traceback (most
_rules =(Rule(LinkExtractor(restrict_xpaths=(xpath_str)), follow=True,
callback='parse_url'),)
def parse_url(self, response):
print response.url
...
基本上,当我运行它时,我得到的是:
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/scrapy/utils/defer.py", line 102, in iter_errback
yield next(it)
File "/usr/lib/pymodules/python2.7/scrapy/spidermiddlewares/offsite.py", line 28, in process_spider_output
for x in result:
File "/usr/lib/pymodules/python2.7/scrapy/spidermiddlewares/referer.py", line 22, in <genexpr>
return (_set_referer(r) for r in result or ())
File "/usr/lib/pymodules/python2.7/scrapy/spidermiddlewares/urllength.py", line 37, in <genexpr>
return (r for r in result or () if _filter(r))
File "/usr/lib/pymodules/python2.7/scrapy/spidermiddlewares/depth.py", line 54, in <genexpr>
return (r for r in result or () if _filter(r))
File "/usr/lib/pymodules/python2.7/scrapy/spiders/crawl.py", line 67, in _parse_response
cb_res = callback(response, **cb_kwargs) or ()
TypeError: 'str' object is not callable
将
callback='parse\u url'
更改为callback=self.parse\u url
您是否有一个名为parse_url
的函数?然后您需要指定parse\u url
而不是'parse\u url'
您确定这与规则中的回调相关吗?请发布您的爬行器的完整代码。@falsetru我有一个函数parse_url,并尝试了您的建议,但它不起作用。@alecxe我非常确定,因为当我删除回调时它确实起作用。该错误表明,在尝试调用回调时,规则类中存在问题。这是完整的代码,非常简单。@ilijaluve首先或全部,为什么是\u规则
而不是规则
?另外,检查缩进。
from scrapy.linkextractors import LinkExtractor
from scrapy.spiders import CrawlSpider, Rule
from ..model import Properties
class TestScraper(CrawlSpider):
name = "test"
start_urls = [Properties.start_url]
_rules =( Rule(LinkExtractor(restrict_xpaths=(Properites.xpath)), follow=True, callback='parse_url'), )
def parse_url(self, response):
print response.url