Scrapy 热使用刮擦deltafetch_键和爬行蜘蛛

Scrapy 热使用刮擦deltafetch_键和爬行蜘蛛,scrapy,web-crawler,Scrapy,Web Crawler,如何使爬行蜘蛛在其请求中使用deltafetch_键? 以下定义不起作用,解析函数甚至没有被调用 news_links_extractor = LinkExtractor(allow=[r'/news']) all_links_extractor = LinkExtractor(allow=[r'/.*']) def parse(self, response): nlinks = map(canonicalize_url, news_links_extractor.extract_li

如何使爬行蜘蛛在其请求中使用deltafetch_键? 以下定义不起作用,解析函数甚至没有被调用

news_links_extractor = LinkExtractor(allow=[r'/news'])
all_links_extractor = LinkExtractor(allow=[r'/.*'])

def parse(self, response):
    nlinks = map(canonicalize_url, news_links_extractor.extract_links(response))
    alinks = map(canonicalize_url, all_links_extractor.extract_links(response))

    for link in alinks:
        if link in nlinks:
            yield scrapy.Request(link, callback = self.parse_news,
                                 meta={'deltafetch_key': link})
        else:
            yield scrapy.Request(link, callback = self.parse, 
                                 meta={'deltafetch_key': link})

你能分享你蜘蛛的其他部分吗?