Python 请求url必须是str或unicode,获取列表错误?
我收到以下错误:“请求url必须是str或unicode,已获取列表” 起始url为'https://www.zomato.com/istanbul/restaurants?page=1"Python 请求url必须是str或unicode,获取列表错误?,python,scrapy,paginator,Python,Scrapy,Paginator,我收到以下错误:“请求url必须是str或unicode,已获取列表” 起始url为'https://www.zomato.com/istanbul/restaurants?page=1" import scrapy def parse(self, response): all_css = response.css('.search_left_featured') all_product = all_css.css('a::attr(href)').extract()
import scrapy
def parse(self, response):
all_css = response.css('.search_left_featured')
all_product = all_css.css('a::attr(href)').extract()
yield scrapy.Request(all_product, callback=self.parse_dir_contents)
max_page_number = 10
for i in range(1, max_page_number):
url_next = 'https://www.zomato.com/istanbul/restaurants?page=' + str(i)+''
yield scrapy.Request(url_next, callback=self.parse)
def parse_dir_contents(self, response):
items = ZomatodataItem()
name = response.css('.iNaazl::text').extract()
genre =response.css('.PhzdX::text').extract()
location =response.css('.gqeQEx::text').extract()
tags = response.css('.cunMUz::text').extract()
address = response.css('.clKRrC::text').extract()
phone = response.css('.kKemRh::text').extract()
items['name']= name
items['genre']= genre
items['location']= location
items['tags']= tags
items['address']= address
items['phone_number']= phone
yield items
你的问题是什么?错误似乎很明显:
.css
返回一个选择器列表
,其提取
方法返回一个列表
,然后将其传递给请求
,该请求需要一个URL,而列表不是
迭代您的结果,或者使用来自scrapy的(更现代、更不容易混淆的).get()
和.getall()
方法,extract()
被弃用(就像scrapy在其文档中停止使用它一样),因为它的行为因在选择器(返回字符串)或选择器列表上调用而不同(返回一个列表)
见鬼,两者都做。哪一行发生了错误?