Python can';我无法从scrapy获得输出
当我把Python can';我无法从scrapy获得输出,python,scrapy,Python,Scrapy,当我把 scrapy runspider divar.py -o data.json 在终端,我得到一个空文件。我做错什么了吗?我想从我放在start_URL中的URL中得到类别和子类别的结果,然后输入结果并打印出来,还得到一个json文件。主要是json文件 import scrapy class ws(scrapy.Spider): name = 'wsDivar' result = [] start_urls =["https://divar.ir
scrapy runspider divar.py -o data.json
在终端,我得到一个空文件。我做错什么了吗?我想从我放在start_URL中的URL中得到类别和子类别的结果,然后输入结果并打印出来,还得到一个json文件。主要是json文件
import scrapy
class ws(scrapy.Spider):
name = 'wsDivar'
result = []
start_urls =["https://divar.ir/s/tehran"]
def parse(self, response):
for category in response.xpath("//*/ul[@class='kt-accordion-item__header']"):
x = {'cats' : category.xpath("//*/ul[@class='kt-accordion-item__header']/a").extract_first()}
result.append(x)
yield(x)
print(result)
next_L =response.xpath("//li[@class='next']/a/@href").extract_first()
if next_L is not None:
next_link = response.urljoin(next_L)
yield scrapy.Request(url=next_link, callback=self.parse)
如果XPath工作正常<代码>产量还可以打印结果
与此相反:
scrapy runspider divar.py-o data.json
使用以下命令:
scrapy crawl wsDivar-o data.json
另外,在项目目录中运行该命令,该目录应该包括scrapy.cfg
文件
import scrapy
class ws(scrapy.Spider):
name = 'wsDivar'
start_urls =["https://divar.ir/s/tehran"]
def parse(self, response):
for category in response.xpath("//*/ul[@class='kt-accordion-item__header']"):
x = {'cats' : category.xpath("//*/ul[@class='kt-accordion-item__header']/a").extract_first()}
yield(x)
next_L =response.xpath("//li[@class='next']/a/@href").extract_first()
if next_L is not None:
next_link = response.urljoin(next_L)
yield scrapy.Request(url=next_link, callback=self.parse)