Python 3.x Twisted.internet.error.ReactorNotRestartable
当我运行这个蜘蛛时,我得到了“扭曲。互联网。错误。反应或不可恢复”。事实上,我在项目目录中运行任何spider时都会遇到这个错误。我上次检查时,它工作正常,但从今天起,它给了我这个错误。我对twisted不熟悉。请帮忙Python 3.x Twisted.internet.error.ReactorNotRestartable,python-3.x,scrapy,Python 3.x,Scrapy,当我运行这个蜘蛛时,我得到了“扭曲。互联网。错误。反应或不可恢复”。事实上,我在项目目录中运行任何spider时都会遇到这个错误。我上次检查时,它工作正常,但从今天起,它给了我这个错误。我对twisted不熟悉。请帮忙 import scrapy from quo.items import QuoItem from scrapy.linkextractors import LinkExtractor from scrapy.spiders import CrawlSpider, Rule fro
import scrapy
from quo.items import QuoItem
from scrapy.linkextractors import LinkExtractor
from scrapy.spiders import CrawlSpider, Rule
from twisted.internet import reactor
class ISpider(CrawlSpider):
name='iShopE'
handle_httpstatus_list = [400]
def start_requests(self):
yield scrapy.Request('https://www.ishopping.pk/electronics/home-theatres.html')
def parse(self, response):
for href in response.xpath('//div[@class="category-products-"]').extract():
for product_page in response.xpath('//h2[@class="product-name"]/a/@href').extract():
url=response.urljoin(product_page)
yield scrapy.Request(url, callback=self.parse_productPage)
next_page=response.xpath('(//a[@class="next i-next"]/@href)[1]').extract()
if next_page:
yield scrapy.Request(response.urljoin(next_page[0]), callback=self.parse)
def parse_productPage(self,response):
url_ProductPage=response.url
item=QuoItem()
if url_ProductPage:
item['url_ProductPage']=url_ProductPage
for rev in response.xpath('//div[@class="product-essential"]'):
price=response.xpath('//div[@class="price-box"]/span[@class="regular-price"]/meta[@itemprop="price"]/@content').extract()
if price:
item['price']=price
newPrice=response.xpath('(//div[@class="price-box"]/p[@class="special-price"]/span[@class="price"]/text())[1]').extract()
if newPrice:
newPrice=" ".join(str(x) for x in newPrice)
newPrice=newPrice.strip('\n')
item['price'] =newPrice.replace(" ", "")
item['newPrice']=newPrice.replace(" ","")
oldPrice=response.xpath('(//div[@class="price-box"]/p[@class="old-price"]/span[@class="price"]/text())[1]').extract()
if oldPrice:
oldPrice=" ".join(str(x) for x in oldPrice)
oldPrice=oldPrice.strip('\n')
item['oldPrice'] =oldPrice.replace(" ", "")
Availability=response.xpath('//p[@class="availability in-stock"]/span[@class="value"]/text()').extract()
if Availability:
item['Availability']=Availability
Brand=response.xpath('(//div[@class="box-p-attr"]/span)[1]/text()').extract()
if Brand:
item['Brand']=Brand
deliveryTime=response.xpath('(//div[@class="box-p-attr"]/span)[2]/text()').extract()
if deliveryTime:
item['deliveryTime']=deliveryTime
Waranty=response.xpath('(//div[@class="box-p-attr"]/span)[3]/text()').extract()
if Waranty:
item['Waranty']=Waranty
title=response.xpath('//div[@class="product-name"]/h1[@itemprop="name"]/text()').extract()
if title:
item['Title']=title
yield item
您不需要
reactor.run();reactor.stop()
。Scrapy是为你做的。如果删除这些语句会发生什么?有或没有这些语句,都会出现相同的错误。那么您如何运行spider呢?