Python Scrapy未发送post请求
我试图向URL发出post请求,但scrapy没有发送post请求。我没有得到正确的回答 下面是我的代码Python Scrapy未发送post请求,python,scrapy,Python,Scrapy,我试图向URL发出post请求,但scrapy没有发送post请求。我没有得到正确的回答 下面是我的代码 import scrapy class MainSpider(scrapy.Spider): name = 'main' directory = 'https://www.constructionenquirer.com/directory' start_urls = [directory] def parse(self, response):
import scrapy
class MainSpider(scrapy.Spider):
name = 'main'
directory = 'https://www.constructionenquirer.com/directory'
start_urls = [directory]
def parse(self, response):
sectors = response.xpath('(//select[@name="sector"]/option)[position()>1]')
for sector in sectors:
vals = sector.xpath('.//@value').get()
data = {
'ce-directory-action': 'ce-directory-action',
'sector': vals,
'action': 'find-firms-by-sector'
}
yield scrapy.FormRequest(url=self.directory, formdata=data, callback=self.parse_sectors)
def parse_sectors(self, response):
yield {
"Name": response.xpath('//h3/a/text()').get()
}
您的代码中有一个输入错误:
'ce-directory-action': 'sector-search',
请确保将表单请求发布到正确的url以获取更新的数据