Can';t使用Scrapy登录 已经是会员了吗?
请输入您的帐户详细信息: 您的电子邮件*Can';t使用Scrapy登录 已经是会员了吗?,scrapy,web-crawler,scrapy-spider,scrapy-shell,Scrapy,Web Crawler,Scrapy Spider,Scrapy Shell,请输入您的帐户详细信息: 您的电子邮件* 密码* *必填字段 尝试使用scrapy.FormRequest.from_response() 但似乎不起作用 我需要登录才能完全访问产品详细信息 登录页面:这是如何登录到目标站点的示例。要做到这一点,您需要打开浏览器并了解发送到服务器的所有数据。当您了解它是如何工作的,您就可以编写自己的代码了 <div class="col small-w100 tiny-w100 col1"> <div class="box_alr
密码*
*必填字段
尝试使用scrapy.FormRequest.from_response() 但似乎不起作用 我需要登录才能完全访问产品详细信息
登录页面:这是如何登录到目标站点的示例。要做到这一点,您需要打开浏览器并了解发送到服务器的所有数据。当您了解它是如何工作的,您就可以编写自己的代码了
<div class="col small-w100 tiny-w100 col1">
<div class="box_already_member">
<h2 class="fs22 fwn foro black">Already member ?</h2>
<p>Please enter your account details : </p>
<div class="box_form">
<label>Your email* </label>
<br>
<input id="txtUsernamelogin" type="text" data-parsley-group="glogin"
data-parsley-required="true" data-parsley-errors-container="#lblMessage"
data-parsley-type-message="Please check that your Professional Email is
in correct format" data-parsley-required-message="Please type your
Professional Email" data-parsley-type="email"><br>
<label>Password* </label>
<br>
<input id="txtPasswordlogin" type="password" data-parsley-group="glogin"
data-parsley-required="true" data-parsley-errors-container="#lblMessage"
data-parsley-required-message="Please type your password"><br>
<div class="row pt20 pb20">
<div class="col "><a class="c19" href="/forgot-password"
rel="nofollow">Forgot password ?</a></div>
<div class="col txtright">
<div class="inbl">
<a href="#" id="loginbtnclick" class=" row wauto fs14
c0 bgc18 rounded5 txtcenter h36p vam tdn mb20">
<span class="col vam fs16 pr40 pl40">
<strong>LOGIN</strong></span>
</a>
</div>
</div>
</div>
</div>
<span>*mandatory fields</span><br>
<span id="lblMessage" class="red"></span>
</div>
import scrapy
从scrapy.exceptions导入CloseSpider
从scrapy.spider导入爬行蜘蛛
类特殊蜘蛛(爬行蜘蛛):
名称='specialchem'
允许的_域=[']
起始URL=['https://cosmetics./login']
自定义_设置={'ROBOTSTXT_-obe':False}
def解析(自我,响应):
inputs=response.css('表单输入')
formdata={'Iid':'',
“密码”:“密码”,
“用户”:“电子邮件”,
'Popin':'1'}
返回scrapy.FormRequest(
'https://cosmetics./services/LoginService.ashx',
formdata=formdata,
callback=self.after\u登录
)
登录后的def(自我,响应):
如果响应中没有“确定”。文本:
raise CloseSpider('登录或密码错误。或者您被阻止')
url='1〕https://cosmetics./product/i-eastman-chemical-company-eastman-aq-38s-polymer'
返回scrapy.Request(url,callback=self.product)
def产品(自我、响应):
通过
它将工作时,你取代域,秘密和电子邮件,以纠正一个
import scrapy
from scrapy.exceptions import CloseSpider
from scrapy.spiders import CrawlSpider
class SpecialchemSpider(CrawlSpider):
name = 'specialchem'
allowed_domains = ['<DOMAIN>']
start_urls = ['https://cosmetics.<DOMAIN>/login']
custom_settings = {'ROBOTSTXT_OBEY': False}
def parse(self, response):
inputs = response.css('form input')
formdata = {'Iid': '',
'Password': 'SECRET',
'User': 'EMAIL',
'Popin': '1'}
return scrapy.FormRequest(
'https://cosmetics.<DOMAIN>/services/LoginService.ashx',
formdata=formdata,
callback=self.after_login
)
def after_login(self, response):
if 'OK' not in response.text:
raise CloseSpider('Wrong login or password. Or you was blocked.')
url = 'https://cosmetics.<DOMAIN>/product/i-eastman-chemical-company-eastman-aq-38s-polymer'
return scrapy.Request(url, callback=self.product)
def product(self, response):
pass