Python 2.7 使用scrapy填写两阶段登录
登录后,我正在尝试刮除lynda.com。Lynda.com有一个两步验证过程,我必须输入用户名,然后输入密码。 看了这个之后,我创建了下面的spider,其中我创建了两个函数,一个用于插入用户名,另一个用于插入密码Python 2.7 使用scrapy填写两阶段登录,python-2.7,login,scrapy,Python 2.7,Login,Scrapy,登录后,我正在尝试刮除lynda.com。Lynda.com有一个两步验证过程,我必须输入用户名,然后输入密码。 看了这个之后,我创建了下面的spider,其中我创建了两个函数,一个用于插入用户名,另一个用于插入密码 # -*- coding: utf-8 -*- import scrapy from loginform import fill_login_form class LyndaSpider(scrapy.Spider): name = 'lynda' start
# -*- coding: utf-8 -*-
import scrapy
from loginform import fill_login_form
class LyndaSpider(scrapy.Spider):
name = 'lynda'
start_urls = [
'lynda.com'
]
username_url = 'https://www.lynda.com/signin'
login_user = '*****@gmail.com'
login_password = '****'
def start_requests(self):
# let's start by sending a first request to login page
yield scrapy.Request(self.username_url, self.parse_username_login)
def parse_username_login(self, response):
# got the username page, let's fill the login form...
data, url, method = fill_login_form(response.url, response.body,
username=self.login_user, password='')
# ... and send a request with our login data
return scrapy.http.FormRequest(url, formdata=dict(data),
method=method, callback=self.parse_password_login)
def parse_password_login(self, response):
# got the username page, let's fill the login form...
data, url, method = fill_login_form(response.url, response.body, username='',
password=self.login_password)
# ... and send a request with our login data
return scrapy.http.FormRequest(url, formdata=dict(data),
method=method, callback=self.start_crawl)
def start_crawl(self, response):
# OK, we're in, let's start crawling the protected pages
for url in self.start_urls:
yield scrapy.Request(url)
def parse(self, response):
# do stuff with the logged in response
pass
我得到以下错误:
Traceback (most recent call last):
File "/home/edhiraj/program_files/venv2/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/edhiraj/program_files/PycharmProjects/justin/justin/spiders/lynda_spider.py", line 25, in parse_username_login
username=self.login_user, password='')
File "/home/edhiraj/program_files/venv2/local/lib/python2.7/site-packages/loginform.py", line 80, in fill_login_form
form.fields[passfield] = password
File "/home/edhiraj/program_files/venv2/local/lib/python2.7/site-packages/lxml/html/__init__.py", line 1150, in __setitem__
self.inputs[item].value = value
File "/home/edhiraj/program_files/venv2/local/lib/python2.7/site-packages/lxml/html/__init__.py", line 1216, in __getitem__
"No input element with the name %r" % name)
KeyError: 'No input element with the name None'
为什么不使用FormRequest.from_response()@Verz1Lka谢谢。我是新来的,所以我不知道