Python 使用Mechanize登录
我试图从一个网站上提取一些数据-不是很多-但足以保证一个小脚本。。。我试图首先登录到该网站使用mechanize和cookielib,但我失败了 我现在有Python 使用Mechanize登录,python,cookies,beautifulsoup,mechanize,Python,Cookies,Beautifulsoup,Mechanize,我试图从一个网站上提取一些数据-不是很多-但足以保证一个小脚本。。。我试图首先登录到该网站使用mechanize和cookielib,但我失败了 我现在有 from bs4 import BeautifulSoup import requests import re import urllib2 import cookielib import mechanize cj = cookielib.CookieJar() br = mechanize.Browser() br.set_cooki
from bs4 import BeautifulSoup
import requests
import re
import urllib2
import cookielib
import mechanize
cj = cookielib.CookieJar()
br = mechanize.Browser()
br.set_cookiejar(cj)
br.open("https://squashlevels.com/menu_login.php")
# How do I log in?
r = requests.get('https://squashlevels.com/players.php?all&club=1314')
soup = BeautifulSoup(r.content, "html.parser")
## Do stuff...
我应该使用什么代码登录此站点
谢谢您的时间。您可以使用登录
from selenium import webdriver
import bs4
email = 'username@email.com'
password = 'password'
driver = webdriver.Chrome()
driver.get('https://squashlevels.com/menu_login.php')
# log in
driver.find_element_by_name("email").send_keys(email)
driver.find_element_by_name("password").send_keys(password)
driver.find_element_by_css_selector("input[type='submit'][value='Login']").click()
# get the html then do some stuff
html = driver.page_source
soup = bs4.BeautifulSoup(html,'html.parser')
您可以使用登录
from selenium import webdriver
import bs4
email = 'username@email.com'
password = 'password'
driver = webdriver.Chrome()
driver.get('https://squashlevels.com/menu_login.php')
# log in
driver.find_element_by_name("email").send_keys(email)
driver.find_element_by_name("password").send_keys(password)
driver.find_element_by_css_selector("input[type='submit'][value='Login']").click()
# get the html then do some stuff
html = driver.page_source
soup = bs4.BeautifulSoup(html,'html.parser')
这里有一个仅使用请求的解决方案,我不确定mechanize是否会提供任何附加值。通过使用requests.Session,您可以维护登录过程返回的cookie,以便在请求players.php时包含这些cookie 站点在混合中遇到的一个小麻烦是,您还需要发布密码的MD5散列:
email = 'user@host.com'
password = 'secret'
s = requests.Session()
s.post('https://squashlevels.com/menu_login.php', data={
'action': 'login',
'email': email,
'password': password,
'md5password': hashlib.md5(password.encode('utf-8')).hexdigest()
})
r = s.get('https://squashlevels.com/players.php?all&club=1314')
soup = BeautifulSoup(r.content, 'html.parser')
for row in soup.select('table.ranking tr'):
print([col.text.strip() for col in row.select('td')])
输出:
['1', 'Nathan Miller', 'Bluecoat Sports Horsham', 'East England Masters 2018/19', '6', '15 Dec 2018', '4,706', '70%', '']
['2', 'Kit Pearman', 'Dorking', 'Surrey Winter League 2018/19', '2', '20 Nov 2018', '4,469', '64%', '']
['3', 'Simon Millard', 'Bluecoat Sports Horsham', 'Sussex Mens League 2018/2019', '1', '04 Dec 2018', '2,680', '57%', '']
...
这里有一个仅使用请求的解决方案,我不确定mechanize是否会提供任何附加值。通过使用requests.Session,您可以维护登录过程返回的cookie,以便在请求players.php时包含这些cookie 站点在混合中遇到的一个小麻烦是,您还需要发布密码的MD5散列:
email = 'user@host.com'
password = 'secret'
s = requests.Session()
s.post('https://squashlevels.com/menu_login.php', data={
'action': 'login',
'email': email,
'password': password,
'md5password': hashlib.md5(password.encode('utf-8')).hexdigest()
})
r = s.get('https://squashlevels.com/players.php?all&club=1314')
soup = BeautifulSoup(r.content, 'html.parser')
for row in soup.select('table.ranking tr'):
print([col.text.strip() for col in row.select('td')])
输出:
['1', 'Nathan Miller', 'Bluecoat Sports Horsham', 'East England Masters 2018/19', '6', '15 Dec 2018', '4,706', '70%', '']
['2', 'Kit Pearman', 'Dorking', 'Surrey Winter League 2018/19', '2', '20 Nov 2018', '4,469', '64%', '']
['3', 'Simon Millard', 'Bluecoat Sports Horsham', 'Sussex Mens League 2018/2019', '1', '04 Dec 2018', '2,680', '57%', '']
...
直接回答您的问题:mechanize无法轻松处理此用例,因为此页面上的标记有点奇怪-表单控件实际上位于表单本身之外,表单本身为空: 这将导致在登录表单中检测0控件的机械化
提交按钮的行为是通过javascript控制的。直接回答您的问题:mechanize无法轻松处理此用例,因为此页面上的标记有点奇怪-表单控件实际上在表单本身之外,表单本身是空的: 这将导致在登录表单中检测0控件的机械化 提交按钮的行为是通过javascript控制的。I get-NameErrorname'hashlib'未定义,我假设我需要更新我的请求库?I get-NameErrorname'hashlib'未定义,我假设我需要更新我的请求库?