Pandas 读取html缺少第一个表
第一张表格在竞选网站“擦肩而过”期间无法通过: url= 以下是代码:Pandas 读取html缺少第一个表,pandas,web-scraping,python-requests-html,Pandas,Web Scraping,Python Requests Html,第一张表格在竞选网站“擦肩而过”期间无法通过: url= 以下是代码: import requests from bs4 import BeautifulSoup import pandas as pd headers = { "accept": "application/json, text/javascript, */*; q=0.01", "accept-encoding": "gzip, deflate
import requests
from bs4 import BeautifulSoup
import pandas as pd
headers = {
"accept": "application/json, text/javascript, */*; q=0.01",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en-GB,en-US;q=0.9,en;q=0.8",
"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.99 Safari/537.36",
"x-requested-with": "XMLHttpRequest",
}
url = r'https://electproject.github.io/Early-Vote-2020G/GA_RO.html'
r = requests.get(url, headers=headers).text
soup = BeautifulSoup(r, 'html.parser')
这两种方法都试过了,但仍然没有获得县/选票/投票率的第一名
tables = soup.findAll('table')
dfs = list()
for table in tables:
df = pd.read_html(str(table))[0]
dfs.append(df)
其他尝试:
df = pd.read_html(r, flavor='html5lib')
两人都拉其他所有桌子,但不是第一张。我假设这是由于具有排序功能的标题造成的,但不确定。问题是第一个表是用JavaScript呈现的,HTML中没有该表的
您可以直接从JavaScript获取数据(可以检查页面源代码以找到正确的
元素):
输出:
0 1 2 3
0 APPLING 4453 12240 0.363807
1 ATKINSON 1431 4939 0.289735
2 BACON 3111 7071 0.439966
3 BAKER 872 2297 0.379626
4 BALDWIN 11850 27567 0.429862
.. ... ... ... ...
154 WHITFIELD 16980 57014 0.297822
155 WILCOX 1582 4838 0.326995
156 WILKES 2958 7204 0.410605
157 WILKINSON 2236 6761 0.33072
158 WORTH 4473 14601 0.306349
谢谢你的帮助
0 1 2 3
0 APPLING 4453 12240 0.363807
1 ATKINSON 1431 4939 0.289735
2 BACON 3111 7071 0.439966
3 BAKER 872 2297 0.379626
4 BALDWIN 11850 27567 0.429862
.. ... ... ... ...
154 WHITFIELD 16980 57014 0.297822
155 WILCOX 1582 4838 0.326995
156 WILKES 2958 7204 0.410605
157 WILKINSON 2236 6761 0.33072
158 WORTH 4473 14601 0.306349