Python 如何从';a';标签

Python 如何从';a';标签,python,web-scraping,beautifulsoup,Python,Web Scraping,Beautifulsoup,我尝试了几种方法从以下网页中提取链接,但似乎找不到所需的链接。从这个网页()我试图提取“gamecast”按钮的所有链接。我试图得到的第一个例子是: 当我试图拉页面上的所有链接时,我甚至根本没有得到想要的链接,所以我很困惑我到底哪里出了问题。下面我所做的几次尝试似乎并没有达到我的目的。我在下面尝试的第一种方法 import requests import csv from bs4 import BeautifulSoup import pandas as pd page = requests

我尝试了几种方法从以下网页中提取链接,但似乎找不到所需的链接。从这个网页()我试图提取“gamecast”按钮的所有链接。我试图得到的第一个例子是:

当我试图拉页面上的所有链接时,我甚至根本没有得到想要的链接,所以我很困惑我到底哪里出了问题。下面我所做的几次尝试似乎并没有达到我的目的。我在下面尝试的第一种方法

import requests
import csv
from bs4 import BeautifulSoup
import pandas as pd

page = requests.get('https://www.espn.com/college-football/scoreboard/_/year/2019/seasontype/2/week/1')
soup = BeautifulSoup(page.text, 'html.parser')
# game_id = soup.find(name_='&lpos=college-football:scoreboard:gamecast')
game_id = soup.find('a',class_='button-alt sm')
这是我尝试过的第二种方法。非常感谢您的帮助

for a in soup.find_all('a'):
if 'college-football' in a['href']:
print(link['href'])

编辑:作为澄清,我正在尝试提取所有包含gameID的链接,如示例链接中所示。

包含您试图拥有的链接的按钮已加载javascript。requests模块不会在正在搜索的html中加载javascript。因此,您无法直接点击按钮来找到所需的链接(没有像Selenium这样的网页模拟器)。但是,我在html中找到了json数据,其中包含链接所在的记分板数据。如果您还想从本页中获取更多信息(时间等),我强烈建议您查看代码中变量
json_scoreboard
中的json数据

代码
导入请求、re、json
从bs4导入BeautifulSoup
r=requests.get(r'https://www.espn.com/college-football/scoreboard/_/year/2019/seasontype/2/week/1')
soup=BeautifulSoup(r.text'html.parser')
脚本头=汤。查找('head')。查找所有('script'))
所有链接={}
对于脚本头中的脚本:
如果script.text中的“window.espn.scoreboardData”:
json_scoreboard=json.loads(re.search(r'({.*?});'),script.text.group(1))
对于json_记分板中的事件['events']:
名称=事件['name']
对于事件['links']中的链接:
如果链接['text']=='Gamecast':
gamecast=link['href']
所有链接[名称]=游戏演员
打印(所有链接)
输出
可能重复到:我引用了该页面,并且在我的问题中有一个解决方案不起作用。据我所知,这对我的问题不起作用。这太棒了,谢谢。有一件事我有点麻烦,就是把这本字典变成了一个更传统的数据框架,其中键(迈阿密飓风在佛罗里达鳄鱼)在第一列,值(实际链接)在第二列。这是不是应该在一个新问题中提出或在这里解决的问题?事实上,对于任何感兴趣的人来说,我只是想出来了。data2=pd.DataFrame.from_dict(所有链接,orient='index')
{'Miami Hurricanes at Florida Gators': 'http://www.espn.com/college-football/game/_/gameId/401110723', 'Georgia Tech Yellow Jackets at Clemson Tigers': 'http://www.espn.com/college-football/game/_/gameId/401111653', 'Texas State Bobcats at Texas A&M Aggies': 'http://www.espn.com/college-football/game/_/gameId/401110731', 'Utah Utes at BYU Cougars': 'http://www.espn.com/college-football/game/_/gameId/401114223', 'Florida A&M Rattlers at UCF Knights': 'http://www.espn.com/college-football/game/_/gameId/401117853', 'Tulsa Golden Hurricane at Michigan State Spartans': 'http://www.espn.com/college-football/game/_/gameId/401112212', 'Wisconsin Badgers at South Florida Bulls': 'http://www.espn.com/college-football/game/_/gameId/401117856', 'Duke Blue Devils at Alabama Crimson Tide': 'http://www.espn.com/college-football/game/_/gameId/401110720', 'Georgia Bulldogs at Vanderbilt Commodores': 'http://www.espn.com/college-football/game/_/gameId/401110732', 'Florida Atlantic Owls at Ohio State Buckeyes': 'http://www.espn.com/college-football/game/_/gameId/401112251', 'Georgia Southern Eagles at LSU Tigers': 'http://www.espn.com/college-football/game/_/gameId/401110725', 'Middle Tennessee Blue Raiders at Michigan Wolverines': 'http://www.espn.com/college-football/game/_/gameId/401112222', 'Louisiana Tech Bulldogs at Texas Longhorns': 'http://www.espn.com/college-football/game/_/gameId/401112135', 'Oregon Ducks at Auburn Tigers': 'http://www.espn.com/college-football/game/_/gameId/401110722', 'Eastern Washington Eagles at Washington Huskies': 'http://www.espn.com/college-football/game/_/gameId/401114233', 'Idaho Vandals at Penn State Nittany Lions': 'http://www.espn.com/college-football/game/_/gameId/401112257', 'Miami (OH) RedHawks at Iowa Hawkeyes': 'http://www.espn.com/college-football/game/_/gameId/401112191', 'Northern Iowa Panthers at Iowa State Cyclones': 'http://www.espn.com/college-football/game/_/gameId/401112085', 'Syracuse Orange at Liberty Flames': 'http://www.espn.com/college-football/game/_/gameId/401112434', 'New Mexico State Aggies at Washington State Cougars': 'http://www.espn.com/college-football/game/_/gameId/401114228', 'South Alabama Jaguars at Nebraska Cornhuskers': 'http://www.espn.com/college-football/game/_/gameId/401112238', 'Northwestern Wildcats at Stanford Cardinal': 'http://www.espn.com/college-football/game/_/gameId/401112245', 'Houston Cougars at Oklahoma Sooners': 'http://www.espn.com/college-football/game/_/gameId/401112114', 'Notre Dame Fighting Irish at Louisville Cardinals': 'http://www.espn.com/college-football/game/_/gameId/401112436'}