Fatal编程技术网
  • python/
  • Python Bs4不返回任何数据，因为它'；s仅返回表中已注释的部分

Python Bs4不返回任何数据，因为它'；s仅返回表中已注释的部分

python python-3.x

Python Bs4不返回任何数据，因为它'；s仅返回表中已注释的部分,python,python-3.x,beautifulsoup,Python,Python 3.x,Beautifulsoup,因此，从下一页开始— 我试图从季后赛表格中获取数据，但当我瞄准它时，它只返回给我评论的部分，而不是可以操纵的实际数据。例如，我只想打印出表中所有的通配符游戏 def loopForData(): currentYear = 1994 for i in range(1, 2): currentYear += 1 nfl_url = "https://www.pro-football-reference.com/years/"+st

因此，从下一页开始—

我试图从季后赛表格中获取数据，但当我瞄准它时，它只返回给我评论的部分，而不是可以操纵的实际数据。例如，我只想打印出表中所有的通配符游戏

def loopForData():
    currentYear = 1994
    for i in range(1, 2):
        currentYear += 1
        nfl_url = "https://www.pro-football-reference.com/years/"+str(currentYear)+"/index.htm"
        result = requests.get(nfl_url).text
        data = BeautifulSoup(result, 'html5lib')
        plyOff_table = data.find(id="all_playoff_results")
        print(plyOff_table)


loopForData()
它只返回以下内容：-请注意，它只返回了commneted表。我已经尝试了几个解析器，知道并一直得到相同的结果，哈哈。有什么想法吗

<div class="table_wrapper setup_commented commented" id="all_playoff_results">

<div class="section_heading" id="playoff_results_sh">
  <span class="section_anchor" data-label="Playoff Results" id="playoff_results_link"></span><h2>Playoff Results</h2>    <div class="section_heading_text">
      <ul>
      </ul>
    </div>

</div><div class="placeholder"></div>
<!--

<div class="table_container" id="div_playoff_results">

    <table class="sortable stats_table" id="playoff_results" data-cols-to-freeze=",1">
    <caption>Playoff Results Table</caption>

   <colgroup><col><col><col><col><col><col><col><col><col></colgroup>
   <thead>
      <tr>
         <th aria-label="Week number in season" data-stat="week_num" scope="col" class=" poptip sort_default_asc sorttable_nosort center" data-tip="Week number in season" >Week</th>
         <th aria-label="Day" data-stat="game_day_of_week" scope="col" class=" poptip sort_default_asc left" >Day</th>
         <th aria-label="Date" data-stat="game_date" scope="col" class=" poptip sort_default_asc center" >Date</th>
         <th aria-label="Winner/tie" data-stat="winner" scope="col" class=" poptip sort_default_asc center" >Winner/tie</th>
         <th aria-label="game_location" data-stat="game_location" scope="col" class=" poptip sort_default_asc center" ></th>
         <th aria-label="Loser/tie" data-stat="loser" scope="col" class=" poptip sort_default_asc center" >Loser/tie</th>
         <th aria-label="boxscore_word" data-stat="boxscore_word" scope="col" class=" poptip sort_default_asc center" ></th>
         <th aria-label="Points Scored by the winning team (first one listed)" data-stat="pts_win" scope="col" class=" poptip center" data-tip="Points Scored by the winning team (first one listed)" >PtsW</th>
         <th aria-label="Points Scored by the losing team (second one listed)" data-stat="pts_lose" scope="col" class=" poptip center" data-tip="Points Scored by the losing team (second one listed)" >PtsL</th>
      </tr>
      </thead>
<tbody><tr ><th scope="row" class="right " data-stat="week_num" csk="31" >WildCard</th><td class="left " data-stat="game_day_of_week" >Sat</td><td class="left " data-stat="game_date" csk="zz1" >December 30</td><td class="left " data-stat="winner" ><strong><a href="/teams/buf/1995.htm">Buffalo Bills</a></strong></td><td class="right iz" data-stat="game_location" ></td><td class="left " data-stat="loser" ><a href="/teams/mia/1995.htm">Miami Dolphins</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199512300buf.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>37</strong></td><td class="right " data-stat="pts_lose" >22</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="31" >WildCard</th><td class="left " data-stat="game_day_of_week" >Sat</td><td class="left " data-stat="game_date" csk="zz1" >December 30</td><td class="left " data-stat="winner" ><strong><a href="/teams/phi/1995.htm">Philadelphia Eagles</a></strong></td><td class="right iz" data-stat="game_location" ></td><td class="left " data-stat="loser" ><a href="/teams/det/1995.htm">Detroit Lions</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199512300phi.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>58</strong></td><td class="right " data-stat="pts_lose" >37</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="31" >WildCard</th><td class="left " data-stat="game_day_of_week" >Sun</td><td class="left " data-stat="game_date" csk="zz1" >December 31</td><td class="left " data-stat="winner" ><strong><a href="/teams/clt/1995.htm">Indianapolis Colts</a></strong></td><td class="right " data-stat="game_location" >@</td><td class="left " data-stat="loser" ><a href="/teams/sdg/1995.htm">San Diego Chargers</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199512310sdg.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>35</strong></td><td class="right " data-stat="pts_lose" >20</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="31" >WildCard</th><td class="left " data-stat="game_day_of_week" >Sun</td><td class="left " data-stat="game_date" csk="zz1" >December 31</td><td class="left " data-stat="winner" ><strong><a href="/teams/gnb/1995.htm">Green Bay Packers</a></strong></td><td class="right iz" data-stat="game_location" ></td><td class="left " data-stat="loser" ><a href="/teams/atl/1995.htm">Atlanta Falcons</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199512310gnb.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>37</strong></td><td class="right " data-stat="pts_lose" >20</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="32" >Division</th><td class="left " data-stat="game_day_of_week" >Sat</td><td class="left " data-stat="game_date" csk="zz3" >January 6</td><td class="left " data-stat="winner" ><strong><a href="/teams/pit/1995.htm">Pittsburgh Steelers</a></strong></td><td class="right iz" data-stat="game_location" ></td><td class="left " data-stat="loser" ><a href="/teams/buf/1995.htm">Buffalo Bills</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199601060pit.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>40</strong></td><td class="right " data-stat="pts_lose" >21</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="32" >Division</th><td class="left " data-stat="game_day_of_week" >Sat</td><td class="left " data-stat="game_date" csk="zz3" >January 6</td><td class="left " data-stat="winner" ><strong><a href="/teams/gnb/1995.htm">Green Bay Packers</a></strong></td><td class="right " data-stat="game_location" >@</td><td class="left " data-stat="loser" ><a href="/teams/sfo/1995.htm">San Francisco 49ers</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199601060sfo.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>27</strong></td><td class="right " data-stat="pts_lose" >17</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="32" >Division</th><td class="left " data-stat="game_day_of_week" >Sun</td><td class="left " data-stat="game_date" csk="zz3" >January 7</td><td class="left " data-stat="winner" ><strong><a href="/teams/dal/1995.htm">Dallas Cowboys</a></strong></td><td class="right iz" data-stat="game_location" ></td><td class="left " data-stat="loser" ><a href="/teams/phi/1995.htm">Philadelphia Eagles</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199601070dal.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>30</strong></td><td class="right " data-stat="pts_lose" >11</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="32" >Division</th><td class="left " data-stat="game_day_of_week" >Sun</td><td class="left " data-stat="game_date" csk="zz3" >January 7</td><td class="left " data-stat="winner" ><strong><a href="/teams/clt/1995.htm">Indianapolis Colts</a></strong></td><td class="right " data-stat="game_location" >@</td><td class="left " data-stat="loser" ><a href="/teams/kan/1995.htm">Kansas City Chiefs</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199601070kan.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>10</strong></td><td class="right " data-stat="pts_lose" >7</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="33" >ConfChamp</th><td class="left " data-stat="game_day_of_week" >Sun</td><td class="left " data-stat="game_date" csk="zz5" >January 14</td><td class="left " data-stat="winner" ><strong><a href="/teams/dal/1995.htm">Dallas Cowboys</a></strong></td><td class="right iz" data-stat="game_location" ></td><td class="left " data-stat="loser" ><a href="/teams/gnb/1995.htm">Green Bay Packers</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199601140dal.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>38</strong></td><td class="right " data-stat="pts_lose" >27</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="33" >ConfChamp</th><td class="left " data-stat="game_day_of_week" >Sun</td><td class="left " data-stat="game_date" csk="zz5" >January 14</td><td class="left " data-stat="winner" ><strong><a href="/teams/pit/1995.htm">Pittsburgh Steelers</a></strong></td><td class="right iz" data-stat="game_location" ></td><td class="left " data-stat="loser" ><a href="/teams/clt/1995.htm">Indianapolis Colts</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199601140pit.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>20</strong></td><td class="right " data-stat="pts_lose" >16</td></tr>
<tr ><th scope="row" class="right " data-stat="week_num" csk="34" >SuperBowl</th><td class="left " data-stat="game_day_of_week" >Sun</td><td class="left " data-stat="game_date" csk="zz8" >January 28</td><td class="left " data-stat="winner" ><strong><a href="/teams/dal/1995.htm">Dallas Cowboys</a></strong></td><td class="right " data-stat="game_location" >N</td><td class="left " data-stat="loser" ><a href="/teams/pit/1995.htm">Pittsburgh Steelers</a></td><td class="center " data-stat="boxscore_word" ><a href="/boxscores/199601280dal.htm">boxscore</a></td><td class="right " data-stat="pts_win" ><strong>27</strong></td><td class="right " data-stat="pts_lose" >17</td></tr>

</table>


</div>
-->


</div>


季后赛结果
您正在刮取错误的标签。您要查找的内容存储在：

<div class="table_container is_setup" id="div_playoff_results">
上述代码将返回：

Week Day Date Winner/tie Loser/tie PtsW PtsL
WildCard Sat December 30 Buffalo Bills Miami Dolphins boxscore 37 22
WildCard Sat December 30 Philadelphia Eagles Detroit Lions boxscore 58 37
WildCard Sun December 31 Indianapolis Colts @ San Diego Chargers boxscore 35 20
WildCard Sun December 31 Green Bay Packers Atlanta Falcons boxscore 37 20
Division Sat January 6 Pittsburgh Steelers Buffalo Bills boxscore 40 21
Division Sat January 6 Green Bay Packers @ San Francisco 49ers boxscore 27 17
Division Sun January 7 Dallas Cowboys Philadelphia Eagles boxscore 30 11
Division Sun January 7 Indianapolis Colts @ Kansas City Chiefs boxscore 10 7
ConfChamp Sun January 14 Dallas Cowboys Green Bay Packers boxscore 38 27
ConfChamp Sun January 14 Pittsburgh Steelers Indianapolis Colts boxscore 20 16
SuperBowl Sun January 28 Dallas Cowboys N Pittsburgh Steelers boxscore 27 17
你刮错标签了。您要查找的内容存储在：

<div class="table_container is_setup" id="div_playoff_results">
上述代码将返回：

Week Day Date Winner/tie Loser/tie PtsW PtsL
WildCard Sat December 30 Buffalo Bills Miami Dolphins boxscore 37 22
WildCard Sat December 30 Philadelphia Eagles Detroit Lions boxscore 58 37
WildCard Sun December 31 Indianapolis Colts @ San Diego Chargers boxscore 35 20
WildCard Sun December 31 Green Bay Packers Atlanta Falcons boxscore 37 20
Division Sat January 6 Pittsburgh Steelers Buffalo Bills boxscore 40 21
Division Sat January 6 Green Bay Packers @ San Francisco 49ers boxscore 27 17
Division Sun January 7 Dallas Cowboys Philadelphia Eagles boxscore 30 11
Division Sun January 7 Indianapolis Colts @ Kansas City Chiefs boxscore 10 7
ConfChamp Sun January 14 Dallas Cowboys Green Bay Packers boxscore 38 27
ConfChamp Sun January 14 Pittsburgh Steelers Indianapolis Colts boxscore 20 16
SuperBowl Sun January 28 Dallas Cowboys N Pittsburgh Steelers boxscore 27 17
硒是值得学习和使用的，如果需要的话。但是，这里不需要它。sports-reference.com网站确实在评论中存储了一些表格。因此，虽然它看起来不是静态的，但实际上是，只需要拉出注释。beautfulsoup可以做什么：

import requests
from bs4 import BeautifulSoup, Comment
import pandas as pd

def loopForData():
    headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36'}
    currentYear = 1994
    for i in range(1, 2):
        currentYear += 1
        nfl_url = "https://www.pro-football-reference.com/years/"+str(currentYear)+"/index.htm"
        result = requests.get(nfl_url, headers=headers).text
        data = BeautifulSoup(result, 'html.parser')

        comments = data.find_all(string=lambda text: isinstance(text, Comment))
        
        tables = []
        for each in comments:
            if 'table' in str(each):
                try:
                    tables.append(pd.read_html(str(each), attrs = {'id': 'playoff_results'})[0])
                    break
                except:
                    continue
        print(tables[0].to_string())

loopForData()
输出：

         Week  Day         Date           Winner/tie Unnamed: 4            Loser/tie Unnamed: 6  PtsW  PtsL
0    WildCard  Sat  December 30        Buffalo Bills        NaN       Miami Dolphins   boxscore    37    22
1    WildCard  Sat  December 30  Philadelphia Eagles        NaN        Detroit Lions   boxscore    58    37
2    WildCard  Sun  December 31   Indianapolis Colts          @   San Diego Chargers   boxscore    35    20
3    WildCard  Sun  December 31    Green Bay Packers        NaN      Atlanta Falcons   boxscore    37    20
4    Division  Sat    January 6  Pittsburgh Steelers        NaN        Buffalo Bills   boxscore    40    21
5    Division  Sat    January 6    Green Bay Packers          @  San Francisco 49ers   boxscore    27    17
6    Division  Sun    January 7       Dallas Cowboys        NaN  Philadelphia Eagles   boxscore    30    11
7    Division  Sun    January 7   Indianapolis Colts          @   Kansas City Chiefs   boxscore    10     7
8   ConfChamp  Sun   January 14       Dallas Cowboys        NaN    Green Bay Packers   boxscore    38    27
9   ConfChamp  Sun   January 14  Pittsburgh Steelers        NaN   Indianapolis Colts   boxscore    20    16
10  SuperBowl  Sun   January 28       Dallas Cowboys          N  Pittsburgh Steelers   boxscore    27    17
硒是值得学习和使用的，如果需要的话。但是，这里不需要它。sports-reference.com网站确实在评论中存储了一些表格。因此，虽然它看起来不是静态的，但实际上是，只需要拉出注释。beautfulsoup可以做什么：

import requests
from bs4 import BeautifulSoup, Comment
import pandas as pd

def loopForData():
    headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36'}
    currentYear = 1994
    for i in range(1, 2):
        currentYear += 1
        nfl_url = "https://www.pro-football-reference.com/years/"+str(currentYear)+"/index.htm"
        result = requests.get(nfl_url, headers=headers).text
        data = BeautifulSoup(result, 'html.parser')

        comments = data.find_all(string=lambda text: isinstance(text, Comment))
        
        tables = []
        for each in comments:
            if 'table' in str(each):
                try:
                    tables.append(pd.read_html(str(each), attrs = {'id': 'playoff_results'})[0])
                    break
                except:
                    continue
        print(tables[0].to_string())

loopForData()
输出：

         Week  Day         Date           Winner/tie Unnamed: 4            Loser/tie Unnamed: 6  PtsW  PtsL
0    WildCard  Sat  December 30        Buffalo Bills        NaN       Miami Dolphins   boxscore    37    22
1    WildCard  Sat  December 30  Philadelphia Eagles        NaN        Detroit Lions   boxscore    58    37
2    WildCard  Sun  December 31   Indianapolis Colts          @   San Diego Chargers   boxscore    35    20
3    WildCard  Sun  December 31    Green Bay Packers        NaN      Atlanta Falcons   boxscore    37    20
4    Division  Sat    January 6  Pittsburgh Steelers        NaN        Buffalo Bills   boxscore    40    21
5    Division  Sat    January 6    Green Bay Packers          @  San Francisco 49ers   boxscore    27    17
6    Division  Sun    January 7       Dallas Cowboys        NaN  Philadelphia Eagles   boxscore    30    11
7    Division  Sun    January 7   Indianapolis Colts          @   Kansas City Chiefs   boxscore    10     7
8   ConfChamp  Sun   January 14       Dallas Cowboys        NaN    Green Bay Packers   boxscore    38    27
9   ConfChamp  Sun   January 14  Pittsburgh Steelers        NaN   Indianapolis Colts   boxscore    20    16
10  SuperBowl  Sun   January 28       Dallas Cowboys          N  Pittsburgh Steelers   boxscore    27    17
谢谢你Rustem，我以前从未用过硒，但我将来会去学的。谢谢你，Rustem，我以前从未用过硒，但我以后会去学的。泰