Python ';列表索引超出范围';刮表时出现问题
我正在尝试从维基百科页面中删除表格 我得到了html,找到了包含我想要的表的部分:Python ';列表索引超出范围';刮表时出现问题,python,web-scraping,beautifulsoup,Python,Web Scraping,Beautifulsoup,我正在尝试从维基百科页面中删除表格 我得到了html,找到了包含我想要的表的部分: <table class="wikitable sortable"> <tbody> <tr> <th>Postcode</th> <th>Borough</th> <th>Neighbourhood</th> </tr> <tr> <td&
<table class="wikitable sortable">
<tbody>
<tr>
<th>Postcode</th>
<th>Borough</th>
<th>Neighbourhood</th>
</tr>
<tr>
<td>M1A</td>
<td>Not assigned</td>
<td>Not assigned</td>
</tr>
<tr>
<td>M2A</td>
<td>Not assigned</td>
<td>Not assigned</td>
</tr>
<tr>
<td>M3A</td>
<td><a href="/wiki/North_York" title="North York">North York</a></td>
<td><a href="/wiki/Parkwoods" title="Parkwoods">Parkwoods</a></td>
</tr>
<tr>
<td>M4A</td>
<td><a href="/wiki/North_York" title="North York">North York</a></td>
<td><a href="/wiki/Victoria_Village" title="Victoria Village">Victoria Village</a></td>
</tr>
<tr>
<td>M5A</td>
<td><a href="/wiki/Downtown_Toronto" title="Downtown Toronto">Downtown Toronto</a></td>
<td><a href="/wiki/Harbourfront_(Toronto)" title="Harbourfront (Toronto)">Harbourfront</a></td>
</tr>
<tr>
.
.
.
输出是我所期望的:
['M1A', 'M2A', 'M3A', 'M4A', 'M5A', 'M5A', 'M6A', 'M6A', 'M7A', 'M8A', 'M9A', 'M1B', 'M1B', 'M2B', 'M3B', 'M4B', 'M4B', 'M5B', 'M5B', 'M6B', ...
然而,当我想为自治区和邻里区做同样的事情时,我总是得到“列表索引超出范围”
以下是我用于自治区和居民区的代码:
Borough=[]
for row in My_table.findAll('td') :
Borough_cell=row.findAll('a')[0]
Borough.append(Borough_cell.text)
print(Borough)
我确实注意到,在自治区和社区中存在“未分配”的值(没有“a”),我不知道是不是这些值导致了问题。
我的预期结果是将表转换为pandas格式以便进一步处理。是的,原因是某些邮政编码有链接,而其他邮政编码没有链接 一种更简单的方法是抓取自治区/邻里区的
td
,将文本放在里面:
Borough=[]
for row in My_table.findAll('tr')[1:]:
Borough_cell=row.findAll('td')[1]
Borough.append(Borough_cell.text)
print(Borough)
就像你对邮政编码所做的一样
请注意,现在您可以将代码概括为一个for循环:
postcodes = []
boroughs = []
neighbourhoods = []
for row in My_table.findAll('tr')[1:]:
postcodecell, boroughcell, neighbourcell = row.findAll('td')
postcodes.append(postcodecell.text)
boroughs.append(boroughcell.text)
neighbourhoods.append(neighbourcell.text)
此外,如果您的项目涉及大量Wikipedia页面和大量的抓取/解析,那么我强烈建议您使用两种资源:
是的,原因是一些邮政编码有链接,而另一些没有 一种更简单的方法是抓取自治区/邻里区的
td
,将文本放在里面:
Borough=[]
for row in My_table.findAll('tr')[1:]:
Borough_cell=row.findAll('td')[1]
Borough.append(Borough_cell.text)
print(Borough)
就像你对邮政编码所做的一样
请注意,现在您可以将代码概括为一个for循环:
postcodes = []
boroughs = []
neighbourhoods = []
for row in My_table.findAll('tr')[1:]:
postcodecell, boroughcell, neighbourcell = row.findAll('td')
postcodes.append(postcodecell.text)
boroughs.append(boroughcell.text)
neighbourhoods.append(neighbourcell.text)
此外,如果您的项目涉及大量Wikipedia页面和大量的抓取/解析,那么我强烈建议您使用两种资源:
Postcode Borough Neighbourhood
0 M1A Not assigned Not assigned
1 M2A Not assigned Not assigned
2 M3A North York Parkwoods
3 M4A North York Victoria Village
4 M5A Downtown Toronto Harbourfront
5 M5A Downtown Toronto Regent Park
...
[288 rows x 3 columns]
还可以使用将表直接转换为数据帧
产出:
Postcode Borough Neighbourhood
0 M1A Not assigned Not assigned
1 M2A Not assigned Not assigned
2 M3A North York Parkwoods
3 M4A North York Victoria Village
4 M5A Downtown Toronto Harbourfront
5 M5A Downtown Toronto Regent Park
...
[288 rows x 3 columns]
我照你说的做了,这对自治区很有效。但是当涉及到社区时,我提取的值都包含了结尾处的“/n”。有什么办法可以删除它吗?@Momoxuan是的,做
neightourcell.text.rstrip()
我照你说的做了,这对自治区很有效。但是,当涉及到邻里时,我提取的值在末尾包含“/n”。有办法删除它吗?@Momoxuan是的,请执行neighbourcell.text.rstrip()