Python 美化组忽略表中的嵌套表
使用BeautifulSoup for Python解析网页(不幸的是,该网页大部分是在表中编写的) 下面是我试图处理的内容的摘录Python 美化组忽略表中的嵌套表,python,html,web-scraping,beautifulsoup,html-parsing,Python,Html,Web Scraping,Beautifulsoup,Html Parsing,使用BeautifulSoup for Python解析网页(不幸的是,该网页大部分是在表中编写的) 下面是我试图处理的内容的摘录 <tr> <td colspan="4"> <div class="shortmenucats"> <span style="color: "> -- Fresh Baked Pastries -- </span> </di
<tr>
<td colspan="4">
<div class="shortmenucats">
<span style="color: ">
-- Fresh Baked Pastries --
</span>
</div>
</td>
</tr>
<tr>
<td width="80%" valign="top">
<table width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td>
<div class="shortmenurecipes">
<span style="color: #000000"> Chocolate Doughnut Holes </span>
</div>
</td>
</tr>
</tbody>
</table>
</td>
<td width="5%" valign="top"></td>
<td width="10%" valign="top" align="right" colspan="1">
<div class="shortmenuprices">
<span style="color: #000000"></span>
</div>
</td>
<td width="5%" valign="top" colspan="1">
</td>
</tr>
--新鲜出炉的糕点--
巧克力甜甜圈洞
这是一个表中的两行,其中有10行,它们是这样交替的(td中的div,td中的table,td中的div,td中的table,等等)
我使用BeautifulSoup调用父表上的find_all,它会返回每一行的重复项,因为嵌套表中有嵌套标记
我首先做了一个表。find_all('td',recursive=False)
,但它根本没有返回任何s。
如果我在父表上调用一个findChildren()
,我会得到一个包含一个结果的列表,但它包含结果中的所有子项
我做错什么了吗?我不知道怎么弄清楚
如果您想要我正在解析的实际网站,请访问:
它的编码相当混乱。我只是想分析一下
任何帮助都将不胜感激。即使这只是一种删除重复项的方法
谢谢。您可以通过目标表在HTML中的深度来识别它们 下面是一些代码,用于选择嵌套在深度3处的表:
tables = soup.findAll("table")
depth3 = []
for t in tables:
if len(t.find_parents("table")) == 3:
depth3.append(t)
对于您的页面,这将导致选择6个表格-三个表格用于标题(“早餐”、“午餐”、“晚餐”),三个表格用于菜单。它们交替-标题、菜单、标题、菜单等,因此您可以只处理位置1、3和5处的表格
现在解析应该容易多了。这里的另一个选项是依赖菜单和其中类别的类名,并且只有在转到元素的直接父级时才使用
recursive=False
完成工作代码提取菜单:
from urllib2 import urlopen
from bs4 import BeautifulSoup
url = "http://138.23.12.141/foodpro/shortmenu.asp?sName=University+of+California%2C+Riverside+Dining+Services&locationNum=02&locationName=Lothian+Residential+Restaurant&naFlag=1"
soup = BeautifulSoup(urlopen(url))
container = soup.find('div', class_='shortmenutitle').find_next_sibling('table').tr
for td in container.find_all('td', recursive=False):
title = td.find('div', class_='shortmenumeals')
print title.text.strip()
for item in td.table.find_all('tr', recursive=False)[1].table.find_all('tr', recursive=False):
print item.text.strip()
print "-----"
print
打印完整的菜单(无重复项):
那么你到底想提取哪些部分呢?
span
s中的文本?或者,还有什么?这很有效!谢谢你哦。令人惊叹的!比我的代码更短更可读。这是我第一次使用BeautifulSoup,因此我非常感谢您的帮助!
Breakfast
-- Fresh Baked Pastries --
Chocolate Doughnut Holes
Double Chocolate Mini Muffin
Fresh Baked Blueberry Bagel
Fresh Baked Plain Bagel
-- Breakfast Parfait Bar --
(V,GF) Breakfast Parfait Bar
-- Hot Cereal & Toppings --
(V) Cream of Wheat
-- Breakfast Offerings --
(V) Belgium Waffle Bar with Condiments
(V) Eggs Rancheros
(V) Vanilla Scented French Toast
(V)Hash Browns
(V.GF) Scrambled Eggs
Corned Beef Hash
Turkey Sausage Patty
-- Omelet Bar --
(V,GF) Omelet Bar (Egg Whites Available Upon Request)
-----
Lunch
-- Soup & Deli Bar --
(V) Broccoli Cheese
Artisian Bread Bar
Chicken Tortilla
N Y Style Deli Bar
-- Global Sizzle --
(V) Jasmine Rice
(V) Steamed Sugar Snap Peas
(V) Thai Vegetable Spring Roll
Red Thai Curry Chicken (contains peanuts)
Sweet Thai Chili & Plum Dipping Sauces
Thai Curry Shrimp w/ Green Pepper
-- Urban Kitchen --
(V) Peruvian Beans
Peruvian Rotissere Chicken
-- The Grill --
(GF) Marinated Grilled Chicken Breast
(V) Skinny Fries
Turkey Club Melt
-- Healthy Vegetarian Bar --
(V) Southwestern Corn Salad
Southwest Soy Beef Wrap
-- Desserts --
Bakery Parfait Bar
Chocolate 1/2 Sheet Cake
Chocolate Mousse
Sugar Free Strawberry Orange Jell-O Gems
-- Continous Service 2pm-4:30pm --
(V) Vietnamese Tofu Spring Roll w/Sauces
Rotini w/Chicken Tomato Cream Spinach
Rotini w/Tomato Cream Spinach
Vietnamese Spring Roll with Tofu
-----
Dinner
-- Soup & Deli Bar --
(V) Broccoli Cheese
Artisian Bread Bar
Chicken Tortilla
N Y Style Deli Bar
-- Spinellis Pizza --
(V)French Bread Three Cheese Pizza
French Bread Pepperoni Pizza
-- Global Sizzle --
Beef Pho Bo Bar
Vegetable Pho Chay Bar
-- Urban Kitchen --
Mashed Sweet Potatoes
Rotissere Porkloin w/Apricot Demi Glace
-- The Grill --
(GF) Marinated Grilled Chicken Breast
(V) Seasoned Wedge Fries
Chicken Tenders
-- Healthy Vegetarian Bar --
(V) Cut Corn
(V) Steamed Broccoli
(Vgn) Black Bean Tostadas
-- Desserts --
Bakery Parfait Bar
Chocolate 1/2 Sheet Cake
Chocolate Mousse
M & M Rice Krispy Treats
Oreo Cheesecake
Peach Maple Cobbler
Sugar Free Strawberry Orange Jell-O Gems
-----