Python BeautifulSoup-获取HTML免费内容的简单方法

我使用以下代码查找页面中所有有趣的链接：

soup.findAll('a', href=re.compile('^notizia.php\?idn=\d+'))

它的工作做得很好。不幸的是，在a标记中有许多嵌套的标记，如font、b和不同的东西。。。我只想得到文本内容，不需要任何其他html标记

链接示例：

<A HREF="notizia.php?idn=1134" OnMouseOver="verde();" OnMouseOut="blu();"><FONT CLASS="v12"><B>03-11-2009:&nbsp;&nbsp;<font color=green>CCS Ingegneria Elettronica-Sportello studenti ed orientamento</B></FONT></A>

在文档中，它说在findAll方法中使用

text=True

，但它将忽略我的正则表达式。为什么？我该如何解决这个问题呢？

我用过这个：

def textOf(soup):
    return u''.join(soup.findAll(text=True))

所以

---

对这个问题有兴趣吗

from pyparsing import makeHTMLTags, SkipTo, anyOpenTag, anyCloseTag, ParseException

htmlsrc = """<A HREF="notizia.php?idn=1134" OnMouseOver="verde();" OnMouseOut="blu();"><FONT CLASS="v12"><B>03-11-2009:&nbsp;&nbsp;<font color=green>CCS Ingegneria Elettronica-Sportello studenti ed orientamento</B></FONT></A>"""

# create pattern to find interesting <A> tags
aStart,aEnd = makeHTMLTags("A")
def matchInterestingHrefsOnly(t):
    if not t.href.startswith("notizia.php?"):
        raise ParseException("not interested...")
aStart.setParseAction(matchInterestingHrefsOnly)
patt = aStart + SkipTo(aEnd)("body") + aEnd

# create pattern to strip HTML tags, and convert HTML entities
stripper = anyOpenTag.suppress() | anyCloseTag.suppress()
def stripTags(s):
    s = stripper.transformString(s)
    s = s.replace("&nbsp;"," ")
    return s


for match in patt.searchString(htmlsrc):
    print stripTags(match.body)

---

这实际上对HTML的变化无常很不敏感，因为它考虑了属性的存在/不存在、大小写等等。

PyQuery听起来像是一个非常酷的选择：我认为你需要一个循环——你不能对结果集调用

findAll

，只能对单个结果调用。

texts = [textOf(n) for n in soup.findAll('a', href=re.compile('^notizia.php\?idn=\d+'))]

from pyparsing import makeHTMLTags, SkipTo, anyOpenTag, anyCloseTag, ParseException

htmlsrc = """<A HREF="notizia.php?idn=1134" OnMouseOver="verde();" OnMouseOut="blu();"><FONT CLASS="v12"><B>03-11-2009:&nbsp;&nbsp;<font color=green>CCS Ingegneria Elettronica-Sportello studenti ed orientamento</B></FONT></A>"""

# create pattern to find interesting <A> tags
aStart,aEnd = makeHTMLTags("A")
def matchInterestingHrefsOnly(t):
    if not t.href.startswith("notizia.php?"):
        raise ParseException("not interested...")
aStart.setParseAction(matchInterestingHrefsOnly)
patt = aStart + SkipTo(aEnd)("body") + aEnd

# create pattern to strip HTML tags, and convert HTML entities
stripper = anyOpenTag.suppress() | anyCloseTag.suppress()
def stripTags(s):
    s = stripper.transformString(s)
    s = s.replace("&nbsp;"," ")
    return s


for match in patt.searchString(htmlsrc):
    print stripTags(match.body)

03-11-2009:  CCS Ingegneria Elettronica-Sportello studenti ed orientamento