Python 3.x 从维基百科获取给定类别的所有页面

Python 3.x 从维基百科获取给定类别的所有页面,python-3.x,web-scraping,web-crawler,wikipedia-api,Python 3.x,Web Scraping,Web Crawler,Wikipedia Api,我正在使用从维基百科给定类别的页面中提取所有文本 如本教程所示- def print_categorymembers(categorymembers, level=0, max_level=2): for c in categorymembers.values(): print("%s: %s (ns: %d)" % ("*" * (level + 1), c.title, c.ns)) if c.ns == wikipediaapi.Namespace.

我正在使用从维基百科给定类别的页面中提取所有文本

如本教程所示-

def print_categorymembers(categorymembers, level=0, max_level=2):
    for c in categorymembers.values():
        print("%s: %s (ns: %d)" % ("*" * (level + 1), c.title, c.ns))
        if c.ns == wikipediaapi.Namespace.CATEGORY and level <= max_level:
            print_categorymembers(c.categorymembers, level + 1)


cat = wiki_wiki.page("Category:Physics")
print("Category members: Category:Physics")
print_categorymembers(cat.categorymembers  
def print_categorymembers(categorymembers,level=0,max_level=2):
对于categorymembers.values()中的c:
打印(“%s:%s(ns:%d)”%(“*”*(级别+1),c.title,c.ns))

如果c.ns==wikipediaapi.Namespace.CATEGORY and level如果您想从页面中提取文本,则必须使用

因此,您的代码可以如下所示:

cat = wiki_wiki.page("Category:Physics")
print("Category members: Category:Physics")
for p in cat.categorymembers.values():
  if p.namespace == wikipediaapi.Namespace.CATEGORY:
    # it is category, so you have to make decision
    # if you want to fetch also text from pages that belong
    # to this category
    print(p)
  elif p.namespace == wikipediaapi.Namespace.MAIN:
    # it is page => we can get text
    print(p)
    print(p.text)

如果你想从页面中提取文本,你必须使用

因此,您的代码可以如下所示:

cat = wiki_wiki.page("Category:Physics")
print("Category members: Category:Physics")
for p in cat.categorymembers.values():
  if p.namespace == wikipediaapi.Namespace.CATEGORY:
    # it is category, so you have to make decision
    # if you want to fetch also text from pages that belong
    # to this category
    print(p)
  elif p.namespace == wikipediaapi.Namespace.MAIN:
    # it is page => we can get text
    print(p)
    print(p.text)