Warning: file_get_contents(/data/phpspider/zhask/data//catemap/7/python-2.7/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 2.7 用bs4(靓汤)python 2.7发布刮削网站_Python 2.7_Web Scraping_Beautifulsoup - Fatal编程技术网

Python 2.7 用bs4(靓汤)python 2.7发布刮削网站

Python 2.7 用bs4(靓汤)python 2.7发布刮削网站,python-2.7,web-scraping,beautifulsoup,Python 2.7,Web Scraping,Beautifulsoup,我试图完成的是一个简单的pythonweb抓取脚本,用于googletrends,在抓取类时遇到了一个问题 from bs4 import BeautifulSoup import requests results = requests.get("https://trends.google.com/trends/trendingsearches/daily?geo=US") soup = BeautifulSoup(results.text, 'lxml') keyword_list = s

我试图完成的是一个简单的pythonweb抓取脚本,用于googletrends,在抓取类时遇到了一个问题

from bs4 import BeautifulSoup
import requests


results = requests.get("https://trends.google.com/trends/trendingsearches/daily?geo=US")
soup = BeautifulSoup(results.text, 'lxml')
keyword_list = soup.find_all('.details-top')
for keyword in keyword_list:
    print(keyword)
打印标记时我接收并清空类,但打印soup时,我接收整个HTML文档。我的目标是打印出搜索页面的每个“关键字”的文本

这有一个结果列表:

1. covid-19
2.Woolworths jobs
如果使用google开发者选项,请选择inspect并将鼠标悬停在标题上方,您将看到div.details-top


如何打印每个标题的文本?我可以在dev tools network选项卡中看到从API调用动态检索的数据。您可以向该url发出xhr,然后在响应文本上使用正则表达式解析查询标题

import requests, re
from bs4 import BeautifulSoup as bs

r = requests.get('https://trends.google.com/trends/api/dailytrends?hl=en-GB&tz=0&geo=AU&ns=15').text
p = re.compile(r'"query":"(.*?)"')
titles = p.findall(r)
print(titles) # 2.7 use print titles

我想var p的查询完成了我所需要的一切,非常感谢。我甚至不需要漂亮的汤*edit:lol我在Linux上安装了它,这就是我不需要导入的原因