Python 美丽的汤无法从表中获取信息
进入python几个月后,我在使用BeautifulSoup从表中抓取一些信息时遇到了困难,如有任何帮助,将不胜感激。我没有得到任何错误代码,只是没有从表中接收任何数据Python 美丽的汤无法从表中获取信息,python,python-3.x,beautifulsoup,Python,Python 3.x,Beautifulsoup,进入python几个月后,我在使用BeautifulSoup从表中抓取一些信息时遇到了困难,如有任何帮助,将不胜感激。我没有得到任何错误代码,只是没有从表中接收任何数据 import bs4 as bs import requests resp = requests.get('https://www.thestreet.com/markets/gainers.html') soup = bs.BeautifulSoup(resp.text, "lxml") table = soup.find(
import bs4 as bs
import requests
resp = requests.get('https://www.thestreet.com/markets/gainers.html')
soup = bs.BeautifulSoup(resp.text, "lxml")
table = soup.find('table', {'id': 'nyseData'})
tickers = []
for row in table.findAll('tr')[1:]:
ticker = row.findAll('td')[1].text
tickers.append(ticker)
非常感谢您的帮助 您遇到了页面不允许某些用户代理访问其站点的问题。这可以通过在
请求
标题中设置用户代理字符串来解决
添加了用户代理的代码:
import bs4 as bs
import requests
headers = {'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'}
resp = requests.get('https://www.thestreet.com/markets/gainers.html', headers=headers)
soup = bs.BeautifulSoup(resp.text,'lxml')
table = soup.find('table', {'id': 'nyseData'})
tickers = []
for row in table.findAll('tr')[1:]:
ticker = row.findAll('td')[1].text
tickers.append(ticker)
print tickers
输出:
[u'QUOT', u'BCEI', u'ATEN', u'SKX', u'FBK', u'FBM', u'CGI', u'SDRL', u'ELLI', u'CELP', u'SXCP', u'CUB', u'GLF', u'SID', u'HBM', u'NE', u'CBG', u'PJT', u'VVI', u'ARL']