Python 当HTML没有';你不会变吗?
我目前正在使用Selenium和BeautifulSoup从Google Finance中获取财务报表数据。例如: 打开谷歌的损益表。当我让Selenium点击页面顶部的“余额表”和“现金流量”按钮时,页面上的图表和表格会发生变化,但url不会发生变化,当我拉页面源时,它是带有损益表的原始页面。我的代码发布在下面:Python 当HTML没有';你不会变吗?,python,selenium,web-scraping,beautifulsoup,Python,Selenium,Web Scraping,Beautifulsoup,我目前正在使用Selenium和BeautifulSoup从Google Finance中获取财务报表数据。例如: 打开谷歌的损益表。当我让Selenium点击页面顶部的“余额表”和“现金流量”按钮时,页面上的图表和表格会发生变化,但url不会发生变化,当我拉页面源时,它是带有损益表的原始页面。我的代码发布在下面: driver = webdriver.Firefox() driver.get("http://www.google.com/finance?q=" + ticker[0] + "
driver = webdriver.Firefox()
driver.get("http://www.google.com/finance?q=" + ticker[0] + "&fstype=ii")
url1 = driver.page_source
soup1 = BeautifulSoup(url1)
element = driver.find_element_by_xpath('//*[@id=":1"]/a/b/b')
element.click()
driver.implicity_wait(3.0)
url2 = driver.page_source
soup2 = BeautifulSoup(url2)
element = driver.find_element_by_xpath('//*[@id=":2"]/a/b/b')
element.click()
driver.implicity_wait(3.0)
url3 = driver.page_source
soup3 = BeautifulSoup(url3)
driver.quit()
感谢您的帮助。谢谢 这里不需要
BeautifulSoup
HTML解析器Selenium
本身在中的功能足够强大
您需要的表数据位于具有不同id
s的div
元素中。激活每个选项卡并从相应的div获取数据
下面是一个打印所有选项卡中的表标题的示例:
from selenium import webdriver
def print_header(element):
table = element.find_element_by_id('fs-table')
for row in table.find_elements_by_tag_name('th'):
print row.text
driver = webdriver.Firefox()
driver.get('http://www.google.com/finance?q=GOOG&fstype=ii')
print_header(driver.find_element_by_id('incinterimdiv'))
print "----"
# activate Balance Sheet
element = driver.find_element_by_xpath('//*[@id=":1"]/a/b/b')
element.click()
print_header(driver.find_element_by_id('balinterimdiv'))
print "----"
# activate Cash Flow
element = driver.find_element_by_xpath('//*[@id=":2"]/a/b/b')
element.click()
print_header(driver.find_element_by_id('casinterimdiv'))
driver.quit()
印刷品:
In Millions of USD (except for per share items)
3 months ending 2014-03-31
3 months ending 2013-12-31
3 months ending 2013-09-30
3 months ending 2013-06-30
3 months ending 2013-03-31
----
In Millions of USD (except for per share items)
As of 2014-03-31
As of 2013-12-31
As of 2013-09-30
As of 2013-06-30
As of 2013-03-31
----
In Millions of USD (except for per share items)
3 months ending 2014-03-31
12 months ending 2013-12-31
9 months ending 2013-09-30
6 months ending 2013-06-30
3 months ending 2013-03-31
因此,我会在print_header函数中添加另一个for循环,该循环类似于:for col in table.find_elements_by_tag_name('td'):然后将结果保存在python对象中?@user2395969您可以在
table
中找到元素,每个tr
等-取决于您想要的输出。这里的要点是只使用selenium
。希望能有帮助。是的,我知道了。谢谢你的帮助!