Python 打开每个谷歌搜索结果的链接
我制作了一个网页刮板,可以打印多个谷歌搜索页面(即Python-Wikipedia)的搜索结果标题。我想能够打开一个新的标签页的所有搜索结果,而它打印的结果,并通过每一页 您应该使用Python 打开每个谷歌搜索结果的链接,python,selenium,selenium-webdriver,web-scraping,Python,Selenium,Selenium Webdriver,Web Scraping,我制作了一个网页刮板,可以打印多个谷歌搜索页面(即Python-Wikipedia)的搜索结果标题。我想能够打开一个新的标签页的所有搜索结果,而它打印的结果,并通过每一页 您应该使用BeautifulSoup和webbrowser模块进行此操作。是否可以同时集成selenium和BS,或者我必须使用其中一个模块或另一个模块?是否确实要在firefox浏览器中打开页面?不一定要在firefox浏览器中打开它,但我确实希望从页面中检索特定信息,无论哪种方式。BeautifulSoup最适合此功能您应
BeautifulSoup
和webbrowser
模块进行此操作。是否可以同时集成selenium和BS,或者我必须使用其中一个模块或另一个模块?是否确实要在firefox浏览器中打开页面?不一定要在firefox浏览器中打开它,但我确实希望从页面中检索特定信息,无论哪种方式。BeautifulSoup最适合此功能您应该为此使用BeautifulSoup
和webbrowser
模块,是否可以同时集成selenium和BS,或者我必须使用其中一种或另一种方式?是否确实要在firefox浏览器中打开该页面?不一定要在firefox浏览器中打开它,但我确实希望从页面中检索特定信息,无论哪种方式。BeautifulSoup最适合于此
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
#number of pages, atm i set it to cycle through 5 pages, starting page is 0
max_page = 5
#number of digits (ie: 2 is 1 digit, 10 is 2 digits, increase when appropriate)
max_dig = 1
#Open up firefox browser
driver = webdriver.Firefox()
#search results "acoustics+companies+in+us"
question = input("\nWhat would you like to google today, but replace every space with a '+' (ie: am+i+a+good+carry)\n\n")
search = []
#get multiple pages
for i in range(0, max_page + 1):
page_num = (max_dig - len(str(i))) * "0" + str(i)
#inserts search input and cycles through pages
url = "https://www.google.com/search?q="+ question +"&ei=LV-uXYrpNoj0rAGC8KSYCg&start="+ page_num +"0&sa=N&ved=0ahUKEwjKs8ie367lAhUIOisKHQI4CaM4ChDy0wMIiQE&biw=1356&bih=946"
driver.get(url)
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "LC20lb")))
elements = driver.find_elements_by_class_name('LC20lb')
for element in elements:
search.append(element.text)
#finds the links of the google search, holds down command and clicks the link to open the links in a new tab
links = driver.find_element_by_xpath('/html/body/div[6]/div[3]/div[10]/div[1]/div[2]/div/div[2]/div[2]/div/div/div/div/div[1]/div/div/div[1]')
ActionChains(driver)
.key_down(Keys.COMMAND)
.click(links)
.perform()
for item in search:
print(item)