Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/selenium/4.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 打开每个谷歌搜索结果的链接_Python_Selenium_Selenium Webdriver_Web Scraping - Fatal编程技术网

Python 打开每个谷歌搜索结果的链接

Python 打开每个谷歌搜索结果的链接,python,selenium,selenium-webdriver,web-scraping,Python,Selenium,Selenium Webdriver,Web Scraping,我制作了一个网页刮板,可以打印多个谷歌搜索页面(即Python-Wikipedia)的搜索结果标题。我想能够打开一个新的标签页的所有搜索结果,而它打印的结果,并通过每一页 您应该使用BeautifulSoup和webbrowser模块进行此操作。是否可以同时集成selenium和BS,或者我必须使用其中一个模块或另一个模块?是否确实要在firefox浏览器中打开页面?不一定要在firefox浏览器中打开它,但我确实希望从页面中检索特定信息,无论哪种方式。BeautifulSoup最适合此功能您应

我制作了一个网页刮板,可以打印多个谷歌搜索页面(即Python-Wikipedia)的搜索结果标题。我想能够打开一个新的标签页的所有搜索结果,而它打印的结果,并通过每一页

您应该使用
BeautifulSoup
webbrowser
模块进行此操作。是否可以同时集成selenium和BS,或者我必须使用其中一个模块或另一个模块?是否确实要在firefox浏览器中打开页面?不一定要在firefox浏览器中打开它,但我确实希望从页面中检索特定信息,无论哪种方式。BeautifulSoup最适合此功能您应该为此使用
BeautifulSoup
webbrowser
模块,是否可以同时集成selenium和BS,或者我必须使用其中一种或另一种方式?是否确实要在firefox浏览器中打开该页面?不一定要在firefox浏览器中打开它,但我确实希望从页面中检索特定信息,无论哪种方式。BeautifulSoup最适合于此
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

#number of pages, atm i set it to cycle through 5 pages, starting page is 0
max_page = 5

#number of digits (ie: 2 is 1 digit, 10 is 2 digits, increase when appropriate)
max_dig = 1

#Open up firefox browser
driver = webdriver.Firefox()

#search results "acoustics+companies+in+us"
question = input("\nWhat would you like to google today, but replace every space with a '+' (ie: am+i+a+good+carry)\n\n")

search = []    

#get multiple pages
for i in range(0, max_page + 1):
    page_num = (max_dig - len(str(i))) * "0" + str(i)
    #inserts search input and cycles through pages
    url =  "https://www.google.com/search?q="+ question +"&ei=LV-uXYrpNoj0rAGC8KSYCg&start="+ page_num +"0&sa=N&ved=0ahUKEwjKs8ie367lAhUIOisKHQI4CaM4ChDy0wMIiQE&biw=1356&bih=946"
    driver.get(url)
    WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "LC20lb")))
    elements = driver.find_elements_by_class_name('LC20lb')
    for element in elements:
        search.append(element.text)

    #finds the links of the google search, holds down command and clicks the link to open the links in a new tab
    links = driver.find_element_by_xpath('/html/body/div[6]/div[3]/div[10]/div[1]/div[2]/div/div[2]/div[2]/div/div/div/div/div[1]/div/div/div[1]')
    ActionChains(driver)
    .key_down(Keys.COMMAND)
    .click(links)
    .perform()

for item in search:
    print(item)