Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/selenium/4.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 从ajax加载的内容中提取链接_Python_Selenium - Fatal编程技术网

Python 从ajax加载的内容中提取链接

Python 从ajax加载的内容中提取链接,python,selenium,Python,Selenium,我试图做的是测试从Ajax加载的内容中提取链接,我的代码对我来说似乎没问题,但它没有提取。我使用的是Selenium,我知道在使用Ajax时这是必须的 代码: #!/usr/bin/env python import random import re import requests import time import uuid from selenium import webdriver from selenium.webdriver.chrome.options import Option

我试图做的是测试从Ajax加载的内容中提取链接,我的代码对我来说似乎没问题,但它没有提取。我使用的是Selenium,我知道在使用Ajax时这是必须的

代码:

#!/usr/bin/env python
import random
import re
import requests
import time
import uuid

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup

# function: init driver.
def init_driver(using_linux, proxy):
    options = Options()
    options.headless = False
    options.add_argument('start-maximized')
    options.add_argument('--disable-popup-blocking')
    options.add_argument('--disable-notifications')
    options.add_argument('--disable-extensions')
    options.add_argument('--log-level=3')
    options.add_argument('--ignore-certificate-errors')
    options.add_argument('--ignore-ssl-errors')
    options.add_experimental_option("excludeSwitches", ["enable-automation", "load-extension"])
    prefs = {'profile.default_content_setting_values.notifications': 2}
    options.add_experimental_option('prefs', prefs)
    if not proxy == "0.0.0.0:0":
        print("--> PROXY DISABLED ...")
    else:
        print("--> USING PROXY: " + str(proxy) + " ...")
        options.add_argument('--proxy-server=%s' % proxy)
    if using_linux:
        return webdriver.Chrome("/usr/bin/chromedriver", options=options)
    else:
        return webdriver.Chrome("chromedriver.exe", options=options)


# get the drive init in a var
driver = init_driver(False, "0.0.0.0:00")

# function: goto and scrape
def goto_and_scrape(driver, page):
    try:
        driver.get(page)
        elems = driver.find_elements_by_xpath("//a[@href]")
        for elem in elems:
            print(elem.get_attribute("href"))
        time.sleep(50)
    except Exception as e:
        print(e)

goto_and_scrape(driver, 'https://backlinkshitter.com/?url=https%3A%2F%2Fwww.videopal.io&type=domain_backlinks')
我正在寻找从页面中提取所有的href链接,目前它只提取1,任何帮助将不胜感激