Python 从ajax加载的内容中提取链接
我试图做的是测试从Ajax加载的内容中提取链接,我的代码对我来说似乎没问题,但它没有提取。我使用的是Selenium,我知道在使用Ajax时这是必须的 代码:Python 从ajax加载的内容中提取链接,python,selenium,Python,Selenium,我试图做的是测试从Ajax加载的内容中提取链接,我的代码对我来说似乎没问题,但它没有提取。我使用的是Selenium,我知道在使用Ajax时这是必须的 代码: #!/usr/bin/env python import random import re import requests import time import uuid from selenium import webdriver from selenium.webdriver.chrome.options import Option
#!/usr/bin/env python
import random
import re
import requests
import time
import uuid
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup
# function: init driver.
def init_driver(using_linux, proxy):
options = Options()
options.headless = False
options.add_argument('start-maximized')
options.add_argument('--disable-popup-blocking')
options.add_argument('--disable-notifications')
options.add_argument('--disable-extensions')
options.add_argument('--log-level=3')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
options.add_experimental_option("excludeSwitches", ["enable-automation", "load-extension"])
prefs = {'profile.default_content_setting_values.notifications': 2}
options.add_experimental_option('prefs', prefs)
if not proxy == "0.0.0.0:0":
print("--> PROXY DISABLED ...")
else:
print("--> USING PROXY: " + str(proxy) + " ...")
options.add_argument('--proxy-server=%s' % proxy)
if using_linux:
return webdriver.Chrome("/usr/bin/chromedriver", options=options)
else:
return webdriver.Chrome("chromedriver.exe", options=options)
# get the drive init in a var
driver = init_driver(False, "0.0.0.0:00")
# function: goto and scrape
def goto_and_scrape(driver, page):
try:
driver.get(page)
elems = driver.find_elements_by_xpath("//a[@href]")
for elem in elems:
print(elem.get_attribute("href"))
time.sleep(50)
except Exception as e:
print(e)
goto_and_scrape(driver, 'https://backlinkshitter.com/?url=https%3A%2F%2Fwww.videopal.io&type=domain_backlinks')
我正在寻找从页面中提取所有的href链接,目前它只提取1,任何帮助将不胜感激