如何通过selenium和python找到twitch上视频的href属性？
我正在尝试为特定用户查找所有视频的twitch视频ID。比如在这一页上 所以这里我们链接了所有的视频，但它并没有简单到只抓取html并找到链接，因为它们似乎是动态生成的 所以我听说了硒，并做了类似的事情：如何通过selenium和python找到twitch上视频的href属性？,python,selenium,selenium-webdriver,css-selectors,webdriverwait,Python,Selenium,Selenium Webdriver,Css Selectors,Webdriverwait,我正在尝试为特定用户查找所有视频的twitch视频ID。比如在这一页上 所以这里我们链接了所有的视频，但它并没有简单到只抓取html并找到链接，因为它们似乎是动态生成的 所以我听说了硒，并做了类似的事情： from selenium import webdriver # Change path here obviously driver = webdriver.Chrome('C:/Users/Jason/Downloads/chromedriver') driver.get('https
from selenium import webdriver
# Change path here obviously
driver = webdriver.Chrome('C:/Users/Jason/Downloads/chromedriver')
driver.get('https://www.twitch.tv/dyrus/videos/all')
link_element = driver.find_elements_by_xpath("//*[@href]")
for link in link_element:
print(link.get_attribute('href'))
driver.close()
提前感谢您使用定位器，您将返回页面上包含
hreffrom selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Change path here obviously
driver = webdriver.Chrome('C:/Users/Jason/Downloads/chromedriver')
driver.get('https://www.twitch.tv/dyrus/videos/all')
links = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a[data-a-target='preview-card-image-link']")))
for link in links:
print(link.get_attribute('href'))
driver.close()
这将打印出页面上的40个链接。我仍然建议进行以下几项更改：
- 始终以最大化模式打开Web浏览器，以便所有/大部分所需元素都在浏览器中
- 如果您使用的是Windows操作系统，则需要将扩展名
附加到WebDriver变体名称的末尾，例如chromedriver.exe.exe - 在标识元素时，请始终尝试在定位器策略中包含
属性class - 始终在
结束时调用@Test
，优雅地关闭并销毁WebDriver和Web客户端实例driver.quit（） - 以下是您自己的代码块，包含上述调整：
from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC options = Options() options.add_argument("start-maximized") options.add_argument("disable-infobars") driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\path\to\chromedriver.exe') driver.get('https://www.twitch.tv/dyrus/videos/all') link_elements = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.tw-interactive.tw-link[data-a-target='preview-card-image-link']"))) for link in link_elements: print(link.get_attribute('href')) driver.quit() - 控制台输出：
https://www.twitch.tv/videos/295314690 https://www.twitch.tv/videos/294901947 https://www.twitch.tv/videos/294472813 https://www.twitch.tv/videos/294075254 https://www.twitch.tv/videos/293617036 https://www.twitch.tv/videos/293236560 https://www.twitch.tv/videos/292800601 https://www.twitch.tv/videos/292409437 https://www.twitch.tv/videos/292328170 https://www.twitch.tv/videos/292032996 https://www.twitch.tv/videos/291625563 https://www.twitch.tv/videos/291192151 https://www.twitch.tv/videos/290824842 https://www.twitch.tv/videos/290434348 https://www.twitch.tv/videos/290021370 https://www.twitch.tv/videos/289561690 https://www.twitch.tv/videos/289495488 https://www.twitch.tv/videos/289138003 https://www.twitch.tv/videos/289110429 https://www.twitch.tv/videos/288804893 https://www.twitch.tv/videos/288784992 https://www.twitch.tv/videos/288687479 https://www.twitch.tv/videos/288432438 https://www.twitch.tv/videos/288117849 https://www.twitch.tv/videos/288004968 https://www.twitch.tv/videos/287689102 https://www.twitch.tv/videos/287451192 https://www.twitch.tv/videos/287267032 https://www.twitch.tv/videos/287017431 https://www.twitch.tv/videos/286819343