在Selenium/Python中从Amazon产品页面的图像中提取所有src属性
我正在使用Selenium从Amazon产品页面中获取详细信息([示例][1])。我已经成功地抓取了产品标题,但我还想获得所有产品图片的URL。这是我的密码:在Selenium/Python中从Amazon产品页面的图像中提取所有src属性,python,selenium,xpath,css-selectors,webdriverwait,Python,Selenium,Xpath,Css Selectors,Webdriverwait,我正在使用Selenium从Amazon产品页面中获取详细信息([示例][1])。我已经成功地抓取了产品标题,但我还想获得所有产品图片的URL。这是我的密码: from selenium import webdriver from webdriver_manager.chrome import ChromeDriverManager def search_amazon(): driver = webdriver.Chrome(ChromeDriverManager().install(
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
def search_amazon():
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('https://www.amazon.com/Pendleton-Glacier-National-Queen-Blanket/dp/B003EQ4AYY/?_encoding=UTF8&pd_rd_w=dZURJ&pf_rd_p=ab102187-3a5a-49fd-b43f-4f928775aeae&pf_rd_r=PD8YGV8XA34FMYH7G9TJ&pd_rd_r=2cb55e9c-812a-43de-bf52-7e1976f5374b&pd_rd_wg=KmkoW&ref_=pd_gw_hfp13n_bbn')
productName = driver.find_element_by_id('productTitle').text
print(productName)
imgList = driver.find_element_by_xpath('//*[@id="altImages"]/ul')
options = imgList.find_elements_by_tag_name("li")
for option in options:
print(option.get_attribute("innerHTML"))
search_amazon()
末尾的options循环返回每个LI的innerHTML。但我无法访问IMG src,我尝试的是:
for option in options:
src = option.find_element_by_tag_name("img").get_attribute("src")
这会引发NoTouchElementException:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"img"}
在为每个图像查找
li
元素时,应在路径中指定元素的类别,因为并非元素/*[@id=“altImages”]/ul
的每个li
都引用图像。因此,为了找到URL,您可以这样做:
def search_amazon():
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('https://www.amazon.com/Pendleton-Glacier-National-Queen-Blanket/dp/B003EQ4AYY/?_encoding=UTF8&pd_rd_w=dZURJ&pf_rd_p=ab102187-3a5a-49fd-b43f-4f928775aeae&pf_rd_r=PD8YGV8XA34FMYH7G9TJ&pd_rd_r=2cb55e9c-812a-43de-bf52-7e1976f5374b&pd_rd_wg=KmkoW&ref_=pd_gw_hfp13n_bbn')
productName = driver.find_element_by_id('productTitle').text
print(productName)
imgList = driver.find_element_by_xpath('//*[@id="altImages"]/ul')
options = imgList.find_elements_by_xpath(".//li[contains(@class, 'imageThumbnail')]")
for option in options:
print(option.find_element_by_tag_name("img").get_attribute("src")
要打印
标记的src属性的值,您必须导出所有元素的可见性()
- 使用
CSS\u选择器
:
- 使用
XPATH
:
driver.get('https://www.amazon.com/Pendleton-Glacier-National-Queen-Blanket/dp/B003EQ4AYY/?_encoding=UTF8&pd_rd_w=dZURJ&pf_rd_p=ab102187-3a5a-49fd-b43f-4f928775aeae&pf_rd_r=PD8YGV8XA34FMYH7G9TJ&pd_rd_r=2cb55e9c-812a-43de-bf52-7e1976f5374b&pd_rd_wg=KmkoW&ref_=pd_gw_hfp13n_bbn')
print([my_elem.get_attribute("src") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div#altImages>ul li[data-ux-click] img")))])
- 控制台输出:
driver.get('https://www.amazon.com/Pendleton-Glacier-National-Queen-Blanket/dp/B003EQ4AYY/?_encoding=UTF8&pd_rd_w=dZURJ&pf_rd_p=ab102187-3a5a-49fd-b43f-4f928775aeae&pf_rd_r=PD8YGV8XA34FMYH7G9TJ&pd_rd_r=2cb55e9c-812a-43de-bf52-7e1976f5374b&pd_rd_wg=KmkoW&ref_=pd_gw_hfp13n_bbn')
print([my_elem.get_attribute("src") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@id='altImages']/ul//li[@data-ux-click]//img")))])
- 注意:您必须添加以下导入:
['https://images-na.ssl-images-amazon.com/images/I/41Sj%2BO--J9L._AC_US40_.jpg', 'https://images-na.ssl-images-amazon.com/images/I/41iX14X%2BoRL._AC_US40_.jpg', 'https://images-na.ssl-images-amazon.com/images/I/41wiU-3N5JL._AC_US40_.jpg', 'https://images-na.ssl-images-amazon.com/images/I/41waNtDjTxL._AC_US40_.jpg']
工具书类
您可以在以下内容中找到一些相关的详细讨论:
为了获得实际图像而不是拇指,我使用了悬停功能。添加等待是安全的
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
这将获得实际的全尺寸图像SRC@DecentExperience很高兴能够帮助您。你发现这对未来读者的利益很有帮助。看见
from selenium.webdriver.common.action_chains import ActionChains
...
for i in driver.find_elements_by_css_selector('#altImages .imageThumbnail'):
hover = ActionChains(driver).move_to_element(i)
hover.perform()
driver.find_element_by_css_selector('.image.item.maintain-height.selected img').get_attribute('src'))