Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/350.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 我通过多次抓取脚本获得了价值_Python_Selenium_Web Scraping_Scrapy - Fatal编程技术网

Python 我通过多次抓取脚本获得了价值

Python 我通过多次抓取脚本获得了价值,python,selenium,web-scraping,scrapy,Python,Selenium,Web Scraping,Scrapy,我在amazon上为交易页面创建了抓取代码,脚本运行正常,但我不止一次获得了这个值 # -*- coding: utf-8 -*- from setuptools.command import install import requests import html5lib import time from bs4 import BeautifulSoup from selenium import webdriver driver = webdriver.Chrome(executable_pat

我在amazon上为交易页面创建了抓取代码,脚本运行正常,但我不止一次获得了这个值

# -*- coding: utf-8 -*-
from setuptools.command import install
import requests
import html5lib
import time
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Chrome(executable_path='C:\\Users\\Compu City\\Desktop\\chromedriver.exe')

driver.get('https://www.amazon.com/international-sales-offers/b/?ie=UTF8&node=15529609011&ref_=nav_navm_intl_deal_btn')
time.sleep(10)
res = driver.execute_script("return document.documentElement.outerHTML",'window.scrollBy(0,2000)')
soup = BeautifulSoup(res , 'html5lib')
for x in soup.select('div',{'class':'a-section a-spacing-none tallCellView gridColumn5 singleCell'}):
    for y in x.select('button',{'class':'a-button-text a-text-center'}):
        if('Choose options' in y.text):
            for z in x.find_all('a', {'class': 'a-size-base a-link-normal dealTitleTwoLine singleCellTitle autoHeight'}):
                print(z.text)
driver.close()

我认为这是循环中的问题,但我不知道如何解决它

你想刮什么?页面上有很多东西,你的目标是什么?有直接链接到产品详细信息页面的产品你是说有按钮的产品吗?选择选项如果有,那么你就错了我想,每个产品都会在“产品详细信息”页面上导航,而不管按钮名称的按钮是什么。我得到了我想要的正确值,但现在的问题是,我不止一次地从(z.text)中获得了值,一些产品名称打印了10次或更多次。如果('y.text中的'Choose options'),这是什么意思?