Python 为什么是美丽的汤，拉链和。选择刮削值的顺序错误？

更新

我现在正在使用这个代码

from bs4 import BeautifulSoup
import requests
res=requests.get("https://www.ebay.co.uk/sch/i.html?_from=R40&_nkw=Playstation+1&_sacat=0&_pgn=1")
soup=BeautifulSoup(res.text,'html.parser')
for item,price in zip(soup.select('.lvtitle>a'),soup.select('.lvprice.prc >span')):
    print(item.text + " : " + price.text.strip())

它以一种非常漂亮、易于阅读的格式输出价格和产品名称，但输出顺序与在易趣上的显示顺序不同

---

脚本给出的前四个输出是
（1） 索尼PLAYSTATION 1 PS1控制台/经过测试的工作和控制器/3款免费游戏：28.75英镑 （2） Playstation 1的游戏包括崩溃：20.00英镑
（3） 原厂索尼Playstation 1捆绑销售：29.99英镑
（4） 索尼Playstation 1 PS1控制台捆绑Joblot AV电视销售线索：26.99英镑

但是易趣上的前四项是
（1） 索尼PLAYSTATION 1 PS1控制台/经过测试的工作和控制器/3款免费游戏28.75英镑 （2） 索尼Playstation 1 PS1游戏机捆绑Joblot AV电视销售线索26.99英镑 （3） 索尼Playstation 1 PS1 PSONE控制台捆绑包和电视AV导频测试工作价格为29.99英镑 （4） 新上市的索尼Playstation PS1主机盒，2个控制器，2个存储卡，原版演示版44.99英镑

原始问题

我想要一个网页刮板找到页面上所有50种产品的名称和价格-

我运行了这个代码-

for post in soup.select("h3"):
    print (post)

-这里是输出（还有更多的输出，我没有包括在内）

请启用JavaScript
格式

代码-

title="Click this link to access SONY PLAYSTATION 1 PS1 CONSOLE / Tested Working &amp; Controller / 3 FREE GAMES">SONY PLAYSTATION 1 PS1 CONSOLE / Tested Working &amp; Controller / 3 FREE GAMES</a>
    </h3>

title=“单击此链接访问索尼PLAYSTATION 1 PS1控制台/测试工作和控制器/3免费游戏”>索尼PLAYSTATION 1 PS1控制台/测试工作和控制器；控制器/3免费游戏

-出现两次

---

但是它出现的两次，href值是不同的。在eBay上，该项目出现在列表的顶部，因此我需要重写代码，使其保留第一个实例，但去掉第二个实例。我真的不知道从哪里开始解决这个问题，我不知道我能做什么实验

试试下面的代码。使用Zip可以选择两个项目

from bs4 import BeautifulSoup
import requests
res=requests.get("https://www.ebay.co.uk/sch/i.html?_from=R40&_nkw=Playstation+1&_sacat=0&_pgn=1")
soup=BeautifulSoup(res.text,'html.parser')
for item,price in zip(soup.select('.lvtitle>a'),soup.select('.lvprice.prc >span')):
    print(item.text + " : " + price.text.strip())

---

输出：

SONY PLAYSTATION 1 PS1 CONSOLE / Tested Working & Controller / 3 FREE GAMES : £28.75
Playstation 1 With Games Including Crash : £20.00
Original Sony Playstation 1 Bundle : £29.99
Sony Playstation 1 PS1 Console Bundle Joblot AV TV Lead : £26.99
Sony Playstation 1 PS1 PSONE Console Bundle  & TV AV Lead TESTED WORKING : £29.99
SONY PLAYSTATION 1 PS1 CONSOLE / Tested Working & Controller / 3 FREE GAMES : £28.75
Sony Playstation 1 PS1 Console Bundle Joblot AV TV Lead : £26.99
Slim PAL PlayStation 1 PSOne Console + Official Controller Bundle ~ Fully Tested : £22.95
Sony Playstation 1 PS1 PSONE Console Bundle  & TV AV Lead TESTED WORKING : £29.99
Sony Playstation PS1 Console Boxed, 2 Controllers, 2 Memory Cards, Original Demo : £44.99
Sony Playstation 1 PS1 Console WITH LEADS TESTED and WORKING PAL : £19.95
Sony playstation 1 original console boxed with instructions : £28.00
SONY Playstation 1 PS1 PSONE Games Console Bundle With 14 Games DRIVER - W71 : £20.62
Sony Playstation 1 Grey Console / Model SCPH-1002 / Tested & Working : £7.50
Sony Playstation PS1 Console Boxed, 2 Controllers, 2 Memory Cards, Original Demo : £44.99
Sony Playstation PS One PS1 White Console + 2 Controllers & Memory Card - TESTED : £24.75
PlayStation 1 - Original Grey Console : £29.99
SONY PS1 PlayStation One Console Controller Retro Gaming Classic Boxed VGC : £39.99
SONY PlayStation 1 PS1 Home Games Console Bundle With 9 Games TEKKEN 3 - D37 : £28.00
PLAYSTATION 1 CONSOLE 2 CONTROLLERS 27 GAMES GOOD CONDITION PAL SCPH-55522 : £5.00
PlayStation 1 Console With Leads And Offical Controller Ps1 : £20.00
SONY Playstation 1 with 13 games, 2 controllers, official SONY RFU & RCA cables : £29.99
Sony PlayStation 1 Console Bundle with PS1 Games (Boxed) PAL : £19.99
Sony PlayStation 1 PS1 Grey Console Inc Official Controllers & 10 Game Bundle : £39.95
Playstation One PS1 Boxed in EXCELLENT Condition Bundle with 10 loose Games : £49.99
Sony Playstation 1 Console With Controller And 8 Games : £32.00
Sony PlayStation 1 - PS1 Console - Controller & 2 Memory Cards : £32.99
Sony Playstation 1 SCPH-5502 Grey Console. Orginal playstation with controller : £7.00
Sony Playstation 1 + Leads, Memory Cartridge, Controller & Games : £20.99
Sony Playstation 1 PS1 PS One Original Games Console : £18.00
Sony Playstation 1 One Console, Fully Boxed Complete With Polystyrene And Covers : £34.99
Playstation 1 Console With 11 Games, 2 Controllers, PSU #147 : £26.00
Sony Playstation 1 SCPH-5502 Grey Console + 8 games : £19.30
SONY PLAYSTATION 1 BUNDLE CONSOLE 2 CONTROLLERS MEMORY CARD AND 10 GAMES : £30.00
SONY PLAYSTATION 1 BUNDLE CONSOLE 2 CONTROLLERS MEMORY CARD AND 10 GAMES : £23.00
Sony Playstation PSone Slim 1 2 Controllers With Box : £10.50
20 game PlayStation 1 bundle : £29.95
SONY Playstation 1 PS1 Console Bundle With 9 Games, 2 Controllers, BOXED - C63 : £47.00
128G PS1 MINI True Blue Mini Crackhead Pack For Playstation Built-in 7000 Games : £23.00
Sony Playstation 1 PSone Slim With Crash Bandicoot 3 & DualShock Controller VGC : £20.78
Playstation 1 dual shock boxed with cab no controller. No games great condition : £34.99
Sony PlayStation 1 Dual Shock Bundle Grey Console, 2 controllers, 2 memory cards : £14.40
REGION FREE PLAYSTATION 1 SLIM PSOne 8 WIRE STEALTH CHIPPED / MODDED PS1 CONSOLE : £42.00
Sony Playstation 1 Grey Console+all leads and 2 controller's : £15.00
SONY PLAYSTATION 1 CHIPPED SCPH-5552 - FULLY WORKING PS1 MODDED CONSOLE ONLY : £22.00
Sony PlayStation - Working PS1 Console - Parts Only : £35.00
Sony PlayStation 1 Dual Shock Bundle Grey Console scph 5552 : £18.00
Sony Playstation 1 PS1 PSONE Console & Controller with leads and game : £34.00
Sony PlayStation 1 PS1 Console Bundle - 23 Games -  Controller - All Cables : £2.20
Sony PlayStation 1 PS1 Grey Console Inc Official Controllers & 10 Game Bundle : £35.00
SONY Playstation 1 PS1 PSone Home Games Console Bundle With Controller - B98 : £24.99
Sony PlayStation 1 PS1 Original Console + 3 Games Rayman Crash Banicoot Oddworld : £49.95
SONY PLAYSTATION 1 SCPH-1002 AUDIOPHILE CONSOLE + GAMES : £39.95
Sony Playstation 1 SCPH-102 Console - Grey : £9.99
Sony PS1 PLAYSTATION 1 SCPH-1002 Console : £32.00
Original Sony Playstation 1 PS1 Games Console Leads Controller Bundle Boxed # : £28.99
Playstation 1 Console + 3 Games : £23.50
Sony Playstation 1 Slim Console Bundle- SCPH-102 -Inc Cables & Controller - 6 : £19.99
Playstation 1 Ps1 Psone And 22 Games : £44.95
Sony Playstation 1 Console With Sony Controller, AV Cable and Power Cable PAL : £9.99
Ps1 Console + Memory Card / Choose Slim / Phat / Audiophile - Complete Setup : £24.99

---

我不确定你能不能随着时间的推移，当然也不能以任何可靠的方式。我认为有适当的防刮措施。例如，您可以在短时间内添加

用户代理

和

推荐人

标题，如果您随后运行代码删除赞助链接，您将获得如第页所示的信息：

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36',
    'Referer': 'https://www.ebay.co.uk/sch/i.html?_from=R40&_nkw=Playstation+1&_sacat=0&_pgn=1',
}

r= requests.get('https://www.ebay.co.uk/sch/i.html?_from=R40&_nkw=Playstation+1&_sacat=0&_pgn=1', headers=headers)
soup = bs(r.text,'lxml')

for item in soup.select('.lvresult:has(.promoted-lv)'):
    item.decompose()

for title, price in zip(soup.select('.lvtitle') , soup.select('.lvprice .bold')):
    print(title.text, price.text) 

但是，在尝试再次请求之前，这会很快被检测到并在没有足够停顿的情况下停止工作（我没有计时，但可能是在几分钟内）

注意：这些度量还检测您是否使用自动浏览器，例如，它会去除您可以看到的赞助商链接的附加随机属性（也缺少：before）

---

为什么要将此标记为易趣api？如果您使用的是API，则不需要刮取。如果href不同，则恰好是两个不同的项目具有相似的文本。我检查了易趣，以确保两个项目没有相同的项目名称。我已经排除了上述内容。当我查看该URL的源代码时，我甚至找不到

class=“lvtitle”

。我想知道易趣是否将一些不同的东西返回给了BS而不是浏览器。是的，我也经历过类似的事情。网站有时会根据用户代理给出不同的答案。因此，浏览器从scraper得到不同的结果。该代码似乎解决了重复H3的问题，但由于未知原因，它没有按时间顺序输出。很明显，你比我更了解Python，所以我不知道你是否知道是什么导致了订单混乱。脚本给出的前四个输出是：（1）SONY PLAYSTATION 1 PS1控制台/测试工作和控制器/3个免费游戏：28.75英镑（2）PLAYSTATION 1和包括崩溃在内的游戏：20.00英镑（3）原版索尼Playstation 1捆绑包：29.99英镑

SONY PLAYSTATION 1 PS1 CONSOLE / Tested Working & Controller / 3 FREE GAMES : £28.75
Playstation 1 With Games Including Crash : £20.00
Original Sony Playstation 1 Bundle : £29.99
Sony Playstation 1 PS1 Console Bundle Joblot AV TV Lead : £26.99
Sony Playstation 1 PS1 PSONE Console Bundle  & TV AV Lead TESTED WORKING : £29.99
SONY PLAYSTATION 1 PS1 CONSOLE / Tested Working & Controller / 3 FREE GAMES : £28.75
Sony Playstation 1 PS1 Console Bundle Joblot AV TV Lead : £26.99
Slim PAL PlayStation 1 PSOne Console + Official Controller Bundle ~ Fully Tested : £22.95
Sony Playstation 1 PS1 PSONE Console Bundle  & TV AV Lead TESTED WORKING : £29.99
Sony Playstation PS1 Console Boxed, 2 Controllers, 2 Memory Cards, Original Demo : £44.99
Sony Playstation 1 PS1 Console WITH LEADS TESTED and WORKING PAL : £19.95
Sony playstation 1 original console boxed with instructions : £28.00
SONY Playstation 1 PS1 PSONE Games Console Bundle With 14 Games DRIVER - W71 : £20.62
Sony Playstation 1 Grey Console / Model SCPH-1002 / Tested & Working : £7.50
Sony Playstation PS1 Console Boxed, 2 Controllers, 2 Memory Cards, Original Demo : £44.99
Sony Playstation PS One PS1 White Console + 2 Controllers & Memory Card - TESTED : £24.75
PlayStation 1 - Original Grey Console : £29.99
SONY PS1 PlayStation One Console Controller Retro Gaming Classic Boxed VGC : £39.99
SONY PlayStation 1 PS1 Home Games Console Bundle With 9 Games TEKKEN 3 - D37 : £28.00
PLAYSTATION 1 CONSOLE 2 CONTROLLERS 27 GAMES GOOD CONDITION PAL SCPH-55522 : £5.00
PlayStation 1 Console With Leads And Offical Controller Ps1 : £20.00
SONY Playstation 1 with 13 games, 2 controllers, official SONY RFU & RCA cables : £29.99
Sony PlayStation 1 Console Bundle with PS1 Games (Boxed) PAL : £19.99
Sony PlayStation 1 PS1 Grey Console Inc Official Controllers & 10 Game Bundle : £39.95
Playstation One PS1 Boxed in EXCELLENT Condition Bundle with 10 loose Games : £49.99
Sony Playstation 1 Console With Controller And 8 Games : £32.00
Sony PlayStation 1 - PS1 Console - Controller & 2 Memory Cards : £32.99
Sony Playstation 1 SCPH-5502 Grey Console. Orginal playstation with controller : £7.00
Sony Playstation 1 + Leads, Memory Cartridge, Controller & Games : £20.99
Sony Playstation 1 PS1 PS One Original Games Console : £18.00
Sony Playstation 1 One Console, Fully Boxed Complete With Polystyrene And Covers : £34.99
Playstation 1 Console With 11 Games, 2 Controllers, PSU #147 : £26.00
Sony Playstation 1 SCPH-5502 Grey Console + 8 games : £19.30
SONY PLAYSTATION 1 BUNDLE CONSOLE 2 CONTROLLERS MEMORY CARD AND 10 GAMES : £30.00
SONY PLAYSTATION 1 BUNDLE CONSOLE 2 CONTROLLERS MEMORY CARD AND 10 GAMES : £23.00
Sony Playstation PSone Slim 1 2 Controllers With Box : £10.50
20 game PlayStation 1 bundle : £29.95
SONY Playstation 1 PS1 Console Bundle With 9 Games, 2 Controllers, BOXED - C63 : £47.00
128G PS1 MINI True Blue Mini Crackhead Pack For Playstation Built-in 7000 Games : £23.00
Sony Playstation 1 PSone Slim With Crash Bandicoot 3 & DualShock Controller VGC : £20.78
Playstation 1 dual shock boxed with cab no controller. No games great condition : £34.99
Sony PlayStation 1 Dual Shock Bundle Grey Console, 2 controllers, 2 memory cards : £14.40
REGION FREE PLAYSTATION 1 SLIM PSOne 8 WIRE STEALTH CHIPPED / MODDED PS1 CONSOLE : £42.00
Sony Playstation 1 Grey Console+all leads and 2 controller's : £15.00
SONY PLAYSTATION 1 CHIPPED SCPH-5552 - FULLY WORKING PS1 MODDED CONSOLE ONLY : £22.00
Sony PlayStation - Working PS1 Console - Parts Only : £35.00
Sony PlayStation 1 Dual Shock Bundle Grey Console scph 5552 : £18.00
Sony Playstation 1 PS1 PSONE Console & Controller with leads and game : £34.00
Sony PlayStation 1 PS1 Console Bundle - 23 Games -  Controller - All Cables : £2.20
Sony PlayStation 1 PS1 Grey Console Inc Official Controllers & 10 Game Bundle : £35.00
SONY Playstation 1 PS1 PSone Home Games Console Bundle With Controller - B98 : £24.99
Sony PlayStation 1 PS1 Original Console + 3 Games Rayman Crash Banicoot Oddworld : £49.95
SONY PLAYSTATION 1 SCPH-1002 AUDIOPHILE CONSOLE + GAMES : £39.95
Sony Playstation 1 SCPH-102 Console - Grey : £9.99
Sony PS1 PLAYSTATION 1 SCPH-1002 Console : £32.00
Original Sony Playstation 1 PS1 Games Console Leads Controller Bundle Boxed # : £28.99
Playstation 1 Console + 3 Games : £23.50
Sony Playstation 1 Slim Console Bundle- SCPH-102 -Inc Cables & Controller - 6 : £19.99
Playstation 1 Ps1 Psone And 22 Games : £44.95
Sony Playstation 1 Console With Sony Controller, AV Cable and Power Cable PAL : £9.99
Ps1 Console + Memory Card / Choose Slim / Phat / Audiophile - Complete Setup : £24.99

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36',
    'Referer': 'https://www.ebay.co.uk/sch/i.html?_from=R40&_nkw=Playstation+1&_sacat=0&_pgn=1',
}

r= requests.get('https://www.ebay.co.uk/sch/i.html?_from=R40&_nkw=Playstation+1&_sacat=0&_pgn=1', headers=headers)
soup = bs(r.text,'lxml')

for item in soup.select('.lvresult:has(.promoted-lv)'):
    item.decompose()

for title, price in zip(soup.select('.lvtitle') , soup.select('.lvprice .bold')):
    print(title.text, price.text)