使用Python抓取XMR
我正在尝试刮取这一页: 我试着用漂亮的汤,但上面写着:TypeError:unhable-type:“slice” 此错误发生在分析行上。我不能用普通的页面来浏览靓汤,因为刮东西的功能太长了。我需要通过此页面获取每个项目的信息,例如游戏标题、发布日期、折扣、价格和URL 这是我的密码使用Python抓取XMR,python,json,web-scraping,beautifulsoup,Python,Json,Web Scraping,Beautifulsoup,我正在尝试刮取这一页: 我试着用漂亮的汤,但上面写着:TypeError:unhable-type:“slice” 此错误发生在分析行上。我不能用普通的页面来浏览靓汤,因为刮东西的功能太长了。我需要通过此页面获取每个项目的信息,例如游戏标题、发布日期、折扣、价格和URL 这是我的密码 browser = webdriver.Chrome(ChromeDriverManager().install(), options=options) browser.get("http
browser = webdriver.Chrome(ChromeDriverManager().install(), options=options)
browser.get("https://store.steampowered.com/search/results/?query&start=0&count=50&dynamic_data=&sort_by=_ASC&snr=1_7_7_7000_7&filter=topsellers&infinite=1")
url = browser.current_url
page = requests.get(url)
data = page.json()
soup = BeautifulSoup(data, "html.parser")
无需使用
selenium
(速度太慢)
步骤很简单:解析json,并将results\u html
转换为soup:
导入请求
从bs4导入BeautifulSoup
url='1〕https://store.steampowered.com/search/results/?query&start=0&count=50&dynamic_data=&sort_by=_ASC&snr=1_7_7_7000_7&filter=topsellers&infinite=1'
response=requests.get(url)
soup=BeautifulSoup(response.json()['results\u html'],“html.parser”)
对于汤中的游戏信息。查找所有(class=“search\u result\u row ds\u collapse\u flag”):#重复此操作
打印(“url:,game_info.get”(“href”))
打印(“名称:”,游戏信息.find(class=“title”).text.strip())
打印(“已发布:”,game\u info.find(class=“search\u Released”).text.strip()
对于游戏中的价格_info.find(class=“search_price”):#过滤原产地价格。
如果price和price.name!='span':#只需检查标签。
实际价格=价格
打印(“价格:,实际价格.strip())
折扣=游戏信息.find(class=“search\u折扣”).text.strip()
如果折扣:#检查是否存在折扣
打印(“折扣:”,折扣)
其他:
打印(“无折扣”)
打印()
打印:
url: https://store.steampowered.com/app/1091500/Cyberpunk_2077/?snr=1_7_7_7000_150_1
name: Cyberpunk 2077
Released: 9 Dec, 2020
Price: ¥ 298
No discount
url: https://store.steampowered.com/app/578080/PLAYERUNKNOWNS_BATTLEGROUNDS/?snr=1_7_7_7000_150_1
name: PLAYERUNKNOWN'S BATTLEGROUNDS
Released: 21 Dec, 2017
Price: ¥ 98
No discount
url: https://store.steampowered.com/app/431960/Wallpaper_Engine/?snr=1_7_7_7000_150_1
name: Wallpaper Engine
Released: 16 Nov, 2018
Price: ¥ 19
No discount
url: https://store.steampowered.com/app/1404210/Red_Dead_Online/?snr=1_7_7_7000_150_1
name: Red Dead Online
Released: 4 Dec, 2020
Price: ¥ 35
No discount
url: https://store.steampowered.com/app/1406990/NEKOPARA_Vol_4/?snr=1_7_7_7000_150_1
name: NEKOPARA Vol. 4
Released: 26 Nov, 2020
Price: ¥ 33
Discount: -11%
.....