Python3与Bs4的刮削

Python3与Bs4的刮削,python,beautifulsoup,Python,Beautifulsoup,我正在尝试使用Python3(使用PyCharm)刮取这个网站的HTML。 我只对房子的价格感兴趣,因此我将搜索范围缩小到某些跨度,如: import requests from bs4 import BeautifulSoup page = requests.get('https://www.idealista.com/venta-viviendas/madrid-madrid/') soup = BeautifulSoup(page.text, 'html.parser') pri

我正在尝试使用Python3(使用PyCharm)刮取这个网站的HTML。 我只对房子的价格感兴趣,因此我将搜索范围缩小到某些跨度,如:

import requests
from bs4 import BeautifulSoup


page = requests.get('https://www.idealista.com/venta-viviendas/madrid-madrid/')

soup = BeautifulSoup(page.text, 'html.parser')


prices=soup.findAll("span", {"class": "item-price h2-simulated"})

print(len(prices))
print(prices)
当我运行它时,我得到以下信息:0 []


这意味着它什么也找不到。此外,如果我打印所有内容:打印(soup)那么大的页面,我得到的html内容很少,因此它显然不能获取所有内容。

该网站是动态的,因此,您需要使用浏览器操作工具,例如:

输出:

[price(original=' 369.000€', current='395.000 €', drop='7%'), price(original=' 1.250.000€ Garaje incluido ', current=None, drop=None), price(original=' 1.590.000€ Garaje incluido', current='1.650.000 €', drop='4%'), price(original=' 1.750.000€ Garaje incluido', current='1.875.000 €', drop='7%'), price(original=' 1.090.000€ Garaje incluido', current='1.195.000 €', drop='9%'), price(original=' 795.000€ ', current=None, drop=None), price(original=' 1.095.000€ Garaje incluido ', current=None, drop=None), price(original=' 355.000€ Garaje incluido ', current=None, drop=None), price(original=' 995.000€ Garaje incluido ', current=None, drop=None), price(original=' 1.130.000€ Garaje incluido', current='1.190.000 €', drop='5%'), price(original=' 850.000€ Garaje incluido ', current=None, drop=None), price(original=' 1.200.000€ Garaje incluido ', current=None, drop=None), price(original=' 990.000€ Garaje incluido ', current=None, drop=None), price(original=' 2.100.000€ ', current=None, drop=None), price(original=' 830.000€ Garaje incluido ', current=None, drop=None), price(original=' 2.390.000€ Garaje incluido ', current=None, drop=None), price(original=' 685.000€ ', current=None, drop=None), price(original=' 1.150.000€ Garaje incluido', current='1.200.000 €', drop='4%'), price(original=' 915.000€ ', current=None, drop=None), price(original=' 1.590.000€ Garaje incluido ', current=None, drop=None), price(original=' 625.000€ Garaje incluido', current='640.000 €', drop='2%'), price(original=' 735.000€', current='760.000 €', drop='3%'), price(original=' 890.000€ Garaje incluido', current='950.000 €', drop='6%'), price(original=' 925.000€', current='999.000 €', drop='7%'), price(original=' 975.000€', current='1.100.000 €', drop='11%'), price(original=' 850.000€ Garaje incluido', current='870.000 €', drop='2%'), price(original=' 1.200.000€ ', current=None, drop=None), price(original=' 1.500.000€ ', current=None, drop=None), price(original=' 1.200.000€ ', current=None, drop=None), price(original=' 1.359.000€ ', current=None, drop=None)]

你可以在头等舱找到,我也认为这是找到所有
[price(original=' 369.000€', current='395.000 €', drop='7%'), price(original=' 1.250.000€ Garaje incluido ', current=None, drop=None), price(original=' 1.590.000€ Garaje incluido', current='1.650.000 €', drop='4%'), price(original=' 1.750.000€ Garaje incluido', current='1.875.000 €', drop='7%'), price(original=' 1.090.000€ Garaje incluido', current='1.195.000 €', drop='9%'), price(original=' 795.000€ ', current=None, drop=None), price(original=' 1.095.000€ Garaje incluido ', current=None, drop=None), price(original=' 355.000€ Garaje incluido ', current=None, drop=None), price(original=' 995.000€ Garaje incluido ', current=None, drop=None), price(original=' 1.130.000€ Garaje incluido', current='1.190.000 €', drop='5%'), price(original=' 850.000€ Garaje incluido ', current=None, drop=None), price(original=' 1.200.000€ Garaje incluido ', current=None, drop=None), price(original=' 990.000€ Garaje incluido ', current=None, drop=None), price(original=' 2.100.000€ ', current=None, drop=None), price(original=' 830.000€ Garaje incluido ', current=None, drop=None), price(original=' 2.390.000€ Garaje incluido ', current=None, drop=None), price(original=' 685.000€ ', current=None, drop=None), price(original=' 1.150.000€ Garaje incluido', current='1.200.000 €', drop='4%'), price(original=' 915.000€ ', current=None, drop=None), price(original=' 1.590.000€ Garaje incluido ', current=None, drop=None), price(original=' 625.000€ Garaje incluido', current='640.000 €', drop='2%'), price(original=' 735.000€', current='760.000 €', drop='3%'), price(original=' 890.000€ Garaje incluido', current='950.000 €', drop='6%'), price(original=' 925.000€', current='999.000 €', drop='7%'), price(original=' 975.000€', current='1.100.000 €', drop='11%'), price(original=' 850.000€ Garaje incluido', current='870.000 €', drop='2%'), price(original=' 1.200.000€ ', current=None, drop=None), price(original=' 1.500.000€ ', current=None, drop=None), price(original=' 1.200.000€ ', current=None, drop=None), price(original=' 1.359.000€ ', current=None, drop=None)]