Python WEB抓取-如何访问通过javascript ng绑定呈现的内容？

我已经尝试从中删除所有的球员数据

使用美丽的汤： blog.py

import requests
from bs4 import BeautifulSoup
from csv import writer

response = requests.get('https://www.easports.com/madden-nfl/player-ratings/?i=1&s=ovr_rating:DESC&v=true&=undefined')
soup = BeautifulSoup(response.text, 'html.parser')

posts = soup.find_all(class_='player_rating-value')
print(posts)

但它会给我一个空列表，或者如果我尝试使用

posts = soup.find_all(class_='ratings-hub_database')
print(posts)

它只会给我力量之前的数据

---

我读过其他解决方案，其中说我需要从“网络”选项卡获取XHR数据，但我不知道如何打开“网络”选项卡并按F5刷新页面。然后按Ctrl+F打开搜索框，输入球员姓名，然后按enter键。查看返回的结果，您将发现以下以json形式返回统计信息的内容

from bs4 import BeautifulSoup as bs
import requests

data = requests.get('https://www.easports.com/madden-nfl/ratings/service/data?entityType=madden19_player&filter=iteration:1&sort=ovr_rating:DESC,%20lastName:ASC&limit=25').json()
print(data)

---

您可以写入csv（请注意，您需要添加自己的标题顺序）

---

使用csv

from bs4 import BeautifulSoup as bs
import requests, csv

data = requests.get('https://www.easports.com/madden-nfl/ratings/service/data?entityType=madden19_player&filter=iteration:1&sort=ovr_rating:DESC,%20lastName:ASC&limit=25').json()
with open("data.csv", "w", encoding="utf-8-sig", newline='') as csv_file:
    w = csv.writer(csv_file, delimiter = ",", quoting=csv.QUOTE_MINIMAL)
    w.writerow(list(data['docs'][0].keys()))
    for row in data['docs']:
        w.writerow(list(row.values()))

---

非常感谢你！我以json文件的形式获取xhr数据，并将其转换为csv，但它返回的只是合并后的数据。我想要得到的是类似于这个表单的东西，你需要循环数据['docs'，]并提取感兴趣的信息。在这里查看json：我设法找到了数据并理解了它。再次感谢你！是的，我会接受这个答案

from bs4 import BeautifulSoup as bs
import requests, csv

data = requests.get('https://www.easports.com/madden-nfl/ratings/service/data?entityType=madden19_player&filter=iteration:1&sort=ovr_rating:DESC,%20lastName:ASC&limit=25').json()
with open("data.csv", "w", encoding="utf-8-sig", newline='') as csv_file:
    w = csv.writer(csv_file, delimiter = ",", quoting=csv.QUOTE_MINIMAL)
    w.writerow(list(data['docs'][0].keys()))
    for row in data['docs']:
        w.writerow(list(row.values()))