Python 要输出CSV文件的已刮取json数据
所以我收集了这些数据Python 要输出CSV文件的已刮取json数据,python,pandas,dataframe,web-scraping,Python,Pandas,Dataframe,Web Scraping,所以我收集了这些数据 [ { "id": 4321069, "points": 52535, "name": "Dennis", "avatar": "", "leaderboardPosition": 1, "rank": "
[
{
"id": 4321069,
"points": 52535,
"name": "Dennis",
"avatar": "",
"leaderboardPosition": 1,
"rank": ""
},
{
"id": 9281450,
"points": 40930,
"name": "Dinh",
"avatar": "https://uploads-us-west-2.insided.com/koodo-en/icon/90x90/aeaf8cc1-65b2-4d07-a838-1f078bbd2b60.png",
"leaderboardPosition": 2,
"rank": ""
},
{
"id": 1087209,
"points": 26053,
"name": "Sophia",
"avatar": "https://uploads-us-west-2.insided.com/koodo-en/icon/90x90/c3e9ffb1-df72-46e8-9cd5-c66a000e98fa.png",
"leaderboardPosition": 3,
"rank": ""
And so on... Big leaderboard of 20 ppl
用这个代码擦掉
import json
import requests
import pandas as pd
url_all_time = 'https://community.koodomobile.com/widget/pointsLeaderboard?period=allTime&maxResults=20&excludeRoles='
# print for all time:
data = requests.get(url_all_time).json()
# for item in data:
# uncomment this to print all data:
# print(json.dumps(data, indent=4))
for item in data:
print(item['name'], item['points'])
我希望能够创建一个表来处理这个问题
每次我刮取数据时,我都希望它用标记为标题的新数据更新表中的点数。所以基本上我想的是,我的索引=用户名,标题=日期。问题是,我甚至无法创建具有该名称/点列的csv文件
到目前为止,我唯一成功的做法是将所有数据写入csv文件。我无法在print命令中找到我想要的数据
编辑:在阅读了@Shijith发布的内容后,我成功地将数据传输到了.csv,但考虑到我的想法(随着时间的推移添加更多数据),我问自己应该使用索引还是不使用索引来编写代码
与
没有
import pandas as pd
url_all_time = 'https://community.koodomobile.com/widget/pointsLeaderboard?period=allTime&maxResults=20&excludeRoles='
data = pd.read_json(url_all_time)
table = pd.DataFrame.from_records(data, columns=['points','name'])
table.to_csv('products.csv', index=False, encoding='utf-8')
您是否尝试过直接将json读入数据帧?从这里可以很容易地按照您的意愿对其进行转换。您可以为今天的日期添加一列并对其进行透视
import pandas as pd
url_all_time = 'https://community.koodomobile.com/widget/pointsLeaderboard?period=allTime&maxResults=20&excludeRoles='
df = pd.read_json(url_all_time)
data['date'] = pd.Timestamp.today().strftime('%m-%d-%Y')
data.pivot(index='name',columns='date',values='points')
@NoobAtPython更新了。在末尾的另一列“date”中输出相同的内容。这是我想要的一部分。我将进一步研究熊猫数据框功能
import pandas as pd
url_all_time = 'https://community.koodomobile.com/widget/pointsLeaderboard?period=allTime&maxResults=20&excludeRoles='
df = pd.read_json(url_all_time)
data['date'] = pd.Timestamp.today().strftime('%m-%d-%Y')
data.pivot(index='name',columns='date',values='points')