Android 将Firebase备份数据加载到BigQuery
我正在尝试将以下来自Firebase的JSON集合从我的麻将游戏加载到BigQuery中进行分析。这个游戏每天大约有3万场,我想对游戏结果做一些分析。i、 确定成功玩家的行为举止,并就打麻将的策略提供建议 JSON集合如下所示:Android 将Firebase备份数据加载到BigQuery,android,json,firebase,google-bigquery,Android,Json,Firebase,Google Bigquery,我正在尝试将以下来自Firebase的JSON集合从我的麻将游戏加载到BigQuery中进行分析。这个游戏每天大约有3万场,我想对游戏结果做一些分析。i、 确定成功玩家的行为举止,并就打麻将的策略提供建议 JSON集合如下所示: - app-id-9999999999 - games - GamerID: TallDingo31118 - GameDateTimeStamp: 170731065722722 discardPlayer: 3
- app-id-9999999999
- games
- GamerID: TallDingo31118
- GameDateTimeStamp: 170731065722722
discardPlayer: 3
handle: "TallDingo31118"
level: 6
roundNo: 12
selfDrawn: false
winningAmount: 288
winningLevel: 10
winningPlayer: 0
winningTiles: "C1C1C1C2C2C2GDGDGDC3C3NWNWNW"
{"games":
{"000177811614":
{"170731065722722":
{"concealed":false,"discardPlayer":3,"level":6,"roundNo":0,"selfDrawn":false,"winningAmount":288,"winningLevel":10,"winningPlayer":1,
"winningTiles":"C1C1C1C2C2C2GDGDGDC3C3NWNWNW"}},...
生成的JSON备份如下所示:
- app-id-9999999999
- games
- GamerID: TallDingo31118
- GameDateTimeStamp: 170731065722722
discardPlayer: 3
handle: "TallDingo31118"
level: 6
roundNo: 12
selfDrawn: false
winningAmount: 288
winningLevel: 10
winningPlayer: 0
winningTiles: "C1C1C1C2C2C2GDGDGDC3C3NWNWNW"
{"games":
{"000177811614":
{"170731065722722":
{"concealed":false,"discardPlayer":3,"level":6,"roundNo":0,"selfDrawn":false,"winningAmount":288,"winningLevel":10,"winningPlayer":1,
"winningTiles":"C1C1C1C2C2C2GDGDGDC3C3NWNWNW"}},...
我正在使用Firebase中的自动备份功能,遵循以下文档:
我使用以下文档在BigQuery中使用JSON加载功能:
生成加载作业时,我遇到以下错误:
Errors:
2017-08-10T02:05:48Z_app-id-999999999.json.gz: Failed to parse JSON: Closing quote expected in string; Could not parse value; Could not parse value; Could not parse value (error code: invalid)
Job ID app-id-99999999: bquijob_999999_999999999
Creation Time Aug 15, 2017, 7:18:18 PM
Start Time Aug 15, 2017, 7:18:19 PM
End Time Aug 15, 2017, 7:18:36 PM
Destination Table app-id-99999999999:MahjongGames.Games
Write Preference Write if empty
Source Format JSON (Newline Delimited)
Source URI gs://app-id-999999999-backups/2017-08-10T02:05:48Z-app-id-99999999999_data.json.gz (Open in GCS)
Autodetect Schema true
在我看来,Firebase提供的JSON格式与BigQuery JSON解析器不兼容
我曾试图用BigQuery发布一个bug,但他们把我推荐给Firebase团队,他们说它应该可以工作,并向我指出了文档,我已经遵循了这些文档,但没有成功
这是Firebase的广告功能,应该是开箱即用的。有人用这个吗 问题是JSON不是以换行符分隔的吗?太棒了@ElliottBrossard。为了节省空间,数据库备份是单行的,这是有道理的。如果BigQuery导入需要换行符,则表示格式不兼容。这种情况下的解决方案是
JSON.stringify(JSON.parse(backup),null,”)
。感谢您的快速响应和建议:在BigQuery中如何和/或在何处编写此转换脚本。对不起,我对FireBase很在行,但对BigQuery来说是个新手。问题是JSON没有换行符吗?太棒了@ElliottBrossard。为了节省空间,数据库备份是单行的,这是有道理的。如果BigQuery导入需要换行符,则表示格式不兼容。这种情况下的解决方案是JSON.stringify(JSON.parse(backup),null,”)
。感谢您的快速响应和建议:在BigQuery中如何和/或在何处编写此转换脚本。抱歉,我对FireBase很在行,但对BigQuery来说是新手。