Apache pig 如何在Pig中解码来自列的JSON?
有许多示例说明如何使用Apache pig 如何在Pig中解码来自列的JSON?,apache-pig,Apache Pig,有许多示例说明如何使用JsonLoader()从文件加载带有模式的JSON数据,而不是从任何类型的其他输出加载数据。您正在查找Elephant Bird中提供的JsonStringToMap UDF: 示例文件: foo bar {"version":1, "type":"an event", "count": 1} foo bar {"version":1, "type":"another event", "count": 1} 猪字: REGISTER /p
JsonLoader()
从文件加载带有模式的JSON数据,而不是从任何类型的其他输出加载数据。您正在查找Elephant Bird中提供的JsonStringToMap UDF:
示例文件:
foo bar {"version":1, "type":"an event", "count": 1}
foo bar {"version":1, "type":"another event", "count": 1}
猪字:
REGISTER /path/to/elephant-bird.jar;
DEFINE JsonStringToMap com.twitter.elephantbird.pig.piggybank.JsonStringToMap();
raw = LOAD '/tmp/file.tsv' USING PigStorage('\t') AS (col1:chararray,col2:chararray,json_string:chararray);
parsed = FOREACH raw GENERATE col1,col2,JsonStringToMap(json_string);
ILLUSTRATE parsed; -- Just to show the output
预处理(JSON作为字符/字符串):
后期处理(JSON作为映射):
这个问题是重复的
-------------------------------------------------------------------------------------------------------
| raw | col1:chararray | col2:chararray | json_string:chararray |
-------------------------------------------------------------------------------------------------------
| | foo | bar | {"version":1, "type":"another event", "count": 1} |
-------------------------------------------------------------------------------------------------
| parsed | col1:chararray | col2:chararray | json:map(:chararray) |
-------------------------------------------------------------------------------------------------
| | foo | bar | {count=1, type=another event, version=1} |
-------------------------------------------------------------------------------------------------