Pig-无法推断org.apache.Pig.piggybank.evaluation.datetime.convert.ISOToUnix的匹配函数为多个或不匹配
我只是想把pig中的日期时间格式转换成大纪元时间,这样我就可以用时间做其他计算了。下面是我的(部分)脚本:Pig-无法推断org.apache.Pig.piggybank.evaluation.datetime.convert.ISOToUnix的匹配函数为多个或不匹配,datetime,hadoop,apache-pig,epoch,Datetime,Hadoop,Apache Pig,Epoch,我只是想把pig中的日期时间格式转换成大纪元时间,这样我就可以用时间做其他计算了。下面是我的(部分)脚本: DEFINE ISOToUnix org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix(); A = LOAD 's3://hearstlogfiles/google/NetworkBackfillImpressions_271283/2014/09/24/NetworkBackfillImpressions_2712
DEFINE ISOToUnix org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();
A = LOAD 's3://hearstlogfiles/google/NetworkBackfillImpressions_271283/2014/09/24/NetworkBackfillImpressions_271283_20140924_00.gz' USING PigStorage(',');
B = LIMIT A 10;
C = FOREACH B GENERATE
(chararray)(CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),SUBSTRING($0, 11,19) )) as dt_string:chararray,
DATE_TIME(CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),SUBSTRING($0, 11,19) )) AS dt;
D = FOREACH C GENERATE
dt_string,
dt,
ISOToUnix(dt)/1000 as epoch:long;
DUMP D;
当pig尝试执行下面的行时,我得到了它下面的错误。我知道我将dt转换为正确的格式
ISOToUnix(dt)/1000 as epoch:long
Could not infer the matching function for org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix as multiple or none of them fit. Please use an explicit cast.
当我转储C时,我得到以下结果。所以我知道CDT的格式是正确的
(2014-09-24 02:53:54,2014-09-24T02:53:54.000Z)
(2014-09-24 02:57:54,2014-09-24T02:57:54.000Z)
(2014-09-24 03:05:06,2014-09-24T03:05:06.000Z)
(2014-09-24 03:27:30,2014-09-24T03:27:30.000Z)
(2014-09-24 03:37:00,2014-09-24T03:37:00.000Z)
(2014-09-24 03:39:18,2014-09-24T03:39:18.000Z)
(2014-09-24 03:41:24,2014-09-24T03:41:24.000Z)
(2014-09-24 03:43:18,2014-09-24T03:43:18.000Z)
(2014-09-24 03:58:12,2014-09-24T03:58:12.000Z)
请提供帮助。粘贴示例来自:
如果您注意到,dt(作为参数传递给ISOTONIX UDF)是chararray。因此您需要将“dt”列键入chararray,如下所示:
C = FOREACH B
GENERATE
(chararray)(CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),
SUBSTRING($0, 11,19) )) as dt_string:chararray,
CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),SUBSTRING($0, 11,19) ) AS dt:chararray;
D = FOREACH C
GENERATE
dt_string,
dt,
ISOToUnix((chararray)dt)/1000 as epoch:long;
DUMP D;
希望这能有所帮助。谢谢你的帮助Guarav,我发现错误无法将datetime转换为chararray。如果我找到其他解决方案,我会让社区知道。类似的错误发生在我身上,你找到解决方案了吗?
C = FOREACH B
GENERATE
(chararray)(CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),
SUBSTRING($0, 11,19) )) as dt_string:chararray,
CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),SUBSTRING($0, 11,19) ) AS dt:chararray;
D = FOREACH C
GENERATE
dt_string,
dt,
ISOToUnix((chararray)dt)/1000 as epoch:long;
DUMP D;