Hadoop 清管器日期转换发布样本日期为2009年4月1日上午9:06:00

我能够将给定的字符串转换为日期，没有任何问题。 下面是有效的代码

date.txt

bill_details= FOREACH bill_details_raw GENERATE 
              $0 AS invD_icode, $1 AS invD_InvH_icode, 
              $2 AS invD_InvH_No,
              ToDate(invD_InvH_date,'MM/dd/yyyy hh:mm:ss aa') AS dateString,  
              $5 AS invD_pro_icode,
              $6 AS invD_pro_ucode, 
              $7 AS invD_pro_name,  
              $8 AS invD_qty;



java.lang.Exception: org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-85 Operator Key: scope-85) children: null at []]: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-85 Operator Key: scope-85) children: null at []]: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:338)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:899)
    at org.apache.pig.builtin.ToDate2ARGS.exec(ToDate2ARGS.java:45)
    at org.apache.pig.builtin.ToDate2ARGS.exec(ToDate2ARGS.java:33)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:330)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextDateTime(POUserFunc.java:422)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:329)
    ... 14 more

file_date = load 'date.txt';
dates_only = foreach file_data generate ToDate($0, 'MM/dd/yyyy hh:mm:ss aa');

代码

4/1/2009 9:06:00 AM

输出

bill_details= FOREACH bill_details_raw GENERATE 
              $0 AS invD_icode, $1 AS invD_InvH_icode, 
              $2 AS invD_InvH_No,
              ToDate(invD_InvH_date,'MM/dd/yyyy hh:mm:ss aa') AS dateString,  
              $5 AS invD_pro_icode,
              $6 AS invD_pro_ucode, 
              $7 AS invD_pro_name,  
              $8 AS invD_qty;



java.lang.Exception: org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-85 Operator Key: scope-85) children: null at []]: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-85 Operator Key: scope-85) children: null at []]: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:338)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Invalid format: " invD_InvH_date"
    at org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:899)
    at org.apache.pig.builtin.ToDate2ARGS.exec(ToDate2ARGS.java:45)
    at org.apache.pig.builtin.ToDate2ARGS.exec(ToDate2ARGS.java:33)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:330)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextDateTime(POUserFunc.java:422)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:329)
    ... 14 more

file_date = load 'date.txt';
dates_only = foreach file_data generate ToDate($0, 'MM/dd/yyyy hh:mm:ss aa');

可能的解决方案：

我认为问题是，您使用的是列名，现在我不确定它是否是架构的一部分，但似乎不是

你能试试这个吗

dump file_date;
(4/1/2009 9:06:00 AM)
dump dates_only
(2009-04-01T09:06:00.000Z)

---

提示，您的日志显示

无效格式：“invD\u InvH\u date”

…顺便说一句，Pig不会从文件中删除头文件。您是否尝试过在AM/PM中使用一个“a”？ToDate（invD_InvH_date，'MM/dd/yyyyy hh:MM:ss a'）作为日期字符串，是的，我尝试过，但结果相同bill_details=FOREACH bill_details_raw生成$0作为invD_icode，$1作为invD_InvH_icode，$2作为invD_InvH_No，$3，'M/dd/yyyyy hh:MM:ss aa'）作为日期字符串，$5作为invD_pro icode，6美元作为库存商品代码，7美元作为库存商品名称，8美元作为库存商品数量@PranavChaturvedi我不理解您的评论？不同之处在于To_date函数中，在数据格式字符串中，我在前面提到了空格，它与我的解决方案有什么不同？