Apache nifi 尼菲-乔特转移

Apache nifi 尼菲-乔特转移,apache-nifi,jolt,Apache Nifi,Jolt,NiFi在JOLT处理器中抛出错误-尽管“高级”处理器接口内的测试工作正常 一些流文件转换时没有错误-但大多数文件都失败-例如: JSON: 震动: 以前有人遇到过这种错误吗 2017-03-24 15:12:35,765 ERROR [Timer-Driven Process Thread-6] o.a.n.p.standard.JoltTransformJSON java.lang.RuntimeException: Unable to load JSON object from I

NiFi在JOLT处理器中抛出错误-尽管“高级”处理器接口内的测试工作正常

一些流文件转换时没有错误-但大多数文件都失败-例如:

JSON:

震动:

以前有人遇到过这种错误吗

    2017-03-24 15:12:35,765 ERROR [Timer-Driven Process Thread-6] o.a.n.p.standard.JoltTransformJSON
java.lang.RuntimeException: Unable to load JSON object from InputStream.
        at com.bazaarvoice.jolt.JsonUtilImpl.jsonToObject(JsonUtilImpl.java:105) ~[json-utils-0.0.21.jar:0.0.21]
        at com.bazaarvoice.jolt.JsonUtils.jsonToObject(JsonUtils.java:117) ~[json-utils-0.0.21.jar:0.0.21]
        at org.apache.nifi.processors.standard.JoltTransformJSON.onTrigger(JoltTransformJSON.java:244) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_77]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_77]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_77]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_77]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.ArrayIndexOutOfBoundsException) (through reference chain: java.util.LinkedHashMap["xml"])
        at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:210) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:177) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase.wrapAndThrow(ContainerDeserializerBase.java:88) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:507) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:341) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer.deserialize(UntypedObjectDeserializer.java:220) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3702) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2784) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.bazaarvoice.jolt.JsonUtilImpl.jsonToObject(JsonUtilImpl.java:102) ~[json-utils-0.0.21.jar:0.0.21]
        ... 14 common frames omitted

我无法复制,但我使用的是最新的源代码(在某个时候将成为NiFi 1.2.0),它对Jolt库进行了升级,并对JoltTransformMJSON处理器进行了一些更改。我粘贴了上面的JSON和规范,它转换得很好


JSON是否使用Java默认值(通常为UTF-8)以外的字符集传递给JoltTransferMJSON?如果是这样,请尝试JoltTransformMJSON处理器之前的处理器。

我无法复制,但我使用的是最新的源代码(在某个时候将成为NiFi 1.2.0),它升级了Jolt库,并对JoltTransformMJSON处理器进行了一些更改。我粘贴了上面的JSON和规范,它转换得很好


JSON是否使用Java默认值(通常为UTF-8)以外的字符集传递给JoltTransferMJSON?如果是这样,请在JoltTransferMJSON处理器之前尝试一个处理器。

像mattyb一样,我也尝试过,但也无法复制(使用最新的源代码)。如果您能够捕获失败的内容和帖子,以及您正在使用的NiFi版本,我可能至少能够重新创建问题并从那里进行调查。

像mattyb一样,我也尝试过,但也无法复制(使用最新的源代码)。如果您能够捕获失败的内容和post,以及您正在使用的NiFi版本,我可能至少能够重新创建问题并从中进行调查。

我相信堆栈跟踪表明处理器无法从您的流文件中读取JSON。失败的flowfile的实际内容是否与您在上面粘贴到高级UI中的内容不同?内容中有特殊字符,例如
“1.0 | 2.0 | 3.0”
,但在“内部”中,您是否可以通过使用不同的NiFi处理器或解析JSON的外部工具加载它来检查它是否是格式良好的JSON?EvaluateJsonPath可能会工作。JSON格式良好-我们还没有通过调整xQuery和XSLTI消除震动并解决难题。我相信堆栈跟踪表明处理器无法从流文件中读取JSON。失败的flowfile的实际内容是否与您在上面粘贴到高级UI中的内容不同?内容中有特殊字符,例如
“1.0 | 2.0 | 3.0”
,但在“内部”中,您是否可以通过使用不同的NiFi处理器或解析JSON的外部工具加载它来检查它是否是格式良好的JSON?EvaluateJsonPath可能会工作。JSON的格式很好——我们并没有通过采用xQuery和XSLT来消除JOLT并解决这个难题
[
  {
    "operation": "shift",
    "spec": {
      "abc": {
        "HEADER": { "*": "&" },
        "SUMMARY": { "*": "&" },
        "*": { "$": "ITEM", "@": "VAL" }
      }
    }
  }
]
    2017-03-24 15:12:35,765 ERROR [Timer-Driven Process Thread-6] o.a.n.p.standard.JoltTransformJSON
java.lang.RuntimeException: Unable to load JSON object from InputStream.
        at com.bazaarvoice.jolt.JsonUtilImpl.jsonToObject(JsonUtilImpl.java:105) ~[json-utils-0.0.21.jar:0.0.21]
        at com.bazaarvoice.jolt.JsonUtils.jsonToObject(JsonUtils.java:117) ~[json-utils-0.0.21.jar:0.0.21]
        at org.apache.nifi.processors.standard.JoltTransformJSON.onTrigger(JoltTransformJSON.java:244) ~[nifi-standard-processors-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.1.0-2.jar:1.1.0.2.1.1.0-2]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_77]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_77]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_77]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_77]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.ArrayIndexOutOfBoundsException) (through reference chain: java.util.LinkedHashMap["xml"])
        at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:210) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:177) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase.wrapAndThrow(ContainerDeserializerBase.java:88) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:507) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:341) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer.deserialize(UntypedObjectDeserializer.java:220) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3702) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2784) ~[jackson-databind-2.6.1.jar:2.6.1]
        at com.bazaarvoice.jolt.JsonUtilImpl.jsonToObject(JsonUtilImpl.java:102) ~[json-utils-0.0.21.jar:0.0.21]
        ... 14 common frames omitted