Apache nifi 在nifi groovy中,给出异常的脚本已经标记为要传输
我有输入有效载荷Apache nifi 在nifi groovy中,给出异常的脚本已经标记为要传输,apache-nifi,Apache Nifi,我有输入有效载荷{“ENV”:{“MAC”:“6cececeb5d0302”,“NS”:“NM”,“DATE”:“170619”,“TIME”:“114635”,“PM2.5”:“10”,“PM10”:“8.272681196722012”} 我使用下面的脚本创建了多个流文件,效果很好。但同样的代码不适用于传输一个流文件。在下面基于的代码中,如果代码是我想要的,那么脚本要么将流传输到下一个处理器,要么只记录信息并删除该流文件 import groovy.json.*; 导入org.apache.
{“ENV”:{“MAC”:“6cececeb5d0302”,“NS”:“NM”,“DATE”:“170619”,“TIME”:“114635”,“PM2.5”:“10”,“PM10”:“8.272681196722012”}
我使用下面的脚本创建了多个流文件,效果很好。但同样的代码不适用于传输一个流文件。在下面基于的代码中,如果代码是我想要的,那么脚本要么将流传输到下一个处理器,要么只记录信息并删除该流文件
import groovy.json.*;
导入org.apache.commons.io.IOUtils;
导入java.nio.charset.*;
导入java.lang.*;
def flowFile=session.get();
if(flowFile==null){
回来
}
def flowFiles=[]作为列表
flowFile=session.write(flowFile,
{inputStream,outputStream->
试一试{
def data=IOUtils.toString(inputStream,StandardCharsets.UTF_8);
def input=new JsonSlurper().parseText(数据);
if(Double.parseDouble(input.ENV[“PM2.5”])>7.2){
规划=
[环境:[
日期:input.ENV.DATE,
“PM2.5”:input.ENV[“PM2.5”],
时间:input.ENV.TIME,
日期\时间:input.ENV.DATE,
NS:input.ENV.NS,
MAC:input.ENV.MAC
]];
字符串finalJson=JsonOutput.toJson out;
write(finalJson.getBytes(StandardCharsets.UTF_8))
流程文件代码中的问题-您正在将
传输到成功,然后删除
相同的流程文件
这些操作中只有一个可以应用于一个文件
你好像忘了创建一个新文件
尝试更改这部分代码:
String finalJson=JsonOutput.toJson out;
write(finalJson.getBytes(StandardCharsets.UTF_8))
flowFiles新建JsonBuilder(out).writeTo(w)}
flowFiles new JsonSlurper().parse(r)}
data.ENV.each{item->
如果((项目“PM2.5”为双倍)>7.2){
def outFile=flowFile.clone(false)//克隆没有内容的flowFile
def outData=[ENVIRONMENT:item]//合并新的json
outFile.write(“UTF-8”){w->
new JsonBuilder(outData).writeTo(w)//将json作为输出文件内容写入
}
REL_SUCCESS删除空的try-catch。并显示真实错误。在删除try-catch后,我已更新了问题中的日志您的算法:1。
您将传入流文件读入flowFile
中2。
然后将一些内容写入传入流文件flowFile=session.write(flowFile…
3.
然后将传入的流文件添加到数组flowFiles我需要用java代码生成groovyscript。因此,我使用相同的groovy代码模板生成一对一和一对多映射(使用循环生成一对多流文件)。我只想如果if块中的条件不满足,它应该在else部分中删除流文件,并在一对一和一对多流文件生成的情况下记录相同的内容。我只想使用两个或一个groovy脚本代码/模板,这两个脚本代码/模板将适用于带条件的一对一和一对多流文件生成。请编辑您的问题并提供了输入和预期多个输出的示例,但在我的案例中,为什么多流文件生成的第二个代码可以工作。如果条件不满足,它也会删除该流文件。我解释了为什么您的代码不能工作。您没有创建新的流文件。
{"ENVIRONMENT":{"DATE":"170619","PM2.5":"10","TIME":"114635","DATE_TIME":"170619","NS":"NM","MAC":"6CECEB5D0302"}}
2019-10-07 12:08:29,237 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.script.ExecuteScript ExecuteScript[id=716b08ff-7a78-3806-bdd6-99d57cfd7cb7] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: javax.script.ScriptException: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=c8217e1d-ec25-4017-a175-25ef12ad64ba,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1570308702876-1, container=default, section=1], offset=53731, length=114],offset=0,name=c8217e1d-ec25-4017-a175-25ef12ad64ba,size=114] is already marked for transfer: org.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: javax.script.ScriptException: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=c8217e1d-ec25-4017-a175-25ef12ad64ba,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1570308702876-1, container=default, section=1], offset=53731, length=114],offset=0,name=c8217e1d-ec25-4017-a175-25ef12ad64ba,size=114] is already marked for transfer
org.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: javax.script.ScriptException: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=c8217e1d-ec25-4017-a175-25ef12ad64ba,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1570308702876-1, container=default, section=1], offset=53731, length=114],offset=0,name=c8217e1d-ec25-4017-a175-25ef12ad64ba,size=114] is already marked for transfer
at org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:248)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.script.ScriptException: javax.script.ScriptException: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=c8217e1d-ec25-4017-a175-25ef12ad64ba,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1570308702876-1, container=default, section=1], offset=53731, length=114],offset=0,name=c8217e1d-ec25-4017-a175-25ef12ad64ba,size=114] is already marked for transfer
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:162)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
at org.apache.nifi.script.impl.GroovyScriptEngineConfigurator.eval(GroovyScriptEngineConfigurator.java:54)
at org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:232)
... 11 common frames omitted
Caused by: javax.script.ScriptException: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=c8217e1d-ec25-4017-a175-25ef12ad64ba,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1570308702876-1, container=default, section=1], offset=53731, length=114],offset=0,name=c8217e1d-ec25-4017-a175-25ef12ad64ba,size=114] is already marked for transfer
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:324)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:159)
... 14 common frames omitted
Caused by: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=c8217e1d-ec25-4017-a175-25ef12ad64ba,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1570308702876-1, container=default, section=1], offset=53731, length=114],offset=0,name=c8217e1d-ec25-4017-a175-25ef12ad64ba,size=114] is already marked for transfer
at org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:3139)
at org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:3121)
at org.apache.nifi.controller.repository.StandardProcessSession.remove(StandardProcessSession.java:1979)
at org.apache.nifi.processor.ProcessSession$remove$3.call(Unknown Source)
at Script107.run(Script107.groovy:40)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:321)
... 15 common frames omitted
{
"ENV": [{
"MAC": "6CECEB5D0302",
"NS": "NM",
"DATE": "170619",
"TIME": "114635",
"PM2.5": "7.222410585417936",
"PM10": "8.272681196722012"
},
{
"MAC": "6CECEB5D0302",
"NS": "NM",
"DATE": "170619",
"TIME": "114635",
"PM2.5": "7.222410585417936",
"PM10": "8.272681196722012"
}
]
}
import groovy.json.*;
import org.apache.commons.io.IOUtils;
import java.nio.charset.*;
def flowFile = session.get()
if(!flowFile) return
def flowFiles = [] as List<FlowFile>
def inputStream = session.read(flowFile)
def writer = new StringWriter();
try {
IOUtils.copy(inputStream, writer, "UTF-8");
data=writer.toString();
inputStream.close();
def input = new JsonSlurper().parseText( data );
for (def i=0;i<input.ENV.size();i++){
if( Double.parseDouble(input.ENV[i]["PM2.5"])>7.2) {
def newFlowFile = session.create(flowFile);
Map out =
[ENVIRONMENT: [
DATE: input.ENV[i].DATE,
"PM2.5": input.ENV[i]["PM2.5"],
TIME: input.ENV[i].TIME,
DATE_TIME: input.ENV[i].DATE,
NS: input.ENV[i].NS,
MAC: input.ENV[i].MAC
]];String finalJson = JsonOutput.toJson out;
newFlowFile = session.write(newFlowFile, { outputStream ->
outputStream.write( finalJson.getBytes(StandardCharsets.UTF_8) )
} as OutputStreamCallback)
flowFiles << newFlowFile
}
else{
}
}} catch (IOException e) {
}
session.transfer(flowFiles, REL_SUCCESS)
session.remove(flowFile)