Apache nifi MiNiFi教程失败,工具包错误:连接id为。。。具有无效的目标id

Apache nifi MiNiFi教程失败,工具包错误:连接id为。。。具有无效的目标id,apache-nifi,apache-minifi,Apache Nifi,Apache Minifi,我遵循了本教程: 因此,在我的根进程组和MiNiFi的进程组中有一个输入端口。MiNiFi进程组包含一个尾部文件处理器、一个远程进程组以及它们之间的连接 在MiNiFi进程组中我正在保存模板,然后从模板列表下载模板并运行命令: minifi-toolkit-0.5.0\bin\config.bat transform minifi\MiNiFi2.xml minifi\minifi_config\config.yml 但它失败了,说: The JAVA_HOME environment var

我遵循了本教程:

因此,在我的根进程组和MiNiFi的进程组中有一个
输入端口。
MiNiFi进程组
包含一个
尾部文件处理器
、一个
远程进程组
以及它们之间的连接

MiNiFi进程组中
我正在保存模板,然后从模板列表下载模板并运行命令:

minifi-toolkit-0.5.0\bin\config.bat transform minifi\MiNiFi2.xml minifi\minifi_config\config.yml
但它失败了,说:

The JAVA_HOME environment variable is not defined correctly.
Instead the PATH will be used to find the java executable.

There are validation errors with the template, still outputting YAML but it will need to be edited.
Connection with id 10de7342-41ef-37a5-0000-000000000000 has invalid destination id 9b02eb45-3c2d-3fb4-91cf-880d5038c0cd
我重新创造了一切,问题依然存在

这是我的模板:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.3">
    <description></description>
    <groupId>db4ce9c5-016e-1000-bce2-b531930a6edd</groupId>
    <name>MiNiFi</name>
    <snippet>
        <connections>
            <id>10de7342-41ef-37a5-0000-000000000000</id>
            <parentGroupId>e544a343-e2f0-3112-0000-000000000000</parentGroupId>
            <backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
            <backPressureObjectThreshold>10000</backPressureObjectThreshold>
            <destination>
                <groupId>1e18bdb9-7df3-3fba-0000-000000000000</groupId>
                <id>9b02eb45-3c2d-3fb4-91cf-880d5038c0cd</id>
                <type>REMOTE_INPUT_PORT</type>
            </destination>
            <flowFileExpiration>0 sec</flowFileExpiration>
            <labelIndex>1</labelIndex>
            <loadBalanceCompression>DO_NOT_COMPRESS</loadBalanceCompression>
            <loadBalancePartitionAttribute></loadBalancePartitionAttribute>
            <loadBalanceStatus>LOAD_BALANCE_NOT_CONFIGURED</loadBalanceStatus>
            <loadBalanceStrategy>DO_NOT_LOAD_BALANCE</loadBalanceStrategy>
            <name></name>
            <selectedRelationships>success</selectedRelationships>
            <source>
                <groupId>e544a343-e2f0-3112-0000-000000000000</groupId>
                <id>246f7d78-f8e1-3c88-0000-000000000000</id>
                <type>PROCESSOR</type>
            </source>
            <zIndex>0</zIndex>
        </connections>
        <processors>
            <id>246f7d78-f8e1-3c88-0000-000000000000</id>
            <parentGroupId>e544a343-e2f0-3112-0000-000000000000</parentGroupId>
            <position>
                <x>0.0</x>
                <y>16.0</y>
            </position>
            <bundle>
                <artifact>nifi-standard-nar</artifact>
                <group>org.apache.nifi</group>
                <version>1.10.0</version>
            </bundle>
            <config>
                <bulletinLevel>WARN</bulletinLevel>
                <comments></comments>
                <concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
                <descriptors>
                    <entry>
                        <key>tail-mode</key>
                        <value>
                            <name>tail-mode</name>
                        </value>
                    </entry>
                    <entry>
                        <key>File to Tail</key>
                        <value>
                            <name>File to Tail</name>
                        </value>
                    </entry>
                    <entry>
                        <key>Rolling Filename Pattern</key>
                        <value>
                            <name>Rolling Filename Pattern</name>
                        </value>
                    </entry>
                    <entry>
                        <key>tail-base-directory</key>
                        <value>
                            <name>tail-base-directory</name>
                        </value>
                    </entry>
                    <entry>
                        <key>Initial Start Position</key>
                        <value>
                            <name>Initial Start Position</name>
                        </value>
                    </entry>
                    <entry>
                        <key>File Location</key>
                        <value>
                            <name>File Location</name>
                        </value>
                    </entry>
                    <entry>
                        <key>tailfile-recursive-lookup</key>
                        <value>
                            <name>tailfile-recursive-lookup</name>
                        </value>
                    </entry>
                    <entry>
                        <key>tailfile-lookup-frequency</key>
                        <value>
                            <name>tailfile-lookup-frequency</name>
                        </value>
                    </entry>
                    <entry>
                        <key>tailfile-maximum-age</key>
                        <value>
                            <name>tailfile-maximum-age</name>
                        </value>
                    </entry>
                </descriptors>
                <executionNode>ALL</executionNode>
                <lossTolerant>false</lossTolerant>
                <penaltyDuration>30 sec</penaltyDuration>
                <properties>
                    <entry>
                        <key>tail-mode</key>
                        <value>Single file</value>
                    </entry>
                    <entry>
                        <key>File to Tail</key>
                        <value>/opt/minifi/minifi_test_dir</value>
                    </entry>
                    <entry>
                        <key>Rolling Filename Pattern</key>
                    </entry>
                    <entry>
                        <key>tail-base-directory</key>
                    </entry>
                    <entry>
                        <key>Initial Start Position</key>
                        <value>Beginning of File</value>
                    </entry>
                    <entry>
                        <key>File Location</key>
                        <value>Local</value>
                    </entry>
                    <entry>
                        <key>tailfile-recursive-lookup</key>
                        <value>false</value>
                    </entry>
                    <entry>
                        <key>tailfile-lookup-frequency</key>
                        <value>10 minutes</value>
                    </entry>
                    <entry>
                        <key>tailfile-maximum-age</key>
                        <value>24 hours</value>
                    </entry>
                </properties>
                <runDurationMillis>0</runDurationMillis>
                <schedulingPeriod>0 sec</schedulingPeriod>
                <schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
                <yieldDuration>1 sec</yieldDuration>
            </config>
            <executionNodeRestricted>false</executionNodeRestricted>
            <name>TailFile</name>
            <relationships>
                <autoTerminate>false</autoTerminate>
                <name>success</name>
            </relationships>
            <state>STOPPED</state>
            <style/>
            <type>org.apache.nifi.processors.standard.TailFile</type>
        </processors>
        <remoteProcessGroups>
            <id>1e18bdb9-7df3-3fba-0000-000000000000</id>
            <parentGroupId>e544a343-e2f0-3112-0000-000000000000</parentGroupId>
            <position>
                <x>696.0</x>
                <y>0.0</y>
            </position>
            <comments></comments>
            <communicationsTimeout>30 sec</communicationsTimeout>
            <contents>
                <inputPorts>
                    <batchSettings/>
                    <comments></comments>
                    <concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
                    <connected>true</connected>
                    <exists>true</exists>
                    <groupId>db4d2e20-016e-1000-d3c7-73a67e2b1d67</groupId>
                    <id>9b02eb45-3c2d-3fb4-91cf-880d5038c0cd</id>
                    <name>input_minifi</name>
                    <targetId>dadd7f29-016e-1000-3c91-89495c813674</targetId>
                    <targetRunning>false</targetRunning>
                    <transmitting>false</transmitting>
                    <useCompression>false</useCompression>
                </inputPorts>
            </contents>
            <proxyHost></proxyHost>
            <proxyUser></proxyUser>
            <targetUri>http://nifi-server:8080/nifi</targetUri>
            <targetUris>http://nifi-server:8080/nifi</targetUris>
            <transportProtocol>RAW</transportProtocol>
            <yieldDuration>10 sec</yieldDuration>
        </remoteProcessGroups>
    </snippet>
    <timestamp>12/06/2019 13:04:03 UTC</timestamp>
</template>
编辑:

工具包(由@seb报告)中似乎存在一个bug,其中targetId在模板中用作输入端口id,但连接id没有正确映射

以前的:

如果我没记错的话,工具箱希望它处理的模板是完全自包含的

模板正在引用根进程组中的输入端口

这将导致在仅导出模板并转换为与MiNiFi一起使用时丢失该模板

我建议将您的输入端口向下移动到要创建模板的流程组中。

编辑:

工具包(由@seb报告)中似乎存在一个bug,其中targetId在模板中用作输入端口id,但连接id没有正确映射

以前的:

如果我没记错的话,工具箱希望它处理的模板是完全自包含的

模板正在引用根进程组中的输入端口

这将导致在仅导出模板并转换为与MiNiFi一起使用时丢失该模板


我建议将您的输入端口向下移动到要创建模板的流程组中。

但这似乎是unituitive,而不是要在MiNiFi中运行的模板。为什么MiNiFi要处理一个应该是远程的输入端口?(我只添加了一个输入url的远程PG,输入端口由NiFi添加)该模板用于逻辑,应仅在MiNiFi中运行,您的NiFi流应单独设计。通过在指向NiFi实例的模板中放置一个远程输出端口,然后将其导出,可以将数据从MiNiFi发送到NiFi。通过在NiFi流中设置远程输出端口并将其指向模板中的输入端口,您可以将数据从NiFi发送到MiNiFi。感谢您的帮助。我尝试了输出端口,但没有定义远程主机的设置。RPG似乎是正确的做法,如本文所述:对不起,我说错了,远程进程组就是我所说的“远程输出端口”。看起来您可以通过从该模板中删除targetId来解决此问题,介意尝试一下吗?但这似乎是unituitive,不是在MiNiFi中运行的模板。为什么MiNiFi要处理一个应该是远程的输入端口?(我只添加了一个输入url的远程PG,输入端口由NiFi添加)该模板用于逻辑,应仅在MiNiFi中运行,您的NiFi流应单独设计。通过在指向NiFi实例的模板中放置一个远程输出端口,然后将其导出,可以将数据从MiNiFi发送到NiFi。通过在NiFi流中设置远程输出端口并将其指向模板中的输入端口,您可以将数据从NiFi发送到MiNiFi。感谢您的帮助。我尝试了输出端口,但没有定义远程主机的设置。正如这里所描述的,RPG似乎是正确的做法:对不起,我说错了,远程进程组就是我所说的“远程输出端口”。看起来您可以通过从模板中删除targetId来解决这个问题,介意试试吗?
MiNiFi Config Version: 3
Flow Controller:
  name: MiNiFi
  comment: ''
Core Properties:
  flow controller graceful shutdown period: 10 sec
  flow service write delay interval: 500 ms
  administrative yield duration: 30 sec
  bored yield duration: 10 millis
  max concurrent threads: 1
  variable registry properties: ''
FlowFile Repository:
  partitions: 256
  checkpoint interval: 2 mins
  always sync: false
  Swap:
    threshold: 20000
    in period: 5 sec
    in threads: 1
    out period: 5 sec
    out threads: 4
Content Repository:
  content claim max appendable size: 10 MB
  content claim max flow files: 100
  always sync: false
Provenance Repository:
  provenance rollover time: 1 min
  implementation: org.apache.nifi.provenance.MiNiFiPersistentProvenanceRepository
Component Status Repository:
  buffer size: 1440
  snapshot frequency: 1 min
Security Properties:
  keystore: ''
  keystore type: ''
  keystore password: ''
  key password: ''
  truststore: ''
  truststore type: ''
  truststore password: ''
  ssl protocol: ''
  Sensitive Props:
    key:
    algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
    provider: BC
Processors:
- id: 246f7d78-f8e1-3c88-0000-000000000000
  name: TailFile
  class: org.apache.nifi.processors.standard.TailFile
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list: []
  Properties:
    File Location: Local
    File to Tail: /opt/minifi/minifi_test_dir
    Initial Start Position: Beginning of File
    Rolling Filename Pattern:
    tail-base-directory:
    tail-mode: Single file
    tailfile-lookup-frequency: 10 minutes
    tailfile-maximum-age: 24 hours
    tailfile-recursive-lookup: 'false'
Controller Services: []
Process Groups: []
Input Ports: []
Output Ports: []
Funnels: []
Connections:
- id: 10de7342-41ef-37a5-0000-000000000000
  name: TailFile/success/9b02eb45-3c2d-3fb4-91cf-880d5038c0cd
  source id: 246f7d78-f8e1-3c88-0000-000000000000
  source relationship names:
  - success
  destination id: 9b02eb45-3c2d-3fb4-91cf-880d5038c0cd
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
Remote Process Groups:
- id: 1e18bdb9-7df3-3fba-0000-000000000000
  name: ''
  url: http://nifi-server:8080/nifi
  comment: ''
  timeout: 30 sec
  yield period: 10 sec
  transport protocol: RAW
  proxy host: ''
  proxy port: ''
  proxy user: ''
  proxy password: ''
  local network interface: ''
  Input Ports:
  - id: dadd7f29-016e-1000-3c91-89495c813674
    name: input_minifi
    comment: ''
    max concurrent tasks: 1
    use compression: false
  Output Ports: []
NiFi Properties Overrides: {}