Apache kafka 将Storm 1 Kafka拓扑转换为Heron,有几个问题

Apache kafka 将Storm 1 Kafka拓扑转换为Heron,有几个问题,apache-kafka,heron,Apache Kafka,Heron,正在尝试将Storm 1.0.6拓扑切换到Heron。一步一步地把卡夫卡的壶嘴移开,看看事情进展如何。主要方法如下(从原始通量版本修改): 相关POM条目: <dependency> <groupId>org.apache.heron</groupId> <artifactId>heron-api</artifactId> <version&g

正在尝试将Storm 1.0.6拓扑切换到Heron。一步一步地把卡夫卡的壶嘴移开,看看事情进展如何。主要方法如下(从原始通量版本修改):

相关POM条目:

        <dependency>
            <groupId>org.apache.heron</groupId>
            <artifactId>heron-api</artifactId>
            <version>0.20.3-incubating</version>
        </dependency>

        <dependency>
            <groupId>org.apache.heron</groupId>
            <artifactId>heron-storm</artifactId>
            <version>0.20.3-incubating</version>
        </dependency>

        <dependency>
            <groupId>org.apache.storm</groupId>
            <artifactId>storm-kafka</artifactId>
            <version>1.0.6</version>
        </dependency>
问题1:拓扑立即退出,是否有相当于Flux'--sleep'的Eco标志使其运行一段时间(以进行调试等)

问题2:我有点惊讶于我需要加入storm kafka(本以为会有一个Heron等价物)-这是正确的(或其他神器)吗?如果是,1.0.6是可以使用的版本,还是Heron与另一个版本配合更好

问题3:以上是YAML中的
类型:“storm”
,尝试
类型:“heron”
会出现以下错误:

INFO: Building spouts
Exception in thread "main" java.lang.ClassCastException: class org.apache.storm.kafka.KafkaSpout cannot be cast to class org.apache.heron.api.spout.IRichSpout (org.apache.storm.kafka.KafkaSpout and org.apache.heron.api.spout.IRichSpout are in unnamed module of loader 'app')
    at org.apache.heron.eco.builder.heron.SpoutBuilder.buildSpouts(SpoutBuilder.java:42)
    at org.apache.heron.eco.builder.heron.EcoBuilder.buildTopologyBuilder(EcoBuilder.java:70)
    at org.apache.heron.eco.Eco.submit(Eco.java:125)
    at org.apache.heron.eco.Eco.main(Eco.java:161)
    at KafkaTopology.main(KafkaTopology.java:26)

Process finished with exit code 1
这就是它使用卡夫卡的方式,类型需要是风暴而不是苍鹭,或者这里有一些解决方法吗

    • 有好几条卡夫卡给苍鹭的嘴。我使用Storm(Storm-kafka-client-2.1)克隆并在生产中使用它

    问题1
    :我不确定拓扑为什么会关闭。尝试使用
    --verbose
    标志运行提交。此时,
    --sleep
    参数的功能不存在。如果你需要的话,它可以是一个功能

    问题2
    :有一只苍鹭。在Heron被捐赠给Apache之后,为了发布二进制版本,需要做大量的工作。大部分工作已经完成。在下一个版本中,我希望所有二进制工件都能被适当地分发

    问题3
    :出现此问题的原因是,根据指定的类型,它在某个包中查找螺栓/喷嘴。当输入“storm”时,它希望它实现或扩展的类是“org.apache.storm”。当输入“heron”时,它希望它实现或扩展的类是“org.apache.heron”。如果使用依赖项storm kafka,则类型必须为“storm”。在这里可以找到与苍鹭相当的物种

            <dependency>
                <groupId>org.apache.heron</groupId>
                <artifactId>heron-api</artifactId>
                <version>0.20.3-incubating</version>
            </dependency>
    
            <dependency>
                <groupId>org.apache.heron</groupId>
                <artifactId>heron-storm</artifactId>
                <version>0.20.3-incubating</version>
            </dependency>
    
            <dependency>
                <groupId>org.apache.storm</groupId>
                <artifactId>storm-kafka</artifactId>
                <version>1.0.6</version>
            </dependency>
    
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.parser.EcoParser loadTopologyFromYaml
    INFO: Parsing eco config file
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.parser.EcoParser loadTopologyFromYaml
    INFO: Performing property substitution.
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.parser.EcoParser loadTopologyFromYaml
    INFO: Performing environment variable substitution.
    topology type is Storm
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.builder.storm.EcoBuilder buildConfig
    INFO: Building topology config
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.Eco printTopologyInfo
    INFO: ---------- TOPOLOGY DETAILS ----------
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.Eco printTopologyInfo
    INFO: Topology Name: kafkaTopology-XXX_topologyVersion_XXX
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.Eco printTopologyInfo
    INFO: --------------- SPOUTS ---------------
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.Eco printTopologyInfo
    INFO: kafka-spout [1] (org.apache.storm.kafka.KafkaSpout)
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.Eco printTopologyInfo
    INFO: ---------------- BOLTS ---------------
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.Eco printTopologyInfo
    INFO: --------------- STREAMS ---------------
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.Eco printTopologyInfo
    INFO: --------------------------------------
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.builder.storm.EcoBuilder buildTopologyBuilder
    INFO: Building components
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.builder.storm.EcoBuilder buildTopologyBuilder
    INFO: Building spouts
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.builder.storm.EcoBuilder buildTopologyBuilder
    INFO: Building bolts
    Apr 30, 2021 4:38:49 PM org.apache.heron.eco.builder.storm.EcoBuilder buildTopologyBuilder
    INFO: Building streams
    
    Process finished with exit code 0
    
    INFO: Building spouts
    Exception in thread "main" java.lang.ClassCastException: class org.apache.storm.kafka.KafkaSpout cannot be cast to class org.apache.heron.api.spout.IRichSpout (org.apache.storm.kafka.KafkaSpout and org.apache.heron.api.spout.IRichSpout are in unnamed module of loader 'app')
        at org.apache.heron.eco.builder.heron.SpoutBuilder.buildSpouts(SpoutBuilder.java:42)
        at org.apache.heron.eco.builder.heron.EcoBuilder.buildTopologyBuilder(EcoBuilder.java:70)
        at org.apache.heron.eco.Eco.submit(Eco.java:125)
        at org.apache.heron.eco.Eco.main(Eco.java:161)
        at KafkaTopology.main(KafkaTopology.java:26)
    
    Process finished with exit code 1