Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/apache-kafka/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache kafka Kafka流处理分支访问未筛选记录多个谓词_Apache Kafka_Apache Kafka Streams - Fatal编程技术网

Apache kafka Kafka流处理分支访问未筛选记录多个谓词

Apache kafka Kafka流处理分支访问未筛选记录多个谓词,apache-kafka,apache-kafka-streams,Apache Kafka,Apache Kafka Streams,我试图在SpringBoot Kafka Streams应用程序中对Kafka inputStream应用分支,我知道一旦我使用谓词进行分支,我就可以访问这样的不匹配记录,例如:从 现在,这将使用SpringBoot ConfigurationProperties创建具有以下类的对象列表 package com.acme.org; import org.springframework.boot.context.properties.ConfigurationProperties; import

我试图在SpringBoot Kafka Streams应用程序中对Kafka inputStream应用分支,我知道一旦我使用谓词进行分支,我就可以访问这样的不匹配记录,例如:从

现在,这将使用SpringBoot ConfigurationProperties创建具有以下类的对象列表

package com.acme.org;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;

import java.util.List;
import java.util.Map;

@Component
@ConfigurationProperties(prefix = "topologymappings")
public class ApplicationProps {

    public List<TopicMapping> topicmappings;
    public String inputtopic;
    public String defaulttopic;


    public String getInputtopic() {
        return inputtopic;
    }

    public void setInputtopic(String inputtopic) {
        this.inputtopic = inputtopic;
    }

    public List<TopicMapping> getTopicmappings() {
        return topicmappings;
    }

    public void setTopicmappings(List<TopicMapping> topicmappings) {
        this.topicmappings = topicmappings;
    }


    public String getDefaulttopic() {
        return defaulttopic;
    }

    public void setDefaulttopic(String defaulttopic) {
        this.defaulttopic = defaulttopic;
    }


    public static class TopicMapping {

        public String getFilter_key() {
            return filter_key;
        }

        public void setFilter_key(String filter_key) {
            this.filter_key = filter_key;
        }

        public String getFilter_value() {
            return filter_value;
        }

        public void setFilter_value(String filter_value) {
            this.filter_value = filter_value;
        }

        public String getOutput_topic() {
            return output_topic;
        }

        public void setOutput_topic(String output_topic) {
            this.output_topic = output_topic;
        }

        private String filter_key;
        private String filter_value;
        private String output_topic;

    }
}
package com.acme.org;
导入org.springframework.boot.context.properties.ConfigurationProperties;
导入org.springframework.stereotype.Component;
导入java.util.List;
导入java.util.Map;
@组成部分
@配置属性(前缀=“拓扑映射”)
公共类应用程序道具{
公开主题地图列表;
公共字符串输入;
公共字符串主题;
公共字符串getInputtopic(){
返回输入;
}
公共void设置输入选项(字符串输入选项){
this.inputOpic=inputOpic;
}
公共列表getTopicmappings(){
返回主题地图;
}
公共无效设置topicmappings(列出topicmappings){
this.topicmappings=topicmappings;
}
公共字符串getDefaulttopic(){
返回默认主题;
}
public void setDefaulttopic(字符串defaulttopic){
this.defaulttopic=defaulttopic;
}
公共静态类TopicMapping{
公共字符串getFilter_key(){
返回过滤键;
}
公共无效设置筛选器密钥(字符串筛选器密钥){
this.filter\u key=filter\u key;
}
公共字符串getFilter_value(){
返回filter_值;
}
公共void setFilter_值(字符串filter_值){
this.filter\u value=filter\u value;
}
公共字符串getOutput_topic(){
返回输出主题;
}
公共void设置输出主题(字符串输出主题){
this.output\u topic=输出主题;
}
私有字符串过滤器\u密钥;
私有字符串过滤器_值;
私有字符串输出\u主题;
}
}
然后,我将这个列表传递给我的拓扑类,并迭代每个谓词集,以使用for-each循环来分支记录

 for (ApplicationProps.TopicMapping topMap: appProps.getTopicmappings() //iterate through List of Application config objects
             ) {

            branches = inputStream.branch(new Predicate<String, String>() {
                @Override
                public boolean test(String key, String value) {
                    boolean innerFlag = false;  //this flag can't be accessed outside this inner class 
                    if (value.contains("\"" + topMap.getFilter_key() + "\"" + ":\"" + topMap.getFilter_value() + "\"")) {
                        logger.info("Matched record->"+key+":"+value+"::" + topMap.getFilter_key() + ":" + topMap.getFilter_value());
                        return true;
                    }

                    return false;
                }
            }); 
            branches[0].to(topMap.getOutput_topic()); //I'll basically have one branch for every run of my loop

        }
for(ApplicationProps.TopicMapping topMap:appProps.getTopicmappings()//遍历应用程序配置对象列表
) {
Branchs=inputStream.branch(新谓词(){
@凌驾
公共布尔测试(字符串键、字符串值){
boolean innerFlag=false;//无法在此内部类之外访问此标志
if(value.contains(“\”+topMap.getFilter\u key()+“\”+:\”+topMap.getFilter\u value()+“\”){
logger.info(“匹配记录->”+key+:“+value+”::“+topMap.getFilter_key()+”:“+topMap.getFilter_value());
返回true;
}
返回false;
}
}); 
分支[0]。到(topMap.getOutput_topic());//我的循环每次运行基本上都有一个分支
}
我的问题是:

  • 如何访问两个谓词都不匹配的记录?请注意,我有一个
    (value.contains(“\”“+topMap.getFilter\u key()+”\“+”:\”“+topMap.getFilter\u value()+”)
    ,它替换循环中的键和值,基本上运行两个谓词,我必须先运行两个筛选器,看看记录是否匹配,然后才能执行类似
    (键,值)的操作-true
    ,但是我不能在循环外执行此操作,因为inputStream仍将包含与我的筛选器匹配的所有原始消息

  • 对于我的主题中的每一条新消息,我的inputStream将只有一条记录,还是自我启动此应用程序以来的所有以前的记录

  • 在我的例子中,分支不会从源或inputStream中删除记录,在这种情况下,我可以将inputStream中所有不匹配/剩余的记录输出到我的默认主题

  • package com.acme.org;
    
    import org.springframework.boot.context.properties.ConfigurationProperties;
    import org.springframework.stereotype.Component;
    
    import java.util.List;
    import java.util.Map;
    
    @Component
    @ConfigurationProperties(prefix = "topologymappings")
    public class ApplicationProps {
    
        public List<TopicMapping> topicmappings;
        public String inputtopic;
        public String defaulttopic;
    
    
        public String getInputtopic() {
            return inputtopic;
        }
    
        public void setInputtopic(String inputtopic) {
            this.inputtopic = inputtopic;
        }
    
        public List<TopicMapping> getTopicmappings() {
            return topicmappings;
        }
    
        public void setTopicmappings(List<TopicMapping> topicmappings) {
            this.topicmappings = topicmappings;
        }
    
    
        public String getDefaulttopic() {
            return defaulttopic;
        }
    
        public void setDefaulttopic(String defaulttopic) {
            this.defaulttopic = defaulttopic;
        }
    
    
        public static class TopicMapping {
    
            public String getFilter_key() {
                return filter_key;
            }
    
            public void setFilter_key(String filter_key) {
                this.filter_key = filter_key;
            }
    
            public String getFilter_value() {
                return filter_value;
            }
    
            public void setFilter_value(String filter_value) {
                this.filter_value = filter_value;
            }
    
            public String getOutput_topic() {
                return output_topic;
            }
    
            public void setOutput_topic(String output_topic) {
                this.output_topic = output_topic;
            }
    
            private String filter_key;
            private String filter_value;
            private String output_topic;
    
        }
    }
    
     for (ApplicationProps.TopicMapping topMap: appProps.getTopicmappings() //iterate through List of Application config objects
                 ) {
    
                branches = inputStream.branch(new Predicate<String, String>() {
                    @Override
                    public boolean test(String key, String value) {
                        boolean innerFlag = false;  //this flag can't be accessed outside this inner class 
                        if (value.contains("\"" + topMap.getFilter_key() + "\"" + ":\"" + topMap.getFilter_value() + "\"")) {
                            logger.info("Matched record->"+key+":"+value+"::" + topMap.getFilter_key() + ":" + topMap.getFilter_value());
                            return true;
                        }
    
                        return false;
                    }
                }); 
                branches[0].to(topMap.getOutput_topic()); //I'll basically have one branch for every run of my loop
    
            }