Apache flink 为什么我的flink程序不加入两条流?

Apache flink 为什么我的flink程序不加入两条流?,apache-flink,flink-streaming,Apache Flink,Flink Streaming,我想加入Customer并根据id寻址对象。 这些是我对客户主题的kafka流的输入 {"id": 1,"name": "Yogesh"} {"id": 2,"name": "Swati" } {"id": 3,"name": "Shruti"} {"id": 4,"name": "Amol" } {"id": 5,"name": "Pooja" } {"id": 6,"name": "Kiran" } 下面是fro地址 {"id": 1,"address":"Pune" } {"id":

我想加入Customer并根据id寻址对象。 这些是我对客户主题的kafka流的输入

{"id": 1,"name": "Yogesh"}
{"id": 2,"name": "Swati" }
{"id": 3,"name": "Shruti"}
{"id": 4,"name": "Amol"  }
{"id": 5,"name": "Pooja" }
{"id": 6,"name": "Kiran" }
下面是fro地址

{"id": 1,"address":"Pune" }
{"id": 2,"address":"Pune" }
{"id": 3,"address":"Pune" }
{"id": 4,"address":"Kalyan"}
{"id": 5,"address": "Pimpri"}
我使用了interval join以及使用TumblingEventItemWindows和滑动窗口的JoinFunction,但它没有连接客户和地址流。我不明白我在代码中遗漏了什么

public static void main(String[] args) throws Exception {
            final StreamExecutionEnvironment env = setupEnvironment();
            final MultipleParameterTool params = MultipleParameterTool.fromArgs(args);
            env.getConfig().setGlobalJobParameters(params);
            env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
            init(params);

            FlinkKafkaConsumerBase<String> flinkKafkaConsumer = new FlinkKafkaConsumer<>(customerTopic,
                    new SimpleStringSchema(), CommonConfig.getConsumerProperties(ip, port, customerTopic))
                            .setStartFromEarliest();

            DataStream<Customer> customerStream = env.addSource(flinkKafkaConsumer)
                    .flatMap(new FlatMapFunction<String, Customer>() {
                        private static final long serialVersionUID = 2142214034515856836L;

                        @Override
                        public void flatMap(String value, Collector<Customer> out) throws Exception {
                            Customer customer = null;
                            try {
                                customer = mapper.readValue(value, Customer.class);
                            } catch (Exception exception) {
                                System.out.println(exception);
                            }
                            if (null != customer) {
                                out.collect(customer);
                            }
                        }
                    });

            customerStream.print();

            DataStream<Address> addressStream = env
                    .addSource(new FlinkKafkaConsumer<>(addressTopic, new SimpleStringSchema(),
                            CommonConfig.getConsumerProperties(ip, port, addressTopic)).setStartFromEarliest())
                    .flatMap(new FlatMapFunction<String, Address>() {
                        private static final long serialVersionUID = 2142214034515856836L;

                        @Override
                        public void flatMap(String value, Collector<Address> out) throws Exception {
                            Address address = null;
                            try {
                                address = mapper.readValue(value, Address.class);
                            } catch (Exception exception) {
                                System.out.println(exception);
                            }
                            if (null != address) {
                                out.collect(address);
                            }
                        }
                    });

            addressStream.print();

            customerStream.keyBy(new IdSelectorCustomer()).intervalJoin(addressStream.keyBy(new IdSelectorAddress()))
                    .between(Time.seconds(-2), Time.seconds(1))
                    .process(new ProcessJoinFunction<Customer, Address, CustomerInfo>() {
                        private static final long serialVersionUID = -3658796606815087434L;
                        @Override
                        public void processElement(Customer customer, Address address,
                                ProcessJoinFunction<Customer, Address, CustomerInfo>.Context ctx,
                                Collector<CustomerInfo> collector) throws Exception {
                            collector.collect(new CustomerInfo(customer.getId(), customer.getName(), address.getAddress()));
                        }
                    }).print();

            DataStream<CustomerInfo> joinResultStream = customerStream.join(addressStream).where(new IdSelectorCustomer())
                    .equalTo(new IdSelectorAddress()).window(TumblingEventTimeWindows.of(Time.seconds(5)))
                    .apply(new JoinFunction<Customer, Address, CustomerInfo>() {
                        private static final long serialVersionUID = -8913244745978230585L;
                        @Override
                        public CustomerInfo join(Customer first, Address second) throws Exception {
                            return new CustomerInfo(first.getId(), first.getName(), second.getAddress());
                        }
                    });
            joinResultStream.print();
            env.execute("Execute");
}

    // ===============================================================================
    public class IdSelectorAddress implements KeySelector<Address,Long> {
        private static final long serialVersionUID = 7642739595630647992L;
        @Override
        public Long getKey(Address value) throws Exception {
            return value.getId();
        }
    }

    // ========================================================================
    public class IdSelectorCustomer implements KeySelector<Customer,Long> {
        private static final long serialVersionUID = 7642739595630647992L;
        @Override
        public Long getKey(Customer value) throws Exception {
            return value.getId();
        }
    }
publicstaticvoidmain(字符串[]args)引发异常{
最终StreamExecutionEnvironment env=setupEnvironment();
最终MultipleParameterTool参数=MultipleParameterTool.fromArgs(args);
env.getConfig().setGlobalJobParameters(参数);
环境setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
init(params);
FlinkKafkaConsumerBase flinkKafkaConsumer=新的flinkKafkaConsumer(customerTopic,
新的SimpleStringSchema(),CommonConfig.getConsumerProperties(ip、端口、customerTopic))
.setStartFromEarliest();
DataStream customerStream=env.addSource(flinkKafkaConsumer)
.flatMap(新的flatMap函数(){
私有静态最终长serialVersionUID=2142214034515856836L;
@凌驾
公共void flatMap(字符串值,收集器输出)引发异常{
客户=空;
试一试{
customer=mapper.readValue(值,customer.class);
}捕获(异常){
System.out.println(例外);
}
如果(空!=客户){
出、收(客户);
}
}
});
customerStream.print();
DataStream addressStream=env
.addSource(新的FlinkKafkaConsumer(addressTopic,新的SimpleStringSchema(),
CommonConfig.getConsumerProperties(ip、端口、地址主题)).setStartFromEarliest()
.flatMap(新的flatMap函数(){
私有静态最终长serialVersionUID=2142214034515856836L;
@凌驾
公共void flatMap(字符串值,收集器输出)引发异常{
地址=空;
试一试{
address=mapper.readValue(值,address.class);
}捕获(异常){
System.out.println(例外);
}
如果(空!=地址){
收款人(地址);
}
}
});
addressStream.print();
customerStream.keyBy(新IdSelectorCustomer()).intervalJoin(addressStream.keyBy(新IdSelectorAddress()))
.介于(时间秒(-2),时间秒(1))
.process(新的ProcessJoinFunction(){
私有静态最终长serialVersionUID=-3658796606815087434L;
@凌驾
public void processElement(客户、地址、,
ProcessJoinFunction.Context ctx,
收集器)引发异常{
collector.collect(新CustomerInfo(customer.getId()、customer.getName()、address.getAddress());
}
}).print();
DataStream joinResultStream=customerStream.join(addressStream).where(新IdSelectorCustomer())
.equalTo(新的IdSelectorAddress()).window(TumblingEventTimeWindows.of(Time.seconds(5)))
.apply(新函数(){
私有静态最终长serialVersionUID=-8913244745978230585L;
@凌驾
公共CustomerInfo加入(客户第一,地址第二)引发异常{
返回新的CustomerInfo(first.getId()、first.getName()、second.getAddress());
}
});
joinResultStream.print();
环境执行(“执行”);
}
// ===============================================================================
公共类IdSelectorAddress实现KeySelector{
私有静态最终长serialVersionUID=7642739595630647992L;
@凌驾
公共长getKey(地址值)引发异常{
返回值.getId();
}
}
// ========================================================================
公共类IdSelectorCustomer实现KeySelector{
私有静态最终长serialVersionUID=7642739595630647992L;
@凌驾
公共长getKey(客户值)引发异常{
返回值.getId();
}
}

我假设您没有看到任何结果。原因是您的窗口从未被触发/评估/关闭

您的事件没有时间戳。仍然可以设置
TimeCharacteristic.EventTime
。由于不分配时间戳和水印,Flink无法判断如何以窗口连接或间隔连接方式连接事件

使用
DataStream#assignTimestampsAndWatermarks
FlinkKafkaConsumer#assignTimestampsAndWatermarks
处理事件时间或将时间特征更改为
ProcessingTime


我希望这将引导您朝着正确的方向前进。

我想您不会看到任何结果。原因是您的窗口从未被触发/评估/关闭

您的事件没有时间戳。你仍然设置了
Tim