Apache kafka Flink Kafka使用者在使用DataStream键时引发空指针异常
我正在使用这个示例,在这里我分离数据,因为我已经创建了一个应用程序,它正在将应用程序发送到Kafka&另一个应用程序正在读取Kafka。。。我为类TemperatureWarning生成了制作人,即在卡夫卡,我正在发送与TemperatureWarning相关的数据。下面是我的代码,它正在使用卡夫卡的数据Apache kafka Flink Kafka使用者在使用DataStream键时引发空指针异常,apache-kafka,kafka-consumer-api,apache-flink,flink-streaming,flink-cep,Apache Kafka,Kafka Consumer Api,Apache Flink,Flink Streaming,Flink Cep,我正在使用这个示例,在这里我分离数据,因为我已经创建了一个应用程序,它正在将应用程序发送到Kafka&另一个应用程序正在读取Kafka。。。我为类TemperatureWarning生成了制作人,即在卡夫卡,我正在发送与TemperatureWarning相关的数据。下面是我的代码,它正在使用卡夫卡的数据 StreamExecutionEnvironment env=StreamExecutionEnvironment.getExecutionEnvironment(); env.setStre
StreamExecutionEnvironment env=StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
env.enableCheckpointing(5000);
Properties properties=new Properties();
properties.setProperty("bootstrap.servers", "PUBLICDNS:9092");
properties.setProperty("zookeeper.connect", "PUBLICDNS:2181");
properties.setProperty("group.id", "test");
DataStream<TemperatureWarning> dstream=env.addSource(new FlinkKafkaConsumer09<TemperatureWarning>("MonitoringEvent", new MonitoringEventSchema(), properties));
Pattern<TemperatureWarning, ?> alertPattern = Pattern.<TemperatureWarning>begin("first")
.next("second")
.within(Time.seconds(20));
PatternStream<TemperatureWarning> alertPatternStream = CEP.pattern(
dstream.keyBy("rackID"),
alertPattern);
DataStream<TemperatureAlert> alerts = alertPatternStream.flatSelect(
(Map<String, TemperatureWarning> pattern, Collector<TemperatureAlert> out) -> {
TemperatureWarning first = pattern.get("first");
TemperatureWarning second = pattern.get("second");
if (first.getAverageTemperature() < second.getAverageTemperature()) {
out.collect(new TemperatureAlert(second.getRackID(),second.getAverageTemperature(),second.getTimeStamp()));
}
});
dstream.print();
alerts.print();
env.execute("Flink Kafka Consumer");
}
以下是我的班级监控文采:
public class MonitoringEventSchema implements DeserializationSchema<TemperatureWarning>,SerializationSchema<TemperatureWarning>
公共类MonitoringEventSchema实现反序列化模式、序列化模式
{
@覆盖
公共类型信息getProducedType(){
//TODO自动生成的方法存根
返回null;
}
@凌驾
公共字节[]序列化(TemperatureWarning元素){
//TODO自动生成的方法存根
返回元素.toString().getBytes();
}
@凌驾
public TemperatureWarning反序列化(字节[]消息)引发IOException{
//TODO自动生成的方法存根
如果(消息!=null)
{
String str=新字符串(消息“UTF-8”);
字符串[]val=str.split(“,”);
TemperatureWarning警告=新的TemperatureWarning(Integer.parseInt(val[0])、Double.parseDouble(val[1])、Long.parseLong(val[2]);
返回警告;
}
返回null;
}
@凌驾
公共布尔值isEndOfStream(温度警告下一个远程事件){
//TODO自动生成的方法存根
返回false;
}
}
现在需要做什么keyBy操作,正如我提到的流分区所需的密钥??这里需要做什么来解决此错误???此函数中存在问题:
@Override
public TypeInformation<TemperatureWarning> getProducedType() {
// TODO Auto-generated method stub
return null;
}
@覆盖
公共类型信息getProducedType(){
//TODO自动生成的方法存根
返回null;
}
此处不能返回null。非常感谢。。。我使用了
returntypeextractor.getForClass(TemperatureWarning.class)
解决了这个错误。。。我只是想确认我反序列化DataStream类型类的方式,它是反序列化数据的正确方式吗??您能给我分享一个自定义KafkanConsumer示例吗?当KafkaFlinkConsumer无法java.lang.NullPointerException位于com.yash.source.MonitoringEventSchema.deserialize(MonitoringEventSchema.java:74)时,如何解决此错误在org.apache.flink.streaming.util.serialization.KeyedDeserializationSchemaWrapper.deserialize(KeyedDeserializationSchemaWrapper.java:39)org.apache.flink.streaming.connectors.kafka.flinkafkaConsumer09$ConsumerThread.run上的com.yash.source.MonitoringEventSchemaWtschema.反序列化(MonitoringEventSchemaWrapper.java:1)(FlinkKafkaConsumer09.java:466)
public class MonitoringEventSchema implements DeserializationSchema<TemperatureWarning>,SerializationSchema<TemperatureWarning>
@Override
public TypeInformation<TemperatureWarning> getProducedType() {
// TODO Auto-generated method stub
return null;
}
@Override
public byte[] serialize(TemperatureWarning element) {
// TODO Auto-generated method stub
return element.toString().getBytes();
}
@Override
public TemperatureWarning deserialize(byte[] message) throws IOException {
// TODO Auto-generated method stub
if(message!=null)
{
String str=new String(message,"UTF-8");
String []val=str.split(",");
TemperatureWarning warning=new TemperatureWarning(Integer.parseInt(val[0]),Double.parseDouble(val[1]),Long.parseLong(val[2]));
return warning;
}
return null;
}
@Override
public boolean isEndOfStream(TemperatureWarning nextElement) {
// TODO Auto-generated method stub
return false;
}
@Override
public TypeInformation<TemperatureWarning> getProducedType() {
// TODO Auto-generated method stub
return null;
}