Java 由于重新平衡,无法完成提交

Java 由于重新平衡,无法完成提交,java,hadoop,apache-kafka,kafka-consumer-api,Java,Hadoop,Apache Kafka,Kafka Consumer Api,我的卡夫卡消费代码如下所示,我只有一个消费者 Properties consumerConfig = new Properties(); consumerConfig.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "sandbox.hortonworks.com:6667"); consumerConfig.put(ConsumerConfig.GROUP_ID_CONFIG, "my-group");

我的卡夫卡消费代码如下所示,我只有一个消费者

Properties consumerConfig = new Properties();
        consumerConfig.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "sandbox.hortonworks.com:6667");
        consumerConfig.put(ConsumerConfig.GROUP_ID_CONFIG, "my-group");
        consumerConfig.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
        consumerConfig.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
                "org.apache.kafka.common.serialization.StringDeserializer");
        consumerConfig.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
                "org.apache.kafka.common.serialization.StringDeserializer");
        @SuppressWarnings("resource")
        KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(consumerConfig);
        TestConsumerRebalanceListener rebalanceListener = new TestConsumerRebalanceListener();
        consumer.subscribe(Collections.singletonList("test1"), rebalanceListener);
        HDFSAppendTrial example = new HDFSAppendTrial();
        String coreSite = "/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/core-site.xml";
        String hdfsSite = "/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/hdfs-site.xml";
        String hdfsFilePath = "/appendTo/Trial.csv";

        while (true) {
            ConsumerRecords<String, String> records = consumer.poll(999999999);
            for (ConsumerRecord<String, String> record : records) {
                FileSystem fileSystem = example.configureFileSystem(coreSite, hdfsSite);
                String res = example.appendToFile(fileSystem, record.value(), hdfsFilePath);
                System.out.printf("%s\n", record.value());
                if (res.equalsIgnoreCase( "success")) {
                    System.out.println("Successfully appended to file");
                }
                else
                    System.out.println("couldn't append to file");
                example.closeFileSystem(fileSystem);

            }
            consumer.commitSync();


        }
Properties-consumerConfig=new-Properties();
consumerConfig.put(consumerConfig.BOOTSTRAP_SERVERS_CONFIG,“sandbox.hortonworks.com:6667”);
consumerConfig.put(consumerConfig.GROUP_ID_CONFIG,“我的组”);
consumerConfig.put(consumerConfig.AUTO_OFFSET_RESET_CONFIG,“最早”);
consumerConfig.put(consumerConfig.VALUE\u反序列化程序\u类\u配置,
“org.apache.kafka.common.serialization.StringDeserializer”);
consumerConfig.put(consumerConfig.KEY\u反序列化程序\u类\u配置,
“org.apache.kafka.common.serialization.StringDeserializer”);
@抑制警告(“资源”)
卡夫卡消费者=新卡夫卡消费者(消费者配置);
TestConsumerBalanceListener rebalanceListener=新建TestConsumerBalanceListener();
consumer.subscribe(Collections.singletonList(“test1”)、rebalanceListener);
HDFSAppendTrial示例=新的HDFSAppendTrial();
字符串coreSite=“/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/core-site.xml”;
字符串hdfsSite=“/usr/hdp/2.6.0.3-8/hadoop/etc/hadoop/hdfs site.xml”;
字符串hdfsFilePath=“/appendTo/Trial.csv”;
while(true){
ConsumerRecords records=consumer.poll(999999999);
对于(消费者记录:记录){
FileSystem FileSystem=example.configureFileSystem(coreSite、hdfsSite);
String res=example.appendToFile(文件系统,record.value(),hdfsFilePath);
System.out.printf(“%s\n”,record.value());
如果(同等信号案例(“成功”)){
System.out.println(“成功附加到文件”);
}
其他的
System.out.println(“无法附加到文件”);
closeFileSystem(文件系统);
}
consumer.commitSync();
}
在流式传输一些记录后,我收到以下错误

如何解决上述问题,并向我解释问题的原因,如何克服这些问题!有人能帮我为消费者编写更多的代码吗

提前感谢能够帮助我的人,也感谢至少试一试的人


consumerConfig.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");
consumerConfig.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG,"True");
consumerConfig.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG,"99998");
consumerConfig.put(ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG,"10000");
consumerConfig.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG,"99999");

通过在properties对象中添加上述属性,consumerConfig