Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/apache-kafka/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache kafka kafka.common.FailedToSendMessageException:kafka生成错误_Apache Kafka_Kafka Producer Api - Fatal编程技术网

Apache kafka kafka.common.FailedToSendMessageException:kafka生成错误

Apache kafka kafka.common.FailedToSendMessageException:kafka生成错误,apache-kafka,kafka-producer-api,Apache Kafka,Kafka Producer Api,我正在读取一个json文件,并试图使用kafka生成它。。 这是我的密码: public class FlatFileDataProducer { private String topic = "JsonTopic"; private Producer<String, String> producer = null; KeyedMessage<String, String> message = null; public JsonReade

我正在读取一个json文件,并试图使用kafka生成它。。 这是我的密码:

public class FlatFileDataProducer {

    private String topic = "JsonTopic";
    private Producer<String, String> producer = null;
    KeyedMessage<String, String> message = null;
    public JsonReader reader;

    public void run(String jsonPath) throws ClassNotFoundException, FileNotFoundException, IOException, ParseException{
        reader = new JsonReader();
        System.out.println("---------------------");
        System.out.println("JSON FILE PATH IS : "+jsonPath);
        System.out.println("---------------------");
        Properties prop = new Properties();
        prop.put("metadata.broker.list", "192.168.63.145:9092");
        prop.put("serializer.class", "kafka.serializer.StringEncoder");
        // prop.put("partitioner.class", "example.producer.SimplePartitioner");
        prop.put("request.required.acks", "1");


        ProducerConfig config = new ProducerConfig(prop);
        producer = new Producer<String, String>(config);
        List<Employee> emp = reader.readJsonFile(jsonPath);     
        for (Employee employee : emp) 
        {
            System.out.println("---------------------");
            System.out.println(employee.toString());
            System.out.println("---------------------");
            message = new KeyedMessage<String, String>(topic, employee.toString());

            producer.send(message);
            producer.close();

        }
         System.out.println("Messages to Kafka successfully");
    }
它给出了一个错误,但当我检查cosumershell时,我得到如下结果:对于JSON文件中的一行,我在shell中看到4个条目。。 第二期:

> [root@sandbox ~]# java -jar sparkkafka.jar /root/customer.json
> JSON FILE PATH IS : /root/customer.json
>  log4j:WARN No appenders could be found for logger (kafka.utils.VerifiableProperties). log4j:WARN Please
> initialize the log4j system properly.
> 1,Smith,25
> Exception in thread "main" kafka.common.FailedToSendMessageException: Failed to send messages
> after 3 tries.
>         at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:91)
>         at kafka.producer.Producer.send(Producer.scala:77)
>         at kafka.javaapi.producer.Producer.send(Producer.scala:33)
>         at com.up.jsonType.FlatFileDataProducer.run(FlatFileDataProducer.java:41)
>         at com.up.jsonType.FlatFileDataProducer.main(FlatFileDataProducer.java:49)
[root@sandbox[垃圾箱]# [root@sandboxbin]#./kafka-console-consumer.sh——zookeeper本地主机:2181——主题JsonTopic——从头开始

1,Smith,25
1,Smith,25
1,Smith,25
1,Smith,25

对于相同的数据,我得到了4倍的行数。

您可以尝试添加以下属性:

prop.put("producer.type","async");

您需要删除以下两个属性:

    //prop.put("request.required.acks", "1");
    //prop.put("producer.type","async");
此属性将实际负责确认

    //prop.put("request.required.acks", "1");
    //prop.put("producer.type","async");