Java 当群集关闭时，KafkaPublisher会阻塞

我正在使用kafka-clients-0.8.2.1.jar，发现当集群根本没有启动时，发送会被阻塞。我在网上找到了更多相关信息，发现这是许多人都面临的一个已知问题。 但是，我尝试了上面提到的一些选项，但没有帮助： 1.调用KafkaProducer.partitionsFor（），它仍然被阻止。 2.正在将producer.type设置为async，但无法识别

我想要实现的是检测集群关闭，排队消息（最大限制）并在集群返回时发送。如果这太复杂，至少它不会阻塞，因为它会导致应用程序将所有消息排队并耗尽内存

代码：

---

新的producer API将在其内部缓冲区已满时阻塞（这没有很好的文档记录）。您可以将属性

block.on.buffer.full

添加到

false

，然后您将获得

bufferdepustedexception

---

看

这是非常正确的。如果缓冲区已满，仅通过阅读文档不可能得出生产者api将阻塞的结论。另外，需要注意的是，在版本0.11.0.0中删除了参数

block.on.buffer.full

，现在正确的参数名称是

max.block.ms

。

System.out.println("props:"
producer = new KafkaProducer<String, String>(props);


producer.partitionsFor(record.topic());
producer.send(record, new Callback() {

props:{queue.size=1000, reconnect.backoff.ms=10000, request.timeout.ms=1000, bootstrap.servers=tstaapp001:59092,ewdlxsrv283:59092,devcapp001:59092, value.serializer=org.apache.kafka.common.serialization.StringSerializer, request.required.acks=1, buffer.memory=33554432, retries=0, producer.type=async, key.serializer=org.apache.kafka.common.serialization.StringSerializer, linger.ms=1, topic.metadata.refresh.interval.ms=1000, batch.size=16384, timeout.ms=10000}
20:44:33.584 [RTEMPool-1-RTEMThread-4] DEBUG o.a.k.c.producer.internals.Metadata - Updated cluster metadata version 1 to Cluster(nodes = [Node(tstaapp001, 59092), Node(devcapp001, 59092), Node(ewdlxsrv283, 59092)], partitions = [])
20:44:33.603 [kafka-producer-network-thread | producer-1] DEBUG o.a.k.c.producer.internals.Sender - Starting Kafka producer I/O thread.
20:44:33.603 [RTEMPool-1-RTEMThread-4] WARN  o.a.k.c.producer.ProducerConfig - The configuration topic.metadata.refresh.interval.ms = null was supplied but isn't a known config.
20:44:33.603 [RTEMPool-1-RTEMThread-4] WARN  o.a.k.c.producer.ProducerConfig - The configuration request.timeout.ms = null was supplied but isn't a known config.
20:44:33.603 [RTEMPool-1-RTEMThread-4] WARN  o.a.k.c.producer.ProducerConfig - The configuration producer.type = null was supplied but isn't a known config.
20:44:33.604 [RTEMPool-1-RTEMThread-4] WARN  o.a.k.c.producer.ProducerConfig - The configuration request.required.acks = null was supplied but isn't a known config.
20:44:33.604 [RTEMPool-1-RTEMThread-4] WARN  o.a.k.c.producer.ProducerConfig - The configuration queue.size = null was supplied but isn't a known config.
20:44:33.604 [RTEMPool-1-RTEMThread-4] DEBUG o.a.k.clients.producer.KafkaProducer - Kafka producer started
20:44:42.958 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:42.959 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Init connection to node -2 for sending metadata request in the next iteration
20:44:42.959 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -2 at ewdlxsrv283:59092.
20:44:42.964 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.063 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.163 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.264 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.364 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.464 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.564 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.664 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.764 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.864 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.964 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.064 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.164 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.264 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.364 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.464 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.479 [kafka-producer-network-thread | producer-1] WARN  o.a.kafka.common.network.Selector - Error in I/O with null
java.net.ConnectException: Connection refused: no further information