java Kafka在docker中发送消息的开放文件(lsof)正以接近50/秒的速度增长
我的java web应用程序正在docker容器中运行。Java应用程序正在向Kafka发送消息(~300毫秒/小时)。服务器在几个小时内就崩溃了。现在我需要每隔几个小时重启一次服务器。我看到打开的文件数量在不断增加。以下命令的输出正在增加~50/秒java Kafka在docker中发送消息的开放文件(lsof)正以接近50/秒的速度增长,java,docker,apache-kafka,lsof,Java,Docker,Apache Kafka,Lsof,我的java web应用程序正在docker容器中运行。Java应用程序正在向Kafka发送消息(~300毫秒/小时)。服务器在几个小时内就崩溃了。现在我需要每隔几个小时重启一次服务器。我看到打开的文件数量在不断增加。以下命令的输出正在增加~50/秒 lsof | grep kafka | wc -l 我尝试过卡夫卡版本,从2.4.0到2.6.0。(服务器和Kafka客户端java)但问题仍然存在 **Java代码** public class KafkaProducer {
lsof | grep kafka | wc -l
我尝试过卡夫卡版本,从2.4.0到2.6.0。(服务器和Kafka客户端java)但问题仍然存在
**Java代码**
public class KafkaProducer {
private static KafkaProducer single_instance = null;
public Producer<String, KafkaRequest> kafkaProducer = null;
private KafkaProducer(String url)
{
Properties props = new Properties();
props.put("bootstrap.servers", url);
kafkaProducer = new org.apache.kafka.clients.producer.KafkaProducer<>(props, new StringSerializer(), new KafkaRequestSerializer());
}
public static KafkaProducer getInstance(String url)
{
if (single_instance == null)
single_instance = new KafkaProducer(url);
return single_instance;
}
同一容器中是否有Kafka服务器?如果您有多个分区,则可能会导致问题kafka正在另一个容器中运行。我只有一个分区,因为卷很小。我删除了singleton类,并在每次发送消息后调用close()。这有助于不断增加打开的文件。
public void postKafkaMessage(String topicName, KafkaRequest kafkaRequest, String key, String kafkaUrl) {
KafkaProducer kafkaProducer = KafkaProducer.getInstance(kafkaUrl);
kafkaProducer.kafkaProducer.send(new ProducerRecord<>(topicName, key, kafkaRequest));
}
**COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME**
java 23 1811 FlowRunne root 210u IPv4 95985 0t0 TCP 7adf829e2db1:37696->kafka.server_elk:9092 (ESTABLISHED)
java 23 1811 FlowRunne root 211u sock 0,8 0t0 534942 protocol: TCP
java 23 1811 FlowRunne root 212u sock 0,8 0t0 534943 protocol: TCP
java 23 1811 FlowRunne root 213r FIFO 0,11 0t0 96862 pipe
java 23 1811 FlowRunne root 214w FIFO 0,11 0t0 96862 pipe
java 23 1811 FlowRunne root 215u a_inode 0,12 0 8697 [eventpoll]
java 23 1811 FlowRunne root 216u sock 0,8 0t0 531973 protocol: TCP
java 23 1811 FlowRunne root 217u IPv4 96864 0t0 TCP 7adf829e2db1:37718->kafka.server_elk:9092 (ESTABLISHED)
java 23 1811 FlowRunne root 218u sock 0,8 0t0 534027 protocol: TCP
java 23 1811 FlowRunne root 219r FIFO 0,11 0t0 95990 pipe
java 23 1811 FlowRunne root 220w FIFO 0,11 0t0 95990 pipe
java 23 1811 FlowRunne root 221u a_inode 0,12 0 8697 [eventpoll]
java 23 1811 FlowRunne root 222u sock 0,8 0t0 534028 protocol: TCP
java 23 1811 FlowRunne root 223u IPv4 95992 0t0 TCP 7adf829e2db1:37728->kafka.server_elk:9092 (ESTABLISHED)
java 23 1811 FlowRunne root 224u sock 0,8 0t0 534030 protocol: TCP
java 23 1811 FlowRunne root 225r FIFO 0,11 0t0 95994 pipe
java 23 1811 FlowRunne root 226w FIFO 0,11 0t0 95994 pipe
java 23 1811 FlowRunne root 227u a_inode 0,12 0 8697 [eventpoll]
java 23 1811 FlowRunne root 228u sock 0,8 0t0 534032 protocol: TCP
java 23 1811 FlowRunne root 229u IPv4 98150 0t0 TCP 7adf829e2db1:37738->kafka.server_elk:9092 (ESTABLISHED)
java 23 1811 FlowRunne root 230u sock 0,8 0t0 534944 protocol: TCP
java 23 1811 FlowRunne root 231u sock 0,8 0t0 532895 protocol: TCP
java 23 1811 FlowRunne root 232r FIFO 0,11 0t0 98838 pipe
java 23 1811 FlowRunne root 233w FIFO 0,11 0t0 98838 pipe
java 23 1811 FlowRunne root 234u a_inode 0,12 0 8697 [eventpoll]
java 23 1811 FlowRunne root 235u sock 0,8 0t0 534035 protocol: TCP
java 23 1052 pool-8-th root 434u IPv4 111060 0t0 TCP 7adf829e2db1:38250->kafka.server_elk:9092 (ESTABLISHED)
java 23 1052 pool-8-th root 440u IPv4 112668 0t0 TCP 7adf829e2db1:38260->kafka.server_elk:9092 (ESTABLISHED)
java 23 1052 pool-8-th root 446u IPv4 113865 0t0 TCP 7adf829e2db1:38270->kafka.server_elk:9092 (ESTABLISHED)
java 23 1052 pool-8-th root 452u IPv4 111071 0t0 TCP 7adf829e2db1:38280->kafka.server_elk:9092 (ESTABLISHED)
java 23 1052 pool-8-th root 458u IPv4 113871 0t0 TCP 7adf829e2db1:38290->kafka.server_elk:9092 (ESTABLISHED)
java 23 1052 pool-8-th root 464u IPv4 113876 0t0 TCP 7adf829e2db1:38296->kafka.server_elk:9092 (ESTABLISHED)
java 23 1052 pool-8-th root 470u IPv4 111080 0t0 TCP 7adf829e2db1:38308->kafka.server_elk:9092 (ESTABLISHED)
java 23 1052 pool-8-th root 476u IPv4 112964 0t0 TCP 7adf829e2db1:38328->kafka.server_elk:9092 (ESTABLISHED)
java 23 1052 pool-8-th root 483u IPv4 111102 0t0 TCP 7adf829e2db1:38340->kafka.server_elk:9092 (ESTABLISHED)
java 23 464 kafka-pro root 1189u sock 0,8 0t0 614811 protocol: TCP
java 23 464 kafka-pro root 1190u sock 0,8 0t0 622360 protocol: TCP
java 23 464 kafka-pro root 1191u sock 0,8 0t0 536231 protocol: TCP
java 23 464 kafka-pro root 1192u sock 0,8 0t0 622361 protocol: TCP
java 23 464 kafka-pro root 1193u sock 0,8 0t0 536234 protocol: TCP
java 23 464 kafka-pro root 1194u sock 0,8 0t0 536235 protocol: TCP
java 23 464 kafka-pro root 1195u sock 0,8 0t0 616746 protocol: TCP
java 23 464 kafka-pro root 1196u sock 0,8 0t0 616747 protocol: TCP