Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/sql-server-2008/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Node.js 如何将数据发送到节点JS中的多个Kafka主题分区_Node.js_Apache Kafka_Kafka Producer Api - Fatal编程技术网

Node.js 如何将数据发送到节点JS中的多个Kafka主题分区

Node.js 如何将数据发送到节点JS中的多个Kafka主题分区,node.js,apache-kafka,kafka-producer-api,Node.js,Apache Kafka,Kafka Producer Api,在节点js应用程序中,当我尝试向Kafka主题发送消息时,所有消息都将进入分区0。这个主题是用4个分区创建的,我想用循环机制发布,我尝试了多个选项,但没有成功 有什么办法解决这个问题吗?下面是代码片段 payloads = [ { topic: 'test-topic', messages: ['TestMessage1', 'TestMessage2', 'TestMessage3', 'TestMessage4']}, ]; producer.on('ready', function

在节点js应用程序中,当我尝试向Kafka主题发送消息时,所有消息都将进入分区0。这个主题是用4个分区创建的,我想用循环机制发布,我尝试了多个选项,但没有成功

有什么办法解决这个问题吗?下面是代码片段

payloads = [
    { topic: 'test-topic', messages: ['TestMessage1', 'TestMessage2', 'TestMessage3', 'TestMessage4']},
];
producer.on('ready', function(){
    producer.send(payloads, function(err, data){
        console.log("Successfully written onto Kafka");
    });

在卡夫卡中,具有相同密钥的消息被放置在同一分区中。 您可以手动定义分区:

// Force partitioning - default partition is 0
payloads = [ 
    { topic: 'test-topic', messages: ['TestMessage1'], partition: 0 },
    { topic: 'test-topic', messages: ['TestMessage2'], partition: 1 },
    { topic: 'test-topic', messages: ['TestMessage3'], partition: 2 },
    { topic: 'test-topic', messages: ['TestMessage4'], partition: 3 }
];
或对每条消息使用不同的键:

payloads = [ 
    { topic: 'test-topic', messages: ['TestMessage1'], key: '1' },
    { topic: 'test-topic', messages: ['TestMessage2'], key: '2' },
    { topic: 'test-topic', messages: ['TestMessage3'], key: '3' },
    { topic: 'test-topic', messages: ['TestMessage4'], key: '4' }
];

// Alternatively, you can use KeyedMessage
km = new KeyedMessage('1', 'TestMessage1'),
km2 = new KeyedMessage('2', 'TestMessage2'),
payloads = [
    { topic: 'test-topic', messages: [ km , km2 ] },
];

在卡夫卡中,具有相同密钥的消息被放置在同一分区中。 您可以手动定义分区:

// Force partitioning - default partition is 0
payloads = [ 
    { topic: 'test-topic', messages: ['TestMessage1'], partition: 0 },
    { topic: 'test-topic', messages: ['TestMessage2'], partition: 1 },
    { topic: 'test-topic', messages: ['TestMessage3'], partition: 2 },
    { topic: 'test-topic', messages: ['TestMessage4'], partition: 3 }
];
或对每条消息使用不同的键:

payloads = [ 
    { topic: 'test-topic', messages: ['TestMessage1'], key: '1' },
    { topic: 'test-topic', messages: ['TestMessage2'], key: '2' },
    { topic: 'test-topic', messages: ['TestMessage3'], key: '3' },
    { topic: 'test-topic', messages: ['TestMessage4'], key: '4' }
];

// Alternatively, you can use KeyedMessage
km = new KeyedMessage('1', 'TestMessage1'),
km2 = new KeyedMessage('2', 'TestMessage2'),
payloads = [
    { topic: 'test-topic', messages: [ km , km2 ] },
];

你对此有把握吗?Kafka文档说,如果指定了有效的分区号,则在发送记录时将使用该分区。如果未指定分区,但存在密钥,则将使用密钥的哈希值选择分区。如果既不存在键也不存在分区,则将以循环方式分配分区这可能与卡夫卡节点库有关吗?你确定吗?Kafka文档说,如果指定了有效的分区号,则在发送记录时将使用该分区。如果未指定分区,但存在密钥,则将使用密钥的哈希值选择分区。如果既不存在键也不存在分区,则将以循环方式分配分区这可能只是与卡夫卡节点库有关吗?