向java发送kafka字符串消息以在其他java上启动程序

Read.java

public class Read {
    public static void main(String[] args) {
        String conn = "db_url";
        String username = "*****";
        String pwd = "*****";
        String sql = "INSERT INTO table (coloumn) values (?)";

        Properties props = new Properties();
        props.put("bootstrap.servers", "10.247.36.174:3306");
        props.put("acks", "all");
        props.put("retries", 0);
        props.put("batch.size", 16384);
        props.put("linger.ms", 1);
        props.put("buffer.memory", 33554432);
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        Producer<String, String> producer = null;



        try (Connection con = DriverManager.getConnection(conn, username, pwd);
                PreparedStatement ps = con.prepareStatement(sql);
                BufferedReader br = new BufferedReader(new FileReader("All.log"));) {

            String line = null;

                    processMessages(line, br, listparam, ps);
                    break;

            }
        } catch (Exception ex) {
            System.err.println("Error in \n" + ex);
        } finally {
            producer = new KafkaProducer<>(props);
            String msg = "Done";
            producer.send(new ProducerRecord<String, String>("HelloKafka", msg));
            System.out.println("Sent: " + msg);

        }
        producer.close();

    }

    public static void processMessages(String line, BufferedReader br, List<String> list param, PreparedStatement ps)
            throws Exception {
        StringBuilder message = new StringBuilder();
        message.append(line);
        while ((line = br.readLine()) != null) {
            String firstWord = line.split(" ", 2)[0];
            if (listparam.contains(firstWord)) {
                ps.setString(1, message.toString());
                ps.executeUpdate();
                message.setLength(0);
                message.append(line);
            } else {
                message.append("\n" + line);
            }
        }

        if (message.length() > 0) {
            ps.setString(1, message.toString());
            ps.executeUpdate();
        }
    }
}

---

总结：要使用卡夫卡消费者/制作人，首先必须启动Zookeeper和卡夫卡经纪人

出于测试或开发目的，您可以使用以下方式启动：

文档： Docker图像：

---

如果你的卡夫卡准备好了，你可以开始使用它。您必须记住为bootstrap.server设置正确的值以供本地使用它通常是localhost:9092

请打印整个堆栈跟踪。@PrashantPandey您是指错误消息吗？已经补充：ipadress如何处理您的经纪人地址？这似乎不正确。IP地址是连接到数据库的数据库，对吗？还是不？让我在daniuLet us编辑一下。你知道替代的kafkaConsumer.poll不推荐使用吗。。除了使用poll，还有什么建议吗？@stephen，不推荐使用函数KafkaConsumer:：pollfinal long-timeoutMs。您应该使用：KafkaConsumer:：pollfinal Duration timeout

public class Retrieve {
    public static void main(String[] args) {
        String conn = "db_url";
        String username = "****";
        String pwd = "****";
        String sql = "SELECT * from table1";
        Properties props = new Properties();

        props.put("bootstrap.servers", "ipaddress");
        props.put("group.id", "group-1");
        props.put("enable.auto.commit", "true");
        props.put("auto.commit.interval.ms", "1000");
        props.put("auto.offset.reset", "earliest");
        props.put("session.timeout.ms", "30000");
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

        KafkaConsumer<String, String> kafkaConsumer = new KafkaConsumer<>(props);
        kafkaConsumer.subscribe(Arrays.asList("HelloKafka"));

        while (true) {
            ConsumerRecords<String, String> records = kafkaConsumer.poll(100);
            for (ConsumerRecord<String, String> record : records) {
                if (record.value().equals("Done")) {
                    try (Connection con = ...;
                        PreparedStatement ps = ....;) {
                        ResultSet rs =.....;
                        while (rs.next()) {
                            String rawData = rs.getString("RawData");
                        }
                    } catch (Exception ex) {
                        System.err.println("Error in \n" + ex);
                    }
                }
            }
        }
    }
}

14:23:00.482 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node -1 disconnected.
14:23:00.482 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.532 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.583 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.633 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.683 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.733 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.784 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.834 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.885 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.935 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:00.985 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.036 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.086 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.137 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.187 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.238 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.289 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.339 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.389 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.439 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
14:23:01.490 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.247.36.174:3306 (id: -1 rack: null) for sending metadata request
14:23:01.490 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.247.36.174:3306 (id: -1 rack: null)
14:23:02.012 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -1
14:23:02.012 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Completed connection to node -1. Fetching API versions.
14:23:02.012 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating API versions fetch from node -1.
14:23:02.520 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.247.36.174 disconnected
java.io.EOFException: null