Java 如何使用Producer将自定义对象发送到Kafka主题
我想将我的Account类和Producer一起发送到我的Kafka主题,然后我将使用Kafka流进行聚合。但是,我无法发送收到错误的对象: 原因:org.apache.kafka.common.KafkaException:bank.Account不是org.apache.kafka.common.serialization.Serializer的实例 我的制片人班:Java 如何使用Producer将自定义对象发送到Kafka主题,java,serialization,apache-kafka,kafka-producer-api,Java,Serialization,Apache Kafka,Kafka Producer Api,我想将我的Account类和Producer一起发送到我的Kafka主题,然后我将使用Kafka流进行聚合。但是,我无法发送收到错误的对象: 原因:org.apache.kafka.common.KafkaException:bank.Account不是org.apache.kafka.common.serialization.Serializer的实例 我的制片人班: public static void main(String[] args) { DataAccess d
public static void main(String[] args) {
DataAccess dataAccess = new DataAccess();
List<Account> accountList = dataAccess.read();
final Logger logger = LoggerFactory.getLogger(Producer.class);
Properties properties = new Properties();
properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,LongSerializer.class.getName());
properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,Account.class.getName());
KafkaProducer<Long,Account> producer = new KafkaProducer<Long, Account>(properties);
for (Account account : accountList) {
ProducerRecord<Long,Account> record = new ProducerRecord<Long, Account>("bank_account",account.getFromId(),account);
producer.send(record, new Callback() {
public void onCompletion(RecordMetadata recordMetadata, Exception e) {
if (e == null) {
logger.info("Record sent successfully. \n "+ "Topic : "+recordMetadata.topic() +"\n"+
"Partition : " + recordMetadata.partition() + "\n"+
"Offset : " +recordMetadata.offset() +"\n"+
"Timestamp: " +recordMetadata.timestamp() +"\n");
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
}
else{
logger.info("Error sending producer");
}
}
});
}
producer.flush();
producer.close();
}
所以我的问题是,我们如何向卡夫卡主题发送自定义对象?在那之后,我当然想使用这条消息。这一行
properties.setProperty(ProducerConfig.VALUE\u SERIALIZER\u CLASS\u CONFIG,Account.CLASS.getName());
你必须实现你自己的。它不能是普通类
有些人使用JSON进行序列化,有些人使用Avro或Protobuf。但是,将数据放入
字节[]
只是一个实现细节
//1
prop.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,StringSerializer.CLASS.getName());
prop.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,Employee.CLASS.getName());
//2
卡夫卡制作人=新卡夫卡制作人(道具);
员工emp=新员工(1,“阿伦”);
PRODUCTERRECORD PRODUCTERRECORD=新的PRODUCTERRECORD(“雅利安主题”,emp);
//3
导入org.apache.kafka.common.header.Headers;
导入org.apache.kafka.common.serialization.Serializer;
导入java.io.Serializable;
导入java.util.Map;
//由阿伦·辛格开发
公共类Employee实现可序列化的序列化程序{
整数empId;
字符串名称;
地址添加;
公职人员(){
}
公共雇员(整数empId、字符串empName、地址add){
this.empId=empId;
this.empName=empName;
this.add=添加;
}
公共整数getEmpId(){
返回empId;
}
公共字符串getEmpName(){
返回empName;
}
公共广播getAdd(){
返回添加;
}
public void setEmpId(整数empId){
this.empId=empId;
}
public void setEmpName(字符串empName){
this.empName=empName;
}
公共无效设置添加(地址添加){
this.add=添加;
}
公共无效配置(映射配置、布尔isKey){
}
公共字节[]序列化(字符串s,对象o){
返回新字节[0];
}
公共字节[]序列化(字符串主题、标题、对象数据){
返回新字节[0];
}
公众假期结束(){
}
}
谢谢你的回答,朋友。这是正确的,现在我可以发送我的对象与生产者。我现在陷入卡夫卡团队。我不能插入我的对象,也不能实时流。如果您有时间,请看以下内容:
KafkaProducer<Long,Account> producer = new KafkaProducer<Long, Account>(properties);
public class Account {
private long fromId;
private long amount;
private long toId;
private ZonedDateTime time;
}
//1
prop.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
prop.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Employee.class.getName());
//2
KafkaProducer<String, Employee> producer = new KafkaProducer(prop);
Employee emp = new Employee(1, "Arun");
ProducerRecord prodRecord = new ProducerRecord("aryan_topic", emp);
//3
import org.apache.kafka.common.header.Headers;
import org.apache.kafka.common.serialization.Serializer;
import java.io.Serializable;
import java.util.Map;
//Developed by Arun Singh
public class Employee implements Serializable, Serializer {
Integer empId;
String empName;
Address add;
public Employee() {
}
public Employee(Integer empId, String empName, Address add) {
this.empId = empId;
this.empName = empName;
this.add = add;
}
public Integer getEmpId() {
return empId;
}
public String getEmpName() {
return empName;
}
public Address getAdd() {
return add;
}
public void setEmpId(Integer empId) {
this.empId = empId;
}
public void setEmpName(String empName) {
this.empName = empName;
}
public void setAdd(Address add) {
this.add = add;
}
public void configure(Map configs, boolean isKey) {
}
public byte[] serialize(String s, Object o) {
return new byte[0];
}
public byte[] serialize(String topic, Headers headers, Object data) {
return new byte[0];
}
public void close() {
}
}