Java Apache Kafka的自定义主题路由';s mongoSourceConnector
我正在使用ApacheKafka和MongoSourceConnect来接收数据库更改,例如文档插入,并希望将其发送给Kafka。在我的文档字段中,我将发送一个主题键,该键将把默认卡夫卡主题重写为文档的字段主题值 为此,我尝试了smt ExtractTopic$值(io.confluent.connect.transforms.ExtractTopic) 我的配置如下所示:Java Apache Kafka的自定义主题路由';s mongoSourceConnector,java,mongodb,apache-kafka,connector,Java,Mongodb,Apache Kafka,Connector,我正在使用ApacheKafka和MongoSourceConnect来接收数据库更改,例如文档插入,并希望将其发送给Kafka。在我的文档字段中,我将发送一个主题键,该键将把默认卡夫卡主题重写为文档的字段主题值 为此,我尝试了smt ExtractTopic$值(io.confluent.connect.transforms.ExtractTopic) 我的配置如下所示: name=mongo-source connector.class=com.mongodb.kafka.connect.M
name=mongo-source
connector.class=com.mongodb.kafka.connect.MongoSourceConnector
tasks.max=1
# Connection and source configuration
connection.uri=<my_connection_uri>
database=<my_db>
collection=messages
#topic.prefix=
poll.max.batch.size=1000
poll.await.time.ms=5000
# Change stream options
pipeline=[{"$match": { "$or": [{"operationType": "insert"},{"operationType": "update"}]}}]
batch.size=0
change.stream.full.document=updateLookup
publish.full.document.only=true
collation=
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
key.converter.schemas.enable=true
value.converter.schemas.enable=true
# key.converter=org.apache.kafka.connect.json.JsonConverter
# value.converter=org.apache.kafka.connect.json.JsonConverter
transforms=ValueFieldExample
transforms.ValueFieldExample.type=io.confluent.connect.transforms.ExtractTopic$Value
transforms.ValueFieldExample.field=topic
但如果正在使用smt转换(ExtractTopic),则抛出:
Caused by: org.apache.kafka.connect.errors.DataException: Only Struct objects supported for [field extraction], found: java.lang.String
Caused by: org.apache.kafka.connect.errors.DataException: Only Struct objects supported for [field extraction], found: java.lang.String