Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/postgresql/10.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
每当在Postgresql中执行任何更改事件(如Insert Update Delete)时,DEBEZIUM KAFKA连接器就会抛出一个错误_Postgresql_Apache Kafka Connect_Debezium - Fatal编程技术网

每当在Postgresql中执行任何更改事件(如Insert Update Delete)时,DEBEZIUM KAFKA连接器就会抛出一个错误

每当在Postgresql中执行任何更改事件(如Insert Update Delete)时,DEBEZIUM KAFKA连接器就会抛出一个错误,postgresql,apache-kafka-connect,debezium,Postgresql,Apache Kafka Connect,Debezium,我在用Docker。我是卡夫卡连线的新手。我的用例是这样的,我有一个Postgres数据库,从那里我需要捕获关于卡夫卡主题的任何更改事件(INSERT-UPDATE-DELETE),并进一步处理它。但我被困在捕捉变化事件中。我正在浏览以下链接: 创建具有以下配置的连接器之后: {"name": "postgres-source", "config": {"connector.class":"io.de

我在用Docker。我是卡夫卡连线的新手。我的用例是这样的,我有一个Postgres数据库,从那里我需要捕获关于卡夫卡主题的任何更改事件(INSERT-UPDATE-DELETE),并进一步处理它。但我被困在捕捉变化事件中。我正在浏览以下链接:

创建具有以下配置的连接器之后:

{"name": "postgres-source",
  "config": {"connector.class":"io.debezium.connector.postgresql.PostgresConnector",
    "tasks.max":"1",
    "database.hostname": "postgres",
    "database.port": "5432",
    "database.user": "postgres",
    "database.password": "postgres",
    "database.dbname" : "students",
    "database.server.name": "dbserver15",
    "database.whitelist": "students",
    "database.history.kafka.bootstrap.servers": "kafka:9092",
    "database.history.kafka.topic": "schema-changes.students",
    "key.converter": "org.apache.kafka.connect.storage.StringConverter",
    "value.converter": "org.apache.kafka.connect.storage.StringConverter",
    "key.converter.schemas.enable": "false",
    "value.converter.schemas.enable": "true",
    "value.converter.schema.registry.url": "http://schema-registry:8081",
    "transforms": "unwrap",
    "transforms.unwrap.type": "io.debezium.transforms.UnwrapFromEnvelope"
  }
}
我正在使用以下命令捕获数据库中的快照/更改事件:

kafka控制台使用者--引导服务器localhost:9092--从一开始--主题dbserver15.public.admission

它以这种格式显示数据:Struct{student_id=1,gre=337,toefl=118}

但只要我对该表执行任何INSERT-UPDATE-DELETE操作,卡夫卡连接器就会抛出以下错误:

org.apache.kafka.connect.errors.ConnectException: An exception ocurred in the change event producer. This connector will be stopped.
    at io.debezium.connector.base.ChangeEventQueue.throwProducerFailureIfPresent(ChangeEventQueue.java:170)\n\tat io.debezium.connector.base.ChangeEventQueue.poll(ChangeEventQueue.java:151)
    at io.debezium.connector.postgresql.PostgresConnectorTask.poll(PostgresConnectorTask.java:156)\n\tat org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:244)
    at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:220)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.IllegalArgumentException: Invalid identifier: 
    at io.debezium.relational.TableIdParser$TableIdTokenizer.tokenize(TableIdParser.java:68)
    at io.debezium.text.TokenStream.start(TokenStream.java:445)
    at io.debezium.relational.TableIdParser.parse(TableIdParser.java:28)
    at io.debezium.relational.TableId.parse(TableId.java:39)\n\tat io.debezium.connector.postgresql.PostgresSchema.parse(PostgresSchema.java:218)
    at io.debezium.connector.postgresql.RecordsStreamProducer.process(RecordsStreamProducer.java:238)
    at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$streamChanges$1(RecordsStreamProducer.java:131)
    at io.debezium.connector.postgresql.connection.pgproto.PgProtoMessageDecoder.processMessage(PgProtoMessageDecoder.java:48)
    at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.deserializeMessages(PostgresReplicationConnection.java:265)
    at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.read(PostgresReplicationConnection.java:250)
    at io.debezium.connector.postgresql.RecordsStreamProducer.streamChanges(RecordsStreamProducer.java:131)
    at io.debezium.connector.postgresql.RecordsStreamProducer.lambda$start$0(RecordsStreamProducer.java:117)
    ... 5 more
以下是我研究过的解决方案:


也就是说,snapshot.mode设置为exported,这允许连接器执行无锁快照。
但是当我添加“snapshot.mode”=“exported”时,会抛出一个错误,说明,
“snapshot.mode”值“exported”无效:值必须是always、never、initial\u only、initial和custom中的一个


谁能再详细一点,解释一下我遗漏了什么。我想这与配置有关。

您能分享您正在捕获的表的名称吗?@Jiri Pechanec table name是:admission好的,请查看并确保您使用的是最新的Debezium和postgresimages@ZeeshanKareem你找到事情的真相了吗?我正在使用“debezium/debezium connector postgresql:1.4.0”,在添加“snapshot.mode”:“exported”到我的连接器配置后,我开始看到这些错误。您可以共享您捕获的表的名称吗?@Jiri Pechanec表名为:admissionok,请查看并确保您使用的是最新的Debezium和postgresimages@ZeeshanKareem你找到事情的真相了吗?我正在使用“debezium/debezium connector postgresql:1.4.0”,并在将“snapshot.mode”:“exported”添加到我的连接器配置后开始看到这些错误