Java 连接器任务状态无法连接
连接器的任务状态失败,错误为: 我正在分布式模式下运行kafka connect群集,我正在使用ambari部署的kafka(0.10.0.2.5)connect 我使用export-CLASSPATH=/path-to-connector/给出了debezium mysql连接器路径 使用以下命令将连接器配置加载到Kafka Connect中:Java 连接器任务状态无法连接,java,apache-kafka,apache-kafka-connect,debezium,Java,Apache Kafka,Apache Kafka Connect,Debezium,连接器的任务状态失败,错误为: 我正在分布式模式下运行kafka connect群集,我正在使用ambari部署的kafka(0.10.0.2.5)connect 我使用export-CLASSPATH=/path-to-connector/给出了debezium mysql连接器路径 使用以下命令将连接器配置加载到Kafka Connect中: curl -i -X POST -H "Accept:application/json" \ -H "Content-Type:appli
curl -i -X POST -H "Accept:application/json" \
-H "Content-Type:application/json" http://localhost:8083/connectors/ \
-d '{
"name": "MYSQL_CONNECTOR",
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"database.hostname": "10.224.21.36",
"database.port": "3306",
"database.user": "root",
"database.password": "shobhna",
"database.server.id": "1",
"database.server.name": "demo",
"database.history.kafka.bootstrap.servers": "slnxhadoop04.noid.in:6669",
"database.history.kafka.topic": "dbhistory.demo" ,
"include.schema.changes": "true"
}
}'
现在检查连接器状态后,我发现错误:
- {"name":"MYSQL_CONNECTOR","connector":{"state":"RUNNING","worker_id":"172.26.177.115:8083"},
"tasks":[{"state":"FAILED","trace":"org.apache.kafka.connect.errors.ConnectException:
java.lang.NoClassDefFoundError:
org/apache/kafka/clients/admin/AdminClient\n\tat
io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:218)\n\tat
io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:45)\n\tat
org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:137)\n\tat
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140)\n\tat
org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175)\n\tat
java.util.concurrent.Executors$RunnableAdapter.cal(Executors.java:511)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat
java.lang.Thread.run(Thread.java:745)\nCaused by:
java.lang.NoClassDefFoundError:
org/apache/kafka/clients/admin/AdminClient\n\tat
io.debezium.relational.history.KafkaDatabaseHistory.initializeStorage(KafkaDatabaseHistory.java:336)\n\tat
io.debezium.connector.mysql.MySqlSchema.intializeHistoryStorage(MySqlSchema.java:260)\n\tat
io.debezium.connector.mysql.MySqlTaskContext.initializeHistoryStorage(MySqlTaskContext.java:194)\n\tat
io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:126)\n\t...
9 more\nCaused by: java.lang.ClassNotFoundException:
org.apache.kafka.clients.admin.AdminClient \n\tat
java.net.URLClassLoader.findClass(URLClassLoader.java:381)\n\tat
java.lang.ClassLoader.loadClass(ClassLoader.java:424)\n\tat
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)\n\tat
java.lang.ClassLoader.loadClass(ClassLoader.java:357)\n\t
它找不到内置的Kafka类,也找不到连接器 NoClassDefFoundError: org/apache/kafka/clients/admin/AdminClient 我正在使用卡夫卡(0.10.0.2.5) 确保正在运行1)与Kafka broker相匹配的Connect服务器版本2)使用的连接器使用该版本Connect的代码 例如,AdminClient仅存在于Kafka 0.11+中 在最近的HDP版本中,您可以看到Kafka 1.1(不同于0.11),这是最新Debezium的构建和测试版本 Debezium需要AdminClient来生成和注册主题信息,所以我不确定它是否能在旧版本(如0.10)上工作
Kafka wiki中指出,较新版本的ConnectServer可以与旧代理进行通信,但连接器类使用的协议仍存在争议 粘贴并在问题中包含错误,而不是附加屏幕截图。是的,请尝试Kafka 0.11或更高版本。我认为它是可行的,尽管最近我们将测试限制在1.x。我们还可以解决缺少
AdminClient
的问题,前提是之前已经手动创建了所需的主题。请随意为此创建一个JIRA问题。一般来说,我们现在关注的是Kafka>=1.0,但如果不是太多的努力,我们可能可以应用这个更改。谢谢@Gunnar-请随时代表我创建JIRA。哈哈,玩得好;)申请跟踪此要求。
- {"name":"MYSQL_CONNECTOR","connector":{"state":"RUNNING","worker_id":"172.26.177.115:8083"},
"tasks":[{"state":"FAILED","trace":"org.apache.kafka.connect.errors.ConnectException:
java.lang.NoClassDefFoundError:
org/apache/kafka/clients/admin/AdminClient\n\tat
io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:218)\n\tat
io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:45)\n\tat
org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:137)\n\tat
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140)\n\tat
org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175)\n\tat
java.util.concurrent.Executors$RunnableAdapter.cal(Executors.java:511)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat
java.lang.Thread.run(Thread.java:745)\nCaused by:
java.lang.NoClassDefFoundError:
org/apache/kafka/clients/admin/AdminClient\n\tat
io.debezium.relational.history.KafkaDatabaseHistory.initializeStorage(KafkaDatabaseHistory.java:336)\n\tat
io.debezium.connector.mysql.MySqlSchema.intializeHistoryStorage(MySqlSchema.java:260)\n\tat
io.debezium.connector.mysql.MySqlTaskContext.initializeHistoryStorage(MySqlTaskContext.java:194)\n\tat
io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:126)\n\t...
9 more\nCaused by: java.lang.ClassNotFoundException:
org.apache.kafka.clients.admin.AdminClient \n\tat
java.net.URLClassLoader.findClass(URLClassLoader.java:381)\n\tat
java.lang.ClassLoader.loadClass(ClassLoader.java:424)\n\tat
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)\n\tat
java.lang.ClassLoader.loadClass(ClassLoader.java:357)\n\t