Docker 如何在CI服务器上将Kafka用作测试服务？

是否有人设法在CI环境中将Kafka设置为服务？我目前正在尝试使用

lensesio/fastdatadev

docker映像。也尝试过融合图像，见下文

我有以下用于运行测试的CI作业。通过Gitlab CI服务将

ADV_HOST

设置为同一docker网络上的kafka容器。尝试联系卡夫卡容器时，作业挂起

使用lensesio/fast数据开发的GitLab CI作业

测试：
阶段：测试
变量：
主持人：卡夫卡
禁用：azure documentdb、区块链、bloomberg、cassandra、coap、druid、elastic、elastic 5、ftp、hazelcast、hbase、influxdb、jms、kudu、mongodb、mqtt、redis、Reinsight、voltdb、yahoo、hdfs、jdbc、elasticsearch、s3、twitter
连接堆：512m
迷你水桶：图片
样本数据：0
剩余端口：8082
转发日志：0
运行测试：0
禁用_JMX:1
WEB\u端口：0
禁用：配置单元-1.1
##
#服务
#-卡夫卡
#-莫斯奎托（MQTT）
#-迷你（S3）
##
服务：
-名称：lensesio/fast数据开发：2.5.1-L0
别名：卡夫卡
-名称：dcs3spp/minio:version-1.0.2
别名：minio
-名称：EclipseMosquitto:1.6.9
别名：mqtt
脚本：
- >
dotnet测试--无恢复
--记录器：trx
--设置：测试/coverlet.runsettings
--收集：“XPlat代码覆盖率”
WebApp.sln

试图联系卡夫卡容器时CI作业挂起

info:WebApp.Kafka.Admin.kafkaadmin服务[0]
管理员服务正在尝试创建卡夫卡主题。。。
信息：WebApp.Kafka.Admin.kafkaadmin服务[0]
主题：：eventbus，ReplicationCount：：1，PartitionCount：：3
信息：WebApp.Kafka.Admin.kafkaadmin服务[0]
引导服务器：：kafka:9092

合流卡夫卡码头工人图片/

测试：
阶段：测试
变量：
ZOOKEEPER_客户端_端口：2181
动物园管理员时间：2000
卡夫卡经纪人编号：1
卡夫卡大学动物园管理员连接：动物园管理员：2181
KAFKA_播发的_监听器：明文：//KAFKA:29092，明文：//localhost:9092
卡夫卡侦听器安全协议映射：明文：明文，明文主机：明文
卡夫卡·国际经纪人·听众·姓名：明文
卡夫卡偏移量主题复制系数：1
##
#服务
#-卡夫卡
#-莫斯奎托（MQTT）
#-迷你（S3）
##
服务：
-名称：confluentinc/cp动物园管理员：5.1.0
别名：动物园管理员
-名称：confluentinc/cp卡夫卡：5.1.0
别名：卡夫卡
-名称：dcs3spp/minio:version-1.0.2
别名：minio
-名称：EclipseMosquitto:1.6.9
别名：mqtt
脚本：
- >
dotnet测试--无恢复
--记录器：trx
--设置：测试/coverlet.runsettings
--收集：“XPlat代码覆盖率”
WebApp.sln

融合图像错误日志

2020-11-15T12:15:44.558992574Z [2020-11-15 12:15:44,536] INFO Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.558996094Z [2020-11-15 12:15:44,536] INFO Server environment:java.io.tmpdir=/tmp (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.558999139Z [2020-11-15 12:15:44,542] INFO Server environment:java.compiler=<NA> (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.559002735Z [2020-11-15 12:15:44,542] INFO Server environment:os.name=Linux (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.559006873Z [2020-11-15 12:15:44,542] INFO Server environment:os.arch=amd64 (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.559009945Z [2020-11-15 12:15:44,542] INFO Server environment:os.version=4.19.78-coreos (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.559012987Z [2020-11-15 12:15:44,542] INFO Server environment:user.name=root (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.559015990Z [2020-11-15 12:15:44,542] INFO Server environment:user.home=/root (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.559019219Z [2020-11-15 12:15:44,544] INFO Server environment:user.dir=/ (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.646950748Z [2020-11-15 12:15:44,642] INFO tickTime set to 2000 (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.646965707Z [2020-11-15 12:15:44,642] INFO minSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.646969289Z [2020-11-15 12:15:44,642] INFO maxSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
2020-11-15T12:15:44.748601072Z [2020-11-15 12:15:44,725] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
2020-11-15T12:15:44.780680437Z [2020-11-15 12:15:44,754] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
*********
Pulling docker image mcr.microsoft.com/dotnet/core/sdk:3.1-alpine ...
Using docker image sha256:b0c526e8732fdcf06a1cc277f04523b4d3f10a6554d2b9df855e683524ee7ddf for mcr.microsoft.com/dotnet/core/sdk:3.1-alpine with digest mcr.microsoft.com/dotnet/core/sdk@sha256:3982ac41d8777b78ad7a2efe4c9674338975ebf9a25eeceb943348e45edf91b1 ...
Preparing environment
00:01
ERROR: Job failed (system failure): prepare environment: Error response from daemon: Cannot link to a non running container: /runner-z3wu8uu--project-20752619-concurrent-0-5f56b4323d8253b2-confluentinc__cp-kafka-1 AS /runner-z3wu8uu--project-20752619-concurrent-0-5f56b4323d8253b2-predefined-0/kafka (docker.go:817:0s).

2020-11-15T12:15:44.558992574Z[2020-11-15 12:15:44536]信息服务器环境：java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.558996094Z[2020-11-15 12:15:44536]信息服务器环境：java.io.tmpdir=/tmp（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.558999139Z[2020-11-15 12:15:44542]信息服务器环境：java.compiler=（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.559002735Z[2020-11-15 12:15:44542]信息服务器环境：os.name=Linux（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.559006873Z[2020-11-15 12:15:44542]信息服务器环境：os.arch=amd64（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.559009945Z[2020-11-15 12:15:44542]信息服务器环境：os.version=4.19.78-coreos（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.559012987Z[2020-11-15 12:15:44542]信息服务器环境：user.name=root（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.559015990Z[2020-11-15 12:15:44542]信息服务器环境：user.home=/root（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.559019219Z[2020-11-15 12:15:44544]信息服务器环境：user.dir=/（org.apache.zookeeper.Server.ZooKeeperServer）
2020-11-15T12:15:44.646950748Z[2020-11-15 12:15:44642]信息时间设置为2000（org.apache.zookeeper.server.ZooKeeperServer）
2020-11-15T12:15:44.646965707Z[2020-11-15 12:15:44642]信息分钟会话超时设置为-1（org.apache.zookeeper.server.ZooKeeperServer）
2020-11-15T12:15:44.646969289Z[2020-11-15 12:15:44642]信息maxSessionTimeout设置为-1（org.apache.zookeeper.server.ZooKeeperServer）
2020-11-15T12:15:44.748601072Z[2020-11-15 12:15:44725]使用org.apache.zookeeper.server.NIOServerCnxnFactory作为服务器连接工厂的信息（org.apache.zookeeper.server.ServerCnxnFactory）
2020-11-15T12:15:44.780680437Z[2020-11-15 12:15:44754]绑定到端口0.0.0.0/0.0.0.0:2181的信息（org.apache.zookeeper.server.NIOServerCnxnFactory）
*********
正在提取docker映像mcr.microsoft.com/dotnet/core/sdk:3.1-alpine。。。
使用docker映像sha256:b0c526e8732fdcf06a1cc277f04523b4d3f10a6554d2b9df855e683524ee7ddf for mcr.microsoft.com/dotnet/core/sdk:3.1-alpine和摘要mcr.microsoft.com/dotnet/core/sdk@sha256：3982ac41d8777b78ad7a2efe4c9674338975ebf9a25eeceb943348e45edf91b1。。。
准备环境
00:01
错误：作业失败（系统故障）：准备环境：守护进程的错误响应：无法链接到非运行容器：/runner-z3wu8uu-project-20752619-concurrent-0-5f56b4323d8253b2-concurrent inc\uu cp-kafka-1 AS/runner-z3wu8uu-project-2075219-concurrent-0-5f56b4323d8253b2-预定义-0/kafka（docker.go:817:0s）。

是由kafka容器启动尝试连接到我在“kafka_播发的侦听器”变量中设置的kafka容器引起的错误。例如，它无法连接到kafka:29092，因为容器尚未启动？

阅读此aspnetcore后，发现问题在于实现 我的

IHostedService

i

using System;
using System.Threading;
using System.Threading.Tasks;

using Confluent.Kafka;
using Confluent.Kafka.Admin;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

using KafkaAdmin.Kafka.Config;


namespace KafkaAdmin.Kafka
{
    public delegate IAdminClient KafkaAdminFactory(KafkaConfig config);

    /// <summary>Background Service to make a request from Kafka to create a topic</summary>
    public class KafkaAdminService : BackgroundService, IDisposable
    {
        private KafkaAdminFactory _Factory { get; set; }
        private ILogger<KafkaAdminService> _Logger { get; set; }
        private KafkaConfig _Config { get; set; }


        /// <summary>
        /// Retrieve KafkaConfig from appsettings
        /// </summary>
        /// <param name="config">Config POCO from appsettings file</param>
        /// <param name="clientFactory"><see cref="KafkaAdminFactory"/></param>
        /// <param name="logger">Logger instance</param>
        public KafkaAdminService(
            IOptions<KafkaConfig> config,
            KafkaAdminFactory clientFactory,
            ILogger<KafkaAdminService> logger)
        {
            if (clientFactory == null)
                throw new ArgumentNullException(nameof(clientFactory));

            if (config == null)
                throw new ArgumentNullException(nameof(config));

            _Config = config.Value ?? throw new ArgumentNullException(nameof(config));
            _Factory = clientFactory ?? throw new ArgumentNullException(nameof(clientFactory));
            _Logger = logger ?? throw new ArgumentNullException(nameof(logger));
        }


        /// <summary>
        /// Create a Kafka topic if it does not already exist
        /// </summary>
        /// <param name="token">Cancellation token required by IHostedService</param>
        /// <exception name="CreateTopicsException">
        /// Thrown for exceptions encountered except duplicate topic
        /// </exception>
        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            using (var client = _Factory(_Config))
            {
                try
                {
                    _Logger.LogInformation("Admin service trying to create Kafka Topic...");
                    _Logger.LogInformation($"Topic::{_Config.Topic.Name}, ReplicationCount::{_Config.Topic.ReplicationCount}, PartitionCount::{_Config.Topic.PartitionCount}");
                    _Logger.LogInformation($"Bootstrap Servers::{_Config.Consumer.BootstrapServers}");

                    await client.CreateTopicsAsync(new TopicSpecification[] {
                        new TopicSpecification {
                            Name = _Config.Topic.Name,
                            NumPartitions = _Config.Topic.PartitionCount,
                            ReplicationFactor = _Config.Topic.ReplicationCount
                        }
                    }, null);

                    _Logger.LogInformation($"Admin service successfully created topic {_Config.Topic.Name}");
                }
                catch (CreateTopicsException e)
                {
                    if (e.Results[0].Error.Code != ErrorCode.TopicAlreadyExists)
                    {
                        _Logger.LogInformation($"An error occured creating topic {_Config.Topic.Name}: {e.Results[0].Error.Reason}");
                        throw e;
                    }
                    else
                    {
                        _Logger.LogInformation($"Topic {_Config.Topic.Name} already exists");
                    }
                }
            }

            _Logger.LogInformation("Kafka Consumer thread started");

            await Task.CompletedTask;
        }


        /// <summary>
        /// Call base class dispose
        /// </summary>
        public override void Dispose()
        {
            base.Dispose();
        }
    }
}