Apache kafka 卡夫卡：SASL#u SSL+；ACL可以产生但不能消耗

使用

kafka控制台制作人

我可以使用用户

write

使用

kafka控制台消费者

我无法以用户身份读取主题

acl

中的消息

但是，我可以登录，所有ACL都是正确的，当我使用错误的密码时，它会抱怨，所以SASL_SSL和ACL可以工作。在

kafka authorizer.log

中，启用

DEBUG

模式后：

[2019-10-12 20:33:08,647] DEBUG operation = Read on resource = Topic:LITERAL:acl from host = XXXXXXXX  is Allow based on acl = User:read has Allow permission for operations: All from hosts: * (kafka.authorizer.logger)
[2019-10-12 20:33:08,647] DEBUG Principal = User:read is Allowed Operation = Describe from host = XXXXXXXX  on resource = Topic:LITERAL:acl (kafka.authorizer.logger)
[2019-10-12 20:33:08,652] DEBUG operation = Read on resource = Group:LITERAL:aclRead from host = XXXXXXXX  is Allow based on acl = User:read has Allow permission for operations: Read from hosts: * (kafka.authorizer.logger)
[2019-10-12 20:33:08,652] DEBUG Principal = User:read is Allowed Operation = Describe from host = XXXXXXXX on resource = Group:LITERAL:aclRead (kafka.authorizer.logger)

卡夫卡请求.log中的

[2019-10-12 20:40:33,587] DEBUG Completed request:RequestHeader(apiKey=API_VERSIONS, apiVersion=2, clientId=read, correlationId=1) -- {},response:{error_code=0,api_versions=[{api_key=0,min_version=0,max_version=7},{api_key=1,min_version=0,max_version=11},{api_key=2,min_version=0,max_version=5},{api_key=3,min_version=0,max_version=8},{api_key=4,min_version=0,max_version=2},{api_key=5,min_version=0,max_version=1},{api_key=6,min_version=0,max_version=5},{api_key=7,min_version=0,max_version=2},{api_key=8,min_version=0,max_version=7},{api_key=9,min_version=0,max_version=5},{api_key=10,min_version=0,max_version=2},{api_key=11,min_version=0,max_version=5},{api_key=12,min_version=0,max_version=3},{api_key=13,min_version=0,max_version=2},{api_key=14,min_version=0,max_version=3},{api_key=15,min_version=0,max_version=3},{api_key=16,min_version=0,max_version=2},{api_key=17,min_version=0,max_version=1},{api_key=18,min_version=0,max_version=2},{api_key=19,min_version=0,max_version=3},{api_key=20,min_version=0,max_version=3},{api_key=21,min_version=0,max_version=1},{api_key=22,min_version=0,max_version=1},{api_key=23,min_version=0,max_version=3},{api_key=24,min_version=0,max_version=1},{api_key=25,min_version=0,max_version=1},{api_key=26,min_version=0,max_version=1},{api_key=27,min_version=0,max_version=0},{api_key=28,min_version=0,max_version=2},{api_key=29,min_version=0,max_version=1},{api_key=30,min_version=0,max_version=1},{api_key=31,min_version=0,max_version=1},{api_key=32,min_version=0,max_version=2},{api_key=33,min_version=0,max_version=1},{api_key=34,min_version=0,max_version=1},{api_key=35,min_version=0,max_version=1},{api_key=36,min_version=0,max_version=1},{api_key=37,min_version=0,max_version=1},{api_key=38,min_version=0,max_version=1},{api_key=39,min_version=0,max_version=1},{api_key=40,min_version=0,max_version=1},{api_key=41,min_version=0,max_version=1},{api_key=42,min_version=0,max_version=1},{api_key=43,min_version=0,max_version=0},{api_key=44,min_version=0,max_version=0}],throttle_time_ms=0} from connection 192.168.1.13:9094-XXXXXXXXXX:45642-4;totalTime:0.733,requestQueueTime:0.055,localTime:0.468,remoteTime:0.0,throttleTime:0.432,responseQueueTime:0.052,sendTime:0.172,securityProtocol:SASL_SSL,principal:User:read,listener:SASL_SSL (kafka.request.logger)
[2019-10-12 20:40:33,604] DEBUG Completed request:RequestHeader(apiKey=METADATA, apiVersion=8, clientId=read, correlationId=2) -- {topics=[{name=acl}],allow_auto_topic_creation=true,include_cluster_authorized_operations=false,include_topic_authorized_operations=false},response:{throttle_time_ms=0,brokers=[{node_id=2,host=kafka2.exmaple.com,port=9094,rack=null},{node_id=3,host=kafka3.exmaple.com,port=9094,rack=null},{node_id=1,host=kafka1.exmaple.com,port=9094,rack=null}],cluster_id=TIIhlmDsSv-wfmkf3PQA4w,controller_id=2,topics=[{error_code=0,name=acl,is_internal=false,partitions=[{error_code=0,partition_index=0,leader_id=1,leader_epoch=3,replica_nodes=[1,3],isr_nodes=[3,1],offline_replicas=[]},{error_code=0,partition_index=4,leader_id=2,leader_epoch=1,replica_nodes=[2,3],isr_nodes=[2,3],offline_replicas=[]},{error_code=0,partition_index=1,leader_id=2,leader_epoch=2,replica_nodes=[2,1],isr_nodes=[2,1],offline_replicas=[]},{error_code=0,partition_index=2,leader_id=2,leader_epoch=1,replica_nodes=[3,2],isr_nodes=[2,3],offline_replicas=[]},{error_code=0,partition_index=3,leader_id=1,leader_epoch=2,replica_nodes=[1,2],isr_nodes=[2,1],offline_replicas=[]}],topic_authorized_operations=0}],cluster_authorized_operations=0} from connection 192.168.1.13:9094-XXXXXXXXXXX:45642-4;totalTime:6.546,requestQueueTime:0.085,localTime:1.913,remoteTime:0.0,throttleTime:0.664,responseQueueTime:4.327,sendTime:0.242,securityProtocol:SASL_SSL,principal:User:read,listener:SASL_SSL (kafka.request.logger)
[2019-10-12 20:40:33,606] DEBUG Completed request:RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=2, clientId=read, correlationId=0) -- {key=aclRead,key_type=0},response:{throttle_time_ms=0,error_code=0,error_message=NONE,node_id=2,host=kafka2.exmaple.com,port=9094} from connection 192.168.1.13:9094-XXXXXXXXXXXX:45642-4;totalTime:1.463,requestQueueTime:0.047,localTime:1.209,remoteTime:0.0,throttleTime:0.251,responseQueueTime:0.055,sendTime:0.163,securityProtocol:SASL_SSL,principal:User:read,listener:SASL_SSL (kafka.request.logger)

这基本上意味着一切都很好

我已打开kafdrop安装，并能够连接到kafka群集。我可以看到那里的一切，从主题到消息（！）。但它表示，没有消费者与这个话题有关

当我关闭消费者时，它显示
总共处理了0条消息
我使用以下方法启动它：

bash kafka-console-consumer.sh--引导服务器kafka1.example.com:9094--主题acl--组aclRead--从一开始--consumer.config=/root/consumer.properties

consumer.properties内容：

security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username='read' password='blablabla';
ssl.truststore.location=/root/kafka.truststore.jks
ssl.truststore.password=blablabla

我的ACL正确，否则拒绝连接：

Current ACLs for resource `Group:LITERAL:aclRead`:
        User:read has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:LITERAL:acl`:
        User:read has Allow permission for operations: All from hosts: *

调试日志文件也证实了这一点，它们似乎都喜欢正在发生的事情

我还可以在
\u消费者偏移量
主题中看到一些条目

Offset: 0   Key: aclRead   Timestamp: 2019-10-12 16:43:22.493 Headers: empty

empty

所以发生了一些事情

但是是的。。。。没有留言，救命
 如果有人偶然发现这个：

我在文件
/etc/kafka/tools-log4j.properties
（CentOS）中启用了调试日志记录

然后，当启动消费者时，它显示了大量信息，包括关于
组长不可用的消息

事实证明，我使用server.properties文件中提供的错误默认设置启动了我的3-broker集群。在重新安装并更改服务器后，它成功了！
请注意，我仍在开发中，试图让所有东西都启动并运行，显然，此设置是在第一个消费者连接时使用的

############################# Internal Topic Settings  #############################
# The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state"
# For anything other than development testing, a value greater than 1 is recommended for to ensure availability such as 3.
offsets.topic.replication.factor=3
transaction.state.log.replication.factor=3
transaction.state.log.min.isr=3

上述设置在
server.properties
文件中将
1
作为默认值，该文件在3-broker设置期间破坏了消费者。
是否有其他消费者使用相同的主题+消费者组组合？否，这是我的第一个生产者和第一个消费者，共有5个分区，但是，即使推送50条消息也不会产生结果：（你能启用
kafka控制台消费者
debug并查看其中是否有什么有趣的内容吗？请阅读下面我的答案，我已经这么做了：）