Apache spark 运行结构化流媒体示例时出错;警告网络客户端:引导代理ip-10-28-3-35.ec2.内部:2181已断开;

Apache spark 运行结构化流媒体示例时出错;警告网络客户端:引导代理ip-10-28-3-35.ec2.内部:2181已断开;,apache-spark,pyspark,apache-kafka,hortonworks-data-platform,spark-structured-streaming,Apache Spark,Pyspark,Apache Kafka,Hortonworks Data Platform,Spark Structured Streaming,运行时,我遇到以下错误: WARN NetworkClient: Bootstrap broker ip-10-28-3-35.ec2.internal:2181 disconnected 环境:HDP-2.6.5.0 我能够阅读卡夫卡给出的一个主题的内容 命令来运行这些作业 bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.3.1 examples/src/main/python/sql/stre

运行时,我遇到以下错误:

WARN NetworkClient: Bootstrap broker ip-10-28-3-35.ec2.internal:2181 disconnected
环境:HDP-2.6.5.0

我能够阅读卡夫卡给出的一个主题的内容

命令来运行这些作业

bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.3.1 examples/src/main/python/sql/streaming/structured_kafka_wordcount.py ip-10-28-3-35.ec2.internal:2181 subscribe fifa2
[root@centosspark2]#bin/spark submit--packagesorg.apache.spark:spark-sql-kafka-0-10_2.11:2.3.1 examples/src/main/python/sql/streaming/structured_kafka_wordcount.py ip-10-28-3-35.ec2.内部:2181订阅fifa2
常春藤默认缓存设置为:/root/.ivy2/Cache
存储在:/root/.ivy2/jars中的包的jar
::加载设置::url=jar:file:/usr/hdp/2.6.5.0-292/spark2/jars/ivy-2.4.0.jar/org/apache/ivy/core/settings/ivysettings.xml
org.apache.spark#spark-sql-kafka-0-10_2.11作为依赖项添加
::解析依赖项::org.apache.spark#spark提交父级;1
confs:[默认值]
找到org.apache.spark#spark-sql-kafka-0-10_2.11;2.3.1在中环
找到org.apache.kafka#kafka客户端;中环0.10.0.1
找到net.jpountz.lz4#lz4;中环1.3.0
找到org.xerial.snappy#snappyjava;1.1.2.6位于中环
找到org.slf4j#slf4j api;1.7.16中环
发现org.spark project.spark#未使用;中环1.0.0
::解析报告::解析642ms::工件dl 15ms
::正在使用的模块:
net.jpountz.lz4#lz4;1.3.0从中央输入[默认值]
org.apache.kafka#kafka客户端;0.10.0.1从中央输入[默认值]
org.apache.spark#spark-sql-kafka-0-10_2.11;2.3.1从中央输入[默认]
org.slf4j#slf4j api;1.7.16从中央输入[默认]
org.spark project.spark#未使用;1.0.0从中央输入[默认值]
snappy#snappyjava;1.1.2.6从中央输入[默认]
---------------------------------------------------------------------
||模块| |工件|
|形态|编号|搜索| dwnlded |驱逐|编号| dwnlded|
---------------------------------------------------------------------
|默认值| 6 | 0 | 0 | 6 | 0|
---------------------------------------------------------------------
::检索::org.apache.spark#spark提交父级
confs:[默认值]
已复制0个工件,已检索6个(0 KB/17毫秒)
18/06/22 06:57:26警告Utils:服务“SparkUI”无法在端口4040上绑定。正在尝试端口4041。
18/06/22 06:57:26警告Utils:服务“SparkUI”无法在端口4041上绑定。正在尝试端口4042。
18/06/22 06:57:35警告网络客户端:引导代理ip-10-28-3-35.ec2.内部:2181已断开连接
18/06/22 06:57:35警告网络客户端:引导代理ip-10-28-3-35.ec2.内部:2181已断开连接
18/06/22 06:57:35警告网络客户端:引导代理ip-10-28-3-35.ec2.内部:2181已断开连接
18/06/22 06:57:35警告网络客户端:引导代理ip-10-28-3-35.ec2.内部:2181已断开连接
^Z
[11] +停止bin/spark提交--包org.apache.spark:spark-sql-kafka-0-10_2.11:2.3.1示例/src/main/python/sql/streaming/structured_kafka_wordcount.py ip-10-28-3-35.ec2.内部:2181订阅fifa2

您需要提供正确的卡夫卡服务器,而不是动物园管理员