java.net.SocketException:尝试从spark sql java应用程序将数据加载/写入redis时发生连接重置异常
我正在尝试在基于java的spark sql应用程序中从Redis缓存加载/写入数据。 这是我的密码:java.net.SocketException:尝试从spark sql java应用程序将数据加载/写入redis时发生连接重置异常,java,apache-spark-sql,spark-redis,Java,Apache Spark Sql,Spark Redis,我正在尝试在基于java的spark sql应用程序中从Redis缓存加载/写入数据。 这是我的密码: SparkSession sprk = SparkSession.builder().appName("Bulk processing").master("local").config("spark.redis.host", "127.0.0.1") .config("sp
SparkSession sprk = SparkSession.builder().appName("Bulk processing").master("local").config("spark.redis.host", "127.0.0.1")
.config("spark.redis.port", "57855").getOrCreate();
Dataset<Row> employeeDF = sprk.read().format("org.apache.spark.sql.redis").
option("table","employees").load();
employeeDF.show();
SparkSession sprk=SparkSession.builder().appName(“批量处理”).master(“本地”).config(“spark.redis.host”、“127.0.0.1”)
.config(“spark.redis.port”,“57855”).getOrCreate();
数据集employeeDF=sprk.read().format(“org.apache.spark.sql.redis”)。
选项(“表格”、“员工”).load();
employeeDF.show();
这是我得到的一个例外:
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Connection reset
at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:205)
at redis.clients.jedis.util.RedisInputStream.readByte(RedisInputStream.java:43)
at redis.clients.jedis.Protocol.process(Protocol.java:155)
at redis.clients.jedis.Protocol.read(Protocol.java:220)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:318)
at redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:255)
at redis.clients.jedis.Connection.getBulkReply(Connection.java:245)
at redis.clients.jedis.BinaryJedis.info(BinaryJedis.java:2972)
at com.redislabs.provider.redis.RedisConfig.clusterEnabled(RedisConfig.scala:194)
at com.redislabs.provider.redis.RedisConfig.getNodes(RedisConfig.scala:317)
at com.redislabs.provider.redis.RedisConfig.getHosts(RedisConfig.scala:233)
at com.redislabs.provider.redis.RedisConfig.<init>(RedisConfig.scala:132)
at org.apache.spark.sql.redis.RedisSourceRelation.<init>(RedisSourceRelation.scala:34)
at org.apache.spark.sql.redis.DefaultSource.createRelation(DefaultSource.scala:13)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
at com.apps.employees.spark.Main.main(Main.java:41)
Caused by: java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:126)
at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:199)
... 18 more
线程“main”redis.clients.jedis.exceptions.JedisConnectionException:java.net.SocketException:Connection reset中的异常
位于redis.clients.jedis.util.RedisInputStream.EnsureRefill(RedisInputStream.java:205)
位于redis.clients.jedis.util.RedisInputStream.readByte(RedisInputStream.java:43)
在redis.clients.jedis.Protocol.process(Protocol.java:155)
位于redis.clients.jedis.Protocol.read(Protocol.java:220)
在redis.clients.jedis.Connection.ReadProtocolWithCheckingBreak(Connection.java:318)
位于redis.clients.jedis.Connection.getBinaryBulkReply(Connection.java:255)
在redis.clients.jedis.Connection.getBulkReply(Connection.java:245)中
在redis.clients.jedis.BinaryJedis.info(BinaryJedis.java:2972)
位于com.redislabs.provider.redis.RedisConfig.clusterEnabled(RedisConfig.scala:194)
位于com.redislabs.provider.redis.RedisConfig.getNodes(RedisConfig.scala:317)
位于com.redislabs.provider.redis.RedisConfig.getHosts(RedisConfig.scala:233)
位于com.redislabs.provider.redis.RedisConfig.(RedisConfig.scala:132)
位于org.apache.spark.sql.redis.redisourcerelation.(redisourcerelation.scala:34)
位于org.apache.spark.sql.redis.DefaultSource.createRelation(DefaultSource.scala:13)
位于org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
位于org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
位于org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
位于org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
位于com.apps.employees.spark.Main.Main(Main.java:41)
原因:java.net.SocketException:连接重置
位于java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
位于java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
位于java.base/java.net.SocketInputStream.read(SocketInputStream.java:126)
位于redis.clients.jedis.util.RedisInputStream.EnsureRefill(RedisInputStream.java:199)
... 还有18个
我已经在Windows机器中使用docker和kubectl启动了redis。
我不确定是什么导致了这个问题。有人能帮我解决这个问题吗