Apache spark 如何打印“的值”;spark.driver.maxResultSize“是什么;?
我使用Apache spark 如何打印“的值”;spark.driver.maxResultSize“是什么;?,apache-spark,spark-cassandra-connector,Apache Spark,Spark Cassandra Connector,我使用spark submit运行我的应用程序,并将conf设置如下: val conf = new SparkConf(true) .set("spark.cassandra.connection.host","192.168.1.131") .set("spark.cassandra.auth.username","username") .set("spark.cassandra.auth.password","password") .set("spark.driver.m
spark submit
运行我的应用程序,并将conf
设置如下:
val conf = new SparkConf(true)
.set("spark.cassandra.connection.host","192.168.1.131")
.set("spark.cassandra.auth.username","username")
.set("spark.cassandra.auth.password","password")
.set("spark.driver.maxResultSize", "4G")
.set("spark.executor.memory", "4G")
val sc = new SparkContext("spark://192.168.1.120:7077", "just for fun", conf)
我想知道我的设置
spark.driver.maxResultSize
是否工作正常,所以我想打印它的值,但我不知道怎么做?任何建议都将不胜感激 这一行可能会帮助您:
println(conf.get(“spark.driver.maxResultSize”)
但是,提交作业后,更适合在Spark UI上的环境选项卡下检查Spark属性表