Warning: file_get_contents(/data/phpspider/zhask/data//catemap/1/vue.js/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark Cloudera 5.6 spark memoryFraction设置_Apache Spark_Cloudera - Fatal编程技术网

Apache spark Cloudera 5.6 spark memoryFraction设置

Apache spark Cloudera 5.6 spark memoryFraction设置,apache-spark,cloudera,Apache Spark,Cloudera,我有一个CDH5.6集群和5.6CDM(管理器)。我不确定在哪里可以设置spark的MemoryAction设置- spark.storage.memoryFaction spark.shuffle.memoryFaction 要尝试不同的设置并查看示例应用程序的性能,请在创建spark上下文时执行以下操作: sparkContext = { val conf = new SparkConf() .setAppName("Appname") .set("sp

我有一个CDH5.6集群和5.6CDM(管理器)。我不确定在哪里可以设置spark的MemoryAction设置-

spark.storage.memoryFaction
spark.shuffle.memoryFaction

要尝试不同的设置并查看示例应用程序的性能,请在创建spark上下文时执行以下操作:

  sparkContext = {
    val conf = new SparkConf()
      .setAppName("Appname")
      .set("spark.storage.memoryFaction", "someval")
    new SparkContext(conf)
  }
或在执行spark提交时,添加:

 --conf spark.storage.memoryFaction=1