Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/email/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
C# SaveAsTextFile方法在Mobius(SparkCLR)中不再可用吗?_C#_Apache Spark_Mobius - Fatal编程技术网

C# SaveAsTextFile方法在Mobius(SparkCLR)中不再可用吗?

C# SaveAsTextFile方法在Mobius(SparkCLR)中不再可用吗?,c#,apache-spark,mobius,C#,Apache Spark,Mobius,嗨,我正试图运行下面的例子直接从复制。但是当尝试使用SaveAsTextFile(字符串文件路径)方法时,它找不到该方法。相反,它显示了图1中的错误 我引用的是最新发布的:“Microsoft.SparkCLR”version=“1.6.100” var lines=sparkContext.TextFile(@)hdfs://path/to/input.txt"); var words=lines.FlatMap(s=>s.Split(“”)); var wordCounts=words.

嗨,我正试图运行下面的例子直接从复制。但是当尝试使用SaveAsTextFile(字符串文件路径)方法时,它找不到该方法。相反,它显示了图1中的错误

我引用的是最新发布的:“Microsoft.SparkCLR”version=“1.6.100”

var lines=sparkContext.TextFile(@)hdfs://path/to/input.txt");  
var words=lines.FlatMap(s=>s.Split(“”));
var wordCounts=words.Map(w=>newkeyvaluepair(w.Trim(),1))
.ReduceByKey((x,y)=>x+y);
var wordCountCollection=wordCounts.Collect();
wordCounts.SaveAsTextFile(@)hdfs://path/to/wordcount.txt");  

试试下面这一行,它就会起作用

wordCounts.Map(wc => wc.Key + "," + wc.Value)
          .SaveAsTextFile(@"hdfs://path/to/wordcount.txt");
如图中的错误消息所示,如果RDD的类型为string,则SaveAsTextFile()可用。上面的代码将键值对的RDD转换为字符串RDD。中的代码示例需要更新。如果您感兴趣,请随时发送PR来修复它

Mobius项目中讨论了是否使SaveAsTextFile()可用于所有RDD类型。问题是,在RDD中支持的类型上写入字符串,在写入文本文件时,可能并不总是产生可读字符串。如果您对此有意见,请随意在中创建一个问题

wordCounts.Map(wc => wc.Key + "," + wc.Value)
          .SaveAsTextFile(@"hdfs://path/to/wordcount.txt");