Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/java/380.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/maven/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java &引用;“非法国家例外”;Apache Beam管道从csv文件读取、拆分、groupbyKey并写入文本文件时出错。为什么?_Java_Maven_Apache Beam_Apache Beam Io - Fatal编程技术网

Java &引用;“非法国家例外”;Apache Beam管道从csv文件读取、拆分、groupbyKey并写入文本文件时出错。为什么?

Java &引用;“非法国家例外”;Apache Beam管道从csv文件读取、拆分、groupbyKey并写入文本文件时出错。为什么?,java,maven,apache-beam,apache-beam-io,Java,Maven,Apache Beam,Apache Beam Io,我的输入数据如下所示: id,vin,url,exteriorColor,interiorColor,design,transmission,lastcrawled,mileage,price,certified,dealerId,historyType,MSRP 114722309,19XVC2F35PR012846,http://www.pohankaacura.com/auto/used-2017-acura-ilx-chantilly-va-near-buckeystown-md/247

我的输入数据如下所示:

id,vin,url,exteriorColor,interiorColor,design,transmission,lastcrawled,mileage,price,certified,dealerId,historyType,MSRP
114722309,19XVC2F35PR012846,http://www.pohankaacura.com/auto/used-2017-acura-ilx-chantilly-va-near-buckeystown-md/24742881/,Modern Steel,graystone,0,8-Speed Dual-Clutch,2018-02-05 01:49:47 UTC,1646,22550,0,28453
我想建立一个Beam管道,从csv文件中读取数据,获取vin并计算文件中出现vin的次数。所以我想按vin分组并计算计数。我希望我的最终输出是一个平面文件。我错过了注释,所以我现在添加了它,但是我得到了一个不同的错误,我在这里也找不到解决方案。 下面是我的代码

import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.io.TextIO;
import org.apache.beam.sdk.options.PipelineOptions;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.transforms.*;
import org.apache.beam.sdk.values.KV;

public class p1 {
    public static void main(String[] args) {
        PipelineOptions options = PipelineOptionsFactory.create();

        Pipeline p = Pipeline.create(options);
        p.apply(TextIO.read().from("~/slow_storage_drive/beam_test_files/one_vin.csv"))

                .apply("Parse&ConvertToKV", MapElements.via(
                        new SimpleFunction<String, KV<String, Integer>>() {
                            public KV<String, Integer> apply(String input){
                                String[] split = input.split(",");
                                String key = split[1];
                                Integer value = 1;
                                return KV.of(key, value);
                            }
                        }
                ))

                .apply(GroupByKey.<String, Integer>create())


                .apply("SumOfValuesByKey", ParDo.of(new DoFn<KV<String, Iterable<Integer>>, String>() {
                    @ProcessElement
                    public void processElement(ProcessContext context) {
                        Integer crawlCount = 0;
                        String vin = context.element().getKey();
                        Iterable<Integer> counts = context.element().getValue();
                        for (Integer count : counts){
                            crawlCount += count;
                        }
                        context.output(vin + ": " + crawlCount);
                    }
                }))

                .apply(TextIO.write().to("~/slow_storage_drive/beam_example_files/emr_beam_test/final_output").withoutSharding());

        p.run().waitUntilFinish();
    }

}
我得到以下错误:

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project emr_beam_test: An exception occured while executing the Java class. java.lang.IllegalStateException: Invisible parameter type of p1$2 arg0 for public p1$2$DoFnInvoker(p1$2) -> [Help 1]

我无法理解我做错了什么。有人能帮我吗

必须用@processElement注释匿名类方法processElement


有关注释的更多信息,请参阅

,因为Apache Beam还不支持Java 10,所以我似乎遇到了不可见的参数类型异常。我将JAVA_主页改为指向JAVA 8,程序运行正常。我从这条线索中得到了这个想法:

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project emr_beam_test: An exception occured while executing the Java class. java.lang.IllegalStateException: Invisible parameter type of p1$2 arg0 for public p1$2$DoFnInvoker(p1$2) -> [Help 1]