Java tln(“范围=”+范围); } 公共静态类GroupAssigner实现FlatMapFunction{ @凌驾 公共void平面图(Tuple2输入,收集器输出){ //索引1-5将分配给第1组,索引6-10将分配给第2组,等等。 int n=new Long(input.f0/5).intValue()+1; out.collect(新的Tuple2(n,(Double)input.f1.getField(0)); } }

Java tln(“范围=”+范围); } 公共静态类GroupAssigner实现FlatMapFunction{ @凌驾 公共void平面图(Tuple2输入,收集器输出){ //索引1-5将分配给第1组,索引6-10将分配给第2组,等等。 int n=new Long(input.f0/5).intValue()+1; out.collect(新的Tuple2(n,(Double)input.f1.getField(0)); } },java,apache-flink,Java,Apache Flink,可以将源连接到多个接收器,源只执行一次,记录被广播到多个接收器。看到这个问题了吗 getExecutionEnvironment是在运行作业时获取环境的正确方法createCollectionEnvironment是一种很好的游戏和测试方法。见 异常错误消息非常清楚:如果调用print或collect,则会执行数据流。所以你有两个选择: 要么在数据流结束时调用print/collect,然后执行并打印。这对测试东西很好。请记住,每个数据流只能调用collect/print一次,否则它会在

可以将源连接到多个接收器,源只执行一次,记录被广播到多个接收器。看到这个问题了吗

  • getExecutionEnvironment
    是在运行作业时获取环境的正确方法
    createCollectionEnvironment
    是一种很好的游戏和测试方法。见

  • 异常错误消息非常清楚:如果调用print或collect,则会执行数据流。所以你有两个选择:

    • 要么在数据流结束时调用print/collect,然后执行并打印。这对测试东西很好。请记住,每个数据流只能调用collect/print一次,否则它会在未完全定义的情况下多次执行
    • 要么在数据流末尾添加一个接收器,然后调用env.execute()。这就是你想要做的,一旦你的流量是在一个更成熟的形状

    谢谢您的回复。第一次我不明白你的回答。你的回答很有帮助!顺便问一下,我可以写一个与collect执行相同的接收器吗?再次感谢你!
    Exception in thread "main" java.lang.RuntimeException: No new data sinks have been defined since the last execution. The last execution refers to the latest call to 'execute()', 'count()', 'collect()', or 'print()'. 
        at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:940) 
        at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:922) 
        at org.apache.flink.api.java.CollectionEnvironment.execute(CollectionEnvironment.java:34) 
        at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:816) 
        at MainClass.main(MainClass.java:114)
    
        //Table is used to calculate the standard deviation as I figured that there is no such calculation in DataSet.
        BatchTableEnvironment tableEnvironment = TableEnvironment.getTableEnvironment(env);
    
        //Get Data from a mySql database
        DataSet<Row> dbData =
                env.createInput(
                        JDBCInputFormat.buildJDBCInputFormat()
                                .setDrivername("com.mysql.cj.jdbc.Driver")
                                .setDBUrl($database_url)
                                .setQuery("select value from $table_name where id =33")
                                .setUsername("username")
                                .setPassword("password")
                                .setRowTypeInfo(new RowTypeInfo(BasicTypeInfo.DOUBLE_TYPE_INFO))
                                .finish()
                );
    
        // Add index for assigning group (group capacity is 5)
        DataSet<Tuple2<Long, Row>> indexedData = DataSetUtils.zipWithIndex(dbData);
    
        // Replace index(long) with group number(int), and convert Row to double at the same time
        DataSet<Tuple2<Integer, Double>> rawData = indexedData.flatMap(new GroupAssigner());
    
        //Using groupBy() to combine individual data of each group into a list, while calculating the mean and range in each group
        //put them into a POJO named GroupDataClass
        DataSet<GroupDataClass> groupDS = rawData.groupBy("f0").combineGroup(new GroupCombineFunction<Tuple2<Integer, Double>, GroupDataClass>() {
            @Override
            public void combine(Iterable<Tuple2<Integer, Double>> iterable, Collector<GroupDataClass> collector) {
                Iterator<Tuple2<Integer, Double>> it = iterable.iterator();
                Tuple2<Integer, Double> var1 = it.next();
                int groupNum = var1.f0;
    
                // Using max and min to calculate range, using i and sum to calculate mean
                double max = var1.f1;
                double min = max;
                double sum = 0;
                int i = 1;
    
                // The list is to store individual value
                List<Double> list = new ArrayList<>();
                list.add(max);
    
                while (it.hasNext())
                {
                    double next = it.next().f1;
                    sum += next;
                    i++;
                    max = next > max ? next : max;
                    min = next < min ? next : min;
                    list.add(next);
                }
    
                //Store group number, mean, range, and 5 individual values within the group
                collector.collect(new GroupDataClass(groupNum, sum / i, max - min, list));
            }
        });
    
        //print because if no sink is created, Flink will not even perform the calculation.
        groupDS.print();
    
    
        // Get the max group number and range in each group to calculate average range
        // if group number start with 1 then the maximum of group number equals to the number of group
        // However, because this is the second sink, data will flow from source again, which will double the group number
        DataSet<Tuple2<Integer, Double>> rangeDS = groupDS.map(new MapFunction<GroupDataClass, Tuple2<Integer, Double>>() {
            @Override
            public Tuple2<Integer, Double> map(GroupDataClass in) {
                return new Tuple2<>(in.groupNum, in.range);
            }
        }).max(0).andSum(1);
    
        // collect and print as if no sink is created, Flink will not even perform the calculation.
        Tuple2<Integer, Double> rangeTuple = rangeDS.collect().get(0);
        double range = rangeTuple.f1/ rangeTuple.f0;
        System.out.println("range = " + range);
    }
    
    public static class GroupAssigner implements FlatMapFunction<Tuple2<Long, Row>, Tuple2<Integer, Double>> {
        @Override
        public void flatMap(Tuple2<Long, Row> input, Collector<Tuple2<Integer, Double>> out) {
    
            // index 1-5 will be assigned to group 1, index 6-10 will be assigned to group 2, etc.
            int n = new Long(input.f0 / 5).intValue() + 1;
            out.collect(new Tuple2<>(n, (Double) input.f1.getField(0)));
        }
    }