Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/apache-kafka/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache kafka 使用RollingSink时获取NullPointerException_Apache Kafka_Apache Flink_Flink Streaming - Fatal编程技术网

Apache kafka 使用RollingSink时获取NullPointerException

Apache kafka 使用RollingSink时获取NullPointerException,apache-kafka,apache-flink,flink-streaming,Apache Kafka,Apache Flink,Flink Streaming,我正在使用windows*平台。我正在阅读来自卡夫卡的消息,希望使用RollingSink存储在文件中。我收到消息,但当我将滚动接收器添加到数据流时,它抛出空指针异常。下面是代码和堆栈跟踪。 它正在创建文件夹结构,但其中没有数据。 即 文件夹2016-07-13--2031和该文件夹中的三个文件 ._part-0-0.in-progress.crc、_part-0-0.in-progress、_part-0-0.pending StreamExecutionEnvironment sev = S

我正在使用windows*平台。我正在阅读来自卡夫卡的消息,希望使用RollingSink存储在文件中。我收到消息,但当我将滚动接收器添加到数据流时,它抛出空指针异常。下面是代码和堆栈跟踪。 它正在创建文件夹结构,但其中没有数据。 即 文件夹2016-07-13--2031和该文件夹中的三个文件 ._part-0-0.in-progress.crc、_part-0-0.in-progress、_part-0-0.pending

StreamExecutionEnvironment sev = StreamExecutionEnvironment.getExecutionEnvironment();
    DataStream<String> kafkaStream = sev.addSource(new FlinkKafkaConsumer08<String>("test", new SimpleStringSchema(), properties));

    String basePath = "C:\\project\\IOT\\testData\\SinkData";       
     RollingSink<String> rollingSink = new RollingSink<String>(basePath);
     kafkaStream.addSink(rollingSink);


07/14/2016 00:48:48 Job execution switched to status FAILED.
Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
    at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$7.apply$mcV$sp(JobManager.scala:717)
    at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$7.apply(JobManager.scala:663)
    at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$7.apply(JobManager.scala:663)
    at `enter code here`scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:401)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.NullPointerException
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
    at org.apache.hadoop.util.Shell.run(Shell.java:418)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
    at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:559)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:534)
    at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:42)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1698)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1680)
    at org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:1733)
    at org.apache.flink.streaming.connectors.fs.RollingSink.open(RollingSink.java:339)
    at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:38)
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:91)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:317)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:215)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
    at java.lang.Thread.run(Thread.java:745)
StreamExecutionEnvironment sev=StreamExecutionEnvironment.getExecutionEnvironment();
DataStream kafkaStream=sev.addSource(新的FlinkKafkaConsumer08(“测试”,新的SimpleStringSchema(),属性));
String basePath=“C:\\project\\IOT\\testData\\SinkData”;
RollingSink RollingSink=新RollingSink(基本路径);
kafkaStream.addSink(rollingSink);
2016年7月14日00:48:48作业执行切换到失败状态。
线程“main”org.apache.flink.runtime.client.JobExecutionException中出现异常:作业执行失败。
在org.apache.flink.runtime.jobmanager.jobmanager$$anonfun$handleMessage$1$$anonfun$applyorlse$7.apply$mcV$sp(jobmanager.scala:717)
在org.apache.flink.runtime.jobmanager.jobmanager$$anonfun$handleMessage$1$$anonfun$applyorlse$7.apply(jobmanager.scala:663)
在org.apache.flink.runtime.jobmanager.jobmanager$$anonfun$handleMessage$1$$anonfun$applyorlse$7.apply(jobmanager.scala:663)
在`enter code here`scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
在scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
运行(AbstractDispatcher.scala:41)
在akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:401)
位于scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
位于scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
位于scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
在scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)中
原因:java.lang.NullPointerException
位于java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
位于org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
位于org.apache.hadoop.util.Shell.run(Shell.java:418)
位于org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
位于org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
位于org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
位于org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
位于org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:559)
位于org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:534)
位于org.apache.hadoop.fs.LocatedFileStatus(LocatedFileStatus.java:42)
位于org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1698)
位于org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1680)
位于org.apache.hadoop.fs.FileSystem$5.hasNext(FileSystem.java:1733)
位于org.apache.flink.streaming.connectors.fs.RollingSink.open(RollingSink.java:339)
位于org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:38)
位于org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:91)
位于org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:317)
位于org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:215)
位于org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
运行(Thread.java:745)