Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/docker/9.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Hadoop INFO hdfs.HDFSEventSink:已调用写入程序回调_Hadoop_Docker_Flume - Fatal编程技术网

Hadoop INFO hdfs.HDFSEventSink:已调用写入程序回调

Hadoop INFO hdfs.HDFSEventSink:已调用写入程序回调,hadoop,docker,flume,Hadoop,Docker,Flume,我用谷歌搜索了这个错误。我没有任何解决办法 我正在使用伪分布式hadoop flume。它是停靠的应用程序。我正试图从flume在控制台中写入。它起作用了。我正在尝试用hdfs编写。它说writer回调失败了 flume.conf a2.sources = r1 a2.sinks = k1 a2.channels = c1 a2.sources.r1.type = netcat a2.sources.r1.bind = localhost a2.sources.r1.port

我用谷歌搜索了这个错误。我没有任何解决办法

我正在使用伪分布式hadoop flume。它是停靠的应用程序。我正试图从flume在控制台中写入。它起作用了。我正在尝试用hdfs编写。它说writer回调失败了

flume.conf

  a2.sources = r1
  a2.sinks = k1
  a2.channels = c1

 a2.sources.r1.type = netcat
 a2.sources.r1.bind = localhost
 a2.sources.r1.port = 5140

 a2.sinks.k1.type = hdfs
 a2.sinks.k1.hdfs.fileType = DataStream
 a2.sinks.k1.hdfs.writeFormat = Text
 a2.sinks.k1.hdfs.path = hdfs://localhost:8020/user/root/syslog/%y-%m-%d/%H%M/%S
 a2.sinks.k1.hdfs.filePrefix = events
 a2.sinks.k1.hdfs.roundUnit = minute
 a2.sinks.k1.hdfs.useLocalTimeStamp = true

 # Use a channel which buffers events in memory
 a2.channels.c1.type = memory
 a2.channels.c1.capacity = 10000
 a2.channels.c1.transactionCapacity = 100

 # Bind the source and sink to the channel
 a2.sources.r1.channels = c1
 a2.sinks.k1.channel = c1
水槽运行命令

 /usr/bin/flume-ng agent --conf-file /etc/flume-ng/conf/flume.conf --name a1 -Dflume.root.logger=INFO,console

所有hadoop服务都在运行。如何解决这个错误?有什么想法吗?

你能把错误日志也贴出来吗。我在哪里可以得到日志?你在哪里看到信息hdfs.HDFSEventSink:Writer callback调用了?我正在将数据写入控制台。消息来源是netcat。水槽是控制台。我在控制台中看到了。我想知道你是否解决了这个问题?