Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java Apache Flume/var/log/Flume ng/Flume.log(权限被拒绝)_Java_Hadoop_Flume_Flume Ng - Fatal编程技术网

Java Apache Flume/var/log/Flume ng/Flume.log(权限被拒绝)

Java Apache Flume/var/log/Flume ng/Flume.log(权限被拒绝),java,hadoop,flume,flume-ng,Java,Hadoop,Flume,Flume Ng,我正在尝试从/home/cloudera/Documents/flume/读取日志文件,并使用apacheflume将其写入hdfs。我使用以下命令在hdfs中创建flumeLogTest文件夹: sudo -u hdfs hadoop fs -mkdir flumeLogTest sudo -u hdfs hadoop fs -chown flume:flume flumeLogTest 我的配置文件是: agent.sources = src agent.channels = ch age

我正在尝试从/home/cloudera/Documents/flume/读取日志文件,并使用apacheflume将其写入hdfs。我使用以下命令在hdfs中创建flumeLogTest文件夹:

sudo -u hdfs hadoop fs -mkdir flumeLogTest
sudo -u hdfs hadoop fs -chown flume:flume flumeLogTest
我的配置文件是:

agent.sources = src
agent.channels = ch
agent.sinks = snk

agent.sources.src.type = exec
agent.sources.src.command = tail -f Documents/flume/vmware-0.log
agent.sources.src.channels = ch

agent.sinks.snk.channel = ch
agent.sinks.snk.type = hdfs
agent.sinks.snk.hdfs.path = hdfs://localhost.localdomain:8020/user/hdfs/flumeLogTest
agent.sinks.snk.hdfs.fileType = DataStream
agent.sinks.snk.hdfs.writeFormat = Text

agent.channels.ch.type = memory
agent.channels.ch.capacity = 100
我通过以下命令启动了flume代理:

/usr/bin/flume-ng agent --conf Documents/flume --conf-file Documents/flume/Example1.conf --name agent
其中Documents/flume是/home/cloudera/中的一个文件夹,该文件夹包含以下文件

  • flume-conf.properties.template
  • log4j.properties
  • 水槽-env.sh.模板
  • 水槽-env.sh
  • 示例1.conf
  • 我得到了以下错误:

    log4j:ERROR setFile(null,true) call failed.
    java.io.FileNotFoundException: /var/log/flume-ng/flume.log (Permission denied)
        at java.io.FileOutputStream.openAppend(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:192)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:116)
        at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
        at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
        at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
        at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
        at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
        at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
        at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
        at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
        at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254)
        at org.apache.flume.node.Application.<clinit>(Application.java:58)
    
    log4j:错误setFile(null,true)调用失败。
    java.io.FileNotFoundException:/var/log/flume ng/flume.log(权限被拒绝)
    位于java.io.FileOutputStream.openAppend(本机方法)
    位于java.io.FileOutputStream。(FileOutputStream.java:192)
    位于java.io.FileOutputStream。(FileOutputStream.java:116)
    位于org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
    位于org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
    位于org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
    位于org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
    位于org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
    位于org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
    位于org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
    位于org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
    位于org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
    位于org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
    位于org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
    在org.apache.log4j.helpers.OptionConverter.selectAndConfigure上(OptionConverter.java:526)
    登录org.apache.log4j.LogManager(LogManager.java:127)
    位于org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73)
    位于org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242)
    位于org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254)
    位于org.apache.flume.node.Application.(Application.java:58)
    
    我希望我能清楚地回答这个问题。请帮我整理一下。
    提前感谢。

    检查/var/log/flume ng/flume.log的文件权限并更改它

    sudo chown水槽/var/log/flume ng/flume.log


    [java.io.FileNotFoundException:/var/log/flume ng/flume.log(权限被拒绝)]

    它正在hdfs中写入日志(我之前没有注意到),但由于一些权限问题,它抛出了错误,我刚刚更改了权限:sudo chmod 777-R,它工作得很好。谢谢老兄