Hadoop:使用flume假脱机目录将csv文件复制到hdfs,错误:INFO source.SpoolDirectorySource:假脱机目录源运行程序已关闭

Hadoop:使用flume假脱机目录将csv文件复制到hdfs,错误:INFO source.SpoolDirectorySource:假脱机目录源运行程序已关闭,hadoop,hadoop-streaming,flume-ng,Hadoop,Hadoop Streaming,Flume Ng,我正在尝试使用flume spool dir将csv文件复制到hdfs。因为我是Hadoop概念的初学者。请帮助我解决以下问题 hdfs目录:/home/hdfs 水槽方向:/etc/flume/ 请查找flume-hwdgteam01.conf文件,如下所示 # Define a source, a channel, and a sink hwdgteam01.sources = src1 hwdgteam01.channels = chan1 hwdgteam01.sinks = sink1

我正在尝试使用flume spool dir将csv文件复制到hdfs。因为我是Hadoop概念的初学者。请帮助我解决以下问题

hdfs目录:/home/hdfs
水槽方向:/etc/flume/

请查找flume-hwdgteam01.conf文件,如下所示

# Define a source, a channel, and a sink
hwdgteam01.sources = src1
hwdgteam01.channels = chan1
hwdgteam01.sinks = sink1

# Set the source type to Spooling Directory and set the directory
# location to /home/flume/ingestion/

hwdgteam01.sources.src1.type = spooldir
hwdgteam01.sources.src1.spoolDir = /home/hwdgteam01/nandan/input-data
hwdgteam01.sources.src1.basenameHeader = true

# Configure the channel as simple in-memory queue
hwdgteam01.channels.chan1.type = memory

# Define the HDFS sink and set its path to your target HDFS directory
hwdgteam01.sinks.sink1.type = hdfs
hwdgteam01.sinks.sink1.hdfs.path = /home/datalanding
hwdgteam01.sinks.sink1.hdfs.fileType = DataStream

# Disable rollover functionallity as we want to keep the original files
hwdgteam01.sinks.sink1.rollCount = 0
hwdgteam01.sinks.sink1.rollInterval = 0
hwdgteam01.sinks.sink1.rollSize = 0
hwdgteam01.sinks.sink1.idleTimeout = 0

# Set the files to their original name
hwdgteam01.sinks.sink1.hdfs.filePrefix = %{basename}

# Connect source and sink
hwdgteam01.sources.src1.channels = chan1
hwdgteam01.sinks.sink1.channel = chan1
我执行命令的方式如下:

/usr/bin/flume ng agent--conf conf--conf file/home/hwdgteam01/nandan/config/flume-hwdgteam01.conf-Dflume.root.logger=DEBUG,console--name hwdgteam01

/usr/bin/flume ng agent-n hwdgteam01-f/home/hwdgteam01/nandan/config/flume-hwdgteam01.conf

/usr/bin/flume ng agent-n hwdgteam01-f/home/hwdgteam01/nandan/config/flume-hwdgteam01.conf

/home/hwdgteam01/nandan/config/flume ng代理-n hwdgteam01-f

/home/hwdgteam01/nandan/config/flume-hwdgteam01.conf

但是什么都没有解决,我得到了下面的错误

请告诉我哪里出了问题。 谢谢你的帮助