Hadoop 我没有收到来自twitter的推文

Hadoop 我没有收到来自twitter的推文,hadoop,twitter,hdfs,bigdata,flume,Hadoop,Twitter,Hdfs,Bigdata,Flume,我尝试通过HDFS中的Flume来流式传输推文,但我没有从推文中获得推文。该过程将在下面提到的这一点停止!!除了这一步什么都没有 INFO instrumentation.MonitoredCounterGroup: Monitoried counter group for type: SINK, name: HDFS, registered successfully. INFO instrumentation.MonitoredCounterGroup: Component type: SIN

我尝试通过HDFS中的Flume来流式传输推文,但我没有从推文中获得推文。该过程将在下面提到的这一点停止!!除了这一步什么都没有

INFO instrumentation.MonitoredCounterGroup: Monitoried counter group for type: SINK, name: HDFS, registered successfully.
INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: HDFS started
My flume.conf文件:

TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = HDFS

TwitterAgent.sources.Twitter.type =       org.apache.flume.source.twitter.TwitterSource
TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sources.Twitter.consumerKey = **************
TwitterAgent.sources.Twitter.consumerSecret =  *********
TwitterAgent.sources.Twitter.accessToken =************
TwitterAgent.sources.Twitter.accessTokenSecret = *********

TwitterAgent.sources.Twitter.keywords = big data,cloud,hadoop,data analysis,hive,pig,hbase,flume,datawarehouse
TwitterAgent.sinks.HDFS.channel = MemChannel
TwitterAgent.sinks.HDFS.type = hdfs
TwitterAgent.sinks.HDFS.hdfs.path = hdfs://localhost:9000/Big
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0
TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000

TwitterAgent.channels.MemChannel.type = memory
TwitterAgent.channels.MemChannel.capacity = 10000
TwitterAgent.channels.MemChannel.transactionCapacity = 100
请尝试此代码。。
TwitterAgent.sources=Twitter
TwitterAgent.channels=MemChannel
TwitterAgent.sinks=HDFS
TwitterAgent.sources.Twitter.type=com.cloudera.flume.source.TwitterSource
TwitterAgent.sources.Twitter.channels=MemChannel
TwitterAgent.sources.Twitter.consumerKey=xxxxxxxxxxxxx
TwitterAgent.sources.Twitter.ConsumerCret=xxxxxxxxxxx
TwitterAgent.sources.Twitter.accessToken=XXXXXXXXXXXXX
TwitterAgent.sources.Twitter.accessTokenSecret=xxxxxxxxx
TwitterAgent.sources.Twitter.keywords=大数据、云、hadoop、数据分析、hive、pig、hbase、flume、数据仓库
TwitterAgent.sinks.HDFS.channel=MemChannel
TwitterAgent.sinks.HDFS.type=HDFS
TwitterAgent.sinks.HDFS.HDFS.path=hdfs://localhost:9000/Big
TwitterAgent.sinks.HDFS.HDFS.fileType=数据流
TwitterAgent.sinks.HDFS.HDFS.writeFormat=Text
TwitterAgent.sinks.HDFS.HDFS.batchSize=1000
TwitterAgent.sinks.HDFS.HDFS.rollSize=0
TwitterAgent.sinks.HDFS.HDFS.rollCount=10000
TwitterAgent.sinks.HDFS.HDFS.filePrefix=twitter\u India-
TwitterAgent.sinks.HDFS.HDFS.round=true
TwitterAgent.sinks.HDFS.HDFS.roundValue=2
TwitterAgent.sinks.HDFS.HDFS.roundUnit=分钟
TwitterAgent.channels.MemChannel.type=内存
TwitterAgent.channels.MemChannel.capacity=10000

TwitterAgent.channels.MemChannel.transactionCapacity=100
flume.conf
文件中

TwitterAgent.sinks.HDFS.hdfs.path = hdfs://localhost:**9000**/Big
将您自己的活动localhost instade设置为9000

转到,您可以看到活动的本地主机。就像我的情况一样

TwitterAgent.sinks.HDFS.hdfs.path=hdfs://localhost:54310/user/flume/tweets

您希望共享的任何代码都是有用的,请将其包含在原始帖子中,通过编辑并删除这些评论。如果可以,请帮助我。您可以检查并发布/logs/flume.log文件中的日志吗?此步骤中已停止!!没有建立连接!INFO instrumentation.MonitoredCounterGroup:Component type:SINK,name:HDFS startedit对我来说运行得很好,您需要检查此操作所需的外部jar。。