Hadoop 流式推文收到致命警报时出现异常:Flume中拒绝访问

Hadoop 流式推文收到致命警报时出现异常:Flume中拒绝访问,hadoop,flume,flume-ng,flume-twitter,Hadoop,Flume,Flume Ng,Flume Twitter,我目前在Flume中有此配置 TwitterAgent.sources = Twitter TwitterAgent.channels = MemChannel TwitterAgent.sinks = HDFS TwitterAgent.sources.Twitter.type= org.apache.flume.source.twitter.TwitterSource TwitterAgent.sources.Twitter.channels = MemChannel TwitterAge

我目前在Flume中有此配置

TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = HDFS

TwitterAgent.sources.Twitter.type= org.apache.flume.source.twitter.TwitterSource
TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sources.Twitter.consumerKey=XXXXXXXXXXXX
TwitterAgent.sources.Twitter.consumerSecret=XXXXXXXXXXXX
TwitterAgent.sources.Twitter.accessToken=XXXXXXXXXXXX
TwitterAgent.sources.Twitter.accessTokenSecret=XXXXXXXXXXXX 


TwitterAgent.sources.Twitter.maxBatchSize=1000

TwitterAgent.sinks.HDFS.channel=MemChannel
TwitterAgent.sinks.HDFS.type= hdfs
TwitterAgent.sinks.HDFS.hdfs.path= hdfs://quickstart.cloudera:8020/user/flume/tweets

TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
TwitterAgent.sinks.HDFS.hdfs.batchSize=10
TwitterAgent.sinks.HDFS.hdfs.rollSize=0
TwitterAgent.sinks.HDFS.hdfs.rollCount= 10000
TwitterAgent.sinks.HDFS.hdfs.rollInterval = 600


TwitterAgent.channels.MemChannel.type= memory
TwitterAgent.channels.MemChannel.capacity = 10000
TwitterAgent.channels.MemChannel.transactionCapacity =100
twitter应用程序验证密钥是正确的。我不断地犯这个错误 在flume日志文件中:

18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Added sinks: HDFS Agent: TwitterAgent
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Processing:HDFS
18/01/25 08:01:46 INFO conf.FlumeConfiguration: Post-validation flume configuration contains configuration for agents: [TwitterAgent]
18/01/25 08:01:46 INFO node.AbstractConfigurationProvider: Creating channels
18/01/25 08:01:46 INFO channel.DefaultChannelFactory: Creating instance of channel MemChannel type memory
18/01/25 08:01:46 INFO node.AbstractConfigurationProvider: Created channel MemChannel
18/01/25 08:01:46 INFO source.DefaultSourceFactory: Creating instance of source Twitter, type org.apache.flume.source.twitter.TwitterSource
18/01/25 08:01:46 INFO twitter.TwitterSource: Consumer Key:'XXXXXXXXXXXX'
18/01/25 08:01:46 INFO twitter.TwitterSource: Consumer Secret:'XXXXXXXXXXXX'
18/01/25 08:01:46 INFO twitter.TwitterSource: Access Token:'XXXXXXXXXXXX'
18/01/25 08:01:46 INFO twitter.TwitterSource: Access Token Secret:'XXXXXXXXXXXX'
18/01/25 08:01:46 INFO sink.DefaultSinkFactory: Creating instance of sink: HDFS, type: hdfs
18/01/25 08:01:46 INFO node.AbstractConfigurationProvider: Channel MemChannel connected to [Twitter, HDFS]
18/01/25 08:01:46 INFO node.Application: Starting new configuration:{ sourceRunners:{Twitter=EventDrivenSourceRunner: { source:org.apache.flume.source.twitter.TwitterSource{name:Twitter,state:IDLE} }} sinkRunners:{HDFS=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@58c9defc counterGroup:{ name:null counters:{} } }} channels:{MemChannel=org.apache.flume.channel.MemoryChannel{name: MemChannel}} }
18/01/25 08:01:46 INFO node.Application: Starting Channel MemChannel
18/01/25 08:01:47 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: CHANNEL, name: MemChannel: Successfully registered new MBean.
18/01/25 08:01:47 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: MemChannel started
18/01/25 08:01:47 INFO node.Application: Starting Sink HDFS
18/01/25 08:01:47 INFO node.Application: Starting Source Twitter
18/01/25 08:01:47 INFO twitter.TwitterSource: Starting twitter source org.apache.flume.source.twitter.TwitterSource{name:Twitter,state:IDLE} ...
18/01/25 08:01:47 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: SINK, name: HDFS: Successfully registered new MBean.
18/01/25 08:01:47 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: HDFS started
18/01/25 08:01:47 INFO twitter.TwitterSource: Twitter source Twitter started.
18/01/25 08:01:47 INFO twitter4j.TwitterStreamImpl: Establishing connection.
**18/01/25 08:01:49 INFO twitter4j.TwitterStreamImpl: Received fatal alert: access_denied
18/01/25 08:01:49 ERROR twitter.TwitterSource: Exception while streaming tweets
Received fatal alert: access_denied**
Relevant discussions can be found on the Internet at:
    http://www.google.co.jp/search?q=d0031b0b or
    http://www.google.co.jp/search?q=1db75522
TwitterException{exceptionCode=[d0031b0b-1db75522 db667dea-99334ae4], statusCode=-1, message=null, code=-1, retryAfter=-1, rateLimitStatus=null, version=3.0.3}
    at twitter4j.internal.http.HttpClientImpl.request(HttpClientImpl.java:192)
我在这个路径上创建了twitter.conf文件/home/cloudera/flumeprac

我正在终端上运行以下命令

[cloudera@quickstart flumeprac]$ flume-ng agent --conf-file
 twitter.conf --name TwitterAgent --conf $FLUME_HOME/conf
 _DFflume.root.logger=INFO,console
我通过下面的链接浏览


hii,您解决了这个问题吗?