Flume solrSink,无错误,但不会将数据摄取到Solr
我正在使用Flume为文本文档编制索引,我没有看到任何错误或警告消息,但Solr和Flume的数据都没有被摄取到Solr日志级别,全部设置为跟踪Flume solrSink,无错误,但不会将数据摄取到Solr,solr,flume,Solr,Flume,我正在使用Flume为文本文档编制索引,我没有看到任何错误或警告消息,但Solr和Flume的数据都没有被摄取到Solr日志级别,全部设置为跟踪 Flume version : 1.5.2.2.3 Solr Version : 5.5 **Config files are as below** **Flume Config :** agent.sources = SpoolDirSrc agent.channels = FileChannel agent.sinks = SolrSi
Flume version : 1.5.2.2.3
Solr Version : 5.5
**Config files are as below**
**Flume Config :**
agent.sources = SpoolDirSrc
agent.channels = FileChannel
agent.sinks = SolrSink
# Configure Source
agent.sources.SpoolDirSrc.channels = fileChannel agent.sources.SpoolDirSrc.type = spooldir
agent.sources.SpoolDirSrc.spoolDir=/home/flume/source\u电子邮件agent.sources.SpoolDirSrc.basenameHeader=true agent.sources.SpoolDirSrc.fileHeader=true
agent.sources.SpoolDirSrc.deserializer=org.apache.flume.sink.solr.morpline.BlobDeserializer$Builder
agent.channels.FileChannel.type=文件agent.channels.FileChannel.capacity=10000
agent.sinks.SolrSink.type=org.apache.flume.sink.solr.morpline.morplinesolrsink
agent.sinks.SolrSink.morplinefile=/etc/flume/conf/morpline.conf agent.sinks.SolrSink.batchsize=1000 agent.sinks.SolrSink.batchDurationMillis=2500 agent.sinks.SolrSink.channel=fileChannel agent.sinks.SolrSink.morplineId=morpline1 agent.sources.SpoolDirSrc.channels=fileChannel代理.sinks.SolrSink=fileChannel
“Morphline配置
solrLocator: {
collection : gsearch
zkHost : "codesolr-as-r3p:21810,codesolr-as-r3p:21811,codesolr-as-r3p:21812"
}
morphlines :
[
{
id : morphline1
importCommands : ["org.kitesdk.**", "org.apache.solr.**"]
commands :
[
{ detectMimeType { includeDefaultMimeTypes : true } }
{
solrCell {
solrLocator : ${solrLocator}
captureAttr : true
lowernames : true
capture : [_attachment_body, _attachment_mimetype, basename, content, content_encoding, content_type, file, meta]
parsers : [ { parser : org.apache.tika.parser.txt.TXTParser } ]
}
}
{ generateUUID { field : id } }
{ sanitizeUnknownSolrFields { solrLocator : ${solrLocator} } }
{ logDebug { format : "output record: {}", args : ["@{}"] } }
{ loadSolr: { solrLocator : ${solrLocator} } }
]
}
]
Please help me what could be the issue
Regards,
~Sri
通常在flume日志中,您可以看到更详细的错误,您可以粘贴跟踪吗 可能是morphilines找不到solr配置,您可以在morphilines.conf中添加此属性 solrHomeDir:“/your\u solr\u config\u文件” 我希望是你的帮助