Logging Solr日志未直接推送到kafka，Solr无法连接到ZK_Logging_Solr_Log4j_Apache Zookeeper_Apache Kafka

Logging Solr日志未直接推送到kafka，Solr无法连接到ZK

logging solr log4j apache-zookeeper apache-kafka

Logging Solr日志未直接推送到kafka，Solr无法连接到ZK,logging,solr,log4j,apache-zookeeper,apache-kafka,Logging,Solr,Log4j,Apache Zookeeper,Apache Kafka,我正在尝试使用log4j将日志从solr直接发送到kafka。虽然日志将打印到标准输出，但卡夫卡没有收到任何数据。我可以使用命令行生成器将数据推送到kafka 我收到的警告和错误： WARN - 2015-01-19 12:09:25.545; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Solr cannot talk to ZK, exiting Overseer main queue loop

我正在尝试使用log4j将日志从solr直接发送到kafka。虽然日志将打印到标准输出，但卡夫卡没有收到任何数据。我可以使用命令行生成器将数据推送到kafka

我收到的警告和错误：

WARN  - 2015-01-19 12:09:25.545; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Solr cannot talk to ZK, exiting Overseer main queue loop                                                                                                                             
INFO  - 2015-01-19 12:09:25.552; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Overseer Loop exiting : 10.254.120.50:8900_solr 
WARN  - 2015-01-19 12:09:25.554; org.apache.solr.common.cloud.ZkStateReader$2; ZooKeeper watch triggered, but Solr cannot talk to ZK 
ERROR - 2015-01-19 12:09:25.560; org.apache.solr.cloud.Overseer$ClusterStateUpdater; could not read the data                         
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader

我的Log4j.Properties文件：

 solr.log=/home/solradmin/solr/latest/logs/                                                                   
    log4j.rootLogger=INFO, file, KAFKA                                                                                                   
    log4j.logger.KAFKA=INFO, file                                                                                                        
    log4j.logger.solr=INFO, KAFKA  

    log4j.appender.stdout=org.apache.log4j.ConsoleAppender                                                       
    log4j.appender.stdout.layout=org.apache.log4j.PatternLayout                                                  
    log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n 

    log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender                                                                       
    log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout                                                   
    log4j.appender.KAFKA.layout.ConversionPattern=%-5p: %c - %m%n                                                                        
    log4j.appender.KAFKA.BrokerList=localhost:9092                                                                                       
    log4j.appender.KAFKA.Topic=herpderp                                                                               

    log4j.appender.file=org.apache.log4j.RollingFileAppender                                                                             
    log4j.appender.file.MaxFileSize=100MB                                                                                                
    log4j.appender.file.MaxBackupIndex=9   
                                                                                                                                                                                                                                                                                                                log4j.appender.file.File=${solr.log}/solr.log                                                                                        
    log4j.appender.file.layout=org.apache.log4j.PatternLayout                                                                            
    log4j.appender.file.layout.ConversionPattern=%-5p - %d{yyyy-MM-dd HH:mm:ss.SSS}; %C; %m\n         
    log4j.logger.org.apache.solr=DEBUG                                                                                                   
    log4j.logger.org.apache.zookeeper=WARN                                                                                       
    log4j.logger.org.apache.hadoop=WARN

作为受支持的追加器。然而，log4j的功能很容易配置

log4j需要某种插件来支持卡夫卡吗

我使用以下来源尝试了不同的配置：及

请确保根日志记录器不会自动登录

#bad
log4j.rootLogger=INFO, file, KAFKA                                                                                                   
#good
log4j.rootLogger=INFO, file

我的完整配置

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n

log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout
log4j.appender.KAFKA.layout.ConversionPattern=%-5p: %c - %m%n
log4j.appender.KAFKA.BrokerList=kafka1.example.com:6667,kafka2.example.com:6667,kafka3.example.com:6667
log4j.appender.KAFKA.Topic=foobar
log4j.rootLogger=DEBUG,file

log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.MaxFileSize=100MB
log4j.appender.file.MaxBackupIndex=9

solr.log=/home/solradmin/solr/latest/logs
#- File to log to and log format
log4j.appender.file.File=${solr.log}/solr.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%-5p - %d{yyyy-MM-dd HH:mm:ss.SSS}; %C; %m\n
log4j.logger.org.apache.solr=DEBUG,KAFKA
log4j.logger.org.apache.zookeeper=WARN,KAFKA
log4j.logger.org.apache.hadoop=WARN

# set to INFO to enable infostream log messages
log4j.logger.org.apache.solr.update.LoggingInfoStream=OFF

Java应用程序

package nd.KafkaTest;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * Hello world!
 *
 */
public class App 
{
    public static void main( String[] args )
    {
        Logger logger = LoggerFactory.getLogger(App.class.getName());
        System.out.println( "Hello World!" );
        int i= 100;
        while (i>0)
        {

               logger.debug("Debugging!." + i); 
               logger.info("Exiting application." + i);
               i--;
        }
        System.out.println("here you go");

    }
}

Pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>nd</groupId>
  <artifactId>KafkaTest</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>KafkaTest</name>
   <url>http://maven.apache.org</url>
<build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <configuration>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <mainClass>logToKafka.App</mainClass>
            </manifest>
          </archive>
        </configuration>
      </plugin>
    </plugins>
  </build>
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
   <dependency>
              <groupId>org.apache.kafka</groupId>
              <artifactId>kafka_2.9.2</artifactId>
              <version>0.8.2.1</version>
       </dependency>

       <dependency>
              <groupId>org.slf4j</groupId>
              <artifactId>slf4j-api</artifactId>
              <version>1.7.6</version>
       </dependency>
<!--        <dependency> -->
<!--               <groupId>org.slf4j</groupId> -->
<!--               <artifactId>slf4j-log4j12</artifactId> -->
<!--               <version>1.7.12</version> -->
<!--        </dependency>  -->
       <dependency>
              <groupId>log4j</groupId>
              <artifactId>log4j</artifactId>
              <version>1.2.17</version>
       </dependency>


  </dependencies>
</project>

信用证：认识一下拉杰德夫，我和他是同事，一起工作过

确保根记录器不记录到自身

#bad
log4j.rootLogger=INFO, file, KAFKA                                                                                                   
#good
log4j.rootLogger=INFO, file

我的完整配置

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n

log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout
log4j.appender.KAFKA.layout.ConversionPattern=%-5p: %c - %m%n
log4j.appender.KAFKA.BrokerList=kafka1.example.com:6667,kafka2.example.com:6667,kafka3.example.com:6667
log4j.appender.KAFKA.Topic=foobar
log4j.rootLogger=DEBUG,file

log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.MaxFileSize=100MB
log4j.appender.file.MaxBackupIndex=9

solr.log=/home/solradmin/solr/latest/logs
#- File to log to and log format
log4j.appender.file.File=${solr.log}/solr.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%-5p - %d{yyyy-MM-dd HH:mm:ss.SSS}; %C; %m\n
log4j.logger.org.apache.solr=DEBUG,KAFKA
log4j.logger.org.apache.zookeeper=WARN,KAFKA
log4j.logger.org.apache.hadoop=WARN

# set to INFO to enable infostream log messages
log4j.logger.org.apache.solr.update.LoggingInfoStream=OFF

Java应用程序

package nd.KafkaTest;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * Hello world!
 *
 */
public class App 
{
    public static void main( String[] args )
    {
        Logger logger = LoggerFactory.getLogger(App.class.getName());
        System.out.println( "Hello World!" );
        int i= 100;
        while (i>0)
        {

               logger.debug("Debugging!." + i); 
               logger.info("Exiting application." + i);
               i--;
        }
        System.out.println("here you go");

    }
}

Pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>nd</groupId>
  <artifactId>KafkaTest</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>KafkaTest</name>
   <url>http://maven.apache.org</url>
<build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <configuration>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <mainClass>logToKafka.App</mainClass>
            </manifest>
          </archive>
        </configuration>
      </plugin>
    </plugins>
  </build>
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
   <dependency>
              <groupId>org.apache.kafka</groupId>
              <artifactId>kafka_2.9.2</artifactId>
              <version>0.8.2.1</version>
       </dependency>

       <dependency>
              <groupId>org.slf4j</groupId>
              <artifactId>slf4j-api</artifactId>
              <version>1.7.6</version>
       </dependency>
<!--        <dependency> -->
<!--               <groupId>org.slf4j</groupId> -->
<!--               <artifactId>slf4j-log4j12</artifactId> -->
<!--               <version>1.7.12</version> -->
<!--        </dependency>  -->
       <dependency>
              <groupId>log4j</groupId>
              <artifactId>log4j</artifactId>
              <version>1.2.17</version>
       </dependency>


  </dependencies>
</project>

信用证：认识一下拉杰德夫，我和他是同事，一起工作过