Php rsyslog性能优化

如何设置以获得最佳性能？

• 我们可以允许某些项目在服务器崩溃时丢失或只是丢失
• 我们将把日志保存到MySQL数据库
• 我们希望能够每秒处理至少100次日志写入，延迟为0.001-0.005秒
• 我们正在从PHP应用程序编写日志

---

感谢您的帮助。

我们刚刚使用MongoDB作为数据库进行了类似的练习，因此我将记录我们所做的工作，并希望它对您有所帮助

这是我们第一次使用rsyslog，因此需要花费一些努力才能找到正确的文档并将所有内容组合在一起。最后，我们的测试驱动程序（我们使用的是SoapUI）能够通过一个使用rsyslog编写事务摘要记录的phpweb服务获得1000个TPS

我们发现了以下文章，这些文章让我们开始了学习：

概述是，您将启用rsyslog的队列基础结构，以便在守护进程的内存队列已满时将传入消息写入磁盘。在本例中，我们启用了$ActionQueueSaveOnShutdown，听起来您不需要它。然后，您将配置rsyslog规则集来解析传入消息，并将它们传递给MySQL的输出处理程序。最后，您的php脚本将使用openlog（）和syslog（）编写您想要记录的任何数据。哦，我们还必须从源代码处编译rsyslog，以便启用json/mongo插件，这本身就是一个练习。我们正在Ubuntu 12.04上使用rsyslog 7.4.5

我当然不是rsyslog方面的专家，但是我可以将我们的配置文件和代码作为起点。同样，它们是针对MongoDB的，希望它能让您知道要做什么，以及在哪里为您的实现进行更改

祝你好运

/etc/rsyslog.conf：

$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)

# Load modules for MongoDB integration: json parser and MongoDB output driver
module(load="mmjsonparse")
module(load="ommongodb")

# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Filter duplicated messages
$RepeatedMsgReduction on

# Set the default permissions for all log files.
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

# Where to place spool files
$WorkDirectory /var/spool/rsyslog

# use queue to decouple the db writes from default message handling
# From http://www.rsyslog.com/doc/rsyslog_high_database_rate.html
$MainMsgQueueFileName mainq     # set file name for main queue, also enables disk mode
$ActionQueueType LinkedList     # use asynchronous processing
$ActionQueueFileName mongodbq   # set file name for mongo db queue, enables disk mode
$ActionResumeRetryCount -1      # infinite retries on insert failure
$ActionQueueSaveOnShutdown on   # write all queue data to disk when rsyslogd is
                                #   terminated (default is off)

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

/etc/rsyslog.d/10-mongo.conf：

input(type="imuxsock" socket="/dev/log")

template(name="mongodblocal" type="subtree" subtree="$!")

# use json parser for all "local0" facility messages, 
# if parsed successfully run the template to load the
# message into the MongoDB database.
if $syslogfacility-text == 'local0' then {
        action(type="mmjsonparse")
        if $parsesuccess == "OK" then {
                # set some local vars that are appended onto the 
                # document that's written to MongoDB
                set $!time = $timestamp;
                set $!sys = $hostname;
                set $!procid = $syslogtag;
                set $!syslog_fac = $syslogfacility;
                set $!syslog_sever = $syslogpriority;
                set $!pid = $procid;
                action(type="ommongodb" server="127.0.0.1" db="test" collection="syslog" template="mongodblocal")
        }
}

/etc/rsyslog.d/50-default.conf：注意：这将禁用默认处理的“local0”消息

# First some standard log files.  Log by facility.
auth,authpriv.*         /var/log/auth.log

# don't write "local0" messages to syslog, 
# as they're processed using ommongodb (see 10-mongo.conf)
*.*;local0,auth,authpriv.none   -/var/log/syslog

kern.*              -/var/log/kern.log
mail.*              -/var/log/mail.log

# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
mail.err            /var/log/mail.err

# Logging for INN news system.
news.crit           /var/log/news/news.crit
news.err            /var/log/news/news.err
news.notice         -/var/log/news/news.notice

# Emergencies are sent to everybody logged in.
*.emerg                                :omusrmsg:*

php web服务相关调用：

// open syslog, include the process ID and open the connection to the logger 
// immediately, and use a user defined logging mechanism Local0
openlog($SCRIPT_NAME, LOG_PID | LOG_NDELAY, LOG_LOCAL0);
// note: calling closelog() is optional, and we don't use it

...
// construct $doc, which is what will be logged, change this as appropriate
// for your implementation; here $ary_headers is the request's HTTP headers,
// and $request/$response are what was posted/returned
$doc = array("headers" => $ary_headers
            ,"request" => $request
            ,"response" => $response
            );
...

// write the log entry to syslog, where it queues it and writes it to MongoDB
// NOTE: need the '@cee: ' prefix so the rsyslog json parser will process it
// See:  http://www.rsyslog.com/doc/rsyslog_conf_modules.html/mmjsonparse.html

// JSON_BIGINT_AS_STRING = Encodes large integers as their original string value.
// JSON_NUMERIC_CHECK = Encodes numeric strings as numbers.
// JSON_UNESCAPED_SLASHES = Don't escape "/".

syslog(LOG_INFO, '@cee: ' . json_encode($doc, JSON_BIGINT_AS_STRING | JSON_NUMERIC_CHECK | JSON_UNESCAPED_SLASHES));

---

这可能会有所帮助，尽管这是关于Elasticsearch，而不是MySQL：

但是它解释了rsyslog如何处理队列和线程的基本原理，并且您可以从一个新的样式配置开始。加上发送到rsyslog关于队列、规则集等文档的链接

有关新型配置格式的更多信息，请参阅以下资源：