我已经使用logstash一段时间了,apache访问日志和偶尔的mysql日志都非常成功。我刚开始将它用于qmail日志,但想要一种更好的方法来根据qmail ID对qmail日志进行分组,并能够跟踪退回或其他传递失败和状态。我已经看到了一些关于postfix而不是qmail的东西。
有没有人用qmail这样的logstash?您的logstash配置如何显示?您的Kibana仪表板看起来如何?
任何帮助都将不胜感激。
以下是一些qmail日志的示例:
Oct 15 09:26:08 imappop1-mail qmail: 1413379568.510987 new msg 33592
Oct 15 09:26:08 imappop1-mail qmail: 1413379568.511087 info msg 33592: bytes 10820 from <SmallBusinessLoan.martin.cota-martin.cota=example1.com@example.com> qp 3740 uid 89
Oct 15 09:26:08 imappop1-mail qmail: 1413379568.513616 starting delivery 1314142: msg 33592 to local example1.com-martin.cota@example1.com
Oct 15 09:26:08 imappop1-mail qmail: 1413379568.513686 status: local 1/4 remote 1/120
Oct 15 09:26:08 imappop1-mail qmail: 1413379568.576361 delivery 1314142: success: did_0+0+1/
Oct 15 09:26:08 imappop1-mail qmail: 1413379568.576491 status: local 0/4 remote 1/120
Oct 15 09:26:08 imappop1-mail qmail: 1413379568.576548 end msg 33592
Oct 15 09:26:09 imappop1-mail qmail: 1413379569.579644 new msg 33603
Oct 15 09:26:09 imappop1-mail qmail: 1413379569.579790 info msg 33603: bytes 4370 from <loansfidelity@example2.com> qp 5037 uid 89
Oct 15 09:26:09 imappop1-mail qmail: 1413379569.582804 starting delivery 1314143: msg 33603 to local example3.com-daniel@example3.com
Oct 15 09:26:09 imappop1-mail qmail: 1413379569.582967 status: local 1/4 remote 1/120
Oct 15 09:26:09 imappop1-mail qmail: 1413379569.619422 delivery 1314143: success: did_0+0+1/
Oct 15 09:26:09 imappop1-mail qmail: 1413379569.619512 status: local 0/4 remote 1/120
Oct 15 09:26:09 imappop1-mail qmail: 1413379569.619561 end msg 33603
理想情况下,我希望能够跟踪这些日志的整个解剖结构。这是我在logstash中的输入和过滤器:
{
"network": {
"servers": [ "192.168.115.61:5000" ],
"timeout": 15,
"ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"
},
"files": [
{
"paths": [
"/var/log/messages",
"/var/log/secure",
"/var/log/haraka.log",
"/var/log/maillog"
],
"fields": { "type": "syslog" }
}
]
}
input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
multiline {
pattern => "(([^\s]+)Exception.+)|(at:.+)"
stream_identity => "%{logsource}.%{@type}"
what => "previous"
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}