Question

我将带有Logstash的Glassfish 4日志文件发送到ElasticSearch接收器。如何从Logstash中删除消息字段中的尾随换行符？

我的活动如下：

{
  "@timestamp" => "2013-11-21T13:29:33.081Z",
  "message" => "[2013-11-21T13:29:32.577+0000] [glassfish 4.0] [INFO] [] [javax.resourceadapter.mqjmsra.lifecycle] [tid: _ThreadID=142 _ThreadName=Thread-43] [timeMillis: 1385040572577] [levelValue: 800] [[\n  MQJMSRA_RA1101: GlassFish MQ JMS Resource Adapter stopped.]]\n",
  "@version" => "1",
  "tags" => ["multiline", "date_filtered"],
  "host" => "myhost",
  "path" => "../server.log"
}

Answer 1

第二种解决方案是使用mutate filter of Logstash。它允许您去除字段的值。

filter {
  # Remove leading and trailing whitspaces (including newline etc. etc.)
  mutate {
    strip => "message"
  }
}

Answer 2

您必须使用具有正确模式的多行过滤器，以告诉logstash，具有前面空白行的每行都属于之前的行。将此行添加到您的conf文件中。

filter{
  ...
  multiline {
    type => "gflogs"
    pattern => "\[\#\|\d{4}"
    negate => true
    what => "previous"
  }
  ...
}

您还可以包含grok插件来处理时间戳并过滤索引的不规则行。

在同一台计算机上查看具有单个logstash实例的完整堆栈

input {
  stdin {
    type => "stdin-type"
  }

  file {
    path => "/path/to/glassfish/logs/*.log"
    type => "gflogs"
  }
}

filter{
  multiline {
    type => "gflogs"
    pattern => "\[\#\|\d{4}"
    negate => true
    what => "previous"
  }

  grok {
    type => "gflogs"
    pattern => "(?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL:loglevel}\|%{DATA:server_version}\|%{JAVACLASS:category}\|%{DATA:kv}\|%{DATA:message}\|\#\]"
    named_captures_only => true
    singles => true
  }

  date {
    type => "gflogs"
    match => [ "timestamp", "ISO8601" ]
  }

  kv {
    type => "gflogs"
    exclude_tags => "_grokparsefailure"
    source => "kv"
    field_split => ";"
    value_split => "="
  }
}

output {
  stdout { codec => rubydebug }
  elasticsearch { embedded => true }
}

这对我有用。请看logstash-usergroup上的这篇文章。我也可以建议最新的logstash book。它也是支持logstash作者工作的好方法。

希望能参加任何JUG-Berlin活动！

如何从消息字段中删除尾随换行符

2 个答案: