Spring Boot自定义滚动日志文件

时间:2020-01-08 21:03:27

标签: java json spring spring-boot logging

我们有一个spring boot应用程序,并使用带有滚动文件的logback来创建json和标准日志文件。

我们使用filebeat抓取log.json文件并推送到麋鹿服务器

我想在日志目录中添加一个op.json自定义json滚动日志文件。这是为了收集有关称为操作和性能信息的信息。

很容易,我只想使用一个管理滚动文件的类。在该类中,将有一个logOp函数,该函数会执行Java对象并对其进行序列化,并在json输出中添加一些标准日志字段。

问题: 1){不是链接,但是会很好} logback中是否有功能,使我能够创建一个命名记录器,该记录器会将字符串(在我的情况下为序列化JSON)附加到op.json日志文件的末尾,并实现滚动记录器功能

2)(如果通过登录不可用)(我会理解)。是否有一个Java程序包可以管理将字符串附加到滚动文件中的问题?我可以仅创建此“ Appender”的实例,然后向其中添加字符串。这就像是logback中的标准滚动文件追加器

我的目标是创建一个简单的JSON文件,该文件可以直接查看和/或使用filebeat进行抓取,然后发送至logstash以创建用于提高操作性能的仪表板。

-------------------- spring-boot-logger.log和spring-boot-logger.json文件的配置{用于log.debug(“ my log message},包括侦探----------

<include resource="org/springframework/boot/logging/logback/defaults.xml"/>

<springProperty scope="context" name="site" source="ci2.application"/>
<springProperty scope="context" name="app" source="spring.application.name"/>


<property name="LOGS" value="./logs" />


<!-- You can override this to have a custom pattern -->
<property name="CONSOLE_LOG_PATTERN"
          value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}"/>

<!-- Appender to log to console -->
<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
        <!-- Minimum logging level to be presented in the console logs-->
    </filter>
    <encoder>
        <pattern>${CONSOLE_LOG_PATTERN}</pattern>
        <charset>utf8</charset>
    </encoder>
</appender>



<appender name="RollingFile"
    class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOGS}/spring-boot-logger.log</file>
    <encoder>
        <pattern>${CONSOLE_LOG_PATTERN}</pattern>
        <charset>utf8</charset>
    </encoder>
    <rollingPolicy
        class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <!-- rollover daily and when the file reaches 10 MegaBytes -->
        <fileNamePattern>${LOGS}/archived/spring-boot-logger-%d{yyyy-MM-dd}.%i.log
        </fileNamePattern>
        <timeBasedFileNamingAndTriggeringPolicy
            class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
            <maxFileSize>10MB</maxFileSize>
        </timeBasedFileNamingAndTriggeringPolicy>
    </rollingPolicy>
</appender>


<appender name="Json"
          class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOGS}/spring-boot-logger.json</file>
    <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
        <providers>
            <timestamp>
                <timeZone>UTC</timeZone>
            </timestamp>
            <pattern>
                <pattern>
                    {
                    "severity": "%level",
                    "service": "${app:-}",
                    "trace": "%X{X-B3-TraceId:-}",
                    "span": "%X{X-B3-SpanId:-}",
                    "parent": "%X{X-B3-ParentSpanId:-}",
                    "exportable": "%X{X-Span-Export:-}",
                    "pid": "${PID:-}",
                    "thread": "%thread",
                    "class": "%logger{40}",
                    "rest": "%message",
                    "schema":"ci2",
                    "group":"devops",
                    "site": "${site}"

                    }
                </pattern>
            </pattern>
        </providers>
    </encoder>  
    <rollingPolicy
        class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <!-- rollover daily and when the file reaches 10 MegaBytes -->
        <fileNamePattern>${LOGS}/archived/spring-boot-logger-%d{yyyy-MM-dd}.%i.json
        </fileNamePattern>
        <timeBasedFileNamingAndTriggeringPolicy
            class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
            <maxFileSize>10MB</maxFileSize>
        </timeBasedFileNamingAndTriggeringPolicy>
    </rollingPolicy>

</appender>

<!-- LOG everything at INFO level -->
<root level="warn">
    <appender-ref ref="RollingFile" />
    <appender-ref ref="Console" />
    <appender-ref ref="Json" />
</root>

<!-- LOG everything at INFO level -->
<root name="org.springframework" level="info">
    <appender-ref ref="RollingFile" />
    <appender-ref ref="Console" />
    <appender-ref ref="Json" />
</root>


<!-- LOG "com.siemens.pl*" at INFO level -->
<logger name="com.siemens.pl" level="info" additivity="false">
    <appender-ref ref="RollingFile" />
    <appender-ref ref="Console" />
    <appender-ref ref="Json" />
</logger>

-----------------结束登录配置--------------------------- ------

我正在尝试向json日志中动态添加值。

类似

1)最高级别的支持,将地图中的值添加为生成的json中的JSON条目

 something like



    Map myValues = new HashMap<String,String> ();
    myValues.put("startTime",startTime);
    myValues.put("endTime",endTime);
    myValues.put("op",operation);
    myValues.put("user",user);
    // ideal usage format, would add map as "key":"value"  to json output
    log.debug(myValues);

2)如果将用户生成的字符串附加到日志文件(通过日志记录未添加任何内容),则可以生成日志字符串(json myKeys +标准日志记录),然后追加

    Map myValues = new HashMap<String,String> ();
    myValues.put("startTime",startTime);
    myValues.put("endTime",endTime);
    myValues.put("op",operation);
    myValues.put("user",user);

    #Generate JSON String that includes standard logging fields and dynamic
    # user fileds passed in myValues map
    String myLogStringToAppendToLogFile = generateLogString(myValues);


    #Ideal op would accept string and append user provide string to end of
    #logfile with no modification
    log.debug("RawString", myLogStringToAppendToLogFile );
    log.debug(myValues);

以上是最简单的方法,我不确定记录器上是否有可以接受地图并添加到日志的API

记录器的另一个选项是向json文件添加一组固定的属性,并且该属性值可以在用户日志行中传递。

例如,从上面的logback def -------------------

<appender name="Json"
          class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOGS}/spring-boot-logger.json</file>
    <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
        <providers>
            <timestamp>
                <timeZone>UTC</timeZone>
            </timestamp>
            <pattern>
                <pattern>
                    {
                    "severity": "%level",
                    "service": "${app:-}",
                    "trace": "%X{X-B3-TraceId:-}",
                    "span": "%X{X-B3-SpanId:-}",
                    "parent": "%X{X-B3-ParentSpanId:-}",
                    "exportable": "%X{X-Span-Export:-}",
                    "pid": "${PID:-}",
                    "thread": "%thread",
                    "class": "%logger{40}",
                    "rest": "%message",
                    "schema":"ci2",
                    "group":"devops",
                    "site": "${site}"

                    }
                </pattern>
            </pattern>
        </providers>
    </encoder>  
    <rollingPolicy
        class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <!-- rollover daily and when the file reaches 10 MegaBytes -->
        <fileNamePattern>${LOGS}/archived/spring-boot-logger-%d{yyyy-MM-dd}.%i.json
        </fileNamePattern>
        <timeBasedFileNamingAndTriggeringPolicy
            class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
            <maxFileSize>10MB</maxFileSize>
        </timeBasedFileNamingAndTriggeringPolicy>
    </rollingPolicy>

</appender>
-----------------------

In this appender def, the <pattern> is used to add values to the json def for sleuth.    

How can I define entries in this pattern and pass in the dynamic value to the logger.  It looks possible and sleuth may do it with  

                     "trace": "%X{X-B3-TraceId:-}",
                    "span": "%X{X-B3-SpanId:-}",
                    "parent": "%X{X-B3-ParentSpanId:-}",
                    "exportable": "%X{X-Span-Export:-}",

另一个有效的API是记录器将接受一个字符串并将用户提供的字符串附加到日志文件的末尾

2 个答案:

答案 0 :(得分:0)

总共: 您无需创建记录器,而需要检查附加程序。 幸运的是,Logback已经支持带有JSON布局的Console Appender的json序列化。

logback.xml / logback-spring.xml中配置控制台附加程序:

<appender name="json" class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
        <jsonFormatter
            class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
            <prettyPrint>true</prettyPrint>
        </jsonFormatter>
        <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
    </layout>
</appender>

并在POM中添加几个依赖项。

有关完整的教程,请参见here

答案 1 :(得分:0)

<appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
                <destination>${logging.logstash.uri}</destination>
                <encoder class="net.logstash.logback.encoder.LogstashEncoder">
                    <customFields>
                        {"src": {
                        "component": "${logging.logstash.src.component}",
                        "platform": "${logging.logstash.src.platform}",
                        "app": "${logging.logstash.src.app}",
                        "log_type": "${logging.logstash.src.log_type}"
                        }}
            </customFields>
        </encoder>
</appender>