Question

我正在尝试将cloud-init日志转换为json，以便filebeat可以将其拾取并将其发送到Kibana。我想通过使用Shell脚本或python脚本来做到这一点。有没有将此类日志转换为json的脚本？

我的python脚本在

之下

Submit/Preview

脚本返回json格式的文件，但是msg字段返回空字符串。我想将日志的每一行都转换为消息字符串。

Answer 1

首先，请尝试使用python内置函数读取原始日志文件，而不是使用子进程运行os命令，因为：

它将更加便携（可跨OS使用）
更快，更不容易出错

按以下方式重新编写您的log_as_json函数对我有用：

inputfile = "cloud-init.log"
outputfile = "cloud-init-json.log"

def log_as_json(filename):
    # Open cloud-init log file for reading
    with open(inputfile, 'r') as log:
        # Open the output file to append json entries
        with open(outputfile, 'a') as jsonlog:
            # Read line by line
            for line in log.readlines():
                # Convert to json and write to file
                jsonlog.write(convert_to_json(line)+"\n")

Answer 2

花了一些时间准备自定义脚本后，我最终制作了以下脚本。这可能对许多其他人有帮助。

  import json

def convert_to_json_log(line):
    """ convert each line to json format """
    log = {}
    log['msg'] = json.dumps(line)
    log['logger-name'] = 'cloud-init'
    log['serviceName'] = 'content-processing'
    return json.dumps(log)

# Open the file with read only permit

f = open('/var/log/cloud-init.log', "r")

# use readlines to read all lines in the file

# The variable "lines" is a list containing all lines in the file

lines = f.readlines()

# close the file after reading the lines.
f.close()

jsonData = ''
for line in lines:
    jsonLine = convert_to_json_log(line)
    jsonData = jsonData + "\n" + jsonLine;


with open("/var/log/cloud-init/cloud-init-json.log", 'w') as new:
    new.write(jsonData)

使用脚本将cloud-init日志转换为json

2 个答案: