Question

应该能够处理更大的日志文件并提供异常消息报告
日志分析完成后，将通知通知触发器报告给特定的邮件ID。还请提出哪种框架最适合处理大文件。[例如：spring boot / batch]

Answer 1

我建议使用ELK stack。将日志流化为弹性搜索并在Kibana中设置警报。

Answer 2

可以在系统上使用sendmail客户端并在该系统上运行脚本，以在发生任何异常时发送警报。

exception="Exception" # "Error", "HTTP 1.1 \" 500", etc
ignoredException="ValidationException"

# log file to scan 
logFileToScan=/var/log/tomcat8/log/application.log

# file where we will keep log of this script
logFilePath=/home/ec2-user/exception.log

# a file where we store till what line the log file has been scanned
# initalize it with 0 
countPath=/home/ec2-user/lineCount

# subject with which you want to receive the mail regading Exception
subject="[ALERT] Exception"

# from whom do you want to send the mail regarding Exception
from="abc@abc.com"

# to whom do you want to send the mail
to="xyz@xyz.com"

# number of lines, before the line containing the word to be scanned, to be sent in the mail
linesBefore=1

# number of lines, before the line containing the word to be scanned, to be sent in the mail
linesAfter=4

# start line
fromLine=`cat $countPath`

# current line count in the file
toLine=`wc -l $logFileToScan | awk '{print $1}'`

#logs are rolling so if fromLine has a value greater than toLine then fromLine has to be set to 0
if [ "$fromLine" == "" ]; then
        fromLine=0
        echo `date` fromLine values was empty, set to 0 >> $logFilePath
elif [ $fromLine -gt $toLine ]; then
        echo `date` logfile was rolled, updating fromLine from $fromLine to 0 >> $logFilePath
        fromLine=0
fi

# if from n to lines are equal then no logs has been generated since last scan
if [ "$fromLine" == "$toLine" ]; then
        echo `date` no logs genetared after last scan >> $logFilePath
else
        echo `date` updating linecount to $toLine >> $logFilePath
        echo $toLine > $countPath

        logContent=`tail -n +"$fromLine" $logFileToScan | head -n "$((toLine - fromLine))" | grep -v $ignoredException | grep -A $linesAfter -B $linesBefore $exception`
        logContent=`echo $logContent | cut -c1-2000`
        if [ "$logContent" == "" ]; then
                echo `date` no exception found >> $logFilePath
                else
                /usr/sbin/sendmail $to <<EOF
subject: $subject
from: $from

logContent=$logContent

EOF
        fi
fi

如何在Java中创建自定义日志监视作业，以提供日志文件中发生的异常消息的报告

2 个答案: