清理并过滤日志消息

时间:2016-07-19 13:30:57

标签: bash unix

我目前有一个脚本,该文件包含当前月份审核数据的文件,并将其作为单独的日志文件输出到另一个文件夹中。 但是,我不需要将每条消息的每一点都输出到这些文件中。消息如下所示:

01-07-2016 09:28:23:687|1200-{Context 0x1bf81010} {Id DM_SERVER} {Component ww42697_allow_me_sysadmin} {Table SYS_C_COMP} {Message {Executed 0:Ok} Chg Component: Component=AN_UST_LV_HousePricePublisher_FREE, FriSched=0000-1330, UpdatedBy=ww42697_allow_me_sysadmin} {Time 9282368} {Date 20160701} {TimeStamp 1467379703} {Record {Component AN_UST_LV_HousePricePublisher_FREE} {Enabled 1} {MonSched 0000-1730;1830-2400} {TueSched 0000-1730;1830-2400} {WedSched 0000-1730;1830-2400} {ThuSched 0000-1730;1830-2400} {FriSched 0000-1330} {SatSched } {SunSched 1830-2400} {Path /opt/mkvanalyticsnylnx/1.72_U0/bin} {Argv -classname com/citigroup/fi/analytics/usd/housepricepublisher/HousePricePublisher -appcf usd/app_rv.cf -jinit usd/HousePricePublisher.properties -log4jxml core/log4j.xml -gficfgname GTE -autocp 1 -log4jrolling 0 -pwpstop 1 -rv 1 -java -DRV_VERSION=8.2} {WorkDir /var/opt/} {DbDir } {LogsDir } {KillScript } {KillScriptArgv } {RestartIfNotReg 1} {RestartNums 5} {TimeoutKill 20} {TimeoutNotReg 60} {UpdatedBy ww42697_allow_me_sysadmin} {UpdatedInfo 0} {StopOnCrash 0} {EngineType 1} {Executable mkv_start7} {Version } {ScheduleTemplate } {CompleteExeName 0} {ComponentType } {DirtyFlag 0} {UpdId 420} } {SeqId 286414}

与“Chg User”一行:

01-07-2016 00:00:03:879|0638-{Context 0x1c316458} {Id DM_SERVER} {Component ar89576_PROD_SysAdmin} {Table SYS_C_USERS} {Message {Executed 0:Ok} Chg User: User=GTEApp, LastLoginTime=361} {Time 387} {Date 20160701} {TimeStamp 1467345603} {Record {UserID 149} {UserName GTEApp} {Password B0 C7 CC 83 FE CC D0 } {Type 2} {Passwdlen 21} {Auth 0} {AuthSource } {PasswdExt } {Disabled 0} {LoginMode 1} {DisablePasswdCheck 0} {PersonalInfo } {FailureCount 0} {LastLoginDate 20160701} {LastPwdChangeDate 0} {LastLoginTime 361} {CreationDate 0} {CreationTime 0} {LastEnableDate 0} {LinkedPolicy } {UpdatedBy ar89576_PROD_SysAdmin} {DirtyFlag 0} {UpdId 769760} } {SeqId 253929}

此外,还有一些行记录了平台的初始化。这些格式不同,我不需要:

01-07-2016 00:00:03:879|0050-PSH::Config,ADD,ENT_REQTIMEOUT,60,3,NO DESCRIPTION

我只需要输出的行包含字符串“Chg Component”(有些人用“Chg User”代替),我想摆脱每个字段周围的括号,用空格或逗号分隔就可以了。我只想了几个字段以及下面的示例:

01-07-2016 09:28:23:687,Component AN_UST_LV_HousePricePublisher_FREE,MonSched 0000-1730;1830-2400,TueSched 0000-1730;1830-2400,WedSched 0000-1730;1830-2400,ThuSched 0000-1730;1830-2400,FriSched 0000-1330,SatSched ,SunSched 1830-2400,Path /opt/mkvanalyticsnylnx/1.72_U0/bin,Argv -classname com/citigroup/fi/analytics/usd/housepricepublisher/HousePricePublisher -appcf usd/app_rv.cf -jinit usd/HousePricePublisher.properties -log4jxml core/log4j.xml -gficfgname GTE -autocp 1 -log4jrolling 0 -pwpstop 1 -rv 1 -java -DRV_VERSION=8.2,WorkDir /var/opt/,UpdatedBy ww42697_allow_me_sysadmin

我所拥有的当前脚本只能输出带有“Chg Component”的行仅适用于少量行,而且很多行不会输出到指定的日期文件......

IN=/var/opt/MKV/platform15000/DM_SERVER/LOGS/AUDITING*
grep "Chg Component" $IN > /home/rg39523/AUDIT_BY_DATE/AUDIT_COMP_CHG.log

COMP_CHG=/home/rg39523/AUDIT_BY_DATE/AUDIT_COMP_CHG.log

exec < $COMP_CHG while read line do
    date=$(echo $line|cut -d" " -f 1)
    INTWO=/home/rg39523/AUDIT_BY_DATE/AUDITING.log.$date

    echo $line >> $INTWO done

非常感谢任何帮助。 : - )

1 个答案:

答案 0 :(得分:0)

我不知道为什么您使用exec <,通常是使用cat file | while来删除{}我只使用tr ,假设字段不包含这些字符。另外我假设您不希望每行开头的原始文件名(向grep添加-h选项):

IN=/var/opt/MKV/platform15000/DM_SERVER/LOGS/AUDITING*
grep -h "Chg Component" $IN > /home/rg39523/AUDIT_BY_DATE/AUDIT_COMP_CHG.log

COMP_CHG=/home/rg39523/AUDIT_BY_DATE/AUDIT_COMP_CHG.log

cat $COMP_CHG | while read line do
    date=$(echo $line|cut -d" " -f 1)
    INTWO=/home/rg39523/AUDIT_BY_DATE/AUDITING.log.$date

    echo $line | tr -d {} >> $INTWO
done