从日志文件中提取信息

时间:2011-09-19 15:43:46

标签: bash shell unix

我还是bash脚本的新手,我有一个日志文件,我想在过去5天内提取这些项目。并且在最近10个小时内

日志文件是这样的:它从2002年3月到2003年1月开始有14000个项目

是否可以这样写:

awk '{print $4}' < *.log |uniq -c|sort -g|tail -n

但仍然不是我想要的

这是我正在使用的示例日志文件

172.16.0.3 - - [31/Mar/2002:19:30:41 +0200]
127.0.0.1 - stefan [01/Apr/2002:12:17:23 +0200]
213.64.153.92 - - [26/Sep/2002:02:01:58 +0200]
213.97.240.226 - - [28/Sep/2002:03:50:58 +0200] 
213.64.214.124 - - [29/Sep/2002:09:56:04 +0200]
.......
213.46.27.204 - - [01/Jan/2003:12:55:21 +0100]

1 个答案:

答案 0 :(得分:1)

这是使用'Perl'的解决方案。我修改了一点输入文件。测试一下:

输入文件数据:

$ cat ayda.txt
213.46.27.204 - - [15/Sep/2011:22:55:21 +0100]
213.46.27.204 - - [16/Sep/2011:22:55:21 +0100]
213.46.27.204 - - [17/Sep/2011:22:55:21 +0100]
213.46.27.204 - - [18/Sep/2011:22:55:21 +0100]
213.46.27.204 - - [19/Sep/2011:22:55:21 +0100]

Perl脚本:

$ cat script.pl
use warnings;
use strict;
use Time::Local;

my %MONTH = (
        Jan => 0,
        Feb => 1,
        Mar => 2,
        Apr => 3,
        May => 4,
        Jun => 5,
        Jul => 6,
        Aug => 7,
        Sep => 8,
        Oct => 9,
        Nov => 10,
        Dec => 11
);

my $SECONDS_IN_HOUR = 60 * 60;
my $SECONDS_IN_DAY = $SECONDS_IN_HOUR * 24;

## Extract and check arguments.
@ARGV == 3 or die qq[Usage: perl $0 logfile [-d|-h] num\n];
my ($time, $option) = ( pop @ARGV, pop @ARGV );
die qq[ERROR: Invalid input arguments\n] if $time =~ /\D/;

## Get the utc time to filter data file.
my $current_utc = time;
my $param_utc;
if ( $option eq "-h" ) {
        $param_utc = $current_utc - $time * $SECONDS_IN_HOUR;
}
elsif ( $option eq "-d" ) {
        $param_utc = $current_utc - $time * $SECONDS_IN_DAY;
}
else {
        die qq[ERROR: Invalid input\n];
}

## When true, print data until eof.
my $in_time = 0;

while ( <> ) {
        if ( $in_time .. eof ) {
                print;
                next;
        }

        chomp;

        m|\[(\d{2})/(\w{3})/(\d{4}):(\d{2}):(\d{2}):(\d{2})| || next;
        my $utc = timelocal( $6, $5, $4, $1, $MONTH{ $2 }, $3 - 1900 );
        if ( $param_utc - $utc <= 0 ) {
                $in_time = 1;
                print $_, qq[\n];
        }

}

运行脚本:

$ perl script.pl
Usage: perl script.pl logfile [-d|-h] num
$ perl script.pl ayda.txt -d 4
213.46.27.204 - - [16/Sep/2011:22:55:21 +0100]
213.46.27.204 - - [17/Sep/2011:22:55:21 +0100]
213.46.27.204 - - [18/Sep/2011:22:55:21 +0100]
213.46.27.204 - - [19/Sep/2011:22:55:21 +0100]
$ perl script.pl ayda.txt -h 30
213.46.27.204 - - [18/Sep/2011:22:55:21 +0100]
213.46.27.204 - - [19/Sep/2011:22:55:21 +0100]