Question

我想在reducer中访问myCounter.my值：

public static class Map extends Mapper<LongWritable, Text, ImmutableBytesWritable, ImmutableBytesWritable>
{
    public static enum myCounter{my};

    @Override
    public void map(LongWritable key, Text value, Context context) 
    {
        context.getCounter(myCounter.my).increment(1);
        context.write( new ImmutableBytesWritable ( ),new ImmutableBytesWritable() );
    }
}


public static class Reduce extends Reducer<ImmutableBytesWritable, ImmutableBytesWritable, Text, Text>
{
    @Override
    public void reduce(ImmutableBytesWritable key,Iterable<ImmutableBytesWritable> result,Context context)
    {

    }
}

Accessing a mapper's counter from a reducer(for old API is given ) 如何让它适用于新的API？

或

我想知道映射器输出的总数？有没有更好的方法？（我无法访问Reducer中的计数器：

Group Name->org.apache.hadoop.mapred.Task$Counter Counter Name->MAP_OUTPUT_RECORDS）

谢谢

Answer 1

通过作业对象访问计数器，使其适用于新API。

Configuration conf = context.getConfiguration();
Cluster cluster = new Cluster(conf);
Job currentJob = cluster.getJob(context.getJobID());
long val=currentJob.getCounters().findCounter(myCounter.my).getValue();

Answer 2

您可以使用相同的代码

访问reducer中的计数器值

Counter counter = context.getCounter( myCounter.my );
        counter.getValue();

also see

Answer 3

您可以按如下方式访问：

Map.myCounter.my // static fields you can access by it's class name.

如何在Reducer中访问Mapper Counter值？

3 个答案: