Question

在我的驱动程序类中，我正在运行两个作业，我的第一个作业正在按预期工作，但在我的第二个作业中，reducer类没有执行。
下面是我的驱动程序类（JOb2配置）：

if(job.waitForCompletion(true)){
            Configuration conf2 = new Configuration();
            Job job2 = Job.getInstance(conf2);

            MultipleInputs.addInputPath(job2, inOutPath, TextInputFormat.class, CombinedUserRatingMapper.class);
            MultipleInputs.addInputPath(job2, new Path(oArgs[3]), TextInputFormat.class, MovieMapper.class);
            FileOutputFormat.setOutputPath(job2, new Path(oArgs[4]));
            job2.setReducerClass(GenreCombinedReducer.class);       
            job2.setJarByClass(GroupedAnalysisDriver.class);
            job2.setNumReduceTasks(1);
            job2.setMapOutputKeyClass(IntWritable.class);
            job2.setMapOutputValueClass(Text.class);
            job2.setOutputKeyClass(Text.class);
            job2.setOutputValueClass(Text.class);

            System.out.println(job2.waitForCompletion(true)?0:1);
        }

以下是我的两台Mappers：
映射器1：

public class CombinedUserRatingMapper extends Mapper<LongWritable, Text, IntWritable, Text> {
//age   occ mid rating
private IntWritable mid = new IntWritable();
private Text val = new Text();

public void map(LongWritable key,Text value,Context context) throws IOException, InterruptedException{
    String[] token = value.toString().split("\t");
    int  i= Integer.parseInt(token[2]);
    mid.set(i);
    val.set("C"+"\t"+token[0]+"\t"+token[1]+"\t"+token[3]);
    context.write(mid, val);
}           //  mid     age occ     rating

}

Second Mapper：

public class MovieMapper extends Mapper<LongWritable, Text, IntWritable, Text> {


public void map(LongWritable key, Text value,Context context) throws IOException, InterruptedException {
    // movies.dat - 1::Toy Story (1995)::Animation|Children's|Comedy
    //              id  name                genre
    String[] token = value.toString().split("::");
        int id = Integer.parseInt(token[0]);  //getting ID and name
        String name = token[1];
        String genre = token[2];
        context.write(new IntWritable(id), new Text("G\t"+genre));
                        //mid               G   genre
}
}

最后减少班级：

public class GenreCombinedReducer extends Reducer<IntWritable, Text, Text, Text>{
//  mid     G   genre
//  mid     C   age occ     rating
private Text outvalue=new Text();

public void Reduce(IntWritable key, Iterable<Text> values,Context context) throws IOException, InterruptedException{
    String genre = "";
    String combineInfo = "" ;
    for(Text val : values){
        if(val.charAt(0)=='G'){
            genre = val.toString().split("\t")[1];

        }else if(val.charAt(0)=='C'){
            String[] CI = val.toString().split("\t");
            combineInfo = CI[1]+CI[2]+CI[3];

        }
    }
    context.write(new Text(combineInfo),new Text(genre));
}


}

在输出文件中，我得到了mapper类的输出。我也尝试过调试，但游标从未进入Reducer类。

Answer 1

在你的Reducer课程中，你应该覆盖＆＃34;减少＆＃34;方法而不是＆＃34;减少＆＃34; 。（Java区分大小写）

Reducer代码是代码没有执行？

1 个答案: