Question

我在map.java类中尝试通过此

发送文件名

context.write（new Text（stringWord），new Text（fileName））

recude.java

 import java.io.IOException;

 import org.apache.hadoop.io.*;
 import org.apache.hadoop.mapreduce.*;

 public class Reduce extends Reducer<Text, Text, Text, Text> {

@Override
public void reduce(Text key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
    int sum = 0;
    MapWritable occurenceInFile = new MapWritable();
    for (Text val : values) {
        if(occurenceInFile.containsKey(val)) {
            occurenceInFile.put(val, new IntWritable(1));
        } else {
            IntWritable occurence = (IntWritable) occurenceInFile.get(val);
            occurenceInFile.put(val, new IntWritable(occurence.get() + 1));
        }
        sum += 1;
    }
    String result = key.toString() + " (";
    boolean flag = false;
    for (Writable filenameWritable : occurenceInFile.keySet()) {
        if (flag) {
            result += ", ";
        }
        String filename = ((Text) filenameWritable).toString();
        result += filename + "=" + ((IntWritable) occurenceInFile.get(filenameWritable)).toString();
        flag = true;
    }
    result += ")\nTotal occurence of \"" + key + "\": " + Integer.toString(sum);

    context.write(key, new Text(result));
}
}

错误代码此错误代码显示在stderr

中

Error: java.lang.NullPointerException
at Reduce.reduce(Reduce.java:18)
at Reduce.reduce(Reduce.java:6)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:180)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:656)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:394)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:166)

我开始写一个wordcount程序，但我仍然不知道如何解决这个错误。

Answer 1

我猜行号＃18就是这个：

occurenceInFile.put(val, new IntWritable(occurence.get() + 1));

因此，前一行读取的occurence变量值可能为null。另一个变体是occurence.get()值为null。

因此，很难建议究竟是什么导致它，但你应该调试你的程序并找到，为什么occurenceInFile.get(val);可以返回null值。

错误：map reduce程序的java.lang.NullPointerException错误

1 个答案: