在hadoop集群上运行MRBenchmark的异常

时间:2014-03-16 21:50:09

标签: hadoop

我已经以分布式方式设置了hadoop并且它正常工作。我一直在尝试按照http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/

中给出的说明运行mrbenchmark

但是我得到以下异常, hadoop jar build/hadoop-0.20-test.jar mrbench -numRuns 1 -maps 2 -reduces 1 -inputLines 1 -inputType ascending MRBenchmark.0.0.2 14/03/16 21:37:03 INFO conf.ClientConfigurationUtil: Client configuration lookup disabled/failed. Using default configuration 14/03/16 21:37:04 INFO mapred.MRBench: creating control file: 1 numLines, ASCENDING sortOrder 14/03/16 21:37:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 14/03/16 21:37:04 INFO mapred.MRBench: created control file: /benchmarks/MRBench/mr_input/input_1714244267.txt 14/03/16 21:37:04 INFO fs.FileSystem: File /benchmarks/MRBench is being deleted only through Trash. Moved to trash: /benchmarks/MRBench java.lang.NullPointerException at java.util.Hashtable.put(Hashtable.java:514) at java.util.Properties.setProperty(Properties.java:161) at org.apache.hadoop.conf.Configuration.set(Configuration.java:476) at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:474) at org.apache.hadoop.mapred.MRBench.runJobInSequence(MRBench.java:180) at org.apache.hadoop.mapred.MRBench.run(MRBench.java:289) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:68) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82) at org.apache.hadoop.mapred.MRBench.main(MRBench.java:211) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) at org.apache.hadoop.test.AllTestDriver.main(AllTestDriver.java:87) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

我使用的是Hadoop 20.2版。

提前致谢。

1 个答案:

答案 0 :(得分:0)

包含参数-jar并指定一个示例jar解决了我的问题(我再次提到了mrbenchmark jar)。

例如。 hadoop jar build / hadoop-0.20-test.jar mrbench -numRuns 1 -maps 2 -reduces 1 -inputLines 1 -inputType ascending -jar build / hadoop-0.20-test.jar

即使Benchmark声明参数-jar已初始化为当前jar,但事实并非如此。必须指定-jar参数。