Question

我正在尝试在安装Hadoop之后启动一个相当简单的WordCount（我几乎跟着this tutorial），但我得到了这个：

2018-04-05 16:51:00,192 INFO mapreduce.Job: Job job_1522936330711_0007 failed with state FAILED due to: Application application_1522936330711_0007 failed 2 times due to AM Container for appattempt_1522936330711_0007_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2018-04-05 16:50:59.449]Exception from container-launch.
Container id: container_1522936330711_0007_02_000001
Exit code: 1

[2018-04-05 16:50:59.452]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error : Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster


[2018-04-05 16:50:59.452]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error : Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster


For more detailed output, check the application tracking page: http://iMac-de-chaire-2.local:8088/cluster/app/application_1522936330711_0007 Then click on links to logs of each attempt.
. Failing the application.
2018-04-05 16:51:00,211 INFO mapreduce.Job: Counters: 0

当我尝试访问日志时，它会告诉我"Failed to read the attempts of the application application_1522940117199_0001."

我有以下文件：

hadoop-env.sh：

export HADOOP_HOME=/usr/local/hadoop/

export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

yarn-site.xml：

<configuration>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>

<property>
    <name>yarn.application.classpath</name>
    <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
</property>
</configuration>

mapred-site.xml：

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
<property>
    <name>mapreduce.application.classpath</name>
    <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
</property>
</configuration>

我在macosx high-sierra，这是我第一次尝试设置Hadoop（我跟着this tutorial）。

Answer 1

我在macosx high-sierra上也遇到了同样的问题。要解决此问题，请将这些属性添加到mapred-site.xml：

<property>
 <name>yarn.app.mapreduce.am.env</name>
 <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
 <name>mapreduce.map.env</name>
 <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
 <name>mapreduce.reduce.env</name>
 <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>

Hadoop：无法找到或加载主类org.apache.hadoop.mapreduce.v2.app.MRAppMaster

1 个答案: