Datanode和Namenode运行但未反映在UI

时间:2016-04-28 02:30:07

标签: hadoop hadoop2 bigdata

我在Hadoop中配置我的Master和Slave时遇到了一个小小的挫折,我在Master和Slave中同时拥有了我的namenode和datanode。

然而,WebUI中的LiveNodes计数没有得到反映,但节点正在运行。

我已经尝试过禁用防火墙并格式化节点,但我无法解决问题。

任何帮助都将受到高度赞赏!!!

以下是摘录:

主人:

jps命令输出:

5088 Jps

4446 NameNode

4681 SecondaryNameNode

奴隶:

jps命令输出:

2478 Jps

2410 DataNode

ubuntu@hadoop-master:/usr/local/hadoop/etc/hadoop$ $HADOOP_HOME/bin/hdfs dfsadmin -refreshNodes
16/04/28 02:22:37 WARN ipc.Client: Address change detected. Old: hadoop-master/52.200.230.29:50077 New: hadoop-master/127.0.0.1:50077
refreshNodes: Call From hadoop-master/127.0.0.1 to hadoop-master:50077 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

hadoop-slave-1的日志文件:

2016-04-28 21:23:07,248 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:12,257 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:17,265 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:22,273 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:27,282 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000
2016-04-28 21:23:32,291 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/52.200.230.29:9000

Hadoop-master的日志文件:

2016-04-28 21:21:04,002 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 127.0.0.1
2016-04-28 21:21:04,002 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
2016-04-28 21:21:04,002 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 407
2016-04-28 21:21:04,002 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 22
2016-04-28 21:21:04,003 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 23
2016-04-28 21:21:04,003 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /usr/local/hadoop/hadoop_data/hdfs/namenode/current/edits_inprogress_0000000000000000407 -> /usr/local/hadoop/hadoop_data/hdfs/namenode/current/edits_0000000000000000407-0000000000000000408
2016-04-28 21:21:04,004 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 409

我主人的netstat -pant命令:

ubuntu@hadoop-master:/usr/local/hadoop/etc/hadoop$ netstat -pant
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:50070           0.0.0.0:*               LISTEN      21491/java
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:50077           0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:50078         0.0.0.0:*               LISTEN      21491/java
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:50090           0.0.0.0:*               LISTEN      21726/java
tcp        0      0 172.31.63.189:50070     128.235.8.68:57225      ESTABLISHED 21491/java
tcp        0      0 127.0.0.1:41471         127.0.0.1:50078         TIME_WAIT   -
tcp        0    124 172.31.63.189:22        128.235.8.68:56950      ESTABLISHED -
tcp        0      0 172.31.63.189:50070     128.235.8.68:57224      ESTABLISHED 21491/java
tcp        0      0 172.31.63.189:50070     128.235.8.68:57223      ESTABLISHED 21491/java
tcp        0      0 172.31.63.189:22        128.235.8.68:57084      ESTABLISHED -
tcp        0      0 172.31.63.189:22        58.218.204.215:39402    ESTABLISHED -
tcp        0      0 172.31.63.189:50070     128.235.8.68:57227      ESTABLISHED 21491/java
tcp        0      0 172.31.63.189:50070     128.235.8.68:57228      ESTABLISHED 21491/java
tcp        0      0 172.31.63.189:50070     128.235.8.68:57226      ESTABLISHED 21491/java
tcp6       0      0 :::22                   :::*                    LISTEN      -
tcp6       0      0 :::50077                :::*                    LISTEN      -
tcp6       0      0 :::9000                 :::*                    LISTEN      -

1 个答案:

答案 0 :(得分:0)

拒绝连接 我可以从你的帖子中看到这个错误。我想你需要做3件事

  1. 确保50077端口正在被进程监听,这是您的hadoop进程
  2. 确保使用某些工具(例如telnet
  3. )可以访问它
  4. 除了防火墙。 selinux也可以影响访问。所以关闭它并重新启动服务再试一次
相关问题