hadoop - Hadoop tasktracker问题

这是一个奇怪的问题。当我的lappy断电并关机时，我正在运行MR工作（单节点）。感谢ubuntu 10.04中缺少“无电池”警告

当我再次启动并尝试启动hadoop（单节点）时，一切顺利，当我从命令行发出“jps”时，列出了所有主进程和从进程。我也试过发布“netstat -plten | grep java”。

命令行输出：

hduser@fox:~$ jps
10476 DataNode
1819 
10696 JobTracker
10813 TaskTracker
11741 Jps
10605 SecondaryNameNode
10360 NameNode

hduser@fox:~$ sudo netstat -plten | grep java
tcp        0      0 0.0.0.0:55376           0.0.0.0:*               LISTEN      1000       141925      10476/java      
tcp        0      0 0.0.0.0:42352           0.0.0.0:*               LISTEN      1000       141790      10360/java      
tcp        0      0 0.0.0.0:50070           0.0.0.0:*               LISTEN      1000       142727      10360/java      
tcp        0      0 0.0.0.0:50010           0.0.0.0:*               LISTEN      1000       142818      10476/java      
tcp        0      0 0.0.0.0:50075           0.0.0.0:*               LISTEN      1000       143308      10476/java      
tcp        0      0 0.0.0.0:50020           0.0.0.0:*               LISTEN      1000       146358      10476/java      
tcp        0      0 127.0.0.1:38533         0.0.0.0:*               LISTEN      1000       143217      10813/java      
tcp        0      0 102.173.156.27:54310   0.0.0.0:*               LISTEN      1000       141913      10360/java      
tcp        0      0 102.173.156.27:54311   0.0.0.0:*               LISTEN      1000       142359      10696/java      
tcp        0      0 0.0.0.0:50090           0.0.0.0:*               LISTEN      1000       142827      10605/java      
tcp        0      0 0.0.0.0:56779           0.0.0.0:*               LISTEN      1000       142191      10605/java      
tcp        0      0 0.0.0.0:55373           0.0.0.0:*               LISTEN      1000       142251      10696/java      
tcp        0      0 0.0.0.0:50030           0.0.0.0:*               LISTEN      1000     142921      10696/java

我检查了作业跟踪器日志文件：

2013-01-09 16:40:52,286 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery for block null bad datanode[0] nodes == null
2013-01-09 16:40:52,286 WARN org.apache.hadoop.hdfs.DFSClient: Could not get block locations. Source file "/home/hduser/hadoop/tmpnew/mapred/system/jobtracker.info" - Aborting...
2013-01-09 16:40:52,286 WARN org.apache.hadoop.mapred.JobTracker: Writing to file hdfs://master:54310/home/hduser/hadoop/tmpnew/mapred/system/jobtracker.info failed!
2013-01-09 16:40:52,287 WARN org.apache.hadoop.mapred.JobTracker: FileSystem is not ready yet!
2013-01-09 16:40:52,290 WARN org.apache.hadoop.mapred.JobTracker: Failed to initialize recovery manager. 
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /home/hduser/hadoop/tmpnew/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
    at org.apache.hadoop.ipc.Client.call(Client.java:1070)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
    at $Proxy5.addBlock(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
    at $Proxy5.addBlock(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3510)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3373)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2589)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2829)
2013-01-09 16:41:02,291 WARN org.apache.hadoop.mapred.JobTracker: Retrying...
2013-01-09 16:41:02,331 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /home/hduser/hadoop/tmpnew/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)

    at org.apache.hadoop.ipc.Client.call(Client.java:1070)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
    at $Proxy5.addBlock(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
    at $Proxy5.addBlock(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3510)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3373)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2589)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2829)

我无法使用URL - master：50060

查看tasktracker

因此我觉得tasktracker没有正确启动。

帮助表示赞赏。

Hadoop tasktracker问题

2 个答案: