Question

我按照这些说明操作：
http://www.ics.uci.edu/~shantas/Install_Hadoop-2.6.0_on_Windows10.pdf

完全完成它们（使用确切的书面版本而不是更新的版本）试图安装hadoop 2.6.5。

这是我配置hdfs-site.xml的方式：

HDFS-site.xml中

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 http://www.apache.org/licenses/LICENSE-2.0
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property><name>dfs.replication</name><value>1</value></property>
<property> <name>dfs.namenode.name.dir</name><value>/hadoop-
2.6.5/data/name</value><final>true</final></property>
<property><name>dfs.datanode.data.dir</name><value>/hadoop-
2.6.5/data/data</value><final>true</final> </property>
</configuration>

当我尝试运行hadoop namenode -format时我在命令行上收到此错误：

17/11/28 23:51:24 INFO http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
17/11/28 23:51:24 INFO http.HttpServer2: Jetty bound to port 50070
17/11/28 23:51:24 INFO mortbay.log: jetty-6.1.26
17/11/28 23:51:24 INFO mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:50070
17/11/28 23:51:24 ERROR common.Util: Syntax error in URI /hadoop-
2.6.5/data/name. Please check hdfs configuration.
java.net.URISyntaxException: Illegal character in path at index 8: /hadoop-
2.6.5/data/name
        at java.net.URI$Parser.fail(URI.java:2848)
        at java.net.URI$Parser.checkChars(URI.java:3021)
        at java.net.URI$Parser.parseHierarchical(URI.java:3105)
        at java.net.URI$Parser.parse(URI.java:3063)
        at java.net.URI.<init>(URI.java:588)
        at org.apache.hadoop.hdfs.server.common.Util.stringAsURI(Util.java:48)
        at org.apache.hadoop.hdfs.server.common.Util.stringCollectionAsURIs(Util.java:98)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1435)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1390)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkConfiguration(FSNamesystem.java:675)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:729)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:539)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:598)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:765)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:749)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1446)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1512)
17/11/28 23:51:24 WARN common.Util: Path /hadoop-
2.6.5/data/name should be specified as a URI in configuration files. Please update hdfs configuration.
17/11/28 23:51:24 ERROR common.Util: Error while processing URI: /hadoop-
2.6.5/data/name
java.io.IOException: The filename, directory name, or volume label syntax is incorrect

更多错误......

    java.io.IOException: No image directories available!
        at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1099)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1091)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:150)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:945)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1387)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1512)
17/11/29 00:23:23 FATAL namenode.NameNode: Failed to start namenode.
java.io.IOException: No image directories available!
        at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1099)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1091)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:150)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:945)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1387)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1512)
17/11/29 00:23:23 INFO util.ExitUtil: Exiting with status 1
17/11/29 00:23:23 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at yonatan/192.168.1.24
************************************************************/

有什么想法吗？

顺便说一句：hadoop版似乎没问题：

C:\WINDOWS\system32>hadoop version
Hadoop 2.6.5
Subversion https://github.com/apache/hadoop.git -r e8c9fe0b4c252caf2ebf1464220599650f119997
Compiled by sjlee on 2016-10-02T23:43Z
Compiled with protoc 2.5.0
From source with checksum f05c9fa095a395faa9db9f7ba5d754
This command was run using /D:/hadoop-2.6.5/share/hadoop/common/hadoop-common-2.6.5.jar

C:\WINDOWS\system32>

编辑：在编辑了像@ cricket_007建议的XML之后，hadoop namenode -f工作了，但是当我尝试以管理员身份运行start-dfs时，我收到了这些错误：

    17/11/29 21:10:36 INFO ipc.Server: Starting Socket Reader #1 for port 50020
17/11/29 21:10:36 INFO datanode.DataNode: Opened IPC server at /0.0.0.0:50020
17/11/29 21:10:36 INFO datanode.DataNode: Refresh request received for nameservices: null
17/11/29 21:10:36 INFO datanode.DataNode: Starting BPOfferServices for nameservices: <default>
17/11/29 21:10:36 WARN common.Util: Path /hadoop-2.6.5/data/data should be specified as a URI in configuration files. Please update hdfs configuration.
17/11/29 21:10:36 INFO datanode.DataNode: Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:50071 starting to offer service
17/11/29 21:10:36 INFO ipc.Server: IPC Server listener on 50020: starting
17/11/29 21:10:36 INFO ipc.Server: IPC Server Responder: starting
17/11/29 21:10:38 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:50071. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
17/11/29 21:10:40 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:50071. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

上次修改：发现问题。需要为hadoop位置定义D驱动器

    <value>/D:/hadoop-2.6.5/data/name</value>
    <final>true</final>
  </property>
  <property>
    <name>dfs.datanode.data.dir</name>
    <value>/D:/hadoop-2.6.5/data/data</value>

Answer 1

索引8处的路径中的非法字符：

看起来你有换行符。从XML中删除它们，或至少正确格式化它们。

<!-- Put site-specific property overrides in this file. -->
<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>/hadoop-2.6.5/data/name</value>
    <final>true</final>
  </property>
  <property>
    <name>dfs.datanode.data.dir</name>
    <value>/hadoop-2.6.5/data/data</value>
    <final>true</final>
  </property>
</configuration>

我经常看到路径中提到的硬盘。例如D:\\hadoop\data。否则，我认为它默认为C盘

我还强烈建议不要将HDFS数据放在与提取的Hadoop tarball相同的位置。在真正的Hadoop环境中，namenode和datanode数据应位于完全独立的磁盘上。硬盘驱动器正在计时炸弹。

在Windows上安装Hadoop时出错

1 个答案: