Question

我正在尝试使用Flume将文件从我的Windows机器加载到HDFS。

我收到以下错误：

12:42:02 WARN hdfs.HDFSEventSink: HDFS IO error
java.io.IOException: Incomplete HDFS URI, no host: hdfs://10.74.xxx.217:9000:/user/urmi/FlumeData.1374649892113
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:118)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2150)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2184)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2166)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:302)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
        at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:123)
        at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:183)
        at org.apache.flume.sink.hdfs.HDFSEventSink$1.doCall(HDFSEventSink.java:432)
        at org.apache.flume.sink.hdfs.HDFSEventSink$1.doCall(HDFSEventSink.java:429)
        at org.apache.flume.sink.hdfs.HDFSEventSink$ProxyCallable.call(HDFSEventSink.java:164)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

Answer 1

URL中的端口号后面有一个额外的冒号（:)。所以正确的URL应该是

HDFS：//10.74.162.217：9000 /

Flume中的HDFS IO错误

1 个答案: