在不同服务器上使用avro源和接收器时出错

时间:2013-10-04 07:24:45

标签: flume

我有2台服务器。日志文件附加在服务器A上。服务器B具有HBase。 所以我将Flume NG安装到服务器A并使用tail exec作为源和avro接收器,并将服务器B avro作为源和hbase作为接收器。 在服务器A上运行代理程序时,我遇到以下异常:

 2013-10-04 12:47:33,778 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -     org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event. Exception follows.
 org.apache.flume.EventDeliveryException: Failed to send events
    at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:382)
    at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
    at java.lang.Thread.run(Thread.java:679)
 Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host: sun, port: 41414 }: RPC connection error
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:161)
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:115)
    at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:590)
    at org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:88)
    at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)
    at org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:209)
    at org.apache.flume.sink.AbstractRpcSink.verifyConnection(AbstractRpcSink.java:269)
    at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:339)
    ... 3 more
 Caused by: java.io.IOException: Error connecting to sun/10.xx.xx.xx:41414
    at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
    at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
    at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:147)
    ... 10 more
 Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:597)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

在服务器B上运行flume代理时,发生以下异常:

 2013-10-04 12:27:56,006 (lifecycleSupervisor-1-4) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:253)] Unable to start EventDrivenSourceRunner: { source:Avro source avroSource: { bindAddress: stratos, port: 41414 } } - Exception follows.
 org.jboss.netty.channel.ChannelException: Failed to bind to: stratos/10.xx.xx.xx:41414
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:298)
    at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
    at org.apache.flume.source.AvroSource.start(AvroSource.java:200)
    at     org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at   org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:679)
 Caused by: java.net.BindException: Cannot assign requested address
    at sun.nio.ch.Net.bind(Native Method)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:137)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.bind(NioServerSocketPipelineSink.java:131)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleServerSocket(NioServerSocketPipelineSink.java:83)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:57)
    at org.jboss.netty.channel.Channels.bind(Channels.java:569)
    at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:186)
    at org.jboss.netty.bootstrap.ServerBootstrap$Binder.channelOpen(ServerBootstrap.java:343)
    at org.jboss.netty.channel.Channels.fireChannelOpen(Channels.java:170)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannel.<init>(NioServerSocketChannel.java:80)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:156)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:86)
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:277)
    ... 12 more
  2013-10-04 12:27:59,007 (lifecycleSupervisor-1-3) [INFO -   org.apache.flume.source.AvroSource.start(AvroSource.java:192)] Starting Avro source avroSource: { bindAddress: stratos, port: 41414 }...
 2013-10-04 12:27:59,008 (lifecycleSupervisor-1-3) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:253)] Unable to start EventDrivenSourceRunner: { source:Avro source avroSource: { bindAddress: stratos, port: 41414 } } - Exception follows.
  org.jboss.netty.channel.ChannelException: Failed to bind to stratos/10.xx.xx.xx:41414
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:298)
    at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
    at org.apache.flume.source.AvroSource.start(AvroSource.java:200)
    at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:679)
  Caused by: java.net.BindException: Cannot assign requested address
    at sun.nio.ch.Net.bind(Native Method)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:137)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.bind(NioServerSocketPipelineSink.java:131)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleServerSocket(NioServerSocketPipelineSink.java:83)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:57)
    at org.jboss.netty.channel.Channels.bind(Channels.java:569)
    at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:186)
    at org.jboss.netty.bootstrap.ServerBootstrap$Binder.channelOpen(ServerBootstrap.java:343)
    at org.jboss.netty.channel.Channels.fireChannelOpen(Channels.java:170)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannel.<init>(NioServerSocketChannel.java:80)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:156)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:86)
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:277)
    ... 12 more
服务器A上的水槽配置是:

agent1.sinks.avroSink.channel = memoryChannel
agent1.sinks.avroSink.type = avro
agent1.sinks.avroSink.hostname = sun
agent1.sinks.avroSink.port = 41414

服务器B上的水槽:

agent1.sources.avroSource.type = avro
agent1.sources.avroSource.channels = memoryChannel
agent1.sources.avroSource.bind = 10.xx.yy.zz
agent1.sources.avroSource.port = 41414

1 个答案:

答案 0 :(得分:4)

必须有B主机名。 B必须有其第n个主机名。 请参阅以下链接: http://pic.dhe.ibm.com/infocenter/bigins/v2r1/index.jsp?topic=%2Fcom.ibm.swg.im.infosphere.biginsights.admin.doc%2Fdoc%2FUserScenarioFlume.html

此致 拉吉