节点js风暴客户端发送数据

时间:2015-03-23 08:36:18

标签: node.js apache-storm

我尝试在Ubuntu 14.04中设置我的风暴DRPC开发环境。我已经设置了Zookeeper,Nimbus,drcp服务器,管理程序,ui,并使它们运行并提交了拓扑。 Storm UI snapshot

然后我有以下node.js客户端代码,名为NodeClient.js,当我尝试运行nodejs NodeClient.js时,没有任何反应。我的Nodejs客户端出了什么问题,或者其他什么是错的。 running node client。我想client.execute发送" hello world"数据包到风暴队列。我是对的吗?

======================== NodeClient.js =================

var DRPC = require('storm-drpc-node');


var client = DRPC({
    host: '127.0.0.1',
    port: 3772,
    timeout: 1000,
    keepAlive: true,
    maxConnectCounts: 30
});



client.on('error', function(err) {
    throw err;
});



// callback way 
client.execute('spout-name', 'hello world', function(err, res) {
    if(err) throw err;
    else console.log(res);

});

======被修改====== supervisor started 我开始担任主管,现在它如上图所示运行, 为了使上面的NodeClient.js工作,我是否需要下载npm install storm-node,并创建spout。并且NodeClient应该修改如下:?

===============我需要创建spout ==============

 var storm = require('node-storm');
 var DRPC = require('storm-drpc-node');
var client = DRPC({
    host: '127.0.0.1',
    port: 3772,
    timeout: 1000,
    keepAlive: true,
    maxConnectCounts: 30
});

var myspout = storm.spout(function(sync) {
    // For an unreliable emit:
    this.emit([fieldValue1, fieldValue2])

    // For a reliable emit:
    this.emit([fieldValue1, fieldValue2], {id: 'some unique id'})

    // Tell storm we're done emitting tuples for now
    sync()
})
.declareOutputFields(["field1", "field2"]) // declare output fields
.on('fail', function(data) {
    // Handle tuple failure
   console.log('data is not send in myspount');
})
.on('ack', function(data) {
    // Handle tuple acknowledgement
    console.log('data is in myspout');
});

client.on('error', function(err) {
    throw err;
});

// callback way
client.execute('myspout',JSON.stringify('hello world'), function(err, res) {
    if(err) throw err;
    else console.log(res);
});

=============编辑2 ============ 现在看来服务器部分工作正常。请检查UI仪表板上的拓扑和Spout。我正在使用apache-storm 0.9.3的例子,BasicDRPCTopology。我使用remoteTopology。在代码中,没有Spout。我想知道仪表板上的这个喷口来自哪里?我如何使用Nodejs作为Spout? new topology spout

//命令提交拓扑 ./storm jar ../examples/storm-starter/storm-starter-topologies-0.9.3.jar storm.starter.BasicDRPCTopology callstatio

======================= BasicDRPCTopology ============

public static void main(String[] args) throws Exception {
    LinearDRPCTopologyBuilder builder = new LinearDRPCTopologyBuilder("exclamation");
    builder.addBolt(new ExclaimBolt(), 3);

    Config conf = new Config();

    if (args == null || args.length == 0) {
      LocalDRPC drpc = new LocalDRPC();
      LocalCluster cluster = new LocalCluster();

      cluster.submitTopology("drpc-demo", conf, builder.createLocalTopology(drpc));




for (String word : new String[]{ "hello", "goodbye" }) {
    System.out.println("Result for \"" + word + "\": " + drpc.execute("exclamation", word));
  }

  cluster.shutdown();
  drpc.shutdown();
}
else {
  conf.setNumWorkers(3);
  StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createRemoteTopology());
}

==================编辑3 =============

我把以下Java客户端DRPC,它也不起作用。我的客户部分有问题吗?我在代码之后附上了我的storm.yaml:任何提示?

public class TestSpout {

public static void main(String[] args) throws TException, DRPCExecutionException {
        DRPCClient client = new DRPCClient("127.0.0.1", 3772);
        String result = client.execute("callstatio", "hello world");
        System.out.println("result is:"+result);    
    }

}

====== storm.yaml ====

storm.zookeeper.servers:
     - "localhost"

nimbus.host: "localhost"
storm.zookeeper.port: 2181
storm.local.dir: "/var/stormtmp/"
java.library.path: "/usr/local/lib/"
supervisor.slots.ports:
  -6700
  -6701
  -6702
  -6703

worker.childopts: "-Xmx768m"
nimbus.childopts: "-Xmx512m"
supervisor.childopts: "-Xmx256m"


 drpc.servers:
     - "127.0.0.1"

================编辑4 = 启动./storm supervisor会出现以下异常:任何提示?     java.lang.IllegalArgumentException:不知道如何创建ISeq:java.lang.Integer

1 个答案:

答案 0 :(得分:1)

您需要使用

启动DRPC风暴
storm drpc 

然后在storm.yaml中添加DRPC服务器的URL

drpc.servers:
   - "my.ip.com"

别忘了使用

LinearDRPCTopologyBuilder

而不是传统的TopologyBuilder'在构建拓扑时。

从风暴文档(本地模式等)

中查看更多内容

https://storm.apache.org/documentation/Distributed-RPC.html