Cygnus 0.8.2不起作用

时间:2015-07-23 11:49:28

标签: fiware-cygnus

我已经在装有CentOS-6.5-x64的VM中安装了Cygnus 0.8.2。 在配置文件agent.conf中,我只更改以下内容:

cygnusagent.sinks.hdfs-sink.oauth2_token = xxxxxx
cygnusagent.sinks.hdfs-sink.hdfs_username = myuser

我用这个命令运行cygnus:

/usr/cygnus/bin/cygnus-flume-ng agent --conf /usr/cygnus/conf/ -f /usr/cygnus/conf/agent.conf -n cygnusagent -Dflume.root.logger=INFO,console

但是当我发送xml消息时出现此错误:

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.)
2015-07-23 14:46:02,133 (SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN - com.telefonica.iot.cygnus.sinks.OrionSink.process(OrionSink.java:163)] The event TTL has expired, it is no more re-injected in the channel (id=617320308, ttl=0)
2015-07-23 14:46:02,133 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - com.telefonica.iot.cygnus.sinks.OrionSink.process(OrionSink.java:193)] Finishing transaction (1437651766-108-0000000000)
2015-07-23 14:46:02,637 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - com.telefonica.iot.cygnus.sinks.OrionSink.process(OrionSink.java:128)] Event got from the channel (id=617320308, headers={timestamp=1437651946136, content-type=application/xml, transactionId=1437651766-108-0000000000, fiware-service=def_serv, fiware-servicepath=def_servpath, ttl=0, destination=sensorreading1_sensorreading}, bodyLength=891)
2015-07-23 14:46:02,643 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - com.telefonica.iot.cygnus.sinks.OrionHDFSSink.persist(OrionHDFSSink.java:356)] [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file (def_serv/def_servpath/sensorreading1_sensorreading/sensorreading1_sensorreading.txt), Data ({"recvTime":"2015-07-23T11:45:46.136Z","nodeid":"1", "nodeid_md":[],"sensorid":"1", "sensorid_md":[],"systemid":"1", "systemid_md":[],"value":"-990.6", "value_md":[]})
2015-07-23 14:46:02,644 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - com.telefonica.iot.cygnus.sinks.OrionSink.process(OrionSink.java:143)] Persistence error (The /user/cristina.albaladejo/def_serv/def_servpath/sensorreading1_sensorreading directory could not be created in HDFS. HttpFS response: 503 Service unavailable)
2015-07-23 14:46:02,644 (SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN - com.telefonica.iot.cygnus.sinks.OrionSink.process(OrionSink.java:163)] The event TTL has expired, it is no more re-injected in the channel (id=617320308, ttl=0)
2015-07-23 14:46:02,644 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - com.telefonica.iot.cygnus.sinks.OrionSink.process(OrionSink.java:193)] Finishing transaction (1437651766-108-0000000000)

然后,数据不存储在HDFS中......我怎么解决这个问题?

完整的配置是:

cygnusagent.sources = http-source
cygnusagent.sinks = hdfs-sink mysql-sink ckan-sink
cygnusagent.channels = hdfs-channel mysql-channel ckan-channel

cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
cygnusagent.sources.http-source.port = 5050
cygnusagent.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.OrionRestHandler
cygnusagent.sources.http-source.handler.notification_target = /notify
cygnusagent.sources.http-source.handler.default_service = def_serv
cygnusagent.sources.http-source.handler.default_service_path = def_servpath
cygnusagent.sources.http-source.handler.events_ttl = 10
cygnusagent.sources.http-source.interceptors = ts gi
cygnusagent.sources.http-source.interceptors.ts.type = timestamp
cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf

# OrionHDFSSink configuration
cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
cygnusagent.sinks.hdfs-sink.type = com.telefonica.iot.cygnus.sinks.OrionHDFSSink
cygnusagent.sinks.hdfs-sink.hdfs_host = x1.y1.z1.w1,x2.y2.z2.w2
cygnusagent.sinks.hdfs-sink.hdfs_port = 14000
cygnusagent.sinks.hdfs-sink.hdfs_username = cristina.albaladejo
cygnusagent.sinks.hdfs-sink.oauth2_token = MYTOKEN
cygnusagent.sinks.hdfs-sink.attr_persistence = column
cygnusagent.sinks.hdfs-sink.hive_host = x.y.z.w
cygnusagent.sinks.hdfs-sink.hive_port = 10000
cygnusagent.sinks.hdfs-sink.krb5_auth = false
cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username
cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx
cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf
cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf

cygnusagent.channels.hdfs-channel.type = memory
cygnusagent.channels.hdfs-channel.capacity = 1000
cygnusagent.channels.hdfs-channel.transactionCapacity = 100

1 个答案:

答案 0 :(得分:1)

您好像没有配置以下属性(放置所需的值):

cygnusagent.sinks.hdfs-sink.hdfs_host = cosmos.lab.fiware.org
cygnusagent.sinks.hdfs-sink.hive_host = cosmos.lab.fiware.org

此外,由于您只是配置HDFS持久性,因此请避免引用mysql-sinkckan-sink,即:

cygnusagent.sinks = hdfs-sink
cygnusagent.channels = hdfs-channel

cygnusagent.sources.http-source.channels = hdfs-channel

尝试使用建议的更改,并告诉我它是否有效。