当es安装在不同的端口上时，fluentd无法/显着连接到elasticsearch

时间：2017-09-25 15:55:39

标签： docker elasticsearch kibana fluentd

在我的docker环境中有弹性搜索（ES-1）暴露给端口9200，这是维护DB数据的主要ES，所以为了记录我们正在使用fluentd-elasticsearch-kibna堆栈和新的弹性搜索运行在9700号港口（ES-2）。

我无法将这个新的弹性搜索（ES-2）流畅地连接到端口9700.让我知道如何将流利的连接到端口以外的其他设备＃9; 9200＆＃34;

下面是用于F-E-K

的docker配置详细信息 
 fluentd.config :
    <source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match *.**>
  @type copy
  <store>
    @type elasticsearch
    host elasticsearchkibana
    port 9200
    logstash_format true
    logstash_prefix fluentd
    logstash_dateformat %Y%m%d
    include_tag_key true
    type_name access_log
    tag_key @log_name
    flush_interval 1s
  </store>
  <store>
    @type stdout
  </store>
</match>


docker-compose.yml :

   version: '2'
services:
  web:
    image: httpd
    ports:
      - "80:80"
    links:
      - fluentd
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
        tag: httpd.access

  elasticsearchkibana:
    image: elasticsearch
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    expose:
      - 9800
    ports:
      - "9700:9300"
      - "9800:9200"

  fluentd:
    build: ./fluentd
    volumes:
      - ./fluentd/conf:/fluentd/etc
    links:
      - "elasticsearchkibana"
    ports:
      - "24224:24224"
      - "24224:24224/udp"

  kibana:
    image: kibana
    environment:
      - ELASTICSEARCH_URL=http://elasticsearchkibana:9200
#      - "ES_HOST=192.168.241.82"
#      - "ES_PORT=9700"
    ports:
      - "5601:5601"

以下是我使用上述配置时获得的错误信息

fluentd_1              | 2017-09-25 15:16:36 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:37 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:37 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""}
fluentd_1              | 2017-09-25 15:16:37 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:39 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4"
fluentd_1              |   2017-09-25 15:16:37 +0000 [warn]: suppressed same stacktrace
fluentd_1              | 2017-09-25 15:16:37 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:39 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:39 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""}
fluentd_1              | 2017-09-25 15:16:39 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:43 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4"
fluentd_1              |   2017-09-25 15:16:39 +0000 [warn]: suppressed same stacktrace
fluentd_1              | 2017-09-25 15:16:39 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:43 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:43 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""}
fluentd_1              | 2017-09-25 15:16:43 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:50 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4"
fluentd_1              |   2017-09-25 15:16:43 +0000 [warn]: suppressed same stacktrace
fluentd_1              | 2017-09-25 15:16:43 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:50 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:50 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""}
fluentd_1              | 2017-09-25 15:16:50 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:04 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4"
fluentd_1              |   2017-09-25 15:16:50 +0000 [warn]: suppressed same stacktrace
fluentd_1              | 2017-09-25 15:16:50 +0000 fluent.warn: {"next_retry":"2017-09-25 15:17:04 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:04 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""}
fluentd_1              | 2017-09-25 15:17:04 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:39 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4"
fluentd_1              |   2017-09-25 15:17:04 +0000 [warn]: suppressed same stacktrace
fluentd_1              | 2017-09-25 15:17:04 +0000 fluent.warn: {"next_retry":"2017-09-25 15:17:39 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:39 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""}
fluentd_1              | 2017-09-25 15:17:39 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:18:46 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4"
fluentd_1              | 2017-09-25 15:17:39 +0000 fluent.warn: {"next_retry":"2017-09-25 15:18:46 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:18:46 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""}
fluentd_1              |   2017-09-25 15:17:39 +0000 [warn]: suppressed same stacktrace
fluentd_1              | 2017-09-25 15:18:46 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:20:49 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4"
fluentd_1              |   2017-09-25 15:18:46 +0000 [warn]: suppressed same stacktrace
fluentd_1              | 2017-09-25 15:18:46 +0000 fluent.warn: {"next_retry":"2017-09-25 15:20:49 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:20:49 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""}
root@docker-test:/home/tlirlogin/docker-compose-efk-master2# cd fluentd/conf/

由于

1 个答案:

答案 0 :(得分：0)

您需要更改您的流利配置，如下所示

fluentd.config :
<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match *.**>
  @type copy
  <store>
    @type elasticsearch
    host elasticsearchkibana
    port 9200
    logstash_format true
    logstash_prefix fluentd
    logstash_dateformat %Y%m%d
    include_tag_key true
    type_name access_log
    tag_key @log_name
    flush_interval 1s
  </store>
  <store>
    @type stdout
  </store>
</match>

所以基本上添加host elasticsearchkibana并更改为port 9200。这是因为我们应该关注容器端口而不是主机端口

相关问题
最新问题