Elasticsearch长双映射不起作用

时间:2016-05-04 10:38:21

标签: elasticsearch logstash

我正在使用logstash将日志从DynamoDB推送到ES:

filter {
    json {
        source => "message"
        target => "doc"
    }
    mutate {
        convert => {
            "[doc][dynamodb][keys][customerId][N]" => "integer"
            "[doc][dynamodb][newImage][callDate][N]" => "integer"
            "[doc][dynamodb][newImage][price][S]" => "float"
        }
    }
    date {
        match => [ "[doc][dynamodb][newImage][callDate][N]", "UNIX" ]
        target => "@timestamp"
    }
}

output {
    elasticsearch {
        hosts => ["localhost"]
        codec => "json"
        index => "cdr-%{+YYYY.MM.dd}"
        document_type => "cdr"
        document_id => "%{[doc][dynamodb][keys][uniqueId][S]}"
        template_name => "cdr"
        template => "/opt/logstash/templates/logstash_dynamodb_template.json"
        template_overwrite => true
    }
    stdout { } 
}

实际上,mutate.convert不做任何更改,无论是删除还是添加。

{
    "order": 0,
    "template": "cdr*",
    "settings": {
        "index.refresh_interval": "5s"
    },
    "mappings": {
        "cdr": {
            "dynamic": "false",
            "_all": {
                "enabled": false
            },
            "properties": {
                "doc": {
                    "properties": {
                        "dynamodb": {
                            "properties": {
                                "keys": {
                                    "properties": {
                                        "customerId": {
                                            "properties": {
                                                "N": {
                                                    "type": "long"
                                                }
                                            }
                                        },
                                        "uniqueId": {
                                            "properties": {
                                                "S": {
                                                    "type": "string",
                                                    "index": "not_analyzed"
                                                }
                                            }
                                        }
                                    }
                                },
                                "newImage": {
                                    "properties": {
                                        "callDate": {
                                            "properties": {
                                                "N": {
                                                    "type": "date",
                                                    "format": "epoch_second"
                                                }
                                            }
                                        },
                                        "direction": {
                                            "properties": {
                                                "S": {
                                                    "type": "string",
                                                    "index": "not_analyzed"
                                                }
                                            }
                                        },
                                        "disposition": {
                                            "properties": {
                                                "S": {
                                                    "type": "string",
                                                    "index": "not_analyzed"
                                                }
                                            }
                                        },
                                        "price": {
                                            "properties": {
                                                "S": {
                                                    "type": "double"
                                                }
                                            }
                                        },
                                        "uniqueId": {
                                            "properties": {
                                                "S": {
                                                    "type": "string",
                                                    "index": "not_analyzed"
                                                }
                                            }
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

是的,doc.message包含所有描述的字段,但它们未映射。这是ES的截图: ES interface 正如您所看到的,只有字符串映射可以正常工作。

查询时出错:No mapping found for [doc.dynamodb.newImage.callDate.N] in order to sort on

有谁知道这种行为的原因是什么?

提示:logstash debug bin/logstash -f filters/logstash-dynamodb.conf --debug不包含任何错误。

提前感谢任何想法。

0 个答案:

没有答案