如何搜索与密钥关联的术语而不是搜索该术语的所有出现?

时间:2018-03-28 07:50:11

标签: elasticsearch

我有一个文本文件,其中包含Apache logs的一些JSON表示。我已经从文件中发布了两个这样的记录。

{'remote_host': '83.149.9.216',
 'remote_logname': '-',
 'remote_user': '-',
 'request_first_line': 'GET '
                       '/presentations/logstash-monitorama-2013/images/kibana-search.png '
                       'HTTP/1.1',
 'request_header_referer': 'http://semicomplete.com/presentations/logstash-monitorama-2013/',
 'request_header_user_agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) '
                              'AppleWebKit/537.36 (KHTML, like Gecko) '
                              'Chrome/32.0.1700.77 Safari/537.36',
 'request_http_ver': '1.1',
 'request_method': 'GET',
 'request_url': '/presentations/logstash-monitorama-2013/images/kibana-search.png',
 'request_url_fragment': '',
 'request_url_hostname': None,
 'request_url_netloc': '',
 'request_url_password': None,
 'request_url_path': '/presentations/logstash-monitorama-2013/images/kibana-search.png',
 'request_url_port': None,
 'request_url_query': '',
 'request_url_query_dict': {},
 'request_url_query_list': [],
 'request_url_query_simple_dict': {},
 'request_url_scheme': '',
 'request_url_username': None,
 'response_bytes_clf': '203023',
 'status': '200',
 'time_received': '[17/May/2015:10:05:03 +0000]',
 'time_received_datetimeobj': datetime.datetime(2015, 5, 17, 10, 5, 3),
 'time_received_isoformat': '2015-05-17T10:05:03',
 'time_received_tz_datetimeobj': datetime.datetime(2015, 5, 17, 10, 5, 3, tzinfo='0000'),
 'time_received_tz_isoformat': '2015-05-17T10:05:03+00:00',
 'time_received_utc_datetimeobj': datetime.datetime(2015, 5, 17, 10, 5, 3, tzinfo='0000'),
 'time_received_utc_isoformat': '2015-05-17T10:05:03+00:00'}
{'remote_host': '83.149.9.216',
 'remote_logname': '-',
 'remote_user': '-',
 'request_first_line': 'GET '
                       '/presentations/logstash-monitorama-2013/images/kibana-dashboard3.png '
                       'HTTP/1.1',
 'request_header_referer': 'http://semicomplete.com/presentations/logstash-monitorama-2013/',
 'request_header_user_agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) '
                              'AppleWebKit/537.36 (KHTML, like Gecko) '
                              'Chrome/32.0.1700.77 Safari/537.36',
 'request_http_ver': '1.1',
 'request_method': 'GET',
 'request_url': '/presentations/logstash-monitorama-2013/images/kibana-dashboard3.png',
 'request_url_fragment': '',
 'request_url_hostname': None,
 'request_url_netloc': '',
 'request_url_password': None,
 'request_url_path': '/presentations/logstash-monitorama-2013/images/kibana-dashboard3.png',
 'request_url_port': None,
 'request_url_query': '',
 'request_url_query_dict': {},
 'request_url_query_list': [],
 'request_url_query_simple_dict': {},
 'request_url_scheme': '',
 'request_url_username': None,
 'response_bytes_clf': '171717',
 'status': '200',
 'time_received': '[17/May/2015:10:05:43 +0000]',
 'time_received_datetimeobj': datetime.datetime(2015, 5, 17, 10, 5, 43),
 'time_received_isoformat': '2015-05-17T10:05:43',
 'time_received_tz_datetimeobj': datetime.datetime(2015, 5, 17, 10, 5, 43, tzinfo='0000'),
 'time_received_tz_isoformat': '2015-05-17T10:05:43+00:00',
 'time_received_utc_datetimeobj': datetime.datetime(2015, 5, 17, 10, 5, 43, tzinfo='0000'),
 'time_received_utc_isoformat': '2015-05-17T10:05:43+00:00'}

注意:我已经打印好了记录以便于阅读,否则所有这些都在一个文件中,我已成功将数据推送到弹性云。

现在,当我搜索一个名为200的术语时,它会返回状态代码,它会返回包含搜索字词200的所有相关文档。

我的卷曲命令

curl -H 'Content-Type: application/json' -k -u user:password -XGET https://5a482b9559f*********************.ap-southeast-1.aws.found.io:9243/apache/logs/_search?pretty=true -d '{"query": {"query_string": {"query": "200"}}}'

{
  "took" : 12,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 8065,
    "max_score" : 4.90325,
    "hits" : [
      {
        "_index" : "apache",
        "_type" : "logs",
        "_id" : "qZNUa2IBlesaiAW_X5xr",
        "_score" : 4.90325,
        "_source" : {
          "remote_host" : "69.165.204.172",
          "request_url_netloc" : "",
          "request_url_port" : null,
          "request_url_query" : "",
          "request_url_query_list" : [ ],
          "request_url_query_simple_dict" : { },
          "request_url_scheme" : "",
          "request_method" : "GET",
          "time_received_tz_isoformat" : "2015-05-18T18:05:38+00:00",
          "response_bytes_clf" : "175208",
          "request_url" : "/presentations/logstash-scale11x/images/ahhh___rage_face_by_samusmmx-d5g5zap.png",
          "time_received" : "[18/May/2015:18:05:38 +0000]",
          "request_header_referer" : "http://s-chassis.co.nz/viewtopic.php?f=16&t=9265&start=200",
          "request_first_line" : "GET /presentations/logstash-scale11x/images/ahhh___rage_face_by_samusmmx-d5g5zap.png HTTP/1.1",
          "request_url_username" : null,
          "request_url_query_dict" : { },
          "time_received_utc_isoformat" : "2015-05-18T18:05:38+00:00",
          "request_http_ver" : "1.1",
          "time_received_isoformat" : "2015-05-18T18:05:38",
          "request_header_user_agent" : "Mozilla/5.0 (Windows NT 5.1; rv:27.0) Gecko/20100101 Firefox/27.0",
          "request_url_hostname" : null,
          "status" : "200",
          "request_url_fragment" : "",
          "remote_logname" : "-",
          "request_url_path" : "/presentations/logstash-scale11x/images/ahhh___rage_face_by_samusmmx-d5g5zap.png",
          "remote_user" : "-",
          "request_url_password" : null
        }
      },
      {
        "_index" : "apache",
        "_type" : "logs",
        "_id" : "1ZNUa2IBlesaiAW_X51s",
        "_score" : 4.90325,
        "_source" : {
          "remote_host" : "110.143.13.225",
          "request_url_netloc" : "",
          "request_url_port" : null,
          "request_url_query" : "",
          "request_url_query_list" : [ ],
          "request_url_query_simple_dict" : { },
          "request_url_scheme" : "",
          "request_method" : "GET",
          "time_received_tz_isoformat" : "2015-05-18T20:05:30+00:00",
          "response_bytes_clf" : "175208",
          "request_url" : "/presentations/logstash-scale11x/images/ahhh___rage_face_by_samusmmx-d5g5zap.png",
          "time_received" : "[18/May/2015:20:05:30 +0000]",
          "request_header_referer" : "http://s-chassis.co.nz/viewtopic.php?f=16&t=9265&start=200",
          "request_first_line" : "GET /presentations/logstash-scale11x/images/ahhh___rage_face_by_samusmmx-d5g5zap.png HTTP/1.1",
          "request_url_username" : null,
          "request_url_query_dict" : { },
          "time_received_utc_isoformat" : "2015-05-18T20:05:30+00:00",
          "request_http_ver" : "1.1",
          "time_received_isoformat" : "2015-05-18T20:05:30",
          "request_header_user_agent" : "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko",
          "request_url_hostname" : null,
          "status" : "200",
          "request_url_fragment" : "",
          "remote_logname" : "-",
          "request_url_path" : "/presentations/logstash-scale11x/images/ahhh___rage_face_by_samusmmx-d5g5zap.png",
          "remote_user" : "-",
          "request_url_password" : null
        }
      },

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

但是200不仅可以用于状态代码,还可以用于其他内容,例如网址的一部分或某些内容的值。如何对等内容进行搜索查询给我所有状态代码为200 的文件?

我尝试过这样的事情

curl -H 'Content-Type: application/json' -k -u user:password -XGET https://5a482b9559f*********************.ap-southeast-1.aws.found.io:9243/apache/logs/_search?pretty=true -d '{"query": {"match": {"status": "200"}}}'

但我没有得到任何结果。那么如何根据特定的密钥缩小搜索特定值?

更新:这是我的索引映射

{
  "apache" : {
    "mappings" : {
      "logs" : {
        "properties" : {
          "remote_host" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "remote_logname" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "remote_user" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_first_line" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_header_referer" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_header_user_agent" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_http_ver" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_method" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_url" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_url_fragment" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_url_hostname" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_url_netloc" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_url_path" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_url_query" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_url_query_dict" : {
            "properties" : {
              "C" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "N" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "O" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "_" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "action" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "commentlimit" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "file" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "flav" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "height" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "iframe" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "page" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "source" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "utm_campaign" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "utm_medium" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "utm_source" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "v" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "width" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              }
            }
          },
          "request_url_query_list" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "request_url_query_simple_dict" : {
            "properties" : {
              "C" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "N" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "O" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "_" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "action" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "commentlimit" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "file" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "flav" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "height" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "iframe" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "page" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "source" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "utm_campaign" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "utm_medium" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "utm_source" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "v" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "width" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              }
            }
          },
          "request_url_scheme" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "response_bytes_clf" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "status" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "time_received" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "time_received_isoformat" : {
            "type" : "date"
          },
          "time_received_tz_isoformat" : {
            "type" : "date"
          },
          "time_received_utc_isoformat" : {
            "type" : "date"
          }
        }
      }
    }
  }
}

1 个答案:

答案 0 :(得分:0)

对于那些简单类型的查询,您可以执行以下操作:

{
  "query": {
    "term": {
      "status.keyword": "200"
    }
  }
}

所以,基本上,我们会在这里超级安全,并且要更换匹配查询(正在做一些分析,可能会分解标记,然后形成一个布尔查询)更简单 term 查询,它将按原样匹配。此外,我们会匹配 status.keyword ,这是状态字段的未更改版本

相关问题