版本升级后,ElasticSearch / Nest MatchPhrasePrefix停止工作

时间:2019-03-07 21:45:40

标签: c# elasticsearch nest

我从:

升级
  • ElasticSearch 2.0到6.6.1
  • ElasticSearch.Net Nuget软件包2.4.6至6.5.1
  • NEST NuGet软件包2.4.6到6.5.1

...并且我的执行MatchPhrasePrefix的Nest查询停止返回结果。

该软件是网页的搜索引擎,其功能之一是可以将结果限制为以特定路径开头的URL,例如http://example.com/blog,以仅查看搜索结果中的博客文章

我有一个运行正常的mainQuery。如果用户提供一个urlStartstWith值,则mainQuery会与bool / MatchPhrasePrefix查询一起被加和。

索引包含从100到1000的文档。

我尝试过的不起作用:

  • 完全重建新索引
  • 已删除.Operator(Operator.And),因为此版本的NEST中不存在urlStartstWith
  • 将“ MaxExpansions”最多更新到5000个值
  • URL编码.MinimumShouldMatch(1)
  • 删除var urlStartWithFilter = esQuery.Bool(b => b.Filter(m => m.MatchPhrasePrefix(pre => pre //.MaxExpansions(5000) //did nothing //.Operator(Operator.And) //does not exist in new version of NEST .Query(urlStartsWith) .Field(f => f.Url)) ) .MinimumShouldMatch(1) ); mainQuery = mainQuery && urlStartWithFilter;

如果我针对旧的ElasticSearch 2.0版服务器运行使用新的NEST库构建的查询,则该查询有效。因此,我认为,ElasticSearch本身已经发生了一些变化。

查询

docker pull docker.elastic.co/elasticsearch/elasticsearch:6.6.1
docker network create esnetwork --driver=bridge
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --name elasticsearch -d --network esnetwork docker.elastic.co/elasticsearch/elasticsearch:6.6.1

根据要求-开始完成示例

这是一个显示问题的示例,与我查询实际项目的网页索引非常接近。

运行一个ElasticSearch 6.6.1实例。您可以在docker中通过以下方式进行操作:

Program.cs

创建一个新的.Net Framework 4.6.1控制台应用程序。 将以下内容粘贴到using Nest; using System; using System.Collections.Generic; namespace Loader { class Program { const string ELASTIC_SERVER = "http://localhost:9200"; const string DEFAULT_INDEX = "stack_overflow_api"; private static Uri es_node = new Uri(ELASTIC_SERVER); private static ConnectionSettings settings = new ConnectionSettings(es_node).DefaultIndex(DEFAULT_INDEX); private static ElasticClient client = new ElasticClient(settings); private static bool include_starts_with = true; static void Main(string[] args) { WriteMainMenu(); } static void WriteMainMenu() { //Console.Clear(); Console.WriteLine(""); Console.WriteLine(""); Console.WriteLine("What to do?"); Console.WriteLine("1 - Load Sample Data into ES"); Console.WriteLine("2 - Run a query WITHOUT StartsWith"); Console.WriteLine("3 - Run a query WITH StartsWith"); Console.WriteLine("[Enter] to exit."); Console.WriteLine(""); Console.WriteLine(""); var option = Console.ReadLine(); if (option == "1") { LoadSampleData(); } else if (option == "2") { include_starts_with = false; RunStartsWithQuery(); } else if (option == "3") { include_starts_with = true; RunStartsWithQuery(); } //- exit } private static void LoadSampleData() { var existsResponse = client.IndexExists(DEFAULT_INDEX); if (existsResponse.Exists) //delete existing mapping (and data) { client.DeleteIndex(DEFAULT_INDEX); } var rebuildResponse = client.CreateIndex(DEFAULT_INDEX, c => c.Settings(s => s.NumberOfReplicas(1).NumberOfShards(5))); var response2 = client.Map<Item>(m => m.AutoMap()); var data = GetSearchResultData(); Console.WriteLine($"Indexing {data.Count} items..."); var response = client.IndexMany<Item>(data); client.Refresh(DEFAULT_INDEX); WriteMainMenu(); } private static List<Item> GetSearchResultData() { var jsonPath = System.IO.Path.Combine(Environment.CurrentDirectory, "StackOverflowSampleJson.json"); var jsondata = System.IO.File.ReadAllText(jsonPath); var searchResult = Newtonsoft.Json.JsonConvert.DeserializeObject<List<Item>>(jsondata); return searchResult; } private static void RunStartsWithQuery() { Console.WriteLine("Enter a search query and press enter, or just press enter to search for the default of 'Perl'."); var search = Console.ReadLine().ToLower(); if (string.IsNullOrWhiteSpace(search)) { search = "Perl"; } Console.WriteLine($"Searching for {search}..."); var result = client.Search<Item>(s => s .Query(esQuery => { var titleQuery = esQuery.Match(m => m .Field(p => p.title) .Boost(1) .Query(search) ); var closedReasonQuery = esQuery.Match(m => m .Field(p => p.closed_reason) .Boost(1) .Query(search) ); // search across a couple fields var mainQuery = titleQuery || closedReasonQuery; if (include_starts_with) { var urlStartsWith = "https://stackoverflow.com/questions/"; var urlStartWithFilter = esQuery.Bool(b => b.Filter(m => m.MatchPhrasePrefix(pre => pre //.MaxExpansions(5000) //did nothing //.Operator(Operator.And) //does not exist in new version of NEST .Query(urlStartsWith) .Field(f => f.link)) ) .MinimumShouldMatch(1) ); mainQuery = mainQuery && urlStartWithFilter; } return mainQuery; }) ); if (result.IsValid == false) { Console.WriteLine("ES Query had an error"); } else if (result.Hits.Count > 0) { Console.ForegroundColor = ConsoleColor.DarkGreen; Console.WriteLine($"Found {result.Hits.Count} results:"); foreach (var item in result.Hits) { Console.WriteLine($" {item.Source.title}"); } Console.ForegroundColor = ConsoleColor.White; } else { Console.ForegroundColor = ConsoleColor.DarkRed; Console.WriteLine($"Found 0 results"); Console.ForegroundColor = ConsoleColor.White; } WriteMainMenu(); } } public class Item { public List<string> tags { get; set; } //public Owner owner { get; set; } public bool is_answered { get; set; } public int view_count { get; set; } public int answer_count { get; set; } public int score { get; set; } public int last_activity_date { get; set; } public int creation_date { get; set; } public int last_edit_date { get; set; } public int question_id { get; set; } public string link { get; set; } public string title { get; set; } public int? accepted_answer_id { get; set; } public int? closed_date { get; set; } public string closed_reason { get; set; } public int? community_owned_date { get; set; } } }

StackOverflowSampleJson.json
  • 创建一个名为StackOverflowSampleJson.json的新文件,并粘贴以下示例JSON的内容:https://pastebin.com/s5rcHysp
  • 通过右键单击Copy to Output Directory并选择属性,然后将Always更改为1 - Load Sample Data into ES来设置2 - Run a query WITHOUT StartsWith
  • 运行该应用。
  • 选择StartsWith填充索引
  • 选择MatchPhrasePrefix以在没有3 - Run a query WITH StartsWith / import itertools stuff = [1, 2, 3] for idx in range(len(stuff)+1): for i in itertools.permutations('ABCD', idx): print i Sample output: () ('A',) ('B',) ('C',) ('D',) ('A', 'B') ('A', 'C') ('A', 'D') ('B', 'A') . . . . ('D', 'C', 'A') ('D', 'C', 'B') 的情况下运行查询,以查看正常查询是否有效
  • 选择originalObjectList = A list of many custom objects. allPossibleOrderings=[] for idx in range(len(originalObjectList)+1): originalObjectListOrderings = list(itertools.permutations(originalObjectList, idx)) for objects in originalObjectListOrderings: allPossibleOrderings.insert(0, map(copy.copy,objects)) ,看看是否包含该额外查询使结果计数为零。

0 个答案:

没有答案