Question

我正在使用mysql数据库为asp.net mvc编码的网站构建一个全文搜索工具。本网站适用于非英语语言。我已经开始使用Lucense作为搜索文本的引擎，但我找不到任何关于它是否支持unicode的信息？

有没有人知道Lucene是否支持Unicode？我不想要一个令人讨厌的惊喜......

还将赞赏有关实施lucene.net的初学者文章的链接。

Answer 1

是。它完全支持unicode 但是为了分析，你应该明确指定适当的词干分析器和正确的停用词。至于样品。这是我们上一个项目的副本

directory = new RAMDirectory();
            analyzer = new StandardAnalyzer(version, new Hashtable());
            var indexWriter = new IndexWriter(directory, analyzer, true, IndexWriter.MaxFieldLength.UNLIMITED);
            using (var session = sessionFactory.OpenStatelessSession())
            {
                organizations = session.CreateCriteria(typeof(Organization)).List<Organization>();
                foreach (var organization in organizations)
                {
                    var document = new Document();
                    document.Add(new Field("Id", organization.ID.ToString(), Field.Store.YES, Field.Index.NOT_ANALYZED_NO_NORMS));
                    document.Add(new Field("FullName", organization.FullName, Field.Store.NO, Field.Index.ANALYZED_NO_NORMS));
                    document.Add(new Field("ObjectTypeInvariantName", typeof(Organization).FullName, Field.Store.YES, Field.Index.NOT_ANALYZED_NO_NORMS));
                    indexWriter.AddDocument(document);
                }

                var persistentType = typeof(Order);
                var classMetadata = DbContext.SessionFactory.GetClassMetadata(persistentType);


                var properties = new List<PropertyInfo>();
                for (int i = 0; i < classMetadata.PropertyTypes.Length; i++)
                {
                    var propertyType = classMetadata.PropertyTypes[i];
                    if (propertyType.IsCollectionType || propertyType.IsEntityType) continue;
                    properties.Add(typeof(Order).GetProperty(classMetadata.PropertyNames[i]));
                }

                orders = session.CreateCriteria(typeof(Order)).List<Order>();
                var idProperty = typeof(Order).GetProperty(classMetadata.IdentifierPropertyName);

                foreach (var order in orders)
                {
                    var document = new Document();
                    document.Add(new Field("Id", idProperty.GetValue(order, null).ToString(), Field.Store.YES, Field.Index.NOT_ANALYZED_NO_NORMS));
                    document.Add(new Field("ObjectTypeInvariantName", typeof(Order).FullName, Field.Store.YES, Field.Index.NOT_ANALYZED_NO_NORMS));
                    foreach (var property in properties)
                    {
                        var value = property.GetValue(order, null);
                        if (value != null)
                        {

                            document.Add(new Field(property.Name, value.ToString(), Field.Store.NO, Field.Index.ANALYZED_NO_NORMS));
                        }
                    }
                    indexWriter.AddDocument(document);
                }
                indexWriter.Optimize(true);
                indexWriter.Commit();
                return indexWriter.GetReader();
            }

我正在从NHibernate查询组织对象并将它们放入Lucene.NET

这是简单的搜索

var searchValue = textEdit1.Text;

                var parser = new QueryParser(version, "FullName", analyzer);
                parser.SetLocale(new CultureInfo("ru-RU"));
                Query query = parser.Parse(searchValue);
                var indexSearcher = new IndexSearcher(directory, true);

                var docs = indexSearcher.Search(query, 10);
                lblSearchTotal.Text = string.Format(totalPattern, docs.totalHits, organizations.Count() + orders.Count);
                resultPanel.Controls.Clear();
                foreach (var found in docs.scoreDocs)
                {
                    var document = indexSearcher.Doc(found.doc);
                    var objectId = document.Get("Id");
                    var objectType = document.Get("ObjectTypeInvariantName");

                    if (resultPanel.Controls.Count > 0)
                    {
                        var labelSeparator = CreateSeparatorLabelControl();
                        resultPanel.Controls.Add(labelSeparator);
                    }
                    var labelCard = CreateFoundLabelControl();
                    resultPanel.Controls.Add(labelCard);

                    var organization = organizations.Where(o => o.ID.ToString() == objectId).FirstOrDefault();
                    if (organization != null)
                    {
                        labelCard.Text = string.Format("<b>{0}</b></br>{1}", organization.AccountNumber, organization.FullName);
                        labelCard.Tag = organization;
                        //labels[count].Text = string.Format("<b>{0}</b></br>{1}", organization.AccountNumber, organization.FullName);
                        //labels[count].Visible = true;
                    }
                    else
                    {
                        labelCard.Text = string.Format("Найден объект типа '{0}' с идентификатором '{1}'", objectType, objectId);
                        labelCard.Tag = mainForm.GetObject(objectType, objectId); 
                    }
                    labelCard.Visible = true;
                    //count++;
                }

Answer 2

是的，Lucene支持unicode，因为它以UTF-8格式存储字符串。

http://lucene.apache.org/java/3_0_3/fileformats.html

<强>字符数

Lucene将unicode字符序列写为UTF-8编码字节。

<强>字符串

Lucene将字符串写为UTF-8编码字节。首先，以字节为单位的长度写为VInt，后跟字节。

字符串 - ＆gt;查特，查尔斯

Answer 3

Lucene确实支持unicode，但也存在局限性。例如，某些文档阅读器不支持unicode。此外，lucene还会执行复数或非复数单词等操作。当你使用外语时，其中一些会消失。

Lucene是否支持Unicode？

3 个答案: