所以，我正在考虑在我的公司实现模糊逻辑匹配，并且无法获得好的结果。对于初学者，我试图将公司名称与其他公司提供的名单上的名称相匹配。
我的第一次尝试是使用soundex，但看起来soundex只比较公司名称中的前几个声音，因此较长的公司名称太容易相互混淆。
我现在正在使用levenstein距离比较进行第二次尝试。它看起来很有希望，特别是如果我先删除标点符号。但是，我仍然无法在没有太多误报的情况下找到重复项
我遇到的一个问题是像widgetsco vs widgets inc这样的公司。所以，如果我比较较短名称长度的子串，我也会收到像BBC大学和CBC大学校园这样的东西。我怀疑使用距离和最长公共子串的组合得分可能是解决方案
有没有人设法建立一个与有限误报匹配的算法？
答案 0 :(得分：0)
你想使用Levenshtein Distance或其他字符串比较算法。您可能想看看Codeplex上的这个项目。
答案 1 :(得分：0)
您使用的是Access吗？如果是这样，请考虑＆＃39; *＆＃39;字符，没有引号。如果您正在使用SQL Server，请使用＆＃39;％＆＃39;字符。然而，这真的不是模糊逻辑，它实际上是Like运算符。如果您确实需要模糊逻辑，请将数据集导出到Excel并从下面的URL加载AddIn。
https://www.microsoft.com/en-us/download/details.aspx?id=15011
仔细阅读说明。它绝对有效，而且效果很好，但你需要按照说明进行操作，而且它并不完全直观。我第一次尝试时，我没有按照说明操作，而且我浪费了很多时间试图让它工作。最终我发现了它，并且效果很好!!
答案 2 :(得分：0)
使用Lawrence Philips创建的Metaphone功能，我们在名称和地址匹配方面取得了不错的成绩。它的工作方式与Soundex类似，但为整个值创建声音/辅音模式。您可能会发现这与其他一些技术一起使用很有用，特别是如果您可以去掉一些像'co'这样的绒毛。和'inc。'如其他评论中所述：
create function [dbo].[Metaphone](@str as nvarchar(70), @KeepNumeric as bit = 0)
returns nvarchar(25)
/*
Metaphone Algorithm
Created by Lawrence Philips.
Metaphone presented in article in "Computer Language" December 1990 issue.
*********** BEGIN METAPHONE RULES ***********
Lawrence Philips' RULES follow:
The 16 consonant sounds:
|--- ZERO represents "th"
|
B X S K J T F H L M N P R 0 W Y
Drop vowels
Exceptions:
Beginning of word: "ae-", "gn", "kn-", "pn-", "wr-" ----> drop first letter
Beginning of word: "wh-" ----> change to "w"
Beginning of word: "x" ----> change to "s"
Beginning of word: vowel or "H" + vowel ----> Keep it
Transformations:
B ----> B unless at the end of word after "m", as in "dumb", "McComb"
C ----> X (sh) if "-cia-" or "-ch-"
S if "-ci-", "-ce-", or "-cy-"
SILENT if "-sci-", "-sce-", or "-scy-"
K otherwise
K "-sch-"
D ----> J if in "-dge-", "-dgy-", or "-dgi-"
T otherwise
F ----> F
G ----> SILENT if "-gh-" and not at end or before a vowel
"-gn" or "-gned"
"-dge-" etc., as in above rule
J if "gi", "ge", "gy" if not double "gg"
K otherwise
H ----> SILENT if after vowel and no vowel follows
or "-ch-", "-sh-", "-ph-", "-th-", "-gh-"
H otherwise
J ----> J
K ----> SILENT if after "c"
K otherwise
L ----> L
M ----> M
N ----> N
P ----> F if before "h"
P otherwise
Q ----> K
R ----> R
S ----> X (sh) if "sh" or "-sio-" or "-sia-"
S otherwise
T ----> X (sh) if "-tia-" or "-tio-"
0 (th) if "th"
SILENT if "-tch-"
T otherwise
V ----> F
W ----> SILENT if not followed by a vowel
W if followed by a vowel
X ----> KS
Y ----> SILENT if not followed by a vowel
Y if followed by a vowel
Z ----> S
*/
as
begin
declare @Result varchar(25)
,@str3 char(3)
,@str2 char(2)
,@str1 char(1)
,@strp char(1)
,@strLen tinyint
,@cnt tinyint
set @strLen = len(@str)
set @cnt = 0
set @Result = ''
-- Preserve first 5 numeric values when required
if @KeepNumeric = 1
begin
set @Result = case when isnumeric(substring(@str,1,1)) = 1
then case when isnumeric(substring(@str,2,1)) = 1
then case when isnumeric(substring(@str,3,1)) = 1
then case when isnumeric(substring(@str,4,1)) = 1
then case when isnumeric(substring(@str,5,1)) = 1
then left(@str,5)
else left(@str,4)
end
else left(@str,3)
end
else left(@str,2)
end
else left(@str,1)
end
else ''
end
set @str = right(@str,len(@str)-len(@Result))
end
--Process beginning exceptions
set @str2 = left(@str,2)
if @str2 = 'wh'
begin
set @str = 'w' + right(@str , @strLen - 2)
set @strLen = @strLen - 1
end
else
if @str2 in('ae', 'gn', 'kn', 'pn', 'wr')
begin
set @str = right(@str , @strLen - 1)
set @strLen = @strLen - 1
end
set @str1 = left(@str,1)
if @str1 = 'x'
set @str = 's' + right(@str , @strLen - 1)
else
if @str1 in ('a','e','i','o','u')
begin
set @str = right(@str, @strLen - 1)
set @strLen = @strLen - 1
set @Result = @Result + @str1
end
while @cnt <= @strLen
begin
set @cnt = @cnt + 1
set @str1 = substring(@str,@cnt,1)
set @strp = case when @cnt <> 0
then substring(@str,(@cnt-1),1)
else ' '
end
-- Check if the current character is the same as the previous character.
-- If we are keeping numbers, only compare non-numeric characters.
if case when @KeepNumeric = 1 and @strp = @str1 and isnumeric(@str1) = 0 then 1
when @KeepNumeric = 0 and @strp = @str1 then 1
else 0
end = 1
continue -- Skip this loop
set @str2 = substring(@str,@cnt,2)
set @Result = case when @KeepNumeric = 1 and isnumeric(@str1) = 1
then @Result + @str1
when @str1 in('f','j','l','m','n','r')
then @Result + @str1
when @str1 = 'q'
then @Result + 'k'
when @str1 = 'v'
then @Result + 'f'
when @str1 = 'x'
then @Result + 'ks'
when @str1 = 'z'
then @Result + 's'
when @str1 = 'b'
then case when @cnt = @strLen
then case when substring(@str,(@cnt - 1),1) <> 'm'
then @Result + 'b'
else @Result
end
else @Result + 'b'
end
when @str1 = 'c'
then case when @str2 = 'ch' or substring(@str,@cnt,3) = 'cia'
then @Result + 'x'
else case when @str2 in('ci','ce','cy') and @strp <> 's'
then @Result + 's'
else @Result + 'k'
end
end
when @str1 = 'd'
then case when substring(@str,@cnt,3) in ('dge','dgy','dgi')
then @Result + 'j'
else @Result + 't'
end
when @str1 = 'g'
then case when substring(@str,(@cnt - 1),3) not in ('dge','dgy','dgi','dha','dhe','dhi','dho','dhu')
then case when @str2 in('gi', 'ge','gy')
then @Result + 'j'
else case when @str2 <> 'gn' or (@str2 <> 'gh' and @cnt+1 <> @strLen)
then @Result + 'k'
else @Result
end
end
else @Result
end
when @str1 = 'h'
then case when @strp not in ('a','e','i','o','u') and @str2 not in ('ha','he','hi','ho','hu')
then case when @strp not in ('c','s','p','t','g')
then @Result + 'h'
else @Result
end
else @Result
end
when @str1 = 'k'
then case when @strp <> 'c'
then @Result + 'k'
else @Result
end
when @str1 = 'p'
then case when @str2 = 'ph'
then @Result + 'f'
else @Result + 'p'
end
when @str1 = 's'
then case when substring(@str,@cnt,3) in ('sia','sio') or @str2 = 'sh'
then @Result + 'x'
else @Result + 's'
end
when @str1 = 't'
then case when substring(@str,@cnt,3) in ('tia','tio')
then @Result + 'x'
else case when @str2 = 'th'
then @Result + '0'
else case when substring(@str,@cnt,3) <> 'tch'
then @Result + 't'
else @Result
end
end
end
when @str1 = 'w'
then case when @str2 not in('wa','we','wi','wo','wu')
then @Result + 'w'
else @Result
end
when @str1 = 'y'
then case when @str2 not in('ya','ye','yi','yo','yu')
then @Result + 'y'
else @Result
end
else @Result
end
end
return @Result
end
答案 3 :(得分：0)
我成功实现了我在Stack Overflow上找到的可以找到匹配字符串百分比的功能。然后，您可以调整容差，直到获得适当数量的匹配/不匹配。该函数的实现将在下面列出，但要点是在查询中包含类似的内容。
DECLARE @tolerance DEC(18, 2) = 50;
WHERE dbo.GetPercentageOfTwoStringMatching(first_table.name, second_table.name) > @tolerance
以下百分比匹配功能的贷方为 Dragos Durlut，11月15日。 在代码中， Dragos Durlut 中包含了LEVENSHTEIN函数的功劳。
T-SQL Get percentage of character match of 2 strings
CREATE FUNCTION [dbo].[GetPercentageOfTwoStringMatching]
(
@string1 NVARCHAR(100)
,@string2 NVARCHAR(100)
)
RETURNS INT
AS
BEGIN
DECLARE @levenShteinNumber INT
DECLARE @string1Length INT = LEN(@string1)
, @string2Length INT = LEN(@string2)
DECLARE @maxLengthNumber INT = CASE WHEN @string1Length > @string2Length THEN @string1Length ELSE @string2Length END
SELECT @levenShteinNumber = [dbo].[LEVENSHTEIN] ( @string1 ,@string2)
DECLARE @percentageOfBadCharacters INT = @levenShteinNumber * 100 / @maxLengthNumber
DECLARE @percentageOfGoodCharacters INT = 100 - @percentageOfBadCharacters
-- Return the result of the function
RETURN @percentageOfGoodCharacters
END
-- =============================================
-- Create date: 2011.12.14
-- Description: http://blog.sendreallybigfiles.com/2009/06/improved-t-sql-levenshtein-distance.html
-- =============================================
CREATE FUNCTION [dbo].[LEVENSHTEIN](@left VARCHAR(100),
@right VARCHAR(100))
returns INT
AS
BEGIN
DECLARE @difference INT,
@lenRight INT,
@lenLeft INT,
@leftIndex INT,
@rightIndex INT,
@left_char CHAR(1),
@right_char CHAR(1),
@compareLength INT
SET @lenLeft = LEN(@left)
SET @lenRight = LEN(@right)
SET @difference = 0
IF @lenLeft = 0
BEGIN
SET @difference = @lenRight
GOTO done
END
IF @lenRight = 0
BEGIN
SET @difference = @lenLeft
GOTO done
END
GOTO comparison
COMPARISON:
IF ( @lenLeft >= @lenRight )
SET @compareLength = @lenLeft
ELSE
SET @compareLength = @lenRight
SET @rightIndex = 1
SET @leftIndex = 1
WHILE @leftIndex <= @compareLength
BEGIN
SET @left_char = substring(@left, @leftIndex, 1)
SET @right_char = substring(@right, @rightIndex, 1)
IF @left_char <> @right_char
BEGIN -- Would an insertion make them re-align?
IF( @left_char = substring(@right, @rightIndex + 1, 1) )
SET @rightIndex = @rightIndex + 1
-- Would an deletion make them re-align?
ELSE IF( substring(@left, @leftIndex + 1, 1) = @right_char )
SET @leftIndex = @leftIndex + 1
SET @difference = @difference + 1
END
SET @leftIndex = @leftIndex + 1
SET @rightIndex = @rightIndex + 1
END
GOTO done
DONE:
RETURN @difference
END
注意：如果您需要比较两个或多个字段（我认为您不这样做），则可以在WHERE子句中以最小容限向函数添加另一个调用。我还发现平均百分率匹配并将其与容差进行比较的成功案例。
DECLARE @tolerance DEC(18, 2) = 25;
--could have multiple different tolerances for each field (weighting some fields as more important to be matching)
DECLARE @avg_tolerance DEC(18, 2) = 50;
WHERE AND dbo.GetPercentageOfTwoStringMatching(first_table.name, second_table.name) > @tolerance
AND dbo.GetPercentageOfTwoStringMatching(first_table.address, second_table.address) > @tolerance
AND (dbo.GetPercentageOfTwoStringMatching(first_table.name, second_table.name)
+ dbo.GetPercentageOfTwoStringMatching(first_table.address, second_table.address)
) / 2 > @avg_tolerance
此解决方案的好处是容差变量可以在每个字段中特定（权衡某些字段匹配的重要性），而平均值可以确保所有字段之间的总体匹配。
答案 4 :(得分：0)
首先，我建议你确保你不能匹配任何其他属性和公司名称（因为模糊匹配肯定会给你一些误报）。如果您想继续进行模糊匹配，您可以使用以下步骤：
从文本中删除所有停用词。例如：Co, Inc 等
如果您的数据库非常大，请使用索引方法，例如分块索引或排序邻域索引。
最后使用 Levenshtein 距离计算模糊分数。您可以在 Fuzzywuzzy 中使用 token_set_ratio 或 partial_ratio 函数。
另外，我发现以下视频旨在解决同样的问题：https://www.youtube.com/watch?v=NRAqIjXaZvw
Nanonets 博客还包含有关该主题的一些可能有用的资源。