解析这些500k行的最有效方法是什么?

时间:2015-03-27 14:05:32

标签: c# sql-server tsql sqlclr

我目前有一个充满ACL条目的数据库,如下所示:

ACL database

我需要仔细检查根节点(如\\ chrlcltsvr02 \ AYY_LMO \ ClientServices)与其子节点(例如\\ chrlcltsvr02 \ AYY_LMO \ ClientServices \ Client1)之间的区别。< / p>

我尝试使用ORM和原始T-SQL在C#代码中这样做(事实上我知道每行打开一个会话是一个可怕的想法):

foreach (string path in distinctPaths)
{
    using (session = sessionFactory.OpenSession())
    {
        string query;

        query = String.Format("SELECT DISTINCT UserGroup, AllowDeny, Permissions FROM FilerACLs WHERE FullPath LIKE '{0}'", path.Replace("'", "''"));

        var parentACLs = session.CreateSQLQuery(query).SetResultTransformer(Transformers.AliasToBean<ShareACLEntry>()).List<ShareACLEntry>();

        query = String.Format("SELECT DISTINCT UserGroup, AllowDeny, Permissions FROM FilerACLs WHERE FullPath LIKE '{0}\\%'", path.Replace("'", "''"));

        var childACLs = session.CreateSQLQuery(query).SetResultTransformer(Transformers.AliasToBean<ShareACLEntry>()).List<ShareACLEntry>();

        if (childACLs.Except(parentACLs, comparer).ToList().Count > 0)
            Console.WriteLine("{0} has diffs!", path);
    }
}

最后比较结果数据以查看子节点是否与根节点不同。

根据不同,我的意思是如果我有一个用于组的ACL&#34; CLT-AD \ Full Access Shared-CHRL&#34;如果允许完全控制父节点而不是子节点,我要注意ACL存在于子节点而不存在于父节点上。

不幸的是,这个过程太慢,无法在任何适当的时间内解析500k行。

我想知道是否有人有想法有效地确定数据是否存在差异 - 无论是直接使用T-SQL,SQL CLR功能还是更好的算法。

如果需要澄清,请告诉我。

谢谢!

修改

由于我在这个问题上得到了相当多的仇恨,让我重新澄清我正在寻找的内容,减去我上面概述的失败方法。

我最近对Windows服务器上的~1,000个共享文件夹执行了扫描。此扫描从顶级目录一直向下递归到文件夹层次结构,并为每个文件夹记录每个ACL的行。

因此,数据库看起来就像上面的截图。

我需要做的是从此数据库中提取报告,详细说明从顶级目录记录的ACL与为此顶级目录下的任何目录记录的ACL之间的差异(甚至是否存在差异)。 / p>

希望这更有意义。

2 个答案:

答案 0 :(得分:2)

这是一些TSQL,

DECLARE @parentFullPath NVARCHAR(260) = N'\\chrlcltsvr02\AYY_LMO\ClientServices';
DECLARE @childFullPath NVARCHAR(260) = N'\\chrlcltsvr02\AYY_LMO\ClientServices\Client1';

SELECT
            [UserGroup],
            [AllowDeny],
            [Permissions]
    FROM
            [ACLs]
    WHERE
            [FullPath] = @childFullPath
EXCEPT
SELECT
            [UserGroup],
            [AllowDeny],
            [Permissions]
    FROM
            [ACLs]
    WHERE
            [FullPath] = @parentFullPath;

它可能会或可能不会满足您的要求,很难说清楚。


要查找所有父子对,

WITH [paths] AS (
SELECT
             [FullPath]
    FROM
             [ACLs]
    GROUP BY
             [FullPath])
SELECT
            [P].[FullPath] [ParentFullPath],
            [C].[FullPath] [ChildFullPath]
    FROM
            [paths] [P]
        JOIN
            [paths] [C]
                ON
                        [C].[FullPath] <> [P].[FullPath]
                    AND
                        CHARINDEX([P].[FullPath], [C].[FullPath]) = 1;

所以你实际上可以同时做到这一切,就像这样。

WITH [paths] AS (
SELECT
             [FullPath]
    FROM
             [ACLs]
    GROUP BY
             [FullPath])
SELECT
            [PC].[ParentFullPath],
            [PC].[ChildFullPath],
            [D].[UserGroup],
            [D].[AllowDeny],
            [D].[Permissions]
    FROM (
            SELECT
                        [P].[FullPath] [ParentFullPath],
                        [C].[FullPath] [ChildFullPath]
                FROM
                        [paths] [P]
                    JOIN
                        [paths] [C]
                            ON
                                    [C].[FullPath] <> [P].[FullPath]
                                AND
                                    CHARINDEX([P].[FullPath], [C].[FullPath]) = 1;
        ) [PC]
CROSS APPLY
(
SELECT
            [UserGroup],
            [AllowDeny],
            [Permissions]
    FROM
            [ACLs]
    WHERE
            [FullPath] = [PC].[ChildFullPath]
EXCEPT
SELECT
            [UserGroup],
            [AllowDeny],
            [Permissions]
    FROM
            [ACLs]
    WHERE
            [FullPath] = [PC].[ParentFullPath]
) [D];

最终,如果您希望此代码有效运行,您需要稍微规范化您的架构。只要父子关系仅通过字符串比较推断存在,这将是一个相对较慢的操作。

答案 1 :(得分:1)

如果你想一次性完成整个列表,你可以编写一个SQL表达式(例如使用substring())到get_parent_path_from_child_path并运行以下SQL结构。从一般情况来看,如何将父母与子女分开是不明确的。所以我只是给你一个线框代码。

(
SELECT -- parents
            [UserGroup],
            [AllowDeny],
            [Permissions],
            [FullPath] as parent_path
    FROM
            [ACLs]
  WHERE add a filter for parents here

minus 
SELECT -- children
            [UserGroup],
            [AllowDeny],
            [Permissions],
            get_parent_path_from_child_path([FullPath]) as parent_path
    FROM
            [ACLs]
    WHERE add a filter for children here

)
union 
(
SELECT -- children
            [UserGroup],
            [AllowDeny],
            [Permissions],
            get_parent_path_from_child_path([FullPath]) as parent_path
    FROM
            [ACLs]
    WHERE add a filter for children here
minus
SELECT -- parents
            [UserGroup],
            [AllowDeny],
            [Permissions],
            [FullPath] as parent_path
    FROM
            [ACLs]
  WHERE add a filter for parents here

)