Enumerable.Any()在大型Linq数据集上非常慢

时间:2017-05-05 19:18:21

标签: c# .net entity-framework linq entity-framework-core

我正在运行这样的代码:

var somethings = db.Somethings.Select(s => new SomethingViewModel
{
    Id = s.Id,
    Name = s.Name,
    IsActive = s.IsActive
    SubSomethings = s.SubSomethings.Select(ss => new SubSomethingViewModel
    {
        Id = ss.Id,
        Name = ss.Name,
        IsActive = ss.IsActive
    }).Where(wss => wss.IsActive)                        
}).Where(ws => ws.IsActive && (ws.SubSomethings.Any())) //remove elements if no SubSomethings
.ToList();

如您所见,这是一对多的关系。 Something中有一个SubSomethings列表。如果我拿出&& (ws.SubSomethings.Any()),我得到了一个非常快速的列表。

但是,我想只在列表中包含至少有一个SubSomething的Somethings。我也尝试过以下方法并获得同样可怕的效率:

var somethings = db.Somethings.Select(s => new SomethingViewModel
{
    Id = s.Id,
    Name = s.Name,
    IsActive = s.IsActive
    SubSomethings = s.SubSomethings.Select(ss => new SubSomethingViewModel
    {
        Id = ss.Id,
        Name = ss.Name,
        IsActive = ss.IsActive
    }).Where(wss => wss.IsActive)                        
}).Where(ws => ws.IsActive)
.ToList(); //this finishes very quickly

var somethings2 = somethings.Where(s => s.SubSomethings.Any()).ToList(); //This is where the code bogged down

如何重新编写查询以使代码变得更快?有一点需要注意:这适用于一两条记录。当我点击> 8000条记录时,至少需要四分钟。

这是我在SubSomething表上为SomethingId的外键创建的索引,它对应于Something.Id

CREATE NONCLUSTERED INDEX [IX_SubSomething_SomethingId] ON [dbo].[SubSomething]
(
    [SomethingId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
GO

这是SubSomething.SomethingId:

的外键创建
ALTER TABLE [dbo].[SubSomething]  WITH CHECK ADD  CONSTRAINT [FK_SubSomething_Something_SomethingId] FOREIGN KEY([SomethingId])
REFERENCES [dbo].[Something] ([Id])
GO

ALTER TABLE [dbo].[SubSomething] CHECK CONSTRAINT [FK_SubSomething_Something_SomethingId]
GO

3 个答案:

答案 0 :(得分:6)

EF Core是你的问题。目前,当查询包含子集合投影时,已知执行N + 1子查询。

解决它并将整个事情限制为2个SQL查询的唯一方法是尽可能多地过滤,然后使用预先加载在内存中加载包含子实体的整个实体集,然后切换到LINQ to Objects并执行最后的投影/过滤:

var somethings = db.Somethings
    .Include(s => s.SubSomethings)
    .Where(s => s.IsActive)
    .AsEnumerable()
    .Select(s => new SomethingViewModel
    {
        Id = s.Id,
        Name = s.Name,
        IsActive = s.IsActive,
        SubSomethings = s.SubSomethings.Select(ss => new SubSomethingViewModel
        {
            Id = ss.Id,
            Name = ss.Name,
            IsActive = ss.IsActive
        }).Where(wss => wss.IsActive).ToList()
    })
    .Where(s => s.SubSomethings.Any())
    .ToList();

答案 1 :(得分:0)

我认为问题在于您将第一个查询转换为列表。试试这个:

var products = db.Products.Where(ws => ws.IsActive && ws.SubSomethings.Count(ss => ss.IsActive) > 0)
.Select(p => new SomethingViewModel
{
    Id = p.Id,
    Name = p.Name,
    IsActive = p.IsActive,
    SubSomethings = p.SubSomethings.Select(ss => new PartViewModel
    {
        Id = ss.Id,
        Name = ss.Name,
        IsActive = ss.IsActive
    }).Where(wss => wss.IsActive)   
}).ToList();

答案 2 :(得分:0)

正如Ivan Stoev所说,EF Core是你的问题,我分享了一种不使用LINQ to SQL的替代方案,但这种方式的性能更差:

  1. 使用加入所需数据的逻辑在数据库中创建一个视图
  2. 使用db.Database.SqlQuery(" Query")查询视图并返回数据:

    var results = db.Database.SqlQuery< SomethingViewModel>("从ViewSomething&#34中选择ColA,ColB,ColC;)

    对象SomethingViewModel必须是强类型,其属性与视图的cols匹配。

  3. 我知道这不是很漂亮,并且对视图进行了硬编码查询,但通常是您通过编译器后使用EF获得的最快性能。