C#,LINQ批处理项目 - 最好的方法是什么?

时间:2010-04-20 19:46:39

标签: c# linq-to-sql savechanges

我有2个数据库 - (1)供稿,(2)生产。 feed数据库是从我们每天获得的客户端文件中提供的,从我的观点来看,它是一个只读数据源。收到Feed文件后,Feed应用程序会执行其操作,然后最终调用生产站点上的Web服务。然后,此Web服务在提要DB和Prod DB之间进行同步。实质上这是伪代码:

  • 获取客户的所有Feed项目
  • 获取同一客户的所有产品
  • 使用LINQ for Objects,获取(1)导致UPDATE的所有项目,(2)导致DELETE的所有项目和(3)导致INSERT的所有项目。
  • 流程更新
  • 处理DELETES
  • Process INSERTS

对于分隔INSERTS,UPDATES和DELETES的核心代码:

List<model.AutoWithImage> feedProductList = _dbFeed.AutoWithImage.Where(feedProduct => feedProduct.ClientID == ClientID).ToList();
List<model.vwCompanyDetails> companyDetailList = _dbRiv.vwCompanyDetails.Where(feedProduct => feedProduct.ClientID == ClientID).ToList();
foreach (model.vwCompanyDetails companyDetail in companyDetailList)
{
    List<model.Product> rivProductList = _dbRiv.Product.Include("Company").Where(feedProduct => feedProduct.Company.CompanyId == companyDetail.CompanyId).ToList();

    foreach (model.AutoWithImage feedProduct in feedProductList)
    {
        bool alreadyExists = false;
        model.Company company = null;
        foreach (model.Product rivProduct in rivProductList)
        {
            if (feedProduct.StockNumber == rivProduct.SKU)
            {
                alreadyExists = true;

                // Active feed items...
                if (feedProduct.Active)
                {
                    // Changed since last sync...
                    if (feedProduct.Updated > rivProduct.LastFeedUpdate)
                    {
                        model.Product updateProduct = new model.Product();
                        updateProduct.ProductId = rivProduct.ProductId;
                        // removed for brevity
                        updateProductList.Add(updateProduct);
                    }
                    // Not changed since last sync...
                    else if (feedProduct.Updated <= rivProduct.LastFeedUpdate)
                    {
                        //nop
                    }
                }
                // No longer active feed products...
                else if (!feedProduct.Active)
                {
                    model.Product deleteProduct = new model.Product();
                    deleteProduct = rivProduct;
                    // removed for brevity
                    deleteProductList.Add(deleteProduct);
                }
            }

            if (company == null)
                company = rivProduct.Company;
        }

        // Found feedProduct new product...
        if (!alreadyExists)
        {
            model.Product insertProduct = new Product();
            insertProduct.ProductId = Guid.NewGuid();
            // removed for brevity
            insertProductList.Add(insertProduct);
        }
    }
}

是的,我知道有更有效的方法可以做到这一点,我开始使用它们。但是,上面的代码相对较快,并且将我的数据分成3个List&lt;&gt;集。

我的问题更多是关于处理_dbRiv.SaveChanges()方法。当我发出它时,出现以触发所有3组(上图)。我试图追踪一个唯一的密钥违规,并在批处理中我没有找到违反约束的一两个记录。我确信在思考LINQ for SQL的实际运作方式时,我遗漏了某些东西。

我想做的是:

  • 仅在更新时执行保存。 做一些其他的事情然后
  • 仅在DELETES上执行保存。 做一些其他的事情然后
  • 逐个执行保存(暂时) 关于INSERTS。

有没有办法一次在一个批次上发出SaveChanges? 有没有办法预先插入InsertProductList对象并一次执行一行SaveChanges? 我吠叫错了树吗?


修改 虽然我知道我可以从EF调用存储过程,但我的目的是学习如何将存储过程转换为EF。

我在SQL中编写了我想要的东西,而且它(这正是我们需要它的方式):

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

ALTER PROC [dbo].[ExecuteSync] @ClientID AS BIGINT AS
BEGIN
    DECLARE @cid UNIQUEIDENTIFIER

    DECLARE c1 CURSOR FOR
    SELECT CompanyID FROM CompanyDetails WHERE ClientID = @ClientID
    OPEN c1
    FETCH NEXT FROM c1 INTO @cid
    WHILE @@FETCH_STATUS = 0
    BEGIN
        SET NOCOUNT ON
        SELECT 'Syncing feed data for ' + CompanyName FROM Company WHERE CompanyId = @cid
        SET NOCOUNT OFF
        -- n/a --------------------------------------------------------------------------------------------------------------------------------------------------------------------
        --SELECT a.*
        --     , p.*
        --  FROM RIVFeeds..AutoWithImage a
        -- INNER JOIN Product p ON a.StockNumber = p.SKU
        -- WHERE ClientID = @ClientID
        --   AND a.Active = 1
        --   AND a.Updated <= p.LastFeedUpdate

        -- Needs UPDATE -----------------------------------------------------------------------------------------------------------------------------------------------------------
        PRINT '--[ UPDATE ]--'
        UPDATE Product
           SET [Description] = ''
             , [Image] = a.ImageURL
             , isDeleted = a.Active ^ 1
             , isFromFeed = 1
             , LastFeedUpdate = a.Updated
             , LowestPrice = a.GuaranteedSalePrice
             , RetailPrice = a.ListPrice
             , [Title] = ''
             , Updated = GETUTCDATE()
             , UpdatedBy = 'Feed Sync Process'
          FROM RIVFeeds..AutoWithImage a
         INNER JOIN Product p ON a.StockNumber = p.SKU AND a.AutoID = p.alternateProductID
         WHERE ClientID = @ClientID
           AND p.CompanyID = @cid
           AND a.Updated > p.LastFeedUpdate

        -- Needs BACKUP -----------------------------------------------------------------------------------------------------------------------------------------------------------
        PRINT '--[ BACKUP #1 ]--'
        INSERT INTO ProductDeleted(ProductId, alternateProductID, CompanyID, CharacterId, URLDomain, SKU, Title, Description, ButtonConfig, RetailPrice, LowestPrice, Image
                  , BackgroundColor, FontColor, buttonPositionCSS, isFromFeed, isDeleted, LastFeedUpdate, Created, CreatedBy, Updated, UpdatedBy)
        SELECT p.ProductId, p.alternateProductID, p.CompanyID, p.CharacterId, p.URLDomain, p.SKU, p.Title, p.Description, p.ButtonConfig, p.RetailPrice, p.LowestPrice, p.Image
                  , p.BackgroundColor, p.FontColor, p.buttonPositionCSS, p.isFromFeed, p.isDeleted, p.LastFeedUpdate, p.Created, p.CreatedBy, GETUTCDATE(), 'Feed Sync Process'
          FROM Product p
         WHERE p.isDeleted = 1
           AND p.CompanyID = @cid

        -- Needs DELETE -----------------------------------------------------------------------------------------------------------------------------------------------------------
        PRINT '--[ DELETE #1 ]--'
        DELETE FROM Product
         WHERE CompanyID = @cid
           AND isDeleted = 1

        -- Needs INSERT -----------------------------------------------------------------------------------------------------------------------------------------------------------
        PRINT '--[ INSERT ]--'
        INSERT INTO Product(ProductId, alternateProductID, CompanyID, CharacterId, URLDomain, SKU, Title, Description, ButtonConfig, RetailPrice, LowestPrice, Image
                  , BackgroundColor, FontColor, buttonPositionCSS, isFromFeed, isDeleted, LastFeedUpdate, Created, CreatedBy)
        SELECT NEWID()
             , a.AutoID
             , @cid
             , ''
             , ''
             , a.StockNumber
             , ''
             , ''
             , ''
             , a.ListPrice
             , a.GuaranteedSalePrice
             , COALESCE(a.ImageURL, '')
             , ''
             , ''
             , ''
             , 1
             , 0
             , a.Updated
             , GETUTCDATE()
             , 'Feed Sync Process'
          FROM RIVFeeds..AutoWithImage a
         WHERE a.ClientID = @ClientID
           AND a.StockNumber NOT IN (SELECT p.sku FROM Product p WHERE CompanyID = @cid AND isFromFeed = 1)
           AND a.AutoID NOT IN (SELECT p.alternateProductID FROM Product p WHERE CompanyID = @cid AND isFromFeed = 1)
           AND a.Active = 1

        --PRINT @cid

        FETCH NEXT FROM c1 INTO @cid
    END
    CLOSE c1
    DEALLOCATE c1
END
GO

现在我正在使用实体框架(尚未完成)的代码中编写它:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using RivWorks.Model;
using RivWorks.Model.Entities;
using RivWorks.Model.Feeds;
using RivWorks.Model.RivData;
using model = RivWorks.Model.Entities;

namespace RivWorks.Controller.Sync
{
    public static class Feeds
    {
        #region Public Methods
        public static bool Product(long ClientID)
        {
            bool retFlag = true;
            DateTime startTime = DateTime.Now;
            DateTime splitTime = startTime;
            Guid companyID;
            DateTime createdUpdated = DateTime.UtcNow;
            string createdUpdatedBy = "Feed Sync Process";
            List<SyncMessage> Activity = new List<SyncMessage>();
            List<model.Product> insertProductList = new List<Product>();
            List<model.Product> updateProductList = new List<Product>();
            List<model.Product> deleteProductList = new List<Product>();

            using (RivEntities _dbRiv = new RivWorksStore(Stores.RivConnString).NegotiationEntities())
            {
                using (FeedsEntities _dbFeed = new FeedStoreReadOnly(Stores.FeedConnString).ReadOnlyEntities())
                {
                    List<model.AutoWithImage> feedProductList = _dbFeed.AutoWithImage.Where(a => a.ClientID == ClientID).ToList();
                    List<model.vwCompanyDetails> companyDetailList = _dbRiv.vwCompanyDetails.Where(a => a.ClientID == ClientID).ToList();

                    foreach (model.vwCompanyDetails companyDetail in companyDetailList)
                    {
                        companyID = companyDetail.CompanyId;
                        List<model.Product> rivProductList = _dbRiv.Product.Include("Company").Where(a => a.Company.CompanyId == companyID).ToList();

                        #region Handle UPDATES...
                        var updateFeedProductList = from f in feedProductList
                                                    join r in rivProductList
                                                    on f.AutoID equals r.alternateProductID
                                                    where f.Updated > r.LastFeedUpdate.Value || f.Active == false
                                                    select f;
                        var updateRivProductList = from r in rivProductList
                                                   join f in feedProductList
                                                   on r.alternateProductID equals f.AutoID
                                                   where f.Updated > r.LastFeedUpdate.Value || f.Active == false
                                                   select r;

                        foreach (model.AutoWithImage feedProduct in updateFeedProductList)
                        {
                            bool alreadyExists = false;
                            foreach (model.Product rivProduct in updateRivProductList)
                            {
                                if (feedProduct.StockNumber == rivProduct.SKU && feedProduct.AutoID == rivProduct.alternateProductID)
                                {
                                    alreadyExists = true;

                                    // Active feed items...
                                    if (feedProduct.Active)
                                    {
                                        // Changed since last sync...
                                        if (feedProduct.Updated > rivProduct.LastFeedUpdate)
                                        {
                                            rivProduct.ProductId = rivProduct.ProductId;
                                            rivProduct.Company = rivProduct.Company;
                                            rivProduct.alternateProductID = feedProduct.AutoID;
                                            rivProduct.Description = String.Empty.EnforceNoNull();
                                            rivProduct.Image = feedProduct.ImageURL.EnforceNoNull();
                                            rivProduct.isDeleted = false;
                                            rivProduct.isFromFeed = true;
                                            rivProduct.LastFeedUpdate = feedProduct.Updated;
                                            rivProduct.LowestPrice = feedProduct.GuaranteedSalePrice;
                                            rivProduct.RetailPrice = feedProduct.ListPrice;
                                            rivProduct.Title = String.Empty.EnforceNoNull();
                                            rivProduct.Updated = createdUpdated;
                                            rivProduct.UpdatedBy = createdUpdatedBy;
                                        }
                                        // Not changed since last sync...
                                        else if (feedProduct.Updated <= rivProduct.LastFeedUpdate)
                                        {
                                            // nop
                                        }
                                    }
                                }
                            }
                        }
                        _dbRiv.SaveChanges();
                        #endregion

                        #region Handle DELETES...
                        List<model.Product> deleteRivProductList = _dbRiv.Product
                                                                         .Include("Company")
                                                                         .Where(a => a.Company.CompanyId == companyID
                                                                                  && a.isDeleted == true)
                                                                         .ToList();
                        // transfer to ProductDelete table...
                        foreach (model.Product delProduct in deleteRivProductList)
                        {
                            model.ProductDeleted productDeleted = new ProductDeleted();
                            productDeleted.alternateProductID = delProduct.alternateProductID;
                            productDeleted.BackgroundColor = delProduct.BackgroundColor;
                            productDeleted.ButtonConfig = delProduct.ButtonConfig;
                            productDeleted.buttonPositionCSS = delProduct.buttonPositionCSS;
                            productDeleted.CharacterId = delProduct.CharacterId;
                            productDeleted.CompanyID = companyID;
                            productDeleted.Created = delProduct.Created;
                            productDeleted.CreatedBy = delProduct.CreatedBy;
                            productDeleted.Description = delProduct.Description;
                            productDeleted.FontColor = delProduct.FontColor;
                            productDeleted.Image = delProduct.Image;
                            productDeleted.isDeleted = delProduct.isDeleted;
                            productDeleted.isFromFeed = delProduct.isFromFeed;
                            productDeleted.LastFeedUpdate = delProduct.LastFeedUpdate;
                            productDeleted.LowestPrice = delProduct.LowestPrice;
                            productDeleted.ProductId = delProduct.ProductId;
                            productDeleted.RetailPrice = delProduct.RetailPrice;
                            productDeleted.SKU = delProduct.SKU;
                            productDeleted.Title = delProduct.Title;
                            productDeleted.Updated = createdUpdated;
                            productDeleted.UpdatedBy = createdUpdatedBy;
                            productDeleted.URLDomain = delProduct.URLDomain;
                            _dbRiv.AddToProductDeleted(productDeleted);
                        }
                        int moves = _dbRiv.SaveChanges();

                        // delete the records...
                        foreach (model.Product delProduct in deleteRivProductList)
                        {
                            _dbRiv.DeleteObject(delProduct);
                        }
                        int deletes = _dbRiv.SaveChanges();
                        #endregion

                        #region Handle INSERTS...
                        // to be written...
                        #endregion
                    }
                }
            }
            return retFlag;  // remember to set this...
        }
        #endregion
    }
}

我知道现在有点乱。我包括这个,所以如果有人有关于如何更好地清理它的建议,更好的方法来利用EF这样做,我会很感激。我知道有一些非常光滑的方式可以跨实体进行连接,并希望学习而不是用脚射击自己。

2 个答案:

答案 0 :(得分:0)

您实际在哪里插入或删除记录?我看到您将其添加到insertProductListdeleteProductList,但您永远不会在表_dbRiv.Product上调用插入或删除。

我认为你想要完成的事情如下:

//perform all updates
_dbRiv.SubmitChanges();

//perform all deletes
_dbRiv.Product.DeleteAllOnSubmit(deleteProductList);
_dbRiv.SubmitChanges();

//perform inserts, one at a time
foreach(model.Product p in insertProductList)
{
    _dbRiv.Product.InsertOnSubmit(p);
    _dbRiv.SubmitChanges();
}

但是,目前尚不清楚您希望如何执行更新。看来,您应该更新model.Product的属性,而不是创建rivProduct的新实例并设置其属性。否则,使用您拥有的代码,我相信您需要使用updateProduct附加_dbRiv.Product.Attach(updateProduct, rivProduct),以便L2S知道哪些属性已更改。

答案 1 :(得分:-1)

我现在已经运行了代码。由于没有其他人在回答我必须假设我正朝着正确的方向前进。感谢。