IQueryable故障回复到IEnumerable

时间:2015-03-05 20:23:09

标签: c# linq linq-to-sql

从Linq-2-SQL数据上下文返回IQueryable<T>后,可以将其包含在执行阶段的自定义IQueryable<T>实现(例如,在枚举时)将处理基础 IQueryProvider的{​​{1}}和故障恢复以枚举整个数据上下文并使用内存中NotSupportedException应用Expression({ {1}})?

代码示例:

IQueryProvider

我正在寻找包装IQueryable的方法,如果底层IQueryProvider抛出NotSupportedException,它将自动故障回复到IEnumerable:

System.Linq.EnumerableQuery<T>

要点

这样可以方便地在数据库级别上执行所有查询,并且只有当Linq-to-SQL无法将Expression转换为SQL时才使用慢速EnumerableQuery获取整个数据集。< / p>

3 个答案:

答案 0 :(得分:2)

现在,我一直认为,如果有人问你一些绳索,你应该只问两个问题,&#34;你确定吗?&#34;和#34;你需要多少绳索?&#34;。所以你问我一些绳子,你告诉我们你确定你想要这根绳子,所以我是谁不给你这根绳子?

这就是你问的......我们称之为 v0.5 。它甚至有一个代号:武装和危险。我甚至给了 v0.1 一个代号:使用剪刀运行。这在http://pastebin.com/6qLs8TPt是不可取的。 ( v0.2 迷失在空间,因为我已经忘记了它在另一台计算机上:-))。我已添加到 v0.3使用Fire http://pastebin.com/pRbKt1Z2一个简单的记录器和一个堆栈跟踪增强器,以及 v0.4 Crossing without Looking http://pastebin.com/yEhc9vjg有点EF兼容性。

我用简单的查询和Join(s)检查了它。它似乎工作。很多可以添加。例如,&#34;智能&#34;投影机分析可能从查询中删除的Select并尝试重建它...但这可能是一个好的开始。有一个小问题:正如我在评论中写的那样,根据是否存在Select,LINQ-to-SQL会在其对象跟踪器中加载返回的对象。如果在&#34;原创&#34;查询有一个Select(所以没有对象跟踪),我删除它以在本地执行它,然后将加载完整的对象,并且对象跟踪将被转换为&#34; on&#34;。显然你可以context.ObjectTrackingEnabled = false

现在它应该与简单的EF查询兼容。 AsNoTracking() / Include() 不兼容。

NotSupportedException属性中跟踪Exception。请注意,此属性仅在最多&#34;外部&#34;中修改。 IQueryable的。所以

// Optional :-) You can ignore the logger and live happy...
// Not optional: you can live happy very far from me!
SafeQueryable.Logger = (iqueriable, expression, e) => Console.WriteLine(e);

var q1 = context.Items.AsSafe();
var q2 = q1.Where(x => x.ID.GetHashCode() > 0);
var q3 = q2.Select(x => x.ID);

var ex = ((ISafeQueryable)q3).Exception;

如何使用它?有一种扩展方法AsSafe()。你在IQueryable<T>上使用它,然后你执行查询......并且快乐地生活...... 玩火! :-)

使用示例:

// Queries that are NotSupportedException with LINQ-to-SQL
var q1 = context.Items.AsSafe().Where(x => x.ID.GetHashCode() > 0).Take(2).Select(x => x.ID);
var q2 = context.Items.Where(x => x.ID.GetHashCode() > 0).AsSafe().Take(2).Select(x => x.ID);
var q3 = context.Items.Where(x => x.ID.GetHashCode() > 0).Take(2).AsSafe().Select(x => x.ID);
var q4 = context.Items.Where(x => x.ID.GetHashCode() > 0).Take(2).Select(x => x.ID).AsSafe();

//// Queries that are OK with LINQ-to-SQL
//var q1 = context.Items.AsSafe().Where(x => x.ID > 0).Take(2).Select(x => x.ID);
//var q2 = context.Items.Where(x => x.ID > 0).AsSafe().Take(2).Select(x => x.ID);
//var q3 = context.Items.Where(x => x.ID > 0).Take(2).AsSafe().Select(x => x.ID);
//var q4 = context.Items.Where(x => x.ID > 0).Take(2).Select(x => x.ID).AsSafe();

var r1 = q1.ToList();
var r2 = q2.First();
var r3 = q3.Max();
// The Aggregate isn't normally supported by LINQ-to-SQL
var r4 = q4.Aggregate((x, y) => x + y);

var ex1 = ((ISafeQueryable)q1).Exception;
var ex2 = ((ISafeQueryable)q2).Exception;
var ex3 = ((ISafeQueryable)q3).Exception;
var ex4 = ((ISafeQueryable)q4).Exception;

如您所见,您可以在任何步骤使用AsSafe(),它都可以使用。发生这种情况是因为查询由AsSafe()中的SafeQueryable<T>包裹,该IQueryProvider同时是IQueryable<T>IQueryable(类似于LINQ-to-SQL的类)。通过这种方式,您调用它的每个其他SafeQueryable<T>方法将生成其他using System; using System.Collections; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Linq.Expressions; using System.Reflection; /// <summary> /// v0.5 Codename: Armed and Dangerous /// /// (previous version v0.1 Codename: Running with Scissors: http://pastebin.com/6qLs8TPt) /// (previous version v0.2 Codename: Lost in Space: lost in space :-) ) /// (previous version v0.3 Codename: Playing with Fire: http://pastebin.com/pRbKt1Z2) /// (previous version v0.4 Codename: Crossing without Looking: http://pastebin.com/yEhc9vjg) /// /// Support class with an extension method to make "Safe" an IQueryable /// or IQueryable&lt;T&gt;. Safe as "I work for another company, thousand /// of miles from you. I do hope I won't ever buy/need something from /// your company". /// The Extension methods wraps a IQueryable in a wrapper that then can /// be used to execute a query. If the original Provider doesn't suppport /// some methods, the query will be partially executed by the Provider /// and partially executed locally. /// /// Minimal support for EF. /// /// Note that this **won't** play nice with the Object Tracking! /// /// Not suitable for programmers under 5 years (of experience)! /// Dangerous if inhaled or executed. /// </summary> public static class SafeQueryable { /// <summary> /// Optional logger to log the queries that are "corrected. Note that /// there is no "strong guarantee" that the IQueriable (that is also /// an IQueryProvider) is executing its (as in IQueriable.Expression) /// Expression, so an explicit Expression parameter is passed. This /// because the IQueryProvider.Execute method receives an explicit /// expression parameter. Clearly there is a "weak guarantee" that /// unless you do "strange things" this won't happen :-) /// </summary> public static Action<IQueryable, Expression, NotSupportedException> Logger { get; set; } /// <summary> /// Return a "Safe" IQueryable&lt;T&gt;. /// </summary> /// <typeparam name="T"></typeparam> /// <param name="source"></param> /// <returns></returns> public static IQueryable<T> AsSafe<T>(this IQueryable<T> source) { if (source is SafeQueryable<T>) { return source; } return new SafeQueryable<T>(source); } } /// <summary> /// Simple interface useful to collect the Exception, or to recognize /// a SafeQueryable&lt;T&gt;. /// </summary> public interface ISafeQueryable { NotSupportedException Exception { get; } } /// <summary> /// "Safe" wrapper around a IQueryable&lt;T;&gt; /// </summary> /// <typeparam name="T"></typeparam> public class SafeQueryable<T> : IOrderedQueryable<T>, IQueryProvider, ISafeQueryable { protected static readonly FieldInfo StackTraceStringField = typeof(Exception).GetField("_stackTraceString", BindingFlags.Instance | BindingFlags.NonPublic); // The query. Note that it can be "transformed" to a "safe" version // of itself. When it happens, IsSafe becomes true public IQueryable<T> Query { get; protected set; } // IsSafe means that the query has been "corrected" if necessary and // won't throw a NotSupportedException protected bool IsSafe { get; set; } // Logging of the "main" NotSupportedException. public NotSupportedException Exception { get; protected set; } public SafeQueryable(IQueryable<T> query) { Query = query; } /* IQueryable<T> */ public IEnumerator<T> GetEnumerator() { if (IsSafe) { return Query.GetEnumerator(); } return new SafeEnumerator(this); } IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); } public Type ElementType { get { return Query.ElementType; } } public Expression Expression { get { return Query.Expression; } } public IQueryProvider Provider { get { return this; } } /* IQueryProvider */ public IQueryable<TElement> CreateQuery<TElement>(Expression expression) { return CreateQueryImpl<TElement>(expression); } public IQueryable CreateQuery(Expression expression) { Type iqueryableArgument = GetIQueryableTypeArgument(expression.Type); MethodInfo createQueryImplMethod = typeof(SafeQueryable<T>) .GetMethod("CreateQueryImpl", BindingFlags.Instance | BindingFlags.NonPublic) .MakeGenericMethod(iqueryableArgument); return (IQueryable)createQueryImplMethod.Invoke(this, new[] { expression }); } public TResult Execute<TResult>(Expression expression) { return ExecuteImpl<TResult>(expression); } public object Execute(Expression expression) { Type iqueryableArgument = GetIQueryableTypeArgument(expression.Type); MethodInfo executeImplMethod = typeof(SafeQueryable<T>) .GetMethod("ExecuteImpl", BindingFlags.Instance | BindingFlags.NonPublic) .MakeGenericMethod(iqueryableArgument); return executeImplMethod.Invoke(this, new[] { expression }); } /* Implementation methods */ // Gets the T of IQueryablelt;T&gt; protected static Type GetIQueryableTypeArgument(Type type) { IEnumerable<Type> interfaces = type.IsInterface ? new[] { type }.Concat(type.GetInterfaces()) : type.GetInterfaces(); Type argument = (from x in interfaces where x.IsGenericType let gt = x.GetGenericTypeDefinition() where gt == typeof(IQueryable<>) select x.GetGenericArguments()[0]).FirstOrDefault(); return argument; } protected IQueryable<TElement> CreateQueryImpl<TElement>(Expression expression) { return new SafeQueryable<TElement>(Query.Provider.CreateQuery<TElement>(expression)); } protected TResult ExecuteImpl<TResult>(Expression expression) { if (IsSafe && Query.Expression == expression) { TResult result = Query.Provider.Execute<TResult>(expression); return result; } try { // Note that thanks to how everything knits together, if you // call query1.First(); query1.First(); the second call will // get to use the query cached by the first one (technically // the cached query will be only the "query1" part) // We try executing it directly TResult result = Query.Provider.Execute<TResult>(expression); // Success! if (!IsSafe && CanCache(expression, true)) { IsSafe = true; } return result; } catch (NotSupportedException e1) { // Clearly there was a NotSupportedException :-) Tuple<IEnumerator<T>, bool, TResult> result = HandleEnumerationFailure<TResult>(e1, expression, true); if (result == null) { throw; } // Success! return result.Item3; } } // Is used both indirectly by GetEnumerator() and by Execute<>. // The returned Tuple<,,> has the first two elements that are valid // when used by the GetEnumerator() and the last that is valid // when used by Execute<>. protected Tuple<IEnumerator<T>, bool, TResult> HandleEnumerationFailure<TResult>(NotSupportedException e1, Expression expression, bool singleResult) { // We "augment" the exception with the full stack trace AugmentStackTrace(e1, 3); if (SafeQueryable.Logger != null) { SafeQueryable.Logger(this, expression, e1); } // We save this first exception Exception = e1; { var query = Query; MethodInfo executeSplittedMethod = typeof(SafeQueryable<T>).GetMethod("ExecuteSplitted", BindingFlags.Instance | BindingFlags.NonPublic); MethodCallExpression call; Expression innerExpression = expression; Type iqueryableArgument; // We want to check that there is a MethodCallExpression with // at least one argument, and that argument is an Expression // of type IQueryable<iqueryableArgument>, and we save the // iqueryableArgument while ((call = innerExpression as MethodCallExpression) != null && call.Arguments.Count > 0 && (innerExpression = call.Arguments[0] as Expression) != null && (iqueryableArgument = GetIQueryableTypeArgument(innerExpression.Type)) != null) { try { Tuple<IEnumerator<T>, bool, TResult> result2 = (Tuple<IEnumerator<T>, bool, TResult>)executeSplittedMethod.MakeGenericMethod(iqueryableArgument, typeof(TResult)).Invoke(this, new object[] { expression, call, innerExpression, singleResult }); return result2; } catch (TargetInvocationException e2) { if (!(e2.InnerException is NotSupportedException)) { throw; } } } return null; } } // Is used both indirectly by GetEnumerator() and by Execute<>. // The returned Tuple<,,> has the first two elements that are valid // when used by the GetEnumerator() and the last that is valid // when used by Execute<>. protected Tuple<IEnumerator<T>, bool, TResult> ExecuteSplitted<TInner, TResult>(Expression expression, MethodCallExpression call, Expression innerExpression, bool singleResult) { // The NotSupportedException should happen here IQueryable<TInner> innerQueryable = Query.Provider.CreateQuery<TInner>(innerExpression); // We try executing it directly IEnumerator<TInner> innerEnumerator = innerQueryable.GetEnumerator(); bool moveNextSuccess = innerEnumerator.MoveNext(); IEnumerator<T> enumerator; TResult singleResultValue; // Success! { // Now we wrap the partially used enumerator in an // EnumerableFromStartedEnumerator IEnumerable<TInner> innerEnumerable = new EnumerableFromStartedEnumerator<TInner>(innerEnumerator, moveNextSuccess, innerQueryable); // Then we apply an AsQueryable, that does some magic // to make the query appear to be a Queryable IQueryable<TInner> innerEnumerableAsQueryable = Queryable.AsQueryable(innerEnumerable); // We rebuild a new expression by changing the "old" // inner parameter of the MethodCallExpression with the // queryable we just built var arguments = call.Arguments.ToArray(); arguments[0] = Expression.Constant(innerEnumerableAsQueryable); MethodCallExpression call2 = Expression.Call(call.Object, call.Method, arguments); Expression expressionWithFake = new SimpleExpressionReplacer(call, call2).Visit(expression); // We "execute" locally the whole query through a second // "outer" instance of the EnumerableQuery (this class is // the class that "implements" the "fake-magic" of // AsQueryable) IQueryable<T> queryable = new EnumerableQuery<T>(expressionWithFake); if (singleResult) { enumerator = null; moveNextSuccess = false; singleResultValue = queryable.Provider.Execute<TResult>(queryable.Expression); } else { enumerator = queryable.GetEnumerator(); moveNextSuccess = enumerator.MoveNext(); singleResultValue = default(TResult); } } // We could enter here with a new query from Execute<>(), // with IsSafe == true . It would be useless to try to cache // that query. if (!IsSafe && CanCache(expression, singleResult)) { Stopwatch sw = Stopwatch.StartNew(); // We redo the same things to create a second copy of // the query that is "complete", not partially // enumerated. This second copy will be cached in the // SafeQueryable<T>. // Note that forcing the Queryable.AsQueryable to not // "recast" the query to the original IQueryable<T> is // quite complex :-) We have to // .AsEnumerable().Select(x => x) . IEnumerable<TInner> innerEnumerable = innerQueryable.AsEnumerable().Select(x => x); IQueryable<TInner> innerEnumerableAsQueryable = Queryable.AsQueryable(innerEnumerable); // Note that we cache the SafeQueryable<>.Expression! var arguments = call.Arguments.ToArray(); arguments[0] = Expression.Constant(innerEnumerableAsQueryable); MethodCallExpression call2 = Expression.Call(call.Object, call.Method, arguments); Expression expressionWithFake = new SimpleExpressionReplacer(call, call2).Visit(Expression); IQueryable<T> queryable = new EnumerableQuery<T>(expressionWithFake); // Now the SafeQueryable<T> has a query that *just works* Query = queryable; IsSafe = true; sw.Stop(); Console.WriteLine(sw.ElapsedTicks); } return Tuple.Create(enumerator, moveNextSuccess, singleResultValue); } protected bool CanCache(Expression expression, bool singleResult) { // GetEnumerator() doesn't permit changing the query if (!singleResult) { return true; } // The expression is equal to the one in Query.Expression // (should be very rare!) if (Query.Expression == expression) { return true; } MethodCallExpression call; Expression innerExpression = expression; Type iqueryableArgument; // We walk back the expression to see if a smaller part of it is // the "original" Query.Expression . This happens for example // when one of the operators that returns a single value // (.First(), .FirstOrDefault(), .Single(), .SingleOrDefault(), // .Any(), .All()., .Min(), .Max(), ...) are used. while ((call = innerExpression as MethodCallExpression) != null && call.Arguments.Count > 0 && (innerExpression = call.Arguments[0] as Expression) != null && (iqueryableArgument = GetIQueryableTypeArgument(innerExpression.Type)) != null) { if (Query.Expression == innerExpression) { return true; } } return false; } // The StackTrace of an Exception "stops" at the catch. This method // "augments" it to include the full stack trace. protected static void AugmentStackTrace(Exception e, int skipFrames = 2) { // Playing with a private field here. Don't do it at home :-) // If not present, do nothing. if (StackTraceStringField == null) { return; } string stack1 = e.StackTrace; string stack2 = new StackTrace(skipFrames, true).ToString(); string stack3 = stack1 + stack2; StackTraceStringField.SetValue(e, stack3); } /* Utility classes */ // An IEnumerator<T> that applies the AsSafe() paradigm, knowing that // normally the exception happens only on the first MoveFirst(). protected class SafeEnumerator : IEnumerator<T> { protected readonly SafeQueryable<T> SafeQueryable_; protected IEnumerator<T> Enumerator { get; set; } public SafeEnumerator(SafeQueryable<T> safeQueryable) { SafeQueryable_ = safeQueryable; } public T Current { get { return Enumerator != null ? Enumerator.Current : default(T); } } public void Dispose() { if (Enumerator != null) { Enumerator.Dispose(); } } object IEnumerator.Current { get { return Current; } } public bool MoveNext() { // We handle exceptions only on first MoveNext() if (Enumerator != null) { return Enumerator.MoveNext(); } try { // We try executing it directly Enumerator = SafeQueryable_.Query.GetEnumerator(); bool result = Enumerator.MoveNext(); // Success! SafeQueryable_.IsSafe = true; return result; } catch (NotSupportedException e1) { // Clearly there was a NotSupportedException :-) Tuple<IEnumerator<T>, bool, T> result = SafeQueryable_.HandleEnumerationFailure<T>(e1, SafeQueryable_.Query.Expression, false); if (result == null) { throw; } Enumerator = result.Item1; return result.Item2; } } public void Reset() { if (Enumerator != null) { Enumerator.Reset(); } } } } // A simple expression visitor to replace some nodes of an expression // with some other nodes public class SimpleExpressionReplacer : ExpressionVisitor { public readonly Dictionary<Expression, Expression> Replaces; public SimpleExpressionReplacer(Dictionary<Expression, Expression> replaces) { Replaces = replaces; } public SimpleExpressionReplacer(IEnumerable<Expression> from, IEnumerable<Expression> to) { Replaces = new Dictionary<Expression, Expression>(); using (var enu1 = from.GetEnumerator()) using (var enu2 = to.GetEnumerator()) { while (true) { bool res1 = enu1.MoveNext(); bool res2 = enu2.MoveNext(); if (!res1 || !res2) { if (!res1 && !res2) { break; } if (!res1) { throw new ArgumentException("from shorter"); } throw new ArgumentException("to shorter"); } Replaces.Add(enu1.Current, enu2.Current); } } } public SimpleExpressionReplacer(Expression from, Expression to) { Replaces = new Dictionary<Expression, Expression> { { from, to } }; } public override Expression Visit(Expression node) { Expression to; if (node != null && Replaces.TryGetValue(node, out to)) { return base.Visit(to); } return base.Visit(node); } } // Simple IEnumerable<T> that "uses" an IEnumerator<T> that has // already received a MoveNext(). "eats" the first MoveNext() // received, then continues normally. For shortness, both IEnumerable<T> // and IEnumerator<T> are implemented by the same class. Note that if a // second call to GetEnumerator() is done, the "real" IEnumerator<T> will // be returned, not this proxy implementation. public class EnumerableFromStartedEnumerator<T> : IEnumerable<T>, IEnumerator<T> { public readonly IEnumerator<T> Enumerator; public readonly IEnumerable<T> Enumerable; // Received by creator. Return value of MoveNext() done by caller protected bool FirstMoveNextSuccessful { get; set; } // The Enumerator can be "used" only once, then a new enumerator // can be requested by Enumerable.GetEnumerator() // (default = false) protected bool Used { get; set; } // The first MoveNext() has been already done (default = false) protected bool DoneMoveNext { get; set; } public EnumerableFromStartedEnumerator(IEnumerator<T> enumerator, bool firstMoveNextSuccessful, IEnumerable<T> enumerable) { Enumerator = enumerator; FirstMoveNextSuccessful = firstMoveNextSuccessful; Enumerable = enumerable; } public IEnumerator<T> GetEnumerator() { if (Used) { return Enumerable.GetEnumerator(); } Used = true; return this; } IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); } public T Current { get { // There are various school of though on what should // happens if called before the first MoveNext() or // after a MoveNext() returns false. We follow the // "return default(TInner)" school of thought for the // before first MoveNext() and the "whatever the // Enumerator wants" for the after a MoveNext() returns // false if (!DoneMoveNext) { return default(T); } return Enumerator.Current; } } public void Dispose() { Enumerator.Dispose(); } object IEnumerator.Current { get { return Current; } } public bool MoveNext() { if (!DoneMoveNext) { DoneMoveNext = true; return FirstMoveNextSuccessful; } return Enumerator.MoveNext(); } public void Reset() { // This will 99% throw :-) Not our problem. Enumerator.Reset(); // So it is improbable we will arrive here DoneMoveNext = true; } } 个对象(因此它可以自我复制:-))

var q5 = context.Items.Where(x => x.ID > context.Nodis.Min(y => y.ID.GetHashCode()));
var r5 = q5.ToList();

请注意,LINQ-to-SQL中存在一个错误:

NotSupportedException

这不会抛出context.SomeTable但不会正确执行。我认为这可能是许多使用{{1}}&#34;内部&#34;在查询中。

答案 1 :(得分:-1)

除了自己编写ExtressionTree

之外,我不认为这是一种自动执行此操作的方法。

但是分析你的代码我认为你需要的是使用扩展方法封装你的查询

像这样的东西(我已经使用LinqToSql测试了它的概念并且它支持Length属性):

public static class Extensions
{
    public static IQueryable<User> WhereUserMatches(this IQueryable<User> source)
    {
        return source.Where(x => x.Id + x.Name.Length > 12);
    }
}

用法

var myFileterdUsers = users.WhereUserMatches();

由于您正在使用IQueryable,因此您的情况将被发送到不在内存中的服务器

使用这种方法,您可以封装复杂的查询,并在服务器而不是内存中实际执行它们。

关于隐藏异常的一个词

如果提供商无法处理查询,我建议您不要通过回退使用IEnumerable(在内存查询中)来隐藏异常,因为这可能会导致您的应用程序中的不良行为

如果基础查询提供程序无法将其转换为有效的ExpressionTree

,我建议您明确并抛出异常

例外是好的,他们是我们的朋友,如果我们做错了,他们会告诉我们

隐藏异常是相反的,并且它被认为是一种不好的做法,隐藏异常会给你错误你的应用程序工作,即使它很可能不会做正是你的想法

答案 2 :(得分:-1)

如果我没弄错,你需要在使用你的方法之前通过.ToList()实现集合。尝试将集合转换为列表并将其存储在变量中。之后,尝试在此变量中生成的集合中使用他的方法。