python resettable实例方法memoization装饰器

时间:2010-12-13 17:38:33

标签: python decorator memoization descriptor

我正在尝试为类的实例方法构建一个装饰器来记忆结果。 (之前已经完成了一百万次)但是,我希望能够在任何时候重置memoized缓存(例如,如果实例状态中的某些内容发生更改,这可能会更改无方法的结果)与它的args相关)。因此,我尝试将装饰器构建为类而不是函数,以便我可以作为类成员访问缓存。这导致我学习描述符的路径,特别是__get__方法,这是我实际上被困的地方。我的代码看起来像这样:

import time

class memoized(object):

    def __init__(self, func):
        self.func = func
        self.cache = {}

    def __call__(self, *args, **kwargs):

        key = (self.func, args, frozenset(kwargs.iteritems()))

        try:
            return self.cache[key]
        except KeyError:
            self.cache[key] = self.func(*args, **kwargs)
            return self.cache[key]
        except TypeError:
            # uncacheable, so just return calculated value without caching
            return self.func(*args, **kwargs)

    # self == instance of memoized
    # obj == instance of my_class
    # objtype == class object of __main__.my_class
    def __get__(self, obj, objtype=None):
        """Support instance methods"""
        if obj is None:
            return self

        # new_func is the bound method my_func of my_class instance
        new_func = self.func.__get__(obj, objtype)

        # instantiates a brand new class...this is not helping us, because it's a 
        # new class each time, which starts with a fresh cache
        return self.__class__(new_func)

    # new method that will allow me to reset the memoized cache
    def reset(self):
        print "IN RESET"
        self.cache = {}

class my_class:
    @memoized
    def my_func(self, val):
        print "in my_func"
        time.sleep(2)
        return val


c = my_class()

print "should take time"
print c.my_func(55)
print

print "should be instant"
print c.my_func(55)
print

c.my_func.reset()

print "should take time"
print c.my_func(55)

这是否清晰和/或可能?每次调用__get__时,我都会得到一个memoized类的全新实例,它会丢失包含实际数据的缓存。我一直在与__get__一起努力,但我没有取得多大进展。

我完全错过了这个问题的完全独立的方法吗?并且欢迎并赞赏所有建议/建议。感谢。

3 个答案:

答案 0 :(得分:7)

我没有尝试计算实施的机制,而是从PythonDecoratorLibrary获取了memoized装饰器类,并修改了它以添加reset。以下是结果;我使用的技巧是为装饰函数本身添加一个可调用的reset属性。

    class memoized2(object):
       """Decorator that caches a function's return value each time it is called.
       If called later with the same arguments, the cached value is returned, and
       not re-evaluated.
       """
       def __init__(self, func):
          self.func = func
          self.cache = {}
       def __call__(self, *args):
          try:
             return self.cache[args]
          except KeyError:
             value = self.func(*args)
             self.cache[args] = value
             return value
          except TypeError:
             # uncachable -- for instance, passing a list as an argument.
             # Better to not cache than to blow up entirely.
             return self.func(*args)
       def __repr__(self):
          """Return the function's docstring."""
          return self.func.__doc__
       def __get__(self, obj, objtype):
          """Support instance methods."""
          fn = functools.partial(self.__call__, obj)
          fn.reset = self._reset
          return fn
       def _reset(self):
          self.cache = {}


    class my_class:
        @memoized2
        def my_func(self, val):
            print "in my_func"
            time.sleep(2)
            return val


    c = my_class()

    print "should take time"
    print c.my_func(55)
    print

    print "should be instant"
    print c.my_func(55)
    print

    c.my_func.reset()

    print "should take time"
    print c.my_func(55)

答案 1 :(得分:2)

在@aix给出的原始问题的答案的基础上,我创建了一个我认为可以改进它的类。主要特征是缓存的值存储为正在装饰方法的实例的属性,因此很容易重置它们。

class memoize(object):
  def __init__(self, func):
    #print "Init"
    self.func = func

  def __call__(self, *args):
    #print "Call"
    if not self.func in self.cache:
        self.cache[self.func] = {}
    try:
        return self.cache[self.func][args]
    except KeyError:
        value = self.func(*args)
        self.cache[self.func][args] = value
        return value
    except TypeError:
        # uncachable -- for instance, passing a list as an argument.
        # Better to not cache than to blow up entirely.
        return self.func(*args)

  def __repr__(self):
    """Return the function's docstring."""
    return self.func.__doc__

  def __get__(self, obj, objtype):
    """Support instance methods."""
    #print "Get", obj, objtype
    fn = functools.partial(self.__call__, obj)
    try:
        self.cache = obj.cache
    except:
        obj.cache = {}
        self.cache = obj.cache
    #print self.cache
    return fn

作为使用示例:

class MyClass(object):
    def __init__(self,data):
        self.data = data

    def update(self,data):
        self.data = data
        self.cache = {}

    @memoize
    def func1(self,x):
        print "Computing func1"
        return "I am func1 of %s. Data is %s. x is %s\n" % (self, self.data, x)

    @memoize
    def func2(self,x):
        print "Computing func2"
        return "I am func2 of %s. Data is %s. x is %s\n" % (self, self.data, x)

    def func3(self,x):
        print "Computing func3"
        return "I am func3 of %s. Data is %s. x is %s\n" % (self, self.data, x)

mc1 = MyClass("data1")
mc2 = MyClass("data2")
mc3 = MyClass("data3")

print mc1.func1(1) 
print mc1.func1(1) 
print mc1.func2(1) 
print mc1.func2(1) 
print mc1.func3(1) 
print mc1.func3(1) 

print mc2.func1(1) 
print mc2.func1(1) 
print mc2.func2(1) 
print mc2.func2(1) 
print mc2.func3(1) 
print mc2.func3(1) 

print "Update mc1\n"
mc1.update("data1new")

print mc1.func1(1) 
print mc1.func2(1) 
print mc1.func3(1) 
print mc2.func1(1) 
print mc2.func2(1) 
print mc2.func3(1) 

得到输出:

Computing func1
I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func1
I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Update mc1

Computing func1
I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

答案 2 :(得分:0)

好吧,我想在您的代码中指出两个性能问题。这不是你问题的答案,但我无法作出评论。感谢@delnan指出has_key已被弃用。而不是:

    try:
        return self.cache[key]
    except KeyError:
        self.cache[key] = self.func(*args, **kwargs)
        return self.cache[key]
    except TypeError:
        # uncacheable, so just return calculated value without caching
        return self.func(*args, **kwargs)

我会这样做:

resultDone = False
result = None
try:
  if key in self.cache: return self.cache[key]
  else:
    result = self.func(*args, **kwargs)
    resultDone = True
    self.cache[key] = result
except TypeError: # unhashable key
  pass
if resultDone: return result
else: return self.func(*args, **kwargs)

这可以避免:a)尝试/除KeyError; b)返回时致电cache[key]; c)在不可用密钥上再次调用该函数。

相关问题