Question

我正在编写一个基于MPI的应用程序（但MPI在我的问题中并不重要，我只提到它以揭示基本原理）并且在某些情况下，当工作项少于进程时，我需要创建一个新的沟通者，不包括无关的过程。最后，必须通过有工作的流程（并且只有他们）来释放新的沟通者。

这样做的一个简洁方法是写：

with filter_comm(comm, nworkitems) as newcomm:
    ... do work with communicator newcomm...

正在由有工作的进程执行的正文。

上下文管理器中是否有办法避免执行正文？我理解上下文管理器的设计是为了避免隐藏控制流，但我想知道是否可以规避控制流，因为在我的情况下，我认为为了清晰起见，这是合理的。

Answer 1

如PEP 377中所述，已经提出并拒绝了有条件地跳过上下文管理器主体的能力。

以下是一些如何实现功能的方法。

首先是什么不起作用：不从上下文管理器生成器中产生。

@contextlib.contextmanager
def drivercontext():
  driver, ok = driverfactory()
  try:
    if ok:
      yield driver
    else:
      print 'skip because driver not ok'
  finally:
    driver.quit()

with drivercontext() as driver:
  dostuff(driver)

不屈服会导致RuntimeException引发contextmanager。至少finally可靠地执行。

方法1：手动跳过身体。

@contextlib.contextmanager
def drivercontext():
  driver, ok = driverfactory()
  try:
    yield driver, ok
  finally:
    driver.quit()

with drivercontext() as (driver, ok):
  if ok:
    dostuff(driver)
  else:
    print 'skip because driver not ok'

这虽然明确，否定了上下文管理器主体的大部分简洁性。应该隐藏在上下文管理器中的逻辑溢出到正文，并且必须为每次调用重复。

方法2：滥用发电机。

def drivergenerator():
  driver, ok = driverfactory()
  try:
    if ok:
      yield driver
    else:
      print 'skip because driver not ok'
  finally:
    driver.quit()

for driver in drivergenerator():
  dostuff(driver)

这与可以跳过正文的上下文管理器非常相似。不幸的是，它看起来非常像一个循环。

方法3：手动完成所有事情。

driver, ok = driverfactory()
try:
  if ok:
    dostuff(driver)
  else:
    print 'skip because driver not ok'
finally:
  driver.quit()

呸。这是什么？冗长可以与Java相媲美。

只能通过回调来完成此操作。

def withdriver(callback):
  driver, ok = driverfactory()
  try:
    if ok:
      callback(driver)
    else:
      print 'skip because driver not ok'
  finally:
    driver.quit()

withdriver(dostuff)

好吧，好吧。上下文管理器抽象了许多案例。但总有一些裂缝可以通过。这让我想起了the law of leaky abstractions。

以下是一些演示这些方法和其他方法的代码。

import contextlib
import functools

# ----------------------------------------------------------------------
# this code is a simulation of the code not under my control
# report and ok and fail are variables for use in the simulation
# they do not exist in the real code
# report is used to report certain checkpoints
# ok is used to tell the driver object whether it is ok or not
# fail is used tell dostuff whether it should fail or not

class Driver(object):
  def __init__(self, report, ok):
    # driver can be ok or not ok
    # driver must always quit after use
    # regardless if it is ok or not
    print 'driver init (ok: %s)' % ok
    self.report = report

  def drivestuff(self):
    # if driver is not ok it is not ok to do stuff with it
    self.report.drivestuffrun = True

  def quit(self):
    # driver must always quit regardless of ok or not
    print 'driver quit'
    self.report.driverquit = True

def driverfactory(report, ok=True):
  # driver factory always returns a driver
  # but sometimes driver is not ok
  # this is indicated by second return value
  # not ok driver must still be quit
  return Driver(report, ok), ok

class DoStuffFail(Exception):
  pass

def dostuff(driver, fail=False):
  # this method does a lot of stuff
  # dostuff expects an ok driver
  # it does not check whether the driver is ok
  driver.drivestuff()
  # do stuff can also fail independent of driver
  if fail:
    print 'dostuff fail'
    raise DoStuffFail('doing stuff fail')
  else:
    print 'dostuff'

# ----------------------------------------------------------------------
class AbstractScenario(object):
  def __init__(self, driverfactory, dostuff):
    self.driverfactory = functools.partial(driverfactory, report=self)
    self.dostuff = dostuff
    self.driverquit = False
    self.drivestuffrun = False

# ----------------------------------------------------------------------
class Scenario0(AbstractScenario):

  def run(self):
    print '>>>> not check driver ok and not ensure driver quit'
    driver, ok = self.driverfactory()
    self.dostuff(driver)
    driver.quit()

# ----------------------------------------------------------------------
class Scenario1(AbstractScenario):

  def run(self):
    print '>>>> check driver ok but not ensure driver quit'
    driver, ok = self.driverfactory()
    if ok:
      self.dostuff(driver)
    else:
      print 'skip because driver not ok'
    driver.quit()

# ----------------------------------------------------------------------
class Scenario2(AbstractScenario):

  def run(self):
    print '>>>> check driver ok and ensure driver quit'
    driver, ok = self.driverfactory()
    try:
      if ok:
        self.dostuff(driver)
      else:
        print 'skip because driver not ok'
    finally:
      driver.quit()

# ----------------------------------------------------------------------
class Scenario3(AbstractScenario):

  @contextlib.contextmanager
  def drivercontext(self, driverfactory):
    driver, ok = driverfactory()
    try:
      if ok:
        yield driver
      else:
        print 'skip because driver not ok'
    finally:
      driver.quit()

  def run(self):
    print '>>>> skip body by not yielding (does not work)'
    with self.drivercontext(self.driverfactory) as driver:
      self.dostuff(driver)

# ----------------------------------------------------------------------
class Scenario4(AbstractScenario):

  @contextlib.contextmanager
  def drivercontext(self, driverfactory):
    driver, ok = driverfactory()
    try:
      yield driver, ok
    finally:
      driver.quit()

  def run(self):
    print '>>>> skip body manually by returning flag with context'
    with self.drivercontext(self.driverfactory) as (driver, ok):
      if ok:
        self.dostuff(driver)
      else:
        print 'skip because driver not ok'

# ----------------------------------------------------------------------
class Scenario5(AbstractScenario):

  def drivergenerator(self, driverfactory):
    driver, ok = driverfactory()
    try:
      if ok:
        yield driver
      else:
        print 'skip because driver not ok'
    finally:
      driver.quit()

  def run(self):
    print '>>>> abuse generator as context manager'
    for driver in self.drivergenerator(self.driverfactory):
      self.dostuff(driver)

# ----------------------------------------------------------------------
def doscenarios(driverfactory, dostuff, drivestuffrunexpected=True):
  for Scenario in AbstractScenario.__subclasses__():
    print '-----------------------------------'
    scenario = Scenario(driverfactory, dostuff)
    try:
      try:
        scenario.run()
      except DoStuffFail as e:
        print 'dostuff fail is ok'
      if not scenario.driverquit:
        print '---- fail: driver did not quit'
      if not scenario.drivestuffrun and drivestuffrunexpected:
        print '---- fail: drivestuff did not run'
      if scenario.drivestuffrun and not drivestuffrunexpected:
        print '---- fail: drivestuff did run'
    except Exception as e:
      print '----- fail with exception'
      print '--------', e

# ----------------------------------------------------------------------
notokdriverfactory = functools.partial(driverfactory, ok=False)
dostufffail = functools.partial(dostuff, fail=True)

print '============================================'
print '==== driver ok and do stuff will not fail =='
doscenarios(driverfactory, dostuff)

print '============================================'
print '==== do stuff will fail ================='
doscenarios(driverfactory, dostufffail)

print '==========================================='
print '===== driver is not ok ==================='
doscenarios(notokdriverfactory, dostuff, drivestuffrunexpected=False)

输出。

============================================
==== driver ok and do stuff will not fail ==
-----------------------------------
>>>> not check driver ok and not ensure driver quit
driver init (ok: True)
dostuff
driver quit
-----------------------------------
>>>> check driver ok but not ensure driver quit
driver init (ok: True)
dostuff
driver quit
-----------------------------------
>>>> check driver ok and ensure driver quit
driver init (ok: True)
dostuff
driver quit
-----------------------------------
>>>> skip body by not yielding (does not work)
driver init (ok: True)
dostuff
driver quit
-----------------------------------
>>>> skip body manually by returning flag with context
driver init (ok: True)
dostuff
driver quit
-----------------------------------
>>>> abuse generator as context manager
driver init (ok: True)
dostuff
driver quit
============================================
==== do stuff will fail =================
-----------------------------------
>>>> not check driver ok and not ensure driver quit
driver init (ok: True)
dostuff fail
dostuff fail is ok
---- fail: driver did not quit
-----------------------------------
>>>> check driver ok but not ensure driver quit
driver init (ok: True)
dostuff fail
dostuff fail is ok
---- fail: driver did not quit
-----------------------------------
>>>> check driver ok and ensure driver quit
driver init (ok: True)
dostuff fail
driver quit
dostuff fail is ok
-----------------------------------
>>>> skip body by not yielding (does not work)
driver init (ok: True)
dostuff fail
driver quit
dostuff fail is ok
-----------------------------------
>>>> skip body manually by returning flag with context
driver init (ok: True)
dostuff fail
driver quit
dostuff fail is ok
-----------------------------------
>>>> abuse generator as context manager
driver init (ok: True)
dostuff fail
driver quit
dostuff fail is ok
===========================================
===== driver is not ok ===================
-----------------------------------
>>>> not check driver ok and not ensure driver quit
driver init (ok: False)
dostuff
driver quit
---- fail: drivestuff did run
-----------------------------------
>>>> check driver ok but not ensure driver quit
driver init (ok: False)
skip because driver not ok
driver quit
-----------------------------------
>>>> check driver ok and ensure driver quit
driver init (ok: False)
skip because driver not ok
driver quit
-----------------------------------
>>>> skip body by not yielding (does not work)
driver init (ok: False)
skip because driver not ok
driver quit
----- fail with exception
-------- generator didn't yield
-----------------------------------
>>>> skip body manually by returning flag with context
driver init (ok: False)
skip because driver not ok
driver quit
-----------------------------------
>>>> abuse generator as context manager
driver init (ok: False)
skip because driver not ok
driver quit

Answer 2

此功能似乎是rejected。 Python开发人员通常更喜欢显式变体：

if need_more_workers():
    newcomm = get_new_comm(comm)
    # ...

您还可以使用高阶函数：

def filter_comm(comm, nworkitems, callback):
    if foo:
        callback(get_new_comm())

# ...

some_local_var = 5
def do_work_with_newcomm(newcomm):
    # we can access the local scope here

filter_comm(comm, nworkitems, do_work_with_newcomm)

Answer 3

这样的事情怎么样：

@filter_comm(comm, nworkitems)
def _(newcomm):  # Name is unimportant - we'll never reference this by name.
    ... do work with communicator newcomm...

您实现了filter_comm装饰器，以便对comm和nworkitems执行任何工作，然后根据这些结果决定是否执行它包裹的功能，传入newcomm。

它不如with那么优雅，但我认为这比其他提案更具可读性，更接近你想要的。如果你不喜欢这个名字，你可以将内部函数命名为_之外的其他东西，但是我选择它，因为它是Python中使用的正常名称，当语法需要一个你永远不会实际使用的名称时。

Python上下文管理器：有条件地执行主体？

3 个答案: