确保子程序在退出Python程序时已经死了

时间:2008-11-26 10:21:21

标签: python subprocess kill zombie-process

有没有办法确保所有创建的子进程在Python程序的退出时间都死了?通过子进程,我指的是使用subprocess.Popen()创建的那些。

如果没有,我应该迭代所有发出的杀戮然后杀死-9?什么更干净?

14 个答案:

答案 0 :(得分:41)

您可以使用atexit进行此操作,并注册程序退出时要运行的所有清理任务。

atexit.register(func [,* args [,** kargs]])

在清理过程中,您还可以实现自己的等待,并在发生所需的超时时将其终止。

>>> import atexit
>>> import sys
>>> import time
>>> 
>>> 
>>>
>>> def cleanup():
...     timeout_sec = 5
...     for p in all_processes: # list of your processes
...         p_sec = 0
...         for second in range(timeout_sec):
...             if p.poll() == None:
...                 time.sleep(1)
...                 p_sec += 1
...         if p_sec >= timeout_sec:
...             p.kill() # supported from python 2.6
...     print 'cleaned up!'
...
>>>
>>> atexit.register(cleanup)
>>>
>>> sys.exit()
cleaned up!

注意 - 如果此进程(父进程)被终止,则不会运行已注册的函数。

python> = 2.6

不再需要以下Windows方法

这是一种在Windows中杀死进程的方法。您的Popen对象有一个pid属性,因此您可以通过 success = win_kill(p.pid)(已安装需要pywin32)来调用它:

    def win_kill(pid):
        '''kill a process by specified PID in windows'''
        import win32api
        import win32con

        hProc = None
        try:
            hProc = win32api.OpenProcess(win32con.PROCESS_TERMINATE, 0, pid)
            win32api.TerminateProcess(hProc, 0)
        except Exception:
            return False
        finally:
            if hProc != None:
                hProc.Close()

        return True

答案 1 :(得分:32)

在* nix上,也许使用进程组可以帮助你 - 你也可以捕获子进程产生的子进程。

if __name__ == "__main__":
  os.setpgrp() # create new process group, become its leader
  try:
    # some code
  finally:
    os.killpg(0, signal.SIGKILL) # kill all processes in my group

另一个考虑因素是升级信号:从SIGTERM(kill的默认信号)到SIGKILL(a.k.a kill -9)。在信号之间稍等片刻,让过程有机会在kill -9之前干净地退出。

答案 2 :(得分:14)

subprocess.Popen.wait()是确保他们死亡的唯一方法。实际上,POSIX OS要求您等待您的孩子。许多* nix将创建一个“僵尸”过程:父母没有等待的死孩子。

如果孩子写得合理,它就会终止。孩子们常常从PIPE上读到。关闭输入是一个很大的暗示,孩子应该关闭购物和退出。

如果孩子有虫子并且没有终止,你可能需要杀死它。你应该修复这个bug。

如果孩子是“永远服务”循环,并且不打算终止,你应该杀死它或提供一些输入或消息,这将迫使它终止。


编辑。

在标准操作系统中,您有os.kill( PID, 9 )。 Kill -9很苛刻,BTW。如果你可以用更有礼貌的SIGABRT(6?)或SIGTERM(15)来杀死它们。

在Windows操作系统中,您没有可用的os.kill。请查看此ActiveState Recipe以终止Windows中的进程。

我们有子进程是WSGI服务器。要终止它们,我们在特殊URL上进行GET;这会导致孩子清理并退出。

答案 3 :(得分:4)

警告:仅限Linux!您的孩子在其父母去世时可以收到信号。

首先安装python-prctl == 1.5.0然后更改您的父代码以启动子进程,如下所示

subprocess.Popen(["sleep", "100"], preexec_fn=lambda: prctl.set_pdeathsig(signal.SIGKILL))

这说的是:

  • 启动子进程:sleep 100
  • 在分叉之后和子进程的exec之前,子进程注册"给我发一个SIGKILL 当我的父母终止"。

答案 4 :(得分:3)

  

poll()

     

检查子进程是否已终止。   返回returncode属性。

答案 5 :(得分:3)

我需要一个小问题(清理子进程,但不退出Python程序本身),因为这里没有提到其他答案:

p=subprocess.Popen(your_command, preexec_fn=os.setsid)
os.killpg(os.getpgid(p.pid), 15)

setsid将在新会话中运行程序,从而为其及其子项分配新的进程组。因此调用os.killpg也不会降低你自己的python进程。

答案 6 :(得分:3)

orip的答案很有帮助,但其缺点是它会杀死你的进程并返回你父母的错误代码。我避免这样:

class CleanChildProcesses:
  def __enter__(self):
    os.setpgrp() # create new process group, become its leader
  def __exit__(self, type, value, traceback):
    try:
      os.killpg(0, signal.SIGINT) # kill all processes in my group
    except KeyboardInterrupt:
      # SIGINT is delievered to this process as well as the child processes.
      # Ignore it so that the existing exception, if any, is returned. This
      # leaves us with a clean exit code if there was no exception.
      pass

然后:

  with CleanChildProcesses():
    # Do your work here

当然你可以用try / except / finally来做到这一点,但你必须分别处理例外情况和非例外情况。

答案 7 :(得分:2)

  

有没有办法确保所有创建的子进程在Python程序的退出时间都死了?通过子进程,我指的是使用subprocess.Popen()创建的那些。

您可能会违反封装和测试,所有Popen进程都已通过

终止
subprocess._cleanup()
print subprocess._active == []
  

如果没有,我应该迭代所有发出的杀戮然后杀死-9?什么更干净?

你不能确保所有子过程都没有出去并杀死每一个幸存者。但如果你有这个问题,可能是因为你有更深层次的设计问题。

答案 8 :(得分:2)

Windows的解决方案可能是使用win32 job api,例如How do I automatically destroy child processes in Windows?

这是一个现有的python实现

https://gist.github.com/ubershmekel/119697afba2eaecc6330

答案 9 :(得分:2)

我实际上需要这样做,但它涉及运行远程命令。我们希望能够通过关闭与服务器的连接来停止进程。另外,例如,如果您在python repl中运行,如果您希望能够使用Ctrl-C退出,则可以选择作为前台运行。

import os, signal, time

class CleanChildProcesses:
    """
    with CleanChildProcesses():
        Do work here
    """
    def __init__(self, time_to_die=5, foreground=False):
        self.time_to_die = time_to_die  # how long to give children to die before SIGKILL
        self.foreground = foreground  # If user wants to receive Ctrl-C
        self.is_foreground = False
        self.SIGNALS = (signal.SIGHUP, signal.SIGTERM, signal.SIGABRT, signal.SIGALRM, signal.SIGPIPE)
        self.is_stopped = True  # only call stop once (catch signal xor exiting 'with')

    def _run_as_foreground(self):
        if not self.foreground:
            return False
        try:
            fd = os.open(os.ctermid(), os.O_RDWR)
        except OSError:
            # Happens if process not run from terminal (tty, pty)
            return False

        os.close(fd)
        return True

    def _signal_hdlr(self, sig, framte):
        self.__exit__(None, None, None)

    def start(self):
        self.is_stopped = False
        """
        When running out of remote shell, SIGHUP is only sent to the session
        leader normally, the remote shell, so we need to make sure we are sent 
        SIGHUP. This also allows us not to kill ourselves with SIGKILL.
        - A process group is called orphaned when the parent of every member is 
            either in the process group or outside the session. In particular, 
            the process group of the session leader is always orphaned.
        - If termination of a process causes a process group to become orphaned, 
            and some member is stopped, then all are sent first SIGHUP and then 
            SIGCONT.
        consider: prctl.set_pdeathsig(signal.SIGTERM)
        """
        self.childpid = os.fork()  # return 0 in the child branch, and the childpid in the parent branch
        if self.childpid == 0:
            try:
                os.setpgrp()  # create new process group, become its leader
                os.kill(os.getpid(), signal.SIGSTOP)  # child fork stops itself
            finally:
                os._exit(0)  # shut down without going to __exit__

        os.waitpid(self.childpid, os.WUNTRACED)  # wait until child stopped after it created the process group
        os.setpgid(0, self.childpid)  # join child's group

        if self._run_as_foreground():
            hdlr = signal.signal(signal.SIGTTOU, signal.SIG_IGN)  # ignore since would cause this process to stop
            self.controlling_terminal = os.open(os.ctermid(), os.O_RDWR)
            self.orig_fore_pg = os.tcgetpgrp(self.controlling_terminal)  # sends SIGTTOU to this process
            os.tcsetpgrp(self.controlling_terminal, self.childpid)
            signal.signal(signal.SIGTTOU, hdlr)
            self.is_foreground = True

        self.exit_signals = dict((s, signal.signal(s, self._signal_hdlr))
                                 for s in self.SIGNALS)                                     

    def stop(self):
        try:
            for s in self.SIGNALS:
                #don't get interrupted while cleaning everything up
                signal.signal(s, signal.SIG_IGN)

            self.is_stopped = True

            if self.is_foreground:
                os.tcsetpgrp(self.controlling_terminal, self.orig_fore_pg)
                os.close(self.controlling_terminal)
                self.is_foreground = False

            try:
                os.kill(self.childpid, signal.SIGCONT)
            except OSError:
                """
                can occur if process finished and one of:
                - was reaped by another process
                - if parent explicitly ignored SIGCHLD
                    signal.signal(signal.SIGCHLD, signal.SIG_IGN)
                - parent has the SA_NOCLDWAIT flag set 
                """
                pass

            os.setpgrp()  # leave the child's process group so I won't get signals
            try:
                os.killpg(self.childpid, signal.SIGINT)
                time.sleep(self.time_to_die)  # let processes end gracefully
                os.killpg(self.childpid, signal.SIGKILL)  # In case process gets stuck while dying
                os.waitpid(self.childpid, 0)  # reap Zombie child process
            except OSError as e:
                pass
        finally:
            for s, hdlr in self.exit_signals.iteritems():
                signal.signal(s, hdlr)  # reset default handlers

    def __enter__(self):
        if self.is_stopped:
            self.start()

    def __exit__(self, exit_type, value, traceback):
        if not self.is_stopped:
            self.stop()

感谢Malcolm Handley的初步设计。在linux上用python2.7完成。

答案 10 :(得分:2)

找出适用于linux的解决方案(无需安装prctl):

def _set_pdeathsig(sig=signal.SIGTERM):
    """help function to ensure once parent process exits, its childrent processes will automatically die
    """
    def callable():
        libc = ctypes.CDLL("libc.so.6")
        return libc.prctl(1, sig)
    return callable


subprocess.Popen(your_command, preexec_fn=_set_pdeathsig(signal.SIGTERM)) 

答案 11 :(得分:0)

这就是我为我的posix应用程序所做的:

当您的应用程序存在时,请调用此类的kill()方法: http://www.pixelbeat.org/libs/subProcess.py

此处使用示例: http://code.google.com/p/fslint/source/browse/trunk/fslint-gui#608

答案 12 :(得分:0)

答案 13 :(得分:0)

您可以尝试subalive,这是我为类似问题编写的软件包。它通过RPC使用定期的活动ping,当主服务器出于某种原因停止活动ping时,从属进程会自动终止。

https://github.com/waszil/subalive

主站示例:

from subalive import SubAliveMaster

# start subprocess with alive keeping
SubAliveMaster(<path to your slave script>)

# do your stuff
# ...

从属子过程的示例:

from subalive import SubAliveSlave

# start alive checking
SubAliveSlave()

# do your stuff
# ...
相关问题