Daemon服务中的守护进程线程

时间:2017-09-22 12:18:21

标签: python python-2.7

摘要

我正在编写一个守护进程作为Linux服务运行。它与API(我也在写,但不是问题的一部分)进行通信,获取数据,对数据进行处理,然后将新的数据反馈给API。

当我发出script.py start命令时,它运行正常。守护进程启动脚本,守护程序线程启动并运行。

当我发出script.py stop命令,守护程序线程继续运行时,不会发生什么。主线程(由script.py start启动的线程)的停止并不会停止守护程序线程。

我仍然可以看到它们与ps ux一起运行。并且它们一直在运行直到手动杀死。

问题

如何让script.py stop杀死守护程序线程以及daemon模块启动的主线程?

详情

更深入:它是具有服务器/代理模型的网络设备轮询引擎。这是代理方。

有两个守护程序线程:

  • GetterThread
  • PutterThread

最多有15个类WorkerThread的工作线程可以启动到ping或SNMP轮询给定IP地址的清单。它们只是启动一个执行实际ping或轮询的子进程。

有三个数据队列:

  • ping_request_queue
  • poll_request_queue
  • result_queue

整个事情都包含在一个名为App的自定义类中,该类由daemon模块控制

GetterThread

class GetterThread(threading.Thread):
    """ This thread is responsible for fetching the ping and poll lists from the server and dropping them into the
    appropriate queue """
    server = None                   # type: Server
    ping_request_queue = None       # type: Queue.Queue
    poll_request_queue = None       # type: Queue.Queue

    def __init__(self, server, ping_request_queue, poll_request_queue):
        """
        Create the Thread

        :param Server server: The server to use
        :param Queue.Queue ping_request_queue:
        :param Queue.Queue poll_request_queue:
        """

        threading.Thread.__init__(self)
        self.ctl = ThreadController()
        self.server = server                                # type: Server
        self.ping_request_queue = ping_request_queue        # type: Queue.Queue
        self.poll_request_queue = poll_request_queue        # type: Queue.Queue

    def run(self):
        while self.ctl.run:
            if not self.server.online:
                sleep(30)
                self.server.check_in()
                continue
            sleep(1)
            ping_list, poll_list = self.server.get_lists()
            for r in ping_list:
                req = PingRequest.decode(r)
                self.ping_request_queue.put(req)
            for r in poll_list:
                req = PollRequest.decode(r)
                self.poll_request_queue.put(req)
        self.ctl.remove()

PutterThread

class PutterThread(threading.Thread):
    """ This thread is responsible for picking up results from the results_queue and sending them to the server """
    server = None           # type: Server
    q = None                # type: Queue.Queue

    def __init__(self, server, result_queue):
        """
        Create a thread to put the results on the server

        :param Queue.Queue result_queue:
        """

        threading.Thread.__init__(self)
        self.ctl = ThreadController()
        self.server = server            # type: Server
        self.q = result_queue

    def run(self):
        while self.ctl.run:
            if not self.server.online:
                sleep(30)
                self.server.check_in()
                continue
            sleep(1)
            if self.q.not_empty:
                result = self.q.get()
                if isinstance(result, Request):
                    if result.stage == Request.PINGED:
                        """ Send the ping results """
                        f = self.server.send_ping_results
                        lmsg = 'Sent ping result for request {}'.format(result.uuid)
                    elif result.stage == Request.POLLED:
                        f = self.server.send_poll_results
                        lmsg = 'Sent poll result for request {}'.format(result.uuid)
                    else:
                        continue
                    f(result)
                    logging.debug(lmsg)
                else:
                    logging.info('Bad request in queue: {!r}'.format(result))
        self.ctl.remove()

getter和putter线程实例都设置为守护进程。

我将整个脚本作为守护程序运行:

class App:
    def __init__(self):
        self.pidfile_path = "/var/run/project/poller.agent.pid"
        self.logfile_path = "/var/log/project/poller.agent.log"
        self.handler = logging.FileHandler(self.logfile_path)

    def run(self):

        result_queue = Queue.Queue()
        ping_request_queue = Queue.Queue()
        poll_request_queue = Queue.Queue()

        getter_thread = GetterThread(self.server, ping_request_queue, poll_request_queue)
        getter_thread.setName('GetterThread')
        getter_thread.setDaemon(True)

        putter_thread = PutterThread(self.server, results_queue)
        putter_thread.setName('PutterThread')
        putter_thread.setDaemon(True)

        worker_threads = []

        max_threads = {
            'ping': 5,
            'poll': 10,
        }

        thread_defs = [
            ('ping', ping_request_queue, result_queue),
            ('poll', poll_request_queue, result_queue)
        ]

        while True:
            if ping_request_queue.not_empty or poll_request_queue.not_empty:
                for thread_def in thread_defs:
                    thread_type, input_queue, output_queue = thread_def
                    thread_count = min(input_queue.qsize(), max_threads.get(thread_type))
                    for x in range(thread_count):
                        t = WorkerThread(*thread_def)
                        t.setName('WorkerThread-{}-{:02n}'.format(thread_type, x)
                        worker_threads.append(t)
                        t.start()
            sleep(1)

if __name__ == "__main__":
    app = App()
    daemon_runner = runner.DaemonRunner(app)
    daemon_runner.daemon_context.files_preserve = [app.handler.stream]
    daemon_runner.do_action()

0 个答案:

没有答案