主要

Question

阅读大量文章后，我仍然无法解决我的问题（他们不需要再次无限循环读取管道/队列），这样：

我有一个将命令分发到子流程的流程。每个进程都有一个无限循环，该循环等待传递的命令，然后执行一些操作并返回结果。

我尝试了Pipe（）和Queue（）。当我使用from multiprocessing.dummy import Process, Lock, Queue时，两者都有效。

当我切换到from multiprocessing import Process, Lock, Queue时，Queue版本最终在command, args = self.pipe.get()上陷入僵局，并且Pipe在res = agent['process'].master.recv()上引发EOFError

我没有设法在以下代码中重现Pipe错误，但是Queue的行为相同。

我正在Windows上使用Python 3.6.6。

主要

from testing.filesTesting.processCombination import ProcessCombination

if __name__=='__main__':
    pc = ProcessCombination()
    print('sending comm1')
    r = pc._send_command('comm1')
    print('result:', r)

ProcessCombination

from testing.filesTesting.subProcess import SubProcess

MULTIPROCESSING = True

if MULTIPROCESSING:
    from multiprocessing import Process, Lock, Pipe, Queue
else:
    from multiprocessing.dummy import Process, Lock, Pipe, Queue


class ProcessCombination():
    def __init__(self):
        # init processes
        self.processes = []
        for processID in range(0,3):    
            processArgs = self._get_ProcessArgs(processID)
            process = SubProcess(**processArgs)
            process.start()
            self.processes.append({'process':process,'id':processID})

    def _get_ProcessArgs(self, processID: int) -> dict:
        # prepare process communication
#        master, slave = Pipe()
        master = Queue()
        slave = Queue()
        # return as dict
        return {'processID': processID,
                'master': master,
                'slave': slave}


    def _send_command(self, name, args: dict=None, await_response: bool = True):
        # send command to all processes
        for process in self.processes:
#            process['process'].master.send((name, args))
            process['process'].master.put((name, args))
        # return response if required
        return [self._rcv(process) for process in self.processes] if await_response else []

    def _rcv(self, process):
#        res = process['process'].master.recv()
        res = process['process'].pipe.get()
        return res

SubProcess

MULTIPROCESSING = True

if MULTIPROCESSING:
    from multiprocessing import Process, Lock, Queue
else:
    from multiprocessing.dummy import Process, Lock, Queue

import time

class SubProcess(Process): 
    def __init__(self, processID: int, master, slave):       
        super(Process,self).__init__(daemon=True)
        self.master = master
        self.pipe = slave
        self._processID = processID


    def start(self) -> None:
        super(Process,self).start() 
#        self.pipe.close() # commented out when using queue
#        time consuming class initialization -> time.sleep
        time.sleep(2)


    def run(self):
        '''
        Wait for command from pipe, and then perform selected action
        '''
#        self.master.close() # commented out when using queue
        while True:
#            command, args = self.pipe.recv()
            command, args = self.master.get()

            if command == 'comm1':
                # do action
                # ...
#                self.pipe.send(True)
                self.pipe.put('comm1 result')
            # ...
            elif command == 'close':
                # closing ...
                break

python进程中的队列死锁

主要

ProcessCombination

SubProcess

0 个答案: