在进程之间进行多处理通信的最佳方式。

时间:2014-06-03 22:05:29

标签: python multiprocessing

我正在尝试编写一个函数,该函数将作为一个单独的进程运行,并将获取数据,直到我告诉它停止在主进程上。

以下是它的工作原理:

e = multiprocessing.Event()

def recordLeverPos(e, name): #e = multiprocessing.Event()
    posData = []
    while True:
        datBuffer = str(lever.readCounter()) + '\n'
        posData.append(datBuffer)
        if e.is_set() == True:
            with open('%s_leverPos.tsv' %(name), 'a') as file:
                for i in range(len(posData)):
                    posData[i] = posData[i]
                    file.write(posData[i])
            print 'Done Writing to leverPos file.'
            while e.is_set() == True:
                sleep(0.01)
            posData = []


p = mp.Process(target=recordLeverPos, args = (e, name))

def trialStart():
    global e

    #While trials is going on, some more code regarding the correct task that must be performed.
    e.set() #Indicate to the process that it's time to flush the buffer to the tsv file.


    #Depending on conditions, trial may continue to trial stop a function in which the patient must not do anything to the lever, but I still need to record the lever position.
    e.clear() #Indicate to process to get out of the while loop and start recording a new leverPos buffer.
    trialStop()

def trialStop():
    global e
    #Patient must not do anything here.... but if he/she does.
    if (lever.readCounter > threshold): #Patient  moved it.
        e.set() #Indicate to thread it's time to flush the buffer again.
        e.clear() #Indicate to thread you can start with a new buffer.

        #However the problem seems to be that when I call the e.set() in this function the process does not receive the event and does not save the Buffer... 

    #there's a timer here, if the timer passes I still want to record the position of the lever.

def main():
    p.start() #Begin adding leverPos to the buffer.
    trialStart()

所以我调用该函数并将其转换为与主函数分开运行的进程。

p = mp.Process(target=recordLeverPos, args = (e, name))
p.start()

函数 recordLeverPos 基本上通过将其添加到列表缓冲区来记录杠杆的位置,当试验结束时,我调用e.set(),因此调用函数的下一部分基本上是将列表缓冲区复制到aa tsv文件中。然后该过程将等到我致电e.clear()。 这就是问题所在:我似乎无法对事件进行一致的清除(值得注意的是,我创建了p和e全局变量,以便可以从运行某些阶段的其他函数访问它们)。每当我致电e.set()时,它似乎只适用于我呼叫e.set()的四个不同地方中的两个。 我的问题是,为什么会发生这种情况?有没有更好的方法能够以一种我可以全局称呼它的方式来做到这一点?

有没有更好的方法与流程进行通信? 我尝试自己搜索,但我无法弄清楚如何使用可选对象,说实话,事件函数/类似乎对我来说更直观,但我似乎无法像我一样使用它期待它发挥作用......

如果您想要更多代码,请告诉我,我只是试图超级简化我的代码,这样您就可以获得主要想法,因此您不会浪费时间来了解我的代码是如何工作的。

1 个答案:

答案 0 :(得分:2)

这里有一些允许数据流入的示例代码,然后在发出事件信号时,数据被写入文件。它是独立的,没有依赖性。

注意事项:

  • 在使用Event检查is_set()后,立即清除它。这使父进程再次set()

几乎就是这样,原始代码几乎完成了!

import multiprocessing, sys, time

record_ev = multiprocessing.Event()
name = 'beer'

def recordLeverPos(rec_ev, name): #e = multiprocessing.Event()
    posData = []
    lever = iter(xrange(999))
    while True:
        datBuffer = str(lever.next()) + '\n'
        posData.append(datBuffer)
        print 'posData',posData
        if rec_ev.is_set():
            rec_ev.clear()
            with open('%s_leverPos.tsv' % name, 'a') as dataf:
                for dat in posData:
                    dataf.write(dat)
            print 'wrote: {}'.format(posData)
            print 'Done Writing to leverPos file.'
            posData = []
        time.sleep(1)


record_proc = multiprocessing.Process(
    target=recordLeverPos, args = (record_ev, name)
)
record_proc.start()

time.sleep(2)
print 'triggering record'
record_ev.set()

time.sleep(2)
print 'triggering record #2'
record_ev.set()

time.sleep(2)
record_proc.terminate()
record_proc.join()
sys.exit(0)

输出

posData ['0\n']
posData ['0\n', '1\n']
triggering record
posData ['0\n', '1\n', '2\n']
wrote: ['0\n', '1\n', '2\n']
Done Writing to leverPos file.
posData ['3\n']
triggering record #2
posData ['3\n', '4\n']
wrote: ['3\n', '4\n']
Done Writing to leverPos file.
posData ['5\n']