Question

我有一个python对象 - 我希望在每个dicts中填充键值对但同时使用多个处理器并在python中使用多处理模块的字典列表。为此，我使用Manager模块存储该python对象。以下是代码：

from pylab import *
from numpy.random import *
import multiprocessing
import threading
import random

def tasks_start(id, global_lists):
    counter_lock = threading.Lock()
    with counter_lock:
        num = int(10*random.random())
        global_lists[num] = {'1':'Random'}

    print("Id: ", id)
    print(global_lists[0])


if __name__ == '__main__':
    numProcessors = 6
    pool = multiprocessing.Pool(numProcessors)

    global_list = multiprocessing.Manager().list(range(100))
    for idx in range(100):
        global_list[idx] = multiprocessing.Manager().dict()

    tasks = []
    for id in range(10):
        tasks.append((id, global_list))

    pool.starmap(tasks_start, tasks)
    pool.close()
    pool.join()

所以我在这里做的是创建一个存储为global_list的字典列表，然后使用python的tasks_start()模块调用starmap()方法10次（这样我以后就可以了）扩展到多个参数）以填充字典列表。作为一个简单的测试用例，我只是使用随机生成器每次在列表中随机选取一个字典并用一些值填充它。运行程序时，会出现以下错误：

    multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/home/cysis/inhibition_soum/motif_temporal_patterns/code_versions/2016/09/09_08/parallel_test/test_error_manager.py", line 14, in tasks_start
    print(global_lists[0])
  File "<string>", line 2, in __getitem__
  File "/usr/lib/python3.4/multiprocessing/managers.py", line 732, in _callmethod
    kind, result = conn.recv()
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 251, in recv
    return ForkingPickler.loads(buf.getbuffer())
  File "/usr/lib/python3.4/multiprocessing/managers.py", line 852, in RebuildProxy
    return func(token, serializer, incref=incref, **kwds)
  File "/usr/lib/python3.4/multiprocessing/managers.py", line 706, in __init__
    self._incref()
  File "/usr/lib/python3.4/multiprocessing/managers.py", line 756, in _incref
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 495, in Client
    c = SocketClient(address)
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 624, in SocketClient
    s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cysis/inhibition_soum/motif_temporal_patterns/code_versions/2016/09/09_08/parallel_test/test_error_manager.py", line 29, in <module>
    pool.starmap(tasks_start, tasks)
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 268, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 599, in get
    raise self._value
FileNotFoundError: [Errno 2] No such file or directory

在我看来，在执行最后一次print(global_lists[0)之前，管理员退出，因此无法找到global_lists[0]。任何人都可以对这类事情有所了解吗？

Python多处理过程在操作之前退出

0 个答案: