Question

因此，使用多进程模块可以很容易地并行运行带有不同参数的函数，如下所示：

from multiprocessing import Pool

def f(x):
    return x**2


p = Pool(2)
print(p.map(f, [1, 2]))

但是我有兴趣在相同的参数上执行函数列表。假设我具有以下两个功能：

def f(x):
    return x**2


def g(x):
    return x**3 + 2

如何为相同的参数（例如x = 1）并行执行它们？

Answer 1

您可以使用Pool.apply_async()。您以（函数，argument_tuple）的形式捆绑任务，并将每个任务提供给apply_async()。

from multiprocessing import Pool
from itertools import repeat


def f(x):
    for _ in range(int(50e6)): # dummy computation
        pass
    return x ** 2


def g(x):
    for _ in range(int(50e6)): # dummy computation
        pass
    return x ** 3


def parallelize(n_workers, functions, arguments):
    # if you need this multiple times, instantiate the pool outside and
    # pass it in as dependency to spare recreation all over again
    with Pool(n_workers) as pool:
        tasks = zip(functions, repeat(arguments))
        futures = [pool.apply_async(*t) for t in tasks]
        results = [fut.get() for fut in futures]
    return results


if __name__ == '__main__':

    N_WORKERS = 2

    functions = f, g
    results = parallelize(N_WORKERS, functions, arguments=(10,))
    print(results)

示例输出：

[100, 1000]

Process finished with exit code 0

Answer 2

您可以返回一个元组。使用轻量级模块joblib，可以非常轻松且非常紧凑地完成此操作。我推荐joblib，因为它是轻量级的

public MyUDPClient(String destinationAddr, int port) throws SocketException, UnknownHostException {
  super();
  this.serverAddress = InetAddress.getByName(destinationAddr);
  this.port = port;
  udpSocket = new DatagramSocket();
  //udpSocket = new DatagramSocket(this.port);
  scanner = new Scanner(System.in);
}

使用多处理。您已经在问题中使用了池

from joblib import Parallel, delayed
import multiprocessing
import timeit


# Implementation 1
def f(x):    
    return x**2, x**3 + 2


#Implementation 2 for a more sophisticated second or more functions
def g(x):
    return  x**3 + 2


def f(x):    
    return x**2, g(x)


if __name__ == "__main__":
  inputs = [i for i in range(32)]
  num_cores = multiprocessing.cpu_count()
  t1 = timeit.Timer()
  result = Parallel(n_jobs=num_cores)(delayed(f)(i) for i in inputs)
  print(t1.timeit(1))

示例输出：

from multiprocessing import Pool, cpu_count
import timeit


def g(x):
    return x**3 + 2


def f(x):    
    return x**2, g(x)

if __name__ == "__main__":
  inputs = [i for i in range(32)]
  num_cores = cpu_count()
  p = Pool(num_cores)
  t1 = timeit.Timer()
  result = p.map(f, inputs)
  print(t1.timeit(1))
  print(result)

对于：输入= range（2000），花费了时间： 1.100000190490391e-06

并行执行功能列表

2 个答案: