Question

我想使用带有2个四核处理器的Mac Pro并行化以下python循环。

result_list = []
for a in a_range:
    for b in b_range:
        for c in c_range:
            result = call_fortran_program(a, b, c)
            result_list.append(result)

在我的搜索中，我遇到了像Cython和GIL这样的术语，但我仍然不清楚如何继续。

Answer 1

TypeError                                 Traceback (most recent call last)
<ipython-input-287-376cac3b4f0d> in <module>()
      1 f  = open("hapP.pkl","wb")
----> 2 pickle.dump(hapPkl,f)
      3 f.close()

/usr/lib64/python2.7/copy_reg.pyc in _reduce_ex(self, proto)
     68     else:
     69         if base is self.__class__:
---> 70             raise TypeError, "can't pickle %s objects" % base.__name__
     71         state = base(self)
     72     args = (self.__class__, base, state)

TypeError: can't pickle function objects

Answer 2

尝试ProcessPoolExecutor

这可以通过创建多个进程来绕过GIL锁定。

网站上的一个例子：

import concurrent.futures
import math

PRIMES = [
    112272535095293,
    112582705942171,
    112272535095293,
    115280095190773,
    115797848077099,
    1099726899285419]

def is_prime(n):
    if n % 2 == 0:
        return False

    sqrt_n = int(math.floor(math.sqrt(n)))
    for i in range(3, sqrt_n + 1, 2):
        if n % i == 0:
            return False
    return True

def main():
    with concurrent.futures.ProcessPoolExecutor() as executor:
        for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
            print('%d is prime: %s' % (number, prime))

if __name__ == '__main__':
    main()

并行化python for-loop

2 个答案: