Question

我正在编写这个编码文件中的数字，但是对于所需数字中的数字量，只用一个进程就需要几个小时。

from multiproccessing import Pool

print('What number would you like to start at?')
datastart = raw_input() #user input for datastart

x= int(datastart) #sets x to datastart

print('What number would you like to end at?')
dataend= raw_input() #user input to question at hand and sets var.

print('would you like to write file in normal or reverse? \n (n/r)')
detailer=raw_input() #sets forward or reverse writing mode

print('what password length would you like?')
passwordlen=int(raw_input())# sets length of file to be writen

print('Please wait while file is created!')
f=open(str(datastart)+'-'+str(dataend)+'-'+str(passwordlen)+'.txt','a')

def NumberCrunch(nc):

    while detailer == 'n':
        x=datastart
        while x <= int(dataend):
            num= str(x)
            while len(num) < passwordlen:
                num='0'+num
            if len(num) == passwordlen:
                f.write(num+'\n')
            x+=1


    while detailer == 'r':
        x=int(dataend)
        while x >= int(datastart):
            num= str(x)
            while len(num) < passwordlen:
                num='0'+num
            if len(num +'\n'):
                f.write(num +'\n')


            x-=1

if __name__ == '__main__':
    with Pool(process = 16) as p:
        p.map(NumberCrunch,chunksize =10)
        p.close()
        p.join()


f.close()
print('File has been created! Enjoy!')

我正在使用运行python 2.7 64bit的服务器，决斗xeon 2.20双四核，总共16核，32gb ram和3 tesla卡。修改池（process =（你的cpu核心数））以确保它在你的计算机上运行。

我正在尝试使用所有内核来处理从datastart到dataend的数据范围，在正向写入模式或dataend中以反向写入模式处理数据范围。我尝试过多处理队列。仍然只使用一个进程或提升系统退出。

有没有更好的方法来编写多处理程序，或者我错过了什么。

使用它会更好：

from multiprocessing import Pool

print('What number would you like to start at?')
datastart = raw_input() #user input for datastart

x= int(datastart) #sets x to datastart

print('What number would you like to end at?')
dataend= raw_input() #user input to question at hand and sets var.

print('would you like to write file in normal or reverse? \n (n/r)')
detailer=raw_input() #sets forward or reverse writing mode

print('what password length would you like?')
passwordlen=int(raw_input())# sets length of file to be writen

print('Please wait while file is created!')
f=open(str(datastart)+'-'+str(dataend)+'-'+str(passwordlen)+'.txt','a')

def NumberCrunch(nc):
    while detailer == 'f':
        for i in range(datastart,dataend,1):
            num = str(i)
            while len(num) < passwordlen:
                num = '0' + num
            if len(num)== passwordlen:
                f.write(num+'n')
    while detailer == 'r':
        for i in range(dataend,datastart,-1):
            num= str(i)
            while len(num) < passwordlen:
                num = '0'+num
            if len(num)== passwordlen:
                f.write(num+'\n')
if __name__=='__main____':
    with Pool(process = 16) as p:
        p.map(Numbercrunch,i,chuncksize=10)
        p.close()
        p.join()
f.close()
p.close()
print('File has been created')

任何想法或帮助都会有很大帮助。先谢谢你了！

Answer 1

当我使用多处理时，我使用了Process而不是Pool。像这样：

from multiprocessing import Process

def main_process(myargs):
    #do stuff here

#main_data defined elsewhere with items inside.
for k,myargs in main_data.items(): 
    p = Process(target=main_process, args=(myargs,))
    p.start()

需要帮助多处理这种编码

1 个答案: