Question

When I run the following Thread, it spawns 5 forks. The first time I run it, it goes smooth, everything ok. But when it runs again, after the sleep(60) (while True loop), by the time the thread tries to fork() the processes, I get the following error:

OSError: [Errno 12] Cannot allocate memory

_td = threading.Thread(target=updateProxies,args=())
_td.start()

def updateProxies():
    while True:
        quota = 5
        children = []
        sons = 0
        for i in range(50):
            pid = os.fork()
            if pid:
                children.append(pid)
                sons+=1
                if sons >= quota:
                    os.wait()
                    sons-=1
            else:
                 {CHILD CODE EXECUTION} #database calls, and network requests
                 os._exit(0)
        for x in children:
            os.waitpid(x,0)
        sleep(60)

Things I debugged:

Already tracked if there are any defuncts left from the previous forks and there are none.
Already monitored the server memory, and there are plenty left (at least to spawn more forks again)

My bash script to monitor the memory is:

  #!/bin/bash
  while true
  do
  free | awk '/Mem/{printf("used: %.2f%"), $3/$2*100} /buffers\/cache/{printf(", buffers: %.2f%"), $4/($3+$4)*100} /Swap/{printf(", swap: %.2f%\n"), $3/$2*100}'
  sleep .5
  done

Which marked:

used: 16.41%, swap: 3.86% #before the script execution
used: 21.32%, swap: 3.86% #during the 5 forks execution
used: 19.86%, swap: 3.86% #after the 5 forks ended

This means that the 5 forks only increased around 5% percent of memory when they got spawned. So there is memory available. Then why I just can't spawn the forks again?

Cannot allocate memory when forking in Python, but there is memory

0 个答案: