Question

我有一个symfony 1.4任务，它使用splFileObject打开一个包含超过200000行的csv文件。

这是负责打开文件的代码：

//open the CSV file for read
      $this->file = new SplFileObject($this->filePath, 'r');
      $this->file->setFlags(SplFileObject::READ_CSV);
      $this->file->setCsvControl($this->delimiter);
      $this->file->seek($this->options['limit']);

      $this->limit = $this->options['limit'];

      /*       * read the file* */
      $i = $this->limit + 1;
      foreach ($this->file as $row)
      {
        if ($this->file->valid())
        {
          //initialize invoices and clients object
          $this->initInvoicesArray($row);
          $this->prepareInvoice();

          //clear the row
          unset($row);

          $this->key = $this->file->key();

          //add msisdn to msisdn array, and client to clients array
          self::$InvoicesArray[$this->msisdn] = $this->client;

          //get valid users from table, and save to payment_notifications
          $this->prepareAndSaveValidClients();

          if ($i % $this->options['count'] === 0)
          {
            //launch process again with new $key
            $this->logSection('Payment Notification', $i. 'Lines Processed, Memory Usage : ' . memory_get_usage());
            $this->log($this->fileSys->execute("./symfony mediapi:send-payment-notifications --application=" . $this->options['application']
                    . " --env=" . $this->options['env'] .
                    " --limit=" . $i .
                    ' --count=' . $this->options['count']));

          }
        }

        $i++;
      }

如您所见，一旦限制达到1000行，我就执行该过程。 limit选项用于确定当前进程必须从文件中的哪一行开始。

$this->prepareAndSaveValidClients();是一个用select函数验证数据，准备对象并将数据保存到数据库的函数...这就是prepareAndSaveValidClients（）：

  public function prepareAndSaveValidClients() {

    //get valid clients from database
    $this->query = Doctrine_Query::create()
        ->select('p.gender, p.email2, u.username, u.first_name, u.last_name, u.email_address, u.is_active, p.msisdn, p.user_id, p.city_id, p.street, p.zipcode, p.msisdn_status')
        ->from('sfGuardUser u')
        ->innerJoin('u.Profile as p ON u.id = p.user_id')
        ->whereIn('p.msisdn', $this->msisdn)
        ->whereIn('p.status', self::$AllowedStatus);

    //fetch valid clients result
    $this->results = $this->query->fetchArray();

    //unset the query to free memory
    $this->query->free(true);
    unset($this->query);

    //instanciat an object collection for payment_notifications
    $collection = new Doctrine_Collection("payment_notifications");

    //save valid clients in payment_notifications table
    if (!empty($this->results))
    {
      foreach ($this->results as $key => $client)
      {
        $invoice = self::$InvoicesArray[$client['Profile']['msisdn']];

        //prepare invoices
        $this->initInvoicesArray($invoice);
        $this->prepareInvoice();

        //prepare userprofile
        $this->prepareUserProfile($client);
        $this->prepareClient();

        $paymentNotifications = new paymentNotifications();
        $paymentNotifications->fromArray($this->client);

        $collection->add($paymentNotifications);

        //clear $client and $paymentNotifications
        unset($client);
        unset($paymentNotifications);
      }

      //save the collection
      $collection->save();

      //freeing the collection
      $collection->free(true);

      //clear memory
      $this->results = array();
      unset($collection);

      self::$MsisdnArray = array();
      self::$InvoicesArray = array();

    }
  }

问题是Memory_Usage通过不同的进程增加：

Process 1 : 1000 Lines Processed, Memory Usage : 40947336
Process 2 : 2000Lines Processed, Memory Usage : 69401208
Process 3 : 3000Lines Processed, Memory Usage : 98444272
Process 4: 4000Lines Processed, Memory Usage : 126308156
...

如何启动新进程，并释放以前的进程内存使用情况???

谢谢

即使再次执行该过程，内存使用量也会增加

0 个答案: