Question

我从列表中抓取url，然后将每个url发送到QNetworkRequest并重新接收HTML进行处理。然而，我有成千上万的请求要处理。所以我的应用程序一直挂着，直到我停止它立刻产生所有这些请求。

这是处理大量请求的que的礼仪方式吗？我尝试使用Queue of Urls，然后我将链接到每个QNetworkReply响应后触发的SLOT。

创建作业列表并添加到que

QQueue<QString> jobs;
for (int i = 1; i <= totalPages; i++){
    QString pageUrl = url + "&page=" + QString::number(i);
    jobs.enqueue(pageUrl);
}

qDebug() << "Total Jobs : " << jobs.count() << endl;
for (int i = 0; i < 5; i++){
    processQueue();
}

然后在getHtml函数内部

QNetworkAccessManager *manager = new QNetworkAccessManager(this);
    QNetworkRequest *getHtmlRequest = new QNetworkRequest(pageUrl);
    getHtmlRequest = new QNetworkRequest(url);
    getHtmlRequest->setRawHeader( "User-Agent", "Mozilla/5.0 (X11; U; Linux i686 (x86_64); "
                               "en-US; rv:1.9.0.1) Gecko/2008070206 Firefox/3.0.1" );
    getHtmlRequest->setRawHeader( "charset", "utf-8" );
    getHtmlRequest->setRawHeader( "Connection", "keep-alive" );
    connect(manager, SIGNAL(finished(QNetworkReply*)), this, SLOT(replyGetPageHtmlFinished(QNetworkReply*)));
    connect(manager, SIGNAL(finished(QNetworkReply*)), this, SLOT(processQueue()));
    manager->get(*getHtmlRequest);

触发

void checkNewArrivalWorker::processQueue(){
if (jobs.isEmpty()){
    qDebug() << "Jobs Completed" << endl;
    emit finished();
} else {
    QString pageUrl = jobs.dequeue();
    QString pageNumber = pageUrl.mid(pageUrl.indexOf("page=") + 5);
    getHtml(pageUrl, pageNumber);
}
}

处理多个QNetworkRequest的正确方法

0 个答案: