使用ftplib一次上传多个文件时上传速度慢

时间:2014-01-24 02:49:26

标签: python python-2.7 file-upload ftp ftplib

我正在尝试通过ftplib将30多个文件上传到网络服务器,但我上传时间过长会遇到问题。

如果我使用像CuteFTP,FTPPro这样的程序或直接通过我的网站上的cpanel上传,则上传所有30多个文件需要几秒钟,但是无论我的网络上传速度如何,每次都需要100秒才能使用此代码。我在不同国家的三个不同的连接上尝试过它。

8meg下来,8meg连接花了102秒 8meg下来,0.5meg连接需要106秒 8meg下来,1meg连接花了103秒 8meg下来,0.8meg连接花了101秒

理想情况下,我希望能够压缩所有文件,上传文件并在网络服务器上提取。

以下代码是一些测试代码,它可以从同一个文件夹中上传30多个文件,从另一个文件夹中上传1个单个文件。

import glob
import os
import ftplib
import re
from time import time

starttime = time()
print "Running..."

## Create session
ftp_session = ftplib.FTP('ftp.website.com','admin@website.com','password123','',60)

## ----------------------------------------------------------------------------------------------------

## Output location for single file
destdir = '/MAIN/'
ftp_session.cwd(destdir)

## Stats & Uploading for single file
ftp_file = open("output.html",'rb')
size = os.stat("output.html").st_size / 1024
result = ftp_session.storbinary('STOR output.html.tmp', ftp_file)
ftp_file.close()
data1 = re.findall('(\d*\.\d*)\sseconds', result)
data2 = re.findall('(\d*\.\d*)\sKbytes\sper\ssecond', result)

## Prints and renaming for single file
print "Uploaded output.html (" + str(size) + "KB) in " + str(data1).strip("['").strip("']") + " seconds at " + str(data2).strip("['").strip("']") + " kbps"
if 199 < int(result[0:3]) < 300: # Positive Completion
    ftp_session.rename("output.html.tmp", "index.php")
else:
    print "Upload of output.html resulted in code %s" % (result)

## ----------------------------------------------------------------------------------------------------

## Have to use the full directory for some reason when really they're in \CLASSES\
sourcedir = 'C:\POL\OUTPUT\CLASSES'
filelist = glob.glob(os.path.join(sourcedir, '*.html'))

## Output location for multiple files
destdir = '/MAIN/CLASSES/'
ftp_session.cwd(destdir)

## Cycle through each file
for fname in filelist:
    filename = fname.replace(sourcedir+'\\', "")
    ## Ignore Classes0.html as we dont want to upload this file
    if filename != "CLASSES0.html":
        ## Stats & Uploading for multiple files
        ftp_file = open(fname,'rb')
        size = os.stat(fname).st_size / 1024
        result = ftp_session.storbinary('STOR %s.tmp' % filename, ftp_file)
        ftp_file.close()
        data1 = re.findall('(\d*\.\d*)\sseconds', result)
        data2 = re.findall('(\d*\.\d*)\sKbytes\sper\ssecond', result)

        ## Prints and renaming for multiple files
        print "Uploaded " + filename + " (" + str(size) + "KB) in " + str(data1).strip("['").strip("']") + " seconds at " + str(data2).strip("['").strip("']") + " kbps"
        if 199 < int(result[0:3]) < 300: # Positive Completion
            ftp_session.rename("%s.tmp" % filename, filename)
        else:
            print "Upload of %s resulted in code %s" % (filename, result)
            break

## ----------------------------------------------------------------------------------------------------

## Print end timer
print """The whole routine took %.3f seconds""" % (time() - starttime)

上述代码的输出是:

Running...
Uploaded output.html (505KB) in 7.528 seconds at 67.13 kbps
Uploaded FULL.html (498KB) in 7.265 seconds at 68.55 kbps
Uploaded CLASSES1.html (14KB) in 0.697 seconds at 21.09 kbps
Uploaded CLASSES10.html (14KB) in 0.625 seconds at 23.93 kbps
Uploaded CLASSES11.html (14KB) in 0.788 seconds at 18.78 kbps
Uploaded CLASSES12.html (12KB) in 0.668 seconds at 18.87 kbps
Uploaded CLASSES13.html (12KB) in 0.631 seconds at 20.02 kbps
Uploaded CLASSES14.html (20KB) in 0.815 seconds at 25.21 kbps
Uploaded CLASSES15.html (13KB) in 0.633 seconds at 21.73 kbps
Uploaded CLASSES16.html (20KB) in 0.876 seconds at 23.25 kbps
Uploaded CLASSES17.html (11KB) in 0.675 seconds at 16.97 kbps
Uploaded CLASSES18.html (13KB) in 0.618 seconds at 21.97 kbps
Uploaded CLASSES19.html (20KB) in 0.830 seconds at 24.90 kbps
Uploaded CLASSES2.html (12KB) in 0.610 seconds at 20.62 kbps
Uploaded CLASSES20.html (11KB) in 0.611 seconds at 18.77 kbps
Uploaded CLASSES21.html (26KB) in 0.818 seconds at 32.05 kbps
Uploaded CLASSES22.html (10KB) in 0.646 seconds at 16.00 kbps
Uploaded CLASSES23.html (13KB) in 0.652 seconds at 21.04 kbps
Uploaded CLASSES24.html (22KB) in 0.839 seconds at 27.11 kbps
Uploaded CLASSES25.html (13KB) in 0.627 seconds at 21.85 kbps
Uploaded CLASSES26.html (20KB) in 0.830 seconds at 24.71 kbps
Uploaded CLASSES27.html (11KB) in 0.627 seconds at 18.96 kbps
Uploaded CLASSES28.html (12KB) in 0.612 seconds at 20.43 kbps
Uploaded CLASSES29.html (16KB) in 0.625 seconds at 25.76 kbps
Uploaded CLASSES3.html (11KB) in 0.632 seconds at 17.93 kbps
Uploaded CLASSES30.html (14KB) in 0.636 seconds at 22.93 kbps
Uploaded CLASSES31.html (16KB) in 0.629 seconds at 26.03 kbps
Uploaded CLASSES32.html (19KB) in 0.636 seconds at 30.93 kbps
Uploaded CLASSES33.html (20KB) in 0.869 seconds at 23.69 kbps
Uploaded CLASSES34.html (20KB) in 0.830 seconds at 24.63 kbps
Uploaded CLASSES35.html (20KB) in 0.822 seconds at 25.06 kbps
Uploaded CLASSES36.html (18KB) in 0.647 seconds at 28.32 kbps
Uploaded CLASSES37.html (15KB) in 0.639 seconds at 24.63 kbps
Uploaded CLASSES38.html (16KB) in 0.630 seconds at 25.50 kbps
Uploaded CLASSES39.html (18KB) in 0.614 seconds at 29.76 kbps
Uploaded CLASSES4.html (14KB) in 0.682 seconds at 21.65 kbps
Uploaded CLASSES40.html (17KB) in 0.715 seconds at 24.08 kbps
Uploaded CLASSES41.html (19KB) in 0.690 seconds at 28.17 kbps
Uploaded CLASSES42.html (17KB) in 0.686 seconds at 25.11 kbps
Uploaded CLASSES5.html (11KB) in 0.625 seconds at 18.22 kbps
Uploaded CLASSES6.html (14KB) in 0.649 seconds at 22.91 kbps
Uploaded CLASSES7.html (25KB) in 0.922 seconds at 28.14 kbps
Uploaded CLASSES8.html (16KB) in 0.676 seconds at 23.70 kbps
Uploaded CLASSES9.html (13KB) in 0.697 seconds at 19.46 kbps
The whole routine took 102.950 seconds

关于如何更快地实现这一目标的任何想法?

0 个答案:

没有答案
相关问题