Question

如何使用Python ftplib限制FTP下载？例如，将速度上限设为20Mb / s？

我使用以下代码使用Python ftplib下载文件：

from ftplib import FTP
import os

download_list = 'testlist.txt' # inital list of directories to be downloaded
path_list = [] # initalize a list of all the pathes from download_list
local_folder = 'testStorage' #where files are going to be downloaded to
downloaded_list = 'completedownload.txt' # list of completed downloads
error_list = 'incomplete_downloads.txt' # list of paths that are incomplete


ftp=FTP("ftp.address.com")
ftp.login("user_name","password") #login to FTP account
print "Successfully logged in"

# make a list of files to download from a file
with open(download_list, 'r') as f:
    content = f.readlines()
    path_list = [x.strip() for x in content]

for path in path_list:
    path = path.replace("*","") # strips the * found in the source file
    print '\nChanging directory to ' + path + ':\n'
    #ftp.cwd('/AAA/BBB/CCC/logic-1/')    #the format to change into path note the * is omitted
    #if ftp.cwd(path) == True:


    try:    # tries the path in the file
        ftp.cwd(path)
        #ftp.retrlines('LIST')
        filenames = ftp.nlst()
        for filename in filenames:
            local_directory = local_folder+path # create the local path ie : testStorage/AAA/BBB/CCC/logic-1/
            local_filename = os.path.join(local_directory,filename) #

            if os.path.exists(local_filename) == False: # checks if file already exists
                if not os.path.exists(local_directory): # mimic the remote path locally
                    os.makedirs(local_directory)

                file = open(local_filename,'wb')
                ftp.retrbinary('RETR '+ filename, file.write)
                print filename
                file.close()

            elif os.path.exists(local_filename) == True: # skip the file if it exits
                print 'File ' +filename + ' already exists, skipping this file'

    except: #if path in text file does not exist write to error_list.txt

        print 'Path ' + path + ' does not exist writing path to error_list.txt'
        with open(error_list, 'a') as f2:
            f2.write(path+'\n')
        continue

print "all done closing connection"
ftp.close()  #CLOSE THE FTP CONNECTION

Answer 1

要限制下载，只需根据需要实现file.write和time.sleep的功能。将该函数作为ftp.retrbinary传递给callback（而不是直接代替file.write）。

这个伪代码（我不做Python）应该给你一些想法：

total_length = 0
start_time = time.time()

def write_and_sleep(buf):
    global file
    global total_length
    global start_time
    file.write(buf)
    total_length += sys.getsizeof(buf)
    while (total_length / (time.time() - start_time)) > 100000000:
        time.sleep(0.1)

ftp.retrbinary('RETR '+ filename, write_and_sleep)

减少maxblocksize（ftp.retrbinary的第3个参数）可能有助于实现更顺畅的“下载曲线”。

使用Python ftplib限制FTP下载

1 个答案: