Question

嗨我将数据存储在n个线程的块中。文件的大小是102kb，所以我试图锁定共享资源，即文件，然后当我写第一个块我释放锁，但是当它来自第二个线程的下一个chink，而不是文件继续从它离开的地方开始，如果它开始在顶部写下块...

所以102 kb文件对于两个线程变为51，每个线程具有51kb的块

这是一段代码。

for th in threads:
    th.join()

for th in threads:
    lock.acquire()
    with open(fileName, 'w+') as fh:
        fh.write(th.data)
    lock.release()

我甚至使用模式w+而不是附加覆盖..

更新

def main(url=None, splitBy=2):
    start_time = time.time()
    if not url:
        print "Please Enter some url to begin download."
        return

    fileName = url.split('/')[-1]
    sizeInBytes = requests.head(url, headers={'Accept-Encoding': 'identity'}).headers.get('content-length', None)
    # if os.path.exists(fileName):
    #   if int(sizeInBytes) == os.path.getsize(fileName):
    #       raise SystemExit("File already exists.")

    print "%s bytes to download." % sizeInBytes
    if not sizeInBytes:
        print "Size cannot be determined."
        return
    threads = []
    lock = threading.Lock()

    byteRanges = buildRange(int(sizeInBytes), splitBy)
    for idx in range(splitBy):
        bufTh = SplitBufferThread(url, byteRanges[idx])
        bufTh.daemon = True
        bufTh.start()
        threads.append(bufTh)
    print "--- %s seconds ---" % str(time.time() - start_time)


    for i, th in enumerate(threads):
        th.join()
        lock.acquire()


        with open(fileName, 'a') as fh:
            fh.write(th.data)
            if i == len(threads) - 1:
                fh.seek(0, 0)
                fh.flush()
        lock.release()

更新2

我已经完全删除了额外的线程列表，只需使用join()方法就可以了，但是线程如何等待一个块完成写入是使用with等待一个线程。要写入的数据，然后下一个数据开始附加 ??

 def main(url=None, splitBy=6):
    if not url:
        print "Please Enter some url to begin download."
        return

    fileName = url.split('/')[-1]
    sizeInBytes = requests.head(url, headers={'Accept-Encoding': 'identity'}).headers.get('content-length', None)
    if os.path.exists(fileName):
        if int(sizeInBytes) == os.path.getsize(fileName):
            ask = raw_input('[YES]')
            if not ask or ask.lower() in ['y', 'yes']:
                os.remove(fileName)
            else:
                raise SystemExit("File already exists.")

    start_time = time.time()
    print "%s bytes to download." % sizeInBytes
    if not sizeInBytes:
        print "Size cannot be determined."
        return

    byteRanges = buildRange(int(sizeInBytes), splitBy)
    for idx in range(splitBy):
        bufTh = SplitBufferThread(url, byteRanges[idx])
        bufTh.daemon = True
        bufTh.start()
        with open(fileName, 'a+') as fh:
            bufTh.join()
            fh.write(bufTh.data)

    print "--- %s seconds ---" % str(time.time() - start_time)


    print "Finished Writing file %s" % fileName

Answer 1

``w +''开放阅读和写作。如果没有，则创建该文件存在，否则被截断。流定位于文件的开头。

尝试使用“a +”

当我试图从线程写入文件的每个块时它被覆盖

1 个答案: