Python队列和线程问题

时间:2013-12-04 20:57:04

标签: python multithreading conditional-statements deque

寻找一些关于解决看起来像一个简单问题但却烧掉了我生命中许多小时的指导。我把它简化为一个测试用例:我正在尝试将项添加到主线程中的队列中,并从不同的线程中读取它们,并且由于某种原因,这根本不起作用。测试类如下:

from collections import deque
import Queue
import threading
from threading import Thread
from threading import Lock
from threading import Condition
from pprint import pformat
from time import sleep

class UpdateIndex(Thread):
        def __init__(self):
                Thread.__init__(self)

                # A flag to notify the thread that it should finish up and exit
                self.kill_received = False

                # make it a daemon so it doesn't block
                self.daemon = True

                # queue
                self.indexqueue = deque()
                self.available = Lock()

                self.__class__.processor = self


        def add(self, document):
                with self.available:
                        self.indexqueue.append(document)
                        print "Added: "+str(len(self.indexqueue)) + " to "+hex(id(self.indexqueue))

        @classmethod
        def update(cls, document):
                cls.processor.add(document)

        def run(self):
                sleeptime = 1
                while self.kill_received == False:
                        sleeptime = 1.0
                        with self.available:
                                print "Checking: "+str(len(self.indexqueue)) + " in "+hex(id(self.indexqueue))
                                while len(self.indexqueue) > 0:
                                        sleeptime = 0
                                        source = self.indexqueue.popleft()

                                        print "===>Updating "+pformat(source)

                        if sleeptime > 0:
                                sleep(sleeptime)


if __name__ == "__main__":
        indexthread = UpdateIndex()
        indexthread.start()

        sleep(2)

        UpdateIndex.update({ "id": "one" })
        sleep(5)
        UpdateIndex.update({ "id": "two" })
        sleep(5)
        UpdateIndex.update({ "id": "three" })

我看到的输出是:

Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Added: 1 to 0xb71118ecL
Checking: 1 in 0xb71118ecL
===>Updating {'id': 'one'}
Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Added: 1 to 0xb71118ecL
Checking: 1 in 0xb71118ecL
===>Updating {'id': 'two'}
Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Checking: 0 in 0xb71118ecL
Added: 1 to 0xb71118ecL

由于某种原因,第二个线程永远不会看到对双端队列的添加。我做错了什么?

编辑:同样,对我来说超级混乱的事实是双端队列大小总是一个。似乎有些东西正在排空它,但我不能为我的生活找出什么。我打印出deque的id只是为了确保我们处理相同的对象,它似乎与主线程和更新线程相同。

编辑:刚才意识到上面的测试用例似乎运行正常,但是我的Django项目中添加的完全相同的类给出了:

Checking: 0 in 0x982e614
Checking: 0 in 0x982e614
Checking: 0 in 0x982e614
Added: 1 to 0x982e614
Checking: 0 in 0x982e614
Checking: 0 in 0x982e614

现在我更加困惑了。

0 个答案:

没有答案