对于异步操作,我应该用@ asyncio.coroutine装饰什么?

时间:2015-04-23 09:24:10

标签: python python-3.x asynchronous coroutine python-asyncio

我想以异步方式运行我的代码。我应该用@asyncio.coroutine装饰什么?我应该用yield from调用什么来进行异步操作?

就我而言,我有一些没有装饰器的示例代码。 (简单的聊天机器人看起来像IRC)

import asyncio


class ChatBot:
    def __init__(self, loop):
        conn = asyncio.open_connection(HOST, PORT, loop=loop)
        self.reader, self.writer = yield from conn

    def send(self, msg):
        self.writer.write(msg)

    def read(self):
        msg = yield from self.reader.readline()
        return msg

    def run(self):
        while True:
            msg = self.read()
            self.parse(msg)

    def parse(self, msg):
        if msg.startswith('PING'):
            self.some_work(msg)
        elif msg.startswith('ERROR'):
            self.some_error()
        else:
            self.server_log(msg)

    def some_work(self, msg):
        # some work. It can call asynchronous function like I/O or long operation. It can use self.send().

    def some_error(self, msg):
        # some work. It can call asynchronous function like I/O or long operation. It can use self.send().

    def server_log(self, msg):
        # some work. It can call asynchronous function like I/O or long operation. It can use self.send().


loop = asyncio.get_event_loop()
bot = ChatBot(loop)
loop.run_until_complete(???)
loop.close()

我认为???bot.run()ChatBot.run必须使用@asyncio.coroutine进行修饰。那么,其他方法呢?使用@asyncio.coroutine装饰器并使用yield fromasyncio.async调用方法时,我无法理解。 (我已经阅读了PEP-3156以了解asnycio。但我无法完全理解。)

3 个答案:

答案 0 :(得分:12)

何时使用@asyncio.coroutine装饰器

如果你有一个需要使用yield from来调用协同程序的函数,你应该用asyncio.coroutine来装饰它。还要注意协程通常(并不总是)"病毒"。一旦你将yield from添加到一个函数中它就变成了一个协同程序,另外任何调用的函数,通常(虽然不总是)协同程序也需要来一个协程。

何时使用asyncio.async

为什么协程不总是病毒?因为您实际上并不总是需要使用yield from来调用协程。如果要调用协同程序并等待它完成,则只需使用yield from。如果你只想在后台启动协程,你可以这样做:

asyncio.async(coroutine())

一旦控制权返回到事件循环,这将安排coroutine运行;在继续下一行之前,它不会等coroutine完成。普通函数可以使用它来安排协程运行,而不必自己成为协程。

您还可以使用此方法同时运行多个coroutines。所以,想象你有这两个协程:

@asyncio.coroutine
def coro1():
   yield from asyncio.sleep(1)
   print("coro1")

@asyncio.coroutine
def coro2():
   yield from asyncio.sleep(2)
   print("coro2")

如果你有这个:

@asyncio.coroutine
def main():
    yield from coro1()
    yield from coro2()
    yield from asyncio.sleep(5)

asyncio.get_event_loop().run_until_complete(main())

1秒后,将打印"coro1"。然后,经过两秒钟(总共三秒),将打印"coro2",五秒钟后程序将退出,使总运行时间为8秒。或者,如果您使用asyncio.async

@asyncio.coroutine
def main():
    asyncio.async(coro1())
    asyncio.async(coro2())
    yield from asyncio.sleep(5)


asyncio.get_event_loop().run_until_complete(main())

这将在一秒之后打印"coro1",一秒后"coro2",并且程序将在3秒后退出,总计5秒的运行时间。

这对您的代码有何影响?

遵循这些规则,您的代码需要如下所示:

import asyncio

class ChatBot:
    def __init__(self, reader, writer):
        # __init__ shouldn't be a coroutine, otherwise you won't be able
        # to instantiate ChatBot properly. So I've removed the code that
        # used yield from, and moved it outside of __init__.
        #conn = asyncio.open_connection(HOST, PORT, loop=loop)
        #self.reader, self.writer = yield from conn
        self.reader, self.writer = reader, writer

    def send(self, msg):
        # writer.write is not a coroutine, so you 
        # don't use 'yield from', and send itself doesn't 
        # need to be a coroutine.
        self.writer.write(msg)

    @asyncio.coroutine
    def read(self):
        msg = yield from self.reader.readline()
        return msg

    @asyncio.coroutine
    def run(self):
        while True:
            msg = yield from self.read()
            yield from self.parse(msg)

    @asyncio.coroutine
    def parse(self, msg):
        if msg.startswith('PING'):
            yield from self.some_work(msg)
        elif msg.startswith('ERROR'):
            yield from self.some_error()
        else:
            yield from self.server_log(msg)

    @asyncio.coroutine
    def some_work(self, msg):
        # some work. It can call asynchronous function like I/O or long operation. It can use self.send().

    @asyncio.coroutine
    def some_error(self, msg):
        # some work. It can call asynchronous function like I/O or long operation. It can use self.send().

    @asyncio.coroutine
    def server_log(self, msg):
        # some work. It can call asynchronous function like I/O or long operation. It can use self.send()

@asyncio.coroutine
def main(host, port):
    reader, writer = yield from asyncio.open_connection(HOST, PORT, loop=loop)
    bot = ChatBot(reader, writer)
    yield from bot.run()


loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()

要记住的另一件事 - 在函数前添加yield from并不会神奇地使该调用无阻塞。也没有添加@asyncio.coroutine装饰器。如果函数实际上直接或间接调用本机asyncio协程,它们使用非阻塞I / O并与asyncio事件循环集成,则它们只是非阻塞的。例如,您提到了进行REST API调用。为了使这些REST API调用不阻止事件循环,您需要使用aiohttp库或asyncio.open_connection。使用requestsurllib之类的内容会阻止循环,因为它们未与`asyncio集成。

答案 1 :(得分:2)

您应该装饰使用yield from的所有内容,因为装饰器asyncio.coroutine会将您的函数作为生成器,并在您屈服时执行所有回调/异步作业。

在您的情况下,run需要重写如下:

@asyncio.coroutine
def run(self):
    while True:
        msg = yield from self.read()
        yield from self.parse(msg)

然后,readparse也必须是协程。 你应该在使用它之前阅读异步工作的方法,它会对你有很大帮助。

答案 2 :(得分:-2)

@asyncio.coroutine
def read(self):
    msg = yield from self.reader.readline()
    return msg

@asyncio.coroutine
def run(loop):
    while True:
        msg = yield from read()
        yield from parse(msg)

loop = asyncio.get_event_loop()
loop.run_until_complete(run(loop))
loop.close()