Django + Celery + Requests + Eventlet

Question

I have a Django + Celery project. One of the Celery tasks does a lot of small HTTP requests using the requests library, while others do lots of talking to the database via the Django ORM. The HTTP-heavy task is already running in its own celery worker using its own Celery queue. I'd like to make the HTTP-heavy worker use eventlet while leaving the rest of the tasks to use the prefork execution pool. How do I do this?

The Celery docs seem to suggest that I gain magical concurrency powers by just running celery ... -P eventlet. However, this SO answer says that I need to use a patched version of the requests library. Which is correct? Additionally, if I have to explicitly patch requests, do I have to put this task in a separate module from the rest of the regular tasks so that these other tasks can continue using the regular version of requests?

Answer 1

TL，DR：使用已修补版本的请求库。无需启动单独的模块。

celery -P eventlet为您提供芹菜工作并发性。他们可能会或可能不会致电eventlet.monkey_patch()以使所有代码兼容。他们将来也可能会改变它。明确使用修补版本可以消除歧义，同时还提供有用的文档。

将并发requests与阻止分开是没有意义的。您的prefork池也可以使用并发版本。 Eventlet不会用坏事来毒害。