芹菜-工人有时只接任务

时间:2020-11-09 05:41:35

标签: python django asynchronous redis celery

我正在建立可以在线访问的潜在客户生成门户。请不要介意代码的冗长,我现在正在做大量调试。

我的芹菜工人不一致地执行分配给它的任务,我不确定为什么。

奇怪的是,有时可以100%完美地工作:终端中永远不会有任何显式错误。

我目前是DEBUG = TRUE,并且是REDIS的经纪人!

celery start worker终端命令和响应

celery -A mysite worker -l info --pool=solo
 -------------- celery@DESKTOP-OG8ENRQ v5.0.2 (singularity)
--- ***** -----
-- ******* ---- Windows-10-10.0.19041-SP0 2020-11-09 00:36:13
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         mysite:0x41ba490
- ** ---------- .> transport:   redis://localhost:6379//
- ** ---------- .> results:     redis://localhost:6379/
- *** --- * --- .> concurrency: 12 (solo)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery
[tasks]
  . mysite.celery.debug_task
  . submit

[2020-11-09 00:36:13,899: INFO/MainProcess] Connected to redis://localhost:6379//
[2020-11-09 00:36:14,939: WARNING/MainProcess] c:\users\coole\pycharmprojects\lead_django_retry\venv\lib\site-packages\celery\app\control.py:48: DuplicateNodenameWarning: Received multiple replies from node name: celery@DESKTOP-OG8ENRQ.
Please make sure you give each node a unique nodename using
the celery worker `-n` option.
  warnings.warn(DuplicateNodenameWarning(

[2020-11-09 00:36:14,939: INFO/MainProcess] mingle: all alone
[2020-11-09 00:36:14,947: INFO/MainProcess] celery@DESKTOP-OG8ENRQ ready.

views.py

class LeadInputView(FormView):
template_name = 'lead_main.html'
form_class = LeadInput

def form_valid(self, form):
    print("I'm at views")
    form.submit()
    print(form.submit)
    return HttpResponseRedirect('./success/')

tasks.py

@task(name="submit")
def start_task(city, category, email):
    print("I'm at tasks!")
    print(city, category, email)
    """sends an email when feedback form is filled successfully"""
    logger.info("Submitted")
    return start(city, category, email)

forms.py

class LeadInput(forms.Form):
    city = forms.CharField(max_length=50)
    category = forms.CharField(max_length=50)
    email = forms.EmailField()

    def submit(self):
        print("I'm at forms!")
        x = (start_task.delay(self.cleaned_data['city'], self.cleaned_data['category'], self.cleaned_data['email']))
        return x

celery.py

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')
app = Celery('mysite')

app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

settings.py

BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'UTC'

runserver终端将如下所示:

I'm at views
I'm at forms!
<bound method LeadInput.submit of <LeadInput bound=True, valid=True, fields=(city;category;email)>>

但是工人并没有说捡了什么,只是说“ celery @ DESKTOP-OG8ENRQ准备好了”。除了,什么时候起作用...出于某种原因?我很茫然!

2 个答案:

答案 0 :(得分:0)

默认情况下,如果您愿意在同一主机中使用多个工作程序,然后指定-n选项,那么celery将使用主机名作为工作程序名称。

celery -A mysite worker -l info --pool=solo -n worker2@%h

您的代码工作正常,但任务已传递给第一个工作程序,请参见 DuplicateNodenameWarning with no obvious reason #2938

答案 1 :(得分:0)

您好!事实证明,这是芹菜的一个错误(或者是redis?)……显然,很多Windows用户都遇到了这个错误。 https://github.com/celery/celery/issues/3759

结果是,答案是在启动worker时使-P独奏。我不知道为什么会这样...但是解决了!

感谢纳吉布(Naqib)的帮助!你把我放到右边的兔子洞里找到解决办法。