气流网络服务器会为dags提供cron错误,且计划间隔为无

时间:2018-07-25 13:56:54

标签: python airflow airflow-scheduler

我正在Linux AMI中使用LocalExecutor和PostgreSQL数据库运行Airflow 1.9.0。我想手动触发DAG,但是每当我创建将schedule_interval设置为None@once的DAG时,Web服务器树视图都会崩溃,并显示以下错误(我仅显示最后一个致电):

File "/usr/local/lib/python2.7/site-packages/croniter/croniter.py", line 467, in expand 
    raise CroniterBadCronError(cls.bad_length)
CroniterBadCronError: Exactly 5 or 6 columns has to be specified for iteratorexpression.

此外,当我手动触发DAG时,DAG运行开始,但任务本身从未计划。我环顾四周,但看来我是唯一出现此类错误的人。有人以前遇到过此错误并找到了解决方法吗?

触发问题的最小示例:

import datetime as dt
from airflow import DAG
from airflow.operators.bash_operator import BashOperator

default_args = {
    'owner': 'me'
}

bash_command = """
    echo "this is a test task"
"""

with DAG('schedule_test',
        default_args=default_args,
        start_date = dt.datetime(2018, 7, 24),
        schedule_interval='None',
        catchup=False
        ) as dag:

    first_task = BashOperator(task_id = "first_task", bash_command = bash_command)

1 个答案:

答案 0 :(得分:5)

尝试一下:

  • schedule_interval设置为None,而不使用'',或者干脆不指定schedule_interval in your DAG。它设置为None as a default。有关此信息,请参见:airflow docs-搜索BaseOperator
  • 在dag的底部为您的任务设置业务流程。

像这样:

import datetime
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.dummy_operator import DummyOperator

default_args = {
    'owner': 'me'
}

bash_command = """
    echo "this is a test task"
"""

with DAG('schedule_test',
        default_args=default_args,
        start_date = datetime(2018, 7, 24),
        schedule_interval=None,
        catchup=False
        ) as dag:

t1 = DummyOperator(
    task_id='extract_data',
    dag=dag
)

t2 = BashOperator(
    task_id = "first_task", 
    bash_command = bash_command
)

#####ORCHESTRATION#####
## It is saying that in order for t2 to run, t1 must be done.
t2.set_upstream(t1)
相关问题