可以在DAG中添加一个新的Operator,在作业完成后将DAGRun状态标记为成功或失败。以下是一个示例代码:
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from airflow.models import DagRun
from datetime import datetime, timedelta
def mark_dag_run_status(**kwargs):
dag_run_id = kwargs['dag_run'].run_id
dag_run = DagRun.find(dag_id=kwargs['dag_run'].dag_id, run_id=dag_run_id)
dag_run.state = 'success' if kwargs['ti'].state == 'success' else 'failed'
dag_run.end_date = datetime.now()
dag_run.external_trigger = False
dag_run.verify_integrity()
dag_run.log.info(f"Marking DagRun {dag_run_id} as {dag_run.state}")
dag_run.update()
default_args = {
'owner': 'airflow',
'start_date': datetime(2021, 1, 1),
'retries': 1,
'retry_delay': timedelta(minutes=1)
}
dag = DAG('example_dag', default_args=default_args, schedule_interval=timedelta(days=1))
t1 = PythonOperator(
task_id='task1',
python_callable=lambda: print('Hello World!'),
dag=dag
)
t2 = PythonOperator(
task_id='mark_dagrun_status',
python_callable=mark_dag_run_status,
provide_context=True,
dag=dag
)
t1 >> t2
在这个示例中,新的PythonOperator(t2)标记DAGRun状态为任务成功或失败,并设定end_date。使用provide_context=True向Python Operator提供上下文变量以获取DAG Run ID和Task Instance状态。最后,将t2
插入DAG
以启动标记过程。