Airflow中是否有替代dag_dir_list_interval来从存储上传DAG到调度程序的选项？
Airflow中有两个选项来从存储区上传DAG到调度程序：
- 使用DAGs从存储区中运行。这可以通过将DAGs的文件路径直接指向存储区的路径来实现。这种方式需要运行Airflow worker。
from airflow import DAG from airflow.operators.bash_operator import BashOperator from datetime import datetime, timedelta
default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2021, 1, 1), 'email': ['alerts@airflow.com'], 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=1), }
dag = DAG( 'example_dag', default_args=default_args, description='Example DAG that runs from storage', schedule_interval=timedelta(days=1), catchup=False )
task_1 = BashOperator( task_id='task_1', bash_command='echo "hello world"', dag=dag )
- 将存储库添加到DAG的路径中，然后通过配置DAG的调度程序来传递这些存储库。这种方式不需要运行Airflow worker。
from airflow import DAG from airflow.operators.bash_operator import BashOperator from datetime import datetime, timedelta from airflow.models import Variable
default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2021, 1, 1), 'email': ['alerts@airflow.com'], 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=1), }
dag = DAG( 'example_dag', default_args=default_args, description='Example DAG that runs from storage', schedule_interval=timedelta(days=1), catchup=False )
storage_path = Variable.get('storage_path')
dag.dag_dir_list.append(storage_path)
task_1 = BashOperator( task_id='task_1', bash_command='echo "hello world"', dag=dag )