在Airflow中,默认情况下变量名应该写成全大写字母,以便与其他任务区分开来。同样,在使用BigQueryToGCSOperator时,也需要使用Airflow中默认的变量名称规则。
示例代码:
from airflow import DAG from airflow.operators.bash_operator import BashOperator from datetime import datetime
default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2021, 5, 4), 'email_on_failure': False, 'email_on_retry': False, 'retries': 1 }
dag = DAG( 'example_dag', default_args=default_args, description='A simple tutorial DAG', schedule_interval='@once', )
start_task = BashOperator( task_id='start_task', bash_command='echo "Start task"', dag=dag, )
end_task = BigQueryToGCSOperator( task_id='end_task', bucket='my-bucket', source_project_dataset_table='my-project:my-dataset.my-table', destination_cloud_storage_uris=['gs://my-bucket/my-file.json'], dag=dag, )
start_task >> end_task