DAG Runs and Task State

Re-running DAGs

The history of a DAG’s runs can be accessed by clicking the DAG’s active runs from the main page or by using the Browse > DAG Runs screen:

Browse DAGRuns

Here we see the DAG Runs for example_dag:

Browse DAGRuns-example

Notice the execution dates for these DAGs. These parameters are passed to the DAG and can be used to execute logic within the DAG. This is called accessing the dag run’s current context and is explained in the Airflow documentation here. In general, we’d be interested to access the run context if our data loading flows or task scripts need to know which date ranges this particular run of the task should cover.

These DAG runs were automatically scheduled and triggered by Airflow because of what we defined in the DAG Factory YAML file definition.

Click any Run id to check out the tasks from that run:

Click RunId

This takes us to a view of that specific run of the DAG:

DAG Run View

Here we can see that all of our tasks ran successfully on this DAG run. By hovering on a task we can see it’s status, the task-run timestamp, and other high-level metrics. Click any task to bring up the task controls for that specific run of the task:

Task control

The screen above is the operator’s best friend. From this screen tasks can be marked as successful, failed or cleared. In addition, operators can specify how Airflow should handle upstream and downstream dependencies for the task.

Try to mark a few tasks as failed and see how Airflow responds. Play with upstream and downstream controls to see how Airflow schedules re-runs as a result.

These task-level controls are a key feature in Airflow and can be used to recover from code errors, system dependency outages, and dataset re-builds.