Project Setup

Project setup

Before we start, please complete the following checks:

This project uses AWS CDK library version 1.102.0, hence the same version or higher is required.

Now we can clone the Amazon MWAA Examples repository and navigate to project directory:

git clone git@github.com:aws-samples/amazon-mwaa-examples.git
cd amazon-mwaa-examples/usecases/mwaa-with-codeartifact

This is the project structure that you should be seeing:

.
├── infra/                      // AWS CDK infrastructure 
├── mwaa-ca-bucket-content/     // DAGs and requirements.txt
├── lambda/                     // Lambda handler
├── .env                        // Environment variables
├── Makefile                    // Make rules for automation

Environment variables

We will now set environment variables and create a Python virtual environment. Let’s start by setting environment variables in the .env file:

AWS_REGION=eu-west-1
BUCKET_NAME=REPLACE_WITH_YOUR_BUCKET_NAME
AIRFLOW_VERSION=2.0.2

You will set AWS Region for the deployment, name of an S3 bucket that will be created (must be unique), and the version of Apache Airflow (1.10.12 or 2.0.2).

Python virtual environment

We can now create Python virtual environment. Run the following make rule:

# from the root directory

$ make venv

This rule will create a virtual environment in infra/venv and install all the necessary dependencies.

Deploy the infrastructure

We are now ready to deploy! Please run the deploy rule to provision the infrastructure:

# from the root directory

$ make deploy

AWS CDK CLI will ask for your permissions to deploy specific IAM Roles and IAM Polices resources. When asked, please acknowledge with y and press Enter.

It can take a few minutes to provision a new MWAA environment, so when it’s done head over to next section!