Vengala Vinay

Having 12+ Years of Experience in Software Development

Home » Zero-Downtime Blue-Green Deployment Pipelines for Python Applications on AWS

Zero-Downtime Blue-Green Deployment Pipelines for Python Applications on AWS

Establishing the Blue-Green Foundation with AWS Elastic Beanstalk

Blue-green deployment is a strategy that minimizes downtime and risk by running two identical production environments, “Blue” and “Green.” At any given time, one environment (e.g., Blue) is live and serving production traffic, while the other (Green) is idle. To deploy a new version, we deploy it to the idle environment (Green), test it thoroughly, and then switch traffic from Blue to Green. This post details how to implement this pattern for Python applications on AWS, leveraging Elastic Beanstalk (EB) for its inherent support of this deployment strategy.

Elastic Beanstalk simplifies blue-green deployments by managing the underlying infrastructure and providing built-in mechanisms for environment swapping. We’ll configure an EB application to use immutable or rolling updates with a specific strategy that facilitates this.

Configuring Elastic Beanstalk for Blue-Green Deployments

The core of our blue-green strategy within Elastic Beanstalk lies in its deployment policies. We’ll opt for a “Rolling with additional batch” or “Immutable” deployment policy. For true blue-green, “Immutable” is often preferred as it provisions entirely new instances for the updated environment, ensuring complete isolation. However, “Rolling with additional batch” can also be adapted by carefully managing the traffic shifting.

Let’s assume we have an existing Elastic Beanstalk application named my-python-app. We’ll create a new environment for our “Green” deployment. The key is to ensure both environments share the same configuration, including load balancer settings, security groups, and instance types.

Environment Creation and Configuration

We can create a new environment using the AWS CLI. This environment will initially be idle or serve as our “Green” environment.

First, ensure you have the EB CLI installed and configured.

To create a new environment (our “Green” environment) with an immutable deployment policy:

aws elasticbeanstalk create-environment \
    --application-name my-python-app \
    --environment-name my-python-app-green \
    --version-label initial-version \
    --solution-stack-name "64bit Amazon Linux 2 v3.x.x running Python 3.x" \
    --option-settings Namespace=aws:elasticbeanstalk:environment,OptionName=EnvironmentType,Value=LoadBalanced \
    --option-settings Namespace=aws:elasticbeanstalk:environment,OptionName=DeploymentPolicy,Value=Immutable \
    --option-settings Namespace=aws:autoscaling:launchconfiguration,OptionName=InstanceType,Value=t3.medium \
    --option-settings Namespace=aws:autoscaling:launchconfiguration,OptionName=IamInstanceProfile,Value=aws-elasticbeanstalk-ec2-role \
    --option-settings Namespace=aws:autoscaling:asg,OptionName=MinSize,Value=2 \
    --option-settings Namespace=aws:autoscaling:asg,OptionName=MaxSize,Value=4 \
    --option-settings Namespace=aws:elbv2:loadbalancer,OptionName=Subnets,Value="subnet-xxxxxxxxxxxxxxxxx,subnet-yyyyyyyyyyyyyyyyy" \
    --option-settings Namespace=aws:elbv2:loadbalancer,OptionName=SecurityGroups,Value="sg-zzzzzzzzzzzzzzzzz"

Replace placeholders like my-python-app, initial-version, 64bit Amazon Linux 2 v3.x.x running Python 3.x, instance types, IAM roles, subnet IDs, and security group IDs with your specific values. The crucial setting here is DeploymentPolicy=Immutable.

Automating Deployments with CI/CD

A robust CI/CD pipeline is essential for seamless blue-green deployments. We’ll outline a workflow using AWS CodePipeline, CodeBuild, and CodeDeploy (which Elastic Beanstalk leverages under the hood).

Pipeline Structure

Our pipeline will consist of the following stages:

  • Source: Triggered by commits to a Git repository (e.g., AWS CodeCommit, GitHub).
  • Build: Uses AWS CodeBuild to package the Python application, create a new application version artifact (e.g., a ZIP file), and upload it to an S3 bucket.
  • Deploy to Green: Deploys the new application version artifact to the “Green” Elastic Beanstalk environment. This stage will use CodeDeploy’s deployment capabilities, configured via EB.
  • Test: Executes automated integration and smoke tests against the “Green” environment.
  • Promote to Blue (Traffic Shift): If tests pass, this stage initiates the traffic switch from the “Blue” (current production) environment to the “Green” environment.
  • Terminate Blue (Optional): After a successful traffic shift and a grace period, the old “Blue” environment can be terminated.

CodeBuild Configuration for Packaging

The buildspec.yml file for CodeBuild will handle packaging your Python application. Ensure your application is structured to be deployable by Elastic Beanstalk (e.g., with a requirements.txt and an entry point script).

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.x # Specify your Python version
    commands:
      - echo "Installing dependencies..."
      - pip install --upgrade pip
      - pip install -r requirements.txt -t ./
  build:
    commands:
      - echo "Creating application artifact..."
      # Assuming your application code is in the root directory
      # and you want to package everything except build artifacts
      - zip -r my-python-app-v${CODEBUILD_BUILD_NUMBER}.zip .
  post_build:
    commands:
      - echo "Build completed on `date`"
      - echo "Uploading artifact to S3..."
      - aws s3 cp my-python-app-v${CODEBUILD_BUILD_NUMBER}.zip s3://${CODEBUILD_S3_BUCKET}/${CODEBUILD_PROJECT_NAME}/my-python-app-v${CODEBUILD_BUILD_NUMBER}.zip
      - echo "Creating Elastic Beanstalk application version..."
      - aws elasticbeanstalk create-application-version \
          --application-name my-python-app \
          --version-label "v${CODEBUILD_BUILD_NUMBER}" \
          --source-bundle S3Bucket="${CODEBUILD_S3_BUCKET}",S3Key="${CODEBUILD_PROJECT_NAME}/my-python-app-v${CODEBUILD_BUILD_NUMBER}.zip" \
          --description "Build ${CODEBUILD_BUILD_NUMBER}" \
          --auto-create-application

In your CodePipeline, the “Build” stage will use this buildspec.yml. The output artifact from CodeBuild will be the S3 object containing your application ZIP. This artifact will then be passed to the “Deploy to Green” stage.

Deploying to the “Green” Environment

The “Deploy to Green” stage in CodePipeline will use AWS CodeDeploy to deploy the new application version to the “Green” EB environment. Elastic Beanstalk handles the integration with CodeDeploy for its deployment strategies.

When you create or update an Elastic Beanstalk environment with a specific deployment policy (like Immutable), EB configures the necessary CodeDeploy application and deployment group. You then instruct EB to deploy a specific application version to that environment.

In CodePipeline, you can configure a “Deploy” action that targets your Elastic Beanstalk application and environment. The input artifact for this action will be the application version created in the “Build” stage.

Alternatively, you can use a custom action in CodePipeline that invokes the EB CLI or AWS CLI to perform the deployment:

# Example command to deploy a specific version to the 'my-python-app-green' environment
aws elasticbeanstalk update-environment \
    --environment-name my-python-app-green \
    --version-label "v${CODEBUILD_BUILD_NUMBER}"

This command tells Elastic Beanstalk to deploy the specified version to the “Green” environment. If the environment is configured with an “Immutable” strategy, EB will provision new instances, deploy the version, and then terminate the old ones (if any were present from a previous immutable deployment). If using “Rolling with additional batch,” EB will manage the instance replacement.

Implementing the Traffic Shift

The critical step in blue-green is the traffic switch. With Elastic Beanstalk, this is managed by changing the CNAME record of the load balancer. The “Blue” environment is the one currently serving production traffic (identified by its CNAME), and “Green” is the newly deployed, tested environment.

Manual Traffic Shifting (for testing/initial setup)

You can manually swap environments in the Elastic Beanstalk console. Navigate to your application, select the “Blue” environment, go to “Actions,” and choose “Swap environment URLs.” You’ll select the “Green” environment as the target. This operation atomically swaps the CNAMEs of the two environments.

Automated Traffic Shifting with CodePipeline

For automated blue-green, the “Promote to Blue” stage in CodePipeline needs to orchestrate this swap. This can be achieved using a custom action that executes a script or AWS CLI commands.

First, identify your “Blue” and “Green” environments. Let’s assume “Blue” is my-python-app-blue and “Green” is my-python-app-green. The “Blue” environment is the one currently serving production traffic.

The CodePipeline custom action would execute a script that performs the following:

#!/bin/bash

BLUE_ENV_NAME="my-python-app-blue"
GREEN_ENV_NAME="my-python-app-green"

echo "Initiating environment swap..."

# Check if the green environment is healthy and ready
# (This would involve more sophisticated health checks, e.g., querying load balancer health, running smoke tests)
echo "Performing health checks on ${GREEN_ENV_NAME}..."
# Add your health check commands here. For simplicity, we'll assume it's ready.

# Perform the environment swap
aws elasticbeanstalk swap-environment-cnames \
    --source-environment-name "${GREEN_ENV_NAME}" \
    --destination-environment-name "${BLUE_ENV_NAME}"

if [ $? -eq 0 ]; then
    echo "Environment swap successful. ${GREEN_ENV_NAME} is now serving production traffic."
    # Update the 'production' environment name for the next iteration
    # This might involve updating a parameter store or a configuration file
    # For simplicity, we'll assume the next iteration knows to target the new blue
else
    echo "Environment swap failed."
    exit 1
fi

# Optional: Update DNS records if you're not solely relying on EB CNAMEs
# For example, if you have a Route 53 record pointing to the EB CNAME.
# You would fetch the new CNAMEs and update Route 53.

This script assumes that “Blue” is the environment currently pointed to by your production DNS. The swap-environment-cnames command atomically swaps the CNAMEs. After this, the environment that was previously “Green” is now “Blue” (serving production traffic), and the old “Blue” is now “Green” (idle).

Automated Testing in the “Green” Environment

Before the traffic shift, it’s crucial to validate the “Green” environment. The “Test” stage in CodePipeline should execute a suite of automated tests. These tests should target the CNAME of the “Green” environment.

You can use tools like pytest with custom fixtures to fetch the EB environment URL and run tests against it.

# Example pytest fixture to get EB environment URL
import boto3
import os

def get_eb_environment_url(environment_name):
    eb_client = boto3.client('elasticbeanstalk')
    try:
        response = eb_client.describe_environments(EnvironmentNames=[environment_name])
        if response['Environments']:
            return response['Environments'][0]['CNAME']
        else:
            raise ValueError(f"Environment '{environment_name}' not found.")
    except Exception as e:
        print(f"Error describing environment {environment_name}: {e}")
        return None

# In your test file (e.g., test_app.py)
import pytest

@pytest.fixture(scope="session")
def green_env_url():
    # Assuming you have an environment variable set by CodePipeline
    # or you can fetch it dynamically if the pipeline passes it
    env_name = os.environ.get("GREEN_EB_ENVIRONMENT_NAME", "my-python-app-green")
    url = get_eb_environment_url(env_name)
    if not url:
        pytest.fail(f"Could not retrieve URL for green environment: {env_name}")
    return url

def test_homepage_loads(green_env_url):
    import requests
    response = requests.get(f"http://{green_env_url}/")
    assert response.status_code == 200
    assert "Welcome" in response.text # Example assertion

def test_api_endpoint(green_env_url):
    import requests
    response = requests.get(f"http://{green_env_url}/api/v1/status")
    assert response.status_code == 200
    assert response.json().get("status") == "ok"

The CodeBuild project for the “Test” stage would execute these tests. If any test fails, the pipeline halts, preventing the faulty deployment from reaching production.

Managing Environment States and Rollbacks

With blue-green, rollbacks are inherently simple: if the new “Green” environment proves problematic after the swap, you simply swap the CNAMEs back. The old “Blue” environment (which is now “Green”) is still intact and can resume serving traffic.

Instance Termination and Cleanup

After a successful traffic shift and a sufficient observation period (e.g., a few hours or a day), the old “Blue” environment (which is now “Green” and idle) can be terminated. This is often done manually or as a final step in the CodePipeline, potentially triggered by a manual approval gate.

To terminate an environment:

aws elasticbeanstalk terminate-environment \
    --environment-name my-python-app-old-blue

It’s crucial to have a clear naming convention for your environments to manage this process effectively. For instance, always having a “production” environment and a “staging” or “candidate” environment that gets promoted.

Advanced Considerations

Database Migrations

Database schema changes are a common challenge in zero-downtime deployments. For blue-green, you need a strategy that supports backward compatibility during the transition. This typically involves:

  • Forward and Backward Compatible Changes: Deploy changes that allow both the old and new versions of your application to coexist. This often means adding new columns/tables first, then deploying the new application version that uses them, and finally removing old columns/tables in a subsequent deployment.
  • Staged Rollouts: Use the “Rolling with additional batch” or “Immutable” deployment policies to gradually introduce the new version to a subset of instances. This allows you to monitor database interactions closely.
  • Database Migration Tools: Employ tools like Alembic (for Python/SQLAlchemy) or Django’s migrations, ensuring they are executed as part of the deployment process, often on the new instances before they receive traffic.

For Elastic Beanstalk, you can use .ebextensions to run migration scripts during the deployment lifecycle. However, care must be taken to ensure these scripts are idempotent and handle the blue-green transition gracefully.

Configuration Management

Ensure that environment-specific configurations (database credentials, API keys, feature flags) are managed securely and consistently across both environments. AWS Systems Manager Parameter Store or AWS Secrets Manager are excellent choices for this.

Your application should be configured to fetch these values at runtime. Elastic Beanstalk can inject these parameters as environment variables into your EC2 instances.

Load Balancer Health Checks

Configure your Elastic Load Balancer (ELB) health checks to accurately reflect the health of your application instances. These checks are critical for the load balancer to direct traffic only to healthy instances. During a blue-green deployment, the ELB will automatically start sending traffic to the new “Green” environment’s instances once they pass health checks.

Ensure your health check endpoint (e.g., /health) is robust and returns a 200 OK status code only when the application is fully ready to serve requests, including database connections and other dependencies.

Recent Posts

Top Categories

Our Products

Our Services