Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » Zero-Downtime Blue-Green Deployment Pipelines for C Applications on AWS

Zero-Downtime Blue-Green Deployment Pipelines for C Applications on AWS

Understanding Blue-Green Deployments for C Applications

Blue-green deployment is a strategy that minimizes downtime and risk by running two identical production environments, referred to as “blue” and “green.” At any given time, only one environment is live, serving all production traffic. The other environment is idle, used for deployment and testing. Once the new version is deployed and validated on the idle environment, traffic is switched over. This allows for instant rollback if issues arise.

For C applications, which often have complex build processes and direct system interactions, implementing blue-green deployments requires careful orchestration, especially on cloud platforms like AWS. We’ll focus on a common AWS setup using EC2 instances, an Elastic Load Balancer (ELB), and potentially Auto Scaling Groups (ASGs) for managing the environments.

AWS Infrastructure for Blue-Green

Our setup will involve two distinct sets of EC2 instances, each configured identically. One set will be the “blue” environment, and the other will be the “green” environment. An Elastic Load Balancer will sit in front of these instances, directing traffic to only one environment at a time. We’ll leverage AWS Systems Manager (SSM) for remote execution and potentially CodeDeploy for automating the deployment process.

Consider the following AWS resources:

  • EC2 Instances: Two identical sets, one for blue, one for green. These will run your compiled C application.
  • Elastic Load Balancer (ELB): A single ELB will distribute traffic. We’ll dynamically change its target group to point to either the blue or green instances.
  • Auto Scaling Groups (ASGs): While not strictly mandatory for the blue/green concept itself, ASGs are crucial for maintaining the desired number of instances in each environment and for handling scaling needs. We’ll manage two separate ASGs, one for blue and one for green.
  • AWS Systems Manager (SSM): For executing commands on instances (e.g., starting/stopping services, running health checks) without SSH.
  • IAM Roles: Necessary for EC2 instances and deployment tools to interact with AWS services.
  • S3 Bucket: To store compiled binaries and deployment scripts.

Setting Up the Environments

The core idea is to have two identical deployments. Let’s assume you have a C application that compiles into an executable (e.g., my_c_app) and runs as a service. We’ll use two Auto Scaling Groups, each associated with a different set of EC2 instances and a specific tag to identify them as “blue” or “green.”

EC2 Instance Configuration and Tagging

When launching EC2 instances for your blue and green environments, it’s critical to tag them appropriately. This tagging will be used by the ELB and potentially by deployment scripts to differentiate between the two environments.

Example EC2 launch template configuration (conceptual, via AWS CLI or Console):

For the “Blue” environment ASG:

{
    "LaunchTemplateData": {
        "ImageId": "ami-xxxxxxxxxxxxxxxxx",
        "InstanceType": "t3.medium",
        "KeyName": "my-key-pair",
        "SecurityGroupIds": ["sg-xxxxxxxxxxxxxxxxx"],
        "UserData": "#!/bin/bash\n# User data script to install dependencies and deploy initial version\napt-get update -y\napt-get install -y build-essential\n# ... download and install your C app ...\nsystemctl enable my_c_app\nsystemctl start my_c_app",
        "TagSpecifications": [
            {
                "ResourceType": "instance",
                "Tags": [
                    {"Key": "Environment", "Value": "Production"},
                    {"Key": "DeploymentGroup", "Value": "Blue"}
                ]
            },
            {
                "ResourceType": "volume",
                "Tags": [
                    {"Key": "Environment", "Value": "Production"},
                    {"Key": "DeploymentGroup", "Value": "Blue"}
                ]
            }
        ]
    }
}

For the “Green” environment ASG, the only difference in the TagSpecifications would be {"Key": "DeploymentGroup", "Value": "Green"}.

Elastic Load Balancer (ELB) Configuration

We’ll use an Application Load Balancer (ALB) or Network Load Balancer (NLB). For simplicity, let’s assume an ALB. The key is to configure two Target Groups, one for the blue instances and one for the green instances. The ELB’s listener rules will then be updated to switch traffic between these target groups.

Target Group 1 (Blue):

aws elbv2 create-target-group \
    --name my-app-blue-tg \
    --protocol HTTP \
    --port 80 \
    --vpc-id vpc-xxxxxxxxxxxxxxxxx \
    --target-type instance \
    --health-check-protocol HTTP \
    --health-check-path /health \
    --health-check-interval-seconds 30 \
    --healthy-threshold-count 3 \
    --unhealthy-threshold-count 3

Target Group 2 (Green):

aws elbv2 create-target-group \
    --name my-app-green-tg \
    --protocol HTTP \
    --port 80 \
    --vpc-id vpc-xxxxxxxxxxxxxxxxx \
    --target-type instance \
    --health-check-protocol HTTP \
    --health-check-path /health \
    --health-check-interval-seconds 30 \
    --healthy-threshold-count 3 \
    --unhealthy-threshold-count 3

After creating the ASGs, you’ll need to register the instances launched by them with their respective target groups. This can be done automatically by configuring the ASG to associate with the target group.

Automating Deployments with AWS Systems Manager and CodeDeploy

The core of the blue-green deployment lies in automating the process of deploying the new version to the idle environment and then switching traffic. AWS Systems Manager (SSM) and AWS CodeDeploy are excellent tools for this.

Using AWS Systems Manager (SSM) Run Command

SSM Run Command allows you to execute scripts on your EC2 instances remotely. This is ideal for deploying new binaries, restarting services, and performing health checks.

First, ensure your EC2 instances have the SSM Agent installed and running, and that they have an IAM role with permissions to use SSM (e.g., AmazonSSMManagedInstanceCore policy).

Deployment Script Example (to be stored in S3 and executed via SSM):

#!/bin/bash

# Variables
APP_NAME="my_c_app"
NEW_BINARY_URL="s3://your-deployment-bucket/binaries/my_c_app-v1.2.tar.gz"
INSTALL_DIR="/opt/${APP_NAME}"
SERVICE_FILE="/etc/systemd/system/${APP_NAME}.service"

echo "Starting deployment of ${APP_NAME}..."

# 1. Download the new binary
echo "Downloading new binary from ${NEW_BINARY_URL}..."
aws s3 cp "${NEW_BINARY_URL}" /tmp/new_binary.tar.gz

# 2. Extract and install
echo "Extracting and installing to ${INSTALL_DIR}..."
mkdir -p "${INSTALL_DIR}"
tar -xzf /tmp/new_binary.tar.gz -C "${INSTALL_DIR}"
rm /tmp/new_binary.tar.gz

# 3. Update systemd service if necessary (or just restart)
# For simplicity, we'll assume the service file remains the same and just restart the service.
# In a real-world scenario, you might need to update the ExecStart path if the binary name changes.

# 4. Restart the application service
echo "Restarting ${APP_NAME} service..."
systemctl restart "${APP_NAME}"

# 5. Perform a health check (optional but recommended)
echo "Performing health check..."
HEALTH_CHECK_URL="http://localhost:8080/health" # Assuming your app listens on 8080 and has a /health endpoint
if curl -s --fail "${HEALTH_CHECK_URL}"; then
    echo "Health check passed."
    exit 0
else
    echo "Health check failed!"
    # Attempt to revert or signal failure
    systemctl restart "${APP_NAME}" # Attempt to restart again
    exit 1
fi

Executing the deployment script via SSM Run Command:

# Assuming 'green' instances are tagged with DeploymentGroup: Green
aws ssm send-command \
    --targets "Key=tag:DeploymentGroup,Values=Green" \
    --document-name "AWS-RunShellScript" \
    --parameters "commands=bash -c 'wget -O deploy.sh YOUR_DEPLOY_SCRIPT_URL_OR_INLINE_SCRIPT && bash deploy.sh'" \
    --comment "Deploying new version to Green environment" \
    --output text \
    --query "Command.CommandId"

Using AWS CodeDeploy

CodeDeploy is purpose-built for automating application deployments to EC2 instances, on-premises servers, and serverless Lambda functions. It natively supports blue-green deployments.

To use CodeDeploy for blue-green, you’ll need to:

  • Create a CodeDeploy Application: Define your application in CodeDeploy.
  • Create a Deployment Group: This is where you specify your EC2 instances (using tags like DeploymentGroup: Green), the ELB target group to update, and the blue-green deployment configuration.
  • Create an AppSpec File: This YAML file defines the lifecycle hooks for your deployment (e.g., BeforeInstall, AfterInstall, ApplicationStart, ValidateService).

Example appspec.yml for a C application:

version: 0.0
os: linux
files:
  - source: /opt/your_app_binaries/my_c_app
    destination: /usr/local/bin/my_c_app
hooks:
  BeforeInstall:
    - location: scripts/stop_service.sh
      timeout: 300
      runas: root
  AfterInstall:
    - location: scripts/configure_app.sh
      timeout: 300
      runas: root
  ApplicationStart:
    - location: scripts/start_service.sh
      timeout: 300
      runas: root
  ValidateService:
    - location: scripts/health_check.sh
      timeout: 60
      runas: root

The ValidateService hook is critical for blue-green. CodeDeploy will execute this script. If it succeeds, the deployment is considered successful. If it fails, CodeDeploy will automatically roll back by shifting traffic back to the blue environment and terminating the green instances.

The Blue-Green Traffic Switching Mechanism

The most critical part of a blue-green deployment is the traffic switch. This must be atomic and fast.

Manual Traffic Switching (using AWS CLI)

After deploying the new version to the green environment and verifying its health (e.g., via SSM or CodeDeploy’s validation hook), you can manually switch traffic.

Step 1: Identify the current listener rule.

# Get the listener ARN for your ALB
LISTENER_ARN=$(aws elbv2 describe-listeners --load-balancer-arn YOUR_ALB_ARN --query "Listeners[0].ListenerArn" --output text)

# Get the current rule ARN (assuming it's the default rule)
CURRENT_RULE_ARN=$(aws elbv2 describe-rules --listener-arn $LISTENER_ARN --query "Rules[?Priority=='default'].RuleArn" --output text)
# Or if you have specific priorities, find the one pointing to the blue TG

Step 2: Get the ARNs of your target groups.

BLUE_TG_ARN=$(aws elbv2 describe-target-groups --load-balancer-arn YOUR_ALB_ARN --query "TargetGroups[?TargetGroupName=='my-app-blue-tg'].TargetGroupArn" --output text)
GREEN_TG_ARN=$(aws elbv2 describe-target-groups --load-balancer-arn YOUR_ALB_ARN --query "TargetGroups[?TargetGroupName=='my-app-green-tg'].TargetGroupArn" --output text)

Step 3: Update the listener rule to point to the green target group.

# First, you might want to modify the existing rule or create a new one with higher priority
# For a simple switch, we can modify the default rule's action.
# It's often better to create a new rule with a higher priority and then remove the old one.

# Example: Modifying the default rule's action (use with caution)
aws elbv2 modify-listener \
    --listener-arn $LISTENER_ARN \
    --default-actions "Type=forward,TargetGroupArn=$GREEN_TG_ARN"

# A more robust approach involves creating a new rule with a higher priority and then deleting the old one.
# This ensures a cleaner switch and easier rollback.
# Example: Create a new rule with priority 100 pointing to Green
NEW_RULE_ARN=$(aws elbv2 create-rule \
    --listener-arn $LISTENER_ARN \
    --priority 100 \
    --conditions '{"Field": "path-pattern", "Values": ["/*"]}' \
    --actions "Type=forward,TargetGroupArn=$GREEN_TG_ARN" \
    --query "Rules[0].RuleArn" --output text)

# Once confirmed, you can remove the old rule pointing to Blue
# aws elbv2 delete-rule --rule-id YOUR_OLD_RULE_ID

After this step, all new incoming traffic will be directed to the green environment. The blue environment is now idle and can be used for the next deployment cycle.

Automated Traffic Switching with CodeDeploy

When using CodeDeploy with a blue-green deployment configuration, CodeDeploy handles the traffic shifting automatically. It uses a “traffic shifting” listener rule. You configure CodeDeploy to:

  • Initially send 0% of traffic to the new (green) deployment.
  • After the ValidateService hook passes, shift 100% of traffic to the new deployment.
  • Optionally, you can configure CodeDeploy to shift traffic gradually (e.g., 10% for 5 minutes, then 50%, then 100%).

CodeDeploy manages the creation and modification of ELB listener rules for you based on your deployment group configuration.

Rollback Strategy

The beauty of blue-green is the instant rollback capability.

Manual Rollback

If issues are detected after switching traffic to the green environment:

  • Identify the current listener rule that points to the green environment.
  • Update the listener rule to point back to the blue target group. This is the reverse of the traffic switching step.
  • Terminate the green instances or prepare them for the next deployment.
  • Investigate the issue on the terminated green instances (if logs are preserved) or by redeploying the problematic version to a test environment.

Automated Rollback with CodeDeploy

If the ValidateService hook in your appspec.yml fails, CodeDeploy automatically rolls back. It will shift traffic back to the blue environment and terminate the green instances. This is a significant advantage for ensuring production stability.

Considerations for C Applications

C applications often have specific requirements:

  • Build Process: Ensure your CI/CD pipeline can reliably compile your C code into a deployable artifact (e.g., a static binary or a dynamically linked executable). Store these artifacts in S3.
  • Dependencies: Manage shared libraries (.so files) carefully. They should be included in your deployment artifact or pre-installed on your EC2 instances.
  • Service Management: Use a robust service manager like systemd or SysVinit to ensure your C application runs as a background service. The deployment scripts must correctly start, stop, and restart these services.
  • Configuration Management: Externalize configuration (e.g., database connection strings, API endpoints) into configuration files or environment variables. Ensure these are updated or managed correctly during deployment.
  • Health Checks: Implement a thorough health check endpoint in your C application. This is crucial for both CodeDeploy’s validation and for manual verification. It should check not only if the process is running but also if it can connect to its dependencies (databases, other services).
  • Resource Usage: C applications can sometimes be resource-intensive. Monitor CPU, memory, and I/O usage closely during and after deployments.

Orchestration and CI/CD Integration

A complete blue-green pipeline would look something like this:

  1. Commit Code: Developer commits C code to a Git repository.
  2. CI Pipeline Triggered: Jenkins, GitLab CI, AWS CodePipeline, etc., detects the commit.
  3. Build and Test: The CI server compiles the C code, runs unit tests, and creates a deployable artifact (e.g., a tarball of binaries and necessary config files).
  4. Artifact Storage: The artifact is uploaded to an S3 bucket.
  5. CD Pipeline Triggered: A deployment pipeline (e.g., AWS CodeDeploy, Spinnaker, Argo CD) is initiated.
  6. Deploy to Green: The pipeline triggers a deployment to the “green” environment using CodeDeploy or SSM Run Command. This involves stopping the old version, deploying the new version, and starting the new version.
  7. Validate Green: The ValidateService hook (CodeDeploy) or a custom SSM command runs health checks on the green environment.
  8. Traffic Shift: If validation passes, traffic is shifted from blue to green (automatically by CodeDeploy or manually via AWS CLI).
  9. Monitor: Monitor application performance and error rates in production.
  10. Rollback (if necessary): If issues arise, initiate a rollback by shifting traffic back to blue.
  11. Decommission Blue: Once the green environment is stable, the blue environment can be terminated or prepared for the next deployment cycle.

For C applications, ensuring the build artifact is self-contained and correctly packaged is paramount for a smooth CI/CD integration.