Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Complete Enterprise Migration Guide: Upgrading DigitalOcean Droplets Infrastructure directly to AWS ECS (Fargate)

The Complete Enterprise Migration Guide: Upgrading DigitalOcean Droplets Infrastructure directly to AWS ECS (Fargate)

Phase 1: Pre-Migration Assessment and Planning

Migrating an existing DigitalOcean Droplets infrastructure to AWS Elastic Container Service (ECS) with Fargate requires a meticulous, phased approach. This isn’t a lift-and-shift; it’s a replatforming exercise that leverages containerization and serverless compute. The initial phase focuses on understanding the current state, identifying dependencies, and defining the target architecture.

Inventory Existing Droplets and Applications:

  • Compute: Document all Droplets, their sizes (CPU, RAM), operating systems, and primary roles (web servers, databases, background workers, etc.).
  • Networking: Map out VPCs, subnets, firewalls (ufw rules), load balancers (HAProxy, Nginx), and DNS configurations.
  • Storage: Identify persistent data volumes, their types (block storage), sizes, and access patterns.
  • Databases: List all databases (MySQL, PostgreSQL, Redis, etc.), their versions, sizes, and connection methods.
  • Dependencies: Crucially, map inter-service communication, external API integrations, and any stateful components.
  • Application Stack: Detail the programming languages, frameworks, dependencies, and build processes for each application.

Containerization Strategy:

The core of this migration is containerizing your applications. For each application running on a Droplet, you’ll need to create a Dockerfile. This process involves understanding your application’s runtime environment and packaging it into an immutable image.

Consider a typical PHP application with Nginx and PHP-FPM:

Example Dockerfile for a PHP Application

# Use an official PHP runtime as a parent image
FROM php:8.2-fpm

# Set the working directory in the container
WORKDIR /var/www/html

# Install system dependencies required by your application
# Example: Install git, unzip, and common PHP extensions
RUN apt-get update && apt-get install -y \
    git \
    unzip \
    libzip-dev \
    libpng-dev \
    libjpeg-dev \
    libfreetype6-dev \
    libssl-dev \
    libonig-dev \
    libxml2-dev \
    zip \
    && docker-php-ext-configure gd --with-freetype --with-jpeg \
    && docker-php-ext-install -j$(nproc) gd \
    && docker-php-ext-install pdo pdo_mysql zip exif pcntl opcache \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

# Copy your application code to the container
COPY . /var/www/html

# Install Composer dependencies
COPY --chown=www-data:www-data composer.json composer.lock ./
RUN composer install --no-dev --no-autoloader --no-scripts && composer dump-autoload --optimize --no-dev

# Copy application assets and run migrations (if applicable)
# This might require a multi-stage build for production
COPY --chown=www-data:www-data public/ /var/www/html/public/
RUN composer run-script post-install-cmd

# Expose port 9000 for PHP-FPM
EXPOSE 9000

# Command to run PHP-FPM
CMD ["php-fpm"]

For Nginx, you’ll create a separate container. This is a common pattern in microservices and containerized deployments, allowing for independent scaling and management of web serving and application logic.

Example Dockerfile for Nginx

# Use an official Nginx runtime as a parent image
FROM nginx:stable-alpine

# Remove default Nginx configuration
RUN rm -rf /etc/nginx/conf.d/*

# Copy custom Nginx configuration
COPY nginx.conf /etc/nginx/conf.d/default.conf

# Copy static assets (if any)
COPY public/ /usr/share/nginx/html

# Expose port 80
EXPOSE 80

# Start Nginx in the foreground
CMD ["nginx", "-g", "daemon off;"]

And the corresponding Nginx configuration:

Example Nginx Configuration (nginx.conf)

server {
    listen 80;
    server_name localhost;
    root /var/www/html; # Or /usr/share/nginx/html if serving static files directly

    index index.php index.html index.htm;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        # Use the PHP-FPM service name and port defined in your docker-compose or ECS task definition
        fastcgi_pass php-fpm:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }

    # Deny access to sensitive files
    location ~ /\.ht {
        deny all;
    }

    # Cache static assets for a year
    location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|webp)$ {
        expires 1y;
        add_header Cache-Control "public";
    }
}

AWS Target Architecture Design:

For AWS ECS with Fargate, the primary components will be:

  • ECS Cluster: A logical grouping of ECS services.
  • ECS Service: Manages the desired number of tasks (container instances) for your application.
  • ECS Task Definition: A blueprint for your application, specifying container images, CPU/memory requirements, ports, environment variables, and volumes.
  • AWS Fargate: The serverless compute engine for ECS, abstracting away EC2 instance management.
  • Amazon ECR (Elastic Container Registry): A managed Docker container registry for storing your container images.
  • Application Load Balancer (ALB): To distribute incoming traffic across your Fargate tasks.
  • Amazon RDS or Aurora: For managed relational databases, replacing self-hosted MySQL/PostgreSQL on Droplets.
  • Amazon ElastiCache: For managed Redis/Memcached, replacing self-hosted instances.
  • AWS Secrets Manager or Systems Manager Parameter Store: For securely managing database credentials, API keys, and other secrets.
  • Amazon S3: For object storage, potentially replacing file storage on Droplets.
  • VPC, Subnets, Security Groups: AWS’s networking constructs to isolate and secure your resources.

Phase 2: Containerization and Image Registry Setup

With the Dockerfiles in hand, the next step is to build and push these images to a registry. Amazon ECR is the natural choice within the AWS ecosystem.

Set up Amazon ECR:

  • Create an ECR repository for each distinct application or service.
  • Configure IAM roles and policies to allow your CI/CD pipeline (or local Docker client) to push images.

Build and Push Docker Images:

On a build server or your local machine, after installing Docker and configuring AWS CLI credentials:

Example: Building and Pushing to ECR

# 1. Authenticate Docker to your ECR registry
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com

# 2. Build your application image (e.g., PHP app)
cd /path/to/your/php-app
docker build -t my-php-app:latest .

# 3. Tag the image for ECR
docker tag my-php-app:latest <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-php-app:v1.0.0

# 4. Push the image to ECR
docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-php-app:v1.0.0

# Repeat for Nginx and other services

CI/CD Integration:

Automate this build and push process using a CI/CD pipeline (e.g., AWS CodePipeline, GitLab CI, GitHub Actions). This ensures that every code commit results in a new, versioned container image ready for deployment.

Phase 3: AWS ECS and Fargate Configuration

This phase involves setting up the core AWS services for running your containers.

VPC and Networking:

Ensure you have a VPC with at least two private subnets (for Fargate tasks) and potentially public subnets for NAT Gateways or Bastion hosts. Configure Security Groups to allow ingress from the ALB to your application tasks and egress to necessary AWS services (RDS, ElastiCache, ECR).

Example Security Group Rules

  • ALB Security Group: Allow inbound HTTP (80) and HTTPS (443) from `0.0.0.0/0`.
  • ECS Task Security Group: Allow inbound traffic on the application port (e.g., 80 for Nginx, 9000 for PHP-FPM if directly exposed, though typically proxied) from the ALB Security Group. Allow outbound to RDS, ElastiCache, ECR, etc.

ECS Cluster and Task Definitions:

Create an ECS cluster. Then, define your tasks. A multi-container task definition is common for Nginx and PHP-FPM.

Example ECS Task Definition (JSON)

{
    "family": "my-web-app",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "1024",  // 1 vCPU
    "memory": "2048", // 2 GB
    "executionRoleArn": "arn:aws:iam::<aws_account_id>:role/ecsTaskExecutionRole",
    "containerDefinitions": [
        {
            "name": "nginx",
            "image": "<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-nginx-app:v1.0.0",
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/ecs/my-web-app/nginx",
                    "awslogs-region": "us-east-1",
                    "awslogs-stream-prefix": "ecs"
                }
            }
        },
        {
            "name": "php-fpm",
            "image": "<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-php-app:v1.0.0",
            "essential": true,
            "command": ["php-fpm"],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/ecs/my-web-app/php-fpm",
                    "awslogs-region": "us-east-1",
                    "awslogs-stream-prefix": "ecs"
                }
            }
            // No port mapping needed if only communicating within the task
        }
    ]
}

Note: The `ecsTaskExecutionRole` needs permissions to pull images from ECR and send logs to CloudWatch Logs. The `networkMode: “awsvpc”` is required for Fargate.

ECS Service:

Create an ECS service using the task definition. This service will manage the desired count of Fargate tasks. Configure it to use the Application Load Balancer.

Service Configuration Details

  • Load Balancing: Select “Application Load Balancer”. Create a target group that points to the container port (e.g., 80 for Nginx) within your VPC’s private subnets.
  • Service Discovery: Integrate with AWS Cloud Map for service-to-service communication if needed.
  • Auto Scaling: Configure Application Auto Scaling based on CPU utilization, memory utilization, or custom CloudWatch metrics.
  • Deployment Type: Rolling update is standard. Consider blue/green deployments for zero-downtime releases.

Database and Cache Migration:

This is a critical replatforming step. Migrate your self-hosted databases and caches to managed AWS services.

Database Migration Steps

  • Provision RDS/Aurora Instance: Create a new managed database instance in AWS. Ensure it’s in private subnets and accessible only via its Security Group.
  • Schema Migration: If moving between major versions or engine types, perform schema conversion.
  • Data Migration: Use AWS Database Migration Service (DMS) for minimal downtime migration. For simpler cases, mysqldump/pg_dump and restore can work, but require more downtime.
  • Update Application Configuration: Store new database connection strings in AWS Secrets Manager or Parameter Store and inject them as environment variables into your ECS tasks.

Similarly, migrate Redis/Memcached to Amazon ElastiCache.

Phase 4: Testing, Cutover, and Post-Migration

Thorough testing is paramount before and after the cutover.

Pre-Cutover Testing:

  • Integration Testing: Test all application functionalities against the new AWS infrastructure, including database and cache connections.
  • Performance Testing: Simulate production load to identify bottlenecks and tune Fargate task sizes, ALB configurations, and database/cache resources.
  • Security Testing: Verify Security Group rules, IAM policies, and secret management.

Cutover Strategy:

The cutover involves redirecting traffic from your DigitalOcean Droplets to the AWS ECS Fargate service. This is typically done at the DNS level.

DNS Cutover Example

# 1. Update your DNS records (e.g., in Route 53 or your current DNS provider)
# Point your domain's A record (or CNAME) to the DNS name of your AWS Application Load Balancer.

# Example using AWS CLI for Route 53 (assuming you have a hosted zone and record set)
aws route53 change-resource-record-sets --hosted-zone-id Z1XXXXXXXXXXXXX --change-batch '{
    "Comment": "Update A record for application domain",
    "Changes": [
        {
            "Action": "UPSERT",
            "ResourceRecordSet": {
                "Name": "your.application.com.",
                "Type": "A",
                "AliasTarget": {
                    "HostedZoneId": "ZXXXXXXXXXXXXX",  // ALB's hosted zone ID
                    "DNSName": "your-alb-name-1234567890.us-east-1.elb.amazonaws.com.",
                    "EvaluateTargetHealth": true
                }
            }
        }
    ]
}'

Phased Rollout: Consider using DNS weighting or a staggered rollout to gradually shift traffic, allowing for quick rollback if issues arise.

Post-Migration Monitoring and Optimization:

After the cutover, continuously monitor your application’s health and performance using AWS CloudWatch, ECS service metrics, and ALB access logs. Optimize Fargate task sizes, scaling policies, and database configurations based on real-world usage patterns.

This comprehensive approach, moving from assessment and containerization to robust AWS infrastructure setup and careful cutover, ensures a successful replatforming from DigitalOcean Droplets to AWS ECS Fargate.