Vengala Vinay

Having 12+ Years of Experience in Software Development

Home » Infrastructure as Code: Provisioning Secure Shopify Clusters on AWS Using Terraform

Infrastructure as Code: Provisioning Secure Shopify Clusters on AWS Using Terraform

Terraform Project Structure for Shopify on AWS

A robust Infrastructure as Code (IaC) strategy for deploying complex applications like Shopify on AWS necessitates a well-organized Terraform project. This structure promotes modularity, reusability, and maintainability, especially as the infrastructure scales and evolves. We’ll adopt a standard Terraform directory layout, separating concerns into distinct modules.

Our primary directory will house the root configuration, which orchestrates the deployment of various modules. This root module will define the AWS provider, backend configuration, and the instantiation of our core infrastructure modules.

Root Module Configuration (`main.tf`)

The `main.tf` file in the root directory is the entry point for our Terraform deployment. It defines the AWS provider, specifies the remote state backend (crucial for team collaboration and state locking), and calls our custom modules.

# main.tf

terraform {
  backend "s3" {
    bucket         = "your-terraform-state-bucket-name"
    key            = "shopify/prod/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "your-terraform-state-lock-table"
    encrypt        = true
  }

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

# --- Module Invocations ---

module "vpc" {
  source = "./modules/vpc"

  vpc_cidr_block = "10.0.0.0/16"
  public_subnet_cidrs = [
    "10.0.1.0/24",
    "10.0.2.0/24",
  ]
  private_subnet_cidrs = [
    "10.0.101.0/24",
    "10.0.102.0/24",
  ]
  availability_zones = ["us-east-1a", "us-east-1b"]
}

module "rds" {
  source = "./modules/rds"

  db_instance_class    = "db.r6g.large"
  db_engine            = "postgres"
  db_engine_version    = "14.5"
  db_name              = "shopify_db"
  db_username          = "shopify_user"
  db_password          = var.db_password # Sensitive data managed via variables/secrets manager
  allocated_storage    = 100
  storage_type         = "gp3"
  vpc_id               = module.vpc.vpc_id
  private_subnet_ids   = module.vpc.private_subnet_ids
  security_group_ids   = [module.security_groups.rds_sg_id]
  skip_final_snapshot  = false
  deletion_protection  = true
}

module "ecs_cluster" {
  source = "./modules/ecs"

  cluster_name = "shopify-ecs-cluster"
  vpc_id       = module.vpc.vpc_id
  subnet_ids   = module.vpc.private_subnet_ids
}

module "load_balancer" {
  source = "./modules/alb"

  name               = "shopify-alb"
  vpc_id             = module.vpc.vpc_id
  public_subnet_ids  = module.vpc.public_subnet_ids
  security_group_ids = [module.security_groups.alb_sg_id]
  target_type        = "ip"
}

module "security_groups" {
  source = "./modules/security_groups"

  vpc_id = module.vpc.vpc_id
}

# Example of passing sensitive variables
variable "db_password" {
  description = "Password for the RDS database user."
  type        = string
  sensitive   = true
}

The `backend “s3″` block configures Terraform to store its state file in an S3 bucket. This is critical for managing state in a team environment, preventing conflicts, and enabling state locking via DynamoDB. Ensure you replace placeholder values like your-terraform-state-bucket-name and your-terraform-state-lock-table with your actual AWS resource names. The required_providers block pins the AWS provider version, ensuring consistent behavior.

Modular Design: VPC Module (`modules/vpc`)

A well-defined Virtual Private Cloud (VPC) is the foundation of a secure AWS deployment. Our VPC module will create a custom VPC with public and private subnets across multiple Availability Zones, along with necessary routing and internet gateway configurations.

# modules/vpc/main.tf

resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr_block
  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = {
    Name = "shopify-vpc"
  }
}

resource "aws_internet_gateway" "gw" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "shopify-vpc-igw"
  }
}

# Public Subnets
resource "aws_subnet" "public" {
  count             = length(var.public_subnet_cidrs)
  vpc_id            = aws_vpc.main.id
  cidr_block        = var.public_subnet_cidrs[count.index]
  availability_zone = var.availability_zones[count.index % length(var.availability_zones)] # Distribute across AZs
  map_public_ip_on_launch = true

  tags = {
    Name = "shopify-public-subnet-${count.index}"
  }
}

resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.gw.id
  }

  tags = {
    Name = "shopify-public-rt"
  }
}

resource "aws_route_table_association" "public" {
  count          = length(aws_subnet.public)
  subnet_id      = aws_subnet.public[count.index].id
  route_table_id = aws_route_table.public.id
}

# Private Subnets
resource "aws_subnet" "private" {
  count             = length(var.private_subnet_cidrs)
  vpc_id            = aws_vpc.main.id
  cidr_block        = var.private_subnet_cidrs[count.index]
  availability_zone = var.availability_zones[count.index % length(var.availability_zones)] # Distribute across AZs
  map_public_ip_on_launch = false

  tags = {
    Name = "shopify-private-subnet-${count.index}"
  }
}

resource "aws_nat_gateway" "nat" {
  count         = length(aws_subnet.private) # One NAT Gateway per private subnet for high availability
  allocation_id = aws_eip.nat[count.index].id
  subnet_id     = aws_subnet.public[count.index] # NAT Gateway resides in a public subnet

  tags = {
    Name = "shopify-nat-gw-${count.index}"
  }

  depends_on = [aws_internet_gateway.gw]
}

resource "aws_eip" "nat" {
  count = length(aws_subnet.private)
  domain   = "vpc"
}

resource "aws_route_table" "private" {
  count  = length(aws_subnet.private)
  vpc_id = aws_vpc.main.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.nat[count.index].id
  }

  tags = {
    Name = "shopify-private-rt-${count.index}"
  }
}

resource "aws_route_table_association" "private" {
  count          = length(aws_subnet.private)
  subnet_id      = aws_subnet.private[count.index].id
  route_table_id = aws_route_table.private[count.index].id
}

# --- Outputs ---
output "vpc_id" {
  description = "The ID of the VPC."
  value       = aws_vpc.main.id
}

output "public_subnet_ids" {
  description = "List of IDs of the public subnets."
  value       = aws_subnet.public[*].id
}

output "private_subnet_ids" {
  description = "List of IDs of the private subnets."
  value       = aws_subnet.private[*].id
}

# modules/vpc/variables.tf

variable "vpc_cidr_block" {
  description = "The CIDR block for the VPC."
  type        = string
}

variable "public_subnet_cidrs" {
  description = "List of CIDR blocks for the public subnets."
  type        = list(string)
}

variable "private_subnet_cidrs" {
  description = "List of CIDR blocks for the private subnets."
  type        = list(string)
}

variable "availability_zones" {
  description = "List of Availability Zones to deploy subnets into."
  type        = list(string)
}

This module defines the VPC, an Internet Gateway, public subnets with direct internet access via a route table, and private subnets. For high availability and outbound internet access from private subnets (e.g., for pulling Docker images), we provision NAT Gateways in the public subnets, each with an Elastic IP address. Each private subnet gets its own route table pointing to a NAT Gateway in the corresponding public subnet’s AZ. The outputs provide essential IDs for other modules to reference.

Database Layer: RDS Module (`modules/rds`)

A secure and scalable database is paramount. We’ll use the AWS RDS module to provision a PostgreSQL instance within our private subnets, ensuring it’s not directly accessible from the internet. Parameter groups and option groups can be further customized for performance tuning.

# modules/rds/main.tf

resource "aws_db_instance" "main" {
  allocated_storage       = var.allocated_storage
  storage_type            = var.storage_type
  engine                  = var.db_engine
  engine_version          = var.db_engine_version
  identifier              = "shopify-db-instance"
  instance_class          = var.db_instance_class
  name                    = var.db_name
  username                = var.db_username
  password                = var.db_password
  db_subnet_group_name    = aws_db_subnet_group.main.name
  vpc_security_group_ids  = var.security_group_ids
  skip_final_snapshot     = var.skip_final_snapshot
  deletion_protection     = var.deletion_protection
  publicly_accessible     = false # Crucial for security
  multi_az                = true  # For high availability

  tags = {
    Name = "shopify-rds-instance"
  }
}

resource "aws_db_subnet_group" "main" {
  name       = "shopify-db-subnet-group"
  subnet_ids = var.private_subnet_ids

  tags = {
    Name = "shopify-db-subnet-group"
  }
}

# --- Outputs ---
output "db_instance_endpoint" {
  description = "The connection endpoint for the RDS instance."
  value       = aws_db_instance.main.endpoint
}

output "db_instance_arn" {
  description = "The ARN of the RDS instance."
  value       = aws_db_instance.main.arn
}

# modules/rds/variables.tf

variable "db_instance_class" {
  description = "The instance class for the RDS instance."
  type        = string
}

variable "db_engine" {
  description = "The database engine to use."
  type        = string
}

variable "db_engine_version" {
  description = "The database engine version."
  type        = string
}

variable "db_name" {
  description = "The name of the database to create."
  type        = string
}

variable "db_username" {
  description = "The username for the database master user."
  type        = string
}

variable "db_password" {
  description = "The password for the database master user."
  type        = string
  sensitive   = true
}

variable "allocated_storage" {
  description = "The allocated storage in GB."
  type        = number
}

variable "storage_type" {
  description = "The storage type for the RDS instance."
  type        = string
}

variable "vpc_id" {
  description = "The ID of the VPC to deploy RDS into."
  type        = string
}

variable "private_subnet_ids" {
  description = "List of private subnet IDs for the DB subnet group."
  type        = list(string)
}

variable "security_group_ids" {
  description = "List of security group IDs to associate with the RDS instance."
  type        = list(string)
}

variable "skip_final_snapshot" {
  description = "Determines whether a final snapshot is created before the instance is deleted."
  type        = bool
  default     = true
}

variable "deletion_protection" {
  description = "Determines whether deletion protection is enabled for the RDS instance."
  type        = bool
  default     = false
}

This module creates an aws_db_subnet_group that spans our private subnets and then provisions the aws_db_instance. Setting publicly_accessible = false is critical for security. multi_az = true ensures high availability by creating a standby replica in a different Availability Zone. The db_password is marked as sensitive and should be managed via Terraform Cloud variables, AWS Secrets Manager, or environment variables.

Compute Layer: ECS Cluster Module (`modules/ecs`)

For containerized Shopify applications, Amazon Elastic Container Service (ECS) is a natural fit. This module sets up an ECS cluster, which acts as a logical grouping for our services and tasks. We’ll use the EC2 launch type for maximum control, though Fargate is an alternative for serverless container orchestration.

# modules/ecs/main.tf

resource "aws_ecs_cluster" "main" {
  name = var.cluster_name

  tags = {
    Name = "${var.cluster_name}-cluster"
  }
}

# --- EC2 Launch Type Specific Resources ---
# If using EC2 launch type, you'll need an ECS-optimized AMI and Auto Scaling Group.
# For simplicity, this example focuses on the cluster itself.
# A full EC2 launch type implementation would include:
# - aws_launch_template
# - aws_autoscaling_group
# - aws_iam_role for EC2 instances
# - aws_iam_policy for ECS agent

# --- Fargate Launch Type (Alternative) ---
# If using Fargate, you don't need EC2 instances or ASGs.
# The cluster resource is sufficient.

# --- Outputs ---
output "ecs_cluster_id" {
  description = "The ID of the ECS cluster."
  value       = aws_ecs_cluster.main.id
}

output "ecs_cluster_arn" {
  description = "The ARN of the ECS cluster."
  value       = aws_ecs_cluster.main.arn
}

# modules/ecs/variables.tf

variable "cluster_name" {
  description = "The name for the ECS cluster."
  type        = string
}

variable "vpc_id" {
  description = "The ID of the VPC where the ECS cluster will operate."
  type        = string
}

variable "subnet_ids" {
  description = "List of subnet IDs for ECS services/tasks (especially relevant for Fargate)."
  type        = list(string)
}

This module is straightforward, primarily defining the aws_ecs_cluster. For a production setup using the EC2 launch type, you would extend this module to include an Auto Scaling Group configured with an ECS-optimized AMI, an IAM role for EC2 instances to join the cluster, and necessary security group rules. If opting for Fargate, this module is largely sufficient as AWS manages the underlying compute infrastructure.

Load Balancing: ALB Module (`modules/alb`)

An Application Load Balancer (ALB) is essential for distributing incoming HTTP/S traffic to our Shopify application instances. This module will create the ALB, target groups, and listeners.

# modules/alb/main.tf

resource "aws_lb" "main" {
  name               = var.name
  internal           = false # Set to true for internal-only ALB
  load_balancer_type = "application"
  security_groups    = var.security_group_ids
  subnets            = var.public_subnet_ids

  enable_deletion_protection = true

  tags = {
    Name = "${var.name}-alb"
  }
}

# Example Target Group for Shopify application
resource "aws_lb_target_group" "app" {
  name     = "${var.name}-app-tg"
  port     = 80 # Or your application's port
  protocol = "HTTP"
  vpc_id   = var.vpc_id
  target_type = var.target_type # "instance" or "ip"

  health_check {
    path                = "/health" # Assuming a health check endpoint
    protocol            = "HTTP"
    matcher             = "200"
    interval            = 30
    timeout             = 5
    healthy_threshold   = 3
    unhealthy_threshold = 3
  }

  tags = {
    Name = "${var.name}-app-tg"
  }
}

# Example Listener for HTTP traffic
resource "aws_lb_listener" "http" {
  load_balancer_arn = aws_lb.main.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.app.arn
  }
}

# Example Listener for HTTPS traffic (requires ACM certificate)
# resource "aws_lb_listener" "https" {
#   load_balancer_arn = aws_lb.main.arn
#   port              = "443"
#   protocol          = "HTTPS"
#   certificate_arn   = var.acm_certificate_arn # Pass this via variables

#   default_action {
#     type             = "forward"
#     target_group_arn = aws_lb_target_group.app.arn
#   }
# }

# --- Outputs ---
output "alb_dns_name" {
  description = "The DNS name of the ALB."
  value       = aws_lb.main.dns_name
}

output "alb_zone_id" {
  description = "The canonical hosted zone ID of the ALB."
  value       = aws_lb.main.zone_id
}

output "app_target_group_arn" {
  description = "The ARN of the application target group."
  value       = aws_lb_target_group.app.arn
}

# modules/alb/variables.tf

variable "name" {
  description = "The name of the ALB."
  type        = string
}

variable "vpc_id" {
  description = "The ID of the VPC for the ALB."
  type        = string
}

variable "public_subnet_ids" {
  description = "List of public subnet IDs for the ALB."
  type        = list(string)
}

variable "security_group_ids" {
  description = "List of security group IDs to associate with the ALB."
  type        = list(string)
}

variable "target_type" {
  description = "The type of target that the ALB supports. Valid values are instance or ip."
  type        = string
  default     = "ip"
}

# variable "acm_certificate_arn" {
#   description = "ARN of the ACM certificate for HTTPS listener."
#   type        = string
#   default     = ""
# }

This module provisions an aws_lb (ALB) in the public subnets, making it internet-facing. It also defines an aws_lb_target_group for the Shopify application, including health check configurations. A basic HTTP listener is included. For production, you would uncomment and configure the HTTPS listener, providing an ARN for an AWS Certificate Manager (ACM) certificate. The target_type is set to “ip”, which is generally recommended for ECS services using the `awsvpc` network mode.

Network Security: Security Groups Module (`modules/security_groups`)

Security groups act as virtual firewalls for our AWS resources. This module defines essential security groups for the ALB and the RDS database, enforcing the principle of least privilege.

# modules/security_groups/main.tf

# Security Group for the Application Load Balancer
resource "aws_security_group" "alb_sg" {
  name        = "shopify-alb-sg"
  description = "Allow HTTP and HTTPS inbound traffic"
  vpc_id      = var.vpc_id

  ingress {
    description = "HTTP from anywhere"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "HTTPS from anywhere"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "shopify-alb-sg"
  }
}

# Security Group for the RDS Database
resource "aws_security_group" "rds_sg" {
  name        = "shopify-rds-sg"
  description = "Allow PostgreSQL traffic from application security group"
  vpc_id      = var.vpc_id

  # Ingress rule allowing traffic from the ALB's security group (or ECS task SG)
  # This assumes your ECS tasks will be in a security group that can communicate with RDS.
  # For simplicity, we'll allow from the ALB SG. In a real scenario, you might
  # want a dedicated SG for your application containers.
  ingress {
    description     = "PostgreSQL from ALB SG"
    from_port       = 5432 # Default PostgreSQL port
    to_port         = 5432
    protocol        = "tcp"
    security_groups = [aws_security_group.alb_sg.id] # Restrict to ALB SG
  }

  # If you have a separate SG for your ECS tasks, use that instead of alb_sg.id
  # ingress {
  #   description     = "PostgreSQL from ECS Task SG"
  #   from_port       = 5432
  #   to_port         = 5432
  #   protocol        = "tcp"
  #   security_groups = [aws_security_group.ecs_task_sg.id]
  # }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"] # Allow outbound traffic for potential patching/updates
  }

  tags = {
    Name = "shopify-rds-sg"
  }
}

# --- Outputs ---
output "alb_sg_id" {
  description = "The ID of the ALB security group."
  value       = aws_security_group.alb_sg.id
}

output "rds_sg_id" {
  description = "The ID of the RDS security group."
  value       = aws_security_group.rds_sg.id
}

# modules/security_groups/variables.tf

variable "vpc_id" {
  description = "The ID of the VPC for the security groups."
  type        = string
}

The alb_sg allows inbound HTTP (port 80) and HTTPS (port 443) traffic from anywhere (0.0.0.0/0). The rds_sg is more restrictive: it only allows inbound PostgreSQL traffic (port 5432) from the alb_sg. This ensures that only traffic originating from the load balancer can reach the database. In a more complex setup, you would create a dedicated security group for your ECS tasks and allow RDS access from that group instead of, or in addition to, the ALB’s security group.

Deployment Workflow

With the Terraform code structured into modules, the deployment process is streamlined:

  • Initialize: Navigate to the root directory of your Terraform project (where main.tf resides) and run terraform init. This downloads the necessary provider plugins and configures the backend.
  • Plan: Execute terraform plan to see a preview of the infrastructure changes Terraform will make. Review this output carefully.
  • Apply: Run terraform apply to provision the resources on AWS. You will be prompted to confirm the changes.
  • Destroy: When the infrastructure is no longer needed, use terraform destroy to tear down all provisioned resources.

For managing sensitive variables like the database password, consider using:

  • Terraform Cloud/Enterprise Variables: Securely store and inject sensitive variables.
  • AWS Secrets Manager: Retrieve secrets dynamically during Terraform runs.
  • Environment Variables: Set variables like TF_VAR_db_password before running Terraform commands.

Next Steps and Considerations

This foundational setup provides a secure and scalable infrastructure for a Shopify deployment on AWS. Further enhancements include:

  • ECS Service and Task Definitions: Define how your Shopify application containers run, including resource allocation, environment variables, and networking (using aws_ecs_service and aws_ecs_task_definition).
  • CI/CD Integration: Automate Terraform runs within a CI/CD pipeline (e.g., GitLab CI, GitHub Actions, AWS CodePipeline) for consistent and reliable deployments.
  • Monitoring and Logging: Integrate CloudWatch Alarms, Logs, and potentially Prometheus/Grafana for comprehensive monitoring.
  • Secrets Management: Implement a robust secrets management strategy for application credentials, API keys, etc.
  • Cost Optimization: Regularly review instance types, storage, and resource utilization to manage AWS costs.
  • Disaster Recovery: Plan for multi-region deployments or robust backup and restore strategies for RDS.

Recent Posts

Top Categories

Our Products

Our Services