Disaster Recovery 101: Architecting Auto-Failovers for PostgreSQL and WordPress Deployments on AWS

Key CloudWatch Metrics to Monitor:

• RDS: CPUUtilization, DatabaseConnections, ReadIOPS, WriteIOPS, DiskQueueDepth, FreeableMemory, ReplicaLag (if using read replicas). Crucially, monitor RDS Events for failover notifications.
• EC2 (WordPress Instances): CPUUtilization, NetworkIn, NetworkOut, DiskReadOps, DiskWriteOps.
• ELB: HealthyHostCount, UnHealthyHostCount, HTTPCode_Target_5XX_Count, RequestCount.

Set up CloudWatch Alarms on these metrics. For example:

• Alarm if UnHealthyHostCount on ALB > 0 for more than 5 minutes.
• Alarm if CPUUtilization on RDS instance > 90% for 15 minutes.
• Alarm on specific RDS Events, such as “RDS-EVENT-0006” (Instance rebooted due to failover).

Configure these alarms to send notifications to an SNS topic, which can then trigger emails, Slack messages, or PagerDuty alerts.

Considerations for State and Caching

WordPress deployments often rely on caching mechanisms (e.g., Redis, Memcached) and may store session data. Ensure these components are also architected for high availability.

Caching: Use ElastiCache with replication groups and multi-AZ configurations. Your WordPress instances should be configured to connect to the ElastiCache cluster endpoint, which will automatically point to the primary node after a failover.

Session Management: Avoid storing session data directly on EC2 instances if they are ephemeral. Use a shared session store like ElastiCache or a database table (though this can impact database performance) for session persistence across application instances.

By combining RDS Multi-AZ for database resilience, ELB and Auto Scaling Groups for application availability, and robust monitoring, you can architect a highly available and fault-tolerant WordPress deployment on AWS capable of automated failover.

For applications with complex dependencies or specific business logic that must be available, a simple HTTP health check might not suffice. You can implement a custom PHP script (e.g., /healthcheck.php) that:

• Checks basic web server and PHP functionality.
• Attempts a read-only query to the PostgreSQL database using the RDS endpoint.
• Verifies the status of external APIs or services critical to the application.

This script can return different HTTP status codes or JSON payloads indicating the health status, allowing the ALB to make more informed decisions about routing traffic.

// Example: /healthcheck.php
<?php
require_once('wp-load.php'); // Load WordPress environment

header('Content-Type: application/json');

$response = ['status' => 'unhealthy', 'message' => 'Unknown error'];

try {
    // Check database connection (read-only query)
    global $wpdb;
    $wpdb->query( "SELECT 1" ); // Simple query to check connectivity

    // Add checks for other critical services if needed
    // e.g., $external_api_status = check_external_api();

    if ( $wpdb->last_error === '' ) { // Check for database query errors
        $response = ['status' => 'healthy', 'message' => 'All systems operational'];
        http_response_code(200); // OK
    } else {
        $response = ['status' => 'unhealthy', 'message' => 'Database connection failed'];
        http_response_code(503); // Service Unavailable
    }

} catch ( Exception $e ) {
    $response = ['status' => 'unhealthy', 'message' => 'Exception: ' . $e->getMessage()];
    http_response_code(503); // Service Unavailable
}

echo json_encode($response);
exit;
?>

Ensure your ALB’s health check path is set to this custom script and that the expected healthy status code (e.g., 200) is configured.

Monitoring and Alerting with CloudWatch

Comprehensive monitoring is key to detecting failures and triggering alerts. AWS CloudWatch provides essential metrics for both RDS and EC2 instances.

Key CloudWatch Metrics to Monitor:

• RDS: CPUUtilization, DatabaseConnections, ReadIOPS, WriteIOPS, DiskQueueDepth, FreeableMemory, ReplicaLag (if using read replicas). Crucially, monitor RDS Events for failover notifications.
• EC2 (WordPress Instances): CPUUtilization, NetworkIn, NetworkOut, DiskReadOps, DiskWriteOps.
• ELB: HealthyHostCount, UnHealthyHostCount, HTTPCode_Target_5XX_Count, RequestCount.

Set up CloudWatch Alarms on these metrics. For example:

• Alarm if UnHealthyHostCount on ALB > 0 for more than 5 minutes.
• Alarm if CPUUtilization on RDS instance > 90% for 15 minutes.
• Alarm on specific RDS Events, such as “RDS-EVENT-0006” (Instance rebooted due to failover).

Configure these alarms to send notifications to an SNS topic, which can then trigger emails, Slack messages, or PagerDuty alerts.

Considerations for State and Caching

WordPress deployments often rely on caching mechanisms (e.g., Redis, Memcached) and may store session data. Ensure these components are also architected for high availability.

Caching: Use ElastiCache with replication groups and multi-AZ configurations. Your WordPress instances should be configured to connect to the ElastiCache cluster endpoint, which will automatically point to the primary node after a failover.

Session Management: Avoid storing session data directly on EC2 instances if they are ephemeral. Use a shared session store like ElastiCache or a database table (though this can impact database performance) for session persistence across application instances.

By combining RDS Multi-AZ for database resilience, ELB and Auto Scaling Groups for application availability, and robust monitoring, you can architect a highly available and fault-tolerant WordPress deployment on AWS capable of automated failover.

To test the resilience of your application layer:

• Terminate an EC2 Instance: Manually terminate one of the EC2 instances managed by your ASG. Observe how the ALB stops sending traffic to it (due to failed health checks) and how the ASG launches a replacement instance.
• Simulate Network Issues: Use security group rules or network ACLs to temporarily block traffic to/from specific instances or AZs to mimic network partitions.
• Simulate Application Crashes: Introduce errors in your WordPress code or web server configuration that cause instances to become unhealthy.

Ensure that the ALB correctly identifies unhealthy instances and that the ASG replaces them, maintaining the desired capacity and availability of your WordPress deployment.

Advanced Considerations: Custom Failover Logic and Monitoring

While RDS Multi-AZ and ELB/ASG provide a strong foundation, advanced scenarios might require more granular control or custom logic.

Custom Health Checks and Application-Level Failover

For applications with complex dependencies or specific business logic that must be available, a simple HTTP health check might not suffice. You can implement a custom PHP script (e.g., /healthcheck.php) that:

• Checks basic web server and PHP functionality.
• Attempts a read-only query to the PostgreSQL database using the RDS endpoint.
• Verifies the status of external APIs or services critical to the application.

This script can return different HTTP status codes or JSON payloads indicating the health status, allowing the ALB to make more informed decisions about routing traffic.

// Example: /healthcheck.php
<?php
require_once('wp-load.php'); // Load WordPress environment

header('Content-Type: application/json');

$response = ['status' => 'unhealthy', 'message' => 'Unknown error'];

try {
    // Check database connection (read-only query)
    global $wpdb;
    $wpdb->query( "SELECT 1" ); // Simple query to check connectivity

    // Add checks for other critical services if needed
    // e.g., $external_api_status = check_external_api();

    if ( $wpdb->last_error === '' ) { // Check for database query errors
        $response = ['status' => 'healthy', 'message' => 'All systems operational'];
        http_response_code(200); // OK
    } else {
        $response = ['status' => 'unhealthy', 'message' => 'Database connection failed'];
        http_response_code(503); // Service Unavailable
    }

} catch ( Exception $e ) {
    $response = ['status' => 'unhealthy', 'message' => 'Exception: ' . $e->getMessage()];
    http_response_code(503); // Service Unavailable
}

echo json_encode($response);
exit;
?>

Ensure your ALB’s health check path is set to this custom script and that the expected healthy status code (e.g., 200) is configured.

Monitoring and Alerting with CloudWatch

Comprehensive monitoring is key to detecting failures and triggering alerts. AWS CloudWatch provides essential metrics for both RDS and EC2 instances.

Key CloudWatch Metrics to Monitor:

• RDS: CPUUtilization, DatabaseConnections, ReadIOPS, WriteIOPS, DiskQueueDepth, FreeableMemory, ReplicaLag (if using read replicas). Crucially, monitor RDS Events for failover notifications.
• EC2 (WordPress Instances): CPUUtilization, NetworkIn, NetworkOut, DiskReadOps, DiskWriteOps.
• ELB: HealthyHostCount, UnHealthyHostCount, HTTPCode_Target_5XX_Count, RequestCount.

Set up CloudWatch Alarms on these metrics. For example:

• Alarm if UnHealthyHostCount on ALB > 0 for more than 5 minutes.
• Alarm if CPUUtilization on RDS instance > 90% for 15 minutes.
• Alarm on specific RDS Events, such as “RDS-EVENT-0006” (Instance rebooted due to failover).

Configure these alarms to send notifications to an SNS topic, which can then trigger emails, Slack messages, or PagerDuty alerts.

Considerations for State and Caching

WordPress deployments often rely on caching mechanisms (e.g., Redis, Memcached) and may store session data. Ensure these components are also architected for high availability.

Caching: Use ElastiCache with replication groups and multi-AZ configurations. Your WordPress instances should be configured to connect to the ElastiCache cluster endpoint, which will automatically point to the primary node after a failover.

Session Management: Avoid storing session data directly on EC2 instances if they are ephemeral. Use a shared session store like ElastiCache or a database table (though this can impact database performance) for session persistence across application instances.

By combining RDS Multi-AZ for database resilience, ELB and Auto Scaling Groups for application availability, and robust monitoring, you can architect a highly available and fault-tolerant WordPress deployment on AWS capable of automated failover.

Leveraging AWS RDS Multi-AZ for PostgreSQL High Availability

For critical PostgreSQL deployments, particularly those powering WordPress sites, achieving robust high availability (HA) and automated failover is paramount. Amazon RDS Multi-AZ offers a managed solution that significantly simplifies this. It provisions and maintains a synchronous standby replica in a different Availability Zone (AZ). In the event of a primary instance failure, RDS automatically fails over to the standby replica with minimal interruption.

When creating or modifying an RDS PostgreSQL instance for HA, the key parameter is `MultiAZ`. Setting this to `true` during instance creation is the most straightforward approach. If you have an existing instance, you can modify it to enable Multi-AZ, though this typically involves a brief downtime as RDS creates the standby and performs an initial sync.

Configuring RDS PostgreSQL for Multi-AZ

Here’s a conceptual AWS CLI command to create a new RDS PostgreSQL instance with Multi-AZ enabled. Replace placeholders with your specific values.

Note: For production, always use a dedicated VPC, appropriate security groups, and encrypted storage.

aws rds create-db-instance \
    --db-instance-identifier my-wordpress-db-ha \
    --db-instance-class db.r5.large \
    --engine postgres \
    --allocated-storage 100 \
    --master-username admin \
    --master-user-password 'your_secure_password' \
    --vpc-security-group-ids sg-xxxxxxxxxxxxxxxxx \
    --db-subnet-group-name my-db-subnet-group \
    --multi-az \
    --storage-type gp2 \
    --backup-retention-period 7 \
    --tags Key=Environment,Value=Production Key=Project,Value=WordPress

To verify Multi-AZ status for an existing instance:

aws rds describe-db-instances \
    --db-instance-identifier my-wordpress-db-ha \
    --query "DBInstances[0].MultiAZ" \
    --output text

The output should be `True`. During a failover event, RDS automatically updates the DNS record for your DB instance endpoint to point to the standby replica. Your application, using the standard RDS endpoint, will automatically connect to the new primary after the DNS propagation and failover process completes.

Architecting WordPress Application Layer for Failover Resilience

While RDS Multi-AZ handles database failover, the WordPress application layer also needs to be resilient. A common and effective pattern is to deploy WordPress across multiple Availability Zones using Auto Scaling Groups and Elastic Load Balancing (ELB).

Elastic Load Balancing (ELB) with Auto Scaling Groups

An Application Load Balancer (ALB) is ideal for distributing HTTP/S traffic to your WordPress instances. It can span multiple AZs, providing high availability for the load balancer itself. Auto Scaling Groups (ASG) manage the EC2 instances running your WordPress application. By configuring the ASG to launch instances across multiple AZs within your VPC, you ensure that if one AZ becomes unavailable, your application can continue to serve traffic from other AZs.

Key ELB/ASG Configuration Points:

• VPC and Subnets: Configure your ALB and ASG to use subnets across at least two, preferably three, AZs.
• Health Checks: Implement robust health checks on your ALB. For WordPress, a simple check against /wp-includes/js/jquery/jquery.js or a custom health check endpoint (e.g., /healthcheck.php) is common. The health check should verify that WordPress is responding and ideally that it can connect to the database.
• Auto Scaling Group Launch Configuration/Template: Define EC2 instances with your WordPress installation, web server (Nginx/Apache), and PHP. Ensure these instances are configured to connect to the RDS endpoint.
• Database Connection String: Use the RDS endpoint (e.g., my-wordpress-db-ha.xxxxxxxxxxxx.us-east-1.rds.amazonaws.com) in your wp-config.php. This endpoint automatically resolves to the current primary RDS instance, even after a failover.

Here’s a simplified example of a wp-config.php snippet:

<?php
// ** Database settings ** //
define( 'DB_NAME', 'wordpress_db' );
define( 'DB_USER', 'wp_user' );
define( 'DB_PASSWORD', 'your_db_password' );
define( 'DB_HOST', 'my-wordpress-db-ha.xxxxxxxxxxxx.us-east-1.rds.amazonaws.com:5432' ); // Use RDS endpoint
define( 'DB_CHARSET', 'utf8' );
define( 'DB_COLLATE', '' );

// ** Security Keys ** //
// ... (your security keys) ...

// ** WordPress Database Table prefix ** //
$table_prefix = 'wp_';

// ** Other WordPress settings ** //
define( 'WP_DEBUG', false );

// ** If you're behind a proxy or load balancer ** //
// (Ensure your ALB is configured to forward X-Forwarded-For headers)
if ( isset( $_SERVER['HTTP_X_FORWARDED_FOR'] ) ) {
    $_SERVER['REMOTE_ADDR'] = $_SERVER['HTTP_X_FORWARDED_FOR'];
}

// ** Load WordPress ** //
require_once ABSPATH . 'wp-settings.php';
?>

The Auto Scaling Group should be configured to launch instances in multiple subnets across different AZs. The ALB will then distribute traffic to healthy instances within these AZs.

Simulating and Testing Failover Scenarios

Regular testing is crucial to validate your failover strategy. This involves simulating failures at different layers.

Database Failover Testing

You can manually initiate a failover for your RDS Multi-AZ instance via the AWS Management Console or AWS CLI. Navigate to the RDS dashboard, select your DB instance, choose “Instance actions” -> “Reboot”, and select “Reboot with failover”.

aws rds reboot-db-instance \
    --db-instance-identifier my-wordpress-db-ha \
    --force-failover

Monitor the RDS event logs and your application’s connectivity during this process. The failover typically takes 1-2 minutes, during which your application might experience a brief period of unavailability. Verify that your WordPress site becomes accessible again and that data integrity is maintained.

Application Instance Failure Testing

To test the resilience of your application layer:

• Terminate an EC2 Instance: Manually terminate one of the EC2 instances managed by your ASG. Observe how the ALB stops sending traffic to it (due to failed health checks) and how the ASG launches a replacement instance.
• Simulate Network Issues: Use security group rules or network ACLs to temporarily block traffic to/from specific instances or AZs to mimic network partitions.
• Simulate Application Crashes: Introduce errors in your WordPress code or web server configuration that cause instances to become unhealthy.

Ensure that the ALB correctly identifies unhealthy instances and that the ASG replaces them, maintaining the desired capacity and availability of your WordPress deployment.

Advanced Considerations: Custom Failover Logic and Monitoring

While RDS Multi-AZ and ELB/ASG provide a strong foundation, advanced scenarios might require more granular control or custom logic.

Custom Health Checks and Application-Level Failover

For applications with complex dependencies or specific business logic that must be available, a simple HTTP health check might not suffice. You can implement a custom PHP script (e.g., /healthcheck.php) that:

• Checks basic web server and PHP functionality.
• Attempts a read-only query to the PostgreSQL database using the RDS endpoint.
• Verifies the status of external APIs or services critical to the application.

This script can return different HTTP status codes or JSON payloads indicating the health status, allowing the ALB to make more informed decisions about routing traffic.

// Example: /healthcheck.php
<?php
require_once('wp-load.php'); // Load WordPress environment

header('Content-Type: application/json');

$response = ['status' => 'unhealthy', 'message' => 'Unknown error'];

try {
    // Check database connection (read-only query)
    global $wpdb;
    $wpdb->query( "SELECT 1" ); // Simple query to check connectivity

    // Add checks for other critical services if needed
    // e.g., $external_api_status = check_external_api();

    if ( $wpdb->last_error === '' ) { // Check for database query errors
        $response = ['status' => 'healthy', 'message' => 'All systems operational'];
        http_response_code(200); // OK
    } else {
        $response = ['status' => 'unhealthy', 'message' => 'Database connection failed'];
        http_response_code(503); // Service Unavailable
    }

} catch ( Exception $e ) {
    $response = ['status' => 'unhealthy', 'message' => 'Exception: ' . $e->getMessage()];
    http_response_code(503); // Service Unavailable
}

echo json_encode($response);
exit;
?>

Ensure your ALB’s health check path is set to this custom script and that the expected healthy status code (e.g., 200) is configured.

Monitoring and Alerting with CloudWatch

Comprehensive monitoring is key to detecting failures and triggering alerts. AWS CloudWatch provides essential metrics for both RDS and EC2 instances.

Key CloudWatch Metrics to Monitor:

• RDS: CPUUtilization, DatabaseConnections, ReadIOPS, WriteIOPS, DiskQueueDepth, FreeableMemory, ReplicaLag (if using read replicas). Crucially, monitor RDS Events for failover notifications.
• EC2 (WordPress Instances): CPUUtilization, NetworkIn, NetworkOut, DiskReadOps, DiskWriteOps.
• ELB: HealthyHostCount, UnHealthyHostCount, HTTPCode_Target_5XX_Count, RequestCount.

Set up CloudWatch Alarms on these metrics. For example:

• Alarm if UnHealthyHostCount on ALB > 0 for more than 5 minutes.
• Alarm if CPUUtilization on RDS instance > 90% for 15 minutes.
• Alarm on specific RDS Events, such as “RDS-EVENT-0006” (Instance rebooted due to failover).

Configure these alarms to send notifications to an SNS topic, which can then trigger emails, Slack messages, or PagerDuty alerts.

Considerations for State and Caching

WordPress deployments often rely on caching mechanisms (e.g., Redis, Memcached) and may store session data. Ensure these components are also architected for high availability.

Caching: Use ElastiCache with replication groups and multi-AZ configurations. Your WordPress instances should be configured to connect to the ElastiCache cluster endpoint, which will automatically point to the primary node after a failover.

Session Management: Avoid storing session data directly on EC2 instances if they are ephemeral. Use a shared session store like ElastiCache or a database table (though this can impact database performance) for session persistence across application instances.

By combining RDS Multi-AZ for database resilience, ELB and Auto Scaling Groups for application availability, and robust monitoring, you can architect a highly available and fault-tolerant WordPress deployment on AWS capable of automated failover.