Disaster Recovery 101: Architecting Auto-Failovers for MongoDB and WordPress Deployments on DigitalOcean

Establishing a MongoDB Replica Set for High Availability

A robust disaster recovery strategy for MongoDB hinges on a properly configured replica set. This ensures data redundancy and automatic failover in case of node failure. We’ll focus on a three-node replica set deployed across different DigitalOcean availability zones for maximum resilience.

First, ensure MongoDB is installed on your chosen Droplets. For this example, we’ll assume three Droplets: `mongo-primary` (primary), `mongo-secondary-1`, and `mongo-secondary-2`. Each Droplet will run a MongoDB instance.

MongoDB Configuration File (`mongod.conf`)

On each MongoDB server, the configuration file (`/etc/mongod.conf`) needs to be adjusted. Key parameters include `replication.replSetName`, `net.bindIp`, and `net.port`.

On `mongo-primary`:

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true
net:
  bindIp: 0.0.0.0
  port: 27017
security:
  authorization: enabled
replication:
  replSetName: rs0
processManagement:
  fork: true
  pidFilePath: /var/run/mongodb/mongod.pid

On `mongo-secondary-1` and `mongo-secondary-2`, the configuration is identical, except for `net.bindIp` which should also be `0.0.0.0` to allow connections from other nodes in the replica set.

Initializing the Replica Set

After configuring and starting the MongoDB service on all nodes (e.g., `sudo systemctl start mongod`), connect to the primary node and initiate the replica set configuration.

Connect to `mongo-primary` via the mongo shell:

mongo --port 27017 --host mongo-primary

Inside the mongo shell, initiate the replica set:

rs.initiate(
  {
    _id: "rs0",
    members: [
      { _id: 0, host: "mongo-primary:27017" },
      { _id: 1, host: "mongo-secondary-1:27017" },
      { _id: 2, host: "mongo-secondary-2:27017" }
    ]
  }
)

You can verify the replica set status with `rs.status()` in the mongo shell. The output should indicate that all members are healthy and the primary is correctly elected.

Architecting WordPress for High Availability with MongoDB

For WordPress, we’ll leverage a separate set of Droplets for the web servers and a managed DigitalOcean Managed Database for MongoDB. This separation simplifies management and leverages DigitalOcean’s HA capabilities for the database layer.

WordPress Web Server Configuration

We’ll deploy at least two WordPress web servers for redundancy. These servers will connect to the DigitalOcean Managed MongoDB cluster. For load balancing, we’ll use a DigitalOcean Load Balancer.

The critical part is configuring WordPress to use MongoDB as its database. This is achieved using a plugin like “MongoDB for WordPress” or by directly modifying WordPress’s `wp-config.php` if you’re using a custom solution that integrates with MongoDB.

Assuming you’re using a plugin, the configuration will typically involve providing the connection string to your DigitalOcean Managed MongoDB cluster. The connection string will look something like this:

mongodb://user:password@your-do-mongodb-cluster-hostname:27017/your_database_name?replicaSet=rs0&authSource=admin

Ensure your WordPress web servers have network access to the DigitalOcean Managed MongoDB cluster. This usually involves configuring firewall rules or VPC peering if your Droplets are not in the same VPC as the database cluster.

DigitalOcean Load Balancer Setup

A DigitalOcean Load Balancer will distribute incoming HTTP/S traffic across your WordPress web servers. Configure it with the following:

Frontend Protocol: HTTP (or HTTPS if you have SSL termination configured)
Frontend Port: 80 (or 443)
Backend Protocol: HTTP
Backend Port: 80
Health Check: Configure a health check to ping a specific URL on your WordPress site (e.g., `/healthz.php`). This file should return a 200 OK status if WordPress is responsive.
Droplets: Add your WordPress web server Droplets to the load balancer pool.

The health check is crucial for automatic failover. If a WordPress Droplet becomes unresponsive, the load balancer will automatically stop sending traffic to it.

Automating Failover and Monitoring

True disaster recovery involves automated failover and robust monitoring. For our MongoDB replica set, failover is largely handled by MongoDB itself. For the WordPress layer, we rely on the load balancer and potentially external monitoring tools.

MongoDB Failover

If the primary MongoDB node fails, the remaining secondary nodes will automatically elect a new primary. This process is typically very fast, often within seconds. Applications connected to the replica set will experience a brief interruption as the driver reconnects to the new primary.

To test this, you can shut down the `mongo-primary` Droplet. Observe the `rs.status()` output on one of the secondaries to see the election process. Your WordPress application should automatically reconnect to the new primary.

WordPress Web Server Failover

The DigitalOcean Load Balancer handles WordPress web server failover. When a Droplet fails its health check, the load balancer removes it from the active pool. Traffic is then directed to the remaining healthy Droplets.

To simulate a WordPress web server failure, stop the web server service (e.g., `sudo systemctl stop apache2` or `sudo systemctl stop nginx`) on one of your WordPress Droplets. Monitor the load balancer’s status to see the Droplet being marked as unhealthy and removed from rotation.

Monitoring and Alerting

Beyond the built-in health checks, implement comprehensive monitoring:

DigitalOcean Monitoring: Utilize DigitalOcean’s built-in Droplet and Load Balancer monitoring for CPU, memory, disk I/O, and network traffic.
External Monitoring Tools: Integrate with services like Prometheus, Grafana, or Datadog. Configure exporters for MongoDB (e.g., `mongodb_exporter`) and your web servers.
Alerting: Set up alerts for critical metrics:

MongoDB replica set health (e.g., `rs.status().ok` is not 1).
High latency on MongoDB queries.
WordPress health check failures.
Droplet resource utilization exceeding thresholds.

Alerting should be configured to notify your operations team immediately via email, Slack, or PagerDuty.

Backup and Recovery Strategy

While replication provides high availability, it’s not a substitute for backups. Implement a robust backup strategy:

MongoDB Backups: For DigitalOcean Managed Databases, leverage their automated backup features. For self-hosted replica sets, use `mongodump` with appropriate scheduling and offsite storage.
WordPress Files: Regularly back up your WordPress `wp-content` directory and database (if not solely relying on MongoDB). DigitalOcean’s Droplet snapshots can be a convenient option for full system backups.

Test your restore procedures regularly to ensure they are effective.