Disaster Recovery 101: Architecting Auto-Failovers for MongoDB and Magento 2 Deployments on Google Cloud
Automated MongoDB Failover with Replica Sets and GCP Compute Engine
Achieving automated failover for MongoDB in a production environment, especially when coupled with a demanding application like Magento 2, necessitates a robust replica set configuration. This isn’t merely about having multiple instances; it’s about intelligent health checking, automatic election of a new primary, and ensuring minimal data loss. On Google Cloud Platform (GCP), this translates to leveraging Compute Engine instances, persistent disks, and GCP’s networking capabilities to create a resilient MongoDB cluster.
A standard MongoDB replica set consists of at least three members: one primary and two or more secondaries. For high availability, we’ll deploy these across different GCP zones within the same region. This mitigates the risk of an entire zone outage affecting the database. Each member will have its own persistent disk for data storage, ensuring data durability independent of the underlying VM instance.
GCP Instance and Disk Configuration
We’ll provision Compute Engine instances with sufficient CPU and RAM for MongoDB. For persistent storage, we’ll use SSD persistent disks for optimal performance. The key is to ensure each MongoDB instance has a dedicated disk, mounted consistently across reboots and potential instance replacements.
Here’s a sample `gcloud` command to create a Compute Engine instance and attach an SSD persistent disk:
# Create the first MongoDB instance (e.g., mongo-01)
gcloud compute instances create mongo-01 \
--zone=us-central1-a \
--machine-type=n1-standard-4 \
--image-family=ubuntu-2004-lts \
--image-project=ubuntu-os-cloud \
--boot-disk-size=50GB \
--boot-disk-type=pd-ssd \
--tags=mongodb
# Create a persistent SSD disk for data
gcloud compute disks create mongo-data-01 \
--zone=us-central1-a \
--size=500GB \
--type=pd-ssd
# Attach the data disk to the instance
gcloud compute instances attach-disk mongo-01 \
--zone=us-central1-a \
--disk=mongo-data-01 \
--device-name=mongo-data-01
Repeat this process for `mongo-02` and `mongo-03`, ensuring they are in different zones (e.g., `us-central1-b`, `us-central1-c`) but the same region. The `–tags=mongodb` is crucial for firewall rules.
MongoDB Installation and Replica Set Initialization
On each instance, install MongoDB. Ensure you configure MongoDB to listen on its private IP address and to allow connections from other replica set members. The `mongod.conf` file is central to this.
On each instance, after mounting the data disk (e.g., to `/mnt/mongodb/data`):
# Example for Ubuntu/Debian sudo apt-get update sudo apt-get install -y mongodb # Configure mongod.conf sudo nano /etc/mongod.conf # Inside mongod.conf, ensure these settings: # net: # port: 27017 # bindIp: 0.0.0.0 # Or specific private IPs of your MongoDB nodes # # replication: # replSetName: "rs0" # Your replica set name # Create data directory and set permissions sudo mkdir -p /mnt/mongodb/data sudo chown -R mongodb:mongodb /mnt/mongodb/data sudo chmod -R 755 /mnt/mongodb/data # Ensure mongod uses the correct data directory # In mongod.conf, set: # storage: # dbPath: /mnt/mongodb/data # Restart MongoDB service sudo systemctl restart mongod sudo systemctl enable mongod
Once MongoDB is running on all instances, connect to one of them (e.g., `mongo-01`) and initiate the replica set.
# Connect to mongo-01
mongo --host mongo-01.your-internal-dns.local
# Inside the mongo shell:
rs.initiate(
{
_id : "rs0",
members: [
{ _id: 0, host: "mongo-01.your-internal-dns.local:27017" },
{ _id: 1, host: "mongo-02.your-internal-dns.local:27017" },
{ _id: 2, host: "mongo-03.your-internal-dns.local:27017" }
]
}
)
# Verify replica set status
rs.status()
Replace `.your-internal-dns.local` with your actual internal DNS names or IP addresses. GCP’s internal DNS resolution should handle this if instances are in the same VPC network.
GCP Firewall Configuration
You need to allow traffic between your MongoDB instances on port 27017 and allow SSH for management. Use GCP firewall rules targeting the `mongodb` tag.
# Allow MongoDB traffic within the VPC (from instances with the 'mongodb' tag)
gcloud compute firewall-rules create allow-mongodb-internal \
--allow tcp:27017 \
--source-tags=mongodb \
--target-tags=mongodb \
--description="Allow MongoDB traffic between replica set members"
# Allow SSH from your management IP range (or bastion host)
gcloud compute firewall-rules create allow-ssh-management \
--allow tcp:22 \
--source-ranges=YOUR_MGMT_IP_RANGE/32 \
--target-tags=mongodb \
--description="Allow SSH access for MongoDB management"
Magento 2 Application Configuration for Failover
Magento 2’s database connection configuration is defined in `app/etc/env.php`. To leverage MongoDB replica sets, you need to specify the replica set name and provide a connection string that includes all members.
<?php
return [
'modules' => [
// ... other module configurations
],
'db' => [
'connection' => [
'default' => [
'host' => 'mongo-01.your-internal-dns.local', // Initial connection point
'dbname' => 'magento_prod',
'username' => 'magento_user',
'password' => 'your_secure_password',
'model' => 'mysqlAggr', // This is a placeholder, Magento uses its own adapter
'initStatements' => 'SET NAMES utf8',
'options' => [
'driver' => 'pdo_mysql', // This is also a placeholder for the adapter type
// MongoDB specific options
'replicaSet' => 'rs0', // Crucial for failover
'readPreference' => 'secondaryPreferred', // Or 'primary', 'primaryPreferred', 'secondary', 'nearest'
'connectTimeoutMS' => 5000,
'socketTimeoutMS' => 5000,
'authSource' => 'admin', // If auth is in admin db
],
// For MongoDB adapter, the structure is different.
// Magento 2 uses a custom adapter that maps to MongoDB driver.
// The actual connection string will be constructed by the adapter.
// The key is 'replicaSet' and providing multiple hosts if needed.
// A more direct representation for the adapter might look like:
'host' => 'mongodb://magento_user:[email protected]:27017,mongo-02.your-internal-dns.local:27017,mongo-03.your-internal-dns.local:27017/?replicaSet=rs0&readPreference=secondaryPreferred&authSource=admin'
],
],
// ... other db configurations
],
// ... other configurations
];
?>
Important Note on Magento 2 and MongoDB Adapter: Magento 2’s default database connection is MySQL. To use MongoDB, you need a specific MongoDB adapter for Magento 2 (e.g., from a third-party extension or a custom implementation). The `env.php` structure above is illustrative. The actual adapter will parse the connection parameters. The critical parameters for failover are `replicaSet` and potentially `readPreference`. The adapter will construct a MongoDB connection string, often similar to the example provided in the `host` parameter.
Ensure your Magento 2 application servers can reach the MongoDB instances via their private IPs. This typically means they are in the same VPC network, and appropriate firewall rules are in place to allow traffic from the application servers to the MongoDB instances on port 27017.
Testing Failover
To test the automated failover, you can simulate a primary node failure:
- Connect to the current primary MongoDB instance.
- Stop the `mongod` service: `sudo systemctl stop mongod`.
- Observe the replica set status from a secondary node: `rs.status()`. You should see a new primary elected within seconds.
- Check your Magento 2 application to ensure it can still connect and perform operations.
For a more controlled test, you can also step down the primary:
# Connect to the primary mongo --host mongo-primary.your-internal-dns.local # Inside the mongo shell: rs.stepDown()
Automated Application Failover with GCP Load Balancing and Health Checks
For Magento 2, achieving automated application failover involves ensuring that if an application server instance becomes unhealthy, traffic is automatically routed to healthy instances. This is a multi-layered approach, combining GCP’s robust load balancing with application-level health checks.
GCP Load Balancer Setup
We’ll use a GCP HTTP(S) Load Balancer. This provides a single, stable IP address for your Magento 2 deployment and distributes traffic across your application server instances. The load balancer will perform health checks to determine which instances are available to serve traffic.
The core components are:
- Instance Group: A managed instance group (MIG) that automatically scales and manages your Magento 2 application servers.
- Health Check: A GCP health check that probes your application instances.
- Backend Service: Configured to use the instance group and health check.
- URL Map: Directs traffic to the backend service.
- Target Proxy: For HTTP or HTTPS.
- Forwarding Rule: The public IP address that clients connect to, directing traffic to the target proxy.
Here’s a simplified `gcloud` command sequence to set up a basic HTTP load balancer:
# 1. Create a Managed Instance Group (MIG)
# Assuming you have an instance template configured for Magento 2
gcloud compute instance-groups managed create magento-mig \
--template=magento-instance-template \
--size=2 \
--zone=us-central1-a \
--region=us-central1 \
--initial-delay=300s # Give instances time to start up
# 2. Create a Health Check
gcloud compute health-checks create http magento-health-check \
--request-path=/healthz \
--port=80 \
--check-interval=10s \
--timeout=5s \
--unhealthy-threshold=3 \
--healthy-threshold=2
# 3. Create a Backend Service
gcloud compute backend-services create magento-backend-service \
--protocol=HTTP \
--port-name=http \
--health-checks=magento-health-check \
--timeout=30s \
--enable-cdn \
--region=us-central1
# 4. Add the MIG to the Backend Service
gcloud compute backend-services add-backend magento-backend-service \
--instance-group=magento-mig \
--instance-group-zone=us-central1-a \
--region=us-central1
# 5. Create a URL Map
gcloud compute url-maps create magento-url-map \
--default-service=magento-backend-service
# 6. Create a Target HTTP Proxy
gcloud compute target-http-proxies create magento-http-proxy \
--url-map=magento-url-map
# 7. Create a Global Forwarding Rule (this assigns your public IP)
gcloud compute forwarding-rules create magento-forwarding-rule \
--global \
--target-http-proxy=magento-http-proxy \
--ports=80 \
--address=YOUR_RESERVED_STATIC_IP_ADDRESS # Ensure you have a static IP reserved
For HTTPS, you would create a Target HTTPS Proxy and associate an SSL certificate.
Application Health Check Endpoint
The GCP health check needs a specific endpoint on your Magento 2 application that can quickly and reliably report the application’s health. This endpoint should check critical dependencies, most importantly, the database connection.
Create a custom module in Magento 2 to implement this. The endpoint should return an HTTP 200 OK status if healthy, and a non-200 status (e.g., 503 Service Unavailable) if unhealthy.
// app/code/YourVendor/HealthCheck/Controller/Index/Index.php
namespace YourVendor\HealthCheck\Controller\Index;
use Magento\Framework\App\Action\Action;
use Magento\Framework\App\Action\Context;
use Magento\Framework\App\ResponseInterface;
use Magento\Framework\Controller\ResultInterface;
use Magento\Framework\App\ResourceConnection; // For DB check
class Index extends Action
{
protected $resourceConnection;
public function __construct(
Context $context,
ResourceConnection $resourceConnection
) {
parent::__construct($context);
$this->resourceConnection = $resourceConnection;
}
/**
* Execute action
*
* @return \Magento\Framework\Controller\ResultInterface
*/
public function execute(): ResultInterface
{
$response = $this->getResponse();
$response->setNoCacheHeaders();
try {
// Attempt to connect to the database and run a simple query
$connection = $this->resourceConnection->getConnection();
$connection->fetchOne('SELECT 1'); // Simple query
// If we reach here, DB is accessible
$response->setStatusCode(200);
$response->setContent('OK');
} catch (\Exception $e) {
// Database or other critical service is down
$response->setStatusCode(503);
$response->setContent('Service Unavailable');
// Log the error for debugging
// $this->logger->critical('Health check failed: ' . $e->getMessage());
}
return $response;
}
}
Ensure this controller is mapped to `/healthz` in your `etc/frontend/routes.xml` and `etc/web.xml` (for dispatching). After deploying the module (`bin/magento setup:upgrade`, `bin/magento cache:flush`), the `/healthz` endpoint should be accessible.
Testing Application Failover
To test the application failover:
- Identify an instance in your MIG that is currently serving traffic.
- Stop the web server (e.g., Apache or Nginx) on that instance: `sudo systemctl stop apache2` or `sudo systemctl stop nginx`.
- Wait for the GCP health check interval (e.g., 10 seconds) plus the unhealthy threshold (e.g., 3 checks * 10s = 30s).
- The GCP load balancer should detect the instance as unhealthy and stop sending traffic to it.
- Access your Magento 2 site through the load balancer’s IP address. You should still be able to browse and interact with the site, served by the remaining healthy instances.
- Restart the web server on the previously unhealthy instance. After a few successful health checks, it should be added back into the load balancer’s rotation.
This combination of automated MongoDB replica set failover and GCP Load Balancer with application health checks provides a robust disaster recovery strategy for your Magento 2 deployment, ensuring high availability and minimal downtime.