Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » Server Monitoring Best Practices: Keeping Your WooCommerce App and MongoDB Clusters Alive on AWS

Server Monitoring Best Practices: Keeping Your WooCommerce App and MongoDB Clusters Alive on AWS

Establishing a Robust Monitoring Foundation with AWS CloudWatch

For any production WooCommerce application hosted on AWS, a comprehensive monitoring strategy is paramount. This begins with leveraging AWS CloudWatch effectively. We’ll focus on key metrics for both EC2 instances running your WooCommerce stack and the MongoDB clusters powering your data. The goal is to move beyond basic CPU and memory utilization to actionable insights that prevent outages and performance degradation.

Monitoring EC2 Instances for WooCommerce

Your WooCommerce application likely runs on EC2 instances. Beyond the default CloudWatch metrics, we need to capture application-specific performance indicators and system-level details that directly impact user experience. This involves installing the CloudWatch Agent and configuring custom metrics.

Installing and Configuring the CloudWatch Agent

The CloudWatch Agent allows you to collect system-level metrics (like disk I/O, network traffic beyond basic stats) and log files from your EC2 instances. For a Debian/Ubuntu based system:

First, download and install the agent:

wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent_latest_amd64.deb
sudo dpkg -i amazon-cloudwatch-agent_latest_amd64.deb

Next, create a configuration file. A common location is /opt/aws/amazon-cloudwatch-agent/bin/config.json. This configuration defines which metrics to collect and which log files to stream to CloudWatch Logs.

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "cwagent"
  },
  "metrics": {
    "namespace": "WooCommerce/EC2",
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}"
    },
    "aggregation_dimensions": [
      [ "InstanceId" ]
    ],
    "metrics_collected": {
      "disk": {
        "measurement": [
          "used_percent",
          "free",
          "total",
          "inodes_free",
          "inodes_used_percent"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "/"
        ]
      },
      "mem": {
        "measurement": [
          "mem_used_percent",
          "mem_available_percent",
          "swap_used_percent"
        ],
        "metrics_collection_interval": 60
      },
      "net": {
        "measurement": [
          "bytes_sent",
          "bytes_recv",
          "packets_sent",
          "packets_recv"
        ],
        "metrics_collection_interval": 60
      },
      "statsd": {
        "service_address": "udp:localhost:8125",
        "metrics_collection_interval": 60
      }
    }
  },
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/apache2/access.log",
            "log_group_name": "WooCommerce/Apache/Access",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/var/log/apache2/error.log",
            "log_group_name": "WooCommerce/Apache/Error",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/var/log/php/error.log",
            "log_group_name": "WooCommerce/PHP/Error",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/var/log/woocommerce/app.log",
            "log_group_name": "WooCommerce/App/Logs",
            "log_stream_name": "{instance_id}"
          }
        ]
      }
    }
  }
}

Start the agent with this configuration:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

Crucially, you need to enable StatsD to collect custom application metrics. This involves running a StatsD daemon (like the one bundled with the agent or a separate Node.js instance) and instrumenting your PHP application. For PHP, libraries like php-statsd can be used.

Custom PHP Application Metrics with StatsD

Instrumenting your WooCommerce PHP code to send custom metrics is vital. Consider tracking:

  • Request latency (e.g., for product page loads, checkout process steps).
  • Number of active sessions.
  • Cache hit/miss ratios.
  • Error counts (e.g., WooCommerce-specific exceptions).
  • Database query times (if not already covered by MongoDB metrics).

Example using a hypothetical StatsD client in PHP:

<?php
// Assume $statsdClient is an instance of your StatsD client, configured for localhost:8125

// Track product page load time
$startTime = microtime(true);
// ... load product data ...
$endTime = microtime(true);
$durationMs = ($endTime - $startTime) * 1000;
$statsdClient->timing('woocommerce.product.page_load_time', $durationMs);

// Increment a counter for successful API calls
// ... perform API call ...
if ($success) {
    $statsdClient->increment('woocommerce.api.calls.success');
} else {
    $statsdClient->increment('woocommerce.api.calls.failed');
}

// Track cache hit rate
if ($cacheHit) {
    $statsdClient->increment('woocommerce.cache.hits');
} else {
    $statsdClient->increment('woocommerce.cache.misses');
}
?>

Ensure your StatsD daemon is running and configured to listen on UDP port 8125. The CloudWatch Agent will then pick up these metrics and send them to CloudWatch under the `WooCommerce/EC2` namespace.

Monitoring MongoDB Clusters on AWS DocumentDB or EC2

For MongoDB, whether you’re using AWS DocumentDB or self-managed instances on EC2, monitoring is critical for database performance and availability. DocumentDB offers CloudWatch integration out-of-the-box, while self-managed instances require more manual setup.

Monitoring AWS DocumentDB

DocumentDB automatically publishes metrics to CloudWatch. Key metrics to monitor include:

  • CPUUtilization: Percentage of allocated compute capacity used.
  • DatabaseConnections: Number of active client connections.
  • ReadIOPS and WriteIOPS: Input/Output Operations Per Second.
  • ReadLatency and WriteLatency: Average latency for read and write operations.
  • FreeStorageSpace: Amount of free storage available.
  • NetworkReceiveThroughput and NetworkTransmitThroughput: Network traffic.
  • ReplicationLag: Lag between primary and replica instances (crucial for high availability).

You can create CloudWatch Alarms based on these metrics. For example, an alarm for high replication lag:

aws cloudwatch put-metric-alarm \
    --alarm-name "DocumentDB-High-Replication-Lag" \
    --alarm-description "Alert when DocumentDB replication lag exceeds 5 seconds" \
    --metric-name ReplicationLag \
    --namespace "AWS/DocDB" \
    --statistic Average \
    --period 300 \
    --threshold 5 \
    --comparison-operator GreaterThanThreshold \
    --dimensions InstanceId=your-docdb-instance-id \
    --evaluation-periods 2 \
    --datapoints-to-alarm 2 \
    --treat-missing-data notBreaching \
    --alarm-actions arn:aws:sns:your-region:your-account-id:your-sns-topic

Remember to replace placeholders like your-docdb-instance-id, your-region, your-account-id, and your-sns-topic with your actual values.

Monitoring Self-Managed MongoDB on EC2

For MongoDB instances running on EC2, you’ll combine the CloudWatch Agent with MongoDB’s built-in tools and potentially third-party agents. The CloudWatch Agent can collect system metrics as described earlier, but we need specific MongoDB metrics.

Collecting MongoDB Performance Metrics

The mongostat and mongotop utilities provide real-time insights. To send these to CloudWatch, you can write a script that periodically runs these commands and pushes the data as custom metrics.

Example script (monitor_mongodb.sh) to collect key metrics and send them to CloudWatch using the AWS CLI:

#!/bin/bash

MONGO_HOST="localhost"
MONGO_PORT="27017"
NAMESPACE="WooCommerce/MongoDB"
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)

# Get mongostat output for key metrics
STATS=$(mongostat --host $MONGO_HOST --port $MONGO_PORT --noheaders 1 1 | awk '{print $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16}')

# Parse mongostat output
# Example: inserts, query, update, delete, get_more, command, dirty, used, nread, nwritten, flushes, locks, idx_miss, qr|qw, ar|aw, netIn|netOut
read INSERT_OPS QUERY_OPS UPDATE_OPS DELETE_OPS GET_MORE_OPS COMMAND_OPS DIRTY_PAGES USED_PAGES NREAD_OPS NWRITTEN_OPS FLUSH_OPS LOCK_PERCENTAGE IDX_MISS_RATE QR_QW AR_AW NET_IN NET_OUT <<< "$STATS"

# Send metrics to CloudWatch
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "InsertOps" --dimensions "InstanceId=$INSTANCE_ID" --value $INSERT_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "QueryOps" --dimensions "InstanceId=$INSTANCE_ID" --value $QUERY_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "UpdateOps" --dimensions "InstanceId=$INSTANCE_ID" --value $UPDATE_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "DeleteOps" --dimensions "InstanceId=$INSTANCE_ID" --value $DELETE_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "GetMoreOps" --dimensions "InstanceId=$INSTANCE_ID" --value $GET_MORE_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "CommandOps" --dimensions "InstanceId=$INSTANCE_ID" --value $COMMAND_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "DirtyPages" --dimensions "InstanceId=$INSTANCE_ID" --value $DIRTY_PAGES --unit Percent
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "UsedPages" --dimensions "InstanceId=$INSTANCE_ID" --value $USED_PAGES --unit Percent
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "NReadOps" --dimensions "InstanceId=$INSTANCE_ID" --value $NREAD_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "NWrittenOps" --dimensions "InstanceId=$INSTANCE_ID" --value $NWRITTEN_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "FlushOps" --dimensions "InstanceId=$INSTANCE_ID" --value $FLUSH_OPS --unit Count
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "LockPercentage" --dimensions "InstanceId=$INSTANCE_ID" --value $LOCK_PERCENTAGE --unit Percent
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "IndexMissRatio" --dimensions "InstanceId=$INSTANCE_ID" --value $IDX_MISS_RATE --unit Percent
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "NetworkIn" --dimensions "InstanceId=$INSTANCE_ID" --value $NET_IN --unit Bytes
aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "NetworkOut" --dimensions "InstanceId=$INSTANCE_ID" --value $NET_OUT --unit Bytes

# You can add mongotop metrics similarly, e.g., for read/write lock times
# Example: mongotop --host $MONGO_HOST --port $MONGO_PORT --quiet 1 1 | awk '{print $2, $3}'
# read READ_LOCK_MS WRITE_LOCK_MS <<< $(mongotop --host $MONGO_HOST --port $MONGO_PORT --quiet 1 1 | awk '{print $2, $3}')
# aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "ReadLockMs" --dimensions "InstanceId=$INSTANCE_ID" --value $READ_LOCK_MS --unit Milliseconds
# aws cloudwatch put-metric-data --namespace "$NAMESPACE" --metric-name "WriteLockMs" --dimensions "InstanceId=$INSTANCE_ID" --value $WRITE_LOCK_MS --unit Milliseconds

echo "MongoDB metrics pushed to CloudWatch."
exit 0

This script needs to be scheduled to run periodically (e.g., every minute) using cron. Ensure the EC2 instance profile has permissions to execute cloudwatch:PutMetricData.

Collecting MongoDB Logs

Configure MongoDB to log slow queries and general errors. Then, use the CloudWatch Agent to stream these logs. Edit your mongod.conf (typically in /etc/mongod.conf) to include:

systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true
  timeStampFormat: iso8601
operationProfiling:
  slowOpThresholdMs: 100 # Log queries taking longer than 100ms
  mode: slowOp
net:
  bindIp: 0.0.0.0
  port: 27017

Then, add the MongoDB log file to your CloudWatch Agent configuration (/opt/aws/amazon-cloudwatch-agent/bin/config.json):

    "logs": {
      "logs_collected": {
        "files": {
          "collect_list": [
            // ... other logs ...
            {
              "file_path": "/var/log/mongodb/mongod.log",
              "log_group_name": "WooCommerce/MongoDB/Logs",
              "log_stream_name": "{instance_id}"
            }
          ]
        }
      }
    }

Restart the CloudWatch Agent after updating its configuration.

Advanced Monitoring: Application Performance Monitoring (APM) and Tracing

While CloudWatch provides excellent infrastructure and log monitoring, true application performance insights often require dedicated APM tools. For a WooCommerce application, understanding the performance of individual requests across different services (PHP, database, external APIs) is crucial.

Integrating with AWS X-Ray

AWS X-Ray allows you to trace requests as they travel through your application and across distributed services. This is invaluable for identifying bottlenecks in complex WooCommerce workflows, such as the checkout process or product search.

To integrate X-Ray with PHP, you'll need the X-Ray SDK for PHP. This involves:

  • Installing the SDK via Composer: composer require aws/aws-xray-sdk-php
  • Configuring the X-Ray daemon to run on your EC2 instances.
  • Instrumenting your PHP code to create segments and subsegments for different operations.
<?php
require 'vendor/autoload.php';

use Aws\XRay\XRayClient;
use Aws\XRay\Exception\SegmentNotFoundException;

// Initialize X-Ray client (defaults to daemon address)
$xrayClient = new XRayClient([
    'region' => 'your-region', // e.g., 'us-east-1'
    'version' => 'latest'
]);

// Start a new segment for the request
$segment = $xrayClient->beginSegment('WooCommerceRequest');

try {
    // Instrument specific operations
    $subsegment = $xrayClient->beginSubsegment('LoadProductData');
    // ... load product data from MongoDB ...
    $xrayClient->endSubsegment();

    $subsegment = $xrayClient->beginSubsegment('ProcessPayment');
    // ... process payment ...
    $xrayClient->endSubsegment();

    // Example of tracing a database call (if using a compatible driver or manual instrumentation)
    // $dbSubsegment = $xrayClient->beginSubsegment('MongoDBQuery');
    // ... execute MongoDB query ...
    // $xrayClient->endSubsegment($dbSubsegment);

    // End the main segment
    $xrayClient->endSegment();

} catch (SegmentNotFoundException $e) {
    // Handle cases where segment might not be found (e.g., during error handling)
    error_log("X-Ray Segment Not Found: " . $e->getMessage());
    $xrayClient->endSegment(); // Attempt to close segment gracefully
} catch (\Exception $e) {
    // Record any exceptions
    $xrayClient->addException($e);
    $xrayClient->endSegment(); // Ensure segment is closed on error
    throw $e; // Re-throw the exception
}
?>

The X-Ray daemon typically runs on UDP port 2000. Ensure your EC2 security groups allow outbound UDP traffic to port 2000 for the daemon to communicate with the X-Ray service. For DocumentDB, X-Ray integration is more direct if you're using compatible drivers or SDKs.

Alerting and Incident Response

Effective monitoring is incomplete without a robust alerting and incident response strategy. CloudWatch Alarms are your primary tool here.

Key CloudWatch Alarms for WooCommerce and MongoDB

  • EC2 Instance Health: High CPU utilization (e.g., > 80% for 15 mins), low disk space (e.g., < 10% free), high network in/out.
  • Application Errors: High rate of errors in PHP logs (e.g., `WooCommerce/PHP/Error` log group), high HTTP 5xx error rates from ELB/ALB.
  • Database Performance: High MongoDB read/write latency, high connection counts, low free storage (for DocumentDB), high replication lag.
  • Custom Application Metrics: High request latency for critical WooCommerce paths (e.g., checkout), low cache hit rates.

Configure these alarms to trigger notifications via Amazon SNS to email, Slack (via Lambda integration), or PagerDuty.

Log Analysis for Proactive Issue Detection

Beyond simple error logging, use CloudWatch Logs Insights to query your logs for patterns that might indicate impending issues. For example, searching for repeated specific error messages or unusual request patterns across your Apache access logs.

fields @timestamp, @message
| filter @logStream like /your-instance-id/
| filter @message like /SpecificWooCommerceError/
| stats count() by bin(5m), @message
| sort @timestamp desc

Regularly review these queries and consider turning them into scheduled CloudWatch Alarms if they indicate a recurring problem.

Conclusion

A multi-layered monitoring approach is essential for maintaining a healthy and performant WooCommerce application on AWS. By combining CloudWatch's infrastructure and log monitoring with custom application metrics via StatsD, detailed database insights for MongoDB (or DocumentDB), and advanced tracing with AWS X-Ray, you gain the visibility needed to proactively manage your environment, prevent outages, and ensure a seamless experience for your customers.