Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » Server Monitoring Best Practices: Keeping Your WooCommerce App and DynamoDB Clusters Alive on AWS

Server Monitoring Best Practices: Keeping Your WooCommerce App and DynamoDB Clusters Alive on AWS

Establishing a Robust Monitoring Foundation with CloudWatch

For any mission-critical application, especially one as demanding as WooCommerce, a comprehensive monitoring strategy is paramount. On AWS, CloudWatch serves as the foundational service for collecting and tracking metrics, collecting log files, and setting alarms. For a WooCommerce application, we need to monitor not just the EC2 instances hosting the web servers and PHP-FPM, but also the underlying database (RDS or Aurora for relational data) and the critical session/cache layer (ElastiCache Redis). Furthermore, our DynamoDB clusters, often used for product catalogs, order metadata, or user sessions, require granular performance insights.

EC2 Instance Monitoring for WooCommerce Web/App Servers

The standard CloudWatch agent provides essential OS-level metrics. However, for deeper insights into PHP performance and web server load, we need to augment this. We’ll focus on CPU Utilization, Memory Utilization (requires the CloudWatch agent configured for detailed memory), Disk I/O, and Network In/Out. Crucially, we must also monitor application-specific metrics.

Configuring the CloudWatch Agent for Enhanced Metrics

The default CloudWatch agent configuration is often too basic. We need to enable detailed metrics and potentially custom metrics. For memory utilization, ensure the agent is configured to collect it. A common configuration file is /opt/aws/amazon-cloudwatch-agent/bin/config.json. Here’s a snippet demonstrating how to enable detailed metrics and memory collection:

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "cwagent"
  },
  "metrics": {
    "namespace": "WooCommerce/EC2",
    "metrics_collected": {
      "cpu": {
        "resources": [
          "*"
        ],
        "measurement": [
          "cpu_usage_idle",
          "cpu_usage_iowait",
          "cpu_usage_user",
          "cpu_usage_system"
        ],
        "totalcpu_time_metrics": true
      },
      "mem": {
        "measurement": [
          "mem_used_percent"
        ]
      },
      "disk": {
        "resources": [
          "/",
          "/var/log"
        ],
        "measurement": [
          "used_percent",
          "read_bytes",
          "write_bytes",
          "read_ops",
          "write_ops"
        ]
      },
      "net": {
        "resources": [
          "eth0"
        ],
        "measurement": [
          "bytes_sent",
          "bytes_recv",
          "packets_sent",
          "packets_recv"
        ]
      }
    }
  }
}

After updating the configuration, restart the agent:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

Monitoring Nginx and PHP-FPM Performance

For Nginx, we need to monitor request rates, error rates (4xx, 5xx), and request latency. PHP-FPM requires monitoring active processes, idle processes, and request counts. These are typically exposed via status pages or log files.

Nginx Metrics via `stub_status` and CloudWatch Agent

Enable the stub_status module in your Nginx configuration:

http {
    # ... other configurations
    server {
        listen 80;
        server_name your_domain.com;

        location /nginx_status {
            stub_status on;
            access_log off;
            allow 127.0.0.1; # Restrict access
            deny all;
        }

        # ... other locations
    }
}

Then, configure the CloudWatch agent to scrape this status page. Add a new section to your config.json:

    "nginx": {
      "metrics_collection_interval": 60,
      "log_collected": {
        "files": [
          "/var/log/nginx/access.log"
        ],
        "log_group_name": "WooCommerce/Nginx/Logs",
        "log_stream_name": "{instance_id}/nginx_access",
        "timezone": "UTC"
      },
      "metrics_collected": {
        "nginx_status": {
          "url": "http://localhost/nginx_status",
          "metrics": {
            "active_connections": {},
            "requests_per_second": {},
            "connections_accepted_per_second": {},
            "connections_handled_per_second": {},
            "requests_total": {}
          }
        }
      }
    }

PHP-FPM Metrics via `pm.status_path` and CloudWatch Agent

In your PHP-FPM pool configuration (e.g., /etc/php/7.4/fpm/pool.d/www.conf), enable the status page:

pm.status_path = /fpm_status
ping.path = /fpm_ping
ping.response = pong

You’ll need to configure your web server (Nginx in this case) to proxy requests to this status page. Add a location block to your Nginx configuration:

location ~ ^/(fpm_status|fpm_ping)$ {
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_pass unix:/var/run/php/php7.4-fpm.sock; # Adjust path as needed
    internal;
}

Then, add the PHP-FPM metrics collection to your CloudWatch agent config.json:

    "php_fpm": {
      "metrics_collection_interval": 60,
      "metrics_collected": {
        "php_fpm_status": {
          "url": "http://localhost/fpm_status",
          "metrics": {
            "accepted_conn": {},
            "active_processes": {},
            "idle_processes": {},
            "max_children": {},
            "max_requests": {},
            "requests_per_second": {},
            "slow_requests": {}
          }
        }
      }
    }

DynamoDB Cluster Monitoring: Performance and Cost Optimization

DynamoDB monitoring is critical for performance and cost management. Key metrics include ConsumedReadCapacityUnits, ConsumedWriteCapacityUnits, ThrottledRequests, SuccessfulRequestLatency, and SystemErrors. We’ll use CloudWatch alarms to proactively manage provisioned throughput and identify performance bottlenecks.

Key DynamoDB Metrics and Alarm Thresholds

For tables with provisioned throughput, we need to monitor the utilization of provisioned capacity. A common best practice is to set alarms when utilization consistently exceeds 70-80% to allow for scaling before throttling occurs. For on-demand tables, monitoring ThrottledRequests is crucial to understand if the current throughput is sufficient.

  • ConsumedReadCapacityUnits vs. ProvisionedReadCapacityUnits: Alarm if ConsumedReadCapacityUnits / ProvisionedReadCapacityUnits > 0.8 for 5 minutes.
  • ConsumedWriteCapacityUnits vs. ProvisionedWriteCapacityUnits: Alarm if ConsumedWriteCapacityUnits / ProvisionedWriteCapacityUnits > 0.8 for 5 minutes.
  • ThrottledRequests: Alarm if ThrottledRequests > 0 for 1 minute (especially for on-demand tables or as a secondary check for provisioned).
  • SuccessfulRequestLatency: Alarm if the 95th percentile latency exceeds 500ms for 5 minutes.
  • SystemErrors: Alarm if SystemErrors > 0 for 1 minute.

These alarms can be configured directly in the AWS Management Console under CloudWatch -> Alarms, or programmatically via AWS CLI or SDKs. For example, using the AWS CLI to create an alarm for read capacity utilization:

aws cloudwatch put-metric-alarm \
    --alarm-name "DynamoDB-HighReadCapacityUtilization-MyWooCommerceTable" \
    --alarm-description "Alarm when read capacity utilization exceeds 80% for 5 minutes" \
    --metric-name "ConsumedReadCapacityUnits" \
    --namespace "AWS/DynamoDB" \
    --statistic "Sum" \
    --period 300 \
    --threshold 80 \
    --comparison-operator "GreaterThanThreshold" \
    --dimensions "Name=TableName,Value=MyWooCommerceTable" "Name=Operation,Value=Scan" \
    --evaluation-periods 1 \
    --datapoints-to-alarm 1 \
    --treat-missing-data "notBreaching" \
    --alarm-actions arn:aws:sns:us-east-1:123456789012:MyMonitoringTopic

Note: The --threshold for ConsumedReadCapacityUnits needs to be calculated relative to the provisioned capacity. A more robust approach involves creating a composite alarm or using a Lambda function to dynamically compare consumed vs. provisioned. For simplicity here, we assume a direct comparison or that the threshold is adjusted based on provisioned capacity. A common pattern is to monitor the ProvisionedReadCapacityUnits metric and compare the ConsumedReadCapacityUnits against it.

Log Analysis for DynamoDB Insights

Enable DynamoDB’s detailed logging to CloudWatch Logs. This includes ExecutionLogs and AuditLogs. Analyze these logs for specific errors, slow queries (though DynamoDB doesn’t have “queries” in the traditional sense, it refers to scan/query operations), and access patterns. This is invaluable for debugging and understanding application behavior.

aws dynamodb update-table \
    --table-name MyWooCommerceTable \
    --stream-specification "StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES" \
    --billing-mode PROVISIONED \
    --provisioned-throughput ReadCapacityUnits=100,WriteCapacityUnits=50

To enable logging to CloudWatch Logs, you typically configure this at the table level or via the AWS SDK when creating/updating tables. Ensure the IAM role associated with your application or the CloudWatch agent has permissions to write to CloudWatch Logs.

Application-Level Monitoring with Custom Metrics

Beyond infrastructure and service metrics, application-level insights are crucial for a WooCommerce app. This includes tracking critical business events, API response times for external integrations, and queue lengths for background processing.

Custom WooCommerce Metrics with PHP and CloudWatch SDK

Use the AWS SDK for PHP to publish custom metrics to CloudWatch. This allows you to track things like new orders processed, failed order attempts, cache hit/miss ratios for product data, or the number of items in the WooCommerce background job queue.

<?php
require 'vendor/autoload.php'; // Assuming you use Composer

use Aws\CloudWatch\CloudWatchClient;
use Aws\Exception\AwsException;

$cloudwatch = new CloudWatchClient([
    'region' => 'us-east-1',
    'version' => 'latest'
]);

$namespace = 'WooCommerce/Application';
$instanceId = getenv('EC2_INSTANCE_ID') ?: 'local'; // Get EC2 instance ID or use a placeholder

// Example: Track new orders
function trackNewOrder($orderId) {
    global $cloudwatch, $namespace, $instanceId;
    try {
        $cloudwatch->putMetricData([
            'Namespace' => $namespace,
            'MetricData' => [
                [
                    'MetricName' => 'NewOrders',
                    'Dimensions' => [
                        ['Name' => 'InstanceId', 'Value' => $instanceId]
                    ],
                    'Value' => 1,
                    'Unit' => 'Count',
                    'Timestamp' => time()
                ],
            ],
        ]);
        error_log("Custom metric 'NewOrders' published for order: " . $orderId);
    } catch (AwsException $e) {
        error_log("Error publishing custom metric: " . $e->getMessage());
    }
}

// Example: Track cache hit ratio
function trackCacheHitRatio($hits, $misses) {
    global $cloudwatch, $namespace, $instanceId;
    $total = $hits + $misses;
    if ($total === 0) return;
    $hitRatio = ($hits / $total) * 100;

    try {
        $cloudwatch->putMetricData([
            'Namespace' => $namespace,
            'MetricData' => [
                [
                    'MetricName' => 'CacheHitRatio',
                    'Dimensions' => [
                        ['Name' => 'InstanceId', 'Value' => $instanceId]
                    ],
                    'Value' => $hitRatio,
                    'Unit' => 'Percent',
                    'Timestamp' => time()
                ],
            ],
        ]);
        error_log("Custom metric 'CacheHitRatio' published: " . $hitRatio . "%");
    } catch (AwsException $e) {
        error_log("Error publishing custom metric: " . $e->getMessage());
    }
}

// --- Usage within your WooCommerce application ---
// When a new order is successfully processed:
// trackNewOrder($order->get_id());

// Periodically, or after a batch of cache operations:
// trackCacheHitRatio($cacheHits, $cacheMisses);

?>

Ensure your EC2 instances have an IAM role attached with permissions for cloudwatch:PutMetricData. For local development or environments without an EC2 instance profile, configure AWS credentials via environment variables or a shared credentials file.

Centralized Logging with CloudWatch Logs and Log Insights

Effective log management is crucial for debugging and auditing. Configure the CloudWatch agent to stream Nginx access/error logs, PHP-FPM logs, and application logs to CloudWatch Logs. This provides a centralized, searchable repository.

Configuring Log Streaming

Add log file configurations to your CloudWatch agent config.json:

    "logs": {
      "logs_collected": {
        "files": {
          "collect_list": [
            {
              "file_path": "/var/log/nginx/access.log",
              "log_group_name": "WooCommerce/Nginx/Access",
              "log_stream_name": "{instance_id}/nginx_access"
            },
            {
              "file_path": "/var/log/nginx/error.log",
              "log_group_name": "WooCommerce/Nginx/Error",
              "log_stream_name": "{instance_id}/nginx_error"
            },
            {
              "file_path": "/var/log/php/php7.4-fpm.log",
              "log_group_name": "WooCommerce/PHP-FPM",
              "log_stream_name": "{instance_id}/php_fpm"
            },
            {
              "file_path": "/var/www/html/app/logs/system.log",
              "log_group_name": "WooCommerce/App/Logs",
              "log_stream_name": "{instance_id}/app_log"
            }
          ]
        }
      }
    }

Remember to restart the CloudWatch agent after updating the configuration.

Leveraging CloudWatch Log Insights for Analysis

CloudWatch Log Insights allows you to query your logs using a powerful query language. This is far more efficient than manually sifting through logs. For example, to find all 5xx errors from Nginx:

fields @timestamp, @message
| filter status >= 500
| sort @timestamp desc
| limit 20

To find slow PHP-FPM requests (assuming your logs include execution time):

fields @timestamp, @message
| parse @message "request_slow: * sec" as request_time
| filter request_time > 5
| sort @timestamp desc
| limit 20

You can also create dashboards from Log Insights queries to visualize log trends, such as the rate of specific errors or the distribution of request latencies.

Alerting Strategies and Best Practices

A robust alerting system is the culmination of your monitoring efforts. Alerts should be actionable, specific, and routed to the correct teams. Avoid alert fatigue by tuning thresholds and using composite alarms.

Actionable Alerting Scenarios

  • High CPU/Memory Utilization: Trigger an alarm when CPU or Memory utilization exceeds 85% for more than 10 minutes. This indicates potential performance degradation or a need for scaling.
  • High Error Rates (Nginx/PHP-FPM): Alarm on a sustained increase in 5xx errors (e.g., > 5 errors per minute over 5 minutes).
  • DynamoDB Throttling: As discussed, critical for performance.
  • Application Errors: Use custom metrics or log analysis to trigger alerts on critical application failures (e.g., payment gateway errors, critical API integration failures).
  • Disk Space Running Low: Alarm when disk utilization on critical partitions (e.g., /var/log, /var/www/html) exceeds 90%.
  • High Latency (Application/Database): Monitor custom application latency metrics or RDS/Aurora metrics for sustained increases in response times.

Utilizing SNS and Lambda for Alerting Workflows

AWS Simple Notification Service (SNS) is the backbone for delivering alerts. Alarms can publish to SNS topics, which can then fan out notifications to email, SMS, Slack (via Lambda integration), PagerDuty, etc. For more complex alerting logic (e.g., auto-scaling actions based on alarms, incident creation), AWS Lambda functions can subscribe to SNS topics.

# Example Lambda function triggered by an SNS topic for alert enrichment
import json
import boto3

sns = boto3.client('sns')

def lambda_handler(event, context):
    for record in event['Records']:
        message = json.loads(record['Sns']['Message'])
        alarm_name = message['AlarmName']
        new_state = message['NewStateValue']
        old_state = message['OldStateValue']
        alarm_description = message.get('AlarmDescription', 'No description')
        region = message['Region']
        account_id = message['AWSAccountId']

        print(f"Alarm '{alarm_name}' changed from {old_state} to {new_state} in {region} ({account_id})")
        print(f"Description: {alarm_description}")

        # Example: Send a more detailed Slack message (requires additional setup)
        # slack_message = {
        #     "text": f"🚨 *{new_state}:* {alarm_name}\n>{alarm_description}\n>Region: {region}"
        # }
        # sns.publish(TopicArn='arn:aws:sns:us-east-1:123456789012:SlackNotifications', Message=json.dumps(slack_message))

    return {
        'statusCode': 200,
        'body': json.dumps('Alert processed')
    }

By combining CloudWatch metrics, logs, custom application instrumentation, and a well-defined alerting strategy, you can build a resilient monitoring system that keeps your WooCommerce application and its underlying AWS infrastructure healthy and performant.