Server Monitoring Best Practices: Keeping Your Laravel App and MongoDB Clusters Alive on AWS

Establishing a Robust Monitoring Foundation with AWS CloudWatch

For any production Laravel application hosted on AWS, a comprehensive monitoring strategy is non-negotiable. This begins with leveraging AWS CloudWatch, the cornerstone of AWS’s observability services. We’ll focus on key metrics for EC2 instances running your Laravel app and the MongoDB clusters managed by Amazon DocumentDB or self-hosted on EC2.

EC2 Instance Monitoring for Laravel Applications

Essential metrics for your Laravel application servers include CPU Utilization, Memory Utilization (requires the CloudWatch agent), Disk I/O, and Network In/Out. Setting up alarms on these metrics is crucial for proactive issue detection.

Configuring the CloudWatch Agent for Memory Metrics

By default, CloudWatch only collects CPU metrics from EC2. To get memory utilization, you need to install and configure the CloudWatch agent. This involves creating a configuration file and ensuring the agent runs as a service.

Agent Configuration File (amazon-cloudwatch-agent.json)

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "cwagent"
  },
  "metrics": {
    "namespace": "MyLaravelApp/EC2",
    "metrics_collected": {
      "mem": {
        "measurement": [
          "mem_used_percent"
        ],
        "total_mem": 0,
        "append_dimensions": {
          "InstanceId": "${aws:InstanceId}"
        }
      },
      "disk": {
        "measurement": [
          "used_percent"
        ],
        "resources": [
          "/"
        ],
        "append_dimensions": {
          "InstanceId": "${aws:InstanceId}"
        }
      }
    }
  }
}

Installation and Configuration Steps (Amazon Linux 2)

# Install the agent
sudo yum install amazon-cloudwatch-agent -y

# Copy the configuration file
sudo cp amazon-cloudwatch-agent.json /opt/aws/amazon-cloudwatch-agent/bin/config/amazon-cloudwatch-agent.json

# Start the agent
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config/amazon-cloudwatch-agent.json -s

# Verify agent status
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

Setting Up CloudWatch Alarms

Once metrics are flowing, configure alarms to notify your team. For instance, an alarm for high CPU utilization on your Laravel app servers.

aws cloudwatch put-metric-alarm \
    --alarm-name "HighCPUUtilization-LaravelApp" \
    --alarm-description "Alarm when CPU exceeds 80% for 5 minutes" \
    --metric-name CPUUtilization \
    --namespace AWS/EC2 \
    --statistic Average \
    --period 300 \
    --threshold 80 \
    --comparison-operator GreaterThanThreshold \
    --dimensions "Name=InstanceId,Value=i-0123456789abcdef0" \
    --evaluation-periods 1 \
    --datapoints-to-alarm 1 \
    --treat-missing-data notBreaching \
    --alarm-actions arn:aws:sns:us-east-1:123456789012:MyLaravelAlertsTopic

Monitoring Amazon DocumentDB (or MongoDB on EC2)

For your MongoDB clusters, whether using Amazon DocumentDB or self-managed instances on EC2, monitoring is critical for performance and availability. Key metrics include:

• CPU Utilization
• Memory Usage
• Disk Usage
• Network Throughput
• Read/Write Operations (Ops)
• Latency (Read/Write)
• Connections (Active/Available)
• Oplog Window (for replica sets)

DocumentDB Specific Metrics

DocumentDB exposes many metrics directly through CloudWatch under the AWS/DocDB namespace. You can view these in the AWS console or query them via the AWS CLI.

aws cloudwatch get-metric-statistics \
    --namespace AWS/DocDB \
    --metric-name CPUUtilization \
    --dimensions "Name=DBInstanceIdentifier,Value=my-docdb-instance" \
    --start-time 2023-10-27T00:00:00Z \
    --end-time 2023-10-27T01:00:00Z \
    --period 3600 \
    --statistics Average

Monitoring Self-Hosted MongoDB on EC2

If you’re running MongoDB on EC2, you’ll need to combine the EC2 CloudWatch agent (for system-level metrics) with MongoDB’s own monitoring tools and potentially custom CloudWatch metrics.

Leveraging `mongostat` and `mongotop`

These command-line utilities provide real-time insights into MongoDB performance. For production, you’ll want to script their output and push it to CloudWatch.

# Example: Capture mongostat output periodically
while true; do
    mongostat --host mongodb.example.com --port 27017 --username myuser --password mypassword --authenticationDatabase admin --rowcount 1 --json >> /var/log/mongodb/mongostat.log
    sleep 60
done

Custom CloudWatch Metrics for MongoDB

You can use the CloudWatch agent to push custom metrics derived from MongoDB’s `serverStatus` command or other sources. This requires a more advanced configuration of the agent.

{
  "agent": {
    "metrics_collection_interval": 60
  },
  "metrics": {
    "namespace": "MyMongoDB/EC2",
    "metrics_collected": {
      "exec": {
        "commands": [
          "mongo --host localhost --port 27017 -u myuser -p mypassword --authenticationDatabase admin --eval 'db.serverStatus().connections' --quiet"
        ],
        "name": "mongodb_connections",
        "timeout": 5,
        "append_dimensions": {
          "InstanceId": "${aws:InstanceId}"
        }
      }
    }
  }
}

Application-Level Monitoring with Prometheus and Grafana

While CloudWatch provides excellent infrastructure-level visibility, application-specific metrics are crucial for deep-diving into Laravel performance. Prometheus, coupled with Grafana for visualization, is a powerful combination.

Instrumenting Laravel Applications

You can expose Prometheus metrics from your Laravel application by creating custom middleware or using existing packages. A common approach is to track request durations, queue job processing times, and error rates.

Example: Laravel Prometheus Exporter Middleware

<?php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Prometheus\CollectorRegistry;
use Prometheus\Render\CallbackRenderer;
use Prometheus\Storage\InMemory;

class PrometheusMetrics
{
    protected $registry;

    public function __construct()
    {
        // Use a persistent storage for production, e.g., Redis
        $adapter = new InMemory();
        $this->registry = new CollectorRegistry($adapter);

        // Initialize metrics if they don't exist
        $this->requestCounter = $this->registry->registerCounter(
            'laravel', 'http_requests_total', 'Total HTTP requests', ['method', 'path', 'status']
        );
        $this->requestDuration = $this->registry->registerHistogram(
            'laravel', 'http_request_duration_seconds', 'HTTP request duration in seconds', ['method', 'path']
        );
    }

    public function handle(Request $request, Closure $next)
    {
        if ($request->is('metrics')) {
            $renderer = new CallbackRenderer($this->registry);
            return response($renderer->render(), 200, ['Content-Type' => $renderer->getMimeType()]);
        }

        $start = microtime(true);
        $response = $next($request);
        $duration = microtime(true) - $start;

        $method = $request->method();
        $path = $request->path();
        $status = $response->status();

        $this->requestCounter->inc([$method, $path, $status]);
        $this->requestDuration->observe($duration, [$method, $path]);

        return $response;
    }
}

Register this middleware in app/Http/Kernel.php and ensure a route /metrics is defined to expose the metrics endpoint.

Setting Up Prometheus Server and Grafana

Deploy Prometheus on a dedicated EC2 instance or within a container orchestration system (like EKS). Configure Prometheus to scrape metrics from your Laravel application instances and your database instances (if using MongoDB on EC2, you can use the `mongodb_exporter`).

# prometheus.yml
scrape_configs:
  - job_name: 'laravel_app'
    static_configs:
      - targets: ['app-server-1.example.com:9000', 'app-server-2.example.com:9000'] # Port where your Laravel app exposes metrics

  - job_name: 'mongodb'
    static_configs:
      - targets: ['mongodb-exporter.example.com:9274'] # If using mongodb_exporter

Then, configure Grafana to connect to your Prometheus data source and import or create dashboards to visualize these metrics.

Log Aggregation and Analysis with AWS CloudWatch Logs and Elasticsearch/OpenSearch

Centralized logging is indispensable for debugging and auditing. AWS CloudWatch Logs is a good starting point, but for advanced analysis, consider integrating with Elasticsearch or OpenSearch.

Sending Laravel Logs to CloudWatch Logs

Configure your Laravel application to use the CloudWatch Logs Monolog handler. This requires installing the AWS SDK for PHP and the Monolog CloudWatch handler.

composer require aws/aws-sdk-php monolog/monolog aws/aws-sdk-php-monolog

<?php

// config/logging.php

'channels' => [
    // ... other channels
    'cloudwatch' => [
        'driver' => 'monolog',
        'handler' => Monolog\Handler\StreamHandler::class,
        'with' => [
            'stream' => env('CLOUDWATCH_LOG_GROUP', 'my-laravel-app-logs'),
            'level' => 'debug',
        ],
        'processors' => [
            Monolog\Processor\WebProcessor::class,
            Monolog\Processor\LogTzProcessor::class,
            Monolog\Processor\UidProcessor::class,
            Monolog\Processor\IntrospectionProcessor::class,
            Monolog\Processor\MemoryUsageProcessor::class,
            Monolog\Processor\EnvironmentProcessor::class,
            Monolog\Processor\GitProcessor::class,
            Monolog\Processor\ProcessIdProcessor::class,
            Monolog\Processor\PsrLogMessageProcessor::class,
        ],
        'formatter' => Monolog\Formatter\JsonFormatter::class,
    ],
],

'default' => env('LOG_CHANNEL', 'stack'),

// In your .env file:
// CLOUDWATCH_LOG_GROUP=your-log-group-name
// AWS_ACCESS_KEY_ID=YOUR_KEY
// AWS_SECRET_ACCESS_KEY=YOUR_SECRET
// AWS_REGION=your-aws-region

Then, set your default log channel to cloudwatch in config/logging.php or use it selectively.

Advanced Log Analysis with OpenSearch/Elasticsearch

For large-scale applications, CloudWatch Logs can become expensive and less flexible for complex queries. AWS OpenSearch Service (or self-hosted Elasticsearch) offers powerful full-text search and analytics capabilities.

Data Flow: Laravel -> Fluentd/Logstash -> OpenSearch

You can use Fluentd or Logstash agents running on your EC2 instances to collect logs from files (e.g., Laravel’s storage/logs/laravel.log) or application output and forward them to an OpenSearch cluster.

# Example Fluentd configuration (fluentd.conf)
<source>
  @type tail
  path /var/www/html/storage/logs/laravel.log
  pos_file /var/log/fluentd/laravel.log.pos
  tag laravel.app
  <parse>
    @type json # If using JSON formatter in Monolog
  </parse>
</source>

<match laravel.app>
  @type opensearch # or elasticsearch
  host your-opensearch-domain.region.es.amazonaws.com
  port 443
  logstash_format true
  logstash_prefix laravel-app
  include_tag_key true
  tag_key @log_name
  flush_interval 5s
  request_timeout 5s
  ssl_verify false # Set to true in production
  # Add authentication details if required
</match>

Proactive Health Checks and Synthetic Monitoring

Beyond infrastructure and application metrics, actively checking the health of your application endpoints and critical background jobs is vital. AWS offers services like Route 53 Health Checks and CloudWatch Synthetics.

Route 53 Health Checks

Configure Route 53 health checks to ping a specific endpoint on your Laravel application (e.g., /health) at regular intervals. If the endpoint fails to respond correctly, Route 53 can trigger alarms and potentially reroute traffic away from unhealthy instances.

CloudWatch Synthetics Canaries

Canaries allow you to run scripts (written in Node.js or Python) from various AWS regions to simulate user interactions with your application. This is excellent for testing critical user flows, API endpoints, and even background job processing triggers.

// Example CloudWatch Synthetics Canary (Node.js)
exports.handler = async () => {
    const synthetics = require('Synthetics');
    const log = require('SyntheticsLogger');

    const pageURL = 'https://your-laravel-app.com/api/v1/health'; // Your health check endpoint

    const requestOptions = {
        url: pageURL,
        method: 'GET',
        timeout: 20000 // 20 seconds
    };

    try {
        const response = await synthetics.executeHttpRequest(requestOptions);
        log.info('Health check response:', response);

        if (response.statusCode !== 200) {
            throw new Error(`Health check failed with status code: ${response.statusCode}`);
        }

        // Add more assertions here based on expected response body
        // const body = JSON.parse(response.body);
        // if (body.status !== 'ok') {
        //     throw new Error('Health check reported status not OK');
        // }

        synthetics.appendRunnableOutput('Health check passed.');
    } catch (error) {
        log.error('Health check failed:', error);
        throw error; // This will mark the canary run as FAILED
    }
};

Database Connection Pooling and Performance Tuning

Database performance is often the bottleneck. For MongoDB, ensure your application is using connection pooling effectively. For self-hosted MongoDB, monitor connection counts and idle connections.

Laravel’s MongoDB Connection Management

When using packages like jenssegers/mongodb, connection pooling is typically handled automatically. However, monitor the number of active connections reported by MongoDB.

Tuning MongoDB/DocumentDB Parameters

Key parameters to monitor and potentially tune include:

• maxPoolSize (client-side): Ensure it’s adequate for your application’s concurrency needs.
• maxIncomingConnections (server-side, for self-hosted MongoDB): Controls the maximum number of client connections.
• wiredTigerCacheSize (self-hosted MongoDB): Crucial for performance; ensure it’s appropriately sized.

For DocumentDB, AWS manages many of these underlying parameters, but monitoring the exposed metrics is still key.

Alerting Strategy and Incident Response

A sophisticated monitoring setup is only effective if it leads to timely and appropriate action. Define clear alerting rules and an incident response plan.

Alerting Tiers and Notification Channels

Categorize alerts by severity (e.g., Critical, Warning, Info). Use different notification channels for different tiers: PagerDuty or Opsgenie for critical alerts, Slack or email for warnings.

Runbooks and Automated Remediation

For common issues (e.g., high CPU, disk full), create runbooks that detail the steps to diagnose and resolve the problem. Where possible, automate remediation actions using AWS Lambda functions triggered by CloudWatch alarms.

Conclusion: A Multi-Layered Approach

Effectively monitoring a Laravel application and its MongoDB cluster on AWS requires a multi-layered strategy. This involves leveraging AWS native services like CloudWatch for infrastructure, integrating application performance monitoring (APM) tools or Prometheus/Grafana for code-level insights, and implementing robust logging and alerting. Continuous refinement based on observed patterns and incident post-mortems is key to maintaining a highly available and performant system.