Server Monitoring Best Practices: Keeping Your Laravel App and PostgreSQL Clusters Alive on DigitalOcean
Core Metrics for Laravel & PostgreSQL on DigitalOcean
Effective server monitoring hinges on tracking the right metrics. For a Laravel application backed by PostgreSQL on DigitalOcean, this means a multi-layered approach covering infrastructure, application performance, and database health. We’ll focus on actionable insights, not just raw data.
Infrastructure Monitoring with DigitalOcean Droplets
DigitalOcean’s built-in monitoring provides a good starting point, but it’s often insufficient for deep diagnostics. We need to augment this with agent-based monitoring for more granular control and custom alerting.
CPU Utilization
High CPU can indicate inefficient code, traffic spikes, or background processes gone rogue. We’ll monitor both overall CPU and per-process usage.
Memory Usage
Swapping to disk is a performance killer. Tracking available memory and swap usage is critical. We also need to watch for memory leaks within the Laravel application itself.
Disk I/O and Space
Slow disk I/O can bottleneck database operations and application responsiveness. Running out of disk space is a hard stop. Monitoring read/write operations per second and free disk space is essential.
Network Traffic
Unusual spikes in inbound or outbound traffic can signal DDoS attacks, unexpected bot activity, or misconfigured services. Monitoring bandwidth usage helps in capacity planning and identifying anomalies.
Application Performance Monitoring (APM) for Laravel
Infrastructure metrics only tell part of the story. Understanding how your Laravel application is performing requires APM. While commercial solutions like New Relic or Datadog are powerful, we can achieve significant insights with open-source tools and custom instrumentation.
Request Latency
Tracking the average and percentile (e.g., 95th, 99th) response times for your API endpoints and web pages is paramount. High latency directly impacts user experience and can be caused by slow database queries, external API calls, or inefficient PHP code.
Error Rates
Monitoring PHP errors, exceptions, and HTTP status codes (especially 5xx errors) is non-negotiable. We need to capture stack traces and context for rapid debugging.
Queue Performance
Laravel’s queue system is vital for background jobs. Monitoring queue lengths, processing times, and failed jobs prevents backlogs and ensures critical tasks are completed.
PostgreSQL Cluster Health Monitoring
The database is often the heart of a web application. Proactive monitoring of your PostgreSQL cluster is crucial to prevent data corruption, performance degradation, and downtime.
Connection Usage
Tracking the number of active connections, idle connections, and the maximum allowed connections helps prevent “too many connections” errors. Understanding connection pooling is key here.
Query Performance
Slow queries are a primary cause of database performance issues. We need to identify and analyze long-running queries. Enabling the `pg_stat_statements` extension is a must.
Replication Lag (for HA setups)
If you’re running a PostgreSQL cluster with read replicas for high availability or load balancing, monitoring replication lag is critical. Significant lag means read replicas are out of sync, potentially serving stale data.
Disk Space and I/O for Data Directories
Similar to the OS level, but specifically for PostgreSQL’s data directories. High I/O wait times on the database disks can severely impact performance.
PostgreSQL Logs
Configuring PostgreSQL to log errors, slow queries, and deadlocks provides invaluable debugging information.
Implementing Monitoring: Tools and Techniques
We’ll explore practical implementation strategies using a combination of DigitalOcean’s features, open-source agents, and custom scripts.
Agent-Based Monitoring with Prometheus & Node Exporter
Prometheus is a powerful open-source monitoring and alerting system. Node Exporter provides hardware and OS metrics.
Installation on a Droplet (e.g., Ubuntu 22.04)
First, install Node Exporter on each Droplet you want to monitor.
Download and Install Node Exporter
Fetch the latest release from the official Prometheus GitHub repository.
wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz tar xvfz node_exporter-1.7.0.linux-amd64.tar.gz sudo mv node_exporter-1.7.0.linux-amd64/node_exporter /usr/local/bin/
Create a Systemd Service for Node Exporter
This ensures Node Exporter runs as a background service and restarts on boot.
sudo tee /etc/systemd/system/node_exporter.service <<EOF [Unit] Description=Node Exporter Wants=network-online.target After=network-online.target [Service] User=nobody Group=nobody Type=simple ExecStart=/usr/local/bin/node_exporter [Install] WantedBy=multi-user.target EOF
sudo systemctl daemon-reload sudo systemctl start node_exporter sudo systemctl enable node_exporter
Configuring Prometheus Server
You’ll need a separate Droplet (or a dedicated service) to run the Prometheus server. Install Prometheus similarly, then configure its prometheus.yml file to scrape your Node Exporter instances.
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
scrape_configs:
# Scrape Prometheus itself
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
# Scrape Node Exporter on your Laravel app Droplet
- job_name: 'laravel_app_node'
static_configs:
- targets: ['YOUR_LARAVEL_DROPLET_IP:9100'] # Replace with your Droplet's IP
# Scrape Node Exporter on your PostgreSQL Droplet(s)
- job_name: 'postgres_node'
static_configs:
- targets: ['YOUR_POSTGRES_DROPLET_IP_1:9100', 'YOUR_POSTGRES_DROPLET_IP_2:9100'] # Replace with your Droplet IPs
PostgreSQL Monitoring with Prometheus Exporter
The postgres_exporter is essential for collecting PostgreSQL-specific metrics.
Installation and Configuration
Download and run the exporter. You’ll need to provide database connection details.
# Download and install (example for Linux AMD64)
wget https://github.com/prometheus-community/postgres_exporter/releases/download/v0.13.0/postgres_exporter-v0.13.0.linux-amd64.tar.gz
tar xvfz postgres_exporter-v0.13.0.linux-amd64.tar.gz
sudo mv postgres_exporter-v0.13.0.linux-amd64/postgres_exporter /usr/local/bin/
# Create a PostgreSQL user for monitoring
sudo -u postgres psql -c "CREATE USER monitor WITH PASSWORD 'your_secure_password';"
sudo -u postgres psql -c "GRANT pg_read_all_stats TO monitor;" # Grants access to pg_stat_activity, pg_stat_database, etc.
sudo -u postgres psql -c "ALTER USER monitor CREATEDB;" # Required for pg_stat_statements if not already enabled
# Enable pg_stat_statements (if not already)
# Edit postgresql.conf (e.g., /etc/postgresql/14/main/postgresql.conf)
# shared_preload_libraries = 'pg_stat_statements'
# pg_stat_statements.track = all
# pg_stat_statements.max = 10000
# pg_stat_statements.track_utility = off
# Restart PostgreSQL after changes.
# Create a .pgpass file for the exporter user
echo "YOUR_POSTGRES_DROPLET_IP:5432:*:monitor:your_secure_password" | sudo tee -a ~/.pgpass
sudo chmod 600 ~/.pgpass
# Ensure this .pgpass is accessible by the user running postgres_exporter.
# If running as a systemd service, you might need to specify the path or run as a specific user.
# Create a Systemd Service for postgres_exporter
sudo tee /etc/systemd/system/postgres_exporter.service <<EOF
[Unit]
Description=PostgreSQL Prometheus Exporter
After=network.target
[Service]
User=postgres # Or a dedicated user that can access .pgpass
ExecStart=/usr/local/bin/postgres_exporter --web.listen-address=":9187" --extend.query-path=/etc/postgres_exporter/queries.yaml
Restart=always
[Install]
WantedBy=multi-user.target
EOF
# Create a custom queries.yaml for additional metrics (optional but recommended)
sudo mkdir -p /etc/postgres_exporter/
sudo tee /etc/postgres_exporter/queries.yaml <<EOF
# Example custom query for replication lag
replication_lag_seconds:
query: |
SELECT
COALESCE(pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn), 0) AS lag_bytes
FROM pg_stat_replication
WHERE application_name = 'your_replication_slot_name'; # Replace with your slot name
metrics:
- replication_lag_seconds{job="postgres"}
EOF
# Reload systemd, start, and enable the service
sudo systemctl daemon-reload
sudo systemctl start postgres_exporter
sudo systemctl enable postgres_exporter
Adding to Prometheus Configuration
Update your prometheus.yml on the Prometheus server to scrape the PostgreSQL exporter.
scrape_configs:
# ... other jobs ...
- job_name: 'postgres_exporter'
static_configs:
- targets: ['YOUR_POSTGRES_DROPLET_IP:9187'] # Replace with your PostgreSQL Droplet IP
Laravel APM with OpenTelemetry and Jaeger
OpenTelemetry provides a vendor-neutral way to instrument your application. We can send traces to Jaeger for visualization.
Instrumenting Laravel with OpenTelemetry PHP SDK
Install the necessary packages via Composer.
composer require open-telemetry/sdk composer require open-telemetry/opentelemetry-auto-psr18 composer require open-telemetry/exporter-otlp
Create a service provider to initialize the SDK and configure the exporter. This example sends traces to a local Jaeger agent (running on port 6831 UDP).
// app/Providers/OpenTelemetryServiceProvider.php
namespace App\Providers;
use Illuminate\Support\ServiceProvider;
use OpenTelemetry\API\Trace\TracerProviderInterface;
use OpenTelemetry\SDK\Trace\TracerProvider;
use OpenTelemetry\SDK\Trace\SpanProcessor\BatchSpanProcessor;
use OpenTelemetry\SDK\Trace\Sampler\ParentBased;
use OpenTelemetry\SDK\Trace\Sampler\TraceIdRatioSampler;
use OpenTelemetry\SDK\Common\Export\Http\Psr7HttpClientFactory;
use OpenTelemetry\SDK\Trace\SpanExporter\OtlpExporter;
use OpenTelemetry\Context\Propagation\TextMapPropagatorInterface;
use OpenTelemetry\API\Globals;
use OpenTelemetry\Context\Context;
use OpenTelemetry\API\Trace\Span;
use OpenTelemetry\API\Trace\SpanContext;
use OpenTelemetry\API\Trace\SpanKind;
use OpenTelemetry\API\Trace\StatusCode;
use OpenTelemetry\API\Trace\Attributes;
use OpenTelemetry\API\Trace\NoopTracerProvider;
use OpenTelemetry\API\Trace\SpanBuilderFactory;
use OpenTelemetry\API\Trace\TracerInterface;
class OpenTelemetryServiceProvider extends ServiceProvider
{
/**
* Register services.
*
* @return void
*/
public function register()
{
$this->app->singleton(TracerProviderInterface::class, function ($app) {
// Configure the sampler (e.g., sample 1% of traces)
$sampler = new ParentBased(new TraceIdRatioSampler(0.01));
// Configure the exporter (e.g., OTLP to Jaeger agent)
// Jaeger agent listens on UDP port 6831 by default
$exporter = new OtlpExporter('udp://127.0.0.1:6831');
// Create a batch span processor
$spanProcessor = new BatchSpanProcessor($exporter);
// Create the tracer provider
$tracerProvider = new TracerProvider($spanProcessor, $sampler);
// Set the global tracer provider
Globals::setTracerProvider($tracerProvider);
return $tracerProvider;
});
// Register a TracerInterface for easy access
$this->app->singleton(TracerInterface::class, function ($app) {
return $app->make(TracerProviderInterface::class)->getTracer('io.opentelemetry.php');
});
}
/**
* Bootstrap services.
*
* @return void
*/
public function boot()
{
// Optional: Auto-instrumentation for common frameworks/libraries
// This requires the opentelemetry-auto-psr18 package and potentially
// other auto-instrumentation packages for specific libraries.
// For more advanced auto-instrumentation, consider using the OpenTelemetry PHP AutoInstrumentation agent.
}
}
Register this service provider in config/app.php.
Manual Instrumentation Example (e.g., in a Controller)
You can manually create spans to track specific operations.
use OpenTelemetry\API\Trace\TracerInterface;
use OpenTelemetry\API\Trace\Span;
use OpenTelemetry\API\Trace\StatusCode;
class MyController extends Controller
{
protected $tracer;
public function __construct(TracerInterface $tracer)
{
$this->tracer = $tracer;
}
public function processData()
{
$span = $this->tracer->spanBuilder('process_data_operation')
->setSpanKind(SpanKind::INTERNAL)
->start();
try {
// Simulate some work
sleep(1);
// Example: Trace a database query (if not auto-instrumented)
$dbSpan = $this->tracer->spanBuilder('database_query.users_table')
->setSpanKind(SpanKind::CLIENT)
->setAttribute('db.system', 'postgresql')
->setAttribute('db.statement', 'SELECT * FROM users WHERE id = 1')
->start();
try {
// Execute query...
$dbSpan->setStatus(StatusCode::OK);
} catch (\Throwable $e) {
$dbSpan->recordException($e);
$dbSpan->setStatus(StatusCode::ERROR, $e->getMessage());
throw $e; // Re-throw to be caught by outer try-catch
} finally {
$dbSpan->end();
}
// Simulate another task
sleep(2);
$span->setStatus(StatusCode::OK);
} catch (\Throwable $e) {
$span->recordException($e);
$span->setStatus(StatusCode::ERROR, $e->getMessage());
throw $e;
} finally {
$span->end();
}
return response()->json(['message' => 'Data processed']);
}
}
Running Jaeger
You can run Jaeger locally using Docker for development or deploy it on a dedicated Droplet for production.
# For local development (includes agent, collector, query, and UI) docker run -d --name jaeger \ -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \ -p 5775:5775/udp \ -p 6831:6831/udp \ -p 6832:6832/udp \ -p 5778:5778 \ -p 16686:16686 \ -p 4317:4317 \ -p 4318:4318 \ -p 14250:14250 \ -p 14268:14268 \ -p 9411:9411 \ jaegertracing/all-in-one:latest
Access the Jaeger UI at http://localhost:16686.
Log Aggregation with ELK Stack or Loki
Centralized logging is crucial for debugging and auditing. For smaller setups, a simple approach might be sufficient, but for production, consider ELK (Elasticsearch, Logstash, Kibana) or Grafana Loki.
Filebeat for Log Shipping
Filebeat is a lightweight shipper that can send logs from your Laravel application and PostgreSQL server to a central logging system.
# Install Filebeat (example for Debian/Ubuntu)
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.10.2-amd64.deb
sudo dpkg -i filebeat-8.10.2-amd64.deb
# Configure Filebeat to tail Laravel logs (e.g., storage/logs/laravel.log)
# Edit /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /var/www/your-laravel-app/storage/logs/*.log
json.from_line: true # If your Laravel logs are JSON formatted
fields_under_root: true
fields:
environment: production
app_name: laravel-app
# Configure Filebeat to tail PostgreSQL logs (adjust path as needed)
- type: log
enabled: true
paths:
- /var/log/postgresql/postgresql-*.log
fields_under_root: true
fields:
environment: production
app_name: postgresql
# Configure output (e.g., to Elasticsearch or Logstash)
output.logstash:
hosts: ["YOUR_LOGSTASH_IP:5044"] # Or output.elasticsearch
Start and enable the Filebeat service.
sudo systemctl enable filebeat sudo systemctl start filebeat
Alerting with Alertmanager
Prometheus scrapes metrics, but Alertmanager handles alerts. It deduplicates, groups, and routes them to various receivers (email, Slack, PagerDuty).
Prometheus Alerting Rules
Define alerting rules in Prometheus (e.g., in a file like alerts.yml, then include it in prometheus.yml).
groups:
- name: laravel_postgres_alerts
rules:
- alert: HighCpuUsage
expr: node_cpu_seconds_total{mode="idle"} == 0
for: 5m
labels:
severity: critical
annotations:
summary: "High CPU usage detected on {{ $labels.instance }}"
description: "CPU usage on {{ $labels.instance }} is above 90% for the last 5 minutes."
- alert: HighMemoryUsage
expr: (node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes * 100 > 90
for: 5m
labels:
severity: warning
annotations:
summary: "High memory usage on {{ $labels.instance }}"
description: "Memory usage on {{ $labels.instance }} is above 90% for the last 5 minutes."
- alert: HighPostgresReplicationLag
expr: replication_lag_seconds > 60 # Assuming replication_lag_seconds metric is exposed
for: 2m
labels:
severity: critical
annotations:
summary: "PostgreSQL replication lag on {{ $labels.instance }}"
description: "Replication lag on {{ $labels.instance }} is {{ $value }} seconds, exceeding the 60-second threshold."
- alert: LowDiskSpace
expr: node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"} * 100 < 10
for: 10m
labels:
severity: warning
annotations:
summary: "Low disk space on {{ $labels.instance }}"
description: "Disk space on {{ $labels.instance }} is below 10% for the root filesystem."
Configuring Alertmanager
Set up Alertmanager to receive alerts from Prometheus and route them.
global:
resolve_timeout: 5m
route:
group_by: ['alertname', 'cluster', 'service']
group_wait: 30s
group_interval: 5m
repeat_interval: 4h
receiver: 'slack-notifications' # Default receiver
receivers:
- name: 'slack-notifications'
slack_configs:
- api_url: 'YOUR_SLACK_WEBHOOK_URL'
channel: '#alerts'
send_resolved: true
title: '{{ template "slack.default.title" . }}'
text: '{{ template "slack.default.text" . }}'
Configure Prometheus to send alerts to Alertmanager in its prometheus.yml:
alerting:
alertmanagers:
- static_configs:
- targets: ['YOUR_ALERTMANAGER_IP:9093'] # Replace with your Alertmanager IP
DigitalOcean Specific Considerations
Droplet Resource Allocation
Choose Droplet sizes that match your application’s needs. Monitor resource utilization closely and scale vertically (larger Droplets) or horizontally (more Droplets) as required. Use DigitalOcean’s monitoring graphs to identify trends.
Managed Databases
For PostgreSQL, DigitalOcean Managed Databases offer a simplified operational experience. They come with built-in monitoring and automated backups. If using Managed Databases, focus on application-level metrics and database-specific query performance, as infrastructure management is handled by DO.
Firewall Rules
Ensure your DigitalOcean Cloud Firewalls are configured to allow traffic only on necessary ports (e.g., 80, 443 for web, 5432 for PostgreSQL from your app servers, 9090 for Prometheus, 9100 for Node Exporter, 9187 for Postgres Exporter, 6831 for Jaeger agent).
Load Balancers
If using DigitalOcean Load Balancers, monitor their health checks and traffic distribution. Ensure they are correctly routing traffic to your healthy Laravel application Droplets.
Conclusion
A robust monitoring strategy for a Laravel and PostgreSQL stack on DigitalOcean involves a layered approach. By combining infrastructure metrics, application performance insights, and deep database health checks, and by leveraging tools like Prometheus, Node Exporter, Postgres Exporter, OpenTelemetry, and Filebeat, you can achieve high availability, identify performance bottlenecks proactively, and ensure a stable environment for your users.