Server Monitoring Best Practices: Keeping Your PHP App and MySQL Clusters Alive on Linode

Proactive PHP Application Health Checks

Beyond basic uptime checks, your PHP application needs granular health monitoring. This involves not just checking if the web server is responding, but if the application itself is functional and performing within acceptable parameters. A common approach is to implement a dedicated health check endpoint within your PHP application.

This endpoint should perform several critical checks:

• Database connectivity: Verify that the application can connect to its primary MySQL cluster.
• Cache connectivity: If using Redis or Memcached, confirm accessibility.
• Essential service dependencies: Check for any other critical external services.
• Internal application logic: A simple, fast check to ensure core components are initialized correctly.

Here’s a sample PHP health check script. Save this as /var/www/html/healthcheck.php (adjust path as needed) and ensure it’s accessible but not publicly browsable.

healthcheck.php Example

<?php
// Configuration
define('DB_HOST', '192.168.1.100'); // Replace with your MySQL master IP or hostname
define('DB_USER', 'healthcheck_user');
define('DB_PASS', 'secure_password');
define('DB_NAME', 'your_app_db');
define('CACHE_HOST', '192.168.1.101'); // Replace with your Redis/Memcached IP
define('CACHE_PORT', 6379); // Default Redis port

header('Content-Type: application/json');
$response = ['status' => 'error', 'checks' => []];

// --- Database Check ---
$db_check = ['name' => 'database', 'status' => 'fail', 'message' => ''];
try {
    $pdo = new PDO("mysql:host=" . DB_HOST . ";dbname=" . DB_NAME . ";charset=utf8mb4", DB_USER, DB_PASS, [
        PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
        PDO::ATTR_TIMEOUT => 2 // 2-second timeout
    ]);
    // Execute a simple query to ensure connection is active and functional
    $stmt = $pdo->query("SELECT 1");
    if ($stmt && $stmt->fetchColumn() === '1') {
        $db_check['status'] = 'ok';
        $db_check['message'] = 'Successfully connected to database.';
    } else {
        $db_check['message'] = 'Database query failed.';
    }
} catch (PDOException $e) {
    $db_check['message'] = 'Database connection error: ' . $e->getMessage();
}
$response['checks'][] = $db_check;

// --- Cache Check (Example for Redis) ---
$cache_check = ['name' => 'cache', 'status' => 'fail', 'message' => ''];
try {
    $redis = new Redis();
    if ($redis->connect(CACHE_HOST, CACHE_PORT, 1)) { // 1-second timeout
        if ($redis->ping() === '+PONG') {
            $cache_check['status'] = 'ok';
            $cache_check['message'] = 'Successfully connected to Redis.';
        } else {
            $cache_check['message'] = 'Redis PING failed.';
        }
    } else {
        $cache_check['message'] = 'Redis connection refused.';
    }
} catch (RedisException $e) {
    $cache_check['message'] = 'Redis connection error: ' . $e->getMessage();
}
$response['checks'][] = $cache_check;

// --- Application Logic Check (Example) ---
$app_logic_check = ['name' => 'app_logic', 'status' => 'fail', 'message' => ''];
try {
    // Simulate a quick check of a core component or configuration value
    if (defined('APP_VERSION') && !empty(APP_VERSION)) {
        $app_logic_check['status'] = 'ok';
        $app_logic_check['message'] = 'Core application components initialized.';
    } else {
        $app_logic_check['message'] = 'APP_VERSION not defined or empty.';
    }
} catch (Exception $e) {
    $app_logic_check['message'] = 'Application logic check failed: ' . $e->getMessage();
}
$response['checks'][] = $app_logic_check;

// Determine overall status
$all_ok = true;
foreach ($response['checks'] as $check) {
    if ($check['status'] === 'fail') {
        $all_ok = false;
        break;
    }
}

if ($all_ok) {
    $response['status'] = 'ok';
    http_response_code(200);
} else {
    http_response_code(503); // Service Unavailable
}

echo json_encode($response);
exit;
?>

Important Security Note: The database user (‘healthcheck_user’) should have minimal privileges, ideally only `SELECT` on a single, non-sensitive table or even just `SHOW DATABASES` if that’s sufficient for your needs. Restrict its access to only the necessary IP addresses.

To make this script more robust, you’d typically define APP_VERSION in your application’s bootstrap or configuration files.

External Monitoring with UptimeRobot/Prometheus Node Exporter

While the application-level health check is crucial, external monitoring provides an independent view. For simple uptime and response time checks, services like UptimeRobot are excellent. For more advanced metrics, integrating with Prometheus is the industry standard.

UptimeRobot Configuration

Configure UptimeRobot to monitor your healthcheck.php endpoint (e.g., https://yourdomain.com/healthcheck.php) as an HTTP(S) check. Set the expected response to contain "status": "ok". This ensures that not only is the server reachable, but your application’s health check logic is also passing.

Prometheus Integration with Node Exporter and Blackbox Exporter

For a more comprehensive solution, Prometheus is ideal. You’ll typically use:

• Node Exporter: To collect system-level metrics (CPU, RAM, disk, network) from your Linode instances.
• Blackbox Exporter: To probe your application endpoints (including the healthcheck.php) using protocols like HTTP, TCP, ICMP.

1. Install Node Exporter on each Linode instance:

wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
tar xvfz node_exporter-1.7.0.linux-amd64.tar.gz
sudo mv node_exporter-1.7.0.linux-amd64/node_exporter /usr/local/bin/
sudo useradd --no-create-home node_exporter
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Create a systemd service file (/etc/systemd/system/node_exporter.service):

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
ExecStart=/usr/local/bin/node_exporter --web.listen-address=0.0.0.0:9100

[Install]
WantedBy=multi-user.target

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable node_exporter
sudo systemctl start node_exporter
sudo systemctl status node_exporter

2. Install and Configure Blackbox Exporter:

Download and extract the Blackbox Exporter binary. Create a configuration file (e.g., blackbox.yml):

modules:
  http_2xx:
    prober: http
    timeout: 5s
    http:
      method: GET
      # Validate the response body for the health check status
      fail_if_not_equal: '{"status": "ok"}'
      # Optionally, check for specific headers
      # headers:
      #   Host: yourdomain.com
  # Add other modules as needed (e.g., tcp_connect for MySQL ports)

# Example for MySQL port check
#  mysql_tcp:
#    prober: tcp
#    timeout: 1s
#    tcp:
#      # Use the actual MySQL port
#      port: 3306

Run the Blackbox Exporter (adjust paths):

./blackbox_exporter --config.file=blackbox.yml --web.listen-address=0.0.0.0:9115

Set up a systemd service for Blackbox Exporter similarly to Node Exporter.

3. Configure Prometheus Scrape Jobs:

In your prometheus.yml, add scrape configurations for both Node Exporter and Blackbox Exporter:

scrape_configs:
  - job_name: 'node_exporter'
    static_configs:
      - targets: ['linode1_ip:9100', 'linode2_ip:9100'] # Replace with your Linode IPs

  - job_name: 'blackbox_http'
    metrics_path: /probe
    params:
      module: [http_2xx] # Use the module defined in blackbox.yml
    static_configs:
      - targets:
        - https://yourdomain.com/healthcheck.php # Target to probe
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: blackbox_exporter_ip:9115 # Replace with your Blackbox Exporter IP

  # Example for MySQL probe (if using tcp_connect module)
  # - job_name: 'blackbox_mysql_tcp'
  #   metrics_path: /probe
  #   params:
  #     module: [mysql_tcp]
  #   static_configs:
  #     - targets:
  #       - mysql_master_ip:3306
  #       - mysql_replica1_ip:3306
  #   relabel_configs:
  #     - source_labels: [__address__]
  #       target_label: __param_target
  #     - source_labels: [__param_target]
  #       target_label: instance
  #     - target_label: __address__
  #       replacement: blackbox_exporter_ip:9115

Reload Prometheus configuration after changes.

MySQL Cluster Monitoring: Percona Monitoring and Management (PMM)

For robust MySQL cluster monitoring, especially with replication, Percona Monitoring and Management (PMM) is a powerful, open-source solution. It provides deep insights into query performance, replication lag, server health, and more.

PMM Installation and Configuration

PMM typically runs as a Docker container or on a dedicated VM. The core components are the PMM Server and the PMM Client (which runs on your database nodes).

1. Deploy PMM Server (using Docker):

# Ensure Docker and Docker Compose are installed
docker run -d --name pmm-server \
  --restart always \
  -p 80:80 \
  -p 443:443 \
  -p 3306:3306 \
  -p 9000:9000 \
  -p 9003:9003 \
  -p 9009:9009 \
  -v /opt/pmm-server/data:/srv/data \
  -v /opt/pmm-server/configs:/etc/nginx/conf.d \
  percona/pmm-server:latest

Access the PMM Server UI at http://your_pmm_server_ip.

2. Install PMM Client on MySQL Nodes:

Follow the official Percona documentation for installing the PMM client agent on your Linode instances hosting MySQL. This typically involves downloading a package and running an installation command.

3. Add MySQL Instances to PMM:

From the PMM Server UI, navigate to ‘Add Service’ and select ‘MySQL’. You’ll need to provide:

• Service Name: A descriptive name (e.g., mysql_cluster_master).
• MySQL Username: A dedicated user for PMM with sufficient privileges (e.g., pmm_user).
• MySQL Password: The password for the PMM user.
• Advanced: Query Analytics Username/Password: If you want PMM to analyze slow queries.
• Service Address: The IP address and port of your MySQL instance (e.g., 192.168.1.100:3306).

Repeat this for all nodes in your MySQL cluster (master and replicas).

Key MySQL Metrics to Monitor with PMM

Focus on these critical metrics:

• Replication Lag: Essential for high availability. Monitor Seconds_Behind_Master for each replica. PMM visualizes this clearly.
• Query Performance: Identify slow queries, query throughput (QPS), and query latency. Use the ‘Query Analytics’ tab.
• Connections: Monitor Threads_connected, Threads_running, and connection errors.
• InnoDB Metrics: innodb_buffer_pool_read_requests vs. innodb_buffer_pool_reads (buffer pool hit rate), innodb_row_lock_waits, innodb_row_lock_time_avg.
• Disk I/O: Monitor read/write operations per second and latency.
• Network Traffic: Bandwidth usage between nodes, especially for replication.
• Error Logs: PMM can ingest and display MySQL error logs.

Alerting Strategies

Effective alerting is crucial to act on issues before they impact users. Integrate your monitoring tools with an alerting system.

Alertmanager (with Prometheus)

Prometheus integrates seamlessly with Alertmanager. Define alerting rules in Prometheus (e.g., in a separate alert.rules.yml file):

groups:
- name: php_app_alerts
  rules:
  - alert: HighHTTPErrors
    expr: rate(http_requests_total{code=~"5.."}[5m]) > 0.1 # More than 10% 5xx errors in 5 mins
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "High rate of HTTP 5xx errors detected on {{ $labels.instance }}"
      description: "The application is returning a high number of server errors (5xx)."

  - alert: AppHealthCheckFailed
    expr: probe_success{job="blackbox_http", module="http_2xx"} == 0
    for: 2m
    labels:
      severity: critical
    annotations:
      summary: "Application health check failed for {{ $labels.instance }}"
      description: "The /healthcheck.php endpoint returned an error or unexpected response."

- name: mysql_alerts
  rules:
  - alert: MySQLReplicationLagging
    expr: pmm_mysql_replication_lag_seconds > 60 # Replication lag is over 60 seconds
    for: 3m
    labels:
      severity: critical
    annotations:
      summary: "MySQL replication lag detected on {{ $labels.instance }}"
      description: "Replica {{ $labels.instance }} is lagging behind the master by more than 60 seconds."

  - alert: HighMySQLConnections
    expr: mysql_global_status_threads_connected > 500 # More than 500 active connections
    for: 10m
    labels:
      severity: warning
    annotations:
      summary: "High number of MySQL connections on {{ $labels.instance }}"
      description: "The MySQL server {{ $labels.instance }} has {{ $value }} active connections, approaching capacity."

Configure Alertmanager to route these alerts to Slack, PagerDuty, or email.

UptimeRobot Alerts

Configure UptimeRobot’s alert notifications to send alerts via webhook to a dedicated Slack channel or your incident management system.

Log Aggregation and Analysis

Centralized logging is indispensable for diagnosing issues. Use tools like the ELK stack (Elasticsearch, Logstash, Kibana) or Grafana Loki.

Ensure your PHP application logs errors, warnings, and critical events. Configure your web server (Nginx/Apache) to log access and error details. MySQL error logs should also be collected.

Forward these logs to your aggregation system using agents like Filebeat (for ELK) or Promtail (for Loki). This allows you to search, filter, and visualize logs across your entire infrastructure, correlating application behavior with system events.