Server Monitoring Best Practices: Keeping Your Laravel App and MongoDB Clusters Alive on Google Cloud
Proactive Laravel Application Health Checks with Google Cloud Monitoring
Maintaining the health and performance of a Laravel application deployed on Google Cloud Platform (GCP) requires a multi-layered monitoring strategy. Beyond basic uptime checks, we need to delve into application-specific metrics and error reporting. Google Cloud Monitoring (formerly Stackdriver) offers a robust suite of tools, but effective utilization hinges on proper configuration and integration.
For Laravel, key areas to monitor include:
- Application error rates (PHP exceptions, fatal errors).
- Request latency and throughput.
- Queue worker status and job processing times.
- Database query performance.
- Resource utilization (CPU, memory, disk I/O) of the Compute Engine instances or GKE nodes.
Implementing Custom Metrics for Laravel Applications
Google Cloud Monitoring allows for the ingestion of custom metrics, which are invaluable for application-specific insights. We can leverage the Cloud Monitoring API or client libraries to send data points from within our Laravel application.
A common use case is tracking the number of failed jobs in Laravel’s queue. This can be done by creating a scheduled task that queries the queue driver (e.g., Redis, database) for failed jobs and then reports this count as a custom metric.
Scheduled Task for Queue Health
First, define a new command in your Laravel application:
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\DB;
use Google\Cloud\Monitoring\V3\MetricServiceClient;
use Google\Cloud\Monitoring\V3\MetricDescriptor\MetricKind;
use Google\Cloud\Monitoring\V3\MetricDescriptor\ValueType;
use Google\Cloud\Monitoring\V3\MonitoredResource;
use Google\Protobuf\Timestamp;
use DateTime;
class MonitorQueueHealth extends Command
{
/**
* The name and signature of the console command.
*
* @var string
*/
protected $signature = 'monitor:queue-health';
/**
* The console command description.
*
* @var string
*/
protected $description = 'Reports the number of failed queue jobs to Google Cloud Monitoring.';
/**
* Execute the console command.
*
* @return int
*/
public function handle()
{
// Ensure the GOOGLE_APPLICATION_CREDENTIALS environment variable is set
// or that the application is running with a service account that has
// the 'monitoring.metricWriter' role.
$metricService = new MetricServiceClient();
$projectId = env('GOOGLE_CLOUD_PROJECT');
// --- Count Failed Jobs (Example for Database Driver) ---
// Adjust this query if you are using a different queue driver (e.g., Redis)
$failedJobsCount = DB::table('failed_jobs')->count();
// --- Prepare Time Series Data ---
$timeSeries = [
'metric' => [
'type' => 'custom.googleapis.com/laravel/queue/failed_jobs',
'labels' => [
'environment' => env('APP_ENV', 'production'),
],
],
'resource' => MonitoredResource::build([
'type' => 'gce_instance', // Or 'k8s_container' if using GKE
'labels' => [
'project_id' => $projectId,
'instance_id' => gethostname(), // Or pod name/namespace for GKE
'zone' => env('GOOGLE_CLOUD_ZONE', 'us-central1-a'), // Adjust zone
],
]),
'points' => [
[
'interval' => [
'endTime' => new Timestamp(['seconds' => time()]),
],
'value' => [
'int64_value' => $failedJobsCount,
],
],
],
];
try {
$metricService->createTimeSeries($projectId, [$timeSeries]);
$this->info("Successfully reported {$failedJobsCount} failed jobs to Cloud Monitoring.");
} catch (\Exception $e) {
$this->error("Failed to report queue health: " . $e->getMessage());
// Optionally log this error to Cloud Logging as well
return 1;
}
return 0;
}
}
Next, register this command in your app/Console/Kernel.php:
// app/Console/Kernel.php
protected $commands = [
// ... other commands
\App\Console\Commands\MonitorQueueHealth::class,
];
protected function schedule(Schedule $schedule)
{
// Run every 5 minutes
$schedule->command('monitor:queue-health')->everyFiveMinutes();
}
Metric Descriptor Creation
Before sending data, the custom metric descriptor needs to exist. This can be done manually via the GCP Console or programmatically. For automation, use the MetricServiceClient:
<?php
use Google\Cloud\Monitoring\V3\MetricServiceClient;
use Google\Cloud\Monitoring\V3\MetricDescriptor;
use Google\Cloud\Monitoring\V3\MetricDescriptor\MetricKind;
use Google\Cloud\Monitoring\V3\MetricDescriptor\ValueType;
use Google\Api\MonitoredResourceDescriptor;
$metricService = new MetricServiceClient();
$projectId = env('GOOGLE_CLOUD_PROJECT');
$descriptor = new MetricDescriptor([
'type' => 'custom.googleapis.com/laravel/queue/failed_jobs',
'metric_kind' => MetricKind::GAUGE,
'value_type' => ValueType::INT64,
'description' => 'The number of failed jobs in the Laravel queue.',
'displayName' => 'Laravel Failed Jobs',
'labels' => [
[
'key' => 'environment',
'value_type' => ValueType::STRING,
'description' => 'The application environment (e.g., production, staging).',
],
],
]);
try {
// Check if descriptor already exists to avoid errors
$existingDescriptor = $metricService->getMetricDescriptor($projectId, 'custom.googleapis.com/laravel/queue/failed_jobs');
if (!$existingDescriptor) {
$metricService->createMetricDescriptor($projectId, $descriptor);
echo "Metric descriptor 'custom.googleapis.com/laravel/queue/failed_jobs' created successfully.\n";
} else {
echo "Metric descriptor 'custom.googleapis.com/laravel/queue/failed_jobs' already exists.\n";
}
} catch (\Exception $e) {
echo "Error creating metric descriptor: " . $e->getMessage() . "\n";
}
You can run this script once during your deployment process or as a separate setup task. Ensure the service account used has the monitoring.metricWriter and monitoring.metricDescriptorWriter roles.
Monitoring MongoDB Clusters with Google Cloud Monitoring
For MongoDB deployments, whether self-managed on Compute Engine or within a GKE cluster, monitoring is crucial. Google Cloud Monitoring can ingest metrics from MongoDB using the Ops Manager agent or by scraping Prometheus metrics if you’re using a Prometheus-compatible exporter.
Option 1: Using the Ops Manager Agent (for self-managed MongoDB)
If you’re running MongoDB on Compute Engine instances, the Ops Manager agent can be configured to export metrics to Cloud Monitoring. This typically involves:
- Installing the Ops Manager agent on each MongoDB server.
- Configuring the agent to collect relevant MongoDB metrics (e.g., connections, operations, latency, disk usage, replication status).
- Setting up the agent to forward these metrics to Google Cloud Monitoring. This often involves configuring the agent to use the Cloud Monitoring API endpoint.
The exact configuration details depend on the Ops Manager agent version and your specific MongoDB setup. Refer to the official MongoDB documentation for the most up-to-date instructions on agent configuration for cloud monitoring integrations.
Option 2: Prometheus Exporter and Prometheus-to-Cloud-Monitoring
If your MongoDB deployment is within GKE or you prefer a Prometheus-based approach, you can deploy a MongoDB Prometheus exporter. A popular choice is the mongodb_exporter.
1. Deploying mongodb_exporter:
# Example Kubernetes Deployment for mongodb_exporter
apiVersion: apps/v1
kind: Deployment
metadata:
name: mongodb-exporter
labels:
app: mongodb-exporter
spec:
replicas: 1
selector:
matchLabels:
app: mongodb-exporter
template:
metadata:
labels:
app: mongodb-exporter
spec:
containers:
- name: mongodb-exporter
image: percona/mongodb_exporter:latest # Or a specific version
ports:
- containerPort: 9274 # Default Prometheus port
env:
- name: MONGODB_URI
value: "mongodb://your_mongodb_user:your_mongodb_password@your_mongodb_host:27017/admin?authSource=admin" # Use secrets for credentials
# Add resource limits and requests
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
2. Configuring Prometheus to Scrape Metrics:
If you have Prometheus deployed in your GKE cluster (e.g., via Prometheus Operator), configure it to scrape the mongodb_exporter service. This typically involves creating a ServiceMonitor resource:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: mongodb-exporter
labels:
release: prometheus # Adjust to your Prometheus release name
spec:
selector:
matchLabels:
app: mongodb-exporter # Matches the labels on your mongodb-exporter deployment
namespaceSelector:
matchNames:
- default # Namespace where mongodb-exporter is deployed
endpoints:
- port: http # The name of the port in your mongodb-exporter service
interval: 30s
path: /metrics # The metrics endpoint
3. Forwarding Prometheus Metrics to Cloud Monitoring:
Google Cloud offers a managed solution for ingesting Prometheus metrics into Cloud Monitoring. This involves deploying the prometheus-to-sd (service discovery) agent. The agent can be configured to discover Prometheus targets and send their metrics to Cloud Monitoring.
# Example Kubernetes Deployment for prometheus-to-sd
apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus-to-sd
spec:
replicas: 1
selector:
matchLabels:
app: prometheus-to-sd
template:
metadata:
labels:
app: prometheus-to-sd
spec:
containers:
- name: prometheus-to-sd
image: google/prometheus-to-sd:v0.7.0 # Use a specific, stable version
args:
- "--project-id=YOUR_GCP_PROJECT_ID"
- "--source=kubernetes" # Or 'gce' if running on Compute Engine
- "--cluster-name=YOUR_GKE_CLUSTER_NAME" # If using Kubernetes source
- "--namespace=YOUR_NAMESPACE" # If using Kubernetes source
- "--port=8080" # The port prometheus-to-sd listens on for Prometheus scraping
# Add resource limits and requests
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
# Service account with 'monitoring.metricWriter' role is required
Ensure the Kubernetes service account used by the prometheus-to-sd deployment has the necessary IAM permissions (roles/monitoring.metricWriter) in your GCP project.
Alerting and Dashboards in Google Cloud Monitoring
Once metrics are flowing into Cloud Monitoring, the next step is to create meaningful alerts and dashboards. This transforms raw data into actionable insights.
Configuring Alerting Policies
Alerting policies are crucial for proactive issue detection. For our Laravel app and MongoDB clusters, consider alerts for:
- High Error Rate: Trigger an alert if the rate of application errors (e.g., logged via Cloud Logging or custom metrics) exceeds a threshold.
- High Latency: Alert on sustained high request latency for critical API endpoints.
- Queue Backlog: Notify if the number of pending queue jobs exceeds a defined limit for an extended period.
- Failed Jobs Spike: Alert immediately on a sudden increase in failed queue jobs.
- MongoDB Performance Degradation: Monitor key MongoDB metrics like slow operations, high connection counts, or replication lag.
- Resource Exhaustion: Standard alerts for high CPU, memory, or disk usage on your application servers and database instances.
When creating an alerting policy, specify the metric, the condition (e.g., “above threshold”), the duration, and the notification channel (e.g., email, PagerDuty, Slack via Pub/Sub). For custom metrics, ensure you select the correct metric type (e.g., custom.googleapis.com/laravel/queue/failed_jobs).
Building Custom Dashboards
Dashboards provide a consolidated view of your system’s health. Create dashboards that:
- Display key performance indicators (KPIs) for your Laravel application (request rate, error rate, latency percentiles).
- Visualize the health of your MongoDB clusters (connections, operations per second, replication status, disk usage).
- Show resource utilization for your compute instances or GKE nodes.
- Include charts for your custom metrics, like failed queue jobs over time.
Use the “Metrics Explorer” in Google Cloud Monitoring to find your metrics and then add them to a new dashboard. For MongoDB metrics ingested via Prometheus, they will appear with a prometheus/ prefix in their metric type.
Log-Based Metrics and Error Tracking
Leveraging Cloud Logging’s log-based metrics is another powerful way to monitor your Laravel application without modifying application code extensively. You can create metrics from log entries that indicate errors or specific events.
Creating Log-Based Metrics for Laravel Errors
If your Laravel application logs errors in a consistent format to Cloud Logging, you can create metrics based on these logs. For example, if your logs contain lines like:
[2023-10-27 10:30:00] local.ERROR: Uncaught Error: Call to a member function on null in /var/www/html/app/Http/Controllers/SomeController.php:55
You can create a log-based metric in Cloud Monitoring that counts occurrences of lines matching a specific filter, such as severity=ERROR or a more specific string match for your error format.
# Example filter for Cloud Logging log-based metric resource.type="gce_instance" logName="projects/YOUR_PROJECT_ID/logs/laravel.log" # Or wherever your Laravel logs are severity=ERROR
This allows you to create alerts based on error rates directly from your logs, complementing custom metrics and providing a broader view of application stability.
Conclusion
A robust server monitoring strategy for Laravel applications and MongoDB clusters on GCP involves a combination of infrastructure metrics, application-specific custom metrics, and log analysis. By proactively configuring Google Cloud Monitoring with custom metrics, log-based metrics, and well-defined alerting policies, you can ensure the stability, performance, and availability of your critical services.