Vengala Vinay

Having 12+ Years of Experience in Software Development

Home » Server Monitoring Best Practices: Keeping Your Ruby App and Redis Clusters Alive on DigitalOcean

Server Monitoring Best Practices: Keeping Your Ruby App and Redis Clusters Alive on DigitalOcean

Establishing a Baseline: Essential Metrics for Ruby Apps and Redis

Effective server monitoring begins with a deep understanding of what constitutes “normal” for your specific application stack. For a Ruby on Rails application, this means tracking not just CPU and memory, but also application-specific metrics that indicate performance bottlenecks or impending failures. Similarly, Redis, while often perceived as simple, has critical operational metrics that must be observed.

Monitoring Ruby Application Performance

A robust monitoring strategy for a Ruby application on DigitalOcean should encompass:

  • Request Latency: The time it takes for your application to respond to an incoming HTTP request. High latency directly impacts user experience.
  • Throughput (Requests Per Second): The number of requests your application can handle within a given time frame.
  • Error Rates: The percentage of requests that result in errors (e.g., 5xx HTTP status codes).
  • Database Query Performance: Slow database queries are a common culprit for application slowdowns.
  • Memory Usage (RSS & Heap): Ruby’s garbage collector can be sensitive to memory pressure.
  • CPU Utilization: While important, this should be viewed in conjunction with other metrics. High CPU with low throughput is a red flag.

Tools like New Relic, Datadog, or even open-source solutions like Prometheus with the `ruby-prometheus-client` gem can provide these insights. For a self-hosted Prometheus setup, consider the following:

Prometheus Exporter for Ruby Apps

You’ll need to instrument your Ruby application to expose metrics. A common approach is to use a Rack middleware.

Example: Rack Middleware for Prometheus Metrics

Add the `prometheus_client` gem to your `Gemfile`:

Gemfile Snippet

gem 'prometheus_client'

Then, create a Rack middleware to expose metrics. If you’re using Rails, this can be placed in an initializer.

`config/initializers/prometheus.rb`

require 'prometheus_client'
require 'prometheus_client/middleware'

# Initialize Prometheus Client
PrometheusClient.configure do |config|
  config.logger = Rails.logger
end

# Define custom metrics
# Request duration in seconds
REQUEST_DURATION = PrometheusClient::Histogram.new(
  :http_requests_duration_seconds,
  'HTTP request duration in seconds'
)

# Total number of requests
REQUEST_TOTAL = PrometheusClient::Counter.new(
  :http_requests_total,
  'Total HTTP requests'
)

# Register metrics
PrometheusClient.register(REQUEST_DURATION)
PrometheusClient::Registry.default.register(REQUEST_TOTAL)

# Add middleware to Rails application
Rails.application.config.middleware.use PrometheusClient::Middleware,
  registry: PrometheusClient.registry,
  metrics: {
    duration: REQUEST_DURATION,
    total: REQUEST_TOTAL
  }

# Optional: Expose metrics endpoint
# You might want to mount this on a specific path, e.g., /metrics
# For simplicity, we'll assume a separate process or a dedicated endpoint.
# In a real-world scenario, consider a dedicated endpoint or a separate exporter.

With this in place, your application will expose metrics at a configured endpoint (often `/metrics` if integrated directly). You’ll then configure your Prometheus server to scrape this endpoint.

Monitoring Redis Clusters

Redis is a critical component for caching, session management, and message queuing. Monitoring its health and performance is paramount. Key metrics include:

  • Memory Usage: `used_memory` and `used_memory_rss`. Crucial for preventing Redis from being OOM-killed.
  • CPU Usage: `used_cpu_sys` and `used_cpu_user`. High CPU can indicate inefficient commands or heavy load.
  • Connected Clients: `connected_clients`. A sudden spike can indicate connection leaks or a DoS attack.
  • Latency: `instantaneous_ops_per_sec` and `latest_fork_usec`. High latency can be caused by fork operations or heavy commands.
  • Keyspace Notifications: `keyspace_hits` and `keyspace_misses`. Indicates cache hit rate.
  • Replication Status: For Redis Sentinel or Cluster, monitor `master_repl_offset` and `slave_repl_offset` to ensure replication is in sync.

Redis Exporter for Prometheus

The official Redis exporter for Prometheus is the standard choice. Download and run it on a node that can access your Redis instances.

Installation and Configuration (Example for Ubuntu/Debian)

Download the latest release from the redis_exporter releases page.

wget https://github.com/oliver006/redis_exporter/releases/download/v1.48.0/redis_exporter-v1.48.0.linux-amd64.tar.gz
tar xvfz redis_exporter-v1.48.0.linux-amd64.tar.gz
cd redis_exporter-v1.48.0.linux-amd64
sudo mv redis_exporter /usr/local/bin/

Create a systemd service file to manage the exporter.

Systemd Service File (`/etc/systemd/system/redis_exporter.service`)

[Unit]
Description=Redis Exporter
After=network.target

[Service]
User=redis_exporter
Group=redis_exporter
Type=simple
ExecStart=/usr/local/bin/redis_exporter \
  --redis.addr=redis://your_redis_host:6379 \
  --web.listen-address=":9121" \
  --redis.password=your_redis_password \
  --check-keyspace=true \
  --check-clients=true \
  --check-memory=true \
  --check-cpu=true \
  --check-commands=true \
  --check-replication=true

Restart=on-failure

[Install]
WantedBy=multi-user.target

Note: Replace `your_redis_host`, `your_redis_password`, and potentially the port if you are not using the default. For a Redis cluster, you’ll need to configure the exporter to connect to one of the nodes or use a cluster-aware configuration if supported by the exporter version. For Sentinel, you’d typically point it to the Sentinel instances.

Create a user for the exporter:

sudo useradd --system --no-create-home redis_exporter
sudo systemctl daemon-reload
sudo systemctl start redis_exporter
sudo systemctl enable redis_exporter
sudo systemctl status redis_exporter

Configure Prometheus to scrape the `redis_exporter`’s metrics endpoint (defaulting to `http://localhost:9121/metrics` if run on the same host as Prometheus).

DigitalOcean Droplet & Network Monitoring

Beyond application-specific metrics, fundamental infrastructure monitoring is crucial. DigitalOcean provides basic metrics through its control panel, but for deeper insights and alerting, consider integrating with your chosen monitoring solution.

Key Droplet Metrics to Monitor

  • CPU Utilization: Overall CPU load on the Droplet.
  • Memory Usage: Total RAM consumed.
  • Disk I/O: Read/write operations per second and latency.
  • Network Traffic: Inbound and outbound bandwidth.
  • Disk Space: Free space remaining.

You can use the DigitalOcean API to pull these metrics or, more commonly, run an agent (like `node_exporter` for Prometheus) directly on the Droplet.

Prometheus `node_exporter`

The `node_exporter` is a standard Prometheus exporter for hardware and OS metrics.

Installation (Example for Ubuntu/Debian)

wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
tar xvfz node_exporter-1.7.0.linux-amd64.tar.gz
sudo mv node_exporter-1.7.0.linux-amd64/node_exporter /usr/local/bin/

Systemd Service File (`/etc/systemd/system/node_exporter.service`)

[Unit]
Description=Prometheus Node Exporter
After=network.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter \
  --collector.filesystem \
  --collector.diskstats \
  --collector.cpu \
  --collector.meminfo \
  --collector.netdev \
  --web.listen-address=":9100"

Restart=on-failure

[Install]
WantedBy=multi-user.target

Create the user and start the service:

sudo useradd --system --no-create-home node_exporter
sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo systemctl status node_exporter

Configure Prometheus to scrape the `node_exporter`’s metrics endpoint (defaulting to `http://localhost:9100/metrics`).

Alerting Strategies: Proactive Problem Solving

Monitoring is only half the battle; effective alerting ensures you’re notified *before* users are impacted. Prometheus Alertmanager is the de facto standard for this when using Prometheus.

Key Alerts to Configure

  • High Application Error Rate: Alert when the rate of 5xx errors exceeds a threshold (e.g., > 5% over 5 minutes).
  • High Request Latency: Alert when the P95 or P99 latency for critical endpoints exceeds acceptable limits.
  • Redis Memory Pressure: Alert when `used_memory` exceeds 85% of `maxmemory`.
  • Redis High Latency: Alert on sustained high `latest_fork_usec` or `instantaneous_ops_per_sec` if it indicates a problem.
  • Droplet Resource Exhaustion: Alert on high CPU utilization (e.g., > 90% for 10 minutes), low disk space (< 10% free), or high memory usage.
  • Redis Replication Lag: Alert if `master_repl_offset` and `slave_repl_offset` drift significantly.
  • Application Unresponsiveness: A critical alert if the application’s `/health` endpoint (if you have one) starts returning errors or becomes unreachable.

Example Alerting Rule (Prometheus)

Add this to your Prometheus rules file (e.g., `rules.yml`):

groups:
- name: ruby_app_alerts
  rules:
  - alert: HighHttpErrorRate
    expr: |
      sum(rate(http_requests_total{status=~"5..", job="your_ruby_app_job"}[5m]))
      /
      sum(rate(http_requests_total{job="your_ruby_app_job"}[5m]))
      * 100 > 5
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "High HTTP error rate detected for {{ $labels.job }}"
      description: "The HTTP error rate for {{ $labels.job }} has exceeded 5% over the last 5 minutes. Current rate: {{ $value | printf \"%.2f\" }}%."

- name: redis_alerts
  rules:
  - alert: RedisMemoryLow
    expr: |
      redis_used_memory_bytes{job="your_redis_job"}
      /
      redis_maxmemory_bytes{job="your_redis_job"}
      * 100 > 85
    for: 10m
    labels:
      severity: warning
    annotations:
      summary: "Redis memory usage is high on {{ $labels.instance }}"
      description: "Redis instance {{ $labels.instance }} is using {{ $value | printf \"%.2f\" }}% of its max memory."

Ensure your `prometheus.yml` configuration includes these rules and is set up to send alerts to your Alertmanager instance.

Log Aggregation and Analysis

Metrics tell you *what* is happening, but logs often tell you *why*. Centralized log aggregation is essential for debugging issues that metrics alone can’t explain.

Recommended Logging Stack

  • Log Shipper: Fluentd, Filebeat, or Logstash to collect logs from your application servers and Redis.
  • Log Storage/Indexing: Elasticsearch or Loki.
  • Log Visualization: Kibana (for Elasticsearch) or Grafana (for Loki).

Configure your Ruby application to log to standard output (if running in containers) or to a dedicated log file. For Redis, configure `logfile` in `redis.conf` and ensure the log shipper can access it.

Example: Filebeat Configuration for Redis Logs

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/redis/redis-server.log  # Adjust path as per your redis.conf
  fields:
    target: redis
  fields_under_root: true

output.elasticsearch:
  hosts: ["your_elasticsearch_host:9200"]
  # username: "elastic"
  # password: "changeme"

# Or for Logstash:
# output.logstash:
#   hosts: ["your_logstash_host:5044"]

On your Ruby application servers, you’d configure Filebeat similarly, pointing to your application logs (e.g., `log/production.log`).

Conclusion: A Layered Approach

Maintaining the health of your Ruby applications and Redis clusters on DigitalOcean requires a multi-layered monitoring strategy. This involves instrumenting your applications, deploying exporters for critical services like Redis and your host OS, setting up intelligent alerting, and centralizing logs for deep analysis. By implementing these practices, you move from reactive firefighting to proactive system management, ensuring stability and performance.

Recent Posts

Top Categories

Our Products

Our Services