Vengala Vinay

Having 12+ Years of Experience in Software Development

Home » Server Monitoring Best Practices: Keeping Your Ruby App and MongoDB Clusters Alive on Linode

Server Monitoring Best Practices: Keeping Your Ruby App and MongoDB Clusters Alive on Linode

Establishing a Baseline: Essential Metrics for Ruby Apps and MongoDB

Effective server monitoring hinges on understanding what “normal” looks like for your specific stack. For a Ruby application, this means tracking request latency, error rates, and resource utilization. For MongoDB clusters, it’s about query performance, replication lag, disk I/O, and memory usage. Without a baseline, you’re flying blind, unable to distinguish between a minor blip and a critical failure.

Monitoring Ruby Application Performance

We’ll leverage New Relic for application performance monitoring (APM) due to its comprehensive features and ease of integration. For a more granular, open-source approach, consider Prometheus with the `ruby-prometheus-client` gem.

New Relic Integration

Install the New Relic Ruby agent. This typically involves adding the gem to your `Gemfile` and initializing it in your application’s environment configuration.

# Gemfile
gem 'newrelic_rpm'

Then, create or modify a New Relic configuration file, usually `config/newrelic.yml`.

# config/newrelic.yml
common: &common
  license_key: YOUR_NEW_RELIC_LICENSE_KEY
  app_name: YourAppName

development:
  <<: *common

production:
  <<: *common
  # Other production-specific settings...

Ensure your `RAILS_ENV` or equivalent is set to `production` and that the `newrelic_rpm` gem is loaded before your application code. Restart your Ruby application server (e.g., Puma, Unicorn).

Prometheus & Grafana for Ruby (Open Source)

For a self-hosted solution, Prometheus is a powerful choice. You'll need to instrument your Ruby application using a client library.

# Gemfile
gem 'prometheus-client-mruby'

Initialize the client and expose metrics via an HTTP endpoint. This example uses Rack.

# config/initializers/prometheus.rb (for Rails)
require 'prometheus_client/mruby'
require 'prometheus_client/middleware/collector'

# Initialize Prometheus Client
Prometheus::Client.configure do |config|
  config.logger = Rails.logger
end

# Define metrics
REQUEST_DURATION = Prometheus::Client::Histogram.new(:request_duration_seconds, 'Duration of HTTP requests in seconds.')
REQUEST_COUNT = Prometheus::Client::Counter.new(:http_requests_total, 'Total number of HTTP requests.')

# Register metrics
Prometheus::Client.default.register(REQUEST_DURATION)
Prometheus::Client.default.register(REQUEST_COUNT)

# Add middleware to collect metrics
Rails.application.middleware.use Prometheus::Client::Middleware::Collector,
  registry: Prometheus::Client.default,
  metrics: {
    REQUEST_DURATION => { labels: [:method, :path, :status] },
    REQUEST_COUNT => { labels: [:method, :path, :status] }
  }

You'll also need to expose a `/metrics` endpoint. In Rails, this can be done with a simple route and controller.

# routes.rb
get '/metrics' => 'metrics#index'

# controllers/metrics_controller.rb
class MetricsController < ApplicationController
  def index
    render plain: Prometheus::Client.default.render, content_type: 'text/plain; version=0.0.4; charset=utf-8'
  end
end

Configure Prometheus to scrape this endpoint. Add a job to your `prometheus.yml`:

# prometheus.yml
scrape_configs:
  - job_name: 'your_ruby_app'
    static_configs:
      - targets: ['your_app_host:3000'] # Replace with your app's host and port
    metrics_path: '/metrics'

Monitoring MongoDB Clusters

For MongoDB, we'll focus on metrics exposed via the `mongostat` and `mongotop` utilities, and more importantly, through the MongoDB Enterprise Monitoring tools or by scraping the `serverStatus` endpoint.

Key MongoDB Metrics to Track

  • Operations: `insert`, `query`, `update`, `delete` counts per second.
  • Network: Bytes in/out, connections (current, available).
  • Memory: Resident, Virtual, Mapped File usage.
  • Disk: Reads/writes per second, I/O wait times.
  • Replication: Oplog window, replication lag (seconds behind primary).
  • Query Performance: Slow queries (logged or via `db.currentOp()`).
  • Locking: Global lock percentage.

Using `mongostat` and `mongotop`

`mongostat` provides a real-time snapshot of server activity. It's excellent for quick checks and can be scripted.

# Monitor all members of a replica set
mongostat --host rs0/mongo1.example.com:27017,mongo2.example.com:27017 --discover --username admin --password '...' --authenticationDatabase admin --oplog --rowcount 10

`mongotop` tracks read/write activity per collection.

# Monitor read/write activity for a specific database
mongotop --db mydatabase --username admin --password '...' --authenticationDatabase admin --locks --rowcount 5

These tools are useful for interactive debugging but less so for continuous, automated monitoring. For that, we turn to Prometheus.

Prometheus MongoDB Exporter

The `mongodb_exporter` is a popular choice for exposing MongoDB metrics to Prometheus. Download and run it on a server that can connect to your MongoDB cluster.

# Download the exporter (example for Linux AMD64)
wget https://github.com/percona/mongodb_exporter/releases/download/v0.35.0/mongodb_exporter-0.35.0.linux.amd64.tar.gz
tar -xzf mongodb_exporter-0.35.0.linux.amd64.tar.gz
cd mongodb_exporter-0.35.0

# Run the exporter (replace connection string)
./mongodb_exporter --mongodb.uri="mongodb://admin:[email protected]:27017/admin?replicaSet=rs0" --web.listen-address=":9204"

This will expose metrics on port 9204. Configure Prometheus to scrape it:

# prometheus.yml
scrape_configs:
  - job_name: 'mongodb'
    static_configs:
      - targets: ['your_exporter_host:9204'] # Replace with your exporter's host and port

Monitoring Replication Lag

Replication lag is critical. The `mongodb_exporter` exposes metrics like `mongodb_replication_oplog_window_seconds` and `mongodb_replication_member_state`. You can create alerts based on these.

# Example Prometheus Alert Rule (in a .yml file)
groups:
- name: mongodb_alerts
  rules:
  - alert: MongoReplicaLagging
    expr: |
      avg by (replica_set, host) (
        time() - mongodb_replication_oplog_window_seconds
      ) > 600 # Alert if oplog window is more than 10 minutes behind
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "MongoDB replica set {{ $labels.replica_set }} on {{ $labels.host }} is lagging."
      description: "The oplog window is more than 10 minutes behind. Current lag: {{ printf "%.2f" $value }} seconds."

Infrastructure Monitoring on Linode

Linode provides basic infrastructure metrics through its Cloud Manager. For deeper insights and integration with your application/database monitoring, we'll use Node Exporter and potentially specialized exporters.

Node Exporter for System Metrics

Node Exporter is the de facto standard for collecting hardware and OS metrics for Prometheus.

# Download and install Node Exporter (example for Linux AMD64)
wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
tar -xzf node_exporter-1.7.0.linux-amd64.tar.gz
cd node_exporter-1.7.0.linux-amd64

# Run Node Exporter (typically on port 9100)
./node_exporter --web.listen-address=":9100"

Add this to your `prometheus.yml`:

# prometheus.yml
scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets: ['your_linode_host_1:9100', 'your_linode_host_2:9100'] # Add all your Linode IPs

Linode Specifics & Alerts

While Linode's Cloud Manager offers CPU, Disk I/O, and Network traffic graphs, these are often reactive. Integrate Prometheus alerts for critical thresholds:

# Example Prometheus Alert Rule for High CPU
groups:
- name: linode_alerts
  rules:
  - alert: HighCpuUsage
    expr: 100 - avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100 > 90
    for: 10m
    labels:
      severity: critical
    annotations:
      summary: "High CPU usage on Linode instance {{ $labels.instance }}"
      description: "Instance {{ $labels.instance }} has been running at over 90% CPU for 10 minutes."

Similarly, set up alerts for low disk space (`node_filesystem_avail_bytes`), high memory usage, and network saturation.

Centralized Logging and Alerting

Raw metrics are only part of the story. Centralized logging and a robust alerting mechanism are crucial for diagnosing issues and notifying the right people.

Log Aggregation with ELK/Loki

For log aggregation, consider the Elastic Stack (Elasticsearch, Logstash, Kibana) or Grafana Loki. Fluentd or Filebeat can be used as agents on your Linode instances to ship logs.

# Example Filebeat configuration (filebeat.yml)
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/your_app/*.log
    - /var/log/mongodb/mongod.log

output.elasticsearch:
  hosts: ["your_elasticsearch_host:9200"]

# Or for Loki:
# output.logstash:
#   hosts: ["your_logstash_host:5044"]

Alertmanager Configuration

Prometheus Alertmanager handles deduplication, grouping, and routing of alerts generated by Prometheus. Configure receivers (e.g., Slack, PagerDuty, email).

# alertmanager.yml
global:
  resolve_timeout: 5m
  slack_api_url: ''

route:
  group_by: ['alertname', 'cluster', 'service']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  receiver: 'slack-notifications'

receivers:
- name: 'slack-notifications'
  slack_configs:
  - channel: '#alerts'
    send_resolved: true
    text: '{{ template "slack.default.text" . }}'

Ensure your Prometheus configuration points to Alertmanager:

# prometheus.yml
alerting:
  alertmanagers:
  - static_configs:
    - targets: ['your_alertmanager_host:9093']

Proactive Health Checks & Synthetic Monitoring

Beyond passive monitoring, implement active checks to ensure critical functionalities are working.

Application Health Endpoint

Expose a `/health` endpoint in your Ruby application that checks database connectivity, external service availability, etc.

# controllers/health_controller.rb
class HealthController < ApplicationController
  def show
    status = {
      database: check_database,
      # Add other checks
    }

    if status.values.all?
      render json: { status: 'ok', checks: status }, status: :ok
    else
      render json: { status: 'error', checks: status }, status: :service_unavailable
    end
  end

  private

  def check_database
    ActiveRecord::Base.connection.execute('SELECT 1')
    true
  rescue StandardError
    false
  end
end

Use a tool like `blackbox_exporter` (for Prometheus) or a dedicated uptime monitoring service to periodically hit this endpoint.

MongoDB Liveness Probe

For MongoDB, ensure your application can connect and perform a basic read operation. This can be integrated into your application's health check or monitored separately.

# Example Python script for health check
from pymongo import MongoClient
from pymongo.errors import ConnectionFailure

def check_mongo_health(uri="mongodb://admin:[email protected]:27017/admin?replicaSet=rs0"):
    try:
        client = MongoClient(uri, serverSelectionTimeoutMS=5000)
        # The ismaster command is cheap and does not require auth.
        client.admin.command('ismaster')
        return True, "MongoDB connection successful."
    except ConnectionFailure as e:
        return False, f"MongoDB connection failed: {e}"
    except Exception as e:
        return False, f"An unexpected error occurred: {e}"

if __name__ == "__main__":
    is_healthy, message = check_mongo_health()
    print(f"MongoDB Health: {is_healthy} - {message}")
    if not is_healthy:
        exit(1) # Exit with non-zero code for alerting

Schedule this script to run periodically and alert on failures.

Recent Posts

Top Categories

Our Products

Our Services