Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and PostgreSQL on DigitalOcean for Ruby

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and PostgreSQL on DigitalOcean for Ruby

Nginx as a High-Performance Frontend Proxy

For Ruby applications, Nginx excels as a reverse proxy, efficiently handling static assets, SSL termination, and load balancing. Its non-blocking, event-driven architecture makes it ideal for high-concurrency scenarios. We’ll focus on tuning key directives for optimal performance.

Nginx Configuration Tuning

The primary configuration file is typically located at /etc/nginx/nginx.conf. We’ll modify the http block to optimize worker processes and connection handling.

Worker Processes and Connections

The worker_processes directive should ideally be set to the number of CPU cores available on your DigitalOcean droplet. This allows Nginx to utilize all available processing power. The worker_connections directive defines the maximum number of simultaneous connections that each worker process can handle. A common starting point is 1024, but this can be increased based on your application’s needs and system limits.

Tuning `worker_connections`

To determine the optimal number of connections, consider the system’s file descriptor limit. Each connection consumes a file descriptor. You can check the current limit with ulimit -n. The maximum number of file descriptors available to Nginx is the product of worker_processes and worker_connections. Ensure this product does not exceed the system’s open file limit.

Keep-Alive Connections

Enabling HTTP keep-alive connections reduces the overhead of establishing new TCP connections for each request. This is particularly beneficial for clients making multiple requests. The keepalive_timeout directive controls how long an idle keep-alive connection will remain open. A value between 60 and 120 seconds is often a good balance.

Gzip Compression

Compressing responses with Gzip can significantly reduce bandwidth usage and improve load times. Ensure it’s enabled and configured appropriately.

Nginx Configuration Snippet

Here’s a sample snippet for your /etc/nginx/nginx.conf file, focusing on these optimizations. Remember to adjust worker_processes based on your droplet’s CPU count.

# /etc/nginx/nginx.conf

user www-data;
worker_processes auto; # Set to the number of CPU cores, or 'auto' for automatic detection
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Increased from default 1024
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 120; # Increased keep-alive timeout
    types_hash_max_size 2048;

    server_tokens off; # Hide Nginx version for security

    # Gzip Compression
    gzip on;
    gzip_disable "msie6";
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6; # Compression level (1-9)
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Gunicorn/Puma Tuning for Ruby Applications

For Ruby applications, you’re likely using either Gunicorn (Python WSGI HTTP Server, often used with frameworks like Flask/Django but can proxy Ruby apps) or Puma (a popular Ruby web server). We’ll focus on Puma as it’s more idiomatic for Ruby. The principles of worker tuning apply broadly.

Puma Worker and Thread Configuration

Puma operates with a master process that spawns worker processes. Each worker process can then spawn multiple threads to handle requests concurrently. The optimal configuration depends heavily on your application’s I/O-bound vs. CPU-bound nature and the resources of your droplet.

Worker Strategy

Puma offers two main worker strategies: fork (default) and exec. For most Ruby applications, fork is preferred as it allows for efficient memory sharing between workers. The workers directive sets the number of worker processes. A common starting point is (number of CPU cores) - 1 to leave one core for the OS and other services.

Thread Count

The threads directive specifies the minimum and maximum number of threads per worker. For I/O-bound applications, a higher thread count can be beneficial. For CPU-bound applications, keeping the thread count low (e.g., 1-4) is usually better to avoid excessive context switching. A common recommendation is to set min_threads and max_threads to the same value, often 5 to 10, depending on your application’s characteristics.

Puma Configuration Example (config/puma.rb)

Here’s a typical config/puma.rb file for a Rails application deployed on DigitalOcean, tuned for performance. Adjust workers and threads based on your droplet size and application profile.

# config/puma.rb

# Specifies the number of processes
# For a 2-core droplet, 1 worker is often sufficient. For 4+ cores, consider 2-3 workers.
workers Integer(ENV.fetch("WEB_CONCURRENCY") { 2 })

# Specifies the minimum and maximum number of threads to use
# For I/O bound apps, higher threads are good. For CPU bound, keep lower.
# A common starting point is 5-10 threads per worker.
threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
threads threads_count, threads_count

preload_app!

rackup File.expand_path('../config.ru', __FILE__)

# Set the environment
environment ENV.fetch("RAILS_ENV") { "production" }

# Allow Puma to be restarted by `rails restart` command.
plugin :tmp_restart

# Logging
on_worker_boot do
  ActiveRecord::Base.establish_connection if defined?(ActiveRecord)
end

# If using a Unix socket with Nginx, configure the socket path
# Example: bind "unix:///path/to/your/app.sock"
# If using TCP, e.g., for load balancing:
bind "tcp://0.0.0.0:3000" # Or your application's port

# Daemonize the process (optional, often handled by systemd/supervisord)
# daemonize false

# PID file location
pidfile "/tmp/pids/server.pid"

# State file location
state_path "/tmp/pids/puma.state"

# Logging configuration
stdout_redirect "/var/log/puma/puma.log", "/var/log/puma/puma.err.log", true

Systemd Service File Example

To manage Puma effectively, a systemd service file is recommended. This ensures Puma starts on boot, restarts on failure, and manages its environment variables.

# /etc/systemd/system/puma.service

[Unit]
Description=Puma application server
After=network.target

[Service]
Type=simple
User=deploy # Replace with your application user
Group=www-data # Replace with your application group

WorkingDirectory=/home/deploy/your_app # Replace with your app's root directory
Environment="RAILS_ENV=production"
Environment="RAILS_LOG_TO_STDOUT=false" # If using stdout_redirect in puma.rb
Environment="WEB_CONCURRENCY=2" # Matches workers in puma.rb
Environment="RAILS_MAX_THREADS=5" # Matches threads in puma.rb

ExecStart=/usr/local/bin/bundle exec puma -C /home/deploy/your_app/config/puma.rb # Adjust path to bundle and puma.rb

Restart=always

[Install]
WantedBy=multi-user.target

PostgreSQL Performance Tuning

PostgreSQL is a robust relational database, but its performance can be significantly impacted by configuration. We’ll focus on key parameters in postgresql.conf.

Key Configuration Parameters

The primary configuration file is typically located at /etc/postgresql/[version]/main/postgresql.conf. You can find the exact path by running sudo -u postgres psql -c 'SHOW config_file;'.

shared_buffers

This parameter defines the amount of memory dedicated to PostgreSQL for caching data. A common recommendation is 25% of your system’s RAM, but not exceeding 8GB on systems with less than 32GB RAM. For larger systems, 25% is still a good starting point.

work_mem

This parameter controls the amount of memory used for internal sort operations and hash tables before writing to temporary disk files. If your queries involve complex sorts or joins, increasing this can be beneficial. However, it’s allocated per operation, so setting it too high can lead to excessive memory consumption if many operations run concurrently. Start with a moderate value like 16MB and tune based on query analysis.

maintenance_work_mem

This parameter is used for maintenance operations like VACUUM, CREATE INDEX, and ALTER TABLE. A higher value can significantly speed up these operations. A common recommendation is 10-25% of system RAM, but capped at around 1-2GB.

effective_cache_size

This parameter informs the query planner about how much memory is available for disk caching by the operating system and PostgreSQL’s shared buffers. Setting this to 50-75% of total RAM is a good practice. It doesn’t allocate memory but influences the planner’s decisions.

wal_buffers

Write-Ahead Log (WAL) buffers are used to store WAL data before it’s written to disk. A value of -1 (auto-tune) is often sufficient, but manually setting it to 16MB can sometimes improve write performance.

max_connections

This defines the maximum number of concurrent connections to the database. Ensure this is set high enough to accommodate your application’s needs but not so high that it exhausts system resources. A common range is 100-300.

PostgreSQL Configuration Snippet

Here’s a sample snippet for postgresql.conf. Remember to adjust values based on your droplet’s RAM and your specific workload. After making changes, restart PostgreSQL: sudo systemctl restart postgresql.

# /etc/postgresql/[version]/main/postgresql.conf

# Adjust based on your droplet's RAM (e.g., 2GB RAM droplet -> 512MB)
shared_buffers = 1GB

# Adjust based on your application's query complexity and droplet RAM
work_mem = 32MB

# For maintenance tasks, can be higher than work_mem
maintenance_work_mem = 512MB

# Informative for the planner, typically 50-75% of total RAM
effective_cache_size = 4GB # Assuming an 8GB RAM droplet

# Usually auto-tuned, but can be set for potential write performance gains
wal_buffers = 16MB

# Set based on application needs, e.g., 200 connections for a medium app
max_connections = 200

# Enable logical replication if needed
# max_replication_slots = 10
# max_wal_senders = 10

# Tune checkpointing for write performance vs. recovery time
# checkpoint_completion_target = 0.9
# max_wal_size = 4GB

# Enable auto-vacuum for regular table maintenance
autovacuum = on
autovacuum_max_workers = 3
autovacuum_naptime = 15s
autovacuum_vacuum_threshold = 50
autovacuum_analyze_threshold = 50

Monitoring and Iterative Tuning

Performance tuning is not a one-time event. Continuous monitoring is crucial. Utilize tools like:

  • Nginx: nginx-module-vts for detailed Nginx metrics, Prometheus exporters.
  • Puma/Gunicorn: Application performance monitoring (APM) tools like New Relic, Datadog, or custom metrics exposed via Prometheus.
  • PostgreSQL: pg_stat_statements extension for query analysis, pg_activity, Prometheus exporters (e.g., postgres_exporter).
  • System Resources: htop, vmstat, iostat, DigitalOcean’s built-in monitoring.

Start with these baseline configurations, monitor your application’s behavior under load, and iteratively adjust parameters. Pay close attention to CPU utilization, memory usage, I/O wait times, and slow query logs in PostgreSQL. This systematic approach will ensure your Ruby application on DigitalOcean runs at peak efficiency.