The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MongoDB on DigitalOcean for Ruby

Nginx as a High-Performance Frontend Proxy

For Ruby applications, Nginx serves as an indispensable frontend proxy, efficiently handling static assets, SSL termination, and load balancing. Its asynchronous, event-driven architecture makes it ideal for managing high concurrency with minimal resource overhead. We’ll focus on tuning key directives for optimal performance.

Nginx Configuration Tuning

The primary configuration file for Nginx is typically located at /etc/nginx/nginx.conf. We’ll adjust the events and http blocks.

Worker Processes and Connections

The worker_processes directive determines how many worker processes Nginx will spawn. Setting this to auto allows Nginx to detect the number of CPU cores and set the worker processes accordingly, which is generally the most performant option. The worker_connections directive sets the maximum number of simultaneous connections that each worker process can handle. This value, combined with the number of worker processes, dictates the total concurrent connection capacity. A common starting point is 1024, but this can be increased based on server resources and expected load.

Example: nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Increased from default 1024
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # Gzip compression for text-based assets
    gzip on;
    gzip_disable "msie6";
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    # SSL settings
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384';

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Static File Serving

Ensure Nginx is configured to serve static assets directly. This offloads the application server, significantly improving response times for non-dynamic content. The sendfile on; directive allows Nginx to send files directly from its file descriptor to the socket, bypassing user space and reducing CPU usage. tcp_nopush on; and tcp_nodelay on; optimize TCP packet transmission.

Gzip Compression

Enabling gzip compression for text-based assets (HTML, CSS, JavaScript, JSON) can drastically reduce bandwidth usage and improve load times. The directives gzip on;, gzip_types, and gzip_comp_level are crucial here. A compression level of 6 offers a good balance between compression ratio and CPU overhead.

Gunicorn Configuration for Ruby/Rack Applications

Gunicorn (Green Unicorn) is a popular WSGI HTTP Server for Python, but it’s also commonly used to serve Ruby Rack applications. It’s a pre-fork worker model server, meaning it spawns worker processes that are kept alive. This is generally more stable and easier to manage than event-driven models for certain types of applications.

Worker Processes and Threads

The key directives for Gunicorn are --workers and --threads. A common recommendation for --workers is (2 * number_of_cpu_cores) + 1. This formula aims to keep all CPU cores busy while accounting for I/O waits. The --threads directive is used with the gthread worker type. If your application is heavily I/O bound (e.g., making many external API calls or database queries), using threads can improve concurrency within a single worker process. However, for CPU-bound tasks, more worker processes are generally better than threads.

Worker Type

Gunicorn supports several worker types: sync (the default, blocking I/O), eventlet, gevent (both asynchronous, non-blocking I/O), and gthread (multi-threaded). For most Ruby Rack applications, especially those that might have blocking I/O operations, the sync worker type is often the most straightforward and stable. If you have a highly concurrent, I/O-bound application and are comfortable with asynchronous programming, gevent or eventlet could offer performance benefits, but require careful testing.

Command-Line Example

Here’s a typical Gunicorn startup command for a Ruby Rack application (assuming your Rack app is defined in config.ru):

Example: Gunicorn Startup Command

# Assuming 4 CPU cores on the DigitalOcean droplet
gunicorn --workers 9 \
         --threads 2 \
         --worker-class sync \
         --bind unix:/path/to/your/app.sock \
         --log-level info \
         --log-file /var/log/gunicorn/app.log \
         config:app

Explanation:

• --workers 9: (2 * 4 cores) + 1 = 9 workers.
• --threads 2: Using 2 threads per worker. This is less critical for sync workers but can still influence how internal operations are handled. For gthread worker type, this is essential.
• --worker-class sync: Using the synchronous worker class.
• --bind unix:/path/to/your/app.sock: Binding to a Unix socket is generally faster than binding to a TCP port when Nginx is on the same machine.
• --log-level info: Sets the logging level.
• --log-file /var/log/gunicorn/app.log: Specifies the log file. Ensure the directory exists and the user running Gunicorn has write permissions.
• config:app: This tells Gunicorn where to find your Rack application. If your config.ru is in the root of your project, this might be simplified or specific to your framework (e.g., my_rails_app.wsgi:application for Python, but for Ruby, it’s often implicitly handled by config.ru). For Ruby, you might need to ensure your Gemfile has unicorn or rack gems.

Process Management (Systemd)

It’s crucial to manage Gunicorn using a process supervisor like systemd to ensure it restarts automatically if it crashes and starts on boot.

Example: /etc/systemd/system/gunicorn.service

[Unit]
Description=Gunicorn instance to serve myapp
After=network.target

[Service]
User=your_app_user
Group=www-data
WorkingDirectory=/path/to/your/app
Environment="PATH=/path/to/your/app/bin:/usr/local/bin:/usr/bin:/bin"
ExecStart=/usr/local/bin/gunicorn --workers 9 --threads 2 --worker-class sync --bind unix:/path/to/your/app.sock --log-level info --log-file /var/log/gunicorn/app.log config:app

# Optional: Restart policy
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

After creating this file, enable and start the service:

sudo systemctl enable gunicorn.service
sudo systemctl start gunicorn.service
sudo systemctl status gunicorn.service

PHP-FPM Tuning (if applicable)

If your infrastructure involves PHP components alongside Ruby (e.g., a microservice architecture, or a legacy PHP application), tuning PHP-FPM is essential. PHP-FPM (FastCGI Process Manager) is used to manage PHP processes and handle requests from web servers like Nginx.

Process Manager Settings

The primary configuration file for PHP-FPM is typically /etc/php/[version]/fpm/php-fpm.conf, with pool configurations in /etc/php/[version]/fpm/pool.d/www.conf. The pm (Process Manager) setting is critical. Common options are static, dynamic, and ondemand.

pm = dynamic

This is often a good balance. It starts with a few child processes and spawns more as needed, up to a defined maximum. It also kills idle processes to save resources.

Key Directives for dynamic PM:

• pm.max_children: The maximum number of child processes that will be spawned. This is the most important setting. Set it based on your server’s RAM. A common rule of thumb is to leave enough RAM for the OS and other services (like Nginx, MongoDB), and then divide the remaining RAM by the average RAM usage per PHP-FPM process.
• pm.start_servers: The number of child processes to start when PHP-FPM starts.
• pm.min_spare_servers: The minimum number of idle (spare) processes to maintain.
• pm.max_spare_servers: The maximum number of idle (spare) processes to maintain.
• pm.max_requests: The number of requests each child process will execute before respawning. This helps to prevent memory leaks. A value between 500 and 1000 is common.

pm = static

This setting keeps a fixed number of child processes running at all times. It can offer slightly better performance for high-traffic sites as there’s no overhead for spawning new processes, but it consumes more memory constantly.

Key Directives for static PM:

• pm.max_children: The exact number of child processes to maintain.
• pm.max_requests: Same as for dynamic.

Example: /etc/php/[version]/fpm/pool.d/www.conf (Dynamic PM)

; Start a new child processor when the number of requests
; reaches this limit. Appended to enable proper graceful recursion,
; and avoid "Segmentation faults". Default value: 0 (forever).
;
;pm.max_requests = 500

; The dynamic PM will generate process(es) dynamically based on the
; following parameters.
;
;pm.max_children = 50  ; Adjust based on server RAM
;pm.start_servers = 2
;pm.min_spare_servers = 1
;pm.max_spare_servers = 3
;pm.max_requests = 300

; Set to 'on' if you want to use the slowlog feature.
; Default value: 'off'.
;slowlog = /var/log/php/php-fpm.slow.log

; The socket to listen on.
; For IPv4, use: tcp://127.0.0.1:9000
; For IPv6, use: tcp://[::1]:9000
; For a Unix socket, use: unix:/var/run/php/php7.4-fpm.sock
; Note: If the socket path is not absolute, it is relative to the PHP-FPM configuration directory.
listen = /run/php/php7.4-fpm.sock ; Example for PHP 7.4

; Set the user and group for the processes.
user = www-data
group = www-data

; Set the number of DNS lookup processes.
; Default value: 0 (disabled).
;pm.dns_max_processes = 0

; Set the maximum number of concurrent connections.
; Default value: 1024
;pm.max_children = 50 ; Example: Adjust based on server RAM
;pm.start_servers = 5
;pm.min_spare_servers = 2
;pm.max_spare_servers = 5
;pm.max_requests = 1000

; Set the maximum execution time of a script.
; Default value: 30
;max_execution_time = 30

; Set the memory limit for a script.
; Default value: 128M
;memory_limit = 256M

After modifying PHP-FPM configuration, restart the service:

sudo systemctl restart php7.4-fpm # Adjust version as needed

MongoDB Performance Tuning on DigitalOcean

MongoDB’s performance is heavily influenced by hardware, configuration, and query patterns. On DigitalOcean, optimizing MongoDB involves careful consideration of disk I/O, RAM, and network latency.

Storage Engine Choice

MongoDB 3.2+ defaults to the WiredTiger storage engine, which is generally recommended for most workloads due to its excellent compression and concurrency features. Ensure you are using WiredTiger. If you are on an older version or have specific needs, consider MMAPv1 (though generally less performant).

Configuration File Tuning (mongod.conf)

The main configuration file is typically /etc/mongod.conf.

Key Directives:

• storage.wiredTiger.engineConfig.cacheSizeGB: This is arguably the most critical setting. WiredTiger uses a portion of RAM for its internal cache. A common recommendation is to allocate 50% of the available RAM to the WiredTiger cache, ensuring enough RAM is left for the OS and other processes. For example, on a 16GB droplet, you might allocate 8GB.
• storage.journal.enabled: Keep this enabled (default). It ensures data durability by writing operations to a journal before applying them to data files. Disabling it can improve write performance but risks data loss in case of a crash.
• net.bindIp: If MongoDB is only accessed from within the same DigitalOcean VPC or from localhost, binding to specific private IPs or 127.0.0.1 can enhance security and potentially performance by avoiding unnecessary network interface checks.
• operationProfiling.mode: Set to slowOp or all for profiling slow queries. This is essential for identifying performance bottlenecks.

Example: /etc/mongod.conf Snippet

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
  wiredTiger:
    engineConfig:
      cacheSizeGB: 6 # Example: For a 12GB RAM droplet, leaving 6GB for OS/other services

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1,10.10.0.5 # Example: Bind to localhost and a private IP

# security:
#   authorization: enabled # Recommended for production

# logging:
#   quiet: false
#   accessLog:
#     enabled: true
#     path: /var/log/mongodb/mongod.log
#   systemLog:
#     enabled: true
#     path: /var/log/mongodb/mongod.log
#     verbosity: 0

# operation profiling
operationProfiling:
  mode: slowOp # Profile slow operations (default is off)
  slowOpThreshold: 100 # Milliseconds. Adjust as needed.

After changing the configuration, restart MongoDB:

sudo systemctl restart mongod

Indexing Strategies

Proper indexing is paramount for MongoDB performance. Analyze your query patterns using explain() and the slow query logs. Ensure indexes cover the fields used in query predicates, sorts, and projections. Compound indexes are powerful but should be ordered carefully.

Monitoring and Diagnostics

Regularly monitor key metrics:

• CPU Usage: High CPU can indicate inefficient queries or insufficient resources.
• RAM Usage: Monitor cache hit rates. A low cache hit rate suggests the working set doesn’t fit in RAM, leading to disk I/O.
• Disk I/O: High disk I/O (reads/writes) is a major bottleneck. Optimize queries and increase RAM if possible.
• Network I/O: Monitor bandwidth usage, especially for distributed setups.
• MongoDB specific metrics: Use db.serverStatus() and db.stats(), or tools like mongostat and mongotop.

Example: Using mongostat

# Monitor MongoDB server status in real-time
mongostat --host your_mongo_host --port 27017 --username your_user --password your_password --authenticationDatabase admin --rows 10

Key columns to watch in mongostat output include:

• insert, query, update, delete: Operations per second.
• dirty %: Percentage of dirty data in WiredTiger cache. High values might indicate insufficient cache or slow writes.
• used %: Percentage of WiredTiger cache used.
• netIn, netOut: Network traffic.
• res: Resident memory usage.
• qr|qw: Queue length for read/write operations. High values indicate contention.
• idx%: Percentage of operations using indexes. Aim for 100% for read operations.

Putting It All Together: The Nginx-Gunicorn-MongoDB Stack

The synergy between these components is key. Nginx acts as the gatekeeper, efficiently routing traffic. Gunicorn (or your chosen Ruby app server) handles the application logic. MongoDB provides persistent storage. Tuning each layer independently is important, but understanding their interactions is crucial for holistic performance optimization. Ensure your Nginx configuration correctly proxies requests to the Gunicorn Unix socket or TCP port, and that your Ruby application is efficiently interacting with MongoDB, leveraging indexes and avoiding N+1 query problems.