Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MySQL on Linode for Ruby

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MySQL on Linode for Ruby

Nginx as a High-Performance Frontend for Ruby Applications

When deploying Ruby applications, particularly those built with frameworks like Ruby on Rails or Sinatra, Nginx serves as an excellent choice for a frontend web server. Its strengths lie in its asynchronous, event-driven architecture, making it highly efficient at handling concurrent connections, serving static assets, and acting as a reverse proxy. This section details critical Nginx tuning parameters for optimal performance in a Linode environment.

Optimizing Nginx Worker Processes and Connections

The core of Nginx’s performance tuning lies in its worker processes and the number of connections each can handle. The ideal number of worker processes is typically set to the number of CPU cores available on the server. This allows Nginx to effectively utilize all available processing power without excessive context switching.

The worker_connections directive defines the maximum number of simultaneous connections that each worker process can open. This value, combined with the number of worker processes, determines the total maximum connections Nginx can handle. A common starting point is 1024, but this can be increased significantly, limited by the system’s file descriptor limits.

System File Descriptor Limits

Before increasing worker_connections, it’s crucial to ensure the operating system’s file descriptor limits are sufficient. Each connection consumes a file descriptor. You can check current limits with ulimit -n and modify them temporarily or permanently.

Temporary Limit Adjustment (Bash)

# Check current limits
ulimit -n

# Set limits for the current session (e.g., to 65536)
ulimit -n 65536

Permanent Limit Adjustment (/etc/security/limits.conf)

For persistent changes, edit /etc/security/limits.conf. Add the following lines, replacing nginx with the user Nginx runs as if different:

# /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
root soft nofile 65536
root hard nofile 65536

After modifying limits.conf, you’ll need to restart the Nginx service or reboot the server for these changes to take effect. Verify the limits again using ulimit -n after the service restart.

Nginx Configuration Snippet

In your nginx.conf (typically located at /etc/nginx/nginx.conf) or within a site-specific configuration file in /etc/nginx/sites-available/, adjust the events block:

# /etc/nginx/nginx.conf or site config
user www-data; # Or the user your Nginx runs as
worker_processes auto; # Set to number of CPU cores, or 'auto'

events {
    worker_connections 4096; # Adjust based on system limits and expected load
    multi_accept on; # Allows workers to accept multiple connections at once
}

http {
    # ... other http configurations ...

    # Enable Gzip compression for text-based assets
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    # Keep-alive timeout for client connections
    keepalive_timeout 65;

    # Buffering settings for proxying
    proxy_buffer_size 128k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;

    # ... server blocks ...
}

After applying these changes, test your Nginx configuration with sudo nginx -t and reload the service with sudo systemctl reload nginx.

Gunicorn/Puma Tuning for Ruby Applications

For Ruby applications, the choice of application server is critical. While Puma is the default for Rails, Gunicorn is a popular choice for Python, and understanding its tuning principles can be illustrative. For Ruby, we’ll focus on Puma. The goal is to configure the application server to efficiently handle incoming requests from Nginx and manage worker processes.

Puma Worker and Thread Configuration

Puma operates with a master process that spawns worker processes. Each worker process can then manage multiple threads. The optimal configuration depends heavily on your application’s I/O-bound vs. CPU-bound nature and the available server resources.

A common strategy is to set the number of workers to roughly the number of CPU cores, and then configure threads per worker. If your application is I/O-bound (e.g., making many external API calls or database queries), you can benefit from more threads. If it’s CPU-bound, fewer threads and more workers might be better.

Example Puma Configuration (config/puma.rb)

In your Rails application’s config/puma.rb file, you’ll find directives for workers and threads. Here’s an example tuned for a Linode instance with 4 CPU cores:

# config/puma.rb
# Example for a 4-core Linode instance

# Set the environment
environment ENV.fetch('RAILS_ENV') { 'production' }

# Number of workers to spawn.
# A good starting point is to match the number of CPU cores.
# If your app is I/O bound, you might increase this.
workers ENV.fetch('WEB_CONCURRENCY') { 4 }.to_i

# Minimum number of threads to use per worker.
# If your app is CPU bound, keep this low. If I/O bound, increase it.
# A common pattern is to set threads to a value that, when multiplied by workers,
# doesn't exceed the total number of CPU cores by too much, unless I/O bound.
# For 4 workers and 4 cores, 5 threads per worker gives 20 total threads.
# This is often suitable for I/O bound apps.
threads_count = ENV.fetch('RAILS_MAX_THREADS') { 5 }.to_i
threads threads_count, threads_count

# Bind to TCP port 3000, or use a Unix socket for Nginx proxying
# For Nginx proxying, a Unix socket is generally preferred for performance.
# If using a socket, ensure Nginx has read/write permissions.
# bind "tcp://0.0.0.0:3000"
bind "unix:///path/to/your/app/shared/tmp/sockets/puma.sock"

# Set the maximum number of connections allowed per worker.
# This is often set to a value that complements the number of threads.
# For example, if you have 5 threads, you might allow 100 connections per worker.
# This is a soft limit and can be adjusted.
max_threads_count = ENV.fetch('RAILS_MAX_THREADS') { 5 }.to_i
min_threads_count = ENV.fetch('RAILS_MIN_THREADS') { max_threads_count / 2 }.to_i
threads min_threads_count, max_threads_count

# Daemonize the server into the background.
# For production, this is usually true.
daemonize true

# PID file location
pidfile ENV.fetch('PIDFILE') { "tmp/pids/puma.pid" }

# State file location
state_path ENV.fetch('STATEFILE') { "tmp/pids/puma.state" }

# Logging
stdout_redirect ENV.fetch('LOG_STDOUT') { "log/puma.stdout.log" }, ENV.fetch('LOG_STDERR') { "log/puma.stderr.log" }
log_requests true

# Activate the master process.
activate_control_group

# Preload the application code.
preload_app!

# Callbacks for worker lifecycle
on_worker_boot do
  # Worker specific setup code.
  # For example, resetting database connections.
  ActiveRecord::Base.establish_connection if defined?(ActiveRecord)
end

# Allow Puma to be restarted by `rails restart` command.
plugin :tmp_restart

When using a Unix socket, ensure the directory for the socket exists and that the user running Puma has write permissions. Nginx will also need read/write permissions to this socket.

Starting Puma with Systemd

It’s best practice to manage Puma with a process supervisor like systemd. Create a service file, e.g., /etc/systemd/system/puma.service:

# /etc/systemd/system/puma.service
[Unit]
Description=Puma application server
After=network.target

[Service]
Type=simple
User=deploy # Replace with your application user
Group=www-data # Replace with your web server group
WorkingDirectory=/path/to/your/app
Environment="RAILS_ENV=production"
Environment="RAILS_LOG_TO_STDOUT=true" # If using stdout_redirect in puma.rb
# If using Unix socket, ensure the path is correct and user has permissions
ExecStart=/usr/local/bin/bundle exec puma -C /path/to/your/app/config/puma.rb

Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

After creating the service file, enable and start it:

sudo systemctl enable puma
sudo systemctl start puma
sudo systemctl status puma

MySQL/MariaDB Performance Tuning

Database performance is often the bottleneck in web applications. Tuning MySQL (or its fork, MariaDB) involves several key parameters within its configuration file, typically my.cnf or files within /etc/mysql/conf.d/.

Key MySQL Tuning Parameters

  • innodb_buffer_pool_size: This is arguably the most critical parameter for InnoDB. It determines the memory allocated for caching InnoDB data and indexes. A common recommendation is to set it to 50-70% of your server’s total RAM if the database is the primary workload. For a Linode with 8GB RAM, 4GB to 5.6GB is a good starting point.
  • innodb_log_file_size: Controls the size of the InnoDB redo log files. Larger log files can improve write performance by reducing the frequency of log flushing, but they also increase recovery time after a crash. A common size is 256MB to 1GB. Changing this requires a specific shutdown and startup procedure.
  • innodb_flush_log_at_trx_commit: Controls how often InnoDB flushes its log buffer to disk.
    • 1 (default): Flushes log to disk at each transaction commit. Safest, but slowest.
    • 0: Flushes log to disk once per second. Faster, but risks losing up to 1 second of transactions on a crash.
    • 2: Flushes log to OS buffer at each commit, and the OS flushes to disk once per second. Faster than 1, safer than 0.
    For many web applications, setting this to 2 offers a good balance of performance and durability.
  • max_connections: The maximum number of concurrent client connections. Set this based on your application’s needs and server resources. Too high can exhaust memory; too low causes connection errors.
  • query_cache_size (Deprecated in MySQL 5.7, removed in 8.0): If using an older version, a small query cache (e.g., 32MB-64MB) might help for read-heavy workloads with identical queries. However, it can cause contention. For modern applications, it’s often best left disabled or very small.
  • tmp_table_size and max_heap_table_size: These control the maximum size of in-memory temporary tables. Increasing them can speed up complex queries that require temporary tables, but be mindful of RAM usage.

MySQL Configuration Snippet

Create or edit a configuration file, for example, /etc/mysql/conf.d/99-custom.cnf:

# /etc/mysql/conf.d/99-custom.cnf
[mysqld]
# General
max_connections = 200 # Adjust based on application needs and server RAM
# For older MySQL versions, consider query cache if appropriate
# query_cache_type = 1
# query_cache_size = 64M

# InnoDB specific settings
innodb_buffer_pool_size = 4G # Example for 8GB RAM server, adjust to 50-70%
innodb_log_file_size = 512M # Example, requires careful restart procedure
innodb_flush_log_at_trx_commit = 2 # Balance between performance and safety
innodb_flush_method = O_DIRECT # Often recommended for performance on Linux
innodb_file_per_table = 1 # Recommended for easier management and space reclamation

# Temporary tables
tmp_table_size = 64M
max_heap_table_size = 64M

# Connection settings
wait_timeout = 600 # Close idle connections faster
interactive_timeout = 600

# Character set
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

Changing innodb_log_file_size

This change requires a specific procedure to avoid data corruption:

# 1. Stop MySQL
sudo systemctl stop mysql

# 2. Remove the old log files (e.g., ib_logfile0, ib_logfile1)
#    Find their location using 'mysqld --verbose --help | grep innodb_log_group_home_dir'
#    Typically in /var/lib/mysql/
sudo rm /var/lib/mysql/ib_logfile*

# 3. Edit your my.cnf to set the new innodb_log_file_size

# 4. Start MySQL
sudo systemctl start mysql

# 5. Verify the new log file size (optional, but good practice)
#    Check the MySQL error log for messages related to InnoDB initialization.
#    You can also query SHOW VARIABLES LIKE 'innodb_log_file_size';

After applying MySQL configuration changes, restart the MySQL service: sudo systemctl restart mysql.

Putting It All Together: Linode Deployment Workflow

A typical deployment workflow on Linode involves these steps:

  1. Provision Linode Instance: Choose an instance size appropriate for your expected load (CPU, RAM, Disk I/O).
  2. Install Dependencies: Install Nginx, your Ruby environment (RVM/rbenv), Node.js (for asset compilation), MySQL/MariaDB, and any other required packages.
  3. Configure Firewall: Use ufw or Linode’s Cloud Firewall to allow SSH (port 22), HTTP (port 80), and HTTPS (port 443).
  4. Tune System Limits: Adjust file descriptor limits as described earlier.
  5. Configure MySQL: Edit my.cnf or a file in conf.d/, apply tuning parameters, and restart MySQL. Create your database and user.
  6. Deploy Application Code: Use Git, Capistrano, or another deployment tool to get your Ruby application onto the server.
  7. Configure Puma: Edit config/puma.rb, set workers/threads, and configure socket or port binding.
  8. Configure Systemd for Puma: Create and enable the puma.service file.
  9. Configure Nginx as Reverse Proxy: Create a server block in /etc/nginx/sites-available/ to proxy requests to Puma’s socket or port. Include static asset serving and SSL configuration.
  10. Test and Reload Services: Test Nginx configuration (nginx -t), reload Nginx (systemctl reload nginx), and check Puma’s status (systemctl status puma).
  11. Monitor Performance: Use tools like htop, iotop, Nginx access/error logs, Puma logs, and MySQL slow query logs to identify bottlenecks.

Example Nginx Server Block for Puma Socket

This configuration assumes Puma is listening on a Unix socket and serves static assets directly from the Rails public/ directory.

# /etc/nginx/sites-available/your_app.conf
server {
    listen 80;
    server_name your_domain.com www.your_domain.com;

    # Serve static assets directly
    root /path/to/your/app/public;
    location ~ ^/(assets|images|javascripts|system)/  {
        try_files $uri $uri/ =404;
        expires 1y; # Cache static assets aggressively
        add_header Cache-Control "public";
    }

    # Proxy all other requests to Puma
    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # If Puma is using a Unix socket:
        proxy_pass http://unix:/path/to/your/app/shared/tmp/sockets/puma.sock;

        # If Puma is using a TCP port (e.g., 3000):
        # proxy_pass http://127.0.0.1:3000;

        proxy_read_timeout 300s; # Increase timeout for long-running requests
        proxy_connect_timeout 75s;
        proxy_send_timeout 300s;
    }

    # Optional: Add SSL configuration here for HTTPS
    # listen 443 ssl;
    # ssl_certificate /etc/letsencrypt/live/your_domain.com/fullchain.pem;
    # ssl_certificate_key /etc/letsencrypt/live/your_domain.com/privkey.pem;
    # include /etc/letsencrypt/options-ssl-nginx.conf;
    # ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    # Error pages
    error_page 500 502 503 504 /500.html;
    location = /500.html {
        root /path/to/your/app/public;
        internal;
    }
}

Remember to enable the site by creating a symbolic link:

sudo ln -s /etc/nginx/sites-available/your_app.conf /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

This comprehensive approach, combining OS-level tuning, Nginx configuration, application server optimization, and database tuning, provides a robust foundation for high-performance Ruby applications on Linode.