The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MySQL on DigitalOcean for Ruby
Nginx as a High-Performance Frontend Proxy
For Ruby applications, especially those built with frameworks like Rails or Sinatra, Nginx serves as an indispensable frontend proxy. It handles static file serving, SSL termination, request buffering, and load balancing, offloading these critical tasks from your application server. This section details optimal Nginx configurations for a DigitalOcean droplet running your Ruby application.
Tuning Worker Processes and Connections
The `worker_processes` directive dictates how many worker processes Nginx will spawn. A common recommendation is to set this to the number of CPU cores available on your server. For a DigitalOcean droplet, you can determine this using `nproc` or `lscpu`.
The `worker_connections` directive sets the maximum number of simultaneous connections that each worker process can handle. The total maximum connections will be `worker_processes * worker_connections`. Ensure this value is sufficiently high to avoid connection exhaustion, but not so high that it over-allocates resources.
Optimizing Keep-Alive and Buffering
Enabling HTTP Keep-Alive (`keepalive_timeout`) allows clients to reuse the same TCP connection for multiple HTTP requests, reducing latency. A value between 60 and 120 seconds is generally a good starting point.
Request buffering (`client_body_buffer_size`, `client_max_body_size`, `large_client_header_buffers`) is crucial for handling large uploads and preventing denial-of-service attacks. Configure these to match your application’s expected needs.
Static File Serving and Caching
Nginx excels at serving static assets. Configure appropriate `expires` headers to leverage browser caching, significantly reducing load on your application server for repeated requests of CSS, JavaScript, and images.
Example Nginx Configuration Snippet
Here’s a sample snippet for your Nginx configuration, typically found in `/etc/nginx/nginx.conf` or within a site-specific configuration file in `/etc/nginx/sites-available/`.
# Determine worker_processes based on CPU cores
worker_processes auto; # or set to 'nproc'
events {
worker_connections 4096; # Adjust based on expected load and server memory
multi_accept on;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
keepalive_requests 1000; # Max requests per keepalive connection
# Buffering settings
client_body_buffer_size 10K;
client_max_body_size 50m; # Adjust for file upload limits
large_client_header_buffers 4 16k;
# Gzip compression for dynamic content
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
# Static file caching
location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
add_header Cache-Control "public";
access_log off;
}
# Proxy pass to your application server (e.g., Gunicorn/FPM)
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300; # Increase for long-running requests
proxy_connect_timeout 75;
proxy_send_timeout 300;
# For Gunicorn (Python WSGI)
# proxy_pass http://unix:/path/to/your/app.sock;
# For PHP-FPM
# proxy_pass http://127.0.0.1:9000;
}
# ... other http configurations ...
}
Application Server Tuning: Gunicorn (Python) vs. PHP-FPM
The choice between Gunicorn (for Python applications) and PHP-FPM (for PHP applications) significantly impacts performance. Each requires distinct tuning parameters to maximize throughput and minimize latency.
Gunicorn Configuration for Python Apps
Gunicorn is a Python WSGI HTTP Server. Its performance is heavily influenced by the number of worker processes and the worker class used. For CPU-bound applications, the `sync` worker class is common, while for I/O-bound applications, `gevent` or `event` can offer better concurrency.
Worker Processes and Threads
The number of worker processes is typically set to `(2 * number_of_cores) + 1`. For I/O-bound applications using `gevent` or `event` workers, you might increase this further. Threads are not directly managed by Gunicorn’s worker count but by the underlying libraries (e.g., `threading` module in Python). For `sync` workers, each worker handles one request at a time.
Timeout and Keep-Alive
The `–timeout` setting in Gunicorn defines how long the worker will wait for a request to be processed before it’s killed and restarted. This should be set higher than your longest expected request but not excessively high to avoid masking performance issues.
Example Gunicorn Command Line / Configuration
You can launch Gunicorn with various options. A common way to manage this is via a systemd service file.
# Example systemd service file for Gunicorn
# /etc/systemd/system/my_ruby_app.service
[Unit]
Description=Gunicorn instance to serve my_ruby_app
After=network.target
[Service]
User=my_app_user
Group=my_app_user
WorkingDirectory=/path/to/your/ruby_app
Environment="PATH=/path/to/your/ruby_app/venv/bin"
ExecStart=/path/to/your/ruby_app/venv/bin/gunicorn \
--workers 3 \
--worker-class sync \
--bind unix:/path/to/your/app.sock \
--timeout 120 \
--log-level info \
--access-logfile /var/log/gunicorn/access.log \
--error-logfile /var/log/gunicorn/error.log \
my_ruby_app.wsgi:application # Replace with your actual WSGI application entry point
[Install]
WantedBy=multi-user.target
Note: While the example above uses Python syntax for `my_ruby_app.wsgi:application`, for a Ruby application, you would typically use a WSGI server like Puma or Unicorn, configured via their respective settings. The principle of worker processes, threads, and timeouts remains similar.
PHP-FPM Configuration for PHP Apps
PHP-FPM (FastCGI Process Manager) is the standard for running PHP applications. Its performance tuning revolves around the process manager settings, primarily the number of child processes and how they are managed.
Process Manager Settings
PHP-FPM offers three main process management strategies: `static`, `dynamic`, and `ondemand`. `dynamic` is often a good balance, starting with a few processes and spawning more as needed, up to a defined maximum. `static` pre-forks a fixed number of processes, which can be beneficial if you have predictable traffic and sufficient memory.
Tuning `pm.max_children`, `pm.start_servers`, `pm.min_spare_servers`, `pm.max_spare_servers`
These directives control the number of child processes. `pm.max_children` is the absolute maximum number of child processes that will be spawned. `pm.start_servers` is the number of children created at FPM startup. `pm.min_spare_servers` and `pm.max_spare_servers` define the range of spare children to maintain. Tuning these requires careful consideration of your server’s RAM and expected concurrent requests.
Example PHP-FPM Configuration Snippet
This configuration is typically found in `/etc/php/X.Y/fpm/pool.d/www.conf` (replace X.Y with your PHP version).
; Example PHP-FPM pool configuration [www] user = www-data group = www-data listen = /run/php/phpX.Y-fpm.sock ; Or use TCP/IP: listen = 127.0.0.1:9000 ; Process Manager settings pm = dynamic pm.max_children = 50 ; Adjust based on RAM and expected load pm.start_servers = 5 ; Initial number of children pm.min_spare_servers = 2 ; Minimum spare children pm.max_spare_servers = 10 ; Maximum spare children pm.max_requests = 500 ; Restart child processes after this many requests ; Other useful settings request_terminate_timeout = 120 ; Corresponds to Nginx's proxy_read_timeout ; rlimit_files = 1024 ; rlimit_nofile = 65536
MySQL/MariaDB Performance Tuning
Database performance is often the bottleneck in web applications. Optimizing your MySQL or MariaDB instance on DigitalOcean involves tuning key configuration variables and understanding query performance.
Key Configuration Variables (`my.cnf` / `my.ini`)
The primary configuration file for MySQL/MariaDB is typically located at `/etc/mysql/my.cnf` or `/etc/mysql/mysql.conf.d/mysqld.cnf`. Tuning these parameters requires an understanding of your server’s RAM and workload.
`innodb_buffer_pool_size`
This is arguably the most critical setting for InnoDB. It determines the amount of memory allocated for caching InnoDB data and indexes. A common recommendation is to set it to 50-70% of your server’s total RAM on a dedicated database server. For a shared droplet, be more conservative.
`innodb_log_file_size` and `innodb_log_buffer_size`
Larger `innodb_log_file_size` can improve write performance by reducing checkpoint flushing frequency, but it also increases recovery time after a crash. `innodb_log_buffer_size` is for buffering transactions before writing to the log file; a larger value can help with high-volume transactions.
`query_cache_size` (Deprecated in MySQL 5.7, Removed in 8.0)
While deprecated, if you are on an older version, a properly tuned query cache can significantly speed up read-heavy workloads. However, it can also become a bottleneck under heavy write loads due to invalidation overhead. For modern versions, rely on other caching mechanisms.
`max_connections`
Sets the maximum number of simultaneous client connections. Ensure this is high enough for your application’s needs but not so high that it exhausts server memory. Each connection consumes resources.
Example MySQL/MariaDB Configuration Snippet
This snippet is illustrative and should be adapted to your specific droplet size and workload.
[mysqld] # General Settings user = mysql pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock port = 3306 basedir = /usr datadir = /var/lib/mysql tmpdir = /tmp lc_messages_dir = /usr/share/mysql lc_messages = en_US # InnoDB Settings (Crucial for performance) innodb_buffer_pool_size = 2G ; Adjust based on server RAM (e.g., 70% of 4GB RAM) innodb_log_file_size = 512M ; Adjust based on write load innodb_log_buffer_size = 16M ; For high-volume transactions innodb_flush_method = O_DIRECT ; Recommended for modern systems innodb_flush_log_at_trx_commit = 1 ; For ACID compliance, 2 for better performance with slight risk # Connection Settings max_connections = 200 ; Adjust based on application needs and server resources thread_cache_size = 16 ; Cache threads for reuse # Query Cache (If applicable and on older MySQL versions) # query_cache_type = 1 # query_cache_size = 64M # query_cache_limit = 1M # Other performance tuning key_buffer_size = 16M ; For MyISAM index caching (less relevant if primarily InnoDB) sort_buffer_size = 1M read_buffer_size = 1M read_rnd_buffer_size = 2M join_buffer_size = 2M tmp_table_size = 64M max_heap_table_size = 64M # Logging (Optional but recommended for debugging) # log_error = /var/log/mysql/error.log # slow_query_log = 1 # slow_query_log_file = /var/log/mysql/mysql-slow.log # long_query_time = 2
Query Optimization and Indexing
Beyond server configuration, inefficient queries are a major performance killer. Regularly analyze your slow query log (if enabled) and use `EXPLAIN` to understand query execution plans. Ensure appropriate indexes are in place for columns used in `WHERE`, `JOIN`, `ORDER BY`, and `GROUP BY` clauses.
Using `EXPLAIN`
Run `EXPLAIN SELECT … FROM … WHERE …;` to see how MySQL executes a query. Look for full table scans (`type: ALL`), missing indexes, and large `rows` examined. Optimize queries and add indexes as needed.
EXPLAIN SELECT users.name, orders.order_date FROM users JOIN orders ON users.id = orders.user_id WHERE users.created_at BETWEEN '2023-01-01' AND '2023-12-31' ORDER BY orders.order_date DESC;
Monitoring and Iterative Tuning
Performance tuning is not a one-time task. Continuous monitoring and iterative adjustments are key to maintaining optimal performance. Utilize DigitalOcean’s monitoring tools, server-level metrics (CPU, RAM, I/O, Network), and application-specific performance monitoring (APM) tools.
Key Metrics to Monitor
- Nginx: Active connections, requests per second, error rates (5xx, 4xx), upstream response times.
- Application Server (Gunicorn/PHP-FPM): Worker utilization, request queue length, response times, memory usage.
- Database (MySQL/MariaDB): Query throughput, slow queries, connection usage, buffer pool hit rate, disk I/O, replication lag (if applicable).
- System: CPU load, memory usage (free vs. used), swap usage, disk I/O wait times, network traffic.
Iterative Tuning Process
1. **Establish Baseline:** Measure current performance under typical load.
2. **Identify Bottlenecks:** Use monitoring tools to pinpoint the slowest component (Nginx, app server, database, or even network).
3. **Make One Change:** Adjust a single configuration parameter or optimize a specific query.
4. **Measure Again:** Re-evaluate performance after the change. Did it improve, degrade, or have no effect?
5. **Repeat:** Continue the cycle, making small, incremental changes and measuring their impact.
By systematically tuning Nginx, your application server (Gunicorn/PHP-FPM), and your database (MySQL/MariaDB), you can build a robust, high-performance infrastructure on DigitalOcean for your Ruby applications.