The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MySQL on DigitalOcean for PHP

Nginx as a High-Performance Frontend Proxy

Nginx excels as a web server and reverse proxy due to its event-driven, asynchronous architecture. For a PHP application, it typically serves static assets directly and forwards dynamic requests to a backend application server (like Gunicorn for Python/Flask/Django, or PHP-FPM for PHP). Optimizing Nginx involves tuning worker processes, connection limits, and caching strategies.

Worker Processes and Connections

The `worker_processes` directive controls how many worker processes Nginx spawns. A common recommendation is to set this to the number of CPU cores available. `worker_connections` defines the maximum number of simultaneous connections that each worker process can handle. The total theoretical maximum connections is `worker_processes * worker_connections`.

Tuning `nginx.conf`

# /etc/nginx/nginx.conf

user www-data;
worker_processes auto; # Or set to the number of CPU cores, e.g., 4
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 1024; # Adjust based on expected load and system limits
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    server_tokens off; # Hide Nginx version for security

    # Gzip compression for text-based assets
    gzip on;
    gzip_disable "msie6";
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # Load balancing configuration (if using multiple PHP-FPM or Gunicorn instances)
    # upstream php_backend {
    #     server 127.0.0.1:9000;
    #     server 127.0.0.1:9001;
    # }

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

After modifying nginx.conf, always test the configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Caching Strategies

Leveraging Nginx’s caching can significantly reduce load on your backend. This includes browser caching for static assets and potentially proxy caching for dynamic content if appropriate.

Browser Caching for Static Assets

Set appropriate `Cache-Control` and `Expires` headers for static files (CSS, JS, images). This is typically done within your site’s server block.

# Inside your server block in /etc/nginx/sites-available/your_app

location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|webp)$ {
    expires 30d;
    add_header Cache-Control "public, no-transform";
    access_log off; # Optionally disable logging for static assets
    try_files $uri =404;
}

PHP-FPM / Gunicorn Tuning for PHP Applications

For PHP applications, PHP-FPM (FastCGI Process Manager) is the standard. For Python applications, Gunicorn is a popular choice. The core principle is managing the pool of worker processes that handle incoming requests.

PHP-FPM Configuration

PHP-FPM’s performance is heavily influenced by its process manager settings. The most common are `pm` (process manager type), `pm.max_children`, `pm.start_servers`, `pm.min_spare_servers`, and `pm.max_spare_servers`.

Tuning `php-fpm.conf` or Pool Configuration

These settings are typically found in /etc/php/[version]/fpm/pool.d/www.conf (or a custom pool file).

; /etc/php/8.1/fpm/pool.d/www.conf (example for PHP 8.1)

; Choose how the process manager (pm) will control the number of child processes.
; The available options are:
;   'static' - a fixed number of child processes.
;   'dynamic' - the number of child processes is adjusted dynamically.
;   'ondemand' - child processes are created only when new requests arrive.
; Default value: 'dynamic'
pm = dynamic

; If pm is set to 'dynamic', these are the values that will be used:
; Maximum number of accepted connections with the pm = dynamic.
; Default value: 50
pm.max_children = 100 ; Adjust based on available RAM and expected concurrency

; Number of child processes to be created when pm becomes available.
; Default value: 5
pm.start_servers = 10 ; A good starting point

; Minimum number of child processes to be kept active. The process manager will
; overhead to keep this number of child processes running.
; Default value: 2
pm.min_spare_servers = 5

; Maximum number of child processes to be kept active. The process manager will
; overhead to keep this number of child processes running.
; Default value: 8
pm.max_spare_servers = 20

; The script is allowed to run for at most this number of seconds.
; Default value: 30
request_terminate_timeout = 60 ; Increase for long-running operations

; The number of requests each child process should execute before respawning.
; This can be useful to prevent memory leaks from accumulating.
; Default value: 0 (disabled)
pm.max_requests = 500

Important Considerations for PHP-FPM Tuning:

• `pm.max_children`: This is the most critical setting. It should be calculated based on your server’s available RAM. A common formula is (Total RAM - RAM used by OS/other services) / Average RAM per PHP process. Monitor memory usage closely.
• `pm.max_requests`: Setting this to a reasonable value (e.g., 500-1000) helps prevent memory leaks in long-running applications by periodically restarting child processes.
• `request_terminate_timeout`: Increase this if your application has operations that legitimately take longer than the default 30 seconds.

After changes, restart PHP-FPM:

sudo systemctl restart php8.1-fpm # Adjust version as needed

Gunicorn Configuration (for Python Apps)

Gunicorn’s performance is primarily controlled by the number of worker processes and the worker type. For I/O-bound applications, `gevent` or `event` workers are often preferred over the default `sync` workers.

Running Gunicorn with Optimal Settings

A common command-line invocation or configuration file (e.g., gunicorn_config.py) would look like this:

# Example gunicorn_config.py
import multiprocessing

# Number of worker processes. A common starting point is (2 * number_of_cores) + 1.
workers = multiprocessing.cpu_count() * 2 + 1

# Worker type. 'sync' is the default. 'event' and 'gevent' are asynchronous.
# 'gevent' requires installing the gevent library: pip install gevent
worker_class = 'sync' # Or 'event', 'gevent'

# Bind to a socket or IP address and port
bind = "127.0.0.1:8000"

# Maximum number of requests a worker will process before restarting.
max_requests = 1000

# Timeout for worker processes.
timeout = 120 # seconds

# Logging configuration (optional but recommended)
# loglevel = 'info'
# accesslog = '-' # Log to stdout
# errorlog = '-'  # Log to stderr

To run Gunicorn with this configuration:

gunicorn --config gunicorn_config.py myapp.wsgi:application

Or directly via command line:

gunicorn -w $(($(nproc) * 2 + 1)) -k sync --max-requests 1000 --timeout 120 -b 127.0.0.1:8000 myapp.wsgi:application

MySQL Performance Tuning

Database performance is often a bottleneck. Tuning MySQL involves adjusting buffer sizes, query cache settings, and connection handling.

Key MySQL Configuration Variables

The primary configuration file is typically /etc/mysql/mysql.conf.d/mysqld.cnf or /etc/my.cnf.

Tuning `mysqld.cnf`

[mysqld]
# General Settings
user                    = mysql
pid-file                = /var/run/mysqld/mysqld.pid
socket                  = /var/run/mysqld/mysqld.sock
port                    = 3306
basedir                 = /usr
datadir                 = /var/lib/mysql
tmpdir                  = /tmp
lc_messages_dir         = /usr/share/mysql
lc_messages             = en_US
skip-external-locking

# InnoDB Settings (Crucial for performance)
# Adjust based on available RAM. A common starting point is 50-70% of RAM.
innodb_buffer_pool_size = 2G ; Example: For a server with 4GB RAM, start with 2GB. Monitor usage.
innodb_log_file_size    = 256M ; Larger log files can improve write performance but increase recovery time.
innodb_log_buffer_size  = 16M
innodb_flush_log_at_trx_commit = 1 ; (Default: 1) ACID compliant. 2 can be faster but less safe. 0 is fastest but riskiest.
innodb_flush_method     = O_DIRECT ; Recommended for modern systems with hardware RAID or SSDs.
innodb_file_per_table   = 1 ; Recommended for easier management and fragmentation control.

# Connection Settings
max_connections         = 200 ; Adjust based on application needs and server resources.
wait_timeout            = 600 ; Close idle connections after 10 minutes.
interactive_timeout     = 600

# Query Cache (Deprecated in MySQL 5.7, removed in 8.0. Consider application-level caching.)
# query_cache_type        = 0
# query_cache_size        = 0

# Other Performance Settings
key_buffer_size         = 16M ; Primarily for MyISAM tables, less critical if using InnoDB exclusively.
sort_buffer_size        = 2M
read_buffer_size        = 1M
read_rnd_buffer_size    = 2M
join_buffer_size        = 2M
tmp_table_size          = 64M
max_heap_table_size     = 64M

# Table Cache
table_open_cache        = 2000
table_definition_cache  = 1000

# Thread Cache
thread_cache_size       = 16

# Logging (Optional, disable in production for performance unless debugging)
# log_error = /var/log/mysql/error.log
# slow_query_log = 1
# slow_query_log_file = /var/log/mysql/mysql-slow.log
# long_query_time = 2

Tuning Notes:

• `innodb_buffer_pool_size`: This is the most important setting for InnoDB. It caches data and indexes. Set it as high as possible without causing the system to swap. Monitor SHOW ENGINE INNODB STATUS\G for buffer pool hit rate.
• `innodb_log_file_size`: Larger log files can improve write performance by reducing the frequency of flushing, but they increase recovery time after a crash.
• `innodb_flush_log_at_trx_commit`: Setting this to 1 (default) provides full ACID compliance but involves an fsync on every commit. Setting to 2 is often a good compromise for performance, as it flushes to the OS buffer but relies on the OS to flush to disk, making commits faster but with a small risk of data loss if the OS crashes.
• `max_connections`: Don’t set this too high, as each connection consumes memory. Ensure your application connection pooling is configured correctly.
• Query Cache: The query cache is generally disabled in modern MySQL versions due to scalability issues. Rely on application-level caching (e.g., Redis, Memcached) or InnoDB’s buffer pool.

After modifying the configuration, restart MySQL:

sudo systemctl restart mysql

Monitoring and Iteration

Tuning is an iterative process. Continuous monitoring is essential to identify bottlenecks and validate the effectiveness of your changes. Key metrics to watch include:

• Server Resources: CPU utilization (top, htop), Memory usage (free -h), Disk I/O (iostat), Network traffic (iftop).
• Nginx Metrics: Active connections, requests per second, error rates (stub_status module).
• PHP-FPM Metrics: Active processes, idle processes, request duration (PM status page).
• MySQL Metrics: Slow query log, connections, buffer pool hit rate, InnoDB row operations, temporary tables created.
• Application Performance Monitoring (APM): Tools like New Relic, Datadog, or open-source alternatives can provide deep insights into application-level performance and database query times.

Regularly review logs (Nginx access/error logs, PHP-FPM logs, MySQL error/slow query logs) for recurring issues or patterns. Use tools like mysqltuner.pl or tuning-primer.sh as a starting point for MySQL recommendations, but always validate their suggestions against your specific workload and server resources.