The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Redis on DigitalOcean for Magento 2

Nginx Configuration for Magento 2

Optimizing Nginx is crucial for serving static assets efficiently and acting as a robust reverse proxy for your Magento 2 application. We’ll focus on key directives that directly impact performance and stability.

Worker Processes and Connections

The worker_processes directive determines how many worker processes Nginx will spawn. A common recommendation is to set it to the number of CPU cores available on your server. worker_connections defines the maximum number of simultaneous connections that each worker process can handle. The total maximum connections will be worker_processes * worker_connections.

Tuning nginx.conf

Locate your main Nginx configuration file, typically /etc/nginx/nginx.conf. Adjust the events block as follows:

# /etc/nginx/nginx.conf

user www-data;
worker_processes auto; # Or set to the number of CPU cores, e.g., 4
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Adjust based on server load and RAM
    multi_accept on;
}

http {
    # ... other http configurations ...
}

After making changes, test the configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Caching and Compression

Leveraging Nginx’s built-in caching and Gzip compression can significantly reduce server load and improve page load times. We’ll configure browser caching for static assets and enable Gzip compression for dynamic content.

Static Asset Caching

Add the following directives within your Magento 2 site’s server block (typically in /etc/nginx/sites-available/your_magento_site) to set appropriate cache headers for static files.

# /etc/nginx/sites-available/your_magento_site

server {
    # ... other server configurations ...

    location ~* ^/(media|static)/ {
        expires 365d;
        add_header Cache-Control "public, immutable";
        access_log off;
        log_not_found off;
    }

    # ... other server configurations ...
}

Gzip Compression

Enable Gzip compression in the http block of your nginx.conf. This compresses text-based responses (HTML, CSS, JS, JSON) before sending them to the client.

# /etc/nginx/nginx.conf

http {
    # ... other http configurations ...

    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6; # Compression level (1-9)
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;
    gzip_disable "msie6"; # Disable for older IE versions

    # ... other http configurations ...
}

Remember to test and reload Nginx after these changes.

Gunicorn/PHP-FPM Tuning for Magento 2

The application server (Gunicorn for Python-based frameworks, or PHP-FPM for PHP) is where your Magento 2 code executes. Proper tuning here is critical for handling application requests efficiently.

Gunicorn Configuration (if applicable)

If you’re running Magento 2 with a Python framework (less common, but possible with custom setups or headless architectures), Gunicorn’s worker count and type are key. For a typical PHP setup, this section would be skipped in favor of PHP-FPM.

Worker Processes and Threads

Gunicorn offers different worker types. The sync worker is the default and simplest, but can block under heavy load. gevent or event workers are asynchronous and generally preferred for I/O-bound applications.

A common starting point for sync workers is (2 * number_of_cores) + 1. For asynchronous workers, you might increase this significantly, but monitor memory usage.

Example Gunicorn Command Line

# Example for a Python application
gunicorn --workers 4 --worker-class gevent --bind 0.0.0.0:8000 myapp:app

For production, it’s recommended to manage Gunicorn via a process manager like systemd or supervisor.

PHP-FPM Configuration

PHP-FPM (FastCGI Process Manager) is the standard for running PHP applications like Magento 2. Tuning its process manager settings is vital.

Process Manager Settings

The primary configuration file is usually /etc/php/[version]/fpm/php-fpm.conf, and pool configurations are in /etc/php/[version]/fpm/pool.d/www.conf (or a custom pool name).

pm Directive

This directive controls how PHP-FPM manages child processes. The common options are:

• static: A fixed number of child processes are always kept alive. Good for predictable loads.
• dynamic: Processes are spawned as needed, up to a maximum. More flexible.
• ondemand: Processes are only created when a request comes in and are killed after a timeout. Saves resources but can have higher latency for the first request.

Tuning www.conf (or your pool file)

For a typical DigitalOcean droplet with multiple cores, dynamic is often a good balance. Adjust pm.max_children, pm.start_servers, pm.min_spare_servers, and pm.max_spare_servers based on your server’s RAM and expected traffic.

; /etc/php/[version]/fpm/pool.d/www.conf

[www]
user = www-data
group = www-data
listen = /run/php/php[version]-fpm.sock
listen.owner = www-data
listen.group = www-data
listen.mode = 0660

pm = dynamic
pm.max_children = 100      ; Adjust based on RAM (e.g., 100 * 20MB/process = 2GB RAM)
pm.start_servers = 10      ; Initial number of processes
pm.min_spare_servers = 5   ; Minimum idle processes
pm.max_spare_servers = 20  ; Maximum idle processes
pm.process_idle_timeout = 10s ; Timeout for idle processes to be killed (if pm.max_spare_servers is reached)
pm.max_requests = 500      ; Restart process after this many requests to prevent memory leaks


After modifying the PHP-FPM pool configuration, restart the service:


sudo systemctl restart php[version]-fpm



PHP Settings for Magento 2



Beyond PHP-FPM, core PHP settings in php.ini also impact Magento 2 performance. Key settings include memory limits, execution time, and opcache.



php.ini Tuning



Locate your php.ini file (often found via php --ini or in /etc/php/[version]/fpm/php.ini). Adjust the following:


; /etc/php/[version]/fpm/php.ini

memory_limit = 512M
max_execution_time = 180
upload_max_filesize = 64M
post_max_size = 64M
date.timezone = UTC ; Or your server's timezone

; Opcache settings (essential for performance)
opcache.enable=1
opcache.memory_consumption=128 ; MB
opcache.interned_strings_buffer=16
opcache.max_accelerated_files=10000
opcache.revalidate_freq=60
opcache.validate_timestamps=1 ; Set to 0 in production for slight performance gain, but requires manual cache clearing on code deploy
opcache.save_comments=1
opcache.enable_cli=1 ; Enable for CLI commands like bin/magento


Restart PHP-FPM after changing php.ini.



Redis Configuration and Best Practices



Redis is indispensable for Magento 2, serving as a cache backend for sessions, configuration, page cache, and more. Proper Redis configuration and management are critical for optimal performance.



Redis Configuration File



The main configuration file is typically /etc/redis/redis.conf.



Key Directives for Magento 2


# /etc/redis/redis.conf

# Binding to localhost is generally recommended for security if Redis is on the same server as Magento.
# If Redis is on a separate server, bind to the private IP of the Redis server.
bind 127.0.0.1 -::1

# Set a strong password for security.
# requirepass your_very_strong_password

# Max memory to use. Crucial for preventing Redis from consuming all system RAM.
# Set this to a value that leaves enough RAM for your OS and Magento processes.
# Example: 2GB for a server with 4GB RAM.
maxmemory 2gb
maxmemory-policy allkeys-lru ; Eviction policy: LRU (Least Recently Used) is common for caches.

# Persistence: For cache roles, disabling RDB snapshots and AOF is often acceptable
# and can improve performance by reducing disk I/O. If you need persistence for
# session data, consider a separate Redis instance or a different strategy.
save ""
appendonly no

# TCP keepalive settings can help maintain connections, especially over slower networks.
tcp-keepalive 300

# Increase client output buffer limits if you encounter "ERR maxclients" or "ERR output buffer overflow"
# These values are aggressive and should be tuned based on observed traffic.
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 0 0 0
client-output-buffer-limit pubsub 32mb 64mb 60


After modifying redis.conf, restart the Redis service:


sudo systemctl restart redis-server



Magento 2 Redis Configuration



Magento 2's configuration for Redis is managed via its app/etc/env.php file. Ensure you have separate Redis databases configured for different Magento functionalities.



env.php Setup



A typical env.php configuration for Magento 2 using Redis:


<?php
return [
    'backend' => [
        'front' => [
            'cache_storage_factory' => 'Magento\\Framework\\Cache\\Frontend\\Factory',
            'cache_storage_config' => [
                'frontend' => [
                    'default' => [
                        'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                        'options' => [
                            'server' => '127.0.0.1',
                            'port' => 6379,
                            'database' => 0, // Magento Cache DB
                            'password' => 'your_very_strong_password',
                            'compress_data' => '10', // Compression level for Redis data
                            'compression_library' => 'gzip'
                        ]
                    ],
                    'page_cache' => [
                        'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                        'options' => [
                            'server' => '127.0.0.1',
                            'port' => 6379,
                            'database' => 1, // Magento Page Cache DB
                            'password' => 'your_very_strong_password',
                            'compress_data' => '10',
                            'compression_library' => 'gzip'
                        ]
                    ]
                ]
            ]
        ]
    ],
    'cache_types' => [
        'config' => 1,
        'layout' => 1,
        'block_html' => 1,
        'collections' => 1,
        'reflection' => 1,
        'db_ddl' => 1,
        'compiled_config' => 1,
        'eav' => 1,
        'customer_notification' => 1,
        'full_page' => 1,
        'translate' => 1,
        'config_integration' => 1,
        'config_integration_api' => 1,
        'config_webservice' => 1,
        'view_preprocessed' => 1
    ],
    'session' => [
        'save' => 'redis',
        'redis' => [
            'host' => '127.0.0.1',
            'port' => 6379,
            'password' => 'your_very_strong_password',
            'timeout' => '2.5',
            'persistent_identifier' => '',
            'database' => 2, // Magento Session DB
            'compression_threshold' => '2048',
            'compression_library' => 'gzip',
            'log_level' => '3'
        ]
    ]
];



After updating env.php, you must clear the Magento cache:


sudo bin/magento cache:clean
sudo bin/magento cache:flush



Monitoring and Diagnostics



Continuous monitoring is essential to identify bottlenecks and ensure your tuned configurations are performing as expected. Use a combination of system-level tools and application-specific metrics.



System Monitoring Tools



htop / top: Monitor CPU, memory usage, and running processes. Identify high-resource consumers.
iostat: Track disk I/O performance. High wait times can indicate storage bottlenecks.
netstat / ss: Analyze network connections and traffic.
vmstat: Report virtual memory statistics.



Nginx Monitoring



Nginx's stub_status module provides basic metrics. Enable it in your Nginx configuration:


# In http block of nginx.conf or a specific server block
server {
    # ...
    location /nginx_status {
        stub_status;
        allow 127.0.0.1; # Restrict access
        deny all;
    }
    # ...
}



Access http://your_domain.com/nginx_status to see:


Active connections: 123
server accepts handled requests
 167890 167890 335780
Reading: 1 (0.1%) Writing: 2 (0.2%) Waiting: 120 (97.6%)



High Waiting connections might indicate slow application response times or insufficient worker connections. High Active connections suggest high traffic.



PHP-FPM Monitoring



PHP-FPM's status page can be enabled similarly to Nginx's stub_status. Ensure the pm.status_path directive is set in your pool configuration (e.g., /php-fpm-status).


; /etc/php/[version]/fpm/pool.d/www.conf

[www]
; ... other settings ...
pm.status_path = /php-fpm_status
; ... other settings ...


Access http://your_domain.com/php-fpm_status (ensure Nginx is configured to proxy this location if it's not directly accessible). You'll see metrics like:


pool: www
process manager: dynamic
 9 active processes
 15 max active processes
 20 max children reached 0 times
 5 idle processes
 100 requests currently processing
 10.15 active processes limit


A high number of active processes approaching max_children, or frequent max children reached messages, indicates PHP-FPM is a bottleneck.



Redis Monitoring



Use the redis-cli tool for monitoring:


redis-cli
INFO memory
INFO stats
INFO clients



Key metrics to watch:



used_memory: Current memory usage. Ensure it's below maxmemory.
used_memory_peak: Peak memory usage.
evicted_keys: Number of keys evicted due to memory limits. High numbers indicate insufficient memory or an aggressive eviction policy.
total_connections_received: Total clients connected.
connected_clients: Current number of connected clients.
rejected_connections: Number of rejected connections (often due to maxclients limit).



For more advanced monitoring, integrate with tools like Prometheus and Grafana using the Redis Exporter.