The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and PostgreSQL on DigitalOcean for Magento 2

Nginx Configuration for Magento 2 Performance

Optimizing Nginx is crucial for serving static assets efficiently and proxying dynamic requests to your Magento 2 application. We’ll focus on key directives that impact performance and security.

Worker Processes and Connections

The worker_processes directive determines how many worker processes Nginx will spawn. Setting this to auto allows Nginx to detect the number of CPU cores and utilize them effectively. worker_connections defines the maximum number of simultaneous connections that each worker process can handle. A common starting point is 1024, but this may need tuning based on your server’s load and RAM.

Edit your main Nginx configuration file (typically /etc/nginx/nginx.conf) and adjust the events block:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 1024;
    multi_accept on;
}

Buffering and Caching

Nginx buffering can significantly improve performance by reducing the number of read/write operations. client_body_buffer_size controls the buffer size for client request bodies. proxy_buffer_size and proxy_buffers are critical for buffering responses from upstream servers (like Gunicorn or PHP-FPM). Large files or slow upstream responses can benefit from larger buffer sizes.

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    client_body_buffer_size 10K;
    proxy_buffer_size 128k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;

    # ... other http configurations
}

Gzip Compression

Enabling Gzip compression reduces the size of text-based assets (HTML, CSS, JS) sent to the client, leading to faster load times. Ensure you’re not compressing already compressed content like images.

http {
    # ... other http configurations

    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;
    gzip_disable "msie6"; # Disable for older IE versions
}

Static File Caching

Leverage browser caching for static assets by setting appropriate expires headers. This reduces server load as clients will fetch these files from their local cache on subsequent requests.

location ~* \.(jpg|jpeg|png|gif|ico|css|js|svg|woff|woff2|ttf|eot)$ {
    expires 30d;
    add_header Cache-Control "public, no-transform";
    access_log off;
}

Magento 2 Specific Nginx Configuration

Magento 2 requires specific Nginx configurations for static files, media, and security. Ensure your Magento 2 site’s server block includes these directives. The try_files directive is critical for routing requests correctly.

server {
    listen 80;
    server_name your_domain.com www.your_domain.com;
    root /var/www/magento2; # Adjust to your Magento installation path
    index index.php index.html index.htm;

    # Magento 2 specific configurations
    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~* ^/(media|static)/ {
        expires 30d;
        add_header Cache-Control "public, no-transform";
        access_log off;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/var/run/php/php8.1-fpm.sock; # Adjust PHP version and socket path
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    # Deny access to sensitive files
    location ~* /(composer\.json|composer\.lock|\.git|\.svn|var/cache|var/session|var/log|app/etc/env\.php) {
        deny all;
    }

    # Other security headers and configurations
    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Content-Type-Options "nosniff";
    add_header Referrer-Policy "strict-origin-when-cross-origin";
    # add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"; # Uncomment for HSTS
}

Gunicorn Configuration for Magento 2 (Python/WSGI)

If you’re using Gunicorn to serve a Python WSGI application that interfaces with Magento 2 (e.g., via a custom API or a headless setup), tuning Gunicorn is essential. We’ll focus on worker types, worker counts, and timeouts.

Worker Types and Count

Gunicorn offers several worker types. For I/O-bound applications, the gevent or eventlet workers are excellent choices due to their asynchronous nature. For CPU-bound tasks, the default sync worker or threads worker might be considered, though sync is generally simpler to reason about. The number of workers is typically set to (2 * number_of_cores) + 1 as a starting point.

A common Gunicorn command-line invocation or configuration file (e.g., gunicorn_config.py) would look like this:

# gunicorn_config.py
import multiprocessing

bind = "0.0.0.0:8000" # Or your desired host:port
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = "gevent" # or "sync", "eventlet", "threads"
threads = 2 # Only applicable if worker_class is "threads"

# Adjust based on your application's needs and server resources
# For gevent/eventlet, threads are not directly used in the same way as 'threads' worker class.
# The 'workers' count is more critical for concurrency.

# Timeout settings
# The timeout is the number of seconds to wait for a worker to respond.
# Magento operations can be long-running, so a higher timeout might be necessary.
# Be cautious: too high a timeout can mask performance issues or lead to resource exhaustion.
timeout = 120 # seconds

# Graceful shutdown
graceful_timeout = 120 # seconds

# Logging
loglevel = "info"
accesslog = "-" # Log to stdout
errorlog = "-"  # Log to stderr

# Other useful settings
# max_requests = 1000 # Restart workers after this many requests
# max_requests_jitter = 50 # Randomize max_requests to avoid thundering herd

To run Gunicorn with this configuration:

gunicorn --config gunicorn_config.py your_wsgi_app:app

Timeouts and Request Handling

Magento 2 operations, especially complex product imports, order processing, or API calls, can take a significant amount of time. The timeout directive in Gunicorn specifies how long the master process will wait for a worker to respond. Setting this too low will result in 504 Gateway Timeout errors. Setting it too high can tie up worker processes and mask underlying performance problems. A value between 60 and 180 seconds is often a reasonable starting point for Magento, but this requires careful monitoring.

Gunicorn and Nginx Integration

Nginx will proxy requests to Gunicorn. Ensure your Nginx configuration correctly points to Gunicorn’s listening address (e.g., a Unix socket or a TCP port).

# In your Magento 2 Nginx server block
location / {
    proxy_pass http://unix:/path/to/your/gunicorn.sock; # If using a Unix socket
    # OR
    # proxy_pass http://127.0.0.1:8000; # If Gunicorn is listening on TCP

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_connect_timeout 75s;
    proxy_read_timeout 300s; # Match or exceed Gunicorn's timeout
}

PHP-FPM Configuration for Magento 2 (PHP)

For traditional PHP deployments of Magento 2, tuning PHP-FPM is critical. We’ll focus on process management, memory limits, and execution times.

Process Manager Settings

PHP-FPM offers different process management strategies: static, dynamic, and ondemand. For Magento 2, which can have spiky traffic, dynamic or ondemand are often preferred. dynamic is a good balance, starting with a few processes and scaling up as needed. ondemand starts with no processes and spawns them only when requests arrive, which can save memory but might introduce a slight delay on the first request after idle periods.

Edit your PHP-FPM pool configuration file (e.g., /etc/php/8.1/fpm/pool.d/www.conf). Adjust the following directives:

; pm = dynamic ; pm.max_children = 50 ; pm.start_servers = 5 ; pm.min_spare_servers = 2 ; pm.max_spare_servers = 10 ; pm.process_idle_timeout = 10s ; pm.max_requests = 500

Explanation:

• pm: Process manager control method (dynamic, static, ondemand).
• pm.max_children: The maximum number of child processes that will be spawned. This is the most important setting for preventing memory exhaustion. Calculate this based on your server’s RAM and the memory footprint of a single PHP-FPM worker.
• pm.start_servers: Number of child processes to start when PHP-FPM starts.
• pm.min_spare_servers: Minimum number of idle supervisor processes.
• pm.max_spare_servers: Maximum number of idle supervisor processes.
• pm.process_idle_timeout: How long a child process can be idle before being killed (used with dynamic).
• pm.max_requests: The number of requests each child process should execute before reexecuting. This helps prevent memory leaks.

For static PM, you’d set a fixed number of children:

pm = static
pm.max_children = 100 ; Adjust based on RAM

Resource Limits

Magento 2 can be memory-intensive. Ensure your PHP memory limit and execution time are set appropriately. These can be configured in php.ini or directly in the FPM pool configuration.

; In php.ini or FPM pool config
memory_limit = 512M ; Or higher, depending on your needs and server RAM
max_execution_time = 180 ; seconds (for CLI scripts, this is different)
max_input_vars = 3000
upload_max_filesize = 64M
post_max_size = 64M

For FPM pool configuration, you can override these:

; In FPM pool config (e.g., www.conf)
php_admin_value[memory_limit] = 512M
php_admin_value[max_execution_time] = 180
php_admin_value[max_input_vars] = 3000

PHP-FPM and Nginx Integration

Ensure Nginx is configured to pass PHP requests to the correct PHP-FPM socket or TCP port. The fastcgi_pass directive in your Nginx server block is key.

# In your Magento 2 Nginx server block
location ~ \.php$ {
    include snippets/fastcgi-php.conf; # Common FastCGI parameters
    # Use the correct socket path for your PHP version
    fastcgi_pass unix:/var/run/php/php8.1-fpm.sock;
    # OR if FPM is listening on TCP
    # fastcgi_pass 127.0.0.1:9000;
}

PostgreSQL Tuning for Magento 2

PostgreSQL is the recommended database for Magento 2. Optimizing its configuration is paramount for database-intensive operations common in e-commerce.

Shared Buffers

shared_buffers is arguably the most critical parameter. It determines how much memory PostgreSQL uses to cache data and indexes. A common recommendation is 25% of your server’s total RAM, but this can be adjusted based on your workload and available memory. For a 16GB RAM server, 4GB (4096MB) is a good starting point.

# postgresql.conf
shared_buffers = 4096MB

WAL (Write-Ahead Logging)

WAL settings impact durability and performance. Tuning wal_buffers and wal_writer_delay can improve write performance. checkpoint_segments (or max_wal_size in newer versions) and checkpoint_timeout control how often checkpoints occur, which can cause I/O spikes.

# postgresql.conf
wal_buffers = 16MB # Typically 1/4 of shared_buffers, up to 16MB
wal_writer_delay = 200ms # Default is 200ms, can be reduced slightly if needed

# For PostgreSQL < 9.5
# checkpoint_segments = 64 # Adjust based on WAL volume and disk speed
# checkpoint_timeout = 5min # Default is 5min

# For PostgreSQL >= 9.5
max_wal_size = 4GB # Adjust based on WAL volume and disk speed
min_wal_size = 1GB # Keep some WAL files around
checkpoint_timeout = 15min # Less frequent checkpoints can reduce I/O spikes
# checkpoint_completion_target = 0.9 # Spread checkpoint I/O over time

Work Memory

work_mem controls the amount of memory that can be used for internal sort operations and hash tables before writing to temporary disk files. Magento queries, especially for reporting and complex catalog operations, can benefit from larger work_mem values. However, setting this too high can lead to excessive memory consumption if many queries require large sorts simultaneously.

# postgresql.conf
work_mem = 64MB # Start with this and tune based on query analysis

You can set work_mem per session for specific queries or users if needed, but a global setting is often sufficient for general tuning.

Connection Pooling

Magento 2 can open many database connections. Using a connection pooler like PgBouncer can significantly reduce connection overhead and improve performance. Configure PgBouncer to use a transaction or session pooling mode.

# pgbouncer.ini
[databases]
pgbouncer_db = host=127.0.0.1 port=5432 dbname=magento_db

[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 6432
auth_file = /etc/pgbouncer/userlist.txt
pool_mode = transaction ; or session
max_client_conn = 1000
default_pool_size = 20
reserve_pool_size = 5

Then, configure your Magento application (or your PHP-FPM/Gunicorn connection string) to connect to PgBouncer’s port (e.g., 6432) instead of directly to PostgreSQL.

Autovacuum Tuning

Autovacuum is essential for reclaiming space from dead tuples and preventing table bloat. Magento’s frequent updates and deletions can lead to significant bloat if autovacuum is not aggressive enough.

# postgresql.conf
autovacuum = on
log_autovacuum_min_duration = 1000ms ; Log autovacuum actions taking longer than 1s
autovacuum_max_workers = 3 ; Number of concurrent autovacuum processes
autovacuum_naptime = 15s ; How often the autovacuum daemon checks for work
autovacuum_vacuum_threshold = 50 ; Minimum number of rows to trigger a vacuum
autovacuum_analyze_threshold = 50 ; Minimum number of rows to trigger an analyze
autovacuum_vacuum_scale_factor = 0.1 ; Percentage of table size to trigger vacuum
autovacuum_analyze_scale_factor = 0.05 ; Percentage of table size to trigger analyze

You may need to tune these parameters further based on your specific table sizes and activity levels. Consider setting per-table storage parameters for critical tables.

Monitoring and Iterative Tuning

Performance tuning is an ongoing process. Implement robust monitoring for all components:

• Nginx: Use stub_status module for active connections, request rates. Monitor error logs for 5xx errors.
• PHP-FPM: Monitor FPM status page for active processes, queue lengths. Check PHP error logs.
• Gunicorn: Monitor worker status, request latency, and error rates.
• PostgreSQL: Use pg_stat_activity, pg_stat_statements (requires extension), and tools like pgtune or pgbench for load testing. Monitor disk I/O, CPU, and memory usage.
• System: Use tools like htop, vmstat, iostat, and Prometheus/Grafana for comprehensive system-level monitoring.

Start with conservative settings, monitor the impact, and incrementally adjust parameters. Always test changes in a staging environment before deploying to production. Analyze slow queries in PostgreSQL and optimize them, as database performance often has the most significant impact on Magento 2.