The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Elasticsearch on Linode for Magento 2

Nginx Configuration for Magento 2 Performance

Optimizing Nginx is paramount for serving Magento 2 efficiently. We’ll focus on key directives that impact request handling, caching, and resource utilization. This assumes a standard Linode setup with Nginx serving as the primary web server and reverse proxy.

Worker Processes and Connections

The worker_processes directive controls how many worker processes Nginx will spawn. A common recommendation is to set it to the number of CPU cores available. worker_connections defines the maximum number of simultaneous connections that each worker process can handle. The total maximum connections will be worker_processes * worker_connections.

Edit your main Nginx configuration file, typically /etc/nginx/nginx.conf:

user www-data;
worker_processes auto; # Or set to the number of CPU cores, e.g., 4
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Adjust based on expected load and server RAM
    multi_accept on;
}

http {
    # ... other http configurations ...
}

Note: auto for worker_processes is often sufficient and dynamically adjusts based on CPU cores. For worker_connections, consider your server’s RAM and expected concurrent user load. A value of 4096 is a good starting point.

Keepalive Timeout and Buffers

keepalive_timeout specifies the time a persistent connection will remain open. Shorter timeouts can free up resources faster, while longer ones can improve performance for clients making multiple requests. client_body_buffer_size and client_header_buffer_size are crucial for handling large requests and headers.

http {
    # ...
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65; # Default is 65, can be tuned down if needed
    types_hash_max_size 2048;

    client_header_timeout 30; # Default is 60
    client_body_timeout 30;   # Default is 60

    client_header_buffer_size 1k; # Default is 1k
    large_client_header_buffers 4 8k; # Default is 2 4k

    client_body_buffer_size 10m; # Increase for large uploads/POST requests
    client_max_body_size 10m; # Ensure this matches or exceeds client_body_buffer_size

    # ...
}

Tuning: Increase client_body_buffer_size and client_max_body_size if you encounter issues with large file uploads or complex POST requests. The client_header_buffer_size and large_client_header_buffers are important for handling potentially large HTTP headers, especially with many cookies or custom headers.

Gzip Compression

Enabling Gzip compression significantly reduces the size of transferred assets, improving page load times. Ensure it’s configured correctly to avoid compressing already compressed assets (like JPEGs or PNGs).

http {
    # ...
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6; # Compression level (1-9)
    gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;
    gzip_min_length 1000; # Minimum response length to compress
    gzip_disable "msie6"; # Disable for older IE versions if necessary
    # ...
}

Tuning: gzip_comp_level offers a trade-off between CPU usage and compression ratio. Level 6 is a good balance. gzip_min_length prevents small files from being compressed, which can sometimes be counterproductive due to overhead.

Caching Directives

Leverage Nginx’s ability to serve static assets directly from its cache. This offloads significant work from your PHP-FPM/Gunicorn processes.

location ~* ^/(media|static)/ {
    expires 30d; # Cache for 30 days
    add_header Cache-Control "public, immutable";
    access_log off; # Optionally disable access logs for static files
    log_not_found off;
}

Tuning: Adjust the expires directive based on how frequently your static assets change. For Magento 2, 30d is generally safe for media and static directories. The immutable directive is a strong hint to the browser that the content will not change.

Magento 2 Specific Nginx Configuration

Ensure your Magento 2 site-specific Nginx configuration (often in /etc/nginx/sites-available/your-magento-site.conf) includes directives for handling static files, media, and routing requests correctly.

server {
    listen 80;
    server_name your-magento-domain.com;
    root /var/www/your-magento-site/public_html; # Adjust to your Magento root

    index index.php index.html index.htm;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    # Magento 2 static content and media
    location ~ ^/(media|static)/ {
        expires 30d;
        add_header Cache-Control "public, immutable";
        access_log off;
        log_not_found off;
    }

    # PHP-FPM configuration (adjust socket path if using TCP)
    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/var/run/php/php8.1-fpm.sock; # Adjust PHP version and socket path
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
        fastcgi_param MAGE_RUN_CODE "your_store_code"; # Optional: for specific store views
        fastcgi_param MAGE_RUN_TYPE "store";         # Optional: for specific store views
    }

    # Deny access to sensitive files
    location ~ /\.ht {
        deny all;
    }

    # Deny access to .git directory
    location ~ /\.git {
        deny all;
    }
}

Tuning: The MAGE_RUN_CODE and MAGE_RUN_TYPE parameters are useful for multi-store Magento installations, allowing Nginx to pass context to PHP-FPM for specific store views. Ensure the fastcgi_pass directive points to your correct PHP-FPM socket or TCP address.

Gunicorn/PHP-FPM Tuning for Magento 2

Whether you’re using Gunicorn for a Python-based backend or PHP-FPM for a traditional PHP setup, tuning these application servers is critical. For Magento 2, we’ll primarily focus on PHP-FPM as it’s the most common deployment.

PHP-FPM Configuration

PHP-FPM’s performance is heavily influenced by its process manager settings. The most common process managers are static, dynamic, and ondemand. For Magento 2, which can have significant memory footprints and varying load patterns, dynamic or ondemand are often preferred.

Edit your PHP-FPM pool configuration file, typically found in /etc/php/X.Y/fpm/pool.d/www.conf (replace X.Y with your PHP version, e.g., 8.1).

[www]
user = www-data
group = www-data
listen = /var/run/php/php8.1-fpm.sock # Or a TCP socket like 127.0.0.1:9000

; Process Manager Settings
; pm = dynamic # or ondemand
; pm.max_children = 50 # Max number of children
; pm.start_servers = 5  # Number of children to start with
; pm.min_spare_servers = 2 # Min number of idle respawners
; pm.max_spare_servers = 10 # Max number of idle respawners
; pm.process_idle_timeout = 10s # For ondemand

; pm.max_requests = 500 # Restart a child after this many requests

; Memory Limit - Crucial for Magento
memory_limit = 512M # Adjust based on server RAM and Magento's needs
max_execution_time = 180 # Long running tasks like indexing
max_input_vars = 3000 # For large forms and configurations

; Error Reporting (for production, log errors, don't display)
display_errors = Off
log_errors = On
error_log = /var/log/php/php-fpm.log

; Other important settings
realpath_cache_size = 4096k
realpath_cache_ttl = 600

Tuning:

• pm: For Magento 2, dynamic is a good starting point. It scales the number of worker processes based on load. ondemand can save memory but might introduce slight latency on initial requests.
• pm.max_children: This is the most critical setting. It dictates the maximum number of PHP-FPM processes that can run concurrently. Set this based on your server’s RAM. A common rule of thumb is to calculate the average memory usage per PHP process (e.g., 50MB-100MB for Magento) and divide your total RAM by this figure, then subtract memory for the OS and other services. For a 4GB RAM server, 50-75 might be appropriate. Monitor memory usage closely.
• pm.start_servers: A good starting point is (min_spare_servers + max_spare_servers) / 2.
• pm.min_spare_servers and pm.max_spare_servers: These control the number of idle processes. Keep them relatively low for dynamic to avoid excessive idle memory consumption.
• pm.max_requests: Setting this to a value like 500 or 1000 helps prevent memory leaks from accumulating over time by restarting worker processes periodically.
• memory_limit: Magento 2 is memory-intensive. 512M is a common minimum for production. For heavy tasks like indexing or imports, you might need to temporarily increase this or use CLI commands with higher limits.
• max_execution_time: Essential for Magento’s cron jobs, indexing, and complex operations. 180 seconds (3 minutes) is a reasonable default.
• max_input_vars: Magento uses this for configuration and form data. A value of 3000 is often recommended to avoid issues with large configuration arrays.

After making changes, reload PHP-FPM: sudo systemctl reload php8.1-fpm.

Gunicorn Tuning (If Applicable)

If your Magento setup involves a Python backend (e.g., custom modules, headless implementations), Gunicorn tuning is relevant. The principles are similar: manage worker processes and timeouts.

gunicorn --workers 4 --threads 2 --timeout 120 --bind 0.0.0.0:8000 your_app.wsgi:application

Tuning:

• --workers: Typically set to (2 * number_of_cpu_cores) + 1.
• --threads: For I/O-bound applications, threads can improve concurrency within a worker process.
• --timeout: Similar to PHP’s max_execution_time, crucial for long-running requests.

Elasticsearch Tuning for Magento 2

Elasticsearch is a critical component for Magento 2’s layered navigation, search, and catalog performance. Proper JVM heap sizing and shard allocation are key.

JVM Heap Size

Elasticsearch runs on the Java Virtual Machine (JVM). The heap size dictates how much memory the JVM can use. Setting this too low causes frequent garbage collection and poor performance; setting it too high can starve other processes or lead to excessive swapping.

Edit the Elasticsearch JVM options file. The location varies by installation method, but it’s often /etc/elasticsearch/jvm.options or within /etc/elasticsearch/jvm.options.d/.

-Xms1g
-Xmx1g

Tuning:

• Set both -Xms (initial heap size) and -Xmx (maximum heap size) to the same value to prevent resizing.
• A common recommendation is to allocate 50% of the server’s available RAM to Elasticsearch, but never more than 30-32GB. Modern JVMs have optimizations that work best when the heap is below this threshold.
• For a Linode with 8GB RAM dedicated to Elasticsearch, -Xmx4g is a reasonable starting point. Monitor memory usage and Elasticsearch performance metrics.

After changing jvm.options, restart Elasticsearch: sudo systemctl restart elasticsearch.

Shard Allocation and Indexing Performance

Magento 2 typically creates one index per store view for Elasticsearch. The number of primary shards per index affects indexing speed and search performance. Too many shards can increase overhead; too few can limit parallelism.

You can adjust the number of shards and replicas using the Elasticsearch API. For Magento 2, this is often managed via configuration settings within Magento itself or through custom scripts.

# Example: Get current settings for a Magento index
curl -X GET "localhost:9200/magento2_product_1/_settings?pretty"

# Example: Update settings (e.g., change number of replicas)
curl -X PUT "localhost:9200/magento2_product_1/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index" : {
    "number_of_replicas" : 1
  }
}'

Tuning:

• Primary Shards: For Magento 2, the default of 1 primary shard per index is often sufficient, especially on smaller to medium-sized deployments. If you have very large catalogs and are experiencing indexing bottlenecks, consider increasing this, but be mindful of the overhead.
• Replicas: Replicas provide high availability and read scalability. For production, at least 1 replica is recommended. More replicas can improve read performance but increase indexing time and storage requirements.
• Index Refresh Interval: By default, Elasticsearch refreshes indices every 1 second, making documents searchable. For large reindexing operations, temporarily increasing this interval (e.g., to 30 or 60 seconds) can significantly speed up indexing.

# Temporarily increase refresh interval during reindex
curl -X PUT "localhost:9200/magento2_product_1/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index" : {
    "refresh_interval" : "30s"
  }
}'

# Revert after reindex
curl -X PUT "localhost:9200/magento2_product_1/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index" : {
    "refresh_interval" : "1s"
  }
}'

Note: Changes to shard count (primary or replica) typically require reindexing or complex index management operations. Adjusting the refresh interval is the easiest way to impact indexing speed.

System-Level Tuning for Elasticsearch

Ensure your Linode server is optimized for Elasticsearch:

• Swappiness: Set vm.swappiness to a low value (e.g., 1 or 10) to discourage the OS from swapping Elasticsearch’s memory. Edit /etc/sysctl.conf and apply with sudo sysctl -p.
• File Descriptors: Increase the maximum number of open file descriptors for the Elasticsearch user. Edit /etc/security/limits.conf.

# In /etc/sysctl.conf
vm.swappiness = 10

# In /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
elasticsearch soft nofile 65536
elasticsearch hard nofile 65536

Remember to restart Elasticsearch after applying system-level changes that affect its environment.