Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Elasticsearch on DigitalOcean for Magento 2

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Elasticsearch on DigitalOcean for Magento 2

Nginx Configuration for Magento 2 on DigitalOcean

Optimizing Nginx is paramount for Magento 2 performance. We’ll focus on key directives that directly impact request handling, caching, and resource utilization. This assumes a standard DigitalOcean droplet with a recent Ubuntu LTS release.

Worker Processes and Connections

The worker_processes directive dictates how many worker processes Nginx will spawn. A common recommendation is to set this to the number of CPU cores available. worker_connections defines the maximum number of simultaneous connections that each worker process can handle. The total maximum connections will be worker_processes * worker_connections.

Edit your main Nginx configuration file, typically /etc/nginx/nginx.conf:

user www-data;
worker_processes auto; # Or set to the number of CPU cores, e.g., worker_processes 4;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 1024; # Adjust based on expected load and server RAM
    multi_accept on;
}

http {
    # ... other http configurations ...
}

After modifying nginx.conf, test the configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Gzip Compression and Caching Headers

Enabling Gzip compression significantly reduces the size of transferred assets. Properly setting cache control headers is crucial for browser and proxy caching, reducing server load.

Add or modify these directives within the http block in /etc/nginx/nginx.conf or a dedicated conf file in /etc/nginx/conf.d/:

http {
    # ... other http configurations ...

    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;

    # Caching headers for static assets
    location ~* \.(jpg|jpeg|png|gif|ico|css|js|svg|webp|woff|woff2|ttf|eot)$ {
        expires 365d;
        add_header Cache-Control "public, immutable";
    }

    # ... other http configurations ...
}

Reload Nginx after these changes.

Magento 2 Specific Nginx Configuration

Magento 2 requires specific Nginx rules for static file serving, security, and routing. The official Magento documentation provides a comprehensive nginx.conf.sample. Ensure your Magento site’s virtual host configuration correctly includes or merges these rules. A common location for this is /etc/nginx/sites-available/your-magento-site.

Key sections to scrutinize:

  • Static File Serving: Directives that serve static assets (CSS, JS, images) directly from the filesystem, bypassing PHP.
  • Media Files: Similar to static files, but often with different caching strategies.
  • Rewrites and Redirects: Magento relies heavily on Nginx rewrites for clean URLs and SEO.
  • Security Headers: Directives for X-Frame-Options, Content-Security-Policy, etc.

A simplified example of static file handling within a Magento virtual host:

server {
    # ... other server configurations ...

    root /var/www/magento2; # Your Magento root directory
    index index.php index.html index.htm;

    # Serve static files directly
    location /static/ {
        alias /var/www/magento2/pub/static/;
        expires 365d;
        add_header Cache-Control "public, immutable";
        access_log off;
    }

    location /media/ {
        alias /var/www/magento2/pub/media/;
        expires 365d;
        add_header Cache-Control "public";
        access_log off;
    }

    # Magento application handling
    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ ^/index\.php(?:/|$) {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/var/run/php/php8.1-fpm.sock; # Adjust PHP version and socket path
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }

    # ... other server configurations ...
}

Gunicorn/PHP-FPM Tuning for Magento 2

Magento 2 is a PHP application, so tuning PHP-FPM (or HHVM, though less common now) is critical. If you’re using Gunicorn as a reverse proxy for PHP-FPM (a common pattern in some setups), its configuration also matters.

PHP-FPM Configuration

The primary configuration file for PHP-FPM is usually located at /etc/php/[version]/fpm/php.ini and /etc/php/[version]/fpm/pool.d/www.conf. We’ll focus on www.conf for pool-specific settings.

Key directives in /etc/php/[version]/fpm/pool.d/www.conf:

  • pm (Process Manager): Determines how PHP-FPM manages child processes. dynamic is common, but ondemand or static can be tuned.
  • pm.max_children: The maximum number of child processes that will be spawned. This is a critical setting. Too low leads to requests being queued; too high exhausts server memory.
  • pm.start_servers: The number of child processes to start when PHP-FPM starts.
  • pm.min_spare_servers: The minimum number of idle (spare) processes to maintain.
  • pm.max_spare_servers: The maximum number of idle (spare) processes to maintain.
  • pm.max_requests: The number of requests each child process will execute before respawning. This helps prevent memory leaks.

A sample tuning for a medium-sized droplet (e.g., 4GB RAM):

; /etc/php/8.1/fpm/pool.d/www.conf
; Adjust PHP version as needed

[www]
user = www-data
group = www-data
listen = /var/run/php/php8.1-fpm.sock
listen.owner = www-data
listen.group = www-data
listen.mode = 0660

pm = dynamic
pm.max_children = 100       ; Start with a value based on RAM, e.g., (Total RAM - OS/Nginx RAM) / Average PHP Process Size
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 500       ; Good for preventing memory leaks

; Other important php.ini settings (in /etc/php/8.1/fpm/php.ini)
; memory_limit = 512M
; max_execution_time = 180
; upload_max_filesize = 64M
; post_max_size = 64M
; opcache.enable=1
; opcache.memory_consumption=128
; opcache.interned_strings_buffer=16
; opcache.max_accelerated_files=10000
; opcache.revalidate_freq=60
; opcache.validate_timestamps=0 ; Set to 1 in development environments
; opcache.save_comments=1
; opcache.enable_cli=1

After changes, test and restart PHP-FPM:

sudo systemctl restart php8.1-fpm # Adjust version as needed

Gunicorn Configuration (if used as a PHP-FPM proxy)

If Gunicorn is used to proxy requests to PHP-FPM (e.g., via php-fpm-socket), its worker count and timeout settings are relevant. This is less common for pure PHP Magento deployments but can occur in mixed environments.

Example gunicorn.conf.py:

# gunicorn.conf.py
import multiprocessing

bind = "0.0.0.0:8000" # Or your desired bind address and port
workers = multiprocessing.cpu_count() * 2 + 1 # A common starting point
threads = 2 # Adjust based on I/O bound nature of PHP requests
timeout = 300 # Magento can have long-running processes, especially during cron or indexing
keepalive = 5

# If proxying to PHP-FPM:
# worker_class = "gevent" # Or other suitable async worker class if needed
# php_fpm_socket = "/var/run/php/php8.1-fpm.sock" # Path to your PHP-FPM socket

Start/restart Gunicorn with this configuration.

Elasticsearch Tuning for Magento 2

Elasticsearch performance is critical for Magento 2’s catalog search and other features. Tuning involves JVM heap size, shard allocation, and indexing strategies.

JVM Heap Size

The Java Virtual Machine (JVM) heap size is arguably the most important Elasticsearch setting. It should be set to no more than 50% of your server’s total RAM, and never exceed 30-32GB due to compressed ordinary object pointers (compressed oops).

Edit the Elasticsearch JVM options file. On Debian/Ubuntu systems, this is typically /etc/elasticsearch/jvm.options.

-Xms4g
-Xmx4g

In this example, for a server with 8GB RAM, we allocate 4GB to the JVM heap. Restart Elasticsearch after changing:

sudo systemctl restart elasticsearch

Shard Allocation and Indexing

Magento 2 uses Elasticsearch for catalog search. The number of primary shards per index affects performance. For a single-node setup, a single primary shard is often sufficient. For multi-node clusters, consider the trade-offs.

You can control shard allocation using the Cluster Allocation API. To disable shard allocation temporarily (e.g., during major reindexing or upgrades):

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "primaries"
  }
}

To re-enable shard allocation:

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}

When reindexing from Magento, ensure you’re using the command line for optimal performance and to avoid timeouts.

php bin/magento indexer:reindex

For large catalogs, consider tuning index refresh intervals and merge policies, though these are advanced topics often managed by Elasticsearch’s defaults or specific index templates.

Monitoring and Iteration

Tuning is an iterative process. Continuously monitor your system’s performance using tools like:

  • Nginx: stub_status module, access.log analysis, error.log.
  • PHP-FPM: pm.status page, slowlog.
  • Elasticsearch: Elasticsearch APIs (_cat/nodes, _cat/indices, _nodes/stats), dedicated monitoring tools (e.g., Cerebro, Grafana with Elasticsearch data source).
  • System: top, htop, vmstat, iostat, Prometheus Node Exporter.

Adjust settings based on observed bottlenecks. For example, if PHP-FPM pm.max_children is consistently hit and requests are queued, increase it cautiously while monitoring RAM. If Elasticsearch is experiencing high garbage collection pauses, consider heap size or JVM tuning.