Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Elasticsearch on OVH for Magento 2

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Elasticsearch on OVH for Magento 2

Nginx Configuration for High-Traffic Magento 2

Optimizing Nginx is paramount for serving Magento 2 efficiently, especially under heavy load. We’ll focus on key directives that directly impact performance and resource utilization on OVH infrastructure.

Worker Processes and Connections

The `worker_processes` directive dictates how many worker processes Nginx will spawn. A common recommendation is to set this to the number of CPU cores available. For `worker_connections`, this defines the maximum number of simultaneous connections that each worker process can handle. The total maximum connections will be `worker_processes * worker_connections`.

On an OVH instance, you can determine the number of CPU cores using `nproc` or by inspecting `/proc/cpuinfo`. Let’s assume a server with 8 cores.

Edit your main Nginx configuration file (typically `/etc/nginx/nginx.conf`):

user www-data;
worker_processes 8; # Set to the number of CPU cores
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Adjust based on expected load and memory
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # ... other http configurations
}

Tuning `worker_connections`

The value of `worker_connections` should be set considering available RAM and the expected number of concurrent users. Each connection consumes a small amount of memory. A common starting point is 4096, but this can be increased if you have ample RAM and anticipate very high concurrency. Ensure your system’s file descriptor limit (`ulimit -n`) is high enough to accommodate `worker_processes * worker_connections`.

To check and increase the file descriptor limit:

# Check current limit
ulimit -n

# Temporarily increase for the current session
ulimit -n 65536

# To make it permanent, edit /etc/security/limits.conf
# Add these lines (replace 'www-data' with your Nginx user if different):
# www-data soft nofile 65536
# www-data hard nofile 65536

# Also, ensure systemd service files (if applicable) have the correct limits.
# For example, in /etc/systemd/system/nginx.service.d/override.conf:
# [Service]
# LimitNOFILE=65536
# LimitNOFILE_N=65536

Caching and Compression

Leveraging Nginx’s caching and Gzip compression can significantly reduce server load and improve response times. For Magento 2, it’s crucial to cache static assets and potentially full page caches (though this is often handled by Varnish or dedicated Magento caching modules).

Enable Gzip compression:

http {
    # ... other http configurations

    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;

    # ... rest of http configuration
}

Configure browser caching for static assets. This is typically done within your Magento 2 site’s server block.

server {
    # ... server configuration

    location ~* ^/(media|static)/ {
        expires 30d; # Cache for 30 days
        add_header Cache-Control "public";
    }

    location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 30d;
        add_header Cache-Control "public";
    }

    # ... rest of server configuration
}

Gunicorn/PHP-FPM Tuning for Magento 2

The application server (Gunicorn for Python-based frameworks, or PHP-FPM for PHP) is where your Magento 2 code executes. Tuning its process management and resource allocation is critical.

Gunicorn Configuration (if applicable)

If you’re running a custom Magento 2 module or a related service using Python, Gunicorn is a common WSGI HTTP Server. Key parameters include `workers`, `threads`, and `worker_connections` (though `threads` is more relevant for Gunicorn). The optimal number of workers is often calculated as `(2 * Number of CPU Cores) + 1`.

# Example Gunicorn configuration (gunicorn.conf.py)
import multiprocessing

bind = "0.0.0.0:8000"
workers = multiprocessing.cpu_count() * 2 + 1
threads = 2 # Adjust based on I/O bound nature of your app
worker_connections = 1000 # Max concurrent connections per worker
timeout = 120 # Request timeout in seconds
keepalive = 5 # Keepalive timeout in seconds

# For Magento 2, consider memory usage and potential for long-running cron jobs.
# Adjust threads and worker_connections based on profiling.

PHP-FPM Configuration

For standard Magento 2 installations, PHP-FPM is the workhorse. Tuning its process manager (`pm`) is essential. The most common modes are `static`, `dynamic`, and `ondemand`. For Magento 2, `dynamic` or `static` are generally preferred for consistent performance.

Edit your PHP-FPM pool configuration file (e.g., `/etc/php/8.1/fpm/pool.d/www.conf` or similar, depending on your PHP version and OVH setup).

`pm = dynamic`

This mode starts a few children initially and spawns more as needed, up to `pm.max_children`. It’s a good balance between resource usage and responsiveness.

; Choose how the process manager will control the number of child processes.
; Possible values: 'static', 'dynamic', 'ondemand'.
pm = dynamic

; The number of child processes to be created when pm = dynamic.
; This is the maximum number of children that will be spawned at the same time.
; A good starting point is (total RAM - OS/other services RAM) / (PHP memory_limit * number of processes)
; For Magento, this can be high. Start with a conservative estimate and monitor.
pm.max_children = 100

; The number of *additional* child processes which will be spawned when the following conditions are met.
; Default value: 0
pm.start_servers = 10

; The minimum number of children to always keep running.
pm.min_spare_servers = 5

; The maximum number of children to leave running in the background.
pm.max_spare_servers = 20

; The number of requests each child process should execute before respawning.
; This helps to prevent memory leaks. For Magento, a higher value might be acceptable
; if memory usage is well-managed, but a moderate value is safer.
pm.max_requests = 500

`pm = static`

This mode pre-forks a fixed number of children. It offers the most consistent performance but can be resource-intensive if `pm.max_children` is set too high.

pm = static
pm.max_children = 150 ; Set this to a value that your server can sustain under peak load
pm.max_requests = 0 ; Disable respawning for maximum consistency (use with caution)

Important Considerations for PHP-FPM:

  • `memory_limit`: Ensure `memory_limit` in `php.ini` is sufficient for Magento 2 operations (e.g., `512M` or `1G`).
  • `max_execution_time`: For CLI scripts (cron jobs), this should be much higher than for web requests. For web requests, a reasonable limit (e.g., `300`) is advisable.
  • OPcache: Always ensure OPcache is enabled and properly configured for significant performance gains.

Monitoring PHP-FPM Performance

Enable the PHP-FPM status page to monitor active processes, idle processes, and request counts. This is invaluable for tuning `pm.max_children` and `pm.max_spare_servers`.

; In your PHP-FPM pool configuration (e.g., www.conf)
pm.status_path = /fpm-status
ping.path = /fpm-ping
ping.response = pong

Then, configure Nginx to proxy requests to this status page:

location ~ ^/fpm-status {
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_pass unix:/var/run/php/php8.1-fpm.sock; # Adjust socket path
    internal;
}

Elasticsearch Tuning for Magento 2

Elasticsearch is a critical component for Magento 2’s search functionality. Performance issues here can severely impact user experience. Tuning involves JVM heap size, indexing settings, and query optimization.

JVM Heap Size

Elasticsearch runs on the Java Virtual Machine (JVM). The heap size (`Xms` and `Xmx`) is the most crucial setting. It should be set to no more than 50% of the system’s available RAM, and never exceed 30-32GB due to compressed ordinary object pointers (compressed oops).

Edit the Elasticsearch JVM options file. The location varies by installation method (e.g., `/etc/elasticsearch/jvm.options` for package installs, or environment variables for Docker/manual builds).

-Xms4g
-Xmx4g
# Adjust '4g' based on your server's RAM. For an 8GB RAM server, 4GB is a good start.
# For a 16GB RAM server, consider 8g. Never exceed ~30g.

Indexing Settings

For Magento 2, the default number of primary shards and replicas might not be optimal. Magento typically creates one index per store view for products. Consider the trade-off between search speed (more shards) and indexing speed/resource usage.

You can adjust these settings during index creation or by updating existing index templates. Magento’s `catalogsearch_fulltext` index is a primary target.

# Example: Update index settings via API (use with caution, ideally via Magento CLI commands)
# This is a conceptual example; actual implementation might involve Magento's indexing logic.

PUT /_template/magento2_template
{
  "index_patterns": ["magento2_*"],
  "settings": {
    "index": {
      "number_of_shards": 4,       # Adjust based on data volume and query load
      "number_of_replicas": 1      # Adjust based on read load and fault tolerance needs
    }
  }
}

# For Magento 2, you might need to reindex after changing shard/replica counts.
# Use Magento's CLI for proper index management:
# bin/magento indexer:reindex catalogsearch_fulltext

`number_of_shards`: Determines how many primary shards an index is split into. More shards can improve parallel query processing but increase overhead. For Magento, consider the number of store views and product catalog size. A common starting point might be 4-8 shards per index.

`number_of_replicas`: Creates copies of primary shards. Increases read throughput and fault tolerance but consumes more disk space and indexing resources. For a single-node setup on OVH, 0 or 1 replica is typical. For a cluster, 1 or 2 is common.

Query Optimization and Monitoring

Slow search queries are a major bottleneck. Use Elasticsearch’s Slow Log feature to identify problematic queries.

# In elasticsearch.yml
index.search.slowlog.threshold.query: 1s  # Log queries taking longer than 1 second
index.search.slowlog.threshold.fetch: 500ms # Log fetch phases taking longer than 500ms
index.indexing.slowlog.threshold.index: 5s # Log indexing operations taking longer than 5 seconds
index.indexing.slowlog.threshold.bulk: 1s  # Log bulk operations taking longer than 1 second

Analyze the slow logs (typically in `/var/log/elasticsearch/`) to understand query patterns. Common issues include overly broad queries, missing filters, or inefficient aggregations. Magento’s default search can sometimes be chatty; consider using a dedicated Elasticsearch extension for Magento that provides more optimized query structures.

Monitor Elasticsearch cluster health using the `_cat` APIs:

curl -X GET "localhost:9200/_cat/health?v"
curl -X GET "localhost:9200/_cat/indices?v"
curl -X GET "localhost:9200/_cat/nodes?v"
curl -X GET "localhost:9200/_cat/thread_pool?v"

Pay close attention to the `thread_pool` for `search` and `write` queues. If they are consistently high, it indicates a bottleneck in query processing or indexing, respectively. This might require scaling up the Elasticsearch instance (more RAM, faster disks) or optimizing queries/indexing strategies.