The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Redis on OVH for Shopify

Nginx as a High-Performance Frontend for Shopify Applications

When deploying Shopify applications, especially those with custom backends (e.g., using Python/Django/Flask with Gunicorn or PHP/Laravel with FPM), Nginx serves as the indispensable frontend. Its role extends beyond simple request routing; it’s a critical component for caching, SSL termination, load balancing, and static file serving. Optimizing Nginx is paramount for achieving low latency and high throughput.

Our OVH infrastructure often involves dedicated servers or VPS instances. A robust Nginx configuration starts with tuning worker processes and connections. The general rule of thumb for worker_processes is to set it to the number of CPU cores available. This allows Nginx to effectively utilize all available processing power.

Nginx Configuration Tuning

The core Nginx configuration file, typically located at /etc/nginx/nginx.conf, is where we’ll make these adjustments. We’ll also leverage /etc/nginx/conf.d/ for modular site-specific configurations.

Worker Processes and Connections

Adjusting worker_processes and worker_connections is foundational. worker_connections defines the maximum number of simultaneous connections that each worker process can handle. The total maximum connections will be worker_processes * worker_connections.

Example: nginx.conf Snippet

# /etc/nginx/nginx.conf

user www-data;
worker_processes auto; # Set to the number of CPU cores, or 'auto' for Nginx to decide.
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Adjust based on server RAM and expected load.
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    server_tokens off; # Important for security.

    # Gzip Compression
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # SSL Configuration (example)
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384';

    # Load balancing (if applicable)
    # upstream backend_servers {
    #     server 192.168.1.100:8000;
    #     server 192.168.1.101:8000;
    # }

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Explanation:

• worker_processes auto;: Lets Nginx determine the optimal number of worker processes based on CPU cores.
• worker_connections 4096;: A high value that allows each worker to handle many concurrent connections. This should be balanced against available RAM.
• sendfile on;: Enables efficient transfer of files from disk to network socket.
• tcp_nopush on;: Instructs Nginx to try and send HTTP response headers in one packet.
• tcp_nodelay on;: Disables the Nagle algorithm, reducing latency by sending small packets immediately.
• keepalive_timeout 65;: Sets the timeout for persistent connections.
• server_tokens off;: Hides Nginx version information, a minor security hardening step.
• gzip on; ... gzip_types ...: Enables and configures Gzip compression for text-based assets.
• ssl_protocols, ssl_prefer_server_ciphers, ssl_session_cache, ssl_session_timeout, ssl_ciphers: Essential for secure and performant HTTPS. The provided ciphers are a strong, modern set.

Caching Strategies

Leveraging Nginx’s built-in caching can significantly offload your backend application servers. This is particularly effective for static assets and API responses that don’t change frequently.

Browser Caching

Instructing browsers to cache assets reduces the number of requests hitting your server.

# In your server block or http block

location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|webp|woff|woff2|ttf|eot)$ {
    expires 365d;
    add_header Cache-Control "public, immutable";
    access_log off;
}

Nginx FastCGI/Proxy Cache

For dynamic content that can be cached, Nginx’s proxy cache is powerful. This requires defining a cache zone and then using proxy_cache directives.

# In your http block

proxy_cache_path /var/cache/nginx/my_cache levels=1:2 keys_zone=my_cache:10m inactive=60m max_size=1g;

# In your server block or location block

location / {
    proxy_pass http://backend_servers; # Or http://127.0.0.1:8000 for Gunicorn/FPM
    proxy_cache my_cache;
    proxy_cache_valid 200 302 10m; # Cache successful responses for 10 minutes
    proxy_cache_valid 404 1m;      # Cache 404s for 1 minute
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    add_header X-Cache-Status $upstream_cache_status;
    # ... other proxy settings
}

Note: Ensure the cache directory (e.g., /var/cache/nginx/my_cache) exists and is writable by the Nginx user (e.g., www-data).

Gunicorn/FPM Optimization for Backend Performance

The choice between Gunicorn (Python) and PHP-FPM (PHP) depends on your application’s stack. Both require careful tuning to handle concurrent requests efficiently and avoid becoming a bottleneck.

Gunicorn Tuning (Python Applications)

Gunicorn is a Python WSGI HTTP Server. Its performance is heavily influenced by the number of worker processes and the worker class used.

Worker Processes and Threads

A common starting point for the number of workers is (2 * number_of_cores) + 1. For I/O-bound applications, using threads can also be beneficial.

# Example Gunicorn command line or configuration file

# Using command line:
gunicorn --workers 3 --threads 2 --bind 0.0.0.0:8000 myapp.wsgi:application

# Using a Gunicorn configuration file (e.g., gunicorn_config.py):
# workers = 3
# threads = 2
# bind = "0.0.0.0:8000"
# worker_class = "gthread" # or "sync", "eventlet", "gevent"

Explanation:

• --workers 3: Sets the number of worker processes. For a 2-core CPU, (2*2)+1 = 5 might be a good starting point, but 3 is used here for illustration. Adjust based on load testing.
• --threads 2: Sets the number of threads per worker. This is only effective if worker_class supports threading (e.g., gthread, eventlet, gevent).
• --bind 0.0.0.0:8000: The address and port Gunicorn listens on. Nginx will proxy to this.
• worker_class = "gthread": Uses a thread-based worker. For CPU-bound tasks, the default sync worker class might be sufficient, but for I/O-bound tasks (like many web requests), threading or asynchronous workers (eventlet, gevent) are often better.

Worker Type Considerations

The choice of worker_class is crucial:

• sync: The default. Each worker handles one request at a time. Simple but can block under heavy I/O.
• gthread: Uses threads within each worker process. Good for I/O-bound tasks, but limited by Python’s Global Interpreter Lock (GIL) for CPU-bound tasks.
• eventlet / gevent: Asynchronous, non-blocking workers. Excellent for high concurrency and I/O-bound applications, as they can handle thousands of connections per worker. Requires installing the respective libraries (e.g., pip install eventlet).

PHP-FPM Tuning (PHP Applications)

PHP-FPM (FastCGI Process Manager) is the standard for running PHP applications with web servers like Nginx. Its configuration is managed in /etc/php/[version]/fpm/pool.d/www.conf (or a custom pool file).

Process Manager Settings

The key settings are related to how FPM manages its worker processes.

; /etc/php/8.1/fpm/pool.d/www.conf (example for PHP 8.1)

[www]
user = www-data
group = www-data
listen = /run/php/php8.1-fpm.sock ; Or a TCP socket like 127.0.0.1:9000

; Process Manager Settings
; pm = dynamic ; or static or ondemand
; pm.max_children = 50
; pm.start_servers = 5
; pm.min_spare_servers = 2
; pm.max_spare_servers = 10
; pm.process_idle_timeout = 10s
; pm.max_requests = 500

; For static process management (use with caution, requires careful calculation)
pm = static
pm.max_children = 20 ; Adjust based on server resources and expected load

; For dynamic process management (recommended for most cases)
; pm = dynamic
; pm.max_children = 100
; pm.start_servers = 10
; pm.min_spare_servers = 5
; pm.max_spare_servers = 20
; pm.max_requests = 1000 ; Restart workers after X requests to prevent memory leaks

Explanation:

• pm: Process Manager control.
  • dynamic: FPM starts a minimum number of processes and scales up to a maximum as needed.
  • static: FPM maintains a fixed number of processes. Simpler to tune but less flexible.
  • ondemand: Processes are spawned only when a request is received.
• pm.max_children: The maximum number of child processes that will be spawned. This is the most critical setting. Too high can exhaust RAM; too low can lead to request queuing.
• pm.start_servers: The number of processes started when FPM starts.
• pm.min_spare_servers: The minimum number of idle processes to maintain.
• pm.max_spare_servers: The maximum number of idle processes to maintain.
• pm.max_requests: The number of requests each child process should execute before respawning. Useful for mitigating memory leaks in long-running applications.

Tuning Strategy:

• Start with pm = dynamic.
• Calculate pm.max_children: A common formula is (Total RAM - RAM for OS/Nginx/Other Services) / Average RAM per PHP-FPM process. Monitor memory usage closely.
• Set pm.start_servers, pm.min_spare_servers, and pm.max_spare_servers to reasonable values (e.g., 10, 5, 20 for a moderately busy server).
• If using static, calculate pm.max_children based on your server’s resources and expected peak load.
• Monitor php-fpm.log and system resource usage (top, htop, free -m) to identify bottlenecks.

Redis Optimization for Caching and Session Management

Redis is an in-memory data structure store, often used as a cache, message broker, and session store. Optimizing Redis on OVH involves tuning its configuration and understanding its memory usage.

Redis Configuration Tuning

The primary configuration file is typically /etc/redis/redis.conf.

# /etc/redis/redis.conf

# General
daemonize yes
pidfile /var/run/redis/redis-server.pid
logfile /var/log/redis/redis-server.log
dir /var/lib/redis

# Network
bind 127.0.0.1 -::1 ; Bind to localhost for security if Nginx/App are on same server
# bind 0.0.0.0 ; If Redis needs to be accessible from other machines (use with firewall)
protected-mode yes ; Recommended, requires explicit bind or password
port 6379

# Memory Management
maxmemory 2gb ; Set a limit to prevent Redis from consuming all system RAM
maxmemory-policy allkeys-lru ; Eviction policy: LRU (Least Recently Used) is common

# Persistence (adjust based on needs - for caching, often disabled or minimal)
save 900 1   ; Save the DB if 1 key changed in 900 seconds
save 300 10  ; Save the DB if 10 keys changed in 300 seconds
save 60 10000 ; Save the DB if 10000 keys changed in 60 seconds
appendonly no ; Disable AOF if persistence is not critical for cache/session

# Performance
tcp-backlog 511 ; Increase if you have many concurrent connections
tcp-keepalive 300 ; Keep connections alive
databases 16

Explanation:

• maxmemory: Crucial for preventing Redis from consuming all available RAM. Set this to a value that leaves ample memory for the OS and other services.
• maxmemory-policy: Defines how Redis evicts keys when maxmemory is reached. allkeys-lru is a good default for caching.
• save directives: Control RDB snapshotting. For a pure cache or session store, these can often be disabled or set to very aggressive thresholds to minimize disk I/O.
• appendonly no: Disables Append-Only File persistence. If Redis is purely for caching and can be rebuilt, disabling AOF saves disk I/O and CPU. If used for critical data, consider enabling it.
• tcp-backlog: Can help with high connection rates.

Monitoring Redis Performance

Use the redis-cli tool to monitor Redis:

redis-cli
127.0.0.1:6379> INFO memory
127.0.0.1:6379> INFO stats
127.0.0.1:6379> INFO persistence
127.0.0.1:6379> CONFIG GET maxmemory
127.0.0.1:6379> CONFIG GET maxmemory-policy

Key metrics to watch:

• used_memory: Current memory usage.
• used_memory_peak: Peak memory usage.
• evicted_keys: Number of keys evicted due to maxmemory policy. High numbers indicate Redis is too small or the policy needs adjustment.
• keyspace_hits and keyspace_misses: For cache usage, a high hit ratio is desirable.
• instantaneous_ops_per_sec: Current operations per second.

Putting It All Together: OVH Deployment Considerations

On OVH, whether you’re using dedicated servers or VPS, network latency and resource allocation are key. Ensure your Nginx, Gunicorn/FPM, and Redis instances are configured to communicate efficiently. If they are on the same machine, using Unix domain sockets for Nginx to Gunicorn/FPM and Redis can offer a slight performance edge over TCP/IP loopback.

Example Nginx Configuration for Gunicorn/FPM

# In /etc/nginx/sites-available/your_shopify_app

server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;

    # Serve static files directly
    location /static/ {
        alias /path/to/your/app/static/;
        expires 365d;
        add_header Cache-Control "public, immutable";
    }

    location /media/ {
        alias /path/to/your/app/media/;
        expires 365d;
        add_header Cache-Control "public, immutable";
    }

    location / {
        # For Gunicorn (Python)
        proxy_pass http://unix:/path/to/your/app/gunicorn.sock; # Or http://127.0.0.1:8000;

        # For PHP-FPM
        # include snippets/fastcgi-php.conf;
        # fastcgi_pass unix:/run/php/php8.1-fpm.sock; # Or 127.0.0.1:9000;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Add caching headers if applicable
        add_header X-Cache-Status $upstream_cache_status;
    }

    # SSL Configuration (if using HTTPS)
    # listen 443 ssl http2;
    # ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    # ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
    # include /etc/letsencrypt/options-ssl-nginx.conf;
    # ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}

By meticulously tuning Nginx, your chosen backend application server (Gunicorn or PHP-FPM), and Redis, you can build a highly performant and scalable infrastructure on OVH for your Shopify-integrated applications. Continuous monitoring and iterative adjustments based on real-world traffic patterns are key to maintaining optimal performance.