Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Redis on AWS for Magento 2

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Redis on AWS for Magento 2

Nginx Configuration for Magento 2 on AWS

Optimizing Nginx is paramount for serving Magento 2 efficiently, especially under load. We’ll focus on key directives that impact performance and security in an AWS environment.

Worker Processes and Connections

The worker_processes directive dictates how many worker processes Nginx will spawn. A common recommendation is to set this to the number of CPU cores available on your EC2 instance. For worker_connections, this defines the maximum number of simultaneous connections that each worker process can handle. A good starting point is 1024, but this may need tuning based on your traffic patterns.

Caching Strategies

Leveraging Nginx’s built-in caching can significantly reduce backend load. We’ll configure proxy_cache for static assets and potentially dynamic content.

Static Asset Caching

This setup caches static files (images, CSS, JS) for a defined duration. Ensure your proxy_cache_path is configured on an EBS volume or S3 bucket for persistence and scalability.

Nginx Configuration Snippet
# Define cache path and parameters
proxy_cache_path /var/cache/nginx/magento levels=1:2 keys_zone=magento_cache:100m inactive=60m max_size=10g;
proxy_temp_path /var/tmp/nginx/proxy_temp;

server {
    listen 80;
    server_name your_domain.com;

    # ... other server configurations ...

    location ~* \.(jpg|jpeg|png|gif|ico|css|js|svg|woff|woff2|ttf|eot)$ {
        expires 30d;
        add_header Cache-Control "public, no-transform";
        proxy_pass http://your_backend_upstream; # Assuming a backend upstream is defined
        proxy_cache magento_cache;
        proxy_cache_valid 200 302 10m; # Cache successful responses for 10 minutes
        proxy_cache_valid 404 1m;      # Cache 404s for 1 minute to avoid repeated lookups
        proxy_cache_key "$scheme$request_method$host$request_uri";
        add_header X-Cache-Status $upstream_cache_status;
    }

    # ... other location blocks ...
}

Dynamic Content Caching (Use with Caution)

Caching dynamic Magento pages requires careful consideration of cache invalidation. This example shows a basic setup, but a robust solution often involves integrating with Magento’s ESI (Edge Side Includes) or a dedicated Varnish cache layer.

Nginx Configuration Snippet
# Inside your Magento server block
location / {
    proxy_pass http://your_backend_upstream;
    proxy_cache magento_cache;
    proxy_cache_valid 200 302 5m; # Cache dynamic content for 5 minutes
    proxy_cache_valid 404 1m;
    proxy_cache_key "$scheme$request_method$host$request_uri";
    add_header X-Cache-Status $upstream_cache_status;

    # Bypass cache for logged-in users or specific requests
    proxy_cache_bypass $http_cookie;
    proxy_no_cache $http_cookie;

    # ESI support (if configured in Magento)
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
}

Gzip Compression

Enabling Gzip compression significantly reduces the size of text-based assets (HTML, CSS, JS), leading to faster load times. Ensure you don’t double-compress already compressed content (like images).

Nginx Configuration Snippet
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;
gzip_disable "msie6"; # Disable for older IE versions that might have issues

HTTP/2 and TLS/SSL Optimization

Utilize HTTP/2 for multiplexing and header compression. Proper TLS configuration is crucial for security and performance. Use ECDSA certificates for better performance on modern clients.

Nginx Configuration Snippet
server {
    listen 443 ssl http2;
    server_name your_domain.com;

    ssl_certificate /etc/letsencrypt/live/your_domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/your_domain.com/privkey.pem;

    # Modern TLS configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    ssl_session_tickets off; # Consider enabling if performance is critical and security implications understood

    # HSTS (HTTP Strict Transport Security)
    add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;

    # ... rest of your server configuration ...
}

Gunicorn Configuration for Magento 2 (PHP-FPM Backend)

While Gunicorn is typically associated with Python applications, in a Magento 2 context, it’s often used as a process manager for PHP-FPM. This section assumes you’re using Gunicorn to manage PHP-FPM worker processes.

Gunicorn Worker Types and Scaling

For PHP-FPM, the gevent worker type is generally not applicable. You’ll likely be using the default sync worker type, and scaling will be managed by PHP-FPM’s pool configuration. Gunicorn’s role here is primarily to ensure PHP-FPM is running and to manage its lifecycle.

Gunicorn Configuration File (gunicorn_magento.conf)

This configuration focuses on managing the PHP-FPM service. The actual PHP-FPM tuning happens within its own configuration files.

# gunicorn_magento.conf
import os

# Path to your PHP-FPM executable
php_fpm_executable = "/usr/sbin/php-fpm7.4" # Adjust version as needed

# PHP-FPM pool configuration file
php_fpm_pool_conf = "/etc/php/7.4/fpm/pool.d/www.conf" # Adjust path as needed

# Gunicorn settings
bind = "0.0.0.0:9000" # Gunicorn will listen on this port, Nginx will proxy to it
workers = 1 # Gunicorn itself only needs one worker to manage PHP-FPM
threads = 1 # Not directly used for PHP-FPM, but required by Gunicorn

# Logging
accesslog = "/var/log/gunicorn/access.log"
errorlog = "/var/log/gunicorn/error.log"
loglevel = "info"

# Preload application (not directly applicable to PHP-FPM, but good practice)
preload_app = True

# Post-fork hook to start PHP-FPM
def post_fork(server, worker):
    # Start PHP-FPM if it's not already running
    if not os.path.exists("/run/php/php7.4-fpm.sock"): # Check for socket existence
        os.system(f"{php_fpm_executable} --daemon --fpm-config {php_fpm_pool_conf}")
        print(f"Started PHP-FPM with: {php_fpm_executable} --daemon --fpm-config {php_fpm_pool_conf}")

# Post-exec hook to ensure PHP-FPM is running after worker restart
def post_exec(worker):
    # Check if PHP-FPM is running, if not, restart it
    # This is a basic check; a more robust solution might involve checking the socket or PID file
    if not os.path.exists("/run/php/php7.4-fpm.sock"):
        os.system(f"{php_fpm_executable} --daemon --fpm-config {php_fpm_pool_conf}")
        print(f"Restarted PHP-FPM with: {php_fpm_executable} --daemon --fpm-config {php_fpm_pool_conf}")

# Gunicorn command to run:
# gunicorn -c gunicorn_magento.conf your_app_module:app
# In this case, we're not running a Python app, but managing PHP-FPM.
# A common approach is to use Gunicorn as a systemd service.

PHP-FPM Configuration Tuning (www.conf)

This is where the real PHP performance tuning happens. The pm (process manager) setting is critical. dynamic is generally recommended for variable workloads, while ondemand can save resources but might introduce latency on initial requests.

PHP-FPM Pool Configuration Snippet (/etc/php/7.4/fpm/pool.d/www.conf)
; pm = dynamic ; or static or ondemand
pm = dynamic

; Maximum number of accepted connections per request
pm.max_children = 50 ; Adjust based on your EC2 instance's RAM and CPU

; Number of child processes to be created when pm = dynamic
pm.start_servers = 5 ; Start with a reasonable number of workers

; Minimum number of children when pm = dynamic
pm.min_spare_servers = 2

; Maximum number of children when pm = dynamic
pm.max_spare_servers = 10

; The number of requests each child process should execute before respawning.
; This can help prevent memory leaks. A good value is between 500 and 1000.
pm.max_requests = 500

; Process manager settings for 'ondemand'
; pm.process_idle_timeout = 10s ; Timeout for idle processes to be killed
; pm.max_children = 50 ; Still need to define max_children for ondemand

; Listen socket (adjust if not using Unix socket)
listen = /run/php/php7.4-fpm.sock ; Or listen = 127.0.0.1:9000

; Other important settings
; php_admin_value[memory_limit] = 256M
; php_admin_value[max_execution_time] = 180
; php_admin_value[upload_max_filesize] = 64M
; php_admin_value[post_max_size] = 64M

Systemd Service for Gunicorn/PHP-FPM

Managing Gunicorn with systemd ensures it starts on boot and restarts if it crashes. This systemd service will launch Gunicorn, which in turn manages PHP-FPM.

Systemd Service File (/etc/systemd/system/magento-php-fpm.service)
[Unit]
Description=Gunicorn instance to manage PHP-FPM for Magento
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/var/www/html/magento # Adjust to your Magento root directory
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
ExecStart=/usr/local/bin/gunicorn --config /path/to/your/gunicorn_magento.conf gunicorn_magento:app # Assuming gunicorn_magento.py is in the working directory and contains the app object
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

Note: The gunicorn_magento:app part assumes you have a Python file named gunicorn_magento.py in your working directory that defines an app object. For this specific use case of managing PHP-FPM, you might need a minimal Python script that simply imports and runs the Gunicorn configuration. A simpler approach might be to directly manage PHP-FPM with systemd and use Gunicorn for other Python-based services.

Redis Configuration for Magento 2 on AWS

Redis is crucial for Magento 2’s caching mechanisms (session, configuration, page cache, etc.). Optimizing Redis involves tuning its memory usage, persistence, and network settings.

Memory Management

maxmemory is the most critical directive. Set this to a value that leaves ample room for the OS and other processes. Magento’s cache can grow significantly, so allocate generously but avoid swapping.

Redis Configuration Snippet (redis.conf)
# Set a memory limit. Example: 4GB for an instance with 8GB RAM.
# Adjust based on your EC2 instance type and other services running.
maxmemory 4gb

# Choose a memory eviction policy.
# volatile-lru: Evicts keys with an expire set, least recently used. Good for Magento.
# allkeys-lru: Evicts any key, least recently used.
# noeviction: Returns an error when max memory is reached. Not recommended for cache.
maxmemory-policy volatile-lru

Persistence

For caching purposes, RDB persistence can be disabled or configured minimally to reduce I/O overhead. AOF persistence is generally not needed for Magento’s cache unless you require absolute durability for session data (which is often handled separately).

Redis Configuration Snippet (redis.conf)
# Disable RDB snapshots for cache-only Redis instances
save ""

# Disable AOF persistence if not required for durability
appendonly no

Network and Performance Tuning

Tuning TCP settings and Redis’s internal network buffer can improve throughput.

Redis Configuration Snippet (redis.conf)
# TCP Keepalive: Helps detect dead connections.
tcp-keepalive 300

# Increase client output buffer limits if you experience "CLIENT_OUTPUT_BUFFER_LIMIT" errors.
# Format:   
# Example: 256mb 64mb 60
# client-output-buffer-limit normal 0 0 0
# client-output-buffer-limit replica 256mb 64mb 60
# client-output-buffer-limit pubsub 32mb 8mb 60

# Consider using `tcp-backlog` if you have very high connection rates, but be cautious.
# tcp-backlog 511

AWS Specific Considerations

When deploying on AWS, leverage Elasticache for managed Redis, which simplifies operations and provides built-in high availability and scaling. If self-hosting Redis on EC2:

  • Security Groups: Ensure Redis is only accessible from your application servers (e.g., EC2 instances running Nginx/PHP-FPM). Restrict access to port 6379.
  • Instance Type: Choose an instance type with sufficient RAM and network bandwidth.
  • EBS/Instance Store: For persistence (if used), consider Provisioned IOPS (io1/gp3) EBS volumes for consistent performance.
  • Monitoring: Utilize CloudWatch metrics for Redis (CPU, Memory, Network) and set up alarms for critical thresholds.

Magento 2 Configuration

Finally, ensure Magento 2 is configured to use Redis correctly. This is done in app/etc/env.php.

Magento 2 env.php Snippet
<?php
return [
    'backend' => [
        'frontName' => 'admin_your_admin_path'
    ],
    'crypt' => [
        'key' => 'your_encryption_key'
    ],
    'db' => [
        'table_prefix' => '',
        'connection' => [
            'default' => [
                'host' => 'your_db_host',
                'username' => 'your_db_user',
                'password' => 'your_db_password',
                'dbname' => 'your_db_name',
                'model' => 'mysql4',
                'initStatements' => 'SET NAMES utf8',
                'engine' => 'innodb',
            ]
        ]
    ],
    'cache' => [
        'frontend' => [
            'default' => [
                'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                'options' => [
                    'server' => '127.0.0.1', // Or your Redis ElastiCache endpoint
                    'port' => 6379,
                    'database' => '0', // Default DB for Magento cache
                    'password' => '', // If Redis requires authentication
                    'compress_data' => '1', // Enable data compression
                    'compression_library' => 'gzip' // Or 'lzf', 'zlib'
                ]
            ],
            'page_cache' => [
                'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                'options' => [
                    'server' => '127.0.0.1', // Or your Redis ElastiCache endpoint
                    'port' => 6379,
                    'database' => '1', // Separate DB for page cache is recommended
                    'password' => '',
                    'compress_data' => '1',
                    'compression_library' => 'gzip'
                ]
            ]
        ]
    ],
    'session' => [
        'save' => 'redis',
        'redis' => [
            'host' => '127.0.0.1', // Or your Redis ElastiCache endpoint
            'port' => 6379,
            'password' => '',
            'timeout' => '2.5',
            'persistent_identifier' => '',
            'database' => '2', // Separate DB for sessions
            'compression_threshold' => '2048',
            'compression_library' => 'gzip',
            'log_level' => '7', // Adjust log level as needed
            'break_after_failover' => '5',
            'max_concurrency' => '6',
            'connect_retries' => '1',
            'read_timeout' => '10',
            'automatic_cleaning_factor' => '0',
            'compress_data' => '1'
        ]
    ]
];

By meticulously tuning Nginx, PHP-FPM (managed by Gunicorn), and Redis, you can achieve significant performance gains for your Magento 2 deployment on AWS. Remember to monitor your system closely after making changes and iterate based on observed metrics.