The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and DynamoDB on Linode for Magento 2

Nginx Configuration for Magento 2 Performance

Optimizing Nginx is paramount for serving Magento 2 efficiently. We’ll focus on key directives that impact request handling, caching, and resource utilization. This assumes a standard Linode setup with a recent Nginx version.

Worker Processes and Connections

The worker_processes directive controls how many worker processes Nginx will spawn. Setting this to auto is generally recommended, allowing Nginx to determine the optimal number based on available CPU cores. worker_connections defines the maximum number of simultaneous connections that each worker process can handle. A common starting point is 1024, but this may need tuning based on your traffic patterns.

nginx.conf Snippet

worker_processes auto;
events {
    worker_connections 1024;
    multi_accept on;
}

Caching Strategies

Leveraging Nginx’s FastCGI cache for PHP-FPM output can significantly reduce backend processing. This is particularly effective for static or semi-static pages. We’ll configure a cache zone and then apply it within the Magento 2 location block.

Cache Zone Configuration

http {
    # ... other http directives ...

    proxy_cache_path /var/cache/nginx/magento levels=1:2 keys_zone=magento_cache:100m inactive=60m max_size=10g;
    proxy_temp_path /var/tmp/nginx/proxy_temp;

    # ... other http directives ...
}

Explanation:

• /var/cache/nginx/magento: The directory where cache files will be stored. Ensure this directory exists and Nginx has write permissions.
• levels=1:2: Defines the directory structure for the cache.
• keys_zone=magento_cache:100m: Creates a shared memory zone named magento_cache with a size of 100MB to store cache keys.
• inactive=60m: Cache entries that have not been accessed for 60 minutes will be removed.
• max_size=10g: The maximum size of the cache.
• proxy_temp_path: Specifies a temporary directory for proxy operations.

Magento 2 Site Configuration (within server block)

server {
    listen 80;
    server_name your_domain.com;
    root /var/www/html/magento2;
    index index.php index.html index.htm;

    # Enable FastCGI Caching
    set $skip_cache 0;

    # Cache bypass for logged-in users, AJAX requests, POST requests, etc.
    if ($http_cookie ~* "PHPSESSID|wordpress_logged_in_|woocommerce_items_in_cart") {
        set $skip_cache 1;
    }
    if ($request_method = POST) {
        set $skip_cache 1;
    }
    if ($http_x_requested_with = "XMLHttpRequest") {
        set $skip_cache 1;
    }

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 1y;
        add_header Cache-Control "public, no-transform";
        access_log off;
        log_not_found off;
    }

    location ~ \.php$ {
        include snippets/fastcgi_params.conf;
        fastcgi_pass unix:/var/run/php/php8.1-fpm.sock; # Adjust PHP version and socket path
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

        # Cache control directives
        fastcgi_cache_bypass $skip_cache;
        fastcgi_cache_valid 200 60m; # Cache successful responses for 60 minutes
        fastcgi_cache_key "$scheme$request_method$host$request_uri";
        add_header X-Cache-Status $upstream_cache_status;

        # Enable caching for specific responses
        fastcgi_cache magento_cache;
    }

    # Deny access to hidden files
    location ~ /\. {
        deny all;
    }
}

Key Nginx Directives for Magento 2:

• try_files $uri $uri/ /index.php?$args;: This is crucial for Magento’s routing. It attempts to serve the requested file, then a directory, and finally falls back to index.php with query arguments.
• location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$: This block serves static assets with long expiration times and aggressive caching headers.
• fastcgi_cache_bypass $skip_cache;: Uses the variable set earlier to bypass the cache for dynamic requests.
• fastcgi_cache_valid 200 60m;: Caches successful (HTTP 200) responses for 60 minutes. Adjust this based on your content’s volatility.
• fastcgi_cache_key "$scheme$request_method$host$request_uri";: Defines the unique key for each cached item.
• add_header X-Cache-Status $upstream_cache_status;: Adds a header to responses indicating whether the item was served from cache (HIT), bypassed (BYPASS), etc. This is invaluable for debugging.
• fastcgi_cache magento_cache;: Applies the previously defined cache zone to this location.

Gunicorn Tuning for Magento 2 (Python/WSGI)

When running Magento 2 with a Python-based WSGI server like Gunicorn, proper configuration is essential for handling concurrent requests and managing worker processes. This section assumes you are using Gunicorn to serve a Python WSGI application that interfaces with Magento 2 (e.g., via a custom adapter or a framework that wraps Magento).

Worker Processes and Threads

Gunicorn’s concurrency model is based on worker processes and, optionally, threads within those processes. The optimal configuration depends heavily on your application’s I/O-bound vs. CPU-bound nature and the underlying hardware.

Gunicorn Command Line Arguments

gunicorn --workers 4 --threads 2 --bind 0.0.0.0:8000 --timeout 120 --log-level info your_module.wsgi:application

Explanation:

• --workers 4: The number of worker processes. A common starting point is (2 * number_of_cpu_cores) + 1. For a Linode with 2 vCPUs, 4 workers is a reasonable starting point.
• --threads 2: The number of threads per worker process. If your Magento 2 integration involves significant I/O operations (database calls, external API requests), threads can improve concurrency without the overhead of full processes. If your application is CPU-bound, threads might not offer much benefit and could even introduce contention.
• --bind 0.0.0.0:8000: The address and port Gunicorn will listen on. This is typically proxied by Nginx.
• --timeout 120: The maximum time in seconds a worker can spend on a request before being killed. Magento 2 operations can sometimes be lengthy, so a higher timeout (e.g., 120 seconds) is often necessary.
• --log-level info: Sets the logging level.
• your_module.wsgi:application: The Python path to your WSGI application object.

Worker Types

Gunicorn supports several worker types:

• Sync Workers (default): Each worker handles one request at a time. Simple but can be a bottleneck for I/O-bound applications.
• Async Workers (e.g., gevent, eventlet): These workers can handle multiple requests concurrently using cooperative multitasking. If your Magento 2 integration is heavily I/O-bound, using async workers with appropriate threading can yield significant performance gains.
• Gevent Workers: Requires the gevent library to be installed.

Example with Gevent Workers

pip install gevent
gunicorn --worker-class gevent --workers 4 --threads 10 --bind 0.0.0.0:8000 --timeout 120 --log-level info your_module.wsgi:application

With gevent, you might use fewer worker processes but a higher number of threads (e.g., 10-50) to handle many concurrent I/O operations efficiently.

PHP-FPM Tuning for Magento 2

If you’re using PHP-FPM (which is the standard for PHP applications served by Nginx), tuning its process manager is critical. Magento 2 is PHP-intensive, and efficient process management prevents resource exhaustion and speeds up request processing.

Process Manager Settings

The primary configuration file for PHP-FPM is typically located at /etc/php/[version]/fpm/php.conf.d/www.conf (or similar). We’ll focus on the pm (process manager) settings.

www.conf Snippet

; Choose how the process manager (pm) should take on managing child processes.
; This can be either 'static', 'dynamic' or 'ondemand'.
pm = dynamic

; The number of child processes to be created when pm is set to 'static' or 'ondemand'.
; pm.max_children = 50 ; (Used with static or ondemand)

; The desired number of child processes to be available at any time.
; pm.min_spare_servers = 5 ; (Used with dynamic)

; The maximum number of child processes that can be started at any time.
; pm.max_spare_servers = 10 ; (Used with dynamic)

; The maximum number of requests each child process should execute before respawning.
; This can help prevent memory leaks.
pm.max_requests = 500

; The number of child processes to be spawned initially on startup.
; pm.start_servers = 2 ; (Used with dynamic)

; The maximum number of concurrent connections to the socket.
; Should be equal to the number of worker connections in Nginx.
; pm.max_children = 100 ; (If using pm = static)
; pm.max_children = 150 ; (If using pm = dynamic, this is the upper limit)

; Set to 'no' for production environments.
; pm.disable_notations = no

; Set to 'on' for production environments.
; request_terminate_timeout = 0 ; (e.g., 30s for 30 seconds)

; Adjust memory_limit and max_execution_time for Magento 2
memory_limit = 512M
max_execution_time = 180
upload_max_filesize = 64M
post_max_size = 64M

Tuning Strategy:

• pm = dynamic: This is often the best choice for variable workloads. PHP-FPM will manage the number of child processes between min_spare_servers and max_spare_servers, spawning new ones as needed up to max_children.
• pm.max_children: This is the most critical setting. It defines the absolute maximum number of PHP-FPM processes that can run concurrently. A common formula is (Total RAM - RAM used by OS/other services) / Average RAM per PHP process. Monitor your server’s RAM usage closely. If you see excessive swapping, max_children is too high. If requests are queued or timing out, it might be too low. For a Linode with 4GB RAM, starting with pm.max_children = 100 and monitoring is reasonable.
• pm.max_requests: Setting this to a value like 500 helps mitigate memory leaks in long-running PHP processes. Magento can be memory-intensive, so this is important.
• memory_limit, max_execution_time, upload_max_filesize, post_max_size: These PHP settings are crucial for Magento 2. Default values are often too low. 512MB for memory limit and 180 seconds for execution time are good starting points for production.

DynamoDB Performance Tuning for Magento 2

While not directly on Linode, if your Magento 2 instance relies on AWS DynamoDB for session storage, caching, or other data persistence, tuning its performance is vital. This involves understanding read/write capacity units (RCUs/WCUs) and indexing.

Provisioned Throughput (RCUs & WCUs)

DynamoDB’s performance is governed by provisioned throughput. You pay for the capacity you provision, whether you use it or not. Magento 2’s access patterns to DynamoDB (e.g., session reads/writes, cache get/set) will dictate the required RCUs and WCUs.

Monitoring and Auto Scaling

Use CloudWatch metrics to monitor ConsumedReadCapacityUnits and ConsumedWriteCapacityUnits. If these consistently approach your provisioned capacity, you’re likely to experience throttling (indicated by ThrottledRequests metric). Conversely, if consumption is very low, you’re overpaying.

AWS CLI Example for Auto Scaling

aws application-autoscaling put-scaling-policy --service-namespace dynamodb --resource-id "table/your_magento_session_table" --policy-name "RcuTargetPolicy" --policy-type TargetTrackingScaling --target-tracking-scaling-policy-configuration '{
    "TargetValue": 70.0,
    "PredefinedMetricSpecification": {
        "PredefinedMetricType": "DynamoDBReadCapacityUtilization"
    },
    "ScaleInCooldown": 300,
    "ScaleOutCooldown": 300
}'

aws application-autoscaling put-scaling-policy --service-namespace dynamodb --resource-id "table/your_magento_session_table" --policy-name "WcuTargetPolicy" --policy-type TargetTrackingScaling --target-tracking-scaling-policy-configuration '{
    "TargetValue": 70.0,
    "PredefinedMetricSpecification": {
        "PredefinedMetricType": "DynamoDBWriteCapacityUtilization"
    },
    "ScaleInCooldown": 300,
    "ScaleOutCooldown": 300
}'

This configures auto-scaling to maintain 70% read and write capacity utilization. Adjust the TargetValue and cooldown periods based on your traffic patterns.

Indexing Strategies

For tables used by Magento 2 (e.g., session tables, cache tables), ensure you have appropriate Global Secondary Indexes (GSIs) or Local Secondary Indexes (LSIs) if your queries require filtering or sorting on attributes other than the primary key. Magento’s session handler for DynamoDB typically uses session_id as the primary key. If you need to query sessions by user ID or other attributes, a GSI is necessary.

Example GSI for Session Table

{
    "TableName": "your_magento_session_table",
    "AttributeDefinitions": [
        {"AttributeName": "session_id", "AttributeType": "S"},
        {"AttributeName": "user_id", "AttributeType": "S"}
    ],
    "GlobalSecondaryIndexUpdates": [
        {
            "Create": {
                "IndexName": "UserIdIndex",
                "KeySchema": [
                    {"AttributeName": "user_id", "KeyType": "HASH"}
                ],
                "Projection": {
                    "ProjectionType": "ALL"
                },
                "ProvisionedThroughput": {
                    "ReadCapacityUnits": 10,
                    "WriteCapacityUnits": 10
                }
            }
        }
    ]
}

Note: Provisioned throughput for GSIs/LSIs must be managed separately. Auto-scaling can be applied to GSIs as well.

On-Demand vs. Provisioned Capacity

For highly unpredictable workloads, DynamoDB On-Demand capacity mode can be simpler to manage, as you pay per request. However, for predictable, high-traffic Magento 2 instances, provisioned capacity with auto-scaling often offers better cost-efficiency. Evaluate your traffic patterns carefully.