Tuning Magento 2 Varnish 7 Cache and SSL Termination behind Nginx on Rocky Linux 9 for Sub-100ms P99 responses

Varnish 7 Configuration for Magento 2 Enterprise

Achieving sub-100ms P99 response times for a high-traffic Magento 2 Enterprise deployment hinges on a meticulously tuned Varnish cache layer. This section details critical Varnish 7 configuration parameters and VCL snippets essential for optimizing Magento 2 performance, specifically when terminating SSL at Nginx and serving content from Varnish. We’ll focus on Rocky Linux 9 environments.

Core Varnish Daemon Tuning

The primary Varnish daemon configuration resides in /etc/varnish/varnish.params. Adjusting these parameters directly impacts Varnish’s memory allocation, listening interfaces, and overall resource utilization. For a high-throughput Magento instance, consider the following:

Memory Allocation (-s)

The -s parameter defines the storage backend and size. For large catalogs and frequent cache hits, a memory-based storage (malloc) is generally preferred for its speed. The size should be generous, often a significant portion of available RAM, leaving enough for the OS, Nginx, and the Magento application itself. A common starting point for a dedicated cache server might be 80% of available RAM.

VARNISH_STORAGE_SIZE="100G" # Example: 100 Gigabytes

The storage backend type is also crucial. malloc is fast but volatile. For persistence across restarts (though less common in pure caching tiers), persistent or file could be considered, but they introduce I/O overhead.

VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"

Listening Interface and Port (-a)

Varnish typically listens on port 80 for HTTP traffic. Since Nginx will handle SSL termination and forward plain HTTP to Varnish, this remains the standard. Ensure it’s bound to the correct interface if you have multiple network interfaces.

VARNISH_LISTEN_ADDRESS="0.0.0.0" # Listen on all interfaces
VARNISH_LISTEN_PORT="80"

Backend Connection (-b)

This parameter specifies the default backend if no specific VCL rule matches. In our Nginx-terminated setup, Varnish will forward requests to Nginx’s internal HTTP port (e.g., 8080) for dynamic content or cache misses. However, the primary backend definition is usually handled within the VCL itself for more granular control.

Varnish Configuration Language (VCL) for Magento 2

The VCL file, typically located at /etc/varnish/default.vcl, is the heart of Varnish’s caching logic. For Magento 2 Enterprise, a robust VCL must intelligently cache static assets, handle dynamic pages, and respect Magento’s cache-control headers. We’ll assume Nginx is listening on port 8080 for backend requests from Varnish.

Backend Definition

Define your backend server(s). If you have multiple Magento instances or load-balanced Nginx frontends, you can define them here. For simplicity, we’ll define a single backend pointing to Nginx.

backend default {
    .host = "127.0.0.1"; # Or the IP of your Nginx instance
    .port = "8080";     # Nginx listening port for backend traffic
    .connect_timeout = 5s;
    .first_byte_timeout = 60s;
    .between_bytes_timeout = 60s;
}

Cacheability Rules (vcl_recv)

The vcl_recv subroutine is executed when Varnish receives a request. This is where we decide whether to serve from cache, pass to the backend, or discard the request. Magento’s cacheability is complex, involving cookies, session IDs, and specific URL patterns.

sub vcl_recv {
    # Normalize request method
    if (req.method != "GET" && req.method != "HEAD") {
        return (pass);
    }

    # Remove cookies that prevent caching for anonymous users
    if (req.http.Cookie) {
        set req.http.Cookie = regsuball(req.http.Cookie, "(^|; )(_ga|_gid|_fbp|_fpc|_utmz|_utmc|_utmb|_utmk|_ga_.*|_gac_.*|PHPSESSID|mage-cache-sessid|mage-cache-user|mage-cache-validation)=[^;]*", "");
        set req.http.Cookie = regsuball(req.http.Cookie, "^; +|; +$", "");
        if (req.http.Cookie ~ "^$") {
            unset req.http.Cookie;
        }
    }

    # Cache static assets aggressively
    if (req.url ~ "(?i)\.(css|js|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$") {
        return (deliver); # Let vcl_deliver handle cache headers
    }

    # Magento specific cacheable URLs (e.g., product pages, category pages)
    # This is a simplified example. A comprehensive list requires deep Magento knowledge.
    if (req.url ~ "^/(catalog/product/view/id|catalog/category/view/id|cms/index/index)") {
        # Allow caching for anonymous users
        if (!req.http.Cookie) {
            return (hash);
        }
    }

    # For all other requests, pass to backend
    return (pass);
}

Cache Hash Calculation (vcl_hash)

The vcl_hash subroutine determines the cache key. It’s crucial that the hash is unique for distinct content variations. For Magento, this often means including relevant query parameters but excluding session-specific ones.

sub vcl_hash {
    if (req.http.Cookie) {
        # Include cookies that affect content for anonymous users if they exist
        set req.hash += req.http.Cookie;
    }
    # Include URL and query string for hashing
    set req.hash += req.url;
    return (hash);
}

Handling Backend Responses (vcl_backend_response)

This subroutine processes the response from the backend. Here, we respect Magento’s Cache-Control and Expires headers for dynamic content and set aggressive caching for static assets.

sub vcl_backend_response {
    # Respect backend Cache-Control and Expires headers
    if (beresp.http.Cache-Control ~ "no-cache|no-store|private") {
        set beresp.uncacheable = true;
        return (deliver);
    }
    if (beresp.http.Expires || beresp.http.Cache-Control) {
        # If backend provides caching headers, use them
        set beresp.ttl = 1h; # Default TTL if headers are present but not explicit enough
        return (deliver);
    }

    # Aggressive caching for static assets identified in vcl_recv
    if (req.url ~ "(?i)\.(css|js|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$") {
        set beresp.ttl = 1d; # Cache static assets for 1 day
        set beresp.grace = 1h; # Allow stale content for 1 hour
        return (deliver);
    }

    # Default TTL for cacheable dynamic content if no headers are present
    set beresp.ttl = 5m; # Cache dynamic content for 5 minutes
    set beresp.grace = 1m; # Allow stale content for 1 minute

    return (deliver);
}

Delivering Cached Content (vcl_deliver)

The vcl_deliver subroutine is executed before Varnish sends a response to the client. This is where we can add Varnish-specific headers for debugging and ensure correct cache headers are sent.

sub vcl_deliver {
    # Add Varnish headers for debugging
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
    } else {
        set resp.http.X-Cache = "MISS";
    }
    set resp.http.X-Varnish-Cacheable = "true"; # Indicate this response was cacheable by Varnish

    # Ensure no-cache headers are removed if content is cacheable
    unset resp.http.Pragma;
    unset resp.http.Cache-Control; # We manage caching via TTL and grace

    # Set appropriate Cache-Control for static assets
    if (req.url ~ "(?i)\.(css|js|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$") {
        set resp.http.Cache-Control = "public, max-age=86400"; # 1 day
    } else {
        # For dynamic content, rely on TTL and grace period.
        # Avoid sending explicit Cache-Control: max-age to clients unless absolutely necessary.
        # Magento's ESI or specific page caching might set this.
    }

    return (deliver);
}

Nginx Configuration for SSL Termination and Backend Proxy

Nginx acts as the SSL terminator and the reverse proxy to Varnish. It listens on port 443 for HTTPS traffic, decrypts it, and forwards plain HTTP requests to Varnish on port 80. For cache misses or dynamic content, Nginx then acts as the backend, listening on port 8080 and forwarding requests to the Magento application (e.g., PHP-FPM).

Nginx Main Configuration (nginx.conf)

Ensure Nginx is configured to handle a high volume of connections and has sufficient worker processes.

user nginx;
worker_processes auto; # Or a specific number based on CPU cores
error_log /var/log/nginx/error.log warn;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Adjust based on system limits and expected load
    multi_accept on;
}

http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    server_tokens off; # Hide Nginx version

    # Gzip compression
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;

    # Buffers and timeouts for backend connections
    proxy_buffer_size 128k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
    proxy_connect_timeout 60s;
    proxy_send_timeout 60s;
    proxy_read_timeout 60s;

    # Include virtual host configurations
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Varnish Proxy Server Block (varnish.conf)

This server block listens on port 80 and proxies requests to Varnish. It should be included in your Nginx configuration (e.g., in /etc/nginx/conf.d/varnish.conf).

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    server_name your_domain.com www.your_domain.com; # Replace with your domain

    # Redirect HTTP to HTTPS (if SSL is handled by a separate edge device or load balancer)
    # If Nginx handles SSL termination, this block might be different or absent.
    # For this setup, Nginx listens on 80 for Varnish, and 443 for clients.
    # This block is for the *backend* traffic from Nginx to Varnish.
    location / {
        proxy_pass http://127.0.0.1:80; # Forward to Varnish on port 80
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $host;
        proxy_http_version 1.1;
        proxy_set_header Connection ""; # Important for HTTP/1.1 keepalive
    }
}

SSL Termination Server Block (magento-ssl.conf)

This server block handles incoming HTTPS traffic, terminates SSL, and forwards requests to Varnish. Place this in /etc/nginx/sites-available/magento-ssl.conf and symlink to sites-enabled.

server {
    listen 443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;

    server_name your_domain.com www.your_domain.com; # Replace with your domain

    # SSL Configuration
    ssl_certificate /etc/letsencrypt/live/your_domain.com/fullchain.pem; # Path to your SSL certificate
    ssl_certificate_key /etc/letsencrypt/live/your_domain.com/privkey.pem; # Path to your SSL private key
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    ssl_session_tickets off;
    ssl_stapling on;
    ssl_stapling_verify on;
    resolver 8.8.8.8 8.8.4.4 valid=300s; # Use your preferred DNS resolvers
    resolver_timeout 5s;

    # HSTS (HTTP Strict Transport Security)
    add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;

    # Magento specific configurations
    root /var/www/html/public; # Path to your Magento public directory
    index index.php index.html index.htm;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    # Proxy to Varnish for cacheable content
    location / {
        proxy_pass http://127.0.0.1:80; # Forward to Varnish on port 80
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $host;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_connect_timeout 5s; # Shorter timeout for Varnish connection
        proxy_read_timeout 60s;
    }

    # Pass non-cacheable requests or cache misses to Magento backend (PHP-FPM)
    location ~ ^/index\.php(/|$) {
        # This block is reached if Varnish passes the request through.
        # Nginx then acts as the backend proxy to PHP-FPM.
        # Ensure Varnish is configured to pass requests that should hit PHP-FPM.
        # The 'proxy_pass' above to Varnish on port 80 is the primary path.
        # This block is more for direct Nginx-to-PHP-FPM if Varnish is bypassed.
        # In a Varnish-first setup, this might be less critical if Varnish handles all passes.

        # However, if Varnish passes to Nginx on port 80, and Nginx needs to decide
        # whether to serve from its own static files or pass to PHP-FPM, this is relevant.
        # For this setup, we assume Varnish passes to Nginx on port 80, and Nginx
        # then forwards to PHP-FPM on port 9000.

        # If Varnish passes to Nginx on port 80, and Nginx needs to proxy to PHP-FPM:
        # This requires a separate upstream for PHP-FPM.
        # upstream php-fpm {
        #     server unix:/run/php-fpm/www.sock; # Or TCP socket
        # }
        # proxy_pass http://php-fpm;

        # For simplicity in this example, we'll assume Varnish passes to Nginx on port 80,
        # and Nginx's *own* configuration (not shown here, but typically in Magento's
        # default Nginx conf) handles the proxying to PHP-FPM.
        # The key is that Nginx on port 80 is the *backend* for Varnish.
        # The Nginx on port 443 is the *frontend* for clients.

        # If Nginx on port 443 needs to proxy to PHP-FPM directly (e.g., for admin panel
        # or specific non-cached routes that bypass Varnish entirely):
        # This is a common pattern.
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_pass unix:/run/php-fpm/www.sock; # Adjust to your PHP-FPM socket
        fastcgi_read_timeout 300; # Longer timeout for PHP execution
    }

    # Deny access to hidden files
    location ~ /\. {
        deny all;
    }

    # Static files served directly by Nginx (if not cached by Varnish)
    location ~* \.(jpg|jpeg|gif|png|ico|css|js|svg|woff|woff2|ttf|eot)$ {
        expires 30d;
        add_header Cache-Control "public";
        access_log off;
    }
}

Magento Backend Proxy Server Block (magento-backend.conf)

This server block is crucial. It’s the backend that Varnish connects to. It listens on port 8080 and proxies requests to PHP-FPM. This configuration should be in /etc/nginx/conf.d/magento-backend.conf.

server {
    listen 8080;
    server_name localhost; # Or the specific IP Varnish connects to

    root /var/www/html/public; # Path to your Magento public directory
    index index.php index.html index.htm;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ ^/index\.php(/|$) {
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_pass unix:/run/php-fpm/www.sock; # Adjust to your PHP-FPM socket
        fastcgi_read_timeout 300; # Longer timeout for PHP execution
    }

    # Deny access to hidden files
    location ~ /\. {
        deny all;
    }

    # Serve static files directly if Varnish passes them through and Nginx needs to serve them
    # This is less common if Varnish is configured to cache static assets itself.
    # However, if Varnish passes static assets to Nginx on port 8080, Nginx can serve them.
    location ~* \.(jpg|jpeg|gif|png|ico|css|js|svg|woff|woff2|ttf|eot)$ {
        expires 30d;
        add_header Cache-Control "public";
        access_log off;
    }
}

System Tuning and Verification

Beyond Varnish and Nginx configurations, several system-level tunings are vital for achieving sub-100ms P99 responses.

File Descriptors

Ensure your system can handle a high number of open files. Both Varnish and Nginx are I/O intensive.

# Check current limits
ulimit -n

# Increase limits in /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
root soft nofile 65536
root hard nofile 65536

# Increase limits for systemd services (Varnish, Nginx, PHP-FPM)
# Example for Varnish service file (/etc/systemd/system/varnish.service.d/override.conf)
[Service]
LimitNOFILE=65536

# Example for Nginx service file (/etc/systemd/system/nginx.service.d/override.conf)
[Service]
LimitNOFILE=65536

# Example for PHP-FPM service file (/etc/systemd/system/php-fpm.service.d/override.conf)
[Service]
LimitNOFILE=65536

# Reload systemd and restart services after changes
sudo systemctl daemon-reload
sudo systemctl restart varnish nginx php-fpm

TCP Tuning (sysctl.conf)

Optimize TCP settings for high concurrency.

# Add to /etc/sysctl.conf
net.core.somaxconn = 4096
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_syncookies = 1

# Apply changes
sudo sysctl -p

Monitoring and Profiling

Continuous monitoring is essential. Use tools like:

• varnishstat and varnishlog for Varnish insights.
• nginx -s reload and Nginx access/error logs for Nginx behavior.
• htop, iotop, vmstat for system resource utilization.
• Application Performance Monitoring (APM) tools (e.g., New Relic, Datadog) to pinpoint Magento-level bottlenecks.
• Load testing tools (e.g., k6, JMeter) to simulate traffic and measure P99 latency under load.

Pay close attention to cache hit rates in varnishstat. A high hit rate (e.g., > 90%) is indicative of effective caching. Monitor backend response times from Varnish’s perspective (backend_fail, backend_retry) and Nginx’s proxy read times.

Deployment and Testing Workflow

1. Backup: Always back up existing Varnish and Nginx configurations.

2. Varnish Configuration Test: Use varnishd -C -f /etc/varnish/default.vcl to check VCL syntax.

3. Nginx Configuration Test: Use nginx -t to validate Nginx configuration files.

4. Restart Services: Restart Varnish and Nginx sequentially. Check logs for errors.

5. Incremental Load Testing: Start with moderate load and gradually increase, monitoring P99 response times, cache hit rates, and server resource utilization. Adjust Varnish TTLs, grace periods, and Nginx buffer sizes as needed.

6. Production Monitoring: Deploy to production and continue intensive monitoring. Be prepared to roll back if performance degrades.