Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MongoDB on Google Cloud for Magento 2

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MongoDB on Google Cloud for Magento 2

Nginx Configuration for Magento 2 on Google Cloud

Optimizing Nginx is paramount for serving Magento 2 efficiently, especially on Google Cloud where network latency and resource availability are key factors. We’ll focus on tuning worker processes, caching mechanisms, and static file serving.

Nginx Worker Processes and Connections

The worker_processes directive controls how many worker processes Nginx spawns. A common recommendation is to set it to the number of CPU cores available. For Google Cloud Compute Engine instances, this can be dynamically determined or set to a fixed value based on your instance type. The worker_connections directive limits the number of simultaneous connections a worker can handle.

Edit your main Nginx configuration file (typically /etc/nginx/nginx.conf):

# Determine the number of CPU cores. For a 4-core instance, this would be 4.
# You can also use 'auto' if Nginx supports it on your version and OS.
worker_processes 4;

# Set the maximum number of open file descriptors per worker process.
# This should be set high enough to accommodate all expected connections.
# A common starting point is 1024, but it can be increased.
# Ensure your OS limits are also increased (e.g., via /etc/security/limits.conf).
worker_connections 4096;

# Enable the event-driven model for better scalability.
events {
    worker_connections 4096; # Redundant if set globally, but good practice
    multi_accept on;
    use epoll; # For Linux systems
}

# ... other Nginx configurations ...

After modifying nginx.conf, test the configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Nginx Caching Strategies

Leveraging Nginx’s built-in caching can significantly reduce load on your backend application servers (Gunicorn/FPM) and database. We’ll configure browser caching for static assets and potentially server-side caching for dynamic content (though Magento’s Varnish or Redis is often preferred for full-page caching).

Browser Caching for Static Assets

Set appropriate Cache-Control and Expires headers for static files like CSS, JS, images, and fonts. This is typically done within your Magento 2 site’s server block.

server {
    # ... other server configurations ...

    # Magento static files
    location ~ ^/static/.* {
        expires 1y;
        add_header Cache-Control "public, immutable";
        access_log off;
        log_not_found off;
    }

    # Magento media files
    location ~ ^/media/.* {
        expires 1y;
        add_header Cache-Control "public";
        access_log off;
        log_not_found off;
    }

    # Other static assets (e.g., favicon, robots.txt)
    location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 1M;
        add_header Cache-Control "public";
        access_log off;
        log_not_found off;
    }

    # ... other location blocks ...
}

Nginx FastCGI Cache (Optional for Dynamic Content)

While Magento’s full-page cache is usually handled by Varnish or Redis, Nginx’s FastCGI cache can be a useful layer for specific API endpoints or less dynamic sections if Varnish is not in use. This requires enabling the ngx_http_fastcgi_cache_module.

# In nginx.conf or a separate conf file loaded by nginx.conf
fastcgi_cache_path /var/cache/nginx/fastcgi levels=1:2 keys_zone=magento_cache:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
fastcgi_cache_valid 200 302 10m;
fastcgi_cache_valid 404 1m;
fastcgi_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
fastcgi_cache_revalidate on;
fastcgi_cache_min_uses 1;
fastcgi_cache_lock on;

# In your Magento server block
location ~ \.php$ {
    # ... other fastcgi_pass and fastcgi_param directives ...

    fastcgi_cache magento_cache;
    fastcgi_cache_bypass $http_cache_control;
    fastcgi_cache_valid 200 302 10m; # Override global settings if needed
    add_header X-FastCGI-Cache $upstream_cache_status;

    # ... rest of fastcgi configuration ...
}

Remember to create the cache directory and set appropriate permissions:

sudo mkdir -p /var/cache/nginx/fastcgi
sudo chown www-data:www-data /var/cache/nginx/fastcgi # Or your Nginx user

Optimizing Static File Serving

Nginx is highly efficient at serving static files directly. Ensure that your Magento 2 pub/static and pub/media directories are configured to be served by Nginx, bypassing PHP entirely for these resources.

server {
    # ... other server configurations ...

    root /var/www/html/magento2; # Adjust to your Magento root directory
    index index.php index.html index.htm;

    # Serve static files directly
    location /pub/static/ {
        alias /var/www/html/magento2/pub/static/;
        expires 1y;
        add_header Cache-Control "public, immutable";
        access_log off;
        log_not_found off;
        try_files $uri $uri/ /pub/static/index.php?$args; # Fallback for symlinks
    }

    location /pub/media/ {
        alias /var/www/html/magento2/pub/media/;
        expires 1y;
        add_header Cache-Control "public";
        access_log off;
        log_not_found off;
        try_files $uri $uri/ /pub/media/index.php?$args; # Fallback for symlinks
    }

    # Magento's front controller
    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ \.php$ {
        # ... fastcgi_pass and other PHP-related directives ...
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_read_timeout 300; # Increase timeout for long-running PHP scripts
        fastcgi_pass unix:/var/run/php/php8.1-fpm.sock; # Adjust to your PHP-FPM version and socket
    }

    # ... other configurations ...
}

The try_files directive is crucial. For static files, it attempts to serve the requested URI directly. If it’s a directory, it looks for an index file. The fallback to index.php is important for Magento’s symlink strategy for static content.

Gunicorn Configuration for Magento 2 (PHP-FPM Backend)

While Gunicorn is primarily a Python WSGI HTTP Server, it’s not directly used for PHP applications like Magento 2. Magento 2 typically runs with PHP-FPM. If you are using Gunicorn for a separate Python API or microservice that interacts with Magento, the tuning principles would apply to that specific application. For Magento 2, we focus on PHP-FPM.

PHP-FPM Tuning for Magento 2

PHP-FPM (FastCGI Process Manager) is the de facto standard for running PHP applications. Tuning its process management and resource allocation is critical for Magento 2’s performance.

Process Manager Settings

PHP-FPM offers several process management strategies: static, dynamic, and ondemand. For Magento 2, especially under heavy load, dynamic or static are generally preferred.

Edit your PHP-FPM pool configuration file (e.g., /etc/php/8.1/fpm/pool.d/www.conf):

; Choose a process management strategy. 'dynamic' is often a good balance.
; 'static' can offer more predictable performance but might waste resources.
; 'ondemand' is good for low-traffic sites but can cause latency on first request.
pm = dynamic

; For 'dynamic' pm:
; Minimum number of child processes.
pm.min_spare_servers = 3
; Maximum number of child processes. This is a critical tuning parameter.
; A common starting point is (number of CPU cores * 3) to (number of CPU cores * 5).
; Magento can be memory-intensive, so monitor memory usage.
pm.max_children = 100
; Number of max idle processes.
pm.max_spare_servers = 5

; For 'static' pm:
; pm.max_children = 50 ; Set to a fixed number based on your server's capacity.

; Process idle timeout.
pm.process_idle_timeout = 10s

; Maximum number of requests each child process should execute before respawning.
; This helps prevent memory leaks. A value between 500 and 1000 is common.
pm.max_requests = 500

; The user and group that will run the processes.
user = www-data
group = www-data

; The address on which to accept FastCGI requests.
; Use a Unix socket for better performance if Nginx and PHP-FPM are on the same server.
; listen = /var/run/php/php8.1-fpm.sock
; Or use a TCP socket if Nginx and PHP-FPM are on different servers.
; listen = 127.0.0.1:9000

; Set the listen permissions if using a Unix socket.
; listen.owner = www-data
; listen.group = www-data
; listen.mode = 0660

; Set environment variables if needed.
; env[MY_ENV_VAR] = 'value'

After changes, restart PHP-FPM:

sudo systemctl restart php8.1-fpm # Adjust version as needed

PHP Configuration Tuning

Beyond PHP-FPM, core PHP settings in php.ini significantly impact Magento 2 performance.

; Adjust memory limit based on Magento's requirements and server capacity.
; Magento can require a significant amount of memory, especially for complex operations.
memory_limit = 512M

; Maximum execution time for scripts. Magento cron jobs or complex page loads might need more.
max_execution_time = 300

; Maximum input variables. Magento uses many. Increase this substantially.
max_input_vars = 3000

; Maximum upload size.
upload_max_filesize = 64M
post_max_size = 64M

; Enable OPcache for significant performance gains.
; Ensure opcache.enable is set to 1.
opcache.enable=1
opcache.memory_consumption=128 ; Adjust based on your code base size
opcache.interned_strings_buffer=16
opcache.max_accelerated_files=10000
opcache.revalidate_freq=60 ; Check for file updates every 60 seconds
opcache.validate_timestamps=1 ; Set to 0 in production for maximum performance if you have a robust deployment process
opcache.save_comments=1
opcache.enable_file_override=0
opcache.fast_shutdown=1

Locate your php.ini file (e.g., /etc/php/8.1/fpm/php.ini) and restart PHP-FPM after modifications.

MongoDB Tuning for Magento 2

Magento 2 uses MongoDB for session storage, caching, and catalog data (if configured). Optimizing MongoDB involves memory management, indexing, and connection pooling.

WiredTiger Storage Engine Configuration

The WiredTiger storage engine is the default and recommended engine. Its performance is heavily influenced by the storage.wiredTiger.engineConfig.cacheSizeGB setting.

Edit your MongoDB configuration file (typically /etc/mongod.conf):

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
  wiredTiger:
    engineConfig:
      # Allocate a significant portion of RAM to WiredTiger's cache.
      # A common recommendation is 50% of available RAM for dedicated MongoDB servers,
      # or less if running other services on the same instance.
      # For a 16GB RAM instance, this might be 8GB.
      cacheSizeGB: 8
    collectionConfig:
      # Compression can save disk space and I/O, but adds CPU overhead.
      # zlib is a good balance. snappy is faster but less compression.
      compression: zlib
    indexConfig:
      prefixCompression: true

# Network and security settings
net:
  port: 27017
  bindIp: 0.0.0.0 # Or specific IPs for security

# Logging settings
systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true
  verbosity: 0

# Process management
processManagement:
  fork: true
  pidFilePath: /var/run/mongodb/mongod.pid
  # Ensure the user and group are correct for your system
  # user: mongodb
  # group: mongodb

# Sharding settings (if applicable)
# sharding:
#   clusterRole: configsvr
#   configsvrFilePermissions:
#     groupReadable: true

Restart MongoDB after applying changes:

sudo systemctl restart mongod

MongoDB Indexing and Query Optimization

Proper indexing is crucial for fast query performance. Magento 2 relies on specific indexes for its operations. Regularly review and optimize these.

Connect to your MongoDB instance and check existing indexes:

mongo
use  # e.g., use magento_prod
db.getCollectionNames().forEach(function(collectionName) {
    print("Indexes for " + collectionName + ":");
    db[collectionName].getIndexes().forEach(function(index) {
        printjson(index);
    });
});

Magento’s default indexes are usually sufficient, but custom modules or specific usage patterns might require additional indexes. For example, if you frequently query the catalog_product_entity collection by SKU:

db.catalog_product_entity.createIndex( { sku: 1 }, { unique: true } )

Use MongoDB’s profiling tools to identify slow queries. Enable the profiler:

db.setProfilingLevel(1) // 0=off, 1=slow, 2=all
db.system.profile.find().pretty()

Analyze the output for queries exceeding your defined slow query threshold (default is 100ms) and add appropriate indexes.

Connection Pooling and MongoDB URI

Ensure your Magento 2 application is configured to use connection pooling. This is typically managed in app/etc/env.php.

<?php
return [
    'backend' => [
        'frontName' => 'admin_your_admin_path'
    ],
    'crypt' => [
        'key' => 'your_crypt_key'
    ],
    'db' => [
        'table_prefix' => ''
    ],
    'install' => [
        'date' => '2023-10-27 10:00:00'
    ],
    'modules' => [
        'Magento_Backend' => 1,
        // ... other modules
    ],
    'cache' => [
        'frontend' => [
            'default' => [
                'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                'options' => [
                    'report_buffer_size' => '10485760',
                    'compress_data' => '1',
                    'compression_lib' => '',
                    'max_concurrency' => '6',
                    'connect_retries' => '1',
                    'read_timeout' => '10',
                    'automatic_cleaning_factor' => '0',
                    'use_stale_backend' => '0',
                    'cache_id_prefix' => 'mage_'
                ]
            ],
            'page_cache' => [
                'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                'options' => [
                    'report_buffer_size' => '10485760',
                    'compress_data' => '1',
                    'compression_lib' => '',
                    'max_concurrency' => '6',
                    'connect_retries' => '1',
                    'read_timeout' => '10',
                    'automatic_cleaning_factor' => '0',
                    'use_stale_backend' => '0',
                    'cache_id_prefix' => 'mage_'
                ]
            ]
        ]
    ],
    'session' => [
        'save' => 'redis',
        'redis' => [
            'host' => '10.0.0.5', // Your Redis host for sessions
            'port' => 6379,
            'password' => '',
            'timeout' => '2.5',
            'persistent_identifier' => '',
            'database' => '0',
            'compression_threshold' => '2048',
            'compression_library' => '',
            'log_level' => '1',
            'max_concurrency' => '6',
            'break_after_frontend' => '5',
            'break_after_frontend_exception' => '0'
        ]
    ],
    'resource' => [
        'default_setup' => [
            'connection' => [
                'host' => '10.0.0.6', // Your MySQL host
                'dbname' => 'magento_prod',
                'username' => 'magento_user',
                'password' => 'your_db_password',
                'model' => 'mysql4',
                'initStatements' => 'SET NAMES utf8',
                'engine' => 'innodb',
                'active' => '1'
            ]
        ],
        'default' => [
            'connection' => [
                'host' => '10.0.0.6', // Your MySQL host
                'dbname' => 'magento_prod',
                'username' => 'magento_user',
                'password' => 'your_db_password',
                'model' => 'mysql4',
                'initStatements' => 'SET NAMES utf8',
                'engine' => 'innodb',
                'active' => '1'
            ]
        ]
    ],
    'Mage_Core_Model_File_Storage_Storage' => [
        'storage_config' => [
            'media_storage' => [
                'type' => 'local', // or 'cloud' for cloud storage
                'path' => 'media'
            ]
        ]
    ],
    'system' => [
        'default' => [
            'dev' => [
                'css' => [
                    'merge_css_files' => '1'
                ],
                'js' => [
                    'merge_files' => '1',
                    'enable_js_bundling' => '1'
                ]
            ]
        ]
    ],
    'http_client' => [
        'timeout' => '30',
        'user_agent' => 'Magento'
    ],
    'mongo' => [
        'connection' => [
            'host' => '10.0.0.7', // Your MongoDB host
            'port' => '27017',
            'username' => 'magento_user',
            'password' => 'your_mongo_password',
            'dbname' => 'magento_prod',
            'options' => [
                'replicaSet' => 'rs0', // If using replica set
                'authSource' => 'admin', // If auth is not on the dbName
                'connectTimeoutMS' => '5000',
                'socketTimeoutMS' => '5000',
                'w' => 'majority', // For write concern
                'readPreference' => 'primaryPreferred' // Or other read preferences
            ]
        ]
    ]
];

The options array in the mongo configuration is critical. Ensure connectTimeoutMS and socketTimeoutMS are set appropriately to prevent connection issues without introducing excessive delays. For replica sets, configure replicaSet and readPreference according to your availability and consistency needs.

Google Cloud Specific Considerations

When deploying on Google Cloud, consider the following:

  • Instance Sizing: Choose Compute Engine instances with sufficient vCPUs and RAM for your expected load. Monitor resource utilization (CPU, memory, network I/O) using Cloud Monitoring.
  • Persistent Disks: Use SSD Persistent Disks for MongoDB and your Magento application files for better I/O performance.
  • Network Latency: If your MongoDB instance is separate from your web servers, ensure they are in the same GCP region and preferably the same zone to minimize latency. Consider using Private Google Access for internal communication.
  • Load Balancing: Utilize Google Cloud Load Balancing to distribute traffic across multiple Nginx instances. Configure health checks to ensure traffic is only sent to healthy servers.
  • Managed Services: For MongoDB, consider using Google Cloud’s Memorystore for Redis (for caching/sessions) or exploring managed MongoDB solutions if self-management becomes too burdensome.
  • Firewall Rules: Configure GCP firewall rules to restrict access to your MongoDB port (27017) only from your application servers.