The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and PostgreSQL on Google Cloud for Magento 2
Nginx Configuration for Magento 2 on Google Cloud
Optimizing Nginx is crucial for serving Magento 2 efficiently, especially under load. We’ll focus on key directives that impact performance and security in a Google Cloud environment.
The primary configuration file is typically located at /etc/nginx/nginx.conf, with site-specific configurations often in /etc/nginx/sites-available/ and symlinked to /etc/nginx/sites-enabled/.
Worker Processes and Connections
These settings dictate how Nginx handles concurrent requests. For multi-core VMs on Google Cloud, setting worker_processes to the number of CPU cores is a good starting point. worker_connections defines the maximum number of simultaneous connections a worker process can handle. The total connections will be worker_processes * worker_connections.
Ensure your system’s file descriptor limit (ulimit -n) is set high enough to accommodate these connections. You can adjust this in /etc/security/limits.conf.
worker_processes auto; # Or set to the number of CPU cores worker_connections 4096; # Adjust based on expected load and ulimit
Buffering and Caching
Nginx’s buffering directives control how it handles request and response bodies. For Magento 2, especially with large file uploads or API requests, tuning these can prevent memory exhaustion and improve performance.
client_body_buffer_size: Sets the buffer size for client request bodies. A larger value might be needed for large uploads.
client_max_body_size: Defines the maximum allowed size of a client request body. Crucial for file uploads.
proxy_buffer_size and proxy_buffers: These are critical when Nginx acts as a reverse proxy to Gunicorn/PHP-FPM. They control the buffering of responses from the upstream server.
client_body_buffer_size 128k; client_max_body_size 100m; # Adjust as needed for file uploads proxy_buffer_size 128k; proxy_buffers 8 128k; proxy_busy_buffers_size 256k;
Keepalive Connections
keepalive_timeout: Specifies the time a persistent connection will remain open. Shorter timeouts can free up resources faster, while longer ones can reduce latency for repeated requests from the same client.
keepalive_requests: Limits the number of requests that can be made over a single keep-alive connection. This prevents a single client from monopolizing a connection.
keepalive_timeout 65; keepalive_requests 1000;
Gzip Compression
Enabling Gzip compression significantly reduces the size of transferred data, improving page load times. Ensure it’s configured to compress HTML, CSS, and JavaScript files.
gzip on; gzip_vary on; gzip_proxied any; gzip_comp_level 6; # Compression level (1-9) gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;
SSL/TLS Optimization
For secure connections, optimize SSL/TLS settings. Enable HTTP/2 for multiplexing and header compression. Use modern cipher suites and enable session caching.
# Inside your server block for SSL listen 443 ssl http2; ssl_protocols TLSv1.2 TLSv1.3; ssl_prefer_server_ciphers on; ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384; ssl_session_cache shared:SSL:10m; # 10MB cache size ssl_session_timeout 10m; ssl_session_tickets off; # Consider security implications if enabling
Magento 2 Specific Nginx Configuration
Magento 2 requires specific Nginx directives for static file serving, media handling, and routing. This is typically defined in the Magento 2 Nginx configuration file provided by Magento or your deployment tool.
# Example snippet for static files and media
location /static/ {
alias /var/www/html/magento2/pub/static/;
expires max;
access_log off;
add_header Cache-Control "public, immutable";
}
location /media/ {
alias /var/www/html/magento2/pub/media/;
expires 30d; # Cache media files for 30 days
access_log off;
add_header Cache-Control "public";
}
# Magento 2 PHP-FPM or Gunicorn configuration
location ~ ^/index.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php/php7.4-fpm.sock; # Or your Gunicorn socket
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
# Deny access to sensitive files
location ~ /\.ht {
deny all;
}
location ~ /\.svn {
deny all;
}
location ~ /\.git {
deny all;
}
location ~ /composer.json {
deny all;
}
location ~ /composer.lock {
deny all;
}
location ~ /README.md {
deny all;
}
location ~ /LICENSE.txt {
deny all;
}
location ~ /CHANGELOG.md {
deny all;
}
location ~ /UPGRADE.txt {
deny all;
}
location ~ /var/ {
deny all;
}
location ~ /app/etc/env.php {
deny all;
}
location ~ /app/etc/local.xml {
deny all;
}
location ~ /app/etc/config.php {
deny all;
}
location ~ /setup/ {
deny all;
}
location ~ /update/ {
deny all;
}
location ~ /bin/ {
deny all;
}
location ~ /vendor/ {
deny all;
}
location ~ /phpscripts/ {
deny all;
}
location ~ /cron.php {
deny all;
}
location ~ /install.php {
deny all;
}
location ~ /recovery.php {
deny all;
}
location ~ /robots.txt {
allow all;
}
location ~ \.(css|js|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
access_log off;
add_header Cache-Control "public";
}
Gunicorn Configuration for Magento 2 (Python/WSGI)
When running Magento 2 with a Python WSGI application server like Gunicorn, careful configuration is needed to handle the application’s demands. This is less common for Magento 2, which is primarily PHP-based, but can be relevant for custom integrations or specific microservices.
Worker Processes and Threads
Gunicorn’s worker model is critical. For CPU-bound tasks, a worker-per-core model is often best. For I/O-bound tasks, you might use more workers with threads.
--workers: The number of worker processes. A common starting point is (2 * CPU_CORES) + 1.
--threads: The number of threads per worker. If using threads, this allows a single worker process to handle multiple requests concurrently.
# Example Gunicorn command gunicorn --workers 4 --threads 2 --bind 0.0.0.0:8000 your_wsgi_app:app
For Magento 2, which is PHP-based, Gunicorn would typically be used to proxy requests to a PHP-FPM backend, rather than running Magento directly. In such a scenario, Gunicorn’s role is more about managing the Nginx-to-PHP-FPM connection or serving static assets.
Timeouts and Keepalive
--timeout: The number of seconds to wait for a worker to respond. Magento operations can be long-running, so this might need to be increased.
--keep-alive: Enables keep-alive connections. This is generally beneficial for performance.
# Example with increased timeout gunicorn --workers 4 --threads 2 --timeout 120 --keep-alive 2 --bind 0.0.0.0:8000 your_wsgi_app:app
Worker Class
The default worker class is sync. For I/O-bound applications, consider gevent or eventlet if your application is compatible and you’re using threads.
# Example with gevent worker class gunicorn --worker-class gevent --workers 4 --threads 2 --timeout 120 --keep-alive 2 --bind 0.0.0.0:8000 your_wsgi_app:app
PHP-FPM Configuration for Magento 2
PHP-FPM (FastCGI Process Manager) is the standard for running PHP applications like Magento 2. Tuning its pool configuration is critical for performance and stability.
Process Management (pm)
PHP-FPM offers three process management modes:
static: A fixed number of child processes are spawned at startup. Good for predictable loads.dynamic: Processes are spawned dynamically based on load, up to a defined maximum.ondemand: Processes are spawned only when a request is received. Can save resources but introduces latency.
For Magento 2, dynamic is often a good balance. You’ll need to tune pm.max_children, pm.start_servers, pm.min_spare_servers, and pm.max_spare_servers.
; Example PHP-FPM pool configuration (e.g., /etc/php/7.4/fpm/pool.d/www.conf) [www] user = www-data group = www-data listen = /var/run/php/php7.4-fpm.sock listen.owner = www-data listen.group = www-data listen.mode = 0660 pm = dynamic pm.max_children = 150 ; Adjust based on VM memory and CPU pm.start_servers = 10 ; Number of processes started on startup pm.min_spare_servers = 5 ; Minimum idle processes pm.max_spare_servers = 20 ; Maximum idle processes pm.max_requests = 500 ; Restart a child process after this many requests
pm.max_children is the most critical. It should be set based on the available RAM on your Google Cloud VM. Each PHP-FPM worker consumes memory. A common calculation is: (Total RAM - RAM for OS/Nginx/DB) / Average RAM per PHP-FPM worker.
pm.max_requests: Setting this to a reasonable number (e.g., 500-1000) helps prevent memory leaks from accumulating over time by recycling worker processes.
Request Execution Timeouts
Magento operations can be lengthy. Ensure PHP’s execution time limits are sufficient.
; In php.ini (e.g., /etc/php/7.4/fpm/php.ini) max_execution_time = 300 ; Maximum execution time of each script, in seconds max_input_time = 300 ; Maximum amount of time each script may spend parsing input data memory_limit = 512M ; Adjust based on Magento's requirements and VM memory
These settings are also configurable within the PHP-FPM pool configuration file (php_admin_value directives) if you need per-pool settings.
; In pool.d/www.conf php_admin_value[max_execution_time] = 300 php_admin_value[memory_limit] = 512M
OpCache Configuration
OpCache is essential for PHP performance. Ensure it’s enabled and properly configured.
; In php.ini opcache.enable=1 opcache.memory_consumption=128 ; MB opcache.interned_strings_buffer=16 opcache.max_accelerated_files=10000 opcache.revalidate_freq=60 ; Check for file updates every 60 seconds opcache.validate_timestamps=1 ; Set to 0 in production for max performance if you have a deployment process that clears cache opcache.save_comments=1 opcache.enable_cli=1
For production environments where you want maximum performance and have a controlled deployment process, setting opcache.validate_timestamps=0 can be beneficial. This disables checking for file modifications, relying solely on cache clearing during deployments.
PostgreSQL Tuning for Magento 2
PostgreSQL performance is critical for Magento 2’s database operations. Tuning involves adjusting memory parameters, connection settings, and query optimization.
Shared Buffers and WAL
shared_buffers: This is the most important parameter. It’s the amount of memory PostgreSQL uses for caching data. A common recommendation is 25% of system RAM, but this can vary. For large databases and high-traffic sites, you might increase this.
wal_buffers: Memory for Write-Ahead Log (WAL) data. A value of 16MB is often sufficient.
wal_writer_delay: How often the WAL writer flushes data to disk. Lowering this can improve write performance but increase I/O.
# In postgresql.conf (e.g., /etc/postgresql/12/main/postgresql.conf) shared_buffers = 2GB ; Adjust based on VM RAM (e.g., 25% of total RAM) wal_buffers = 16MB wal_writer_delay = 200ms ; Default is 200ms, can be reduced for higher write throughput
Work Memory and Maintenance Work Memory
work_mem: Memory used for internal sort operations and hash tables. Increasing this can significantly speed up complex queries with sorts and joins. Be cautious, as this memory is allocated per sort operation, per query.
maintenance_work_mem: Memory used for maintenance operations like VACUUM, CREATE INDEX, and ALTER TABLE. A larger value can speed up these operations.
# In postgresql.conf work_mem = 64MB ; Adjust based on query complexity and available RAM maintenance_work_mem = 512MB ; Can be set higher than work_mem
Connection Pooling
Magento 2 can open many database connections. Using a connection pooler like PgBouncer can significantly reduce overhead and improve performance by reusing connections.
Install PgBouncer and configure it to connect to your PostgreSQL instance. Then, configure your Magento application to connect to PgBouncer instead of directly to PostgreSQL.
# Example pgbouncer.ini [databases] pgbouncer_db = host=127.0.0.1 port=5432 dbname=magento_db [pgbouncer] listen_addr = 127.0.0.1 listen_port = 6432 auth_type = md5 auth_file = /etc/pgbouncer/userlist.txt pool_mode = session ; Or transaction for higher throughput but potential issues with some apps max_client_conn = 1000 default_pool_size = 20
In userlist.txt, you’d define users and their passwords for PgBouncer to authenticate against PostgreSQL.
# Example userlist.txt "magento_user" "md5" "hashed_password_for_magento_user"
Then, update Magento’s app/etc/env.php to point to PgBouncer:
return [
'db' => [
'connection' => [
'host' => '127.0.0.1',
'port' => '6432', // PgBouncer port
'dbname' => 'magento_db',
'username' => 'magento_user',
'password' => 'your_pgbouncer_password',
'model' => 'mysql4',
'initStatements' => 'SET NAMES utf8',
'options' => [
PDO::ATTR_PERSISTENT => true, // Important for connection pooling
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
],
],
'default_setup' => [
'connection' => [
'host' => '127.0.0.1',
'port' => '6432', // PgBouncer port
'dbname' => 'magento_db',
'username' => 'magento_user',
'password' => 'your_pgbouncer_password',
'model' => 'mysql4',
'initStatements' => 'SET NAMES utf8',
'options' => [
PDO::ATTR_PERSISTENT => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
],
],
],
],
// ... other config
];
Autovacuum Tuning
PostgreSQL’s autovacuum process reclaims space from dead tuples. Proper tuning is essential to prevent table bloat and maintain query performance.
# In postgresql.conf autovacuum = on log_autovacuum_min_duration = 0 ; Log all autovacuum actions for monitoring autovacuum_max_workers = 3 ; Number of concurrent autovacuum processes autovacuum_naptime = 15s ; How often to check for work autovacuum_vacuum_threshold = 50 ; Minimum number of rows to trigger a vacuum autovacuum_analyze_threshold = 50 ; Minimum number of rows to trigger an analyze
You might need to adjust autovacuum_vacuum_scale_factor and autovacuum_analyze_scale_factor for very large tables. For Magento, which has many tables, a balance is key.
Query Performance and Indexing
Regularly analyze slow queries using PostgreSQL’s pg_stat_statements extension and Magento’s built-in profiling tools. Ensure all necessary database indexes are in place. Magento’s EAV model can lead to complex queries, so proper indexing is paramount.
-- Enable pg_stat_statements
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
-- Add to postgresql.conf:
-- shared_preload_libraries = 'pg_stat_statements'
-- Then restart PostgreSQL.
-- Example query to find slow queries:
SELECT
query,
calls,
total_time,
mean_time,
rows
FROM pg_stat_statements
ORDER BY mean_time DESC
LIMIT 10;
Use tools like EXPLAIN ANALYZE to understand query execution plans and identify bottlenecks.
Google Cloud Specific Considerations
When deploying on Google Cloud, leverage its managed services and network capabilities.
Instance Sizing and Machine Types
Choose machine types that balance CPU and memory for your workload. For Magento, memory is often a bottleneck for PHP-FPM and PostgreSQL. Consider memory-optimized or general-purpose machine types.
Persistent Disks
Use SSD Persistent Disks for your application and database instances. This provides significantly better I/O performance compared to standard persistent disks.
Load Balancing
Utilize Google Cloud Load Balancing to distribute traffic across multiple Nginx instances. This enhances availability and scalability. Configure health checks to ensure traffic is only sent to healthy instances.
Cloud SQL for PostgreSQL
For managed PostgreSQL, Google Cloud SQL is an excellent option. It handles patching, backups, and replication, allowing you to focus on application-level tuning. Ensure you select an appropriate instance size and storage type (SSD).
When using Cloud SQL, many of the postgresql.conf parameters are managed by Google. You can tune some parameters via the Cloud Console or `gcloud` CLI, but direct access to the configuration file is not available. Focus on instance sizing, storage, and connection pooling from your application.
Monitoring and Logging
Leverage Google Cloud’s operations suite (formerly Stackdriver) for comprehensive monitoring and logging. Set up alerts for key metrics like CPU utilization, memory usage, disk I/O, and error rates for Nginx, PHP-FPM, and PostgreSQL.
# Example of setting up alerts for high CPU on a Compute Engine instance: # Using gcloud CLI or Cloud Console to create a metric-based alert policy # for 'compute.googleapis.com/instance/cpu/utilization' exceeding a threshold.