The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and PostgreSQL on AWS for WooCommerce
Nginx as a High-Performance Frontend for WooCommerce
When deploying WooCommerce on AWS, Nginx serves as an indispensable frontend. Its event-driven architecture excels at handling a high volume of concurrent connections, making it ideal for serving static assets and proxying dynamic requests to your application servers (Gunicorn for Python/Django or PHP-FPM for PHP/WordPress). Proper tuning of Nginx is crucial for minimizing latency and maximizing throughput.
Nginx Worker Processes and Connections
The primary directives for tuning Nginx’s concurrency are worker_processes and worker_connections. Setting worker_processes to auto allows Nginx to dynamically adjust the number of worker processes based on the number of CPU cores available on your EC2 instance. This is generally the most efficient setting for modern multi-core systems.
worker_connections defines the maximum number of simultaneous connections that each worker process can handle. The theoretical maximum is limited by the operating system’s file descriptor limit. A good starting point is to set this to a value that is significantly higher than your expected peak concurrent users, considering that each connection consumes a file descriptor. For a typical t3.medium instance (2 vCPUs), a value of 2048 or 4096 is often appropriate. You’ll need to ensure your OS’s file descriptor limit is also increased.
Tuning OS File Descriptors
Before increasing worker_connections, you must adjust the operating system’s limits. On Amazon Linux 2 or Ubuntu, this is typically done by editing /etc/security/limits.conf.
Amazon Linux 2
Add the following lines to /etc/security/limits.conf:
* soft nofile 65536 * hard nofile 65536 root soft nofile 65536 root hard nofile 65536
You’ll also need to configure the systemd service for Nginx to inherit these limits. Create or edit a file like /etc/systemd/system/nginx.service.d/override.conf:
[Service] LimitNOFILE=65536
After making these changes, reload the systemd daemon and restart Nginx:
sudo systemctl daemon-reload sudo systemctl restart nginx
Ubuntu
Similar to Amazon Linux 2, edit /etc/security/limits.conf:
* soft nofile 65536 * hard nofile 65536 root soft nofile 65536 root hard nofile 65536
For systemd services, edit /etc/systemd/system/nginx.service.d/override.conf:
[Service] LimitNOFILE=65536
Reload systemd and restart Nginx:
sudo systemctl daemon-reload sudo systemctl restart nginx
Nginx Keepalive Connections
Enabling keepalive connections significantly reduces the overhead of establishing new TCP connections for each request. This is particularly beneficial for HTTP/1.1. The keepalive_timeout directive controls how long an idle keepalive connection will remain open. A value between 60 and 120 seconds is a common starting point. keepalive_requests limits the number of requests that can be made over a single keepalive connection; 1000 is a reasonable default.
Gzip Compression
Compressing responses with Gzip can dramatically reduce bandwidth usage and improve page load times. Ensure it’s enabled and configured appropriately.
http {
# ... other http settings ...
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;
gzip_min_length 1000;
gzip_disable "msie6"; # Disable for older IE versions if necessary
# ... other http settings ...
}
Caching Static Assets
Leveraging browser caching for static assets (CSS, JS, images) is critical. Configure Nginx to send appropriate cache-control headers.
server {
# ... other server settings ...
location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 365d;
add_header Cache-Control "public, immutable";
access_log off;
}
# ... other server settings ...
}
Proxying to Gunicorn (Python/Django)
When using Gunicorn to serve a Python-based WooCommerce backend (e.g., a custom Django application), Nginx acts as a reverse proxy. The key is to configure the proxy_pass directive and associated settings for optimal performance.
Gunicorn Worker Configuration
Gunicorn’s worker count and type are crucial. For CPU-bound applications, the sync worker type is common, but for I/O-bound tasks (like many web applications), gevent or event workers can offer better concurrency. A common heuristic for worker count is (2 * number_of_cores) + 1. However, this should be tuned based on your application’s specific behavior and the EC2 instance type.
# Example Gunicorn command gunicorn --workers 4 --worker-class gevent --bind 0.0.0.0:8000 myapp.wsgi:application
Nginx Proxy Settings
Configure Nginx to efficiently proxy requests to Gunicorn. Set appropriate timeouts and buffer sizes. Ensure proxy_http_version 1.1 is used for keepalive connections.
location / {
proxy_pass http://your_gunicorn_backend_ip:8000; # Or your Gunicorn service name if using ECS/Kubernetes
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 75s;
proxy_send_timeout 75s;
proxy_read_timeout 75s;
proxy_buffer_size 16k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_http_version 1.1;
proxy_set_header Connection ""; # Important for HTTP/1.1 keepalives
}
Proxying to PHP-FPM (WordPress/WooCommerce)
For traditional WordPress/WooCommerce deployments, Nginx communicates with PHP-FPM. Tuning PHP-FPM is as critical as tuning Nginx itself.
PHP-FPM Pool Configuration
The pm (process manager) settings in your PHP-FPM pool configuration (e.g., /etc/php/8.1/fpm/pool.d/www.conf) are paramount. For busy WooCommerce sites, a dynamic process manager is usually preferred.
pm = dynamic: Allows FPM to dynamically manage the number of child processes.pm.max_children: The maximum number of child processes that will be spawned. This should be set based on your server’s RAM. A common starting point is(Total RAM - RAM for OS/Nginx) / Average child process size.pm.start_servers: The number of child processes to start when FPM starts.pm.min_spare_servers: The minimum number of idle supervisor processes.pm.max_spare_servers: The maximum number of idle supervisor processes.pm.process_idle_timeout: The number of seconds after which an idle process will be killed.pm.max_requests: The number of requests each child process should execute before respawning. This helps prevent memory leaks. A value between 500 and 1000 is typical.
; Example PHP-FPM pool configuration [www] user = www-data group = www-data listen = /run/php/php8.1-fpm.sock ; Or a TCP socket like 127.0.0.1:9000 pm = dynamic pm.max_children = 100 pm.start_servers = 10 pm.min_spare_servers = 5 pm.max_spare_servers = 20 pm.process_idle_timeout = 10s pm.max_requests = 500 ; Other settings request_terminate_timeout = 120s catch_workers_output = yes
After modifying PHP-FPM configuration, restart the service:
sudo systemctl restart php8.1-fpm
Nginx FastCGI Settings
Configure Nginx to communicate efficiently with PHP-FPM. Use a Unix socket if PHP-FPM is on the same server for lower latency, or a TCP socket if they are on different instances.
location ~ \.php$ {
include snippets/fastcgi-php.conf;
# If using Unix socket:
fastcgi_pass unix:/run/php/php8.1-fpm.sock;
# If using TCP socket:
# fastcgi_pass 127.0.0.1:9000;
fastcgi_read_timeout 300; # Increase timeout for potentially long PHP operations
fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;
}
PostgreSQL Tuning for WooCommerce
PostgreSQL is the backbone for WooCommerce’s data. Optimizing its configuration is critical for query performance, especially with large product catalogs, order histories, and user data.
Key PostgreSQL Configuration Parameters
The primary configuration file is postgresql.conf. The following parameters are most impactful for performance tuning on AWS RDS or self-managed instances.
Memory Allocation
shared_buffers: This is the most important parameter. It defines the amount of memory PostgreSQL uses for shared memory buffers. A common recommendation is 25% of the total system RAM. For a 32GB RDS instance, start with 8GB (8192MB).
work_mem: This is the maximum memory that can be used for internal sort operations and hash tables before disk-based temporary tables are used. For complex queries with joins and sorts, increasing this can prevent slow disk I/O. Start with 16MB and tune upwards based on query analysis. Be cautious, as this is allocated per sort operation within a query, so a high value with many concurrent complex queries can exhaust memory.
maintenance_work_mem: This is the maximum memory to be used for maintenance operations like VACUUM, CREATE INDEX, and ALTER TABLE ADD FOREIGN KEY. A larger value can significantly speed up these operations. 128MB or 256MB is a good starting point.
effective_cache_size: This parameter informs the query planner about how much memory is available for disk caching by the operating system and PostgreSQL’s shared buffers. It doesn’t allocate memory itself. Set it to roughly 50-75% of total RAM.
Checkpoint Tuning
Checkpoints are crucial for data durability but can cause I/O spikes. Tuning them balances durability with performance.
max_wal_size: The maximum amount of WAL (Write-Ahead Log) files that can accumulate before a checkpoint is forced. Increasing this reduces checkpoint frequency. Start with a value like 1GB or 2GB and monitor.
min_wal_size: The minimum amount of WAL files to keep. This ensures that even after checkpoints, there’s enough WAL data for recovery. A value like 512MB or 1GB is reasonable.
checkpoint_completion_target: This specifies the target duration for the checkpoint process. Setting it to 0.9 means the checkpoint will spread its I/O over 90% of the time between checkpoints. This smooths out I/O spikes.
Connection Pooling
max_connections: The maximum number of concurrent connections allowed. This should be set based on your application’s needs and server resources. For a busy WooCommerce site, 100-200 might be necessary, but each connection consumes memory. It’s often better to use external connection pooling (like PgBouncer) than to set this too high.
effective_io_concurrency: For systems with fast storage (like AWS EBS Provisioned IOPS or NVMe SSDs), this parameter helps the planner by estimating how many concurrent I/O operations can be issued to the storage system. Set this to the number of parallel disk I/O threads your storage can handle. For EBS Provisioned IOPS, this can be set to 100 or more.
Example `postgresql.conf` Snippet (for a 32GB RDS instance)
# Memory shared_buffers = 8GB work_mem = 32MB # Tune based on query analysis maintenance_work_mem = 256MB effective_cache_size = 24GB # ~75% of 32GB RAM # Checkpoints max_wal_size = 2GB min_wal_size = 1GB checkpoint_completion_target = 0.9 # Connections max_connections = 150 # Consider PgBouncer for higher loads # I/O effective_io_concurrency = 100 # For Provisioned IOPS EBS # Logging (useful for debugging) log_destination = 'stderr' logging_collector = on log_directory = 'pg_log' log_filename = 'postgresql-%Y-%m-%d_%H-%M-%S.log' log_statement = 'ddl' # Log DDL statements, or 'all' for debugging log_min_duration_statement = 1000 # Log statements longer than 1s
After modifying postgresql.conf, you must restart the PostgreSQL service or RDS instance for changes to take effect.
Monitoring and Iteration
Tuning is an iterative process. Utilize AWS CloudWatch metrics for EC2 (CPU Utilization, Network In/Out), RDS (CPU Utilization, Database Connections, Read/Write IOPS, Freeable Memory), and Nginx (request counts, error rates). For PostgreSQL, use pg_stat_activity, pg_stat_statements (requires extension), and slow query logs to identify bottlenecks. Regularly review these metrics and adjust configurations accordingly.