The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MySQL on AWS for PHP
Nginx Tuning for High-Traffic PHP Applications
Optimizing Nginx as a reverse proxy and static file server is crucial for any high-performance PHP application. We’ll focus on key directives that directly impact throughput and latency, particularly when serving dynamic content proxied to Gunicorn or FPM.
Worker Processes and Connections
The worker_processes directive dictates how many worker processes Nginx will spawn. Setting this to auto is generally recommended on multi-core systems, allowing Nginx to dynamically adjust based on available CPU cores. The worker_connections directive limits the number of simultaneous connections a single worker process can handle. The total maximum connections will be worker_processes * worker_connections. Ensure this value is sufficiently high to avoid connection exhaustion.
Example Nginx Configuration Snippet
worker_processes auto;
events {
worker_connections 4096; # Adjust based on system limits and expected load
multi_accept on;
}
http {
# ... other http directives ...
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
server_tokens off; # Important for security
# ... other http directives ...
}
Buffering and Timeouts
Nginx buffering can significantly impact performance. While buffering can reduce the number of write operations to disk, excessively large buffers can consume considerable memory. For proxied requests, tuning proxy_buffer_size and proxy_buffers is essential. Setting proxy_buffering on; is standard. The proxy_connect_timeout, proxy_send_timeout, and proxy_read_timeout directives control how long Nginx waits for a response from the upstream server. These should be set to reasonable values to prevent hanging requests from consuming resources.
Example Proxy Configuration
location / {
proxy_pass http://your_app_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
Gzip Compression
Enabling Gzip compression for dynamic content can drastically reduce bandwidth usage and improve perceived load times. Configure it within the http block. Be mindful of CPU overhead; on very high-traffic sites, consider offloading compression to a CDN or using hardware acceleration if available. The gzip_types directive should include common MIME types for dynamic content.
Gzip Configuration
gzip on; gzip_vary on; gzip_proxied any; gzip_comp_level 6; # Compression level (1-9) gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml; gzip_min_length 1000; # Only compress responses larger than 1000 bytes
Gunicorn Tuning for Python WSGI Applications
When deploying Python web applications using Gunicorn, tuning its worker processes and threads is paramount for handling concurrent requests efficiently. Gunicorn’s architecture relies on a master process managing multiple worker processes, each capable of handling multiple requests concurrently through threads or asynchronous workers.
Worker Types and Counts
Gunicorn supports several worker types:
- Sync Workers (
sync): The default. Each worker process handles one request at a time. Performance is limited by the slowest request. - Asynchronous Workers (
gevent,eventlet): Use non-blocking I/O and greenlets/coroutines to handle many requests concurrently within a single process. Ideal for I/O-bound applications. - Threaded Workers (
gthread): Uses threads within a worker process. Can be simpler to implement than async but may be subject to Python’s Global Interpreter Lock (GIL) for CPU-bound tasks.
The number of worker processes (-w or --workers) is typically set to (2 * number_of_cpu_cores) + 1. For asynchronous or threaded workers, the number of workers is less critical than the number of concurrent requests each worker can handle (--threads for gthread, or implicitly handled by the async worker’s event loop).
Example Gunicorn Command Line
# For sync workers (CPU-bound or simple I/O) gunicorn -w 4 -k sync --bind 0.0.0.0:8000 myapp.wsgi:application # For gevent workers (I/O-bound) gunicorn -w 2 -k gevent --worker-connections 1000 --bind 0.0.0.0:8000 myapp.wsgi:application # For threaded workers (mixed workloads, careful with GIL) gunicorn -w 2 --threads 4 --bind 0.0.0.0:8000 myapp.wsgi:application
Timeouts and Keep-Alive
The --timeout setting determines how long Gunicorn will wait for a worker to process a request before timing out. This should be set slightly higher than your application’s expected longest request processing time. --keep-alive controls the number of requests a worker will process before restarting, which can help mitigate memory leaks.
Timeout Configuration
gunicorn --workers 4 --timeout 120 --keep-alive 1000 --bind 0.0.0.0:8000 myapp.wsgi:application
PHP-FPM Tuning for PHP Applications
PHP-FPM (FastCGI Process Manager) is the de facto standard for running PHP applications in production. Its performance hinges on the management of its worker processes, which handle incoming PHP requests. Tuning FPM involves configuring its process manager settings and resource allocation.
Process Manager Settings
PHP-FPM offers three primary process management strategies:
- Static: A fixed number of child processes are spawned when FPM starts and remain active. Offers predictable performance but can be inefficient if load fluctuates significantly.
- Dynamic: FPM starts a few children and spawns more as needed, up to a defined maximum. It also kills idle children to save resources. A good balance for varying loads.
- On-Demand: FPM only starts child processes when a request comes in and kills them after a period of inactivity. Lowest resource usage but can introduce latency for the first request after idle periods.
The most common and recommended setting for general-purpose applications is dynamic. Key parameters include pm.max_children (maximum number of simultaneous child processes), pm.start_servers (number of servers to start at boot), pm.min_spare_servers (minimum number of idle servers to maintain), and pm.max_spare_servers (maximum number of idle servers to maintain).
Example PHP-FPM Pool Configuration (www.conf)
; /etc/php/8.1/fpm/pool.d/www.conf [www] user = www-data group = www-data listen = /run/php/php8.1-fpm.sock listen.owner = www-data listen.group = www-data listen.mode = 0660 pm = dynamic pm.max_children = 100 ; Adjust based on server RAM and expected load pm.start_servers = 10 pm.min_spare_servers = 5 pm.max_spare_servers = 20 pm.process_idle_timeout = 10s ; For dynamic/ondemand, how long to keep idle processes ; For static process management: ; pm = static ; pm.max_children = 50 ; Fixed number of processes request_terminate_timeout = 120s ; Timeout for individual requests request_slowlog_timeout = 10s ; Log requests exceeding this time slowlog = /var/log/php-fpm/slow.log catch_workers_output = yes ; rlimit_files = 1024 ; rlimit_nofile = 65536 ;
Request Termination and Slow Log
request_terminate_timeout is crucial for preventing runaway scripts from consuming resources indefinitely. Set this slightly higher than your application’s longest expected script execution time. The request_slowlog_timeout and slowlog directives are invaluable for identifying performance bottlenecks within your PHP code. Regularly monitor the slow.log file.
MySQL Tuning on AWS RDS/EC2
Optimizing MySQL, whether running on EC2 or managed via AWS RDS, requires careful consideration of its configuration parameters (my.cnf or parameter groups in RDS) and instance sizing.
Key Configuration Parameters
Several parameters significantly impact MySQL performance:
innodb_buffer_pool_size: The most critical setting for InnoDB. It caches data and indexes. Aim to set this to 70-80% of your instance’s available RAM on dedicated database servers. For RDS, this is often managed via parameter groups.innodb_log_file_sizeandinnodb_log_buffer_size: Larger log files can improve write performance by reducing flushing frequency, but increase recovery time.innodb_log_buffer_sizecaches log writes before flushing.max_connections: The maximum number of simultaneous client connections. Set this based on your application’s needs and server capacity. Too high can lead to resource exhaustion.query_cache_size(Deprecated/Removed in MySQL 8.0): If using older MySQL versions, a small query cache can help for read-heavy workloads with identical queries. However, it has significant locking overhead and is often disabled in modern deployments.tmp_table_sizeandmax_heap_table_size: Control the maximum size of in-memory temporary tables. If complex queries create large temporary tables that exceed these limits, they spill to disk, severely degrading performance.
Example `my.cnf` Snippet (for EC2)
[mysqld] # General max_connections = 500 # table_open_cache = 2000 # thread_cache_size = 16 # InnoDB Specific innodb_buffer_pool_size = 4G # Example for a 6GB RAM instance innodb_log_file_size = 512M innodb_log_buffer_size = 16M innodb_flush_log_at_trx_commit = 1 # ACID compliance, set to 2 for better performance if slight risk is acceptable innodb_flush_method = O_DIRECT # Recommended for Linux # Temporary Tables tmp_table_size = 64M max_heap_table_size = 64M # Logging (adjust as needed) # log_error = /var/log/mysql/error.log # slow_query_log = 1 # slow_query_log_file = /var/log/mysql/mysql-slow.log # long_query_time = 2
AWS RDS Parameter Groups
For AWS RDS, you’ll manage these parameters via Custom Parameter Groups. Create a new parameter group based on the engine version you are using, modify the relevant parameters, and associate it with your RDS instance. Remember that changes often require an instance reboot to take effect.
Instance Sizing and IOPS
Choosing the right RDS instance type (e.g., db.m5.large, db.r5.xlarge) is critical. Memory-optimized instances (r series) are generally preferred for databases. For storage, consider Provisioned IOPS (io1/gp3) volumes if your workload is heavily I/O bound and requires consistent performance, especially for write-intensive applications.
Monitoring and Diagnostics
Utilize AWS CloudWatch metrics for RDS (CPU Utilization, Read/Write IOPS, Database Connections, Buffer Cache Hit Ratio) and general EC2 metrics. For MySQL on EC2, use tools like mysqltuner.pl, pt-mysql-summary, and analyze slow query logs. For PHP-FPM, monitor php-fpm.log and the slow log. For Gunicorn, check application logs and system metrics (CPU, Memory, Network).