The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MySQL on Google Cloud for PHP
Nginx Configuration for High-Traffic PHP Applications
Optimizing Nginx is crucial for serving PHP applications efficiently, especially under heavy load. The primary goals are to minimize latency, maximize throughput, and ensure graceful handling of concurrent connections. We’ll focus on key directives that directly impact performance and resource utilization.
Worker Processes and Connections
The worker_processes directive dictates how many worker processes Nginx will spawn. A common recommendation is to set this to the number of CPU cores available on the server. For I/O-bound workloads, setting it to ‘auto’ allows Nginx to determine the optimal number based on CPU cores.
worker_connections defines the maximum number of simultaneous connections that each worker process can handle. This value, combined with worker_processes, determines the total maximum connections Nginx can manage. It’s essential to set this high enough to avoid connection limits, but not so high that it exhausts system resources (like file descriptors).
Example Nginx Configuration Snippet
worker_processes auto; # Or set to the number of CPU cores
events {
worker_connections 4096; # Adjust based on system limits and expected load
multi_accept on;
}
http {
# ... other http configurations ...
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
server_tokens off; # Important for security, hides Nginx version
# Gzip compression for static assets and API responses
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
# Caching for static assets
location ~* \.(jpg|jpeg|png|gif|ico|css|js|svg|woff|woff2)$ {
expires 30d;
add_header Cache-Control "public, no-transform";
}
# ... server blocks ...
}
Explanation:
worker_processes auto;: Nginx automatically scales worker processes to match CPU cores.worker_connections 4096;: Each worker can handle up to 4096 connections. This is a common starting point; monitor system limits (ulimit -n) and adjust.multi_accept on;: Allows workers to accept as many new connections as possible at once.sendfile on;: Efficiently transfers files from disk to network socket without user-space buffering.tcp_nopush on;: Instructs Nginx to send header information and the first part of a file in one packet.tcp_nodelay on;: Disables the Nagle algorithm, reducing latency for small packets.keepalive_timeout 65;: Sets the timeout for persistent connections. Shorter timeouts can free up resources faster but might increase overhead for clients making multiple requests.server_tokens off;: Hides the Nginx version in HTTP headers, a minor security hardening measure.gzip on;: Enables Gzip compression.gzip_types ...;: Specifies MIME types to compress.location ~* \.(...)$: Configures aggressive caching for static assets.
Proxying to PHP-FPM or Gunicorn
When Nginx acts as a reverse proxy, its configuration for upstream communication is critical. For PHP, this typically means proxying to PHP-FPM. For Python applications, it’s often Gunicorn.
Nginx to PHP-FPM Configuration
location ~ \.php$ {
include snippets/fastcgi-php.conf;
# With php-fpm (or other unix sockets):
fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
# Or using TCP/IP:
# fastcgi_pass 127.0.0.1:9000;
# FastCGI buffer settings (tune these based on your application's needs)
fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;
fastcgi_read_timeout 300s; # Increase for long-running scripts
}
Explanation:
include snippets/fastcgi-php.conf;: Includes standard FastCGI parameters.fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;: Specifies the PHP-FPM socket. Ensure this path is correct for your PHP version and installation. Using Unix sockets is generally faster than TCP/IP for local communication.fastcgi_buffers 8 16k;: Sets the number and size of buffers for FastCGI responses.fastcgi_buffer_size 32k;: Sets the size of the FastCGI buffer used for the first part of the response.fastcgi_read_timeout 300s;: Crucial for applications that might have long-running scripts (e.g., report generation, complex calculations). Default is often too low (e.g., 60s).
Nginx to Gunicorn Configuration (Python)
location / {
proxy_pass http://unix:/path/to/your/app.sock; # Or http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 75s;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
}
Explanation:
proxy_pass http://unix:/path/to/your/app.sock;: Directs traffic to the Gunicorn application, either via a Unix socket or a TCP/IP address and port.proxy_set_header ...;: Passes essential client information to the backend application.proxy_connect_timeout 75s;: Timeout for establishing a connection with the upstream server.proxy_read_timeout 300s;: Timeout for reading a response from the upstream server.proxy_send_timeout 300s;: Timeout for sending a request to the upstream server.
Gunicorn Tuning for PHP Developers (or Python Apps)
While the title mentions PHP, many modern applications use a microservices architecture or a Python backend served by Gunicorn. Understanding Gunicorn’s tuning parameters is vital for performance. Gunicorn’s performance is heavily influenced by its worker class, number of workers, and timeout settings.
Worker Class and Number of Workers
Gunicorn supports several worker classes. The most common are:
sync: The default, synchronous worker. Each worker can handle one request at a time. Simple but can block under heavy load.gevent: Uses greenlets for concurrency. Can handle many requests concurrently within a single process, making it suitable for I/O-bound applications. Requiresgeventto be installed.eventlet: Similar togevent, uses greenlets for asynchronous I/O.
The number of workers is typically set based on the number of CPU cores. A common heuristic is (2 * number_of_cores) + 1. This formula aims to keep CPUs busy while accounting for I/O waits.
Gunicorn Command-Line Arguments / Configuration File
You can configure Gunicorn via command-line arguments or a Python configuration file (e.g., gunicorn_config.py).
Command-Line Example
gunicorn --workers 4 \
--worker-class gevent \
--bind unix:/path/to/your/app.sock \
--timeout 300 \
--graceful-timeout 300 \
your_module:your_app
Explanation:
--workers 4: Starts 4 worker processes. Adjust based on CPU cores.--worker-class gevent: Uses thegeventworker class for concurrency.--bind unix:/path/to/your/app.sock: Binds Gunicorn to a Unix socket.--timeout 300: Sets the worker timeout to 300 seconds.--graceful-timeout 300: Timeout for graceful worker shutdown.
Configuration File Example (gunicorn_config.py)
import multiprocessing bind = "unix:/path/to/your/app.sock" workers = multiprocessing.cpu_count() * 2 + 1 worker_class = "gevent" timeout = 300 graceful_timeout = 300 # accesslog = "/var/log/gunicorn/access.log" # errorlog = "/var/log/gunicorn/error.log" # loglevel = "info" # capture_output = True
Explanation: This file provides the same configuration as the command-line arguments but is often cleaner for complex setups.
Gunicorn Timeouts
The --timeout (or timeout) setting is critical. It defines how long Gunicorn will wait for a worker to process a request before killing it. This should be aligned with Nginx’s proxy_read_timeout and proxy_send_timeout. If your application has long-running tasks, these timeouts need to be sufficiently high. However, excessively high timeouts can mask performance issues or lead to resource exhaustion if workers hang indefinitely.
PHP-FPM Tuning for Performance
For PHP applications, PHP-FPM (FastCGI Process Manager) is the standard way to interface with web servers like Nginx. Tuning PHP-FPM involves managing its pool of worker processes, which handle PHP script execution.
Process Manager Settings
PHP-FPM offers three process management strategies:
static: A fixed number of child processes are spawned when the FPM master process starts. This offers the most predictable performance but can be inefficient if traffic fluctuates significantly.dynamic: The number of child processes varies between a specified minimum and maximum based on traffic. This is a good balance between performance and resource utilization.ondemand: Child processes are spawned only when a request arrives. This conserves resources but can introduce latency for the first request after a period of inactivity.
The most common and recommended setting for production is dynamic.
PHP-FPM Pool Configuration (www.conf)
PHP-FPM pool configurations are typically found in files like /etc/php/7.4/fpm/pool.d/www.conf (path may vary by OS and PHP version).
; Start a new pool named 'www'. [www] ; Unix user/group of processes user = www-data group = www-data ; The address on which to accept FastCGI requests. ; Valid syntax is 'ip:port', 'port', 'unix:/path/to/socket'. ; Default is '/run/php/php7.4-fpm.sock' listen = /run/php/php7.4-fpm.sock ; Choose how the process manager (pm) will control the number of child processes. ; Possible values: 'static', 'dynamic', 'ondemand'. pm = dynamic ; If pm is 'dynamic', these are the parameters that will be used: ; pm.max_children The maximum number of children that can be started. pm.max_children = 50 ; Adjust based on RAM and expected concurrent requests ; pm.start_servers The number of children that will be started at each pool init. pm.start_servers = 5 ; A reasonable starting point ; pm.min_spare_servers The minimum number of children that should be kept active. pm.min_spare_servers = 2 ; Keep a few warm ; pm.max_spare_servers The maximum number of children that should be kept active. pm.max_spare_servers = 10 ; Don't let too many idle ; pm.process_idle_timeout The number of seconds after which a child process ; will be killed after it becomes idle. pm.process_idle_timeout = 10s ; Aggressively clean up idle processes ; If pm is 'ondemand', these are the parameters that will be used: ; pm.max_children The maximum number of children that can be started. ; pm.start_time_limit The number of seconds after which a child process ; will be killed after it becomes idle. ; The maximum number of execution time allowed for a script. ; This directive allows you to set a maximum time limit in seconds for scripts to execute. ; This is a global setting for all scripts. ; Default value: 0 (no limit) ; For long-running scripts, increase this value. ; Consider using Nginx's proxy_read_timeout for the web server side. request_terminate_timeout = 300s ; Set to 'on' if you want to use CPU affinity. ; cpu_affinity = ; Set to 'on' if you want to use event-based trigger. ; event_based = no ; Set to 'on' if you want to use process group. ; process_group = no ; Set to 'on' if you want to use process rotation. ; process_rotation = no ; Set to 'on' if you want to use graceful shutdown. ; graceful_shutdown = yes ; Set to 'on' if you want to use graceful shutdown with timeout. ; graceful_shutdown_timeout = 0 ; The maximum number of POST Request body that can be sent to PHP. ; Default value: 8M ; post_max_size = 16M ; Define the maximum number of variables that may be supplied to an application. ; Default value: 1000 ; max_input_vars = 3000 ; Define the maximum number of GET, POST and COOKIE variables that may be accepted. ; Default value: 1000 ; max_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files that will be accepted. ; Default value: 2M ; upload_max_filesize = 10M ; Define the maximum size of data that can be sent to the script. ; Default value: 1024k ; limit_input_vars = 3000 ; Define the maximum size of POST data that will be accepted. ; Default value: 8M ; post_max_size = 16M ; Define the maximum size of uploaded files