Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MySQL on OVH for C

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MySQL on OVH for C

Nginx Configuration for High Throughput

Optimizing Nginx for a high-traffic environment on OVH requires a deep dive into its worker processes, connection handling, and caching strategies. We’ll focus on tuning parameters that directly impact concurrency and resource utilization.

Worker Processes and Connections

The worker_processes directive dictates how many worker processes Nginx will spawn. Setting this to auto is generally recommended, allowing Nginx to detect the number of CPU cores and utilize them efficiently. The worker_connections directive limits the number of simultaneous connections a single worker process can handle. This value, combined with worker_processes, determines the maximum number of concurrent connections Nginx can manage. A common starting point is 1024, but this should be adjusted based on your server’s RAM and expected load.

Example Nginx Configuration Snippet

worker_processes auto;
events {
    worker_connections 4096; # Increased from default 1024
    multi_accept on;        # Accept multiple connections at once
    use epoll;              # Linux-specific, high-performance event notification mechanism
}

http {
    sendfile on;            # Efficiently transfer files from one file descriptor to another
    tcp_nopush on;          # Improves efficiency of sending data over TCP
    tcp_nodelay on;         # Disables the Nagle algorithm, reducing latency
    keepalive_timeout 65;   # Timeout for keep-alive connections
    keepalive_requests 1000; # Max requests per keep-alive connection

    # Gzip compression for static assets
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    # Caching for static assets
    location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
        expires 365d;
        add_header Cache-Control "public, no-transform";
    }

    # ... other http configurations ...
}

Tuning Gunicorn/PHP-FPM for Application Performance

The application server layer is critical. For Python applications, Gunicorn is a popular choice. For PHP, PHP-FPM is the standard. Tuning these involves managing worker processes, request handling, and memory limits.

Gunicorn Configuration (Python)

Gunicorn’s performance is heavily influenced by the number of worker processes and the worker class. For CPU-bound applications, a sync worker class is common, with the number of workers typically set to (2 * number_of_cores) + 1. For I/O-bound applications, consider using an asynchronous worker class like gevent or eventlet, and adjust the worker count accordingly.

Example Gunicorn Command Line

gunicorn --workers 4 --worker-class gevent --bind 0.0.0.0:8000 myapp.wsgi:application

In this example, we’re using 4 worker processes with the gevent worker class, suitable for I/O-bound tasks. The binding is set to all interfaces on port 8000.

PHP-FPM Configuration

PHP-FPM’s configuration resides in its php-fpm.conf and pool configuration files (e.g., www.conf). Key directives include pm (process manager), pm.max_children, pm.start_servers, pm.min_spare_servers, and pm.max_spare_servers. For dynamic process management, pm = dynamic is recommended. For static, pm = static offers more predictable performance but less flexibility.

Example PHP-FPM Pool Configuration (www.conf)

; Choose how the process manager will control the number of child processes.
; Possible values:
;   'static' - static child process number
;   'dynamic' - dynamic number of child processes or use pm.min_spare_servers and pm.max_spare_servers
;   'ondemand' - dynamic number of child processes or use pm.max_children, pm.start_time, pm.max_requests
pm = dynamic

; The number of child processes to be created when pm is set to 'static' or 'dynamic'.
pm.max_children = 100

; The desired maximum number of idle server processes.
pm.min_spare_servers = 10

; The desired maximum number of idle server processes.
pm.max_spare_servers = 20

; The number of requests each child process should execute before respawning.
; This can help prevent memory leaks.
pm.max_requests = 500

; The TCP socket or the UNIX socket to listen on.
; For example: '/var/run/php/php7.4-fpm.sock' or '127.0.0.1:9000'
listen = /run/php/php7.4-fpm.sock

; Set to 'no' for Apache mod_php compatibility.
; Default value is 'yes'.
access.log = /var/log/php/php7.4-fpm.access.log
slowlog = /var/log/php/php7.4-fpm.slow.log
request_slowlog_timeout = 10s

Adjusting pm.max_children is crucial. Too high, and you risk OOM errors; too low, and you’ll starve your application of resources. Monitor your server’s memory usage and adjust accordingly. The request_slowlog_timeout is invaluable for identifying slow PHP scripts.

MySQL Performance Tuning on OVH

Database performance is often the bottleneck. Tuning MySQL involves optimizing configuration parameters, indexing, and query optimization. OVH’s managed database services might have some limitations on direct configuration file access, but many parameters can still be adjusted via the OVH control panel or API.

Key MySQL Configuration Parameters

The my.cnf (or my.ini) file is where most MySQL tuning happens. For OVH, you might need to use their specific interface to modify these. Critical parameters include:

  • innodb_buffer_pool_size: The most important setting for InnoDB. It caches data and indexes. Aim for 70-80% of available RAM on a dedicated database server.
  • innodb_log_file_size: Larger log files can improve write performance but increase recovery time.
  • innodb_flush_log_at_trx_commit: Setting this to 2 (instead of the default 1) can significantly boost write performance at a slight risk of data loss in case of a crash (data loss is limited to the last second of transactions). For maximum durability, keep it at 1.
  • max_connections: The maximum number of simultaneous client connections.
  • query_cache_size: While deprecated in newer MySQL versions, if you’re on an older version, a small query cache can help for read-heavy workloads with identical queries.
  • tmp_table_size and max_heap_table_size: Affect the performance of temporary tables created during complex queries.

Example `my.cnf` Snippet (Illustrative)

[mysqld]
innodb_buffer_pool_size = 8G
innodb_log_file_size = 512M
innodb_flush_log_at_trx_commit = 2
max_connections = 500
query_cache_type = 1
query_cache_size = 64M
tmp_table_size = 128M
max_heap_table_size = 128M
long_query_time = 2
log_error = /var/log/mysql/error.log
slow_query_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log

The long_query_time and slow query log are essential for identifying inefficient queries that need indexing or rewriting.

Monitoring and Diagnostics

Effective tuning is an iterative process that relies heavily on monitoring. Key tools and metrics include:

  • Nginx: Access logs, error logs, ngx_http_stub_status_module (for active connections, requests per second), and system-level metrics (CPU, memory, network I/O).
  • Gunicorn/PHP-FPM: Application logs, error logs, process status, and system-level metrics. For Gunicorn, tools like gunicorn_stats can be useful. For PHP-FPM, the slow log is critical.
  • MySQL: Slow query log, SHOW GLOBAL STATUS;, SHOW ENGINE INNODB STATUS;, and tools like Percona Monitoring and Management (PMM) or Prometheus with mysqld_exporter.
  • System-wide: top, htop, vmstat, iostat, netstat, and specialized APM tools.

Example Slow Query Analysis

If your slow query log (e.g., /var/log/mysql/mysql-slow.log) shows a query like this:

# User@Host: webuser[webuser] @ localhost []  Id:    12
# Query_time: 5.678901  Lock_time: 0.000123 Rows_sent: 10  Rows_examined: 1500000
SET timestamp=1678886400;
SELECT * FROM orders WHERE order_date < '2023-01-01';

This indicates a full table scan on a large `orders` table. The solution is to add an index:

ALTER TABLE orders ADD INDEX idx_order_date (order_date);

Regularly review these logs and metrics to identify and address performance regressions. On OVH, leverage their monitoring tools and consider deploying your own agents for deeper insights.