Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Elasticsearch on Linode for PHP

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and Elasticsearch on Linode for PHP

Nginx as a High-Performance Frontend Proxy

For a PHP application, Nginx serves as the ideal frontend. Its asynchronous, event-driven architecture excels at handling a high volume of concurrent connections, offloading static file serving, and acting as a robust reverse proxy to your application server (Gunicorn for Python/Flask/Django, or PHP-FPM for PHP). We’ll focus on tuning Nginx for optimal performance in this role.

Nginx Configuration Tuning

The core of Nginx performance tuning lies within its nginx.conf file, typically located at /etc/nginx/nginx.conf or within /etc/nginx/conf.d/. We’ll adjust global worker processes and connection limits.

Worker Processes and Connections

The worker_processes directive should ideally be set to the number of CPU cores available on your Linode instance. This allows Nginx to utilize all available processing power for handling requests. The worker_connections directive defines the maximum number of simultaneous connections that each worker process can open. A common starting point is 1024, but this can be increased significantly based on your server’s RAM and expected load.

Example nginx.conf Snippet

# /etc/nginx/nginx.conf

user www-data;
worker_processes auto; # Or set to the number of CPU cores, e.g., 4
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Increased from default 1024
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    server_tokens off; # Hide Nginx version for security

    # Gzip Compression
    gzip on;
    gzip_disable "msie6";
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # Loadbalancer configuration (if using multiple app servers)
    # upstream php-app {
    #     server 127.0.0.1:9000; # Example for PHP-FPM
    #     # server 127.0.0.1:8000; # Example for Gunicorn
    # }

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Optimizing Static File Serving

Nginx is exceptionally fast at serving static assets (CSS, JS, images). Configure your virtual host to leverage browser caching and efficient file serving directives.

Example Virtual Host Configuration

# /etc/nginx/sites-available/your-app.conf

server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;
    root /var/www/your-app/public; # Adjust to your public directory
    index index.php index.html index.htm;

    # Serve static files directly
    location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 30d; # Cache for 30 days
        add_header Cache-Control "public";
        access_log off; # Don't log access for static files
        try_files $uri =404;
    }

    # PHP-FPM configuration
    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        # With php-fpm (unix sockets are faster)
        fastcgi_pass unix:/run/php/php8.1-fpm.sock; # Adjust PHP version
        # Or with TCP/IP
        # fastcgi_pass 127.0.0.1:9000;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    # Deny access to hidden files
    location ~ /\.ht {
        deny all;
    }

    # Proxy to Gunicorn (if using Python app)
    # location / {
    #     proxy_pass http://127.0.0.1:8000;
    #     proxy_set_header Host $host;
    #     proxy_set_header X-Real-IP $remote_addr;
    #     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    #     proxy_set_header X-Forwarded-Proto $scheme;
    # }
}

PHP-FPM / Gunicorn Tuning

Whether you’re using PHP-FPM for a traditional PHP application or Gunicorn as a WSGI server for Python frameworks like Flask or Django, tuning its process management is crucial for handling application-level requests efficiently.

PHP-FPM Configuration

The primary configuration file for PHP-FPM is typically /etc/php/[version]/fpm/pool.d/www.conf. The key directives to tune are related to process management.

Process Management Directives

  • pm: Process manager control. Options are static, dynamic, or ondemand. dynamic is generally recommended for most workloads.
  • pm.max_children: The maximum number of child processes that will be spawned. This is the most critical setting. Set it based on your server’s RAM and the memory footprint of your PHP application. A common formula is (Total RAM - RAM for OS/Nginx) / Average PHP Process Memory.
  • pm.start_servers: The number of child processes to start when PHP-FPM starts.
  • pm.min_spare_servers: The minimum number of idle (spare) processes.
  • pm.max_spare_servers: The maximum number of idle (spare) processes.
  • pm.process_idle_timeout: The number of seconds after which an idle process will be killed.
  • pm.max_requests: The number of requests each child process will serve before respawning. This helps prevent memory leaks.

Example www.conf Snippet (Dynamic PM)

; /etc/php/8.1/fpm/pool.d/www.conf (Adjust PHP version as needed)

[www]
user = www-data
group = www-data
listen = /run/php/php8.1-fpm.sock # Or listen = 127.0.0.1:9000
listen.owner = www-data
listen.group = www-data
listen.mode = 0660

pm = dynamic
pm.max_children = 100       ; Adjust based on RAM and app memory usage
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.process_idle_timeout = 10s
pm.max_requests = 500       ; Restart process after 500 requests

Gunicorn Configuration (Python WSGI)

Gunicorn is configured via command-line arguments or a configuration file. For production, using a configuration file is cleaner.

Example Gunicorn Configuration File

# gunicorn_config.py

import multiprocessing

bind = "127.0.0.1:8000"
workers = multiprocessing.cpu_count() * 2 + 1 # Recommended formula for Gunicorn workers
threads = 2 # Number of threads per worker
backlog = 2048 # Maximum number of pending connections
worker_connections = 1000 # Max concurrent connections per worker (if using gevent/eventlet)
worker_class = "sync" # Or "gevent", "eventlet" for async I/O
timeout = 30 # Request timeout in seconds
keepalive = 2 # Number of seconds to keep alive connections

# Logging
accesslog = "/var/log/gunicorn/access.log"
errorlog = "/var/log/gunicorn/error.log"
loglevel = "info"

# Daemonization
daemon = True
pidfile = "/var/run/gunicorn.pid"

To run Gunicorn with this configuration:

gunicorn --config gunicorn_config.py your_app.wsgi:application

Elasticsearch Performance Tuning

Elasticsearch, while not directly part of the web request path, is often a critical component for search and logging. Its performance directly impacts features that rely on it. Tuning involves JVM heap size, shard allocation, and indexing strategies.

JVM Heap Size

Elasticsearch runs on the Java Virtual Machine (JVM). The heap size is the most critical JVM setting. It should be set to no more than 50% of your system’s RAM, and never exceed 30-32GB due to compressed ordinary object pointers (compressed oops).

Setting JVM Heap Size

Edit the jvm.options file, typically located at /etc/elasticsearch/jvm.options.

# /etc/elasticsearch/jvm.options

# Xms represents the initial size of the heap, and Xmx represents the maximum size.
# Set both to the same value to avoid heap resizing.
# Example for a Linode with 16GB RAM:
-Xms8g
-Xmx8g

# Other JVM options...

Shard Allocation and Settings

The number and size of shards significantly impact Elasticsearch performance. Too many small shards can overwhelm the cluster, while too few large shards can lead to slow recovery and inefficient resource utilization.

Shard Allocation Awareness

If you have multiple Linode instances in a cluster, configure shard allocation awareness to ensure replicas are placed on different nodes.

# PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.awareness.attributes": "zone"
  }
}

Then, when creating nodes, assign them to a zone:

# PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}

Index Settings for Performance

Consider the following when creating or updating indices:

  • Number of Shards: Aim for shards between 10GB and 50GB. A common strategy is to set the number of primary shards based on your expected data volume and growth, and then let Elasticsearch manage replica shards.
  • Number of Replicas: Start with 1 replica for high availability. Increase if read performance is critical and you have sufficient nodes.
  • Refresh Interval: The index.refresh_interval setting controls how often new documents become visible to search. The default is 1s. For high-volume indexing, increasing this to 30s or even -1 (disabling refresh) during bulk imports can significantly improve indexing speed. Remember to re-enable it afterward.

Example Index Creation with Optimized Settings

PUT /my-logs-index
{
  "settings": {
    "index": {
      "number_of_shards": 3,
      "number_of_replicas": 1,
      "refresh_interval": "30s"
    }
  },
  "mappings": {
    "properties": {
      "@timestamp": { "type": "date" },
      "message": { "type": "text" },
      "level": { "type": "keyword" }
      // ... other fields
    }
  }
}

Monitoring and Diagnostics

Continuous monitoring is key to identifying bottlenecks. Use tools like htop, netstat, Nginx’s stub_status module, PHP-FPM’s status page, and Elasticsearch’s `_cat` APIs.

Nginx Stub Status

Enable the stub_status module in your Nginx configuration to get real-time connection metrics.

# In your http block or server block
location /nginx_status {
    stub_status;
    allow 127.0.0.1; # Restrict access
    deny all;
}

Access http://yourdomain.com/nginx_status to see output like:

Active connections: 1234
server accepts handled requests requests per second accept rate handle rate
 1674467 1674467 10000000 123.45 100.00 100.00

PHP-FPM Status Page

Enable the status page in your PHP-FPM pool configuration.

; /etc/php/8.1/fpm/pool.d/www.conf
; ...
pm.status_path = /php-fpm-status
; ...

Then, configure Nginx to proxy to it:

# In your server block
location ~ ^/php-fpm-status {
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_pass unix:/run/php/php8.1-fpm.sock; # Match your listen directive
    internal; # Or allow specific IPs
}

Elasticsearch `_cat` APIs

Use these APIs to inspect cluster health, node status, and index details.

# Cluster Health
curl -X GET "http://localhost:9200/_cat/health?v"

# Node Information
curl -X GET "http://localhost:9200/_cat/nodes?v"

# Index Information
curl -X GET "http://localhost:9200/_cat/indices?v"

# Shard Information
curl -X GET "http://localhost:9200/_cat/shards?v"

By systematically tuning these components and continuously monitoring their performance, you can build a highly scalable and resilient infrastructure on Linode for your PHP applications.