Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MongoDB on OVH for Perl

The Ultimate DevOps Playbook: Tuning Nginx, Gunicorn/FPM, and MongoDB on OVH for Perl

OVH Infrastructure Tuning for Perl Applications: Nginx, Gunicorn/FPM, and MongoDB Deep Dive

This playbook focuses on optimizing a Perl-based web application stack deployed on OVH infrastructure. We’ll cover granular tuning of Nginx as a reverse proxy, Gunicorn (for WSGI-compatible Perl frameworks like Mojolicious) or PHP-FPM (for traditional CGI/FastCGI Perl applications), and MongoDB for data persistence. The goal is to achieve maximum throughput, minimal latency, and robust stability under load.

Nginx as a High-Performance Reverse Proxy

Nginx is the frontline of our web serving infrastructure. Its event-driven architecture makes it ideal for handling a large number of concurrent connections efficiently. We’ll configure it to proxy requests to our application servers (Gunicorn or PHP-FPM) and manage static assets.

Core Nginx Configuration (`nginx.conf`) Tuning

The main `nginx.conf` file is crucial. We’ll adjust worker processes, connection limits, and buffer sizes. For OVH’s typical VPS or dedicated server environments, setting `worker_processes` to the number of CPU cores is a good starting point. `worker_connections` should be set high enough to accommodate anticipated concurrent users, considering that each connection consumes resources.

`nginx.conf` Snippet

user www-data;
worker_processes auto; # Or set to the number of CPU cores
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096; # Adjust based on expected load and server memory
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # SSL configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m; # Adjust size as needed
    ssl_session_timeout 10m;
    ssl_session_tickets off;

    # Gzip compression for text-based assets
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    # Buffers and timeouts for upstream communication
    proxy_connect_timeout 60s;
    proxy_send_timeout 60s;
    proxy_read_timeout 60s;
    proxy_buffer_size 16k;
    proxy_buffers 4 32k;
    proxy_busy_buffers_size 64k;

    # Logging configuration
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log warn;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Application-Specific Server Block Configuration

For each Perl application, a dedicated server block is essential. This block defines how Nginx handles requests for that specific domain, including proxying to the application server and serving static files.

Example Nginx Server Block (for Gunicorn/Mojolicious)

server {
    listen 80;
    server_name your-perl-app.example.com;

    # Redirect HTTP to HTTPS (assuming you have SSL configured)
    # return 301 https://$host$request_uri;

    # Serve static assets directly
    location /static/ {
        alias /var/www/your-perl-app/public/static/;
        expires 30d;
        access_log off;
        add_header Cache-Control "public";
    }

    # Proxy requests to Gunicorn
    location / {
        proxy_pass http://127.0.0.1:5000; # Assuming Gunicorn runs on port 5000
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s; # Increase timeout for long-running requests
        proxy_connect_timeout 75s;
    }

    # Error pages
    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
        root /usr/share/nginx/html;
    }
}

Example Nginx Server Block (for PHP-FPM)

server {
    listen 80;
    server_name your-perl-cgi-app.example.com;
    root /var/www/your-perl-cgi-app/public_html;
    index index.pl index.php index.html index.htm;

    location / {
        try_files $uri $uri/ /index.pl?$args;
    }

    # Pass PHP scripts to PHP-FPM
    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        # Adjust the socket path based on your PHP-FPM configuration
        fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    # Pass Perl CGI scripts to PHP-FPM (if using a module like fcgiwrap or similar setup)
    # This is a simplified example; actual configuration might involve a dedicated CGI handler
    location ~ \.pl$ {
        # Assuming fcgiwrap or similar is configured to pass .pl files to a Perl interpreter
        # This might require a specific fastcgi_pass directive pointing to the fcgiwrap socket
        # Example: fastcgi_pass unix:/var/run/fcgiwrap.socket;
        # include fastcgi_params;
        # fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

        # For direct CGI execution via Nginx (less common, often requires specific modules or wrappers)
        # This example assumes a setup where Nginx can directly invoke CGI scripts.
        # A more robust solution often involves a FastCGI wrapper.
        # If using a Perl FastCGI module directly:
        # fastcgi_pass unix:/var/run/perl-fastcgi.sock;
        # include fastcgi_params;
        # fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

        # For demonstration, let's assume a common setup where .pl files are handled by PHP-FPM
        # if PHP-FPM is configured to execute them (e.g., via a custom handler or a wrapper).
        # This is NOT standard PHP-FPM behavior for .pl files.
        # A more typical approach for Perl CGI is a dedicated FastCGI server or fcgiwrap.

        # If you are using a Perl FastCGI server:
        # fastcgi_pass 127.0.0.1:9000; # Or your Perl FastCGI server address
        # include fastcgi_params;
        # fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

        # For a more direct CGI approach, consider fcgiwrap:
        # include fastcgi_params;
        # fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        # fastcgi_pass unix:/var/run/fcgiwrap.socket; # Ensure fcgiwrap is installed and configured

        # Fallback to PHP-FPM for .pl if it's specifically configured to handle them (unlikely by default)
        # This is a placeholder and may not work without specific server-side configuration.
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    # Deny access to hidden files
    location ~ /\. {
        deny all;
    }
}

Note on Perl CGI/FastCGI: Directly proxying `.pl` files to PHP-FPM is generally not how PHP-FPM is intended to be used. For Perl CGI or FastCGI applications, you’d typically use a dedicated Perl FastCGI server (like `fcgiwrap` or a custom Perl FastCGI implementation) and configure Nginx to proxy to its socket or TCP port. The example above for `.pl` files is illustrative and might require specific setup on the server to function correctly.

Application Server Tuning: Gunicorn vs. PHP-FPM

Gunicorn (for WSGI Perl Frameworks like Mojolicious)

Gunicorn is a Python WSGI HTTP Server, but it’s commonly used to serve Perl applications built with WSGI-compatible frameworks. Its configuration is straightforward and focuses on worker processes and threads.

Gunicorn Command-Line Configuration

# Example command to start Gunicorn for a Mojolicious app
# Assuming your Mojolicious app is in 'my_app.pl' and you have a 'psgi' file
# or a direct entry point. For Mojolicious, you might run it directly.

# For a Mojolicious app (often run directly or via a wrapper script)
# Example: myapp.pl daemon -l http://127.0.0.1:5000 -m 6

# If using a PSGI wrapper (e.g., for Dancer2 or other PSGI apps)
# gunicorn --workers 4 --threads 2 --bind 127.0.0.1:5000 my_app:app
# --workers: Number of worker processes. A common starting point is (2 * number_of_cores) + 1.
# --threads: Number of threads per worker. Useful for I/O bound tasks.
# --bind: The address and port Gunicorn listens on.
# --timeout: Request timeout in seconds.
# --worker-class: e.g., 'sync', 'gevent', 'eventlet'. 'sync' is default, 'gevent'/'eventlet' for async I/O.

# For a more robust setup, use a process manager like systemd or supervisord.

Systemd Service File for Gunicorn

[Unit]
Description=Gunicorn instance to serve my_perl_app
After=network.target

[Service]
User=your_app_user
Group=your_app_group
WorkingDirectory=/path/to/your/perl/app
Environment="PATH=/path/to/your/venv/bin:/usr/local/bin:/usr/bin:/bin" # Adjust PATH if using virtualenv
ExecStart=/path/to/your/venv/bin/gunicorn --workers 4 --threads 2 --bind unix:/path/to/your/app/gunicorn.sock my_app:app
# Or for TCP: ExecStart=/path/to/your/venv/bin/gunicorn --workers 4 --threads 2 --bind 127.0.0.1:5000 my_app:app

Restart=always
RestartSec=10

[Install]
StandardOutput=journal
StandardError=journal
SyslogIdentifier=gunicorn-my_perl_app

Tuning Gunicorn: The optimal number of workers depends heavily on the application’s I/O patterns and the server’s CPU cores. For CPU-bound tasks, fewer workers with more threads might be beneficial. For I/O-bound tasks, more workers and fewer threads (or async workers) are often better. Monitor CPU and memory usage to find the sweet spot.

PHP-FPM (for Perl CGI/FastCGI via wrappers)

If your Perl application is structured as CGI scripts or uses the FastCGI protocol, and you’re leveraging PHP-FPM (perhaps through a wrapper like `fcgiwrap` or a custom setup), tuning PHP-FPM is critical. The primary configuration file is `php-fpm.conf` or files within `php-fpm.d/`. We’ll focus on the pool configuration.

PHP-FPM Pool Configuration (`www.conf`)

; Example pool configuration for a Perl application
; Typically located in /etc/php/7.4/fpm/pool.d/your_app.conf

[your_app]
user = your_app_user
group = your_app_group
listen = /var/run/php/php7.4-fpm-your_app.sock ; Use a dedicated socket for isolation
listen.owner = www-data
listen.group = www-data
listen.mode = 0660

pm = dynamic
pm.max_children = 50       ; Max number of child processes
pm.start_servers = 5       ; Number of servers started on boot
pm.min_spare_servers = 2   ; Min number of idle servers
pm.max_spare_servers = 10  ; Max number of idle servers
pm.process_idle_timeout = 10s ; Timeout for idle processes

pm.max_requests = 500      ; Max requests per child process before restart (prevents memory leaks)

request_terminate_timeout = 120s ; Timeout for individual requests
request_slowlog_timeout = 30s    ; Log requests slower than this
slowlog = /var/log/php/php7.4-fpm-your_app-slow.log

catch_workers_output = yes
; php_admin_value[memory_limit] = 256M ; Adjust memory limit if needed
; php_admin_value[upload_max_filesize] = 64M
; php_admin_value[post_max_size] = 64M

Tuning PHP-FPM: The `pm` (process manager) settings are key. `dynamic` is often a good balance. `pm.max_children` should be set based on available RAM. A rough guideline: `max_children * (average_memory_per_process) < available_RAM`. Monitor memory usage closely. `pm.max_requests` helps prevent memory leaks by recycling worker processes.

MongoDB Performance Tuning

For data persistence, MongoDB is a common choice. Performance tuning involves indexing, query optimization, and server configuration.

Indexing Strategies

Proper indexing is paramount. Analyze slow queries using `db.slowQueries.find()` (if enabled) or MongoDB’s profiler. Use `explain()` on your queries to identify missing indexes.

Example Indexing

// Example: Indexing a 'users' collection
db.users.createIndex( { username: 1 }, { unique: true } );
db.users.createIndex( { email: 1 }, { unique: true } );
db.users.createIndex( { "profile.last_login": -1 } ); // For sorting by last login

// Example: Compound index for a common query pattern
// Query: db.orders.find( { status: "processing", created_at: { $gt: ISODate("2023-01-01") } } )
db.orders.createIndex( { status: 1, created_at: 1 } );

MongoDB Server Configuration (`mongod.conf`)

The `mongod.conf` file (typically YAML) allows tuning of various server parameters.

`mongod.conf` Snippet

systemLog:
  destination: file
  path: /var/log/mongodb/mongod.log
  logAppend: true
  verbosity: 0 # 0 is default, higher values increase logging detail

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true # Essential for durability and performance
  # engine: wiredTiger # Default engine, highly optimized
  wiredTiger:
    engineConfig:
      cacheSizeGB: 0.5 # Allocate a portion of system RAM for the WiredTiger cache.
                       # A common starting point is 50% of RAM, but monitor usage.
                       # For a 4GB RAM server, start with 1-2GB.

net:
  port: 27017
  bindIp: 127.0.0.1 # Or specific IPs for remote access, e.g., 0.0.0.0 for all interfaces (use with caution and firewall)

# Sharding configuration (if applicable)
# sharding:
#   clusterRole: configsvr
#   configsvrFilePermissions:
#     - mode: "0600"
#       user: "mongodb"
#       group: "mongodb"

# Security settings (essential for production)
# security:
#   authorization: enabled

# Replication settings (for high availability)
# replication:
#   replSetName: "rs0"

Tuning MongoDB: The `wiredTiger.cacheSizeGB` is the most impactful setting for performance. Ensure it’s not set too high, leaving enough RAM for the OS and other processes. Enabling the journal is crucial for data integrity. For production, enable security and replication.

Monitoring and Diagnostics

Continuous monitoring is key to identifying bottlenecks. Utilize tools like:

  • Nginx: `nginx -t` (config test), `nginx status`, `tail -f /var/log/nginx/access.log`, `tail -f /var/log/nginx/error.log`, `netstat -anp | grep nginx`, `htop`.
  • Gunicorn/PHP-FPM: `systemctl status gunicorn`, `systemctl status php7.4-fpm`, `journalctl -u gunicorn`, `tail -f /var/log/php/php7.4-fpm-your_app.log`, `htop`.
  • MongoDB: `mongostat`, `mongotop`, `db.serverStatus()`, `db.stats()`, `db.collection.stats()`, `explain()`, MongoDB Atlas monitoring tools (if applicable).
  • System: `htop`, `vmstat`, `iostat`, `dmesg`.

Regularly review logs for errors and warnings. Use `strace` on specific processes for deep-dive debugging if necessary.