Installing and Configuring Gunicorn with Gevent Workers on Rocky Linux 9 for High-Concurrency Python Django Sites

Rocky Linux 9 System Preparation

Before installing Gunicorn and Gevent, ensure your Rocky Linux 9 system is up-to-date and has the necessary development tools. This foundational step is critical for a stable and secure deployment.

Begin by updating your package repositories and upgrading existing packages:

sudo dnf update -y
sudo dnf upgrade -y

Next, install essential build tools and Python development headers. These are required for compiling C extensions that Gevent might depend on, and for general Python package management.

sudo dnf install -y gcc python3-devel python3-pip openssl-devel libffi-devel bzip2-devel

Python Virtual Environment Setup

It is a best practice to isolate your Python application dependencies within a virtual environment. This prevents conflicts between different projects and system-wide Python packages. We’ll use Python’s built-in `venv` module.

Create a project directory and navigate into it. Then, create a virtual environment named `venv`:

mkdir -p /opt/my_django_app
cd /opt/my_django_app
python3 -m venv venv

Activate the virtual environment. You’ll notice your shell prompt changes to indicate that the environment is active.

source venv/bin/activate

Installing Gunicorn and Gevent

With the virtual environment activated, you can now install Gunicorn and the Gevent library. Gevent is a coroutine-based networking library that uses greenlet to provide a high-level synchronous API around the `libev` and `libuv` event loops. This allows Gunicorn to handle many concurrent connections efficiently with its worker processes.

pip install gunicorn gevent

For a typical Django project, you would also install Django and any other project-specific dependencies here. For example:

pip install django djangorestframework psycopg2-binary # Example dependencies

Configuring Gunicorn with Gevent Workers

Gunicorn needs to be configured to use Gevent workers. This is typically done via command-line arguments or a configuration file. For production, a configuration file is recommended for better organization and maintainability.

Create a Gunicorn configuration file, for instance, `gunicorn_config.py`, in your project’s root directory (the same directory where `manage.py` resides if it’s a Django project).

import multiprocessing

# Bind to a specific IP address and port
bind = "0.0.0.0:8000"

# Number of worker processes. A common starting point is (2 * number_of_cores) + 1.
# For Gevent, this number can often be higher as workers are I/O bound and can
# yield control efficiently. Experimentation is key.
workers = multiprocessing.cpu_count() * 2 + 1

# Use the gevent worker class
worker_class = "gevent"

# Maximum number of simultaneous connections per worker.
# Gevent workers can handle a large number of connections due to their non-blocking nature.
# A value between 1000 and 10000 is common, but depends heavily on your application's I/O patterns.
worker_connections = 1000

# Timeout for worker processes. If a worker takes longer than this to respond,
# it will be killed and restarted.
timeout = 30

# Log level. Options: 'debug', 'info', 'warning', 'error', 'critical'.
loglevel = "info"

# Path to the access log file
accesslog = "/var/log/gunicorn/access.log"

# Path to the error log file
errorlog = "/var/log/gunicorn/error.log"

# If using Django, specify the WSGI application path
# For example, if your Django project is named 'myproject', and wsgi.py is in
# /opt/my_django_app/myproject/wsgi.py, then:
# wsgi_app = "myproject.wsgi:application"
# If you are not using Django, replace this with your application's WSGI callable.
# wsgi_app = "your_module:your_wsgi_application"

Ensure the log directory exists and has appropriate permissions for Gunicorn to write to it.

sudo mkdir -p /var/log/gunicorn
sudo chown -R $(whoami):$(whoami) /var/log/gunicorn # Adjust ownership as needed for your deployment user

Running Gunicorn with Gevent Workers

Now, you can start Gunicorn using the configuration file. Navigate to your project’s root directory (where `gunicorn_config.py` and `manage.py` are located) and execute the following command. Replace `your_project_name.wsgi:application` with the actual path to your WSGI application.

# Ensure you are in your project's root directory and venv is activated
# Example for a Django project named 'myproject'
gunicorn --config gunicorn_config.py myproject.wsgi:application

To run Gunicorn in the background as a service, it’s highly recommended to use a process manager like `systemd`. Create a systemd service file.

[Unit]
Description=Gunicorn instance to serve my_django_app
After=network.target

[Service]
User=your_deploy_user # Replace with your deployment user
Group=your_deploy_group # Replace with your deployment group
WorkingDirectory=/opt/my_django_app
Environment="PATH=/opt/my_django_app/venv/bin"
ExecStart=/opt/my_django_app/venv/bin/gunicorn --config /opt/my_django_app/gunicorn_config.py myproject.wsgi:application

# Optional: If you need to specify environment variables for your application
# EnvironmentFile=/etc/my_django_app/environment

Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

Replace `your_deploy_user`, `your_deploy_group`, `/opt/my_django_app`, and `myproject.wsgi:application` with your specific values. Save this file as `/etc/systemd/system/gunicorn.service`.

Reload the systemd daemon, start the Gunicorn service, and enable it to start on boot:

sudo systemctl daemon-reload
sudo systemctl start gunicorn
sudo systemctl enable gunicorn

You can check the status of the Gunicorn service and view logs with:

sudo systemctl status gunicorn
sudo journalctl -u gunicorn -f

Integrating with a Reverse Proxy (Nginx)

For production environments, Gunicorn should typically be run behind a reverse proxy like Nginx. Nginx will handle SSL termination, serve static files, and forward dynamic requests to Gunicorn.

Install Nginx if you haven’t already:

sudo dnf install -y nginx

Create an Nginx server block configuration file for your application. For example, `/etc/nginx/conf.d/my_django_app.conf`:

server {
    listen 80;
    server_name your_domain.com www.your_domain.com; # Replace with your domain

    # Optional: For SSL, uncomment and configure
    # listen 443 ssl http2;
    # ssl_certificate /etc/letsencrypt/live/your_domain.com/fullchain.pem;
    # ssl_certificate_key /etc/letsencrypt/live/your_domain.com/privkey.pem;
    # include /etc/letsencrypt/options-ssl-nginx.conf;
    # ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location /static/ {
        alias /opt/my_django_app/static_collected/; # Path where Django's collectstatic puts files
    }

    location /media/ {
        alias /opt/my_django_app/media/; # Path for user-uploaded media
    }

    location / {
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://127.0.0.1:8000; # Gunicorn's bind address
        proxy_read_timeout 300s; # Adjust if your app has long-running requests
        proxy_connect_timeout 75s;
    }

    # If using SSL, redirect HTTP to HTTPS
    # if ($scheme != "https") {
    #     return 301 https://$host$request_uri;
    # }
}

Ensure your Django `settings.py` has `STATIC_ROOT` configured correctly if you are serving static files via Nginx. Run `python manage.py collectstatic` to gather all static files into the `STATIC_ROOT` directory.

# In your Django project's root directory, with venv activated
python manage.py collectstatic

Test the Nginx configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Tuning and Monitoring

The `worker_connections` setting in `gunicorn_config.py` is crucial for Gevent. A higher value allows each worker to handle more concurrent requests. However, excessively high values can lead to resource exhaustion (memory, file descriptors). Monitor your system’s resource usage (CPU, memory, network sockets) under load to find the optimal balance.

Key metrics to monitor include:

• Gunicorn worker status (running, restarting, dead).
• CPU and memory usage per worker process.
• Network connection counts (e.g., using ss -an | grep :8000 | wc -l).
• Application response times and error rates.
• Nginx access and error logs for insights into request handling.

Tools like Prometheus with `node_exporter` and `gunicorn_exporter` (if available or custom metrics are exposed), or ELK stack for log aggregation, are invaluable for comprehensive monitoring in production. Adjusting `workers` and `worker_connections` should be an iterative process based on observed performance and resource utilization.