Advanced Server Management: Setting Up an Ollama Reverse Proxy

If you’re running Ollama on your Ubuntu server, you likely know it defaults to 127.0.0.1:11434. While this works for local testing, exposing it securely to external platforms requires a robust reverse proxy. In this guide, we’ll use Apache to bridge that gap and secure your firewall.

Step 1: Enable Necessary Apache Modules

To act as a proxy, Apache needs its proxy and proxy_http modules enabled. Run the following commands:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo systemctl restart apache2

Step 2: Configure the VirtualHost

Create or edit your site configuration file (e.g., /etc/apache2/sites-available/qwen7b.vengalavinay.com.conf). This setup handles the internal rerouting and security layers.

<VirtualHost *:80>
    ServerName qwen7b.vengalavinay.com

    # Reroute traffic to the local Ollama instance
    ProxyPreserveHost Off
    ProxyPass / http://127.0.0.1:11434/
    ProxyPassReverse / http://127.0.0.1:11434/

    # Security: Basic Authentication
    <Location />
        AuthType Basic
        AuthName "Restricted AI Access"
        AuthUserFile /etc/apache2/.htpasswd
        Require valid-user
    </Location>

    # Important: Prevent Ollama from parsing the Auth header
    RequestHeader unset Authorization
</VirtualHost>

💡 Pro Tip: ProxyPreserveHost Off. Setting this to Off ensures that Ollama sees the request as coming from localhost. Since Ollama is configured to trust local traffic, this prevents cross-origin rejection (CORS) errors.

Step 3: Secure Your Firewall (UFW)

One of the most critical steps in any reverse proxy setup is ensuring your internal ports are not accessible from the outside. You want all traffic to go through Apache (Port 80/443), where your authentication and SSL are handled.

1. Deny Remote Access to Ollama

By default, Ollama only listens on 127.0.0.1, but if you’ve ever changed that setting, you must ensure the firewall blocks external attempts to reach port 11434:

sudo ufw deny 11434/tcp

2. Only Allow Standard Web Ports

Ensure that only the standard HTTP and HTTPS ports are open globally:

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw status verbose

With these rules, anyone attempting to reach http://your-ip:11434 will be blocked, forcing them to use https://qwen7b.vengalavinay.com where they will be prompted for your Basic Auth credentials.

Step 4: Handling Authorization Conflicts

One common “gotcha” when proxying Ollama is that Ollama’s built-in API parser might try to interpret the Authorization header you’re using for Apache. This often leads to a 401 Unauthorized response even if your credentials are correct.

The solution is to use RequestHeader unset Authorization within your Apache config. This strips the header after Apache has verified the user but before it hands the request over to Ollama.

✅ Verification

After saving your configuration and restarting Apache, try to access your endpoint using curl:

curl -u your_username:your_password https://qwen7b.vengalavinay.com/v1/chat/completions

You should now have a fully functional, secure gateway to your local AI models!