Troubleshooting AppArmor Profiles blocking Nginx Unix Sockets on openSUSE Leap 15.5 during Startup cycles

Identifying the Root Cause: AppArmor Denials

When Nginx fails to start on openSUSE Leap 15.5 and the logs point to issues with Unix domain sockets, AppArmor is a frequent culprit. AppArmor, a mandatory access control (MAC) system, restricts program capabilities. By default, Nginx profiles might not explicitly permit access to specific socket paths, especially if they deviate from standard configurations or are dynamically created during startup. The primary diagnostic step is to examine the system logs for AppArmor denial messages.

The most reliable place to find these denials is the kernel’s audit log, accessible via dmesg or the systemd journal. Look for messages containing “apparmor=” and “DENIED”.

Leveraging `journalctl` for AppArmor Denials

To efficiently filter the systemd journal for AppArmor-related denial messages during Nginx startup, use the following command. This targets messages from the kernel’s audit subsystem that specifically mention “apparmor” and “denied”.

sudo journalctl -k | grep -i apparmor | grep -i denied

If Nginx is configured to use a Unix socket, for instance, in /var/run/nginx/php-fpm.sock, and AppArmor is blocking access, you might see output similar to this:

[<timestamp>] audit: type=1400 audit(1678886400.123:456): apparmor="DENIED" operation="connect" profile="nginx" pid=<nginx_pid> comm="nginx" src="/var/run/nginx/php-fpm.sock" dst="/var/run/nginx/php-fpm.sock" protocol="unix"

The key elements here are: profile="nginx" indicating the Nginx AppArmor profile, operation="connect" showing the attempted action, and src="/var/run/nginx/php-fpm.sock" (or similar) detailing the resource being accessed.

Analyzing and Modifying AppArmor Profiles

Once a denial is identified, the next step is to adjust the Nginx AppArmor profile. AppArmor profiles are typically located in /etc/apparmor.d/. The Nginx profile is often named /etc/apparmor.d/usr.sbin.nginx or a similarly derived name.

To understand the current state of AppArmor profiles, you can list them and check their modes:

sudo aa-status

This command will show which profiles are loaded and whether they are in ‘enforce’ or ‘complain’ mode. For troubleshooting, temporarily switching the Nginx profile to ‘complain’ mode is highly effective. In ‘complain’ mode, AppArmor logs violations but does not prevent them, allowing Nginx to start and operate while logging potential issues.

sudo aa-complain /etc/apparmor.d/usr.sbin.nginx

After setting to ‘complain’ mode, attempt to start Nginx again. If it starts successfully, the issue is confirmed to be an AppArmor restriction. Now, examine the logs again (using journalctl -k | grep apparmor) to see the *new* denial messages generated while in complain mode. These messages will provide the exact path and operation that needs to be explicitly allowed.

Permitting Unix Socket Access in the Nginx Profile

Edit the Nginx AppArmor profile file (e.g., /etc/apparmor.d/usr.sbin.nginx) using a text editor with root privileges.

sudo nano /etc/apparmor.d/usr.sbin.nginx

Locate the section that defines network access rules. You’ll likely see rules for TCP/IP sockets. To allow access to a specific Unix domain socket, you need to add a rule for the unix protocol. For example, if Nginx needs to connect to a PHP-FPM socket located at /var/run/php-fpm/php-fpm.sock, you would add a line like this:

# Allow Nginx to connect to PHP-FPM Unix socket
  unix,
  # The path below should match your actual PHP-FPM socket location
  /var/run/php-fpm/php-fpm.sock ix,

Explanation of the rule:

• unix,: This directive indicates that the following rules apply to Unix domain sockets.
• /var/run/php-fpm/php-fpm.sock ix,: This is the specific rule for the socket.
  • /var/run/php-fpm/php-fpm.sock: The absolute path to the Unix domain socket.
  • i: This permission allows the profile to ‘initiate’ a connection.
  • x: This permission allows the profile to ‘execute’ operations on the socket (e.g., send/receive data).

If Nginx itself is supposed to *create* the socket (less common for inter-process communication but possible for internal Nginx modules), you might need different permissions like mrw (read, write, create). However, for typical Nginx-to-PHP-FPM communication, ix is usually sufficient for the Nginx profile to connect to an existing socket.

After adding the necessary rule, save the file and reload the AppArmor profiles to apply the changes:

sudo systemctl reload apparmor

Finally, attempt to start Nginx again:

sudo systemctl start nginx

If Nginx starts without errors, you can optionally switch the profile back to ‘enforce’ mode:

sudo aa-enforce /etc/apparmor.d/usr.sbin.nginx

Handling Dynamic Socket Paths and Wildcards

In scenarios where the socket path might vary (e.g., based on user or a dynamic naming convention), AppArmor supports wildcards. For instance, if PHP-FPM sockets are created in /var/run/php-fpm/www-data/*.sock, you can use a wildcard:

# Allow Nginx to connect to PHP-FPM Unix sockets in a specific directory
  unix,
  /var/run/php-fpm/www-data/*.sock ix,

Be cautious with overly broad wildcards (e.g., /var/run/**/*.sock) as they can weaken security. Always restrict permissions to the minimum necessary path.

Advanced Troubleshooting: Profile Generation and Auditing

If the exact path or required permissions are unclear, AppArmor provides tools to help generate or refine profiles. The aa-genprof and aa-logprof utilities are invaluable.

aa-genprof can be used to create a new profile or update an existing one by observing program execution. However, for existing profiles like Nginx, it’s often more practical to use aa-logprof to parse existing audit logs and suggest additions.

sudo aa-logprof

This command will read the audit logs, identify denied operations, and prompt you interactively to decide whether to allow them and update the relevant profile. When prompted about the Unix socket connection, select ‘A’ (Allow) to add the necessary rule to the Nginx profile.

For persistent logging of AppArmor events, ensure the auditd service is running and configured correctly:

sudo systemctl status auditd
sudo systemctl enable auditd
sudo systemctl start auditd

The audit logs themselves are typically found in /var/log/audit/audit.log, which can be parsed by ausearch or directly by aa-logprof.