Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » How We Audited a High-Traffic Perl Enterprise Stack on Google Cloud and Mitigated Remote Code Execution (RCE) via eval block syntax flaws

How We Audited a High-Traffic Perl Enterprise Stack on Google Cloud and Mitigated Remote Code Execution (RCE) via eval block syntax flaws

Initial Reconnaissance and Attack Surface Identification

Our engagement began with a deep dive into the existing infrastructure. The core of the application was a Perl monolith, handling millions of requests daily, hosted on Google Cloud Platform (GCP). The primary attack vectors we focused on were user-supplied input points that could potentially reach dangerous Perl constructs, particularly eval.

The first step was to map out all external-facing services and identify potential input sinks. This involved:

  • Reviewing Nginx access logs for unusual request patterns and parameters.
  • Analyzing application code for functions that accept user input and pass it to potentially unsafe Perl constructs.
  • Identifying all API endpoints and form submissions.
  • Scanning for exposed administrative interfaces or debugging endpoints.

We utilized GCP’s Cloud Logging and Cloud Monitoring to extract relevant log data. A key tool for this phase was a custom Bash script to aggregate and filter Nginx logs, looking for parameters that might be candidates for injection:

Log Aggregation and Parameter Extraction Script

#!/bin/bash

LOG_FILE="/var/log/nginx/access.log"
OUTPUT_FILE="potential_injection_points.txt"

echo "Extracting potential injection parameters from $LOG_FILE..."

# Extract parameters from GET and POST requests
# This is a simplified example; a real-world scenario would involve more sophisticated parsing
# to handle different content types (e.g., JSON, form-urlencoded) and URL encoding.
grep -oP 'GET .*? HTTP/1\.[01]' "$LOG_FILE" | sed -E 's/GET (.*?) HTTP\/1\.[01]/\1/' | grep -oP '\?[^ ]+' | sed 's/^[?&]//' | tr '&' '\n' | cut -d'=' -f1 | sort -u > "$OUTPUT_FILE"
grep -oP 'POST .*? HTTP/1\.[01]' "$LOG_FILE" | sed -E 's/POST (.*?) HTTP\/1\.[01]/\1/' | grep -oP '\?[^ ]+' | sed 's/^[?&]//' | tr '&' '\n' | cut -d'=' -f1 | sort -u >> "$OUTPUT_FILE"

# Further refinement to look for parameters that might be passed to eval
# This requires deeper code analysis, but as a proxy, we look for common parameter names.
grep -iE 'query|cmd|eval|script|code|payload' "$OUTPUT_FILE" | sort -u > "suspicious_parameters.txt"

echo "Potential injection parameters saved to $OUTPUT_FILE"
echo "Suspicious parameters (heuristic) saved to suspicious_parameters.txt"

This script, while basic, helped us identify parameters like cmd, eval_code, and query that were frequently present in requests and were prime candidates for further investigation within the Perl codebase.

Deep Dive into Perl Code: The eval Vulnerability

The core of the vulnerability lay in how user-supplied data was being processed by Perl’s eval function. In many Perl applications, eval is used for dynamic code execution, which is powerful but extremely dangerous if not handled with extreme care. We identified several instances where input parameters were directly or indirectly passed into an eval block without proper sanitization or validation.

Consider a simplified, vulnerable code snippet:

Vulnerable Perl Snippet

use strict;
use warnings;

my $user_input = <>; # Simulating input from a web request parameter

# BAD: Direct eval of user input
eval $user_input;
if ($@) {
    print "Error: $@\n";
}

In this example, any string passed to eval would be executed as Perl code. An attacker could send a string like system('ls -la /'), and the Perl interpreter would execute it, leading to Remote Code Execution (RCE).

A more insidious variant involved string interpolation within the eval block, where the user input was concatenated into a larger string that was then evaluated. This often bypassed simple checks if the attacker could control parts of the string that influenced the final evaluated code.

Example of Interpolation Vulnerability

use strict;
use warnings;

my $user_command_part = $ENV{USER_CMD_PARAM}; # Input from an environment variable or request parameter

# BAD: User input is interpolated into a string that is then eval'd
my $eval_string = "print 'Executing: ' . '$user_command_part';\n";
eval $eval_string;
if ($@) {
    print "Error: $@\n";
}

An attacker could craft $user_command_part to break out of the intended string context and execute arbitrary code. For instance, if $user_command_part was set to ' . system('id') . ', the evaluated string would become:

print 'Executing: ' . ' . system('id') . ';

This would execute the id command. The key was to find these eval calls and trace back the origin of the data being passed to them.

Mitigation Strategy: Input Validation and Sanitization

The primary mitigation strategy involved a multi-layered approach: strict input validation at the entry points and, where dynamic execution was absolutely necessary, extremely careful sanitization and sandboxing of the evaluated code.

1. Strict Input Validation

For parameters that were not intended to be code, we implemented strict validation. This meant defining expected data types, formats, and lengths, and rejecting anything that didn’t conform. For example, if a parameter was expected to be a numerical ID, we would ensure it was purely numeric and within a reasonable range.

2. Whitelisting Allowed Operations

In cases where a limited set of dynamic operations was required, we moved away from direct eval of user input. Instead, we implemented a whitelisting mechanism. This involved defining a set of allowed commands or operations, and mapping user-provided inputs to these predefined, safe operations.

3. Replacing eval with Safer Alternatives

Where eval was used for parsing or data manipulation, we sought to replace it with safer, dedicated modules. For instance:

  • For parsing configuration-like data, modules like Config::Simple or JSON were used instead of eval.
  • For performing calculations or simple logic, dedicated functions or a domain-specific language (DSL) interpreter could be built, avoiding general-purpose code execution.

4. Escaping and Contextual Encoding

If eval was unavoidable for a specific, highly controlled use case, the input had to be meticulously escaped. This is notoriously difficult to get right. The general principle is to ensure that any characters in the user input that have special meaning in Perl syntax (like ;, &, |, (, ), $, @, %, ', ", `, \) are neutralized. However, the exact escaping depends heavily on the context within the eval string.

A safer approach is to use modules designed for safe evaluation, such as Safe.pm, which provides a restricted execution environment. However, even Safe.pm has its limitations and requires careful configuration of the allowed operations.

Refactored Perl Code (Illustrative)

use strict;
use warnings;
use CGI qw(:standard); # Assuming CGI for web input

my $q = CGI->new;
my $user_param = $q->param('user_data'); # Get input from a web parameter

# Example: If 'user_data' is expected to be a simple string for display
# BAD: eval $user_param;

# GOOD: Sanitize for display, e.g., HTML escaping
use HTML::Entities;
print header;
print start_html('Safe Output');
print p("User data: ", encode_entities($user_param));
print end_html;

# Example: If 'user_data' is expected to be a number for a calculation
# BAD: my $result = eval $user_param + 5;

# GOOD: Validate as a number
if ($user_param =~ /^\d+$/) {
    my $number = int($user_param);
    my $result = $number + 5;
    print "Calculation result: $result\n";
} else {
    print "Invalid input: expected a number.\n";
}

# Example: If a specific, limited command execution is needed
# BAD: eval "system('$user_param')";

# GOOD: Whitelist and use system() safely (still requires careful validation of $user_param)
my %allowed_commands = (
    'status' => 'date',
    'version' => 'perl -v',
);

if (exists $allowed_commands{$user_param}) {
    my $command_to_run = $allowed_commands{$user_param};
    print "Executing: $command_to_run\n";
    system($command_to_run);
} else {
    print "Invalid command specified.\n";
}

Infrastructure-Level Hardening on GCP

Beyond code-level fixes, we implemented several infrastructure-level controls within GCP to further bolster security:

  • VPC Network Segmentation: Ensured that the application servers were in private subnets, with strict firewall rules allowing only necessary ingress and egress traffic.
  • Identity and Access Management (IAM): Reviewed and minimized service account permissions for the compute instances running the Perl application. Principle of least privilege was strictly applied.
  • Web Application Firewall (WAF): Configured Google Cloud Armor to provide an additional layer of defense against common web exploits, including those targeting injection vulnerabilities. Rules were tuned to detect suspicious patterns in requests targeting the identified input parameters.
  • Intrusion Detection Systems (IDS): Leveraged GCP’s Security Command Center and integrated third-party IDS solutions to monitor for anomalous network activity and potential exploit attempts.
  • Regular Patching and Updates: Established a robust process for keeping the operating system, Perl interpreter, and all installed Perl modules up-to-date to mitigate known vulnerabilities.

GCP Cloud Armor Rule Example (Conceptual)

# Example Cloud Armor rule to block requests with suspicious eval patterns
# This is a simplified representation; actual rules are more complex.

allow tcp:80;
allow tcp:443;

# Block requests containing common eval injection patterns in query parameters
# This rule is illustrative and would need significant tuning to avoid false positives.
deny request.query.param('user_data') =~ /eval\s*\(?\s*['"]?system/i;
deny request.query.param('cmd') =~ /eval\s*\(?\s*['"]?system/i;
deny request.query.param('script') =~ /eval\s*\(?\s*['"]?system/i;

# More specific rules targeting known vulnerable parameters
deny request.query.param('eval_code') =~ /system\(/i;

The combination of code remediation, input validation, and infrastructure hardening significantly reduced the attack surface and mitigated the identified RCE vulnerabilities. Continuous monitoring and periodic security audits remain critical for maintaining the security posture of such a high-traffic enterprise application.