Vengala Vinay

Having 9+ Years of Experience in Software Development

Home » Zero-Downtime Blue-Green Deployment Pipelines for C Applications on OVH

Zero-Downtime Blue-Green Deployment Pipelines for C Applications on OVH

Understanding Blue-Green Deployments for C Applications

Achieving zero-downtime deployments for C applications, especially those with long-running processes or critical dependencies, requires a robust strategy. Blue-Green deployment offers a pragmatic approach by maintaining two identical production environments: “Blue” (current production) and “Green” (staging/new version). Traffic is initially directed to Blue. Once Green is fully deployed and tested, traffic is switched from Blue to Green. This allows for instant rollback by simply switching traffic back to Blue if issues arise.

For C applications, this often involves managing compiled binaries, configuration files, and potentially shared libraries. The key challenge is ensuring that the state of the application and its dependencies are consistent across both environments and that the transition is seamless.

OVH Infrastructure Setup for Blue-Green

Our OVH infrastructure will leverage Load Balancers (LBaaS) for traffic management and potentially multiple instances of our C application running on dedicated servers or virtual machines (Public Cloud Instances). We’ll assume a basic setup where the C application listens on a specific port (e.g., 8080) and is managed by a systemd service.

The OVH Load Balancer will be the central point for directing traffic. We’ll configure it to point to either the Blue or Green environment. For this example, let’s assume we have two distinct sets of servers, each capable of running the full application stack.

Load Balancer Configuration (Conceptual)

While OVH’s LBaaS configuration is typically done via their control panel or API, the underlying principle involves defining backend pools and health checks. We’ll need two backend pools:

  • Blue Pool: Points to the IP addresses/hostnames of the servers running the current production version (Blue).
  • Green Pool: Points to the IP addresses/hostnames of the servers running the new version (Green).

Health checks are crucial. They should be configured to ping a specific endpoint on the application (e.g., a `/health` HTTP endpoint if your C application exposes one, or a TCP check on the application port). The LBaaS will only send traffic to healthy instances within a pool.

Automating the Deployment Pipeline

A robust CI/CD pipeline is essential. We’ll use a combination of Git, a CI server (e.g., Jenkins, GitLab CI, GitHub Actions), and scripting to manage the deployment process. The pipeline will be triggered by a merge to the main branch or a specific release tag.

CI Stage: Building and Testing

The Continuous Integration stage focuses on compiling the C application and running automated tests. This should happen in an isolated environment to ensure consistency.

Example CI Script (Conceptual – e.g., GitLab CI `.gitlab-ci.yml`):

stages:
  - build
  - test
  - deploy_green

variables:
  APP_NAME: my_c_app
  BUILD_DIR: build
  ARTIFACT_PATH: /opt/artifacts/${CI_COMMIT_REF_SLUG}/${CI_COMMIT_SHA}.tar.gz

build_app:
  stage: build
  image: gcc:latest # Or a custom Docker image with your build tools
  script:
    - mkdir -p ${BUILD_DIR}
    - cd ${BUILD_DIR}
    - cmake .. -DCMAKE_BUILD_TYPE=Release
    - make
    - make install DESTDIR=./install
    - tar -czvf ${ARTIFACT_PATH} -C ./install .
  artifacts:
    paths:
      - ${ARTIFACT_PATH}
    expire_in: 1 week

run_unit_tests:
  stage: test
  image: gcc:latest # Or a custom Docker image
  script:
    - cd ${BUILD_DIR}
    - make test # Assuming your build system supports running tests
  needs:
    - build_app

# Further stages for integration tests, security scans, etc. would go here.

CD Stage: Deploying to Green

The Continuous Deployment stage handles pushing the built artifact to the Green environment and starting the application. This is where the blue-green logic truly begins.

We’ll need a mechanism to provision and configure the Green servers. This could involve Ansible, Terraform, or custom scripts. For simplicity, we’ll focus on the deployment script itself, assuming servers are already provisioned and have the necessary base OS and dependencies.

Deployment Script (e.g., `deploy_green.sh`):

#!/bin/bash

# --- Configuration ---
GREEN_SERVERS=("192.168.1.101" "192.168.1.102") # IPs of Green environment servers
APP_USER="appuser"
APP_INSTALL_DIR="/opt/my_c_app"
APP_SERVICE_NAME="my_c_app.service"
ARTIFACT_URL="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/artifacts/${CI_COMMIT_SHA}?job=${CI_JOB_ID}" # Example for GitLab CI

# --- Functions ---
deploy_to_server() {
  local server=$1
  echo "Deploying to $server..."

  # 1. Download artifact (adjust based on your CI/CD artifact storage)
  echo "Downloading artifact from $ARTIFACT_URL..."
  curl -fLO "$ARTIFACT_URL" -o "${APP_NAME}-${CI_COMMIT_SHA}.tar.gz"

  # 2. Extract artifact to a temporary location
  local temp_extract_dir="/tmp/${APP_NAME}_deploy_${CI_COMMIT_SHA}"
  mkdir -p "$temp_extract_dir"
  tar -xzf "${APP_NAME}-${CI_COMMIT_SHA}.tar.gz" -C "$temp_extract_dir"

  # 3. Stop the application service on the Green server
  echo "Stopping $APP_SERVICE_NAME on $server..."
  ssh "${APP_USER}@${server}" "sudo systemctl stop ${APP_SERVICE_NAME}"

  # 4. Remove old application files
  echo "Removing old application files on $server..."
  ssh "${APP_USER}@${server}" "sudo rm -rf ${APP_INSTALL_DIR}/*"

  # 5. Transfer new application files
  echo "Transferring new application files to $server..."
  scp -r "$temp_extract_dir"/* "${APP_USER}@${server}:${APP_INSTALL_DIR}/"

  # 6. Set correct permissions (if necessary)
  echo "Setting permissions on $server..."
  ssh "${APP_USER}@${server}" "sudo chown -R ${APP_USER}:${APP_USER} ${APP_INSTALL_DIR}"
  ssh "${APP_USER}@${server}" "sudo chmod +x ${APP_INSTALL_DIR}/bin/*" # Ensure executables are executable

  # 7. Update configuration files (if any)
  # Example: Copying a new config file
  # scp /path/to/new/config.conf "${APP_USER}@${server}:${APP_INSTALL_DIR}/conf/config.conf"

  # 8. Start the application service
  echo "Starting $APP_SERVICE_NAME on $server..."
  ssh "${APP_USER}@${server}" "sudo systemctl start ${APP_SERVICE_NAME}"

  # 9. Verify application health (crucial!)
  echo "Verifying application health on $server..."
  # This is a simplified check. A real-world scenario would involve
  # checking a health endpoint or performing more thorough checks.
  sleep 5 # Give the app time to start
  if ssh "${APP_USER}@${server}" "sudo systemctl is-active ${APP_SERVICE_NAME}" | grep -q "active"; then
    echo "Application started successfully on $server."
  else
    echo "ERROR: Application failed to start on $server."
    # Attempt to restart the old version if possible, or at least log the error
    ssh "${APP_USER}@${server}" "sudo systemctl start ${APP_SERVICE_NAME}.old" # Assuming a mechanism for old service
    exit 1
  fi

  # 10. Clean up local artifact
  rm "${APP_NAME}-${CI_COMMIT_SHA}.tar.gz"
  rm -rf "$temp_extract_dir"
}

# --- Main Execution ---
echo "Starting deployment to Green environment..."
for server in "${GREEN_SERVERS[@]}"; do
  deploy_to_server "$server"
done

echo "Deployment to Green environment completed. Manual traffic switch required."
exit 0

Systemd Service File Example (`my_c_app.service`):

[Unit]
Description=My C Application Service
After=network.target

[Service]
User=appuser
Group=appuser
WorkingDirectory=/opt/my_c_app
ExecStart=/opt/my_c_app/bin/my_c_app --config=/opt/my_c_app/conf/config.ini
Restart=on-failure
# Consider adding a mechanism to manage the 'old' version for rollback
# ExecStop=/opt/my_c_app/bin/stop_script.sh

[Install]
WantedBy=multi-user.target

The Traffic Switching Mechanism

This is the critical step for achieving zero downtime. Once the Green environment is deployed and verified, traffic needs to be switched. This is typically done by reconfiguring the Load Balancer.

Manual Switch (via OVH Control Panel/API):

  • Navigate to your Load Balancer configuration in the OVH control panel.
  • Locate the backend pool associated with the “Blue” environment and disable it or remove its servers.
  • Locate the backend pool associated with the “Green” environment and enable it or add its servers.
  • Monitor traffic and application health closely.

Automated Switch (using OVH API and scripting):

For full automation, you’d integrate OVH API calls into your CI/CD pipeline. This requires obtaining API credentials and using a tool like `curl` or a dedicated OVH SDK.

Example Script Snippet for API Switch (Conceptual):

#!/bin/bash

# --- Configuration ---
OVH_API_ENDPOINT="https://api.ovh.com/1.0"
CONSUMER_KEY="YOUR_CONSUMER_KEY"
API_KEY="YOUR_API_KEY" # From your application key
SECRET_KEY="YOUR_SECRET_KEY" # From your application key
SIGNATURE="YOUR_SIGNATURE" # Generated dynamically
TIMESTAMP=$(date +%s)
METHOD="POST" # Or PUT, DELETE depending on the API call
URL_PATH="/cloud/project/YOUR_PROJECT_ID/loadbalancer/YOUR_LOADBALANCER_ID/frontend/YOUR_FRONTEND_ID/defaultBackendSet" # Example path

# --- Function to generate signature ---
generate_signature() {
  local method=$1
  local url_path=$2
  local body=$3
  local timestamp=$4

  local string_to_sign="${SECRET_KEY}+${CONSUMER_KEY}+${method}+${OVH_API_ENDPOINT}${url_path}+${body}+${timestamp}"
  echo -n "$string_to_sign" | openssl dgst -sha1 -hmac "$SECRET_KEY" | awk '{print $2}'
}

# --- Function to call OVH API ---
call_ovh_api() {
  local method=$1
  local url_path=$2
  local body=$3

  local signature=$(generate_signature "$method" "$url_path" "$body" "$TIMESTAMP")

  curl -s -X "$method" \
    -H "X-OVH-Application: $API_KEY" \
    -H "X-OVH-Consumer: $CONSUMER_KEY" \
    -H "X-OVH-Timestamp: $TIMESTAMP" \
    -H "X-OVH-Signature: $SIGNATURE" \
    -H "Content-Type: application/json" \
    "${OVH_API_ENDPOINT}${url_path}" \
    ${body:+-d "$body"}
}

# --- Switch traffic to Green ---
echo "Switching traffic to Green backend set..."

# Assuming 'backendSets' is an array and you know the ID of the Green backend set
# You'll need to query the LB configuration first to get the correct backend set IDs.
GREEN_BACKEND_SET_ID="backendset-green-id" # Replace with actual ID

# Construct the JSON payload to set the default backend set
JSON_PAYLOAD='{"defaultBackendSet": "'${GREEN_BACKEND_SET_ID}'"}'

# Call the API to update the default backend set
response=$(call_ovh_api "PUT" "/cloud/project/YOUR_PROJECT_ID/loadbalancer/YOUR_LOADBALANCER_ID/frontend/YOUR_FRONTEND_ID" "$JSON_PAYLOAD")

if echo "$response" | grep -q '"status": "OK"'; then # Adjust based on actual API response structure
  echo "Traffic switched to Green successfully."
else
  echo "ERROR: Failed to switch traffic."
  echo "API Response: $response"
  # Implement rollback logic here
  exit 1
fi

exit 0

Rollback Strategy

The beauty of Blue-Green is the immediate rollback capability. If the Green deployment introduces critical bugs or performance regressions, the process is reversed:

  • Manual Rollback: Reconfigure the Load Balancer to point back to the Blue backend pool.
  • Automated Rollback: Execute a script that calls the OVH API to switch the default backend set back to the Blue environment.

It’s crucial to have the Blue environment remain stable and untouched during the Green deployment and testing phase. If the Blue environment needs to be updated, it becomes the new Green environment for the subsequent deployment.

Considerations for C Applications

State Management: If your C application manages persistent state (e.g., in-memory caches, local databases), ensure this state is handled correctly during the switch. This might involve shared storage, replication, or a strategy to reload state from a persistent backend.

Database Migrations: If your application interacts with a database, any schema changes must be backward-compatible. Deploying a new application version that requires a breaking database change before the database migration is applied will cause failures. Plan database migrations carefully, often deploying them before the application code that uses them.

Long-Running Connections: For applications with persistent connections (e.g., WebSockets, gRPC), a hard switch can drop these connections. Consider strategies like graceful shutdown on the Blue servers, allowing existing connections to complete before they are taken out of rotation.

Resource Management: Ensure both Blue and Green environments have identical resource allocations (CPU, RAM, network) to avoid performance discrepancies that could mask issues.

Testing Green: Thoroughly test the Green environment *before* switching traffic. This includes automated integration tests, performance tests, and potentially a period of canary release where a small percentage of live traffic is directed to Green.