CI/CD Testing Guide

Overview

Testing CI/CD workflows is highly practical and recommended! This guide covers multiple approaches to test your GitHub Actions workflows.

Why Test CI/CD?

  • ✅ Catch issues early - Find workflow problems before pushing to GitHub
  • ✅ Faster feedback - Test locally without waiting for GitHub Actions
  • ✅ Cost savings - Avoid consuming GitHub Actions minutes during development
  • ✅ Confidence - Ensure workflows work correctly before deployment

Testing Approaches

act is a tool that runs GitHub Actions workflows locally using Docker.

Installation

# Linux
curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash

# macOS
brew install act

# Windows (with Chocolatey)
choco install act-cli

Prerequisites: Docker must be installed and running.

Quick Start

# List all workflows and jobs
cd tasks/04_devops_cicd
act -l

# Run all workflows (push event)
act push

# Run specific job
act push -j python-validation

# Run with verbose output
act push --verbose

Using the Test Script

We provide a convenient test script:

# Make script executable
chmod +x tasks/04_devops_cicd/scripts/test_ci_workflow.sh

# Run the test script
./tasks/04_devops_cicd/scripts/test_ci_workflow.sh

The script will:

  1. Check if act is installed
  2. Check if Docker is running
  3. List available jobs
  4. Let you choose which job to test

Limitations

  • Secrets: Must be provided manually (use act secrets)
  • AWS Services: Cannot test AWS-specific steps (S3, Glue) without credentials
  • GitHub API: Some actions require GitHub API access

Example: Testing with Secrets

# Create secrets file
cat > .secrets <<EOF
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test
EOF

# Run with secrets
act push --secret-file .secrets

2. Workflow Syntax Validation

Validate YAML syntax and workflow structure:

# Using actionlint (recommended)
chmod +x tasks/04_devops_cicd/scripts/validate_workflow_syntax.sh
./tasks/04_devops_cicd/scripts/validate_workflow_syntax.sh

Or manually:

# Install actionlint
# Linux: https://github.com/rhymond/actionlint#installation
# macOS: brew install actionlint

# Validate workflow
actionlint tasks/04_devops_cicd/.github/workflows/ci.yml

3. Manual Testing on GitHub

For full integration testing, push to a test branch:

# Create test branch
git checkout -b test/ci-workflow

# Make a small change (e.g., add a comment)
echo "# Test CI" >> tasks/04_devops_cicd/.github/workflows/ci.yml

# Commit and push
git add .
git commit -m "test: CI workflow"
git push origin test/ci-workflow

# Check GitHub Actions tab for results

Cleanup: 

git checkout main
git branch -D test/ci-workflow
git push origin --delete test/ci-workflow

4. Unit Testing Workflow Steps

Test individual workflow steps in isolation:

Example: Test Python Setup

# Test Python setup step locally
python3 -m venv test-venv
source test-venv/bin/activate
pip install --upgrade pip
pip install -r tasks/01_data_ingestion_transformation/requirements.txt
pip install -r tasks/01_data_ingestion_transformation/requirements-dev.txt

# Test linting
cd tasks/01_data_ingestion_transformation
ruff check src/ tests/

# Test unit tests
pytest tests/test_etl.py tests/test_integration.py -v

Example: Test PySpark Setup

# Test Java setup
java -version  # Should be Java 17

# Test PySpark installation
python3 -c "import pyspark; print(pyspark.__version__)"

# Test PySpark tests
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
pytest tests/test_etl_spark.py -v

Testing Strategy

What to Test

  1. ✅ Workflow Syntax - YAML is valid
  2. ✅ Job Dependencies - Jobs run in correct order
  3. ✅ Step Execution - Each step completes successfully
  4. ✅ Environment Setup - Python, Java, dependencies install correctly
  5. ✅ Test Execution - Unit tests run and pass
  6. ✅ Linting - Code style checks pass
  7. ⚠️ AWS Integration - Requires AWS credentials (test separately)

What's Hard to Test Locally

  • ❌ AWS Services (S3, Glue, Step Functions) - Require AWS credentials
  • ❌ GitHub API - Some actions need GitHub API access
  • ❌ Secrets Management - Must be provided manually
  • ❌ Matrix Builds - Can be slow locally
flowchart LR A[Write Workflow] --> B[Validate Syntax] B --> C[Test Locally with act] C --> D[Test Individual Steps] D --> E[Push to Test Branch] E --> F[Verify on GitHub] F --> G[Merge to Main]

CI/CD Test Checklist

Before merging CI/CD changes:

  • Workflow YAML syntax is valid
  • All jobs can run locally (with act)
  • Python setup works (Python 3.10)
  • Java setup works (Java 17)
  • Dependencies install correctly
  • Linting passes (ruff, sqlfluff)
  • Unit tests pass (pytest)
  • Workflow runs on test branch
  • No secrets exposed in logs

Troubleshooting

act Issues

Problem: act can't find Docker 

# Solution: Ensure Docker is running
docker info

Problem: act uses wrong image size 

# Solution: Select image size on first run
act push
# Choose: micro, medium, or large

Problem: Workflow fails with "secrets not found" 

# Solution: Provide secrets manually
act push --secret-file .secrets

Workflow Issues

Problem: Tests fail locally but pass on GitHub

  • Check Python version (should be 3.10)
  • Check Java version (should be 17)
  • Verify dependencies match requirements.txt

Problem: Linting fails 

# Run linting manually to see errors
cd tasks/01_data_ingestion_transformation
ruff check src/ tests/

Continuous Improvement

Add Workflow Tests to CI

You can even test your CI/CD workflows in CI! Add a workflow validation step:

# .github/workflows/validate-workflows.yml
name: Validate Workflows
on:
  pull_request:
    paths:

      - '.github/workflows/**'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:

      - uses: actions/checkout@v3
      - name: Validate workflow syntax

        uses: schema-tools/actionlint@v1
        with:
          files: '.github/workflows/*.yml'

Resources

  • act: https://github.com/nektos/act
  • actionlint: https://github.com/rhymond/actionlint
  • GitHub Actions Docs: https://docs.github.com/en/actions

Summary

✅ CI/CD testing is practical and recommended!

  • Use act for local testing
  • Validate syntax with actionlint
  • Test individual steps manually
  • Push to test branch for full integration
  • Test AWS-specific steps separately

Time Investment: ~10 minutes to set up, saves hours of debugging later!