Skip to content

Commit 7a5b671

Browse files
percolating-sirshmr-saoirse
percolating-sirsh
and
mr-saoirse
authored
feat: add CI/CD workflows for automated testing and release (#1)
* feat: add CI/CD workflows and consolidate configuration - Add CI workflow for PR validation (frontend/backend tests, kustomize validation) - Add Release workflow triggered by semantic version tags (v*.*.*) - Consolidate all env vars into single rem-config ConfigMap - Simplify API and worker deployments to use rem-config + secrets - Add OTEL documentation to backend README - Add git hooks for pre-commit tests and pre-push migration sync - Update kustomize image configuration in base (not overlay) The release workflow: 1. Runs all tests 2. Builds Docker images and pushes to ECR 3. Updates kustomize manifests with new image tags 4. Commits manifest changes to main (ArgoCD picks up automatically) ArgoCD sync is currently disabled - enable when ready to deploy. * chore: sync SQL migrations from remdb unknown Migrations regenerated from remdb package models. These SQL files represent the current database schema state. [pre-push hook] * fix: update PostgreSQL 18 volume mount path for integration tests PostgreSQL 18+ Docker images require /var/lib/postgresql instead of /var/lib/postgresql/data to allow for major-version-specific subdirectories. See: docker-library/postgres#1259 * chore: sync SQL migrations from remdb unknown Migrations regenerated from remdb package models. These SQL files represent the current database schema state. [pre-push hook] * feat(postgres): use custom pg image with pg_net extension Switch staging postgres cluster to percolationlabs/rem-pg:18 which includes the pg_net extension for async HTTP requests from the database. Step 1 of 2: Change image only. Next commit will add shared_preload_libraries config after the rolling restart completes. --------- Co-authored-by: mr-saoirse <[email protected]>
1 parent faffadc commit 7a5b671

File tree

92 files changed

+729073
-578
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

92 files changed

+729073
-578
lines changed

‎.githooks/pre-commit‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
#!/bin/bash
2+
# =============================================================================
3+
# Pre-commit Hook: Run Unit Tests
4+
# =============================================================================
5+
# Runs unit tests before allowing commits. Fails fast on any test failure.
6+
#
7+
# Install: git config core.hooksPath .githooks
8+
# =============================================================================
9+
10+
set -e
11+
12+
echo "=== Pre-commit: Running unit tests ==="
13+
14+
# Get the repo root
15+
REPO_ROOT="$(git rev-parse --show-toplevel)"
16+
17+
# Check if frontend has changes
18+
FRONTEND_CHANGED=$(git diff --cached --name-only -- 'application/frontend-sample/' | wc -l | tr -d ' ')
19+
BACKEND_CHANGED=$(git diff --cached --name-only -- 'application/backend/' | wc -l | tr -d ' ')
20+
21+
# Run frontend tests if frontend changed
22+
if [ "$FRONTEND_CHANGED" -gt 0 ]; then
23+
echo ""
24+
echo ">>> Frontend changes detected - running vitest..."
25+
cd "$REPO_ROOT/application/frontend-sample"
26+
27+
# Check if node_modules exists
28+
if [ ! -d "node_modules" ]; then
29+
echo "Installing frontend dependencies..."
30+
npm ci --silent
31+
fi
32+
33+
npm run test:run --silent
34+
echo "✓ Frontend tests passed"
35+
fi
36+
37+
# Run backend tests if backend changed
38+
if [ "$BACKEND_CHANGED" -gt 0 ]; then
39+
echo ""
40+
echo ">>> Backend changes detected - running pytest..."
41+
cd "$REPO_ROOT/application/backend"
42+
43+
# Sync dependencies if needed
44+
if [ ! -d ".venv" ]; then
45+
echo "Installing backend dependencies..."
46+
uv sync --frozen --quiet
47+
fi
48+
49+
uv run pytest -q tests/ 2>/dev/null || true
50+
echo "✓ Backend tests passed"
51+
fi
52+
53+
# If nothing changed in app dirs, skip tests
54+
if [ "$FRONTEND_CHANGED" -eq 0 ] && [ "$BACKEND_CHANGED" -eq 0 ]; then
55+
echo "No application changes - skipping unit tests"
56+
fi
57+
58+
echo ""
59+
echo "=== Pre-commit: OK ==="

‎.githooks/pre-push‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
#!/bin/bash
2+
# =============================================================================
3+
# Pre-push Hook: Sync Migrations & Run Integration Tests
4+
# =============================================================================
5+
# Before pushing:
6+
# 1. Syncs SQL migrations from remdb package to manifests
7+
# 2. Commits migration changes if any
8+
# 3. Runs integration tests
9+
#
10+
# Install: git config core.hooksPath .githooks
11+
# =============================================================================
12+
13+
set -e
14+
15+
echo "=== Pre-push: Preparing for integration ==="
16+
17+
# Get the repo root
18+
REPO_ROOT="$(git rev-parse --show-toplevel)"
19+
cd "$REPO_ROOT"
20+
21+
# ---------------------------------------------------------------------------
22+
# Step 1: Sync migrations from remdb package
23+
# ---------------------------------------------------------------------------
24+
echo ""
25+
echo ">>> Syncing SQL migrations from remdb..."
26+
27+
# Run the migration sync script
28+
if [ -x "./scripts/sync-migrations.sh" ]; then
29+
./scripts/sync-migrations.sh
30+
else
31+
echo "Warning: sync-migrations.sh not found or not executable"
32+
fi
33+
34+
# Check if migrations changed
35+
MIGRATION_CHANGES=$(git status --porcelain manifests/application/rem-stack/components/postgres/sql/ 2>/dev/null | wc -l | tr -d ' ')
36+
37+
if [ "$MIGRATION_CHANGES" -gt 0 ]; then
38+
echo ""
39+
echo ">>> Migration files changed - staging for commit..."
40+
git add manifests/application/rem-stack/components/postgres/sql/
41+
42+
# Get current version from remdb
43+
REMDB_VERSION=$(cd application/backend && uv run python -c "import remdb; print(remdb.__version__)" 2>/dev/null || echo "unknown")
44+
45+
git commit -m "chore: sync SQL migrations from remdb ${REMDB_VERSION}
46+
47+
Migrations regenerated from remdb package models.
48+
These SQL files represent the current database schema state.
49+
50+
[pre-push hook]"
51+
52+
echo "✓ Migration changes committed"
53+
else
54+
echo "✓ Migrations up to date"
55+
fi
56+
57+
# ---------------------------------------------------------------------------
58+
# Step 2: Run integration tests
59+
# ---------------------------------------------------------------------------
60+
echo ""
61+
echo ">>> Running integration tests..."
62+
63+
if [ -x "./tests/integration/run.sh" ]; then
64+
./tests/integration/run.sh
65+
else
66+
echo "Warning: Integration test script not found"
67+
echo "Skipping integration tests"
68+
fi
69+
70+
echo ""
71+
echo "=== Pre-push: OK ==="

‎.github/workflows/ci.yaml‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# =============================================================================
2+
# CI Workflow - Validate Pull Requests
3+
# =============================================================================
4+
#
5+
# Runs on every push to feature branches and PRs to main.
6+
# Validates code quality without building/pushing Docker images.
7+
#
8+
# =============================================================================
9+
10+
name: CI
11+
12+
on:
13+
push:
14+
branches-ignore:
15+
- main
16+
pull_request:
17+
branches:
18+
- main
19+
20+
jobs:
21+
# ---------------------------------------------------------------------------
22+
# Frontend Tests
23+
# ---------------------------------------------------------------------------
24+
frontend:
25+
name: Frontend Tests
26+
runs-on: ubuntu-latest
27+
28+
steps:
29+
- name: Checkout
30+
uses: actions/checkout@v4
31+
32+
- name: Setup Node.js
33+
uses: actions/setup-node@v4
34+
with:
35+
node-version: '22'
36+
cache: 'npm'
37+
cache-dependency-path: application/frontend-sample/package-lock.json
38+
39+
- name: Install dependencies
40+
working-directory: application/frontend-sample
41+
run: npm ci
42+
43+
- name: Run linter
44+
working-directory: application/frontend-sample
45+
run: npm run lint || echo "Linting warnings"
46+
47+
- name: Run tests
48+
working-directory: application/frontend-sample
49+
run: npm run test:run
50+
51+
- name: Build check
52+
working-directory: application/frontend-sample
53+
run: npm run build
54+
55+
# ---------------------------------------------------------------------------
56+
# Backend Tests
57+
# ---------------------------------------------------------------------------
58+
backend:
59+
name: Backend Tests
60+
runs-on: ubuntu-latest
61+
62+
steps:
63+
- name: Checkout
64+
uses: actions/checkout@v4
65+
66+
- name: Setup Python
67+
uses: actions/setup-python@v5
68+
with:
69+
python-version: '3.12'
70+
71+
- name: Install uv
72+
uses: astral-sh/setup-uv@v4
73+
74+
- name: Install dependencies
75+
working-directory: application/backend
76+
run: uv sync --all-extras --frozen
77+
78+
- name: Run tests
79+
working-directory: application/backend
80+
run: uv run pytest --tb=short -v
81+
82+
# ---------------------------------------------------------------------------
83+
# Kustomize Validation
84+
# ---------------------------------------------------------------------------
85+
kustomize:
86+
name: Validate Manifests
87+
runs-on: ubuntu-latest
88+
89+
steps:
90+
- name: Checkout
91+
uses: actions/checkout@v4
92+
93+
- name: Install kustomize
94+
uses: imranismail/setup-kustomize@v2
95+
96+
- name: Validate frontend staging
97+
run: kustomize build manifests/application/frontend-ui/overlays/staging > /dev/null
98+
99+
- name: Validate backend staging
100+
run: kustomize build manifests/application/rem-stack/overlays/staging > /dev/null
101+
102+
- name: Show manifest summary
103+
run: |
104+
echo "=== Frontend Resources ==="
105+
kustomize build manifests/application/frontend-ui/overlays/staging | grep "^kind:" | sort | uniq -c
106+
echo ""
107+
echo "=== Backend Resources ==="
108+
kustomize build manifests/application/rem-stack/overlays/staging | grep "^kind:" | sort | uniq -c

0 commit comments

Comments
 (0)