kempersc
caa2690ba4
ci: run all 4 layers on every push
...
Deploy Frontend / build-and-deploy (push) Successful in 3m54s
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Successful in 10m11s
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Successful in 12m4s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Successful in 10m4s
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Successful in 11m7s
DSPy RAG Evaluation / Quality Gate (push) Successful in 1s
Removed conditional execution from Layer 2 and Layer 4.
All layers now run on every push, PR, and schedule.
Quality Gate requires all 4 layers to pass.
2026-01-13 14:05:41 +01:00
kempersc
70a85bd177
fix(ci): use upload-artifact@v3 for Forgejo compatibility
...
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Successful in 10m45s
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Successful in 10m15s
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Successful in 2s
upload-artifact@v4 is not supported on GHES/Forgejo.
All 35 unit tests passed, but job failed due to artifact upload.
Downgrading to v3 which is compatible with self-hosted runners.
2026-01-13 13:28:11 +01:00
kempersc
44061eb736
fix(ci): add pytest-cov to resolve coverage flag error
...
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 10m35s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 1s
pyproject.toml has --cov flags in addopts which require pytest-cov.
Added pytest-cov to all jobs that run pytest.
2026-01-13 13:13:56 +01:00
kempersc
38db05656a
fix(ci): use Python virtual environment for PEP 668 compliance
...
Deploy Frontend / build-and-deploy (push) Successful in 4m2s
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 9m53s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 1s
- Create venv at /opt/venv in each job
- Source venv/bin/activate before pip install and pytest commands
- Add python3-full package for complete venv support
- Fixes 'externally-managed-environment' error on Debian Bookworm
2026-01-13 12:54:41 +01:00
kempersc
0374cfe622
fix(ci): remove container block from dspy-eval, install Python via apt
...
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 5m25s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 2s
The Forgejo runner with label ubuntu-latest:docker://node:20-bookworm
does not properly support custom container overrides. Instead of using
container: image: python:3.11-slim, we now install Python from apt-get
in the node:20-bookworm base container (which is Debian-based).
Changes:
- Remove container: blocks from all 4 layer jobs
- Add 'Install Python' step to each job
- Use python3/python3 -m pip/python3 -m pytest commands
- Remove trigger comment from test file
2026-01-12 19:21:03 +01:00
kempersc
f2b2481272
chore: trigger dspy-eval workflow (touch workflow file)
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 5s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 2s
2026-01-12 11:17:49 +01:00
kempersc
02d5b61e40
fix(ci): use Python container image instead of setup-python action
...
Deploy Frontend / build-and-deploy (push) Successful in 4m3s
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 12s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 1s
- Switch all jobs to use python:3.11-slim container
- Remove setup-python action (not cached on Forgejo runner)
- Add apt-get install for openssh-client and curl in SSH tunnel jobs
- Increased timeout for unit-tests to 10 minutes
- Remove unused PYTHON_VERSION env var
2026-01-11 22:28:49 +01:00
kempersc
8470bf5860
feat(ci): add DSPy RAG evaluation workflow for Forgejo
...
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 6m24s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 1s
Implements 4-layer testing pyramid:
- Layer 1: Fast unit tests (no LLM, ~5 min)
- Layer 2: DSPy module tests with LLM (~20 min)
- Layer 3: Integration tests via SSH tunnel to Oxigraph
- Layer 4: Comprehensive evaluation (nightly)
Includes:
- SSH tunnel setup for Oxigraph access
- Quality gate checks
- JUnit XML output for test results
- Scheduled nightly runs at 2 AM UTC
- Manual trigger with evaluation level selection
2026-01-11 21:19:40 +01:00
