kempersc/glam
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
glam/docs/DEPLOYMENT_GUIDE.md
kempersc 097d116b72 enrich entries
2025-12-01 16:06:34 +01:00

9.9 KiB
Raw Blame History

GLAM Deployment Guide

Complete guide for deploying the GLAM (Galleries, Libraries, Archives, Museums) ontology project.

Overview

This project uses a local deployment model - all deployments are executed from your local machine via SSH and rsync. There is NO CI/CD pipeline or GitHub Actions.

Why Local Deployment?

  • Full control over what gets deployed
  • No exposure of secrets to third-party CI systems
  • Immediate feedback during deployment
  • Easy debugging and rollback

Infrastructure

Server Details

Property Value
Provider Hetzner Cloud
Server Name glam-sparql
IP Address 91.98.224.44
SSH User root
Frontend Path /var/www/glam-frontend/
Data Path /mnt/data/
Oxigraph Data /var/lib/oxigraph/

Services Running on Server

Service Port Description
Caddy 80, 443 Reverse proxy with automatic HTTPS
Oxigraph 7878 SPARQL triplestore (internal only)
Frontend - Static files served by Caddy

Prerequisites

1. Environment Variables

Create a .env file in the project root:

# Required for deployment
HETZNER_HC_API_TOKEN=your_hetzner_cloud_api_token

# Optional - defaults shown
GLAM_DOMAIN=sparql.glam-ontology.org
ADMIN_EMAIL=admin@glam-ontology.org

Getting your Hetzner API Token:

  1. Log in to Hetzner Cloud Console
  2. Select your project
  3. Go to Security > API Tokens
  4. Create a new token with Read & Write permissions
  5. Copy the token to your .env file

2. SSH Access

Ensure you can SSH to the server:

# Test SSH connection
ssh root@91.98.224.44 "echo 'Connection successful'"

If this fails:

  • Check that your SSH key is added to the server's ~/.ssh/authorized_keys
  • Verify the server is running (./infrastructure/deploy.sh --status)

3. Local Dependencies

For frontend deployment:

# Ensure Node.js is installed (v18+)
node --version

# Ensure npm is installed
npm --version

For infrastructure changes:

# Ensure Terraform is installed
terraform --version

# Ensure jq is installed (for JSON parsing)
jq --version

Deployment Script

All deployments use the script at infrastructure/deploy.sh.

Command Options

Option Description
--frontend Build and deploy React frontend
--data Deploy ontology/schema data files
--infra Deploy infrastructure via Terraform
--reload Reload data into Oxigraph triplestore
--all Deploy everything
--status Check server and service status

Quick Reference

# Most common - deploy frontend changes
./infrastructure/deploy.sh --frontend

# Deploy updated ontology/schema files
./infrastructure/deploy.sh --data

# Check what's running on the server
./infrastructure/deploy.sh --status

# Full deployment (rare)
./infrastructure/deploy.sh --all

Deployment Scenarios

Scenario 1: Frontend Changes (Most Common)

After making changes to frontend/:

# Option A: Use deploy script (recommended)
./infrastructure/deploy.sh --frontend

# Option B: Manual build + deploy
cd frontend
npm run build
cd ..
rsync -avz --delete frontend/dist/ root@91.98.224.44:/var/www/glam-frontend/

What the script does:

  1. Runs npm ci to install exact dependencies
  2. Runs npm run build to create production build
  3. Syncs frontend/dist/ to server via rsync

Verification:

# Check the server is serving updated content
curl -I https://91.98.224.44

Scenario 2: Schema/Ontology Updates

After regenerating LinkML schemas or RDF files:

./infrastructure/deploy.sh --data

What gets synced:

  • data/ontology/*.{ttl,rdf,owl,jsonld} → /mnt/data/ontologies/
  • schemas/20251121/rdf/ → /mnt/data/rdf/
  • schemas/20251121/linkml/*.yaml → /mnt/data/linkml/
  • schemas/20251121/uml/mermaid/*.mmd → /mnt/data/uml/

To also reload into Oxigraph:

./infrastructure/deploy.sh --data --reload

Scenario 3: Infrastructure Changes

For changes to server configuration (rare):

./infrastructure/deploy.sh --infra

Warning: This will:

  1. Run terraform plan to show changes
  2. Ask for confirmation before applying
  3. Potentially recreate the server (be careful!)

Scenario 4: Check Server Status

./infrastructure/deploy.sh --status

Output includes:

  • Server name, status, IP, type, location
  • Oxigraph service status
  • Caddy service status
  • Triple count in SPARQL store

Manual Deployment Steps

If the script fails or you need manual control:

Frontend (Manual)

# 1. Build frontend
cd frontend
npm ci
npm run build

# 2. Sync schemas to frontend public directory (if needed)
mkdir -p public/schemas
cp -r ../schemas/20251121/linkml public/schemas/
cp -r ../schemas/20251121/uml public/schemas/

# 3. Deploy to server
rsync -avz --delete dist/ root@91.98.224.44:/var/www/glam-frontend/

# 4. Verify
ssh root@91.98.224.44 "ls -la /var/www/glam-frontend/"

Data Files (Manual)

# Sync ontologies
rsync -avz data/ontology/*.ttl data/ontology/*.rdf data/ontology/*.owl \
  root@91.98.224.44:/mnt/data/ontologies/

# Sync RDF schemas
rsync -avz schemas/20251121/rdf/ \
  root@91.98.224.44:/mnt/data/rdf/

# Sync LinkML schemas
rsync -avz --include='*.yaml' --include='*/' --exclude='*' \
  schemas/20251121/linkml/ \
  root@91.98.224.44:/mnt/data/linkml/

Reload Oxigraph (Manual)

ssh root@91.98.224.44 "/var/lib/glam/scripts/load-ontologies.sh"

Server Management

SSH Access

# Connect to server
ssh root@91.98.224.44

# Check disk usage
ssh root@91.98.224.44 "df -h"

# Check memory usage
ssh root@91.98.224.44 "free -h"

# View logs
ssh root@91.98.224.44 "journalctl -u caddy -n 50"
ssh root@91.98.224.44 "journalctl -u oxigraph -n 50"

Service Management

# Restart Caddy (web server)
ssh root@91.98.224.44 "systemctl restart caddy"

# Restart Oxigraph (SPARQL store)
ssh root@91.98.224.44 "systemctl restart oxigraph"

# Check service status
ssh root@91.98.224.44 "systemctl status caddy"
ssh root@91.98.224.44 "systemctl status oxigraph"

Caddy Configuration

Caddy config is at /etc/caddy/Caddyfile on the server:

# View current config
ssh root@91.98.224.44 "cat /etc/caddy/Caddyfile"

# Edit config (if needed)
ssh root@91.98.224.44 "nano /etc/caddy/Caddyfile"

# Reload after changes
ssh root@91.98.224.44 "systemctl reload caddy"

Troubleshooting

Common Issues

1. "Permission denied" on SSH

# Check SSH key is loaded
ssh-add -l

# Add SSH key if not loaded
ssh-add ~/.ssh/id_ed25519  # or your key file

2. "Server not found" error

# Check server exists in Hetzner
./infrastructure/deploy.sh --status

# If server doesn't exist, create with:
./infrastructure/deploy.sh --infra

3. Frontend not updating

# Force cache clear on browser (Ctrl+Shift+R)

# Or check files were actually deployed
ssh root@91.98.224.44 "ls -la /var/www/glam-frontend/"
ssh root@91.98.224.44 "cat /var/www/glam-frontend/index.html | head -20"

4. SPARQL queries failing

# Check Oxigraph is running
ssh root@91.98.224.44 "systemctl status oxigraph"

# Check triple count
ssh root@91.98.224.44 "curl -s -X POST \
  -H 'Content-Type: application/sparql-query' \
  -H 'Accept: application/sparql-results+json' \
  --data 'SELECT (COUNT(*) AS ?count) WHERE { ?s ?p ?o }' \
  http://localhost:7878/query | jq '.results.bindings[0].count.value'"

# Restart Oxigraph if needed
ssh root@91.98.224.44 "systemctl restart oxigraph"

5. Hetzner API Token Invalid

# Test token manually
curl -s -H "Authorization: Bearer YOUR_TOKEN" \
  "https://api.hetzner.cloud/v1/servers" | jq '.servers[].name'

Log Files

# Caddy access logs
ssh root@91.98.224.44 "journalctl -u caddy --since '1 hour ago'"

# Oxigraph logs
ssh root@91.98.224.44 "journalctl -u oxigraph --since '1 hour ago'"

# System logs
ssh root@91.98.224.44 "journalctl --since '1 hour ago'"

Security Notes

What's Protected

  • HTTPS enabled via Caddy with automatic Let's Encrypt certificates
  • SSH access restricted to key-based authentication
  • Oxigraph only accessible from localhost (not exposed to internet)
  • Caddy reverse proxies SPARQL queries to Oxigraph

Sensitive Files

NEVER commit these to git:

  • .env - Contains Hetzner API token
  • infrastructure/terraform/terraform.tfvars - Contains secrets
  • infrastructure/terraform/*.tfstate* - Contains infrastructure state

These are already in .gitignore.

Deployment Checklist

Before deploying:

  • Changes tested locally
  • .env file exists with valid HETZNER_HC_API_TOKEN
  • SSH access verified (ssh root@91.98.224.44)
  • For frontend: npm run build succeeds locally
  • For schemas: LinkML validation passes

After deploying:

  • ./infrastructure/deploy.sh --status shows services running
  • Frontend loads in browser
  • SPARQL queries return results (if data deployed)
  • No errors in server logs

Quick Commands Reference

# Deploy frontend
./infrastructure/deploy.sh --frontend

# Deploy data
./infrastructure/deploy.sh --data

# Check status
./infrastructure/deploy.sh --status

# SSH to server
ssh root@91.98.224.44

# View Caddy logs
ssh root@91.98.224.44 "journalctl -u caddy -f"

# Restart services
ssh root@91.98.224.44 "systemctl restart caddy"
ssh root@91.98.224.44 "systemctl restart oxigraph"

Related Documentation

  • .opencode/DEPLOYMENT_RULES.md - Rules for AI agents performing deployments
  • AGENTS.md - Rule 7 covers deployment guidelines
  • infrastructure/deploy.sh - The deployment script source code
  • infrastructure/terraform/ - Terraform configuration for server provisioning

Last Updated: 2025-12-01 Maintainer: GLAM Project Team