767 lines
14 KiB
Markdown
767 lines
14 KiB
Markdown
# Ontology Viewer Enhancements Summary and IaC Deployment Plans
|
|
|
|
**Session ID:** ses_52a66c241ffeFIMKCJm0G9SNjH
|
|
**Created:** 11/30/2025, 5:29:41 PM
|
|
**Updated:** 11/30/2025, 6:20:29 PM
|
|
|
|
---
|
|
|
|
## User
|
|
|
|
Summary of Ontology Viewer Enhancements
|
|
All 5 optional enhancements have been successfully implemented:
|
|
1. ✅ Expanded COMMON_PREFIXES (~25 → ~50 prefixes)
|
|
Added many more namespace prefixes organized by category:
|
|
- W3C Standards: rdf, rdfs, owl, xsd, skos, shacl, sh
|
|
- Dublin Core: dc, dct, dcterms, dctype, dcam
|
|
- General Web: foaf, schema
|
|
- W3C Vocabularies: prov, org, dcat, vcard, time, locn, adms, odrl, sosa, ssn
|
|
- Cultural Heritage: crm, crmdig, frbroo, rico, bf, bflc, madsrdf, lido, edm, ore, svcs
|
|
- Government/Public Sector: cpov, tooi, eli, cpsv
|
|
- Provenance: pav, opmv
|
|
- Geographic: geo, wgs84, gn, geosparql, sf
|
|
- Other: cc, sioc, bibo, as, oa, cnt, wd, wdt, dbpedia, dbo
|
|
2. ✅ Search Functionality (Already Existed)
|
|
Search was already implemented (filtering classes/properties by name, URI, or description).
|
|
3. ✅ Bidirectional Navigation ("Used by" Relationships)
|
|
For classes:
|
|
- Shows Subclasses (classes that extend this one)
|
|
- Shows Properties with this domain (properties that use this class as domain)
|
|
- Shows Properties with this range (properties that use this class as range)
|
|
For properties:
|
|
- Shows Sub-properties (properties that extend this one)
|
|
All "Used by" links are styled with a dashed border to visually distinguish them from forward relationships.
|
|
4. ✅ Export Functionality
|
|
Added a dropdown menu with 3 export formats:
|
|
- Original Format (.ttl, .rdf, .owl, .jsonld) - Downloads the raw source file
|
|
- JSON (Parsed) - Exports a structured JSON with all classes, properties, and metadata
|
|
- CSV (Spreadsheet) - Exports a CSV table with Type, URI, Label, Description, Parent/Domain, Range
|
|
5. ✅ Visual Graph View (Class Hierarchy Tree)
|
|
Added a new "Hierarchy" tab that shows:
|
|
- Collapsible tree view of the class hierarchy
|
|
- Expand All / Collapse All buttons
|
|
- Root class count and total class count
|
|
- Click-to-navigate - clicking a class highlights it and updates the URL
|
|
- Visual distinction for highlighted classes
|
|
The tree properly handles:
|
|
- Root classes (no parent in the ontology)
|
|
- Nested hierarchies (any depth)
|
|
- Cycle prevention
|
|
- Alphabetical sorting
|
|
Files Modified
|
|
1. /frontend/src/pages/OntologyViewerPage.tsx
|
|
- Expanded COMMON_PREFIXES
|
|
- Added getClassUsedBy() and getPropertyUsedBy() functions
|
|
- Added inverse relationship rendering in class/property cards
|
|
- Added handleExport() function and export dropdown
|
|
- Added buildClassTree(), TreeNodeComponent, and renderTreeView()
|
|
- Added new "Hierarchy" tab
|
|
2. /frontend/src/pages/OntologyViewerPage.css
|
|
- Added styles for inverse relationship tags (.ontology-viewer__tag--inverse, .ontology-viewer__tag--property)
|
|
- Added export dropdown styles
|
|
- Added tree view styles (.ontology-viewer__tree-*)
|
|
- Added dark mode support for all new elements Now use IaC to start a Hetzner server on which the Ontologies and LinkML and UML files can be stored in a Oxigraph instance and fed to the frontend from there
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
I'll help you set up Infrastructure as Code (IaC) for a Hetzner server running Oxigraph to serve ontologies and LinkML/UML files. Let me first explore the existing project structure to understand the context better.
|
|
|
|
```
|
|
Tool: task
|
|
```
|
|
|
|
```
|
|
Tool: task
|
|
```
|
|
|
|
---
|
|
|
|
## User
|
|
|
|
Change http://localhost:5174/database too as this still refer to IndexedDB, while we actually need to use Postgres and Oxigraph (or just Oxigraph if this is feasibl
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Let me first examine the current database page and understand what needs to be changed.
|
|
|
|
```
|
|
Tool: glob
|
|
```
|
|
|
|
```
|
|
Tool: grep
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now let me check the existing Oxigraph scripts and the indexed-db storage module to understand the current architecture:
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: glob
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: grep
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Let me also check the frontend vite config and App.tsx to understand the routing:
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: grep
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now I have a clear picture. Let me create a comprehensive Database page that shows Oxigraph status and statistics. I'll create:
|
|
|
|
1. A new `useOxigraph` hook to interact with the Oxigraph SPARQL endpoint
|
|
2. An updated Database page that shows Oxigraph status instead of IndexedDB
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: edit
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now let me create the Infrastructure as Code for Hetzner. I'll create a complete Terraform setup:
|
|
|
|
```
|
|
Tool: list
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: bash
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
---
|
|
|
|
## User
|
|
|
|
continue
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now let's create a deployment script to help sync data to the server:
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: bash
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now let's add `.gitignore` for the Terraform directory and add infrastructure to the main `.gitignore`:
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: edit
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Let me verify the files were created correctly and provide a summary:
|
|
|
|
```
|
|
Tool: bash
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: bash
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
All files have been created. Let me also verify the vite.config.ts was updated correctly:
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
All implementations are complete. Here's a summary of what was created:
|
|
|
|
## Summary of Changes
|
|
|
|
### Frontend Changes (Database Page → Oxigraph)
|
|
|
|
1. **`frontend/src/hooks/useOxigraph.ts`** (NEW)
|
|
- React hook for Oxigraph SPARQL endpoint interaction
|
|
- Functions: `executeSparql`, `loadRdfData`, `clearGraph`, `exportGraph`
|
|
- Auto-fetches statistics: triple count, graphs, classes, predicates, namespaces
|
|
|
|
2. **`frontend/src/pages/Database.tsx`** (REWRITTEN)
|
|
- Now shows Oxigraph triplestore status instead of IndexedDB
|
|
- Tabs: Overview, Graphs, Classes, Predicates, Namespaces, Upload
|
|
- SPARQL query editor with JSON result display
|
|
- RDF file upload (Turtle, N-Triples, RDF/XML, JSON-LD, etc.)
|
|
- Export and clear graph functionality
|
|
|
|
3. **`frontend/src/pages/Database.css`** (REWRITTEN)
|
|
- Complete redesign for the new Oxigraph dashboard
|
|
- Dark mode support
|
|
- Responsive design
|
|
|
|
4. **`frontend/vite.config.ts`** (UPDATED)
|
|
- Added proxy for `/sparql` → `http://127.0.0.1:7878`
|
|
|
|
### Infrastructure as Code (Hetzner + Oxigraph)
|
|
|
|
5. **`infrastructure/README.md`** (NEW)
|
|
- Architecture diagram
|
|
- Setup instructions
|
|
- Cost estimates (~€8.45/month)
|
|
|
|
6. **`infrastructure/terraform/main.tf`** (NEW)
|
|
- Hetzner server (CX21: 2 vCPU, 4GB RAM)
|
|
- Persistent volume (50GB SSD)
|
|
- Firewall rules
|
|
- SSH key management
|
|
|
|
7. **`infrastructure/terraform/variables.tf`** (NEW)
|
|
- Configurable: server type, location, domain, email
|
|
- SSH access restrictions
|
|
- Oxigraph version
|
|
|
|
8. **`infrastructure/terraform/outputs.tf`** (NEW)
|
|
- Server IP, SPARQL endpoints, SSH command
|
|
- DNS record instructions
|
|
- Cost estimation
|
|
|
|
9. **`infrastructure/terraform/cloud-init.yaml`** (NEW)
|
|
- Automated server setup on first boot
|
|
- Installs Caddy (reverse proxy with auto-HTTPS)
|
|
- Installs Oxigraph server
|
|
- Systemd services for auto-start
|
|
- Backup script (daily cron)
|
|
- Load ontologies script
|
|
|
|
10. **`infrastructure/terraform/terraform.tfvars.example`** (NEW)
|
|
- Example configuration
|
|
|
|
11. **`infrastructure/deploy-data.sh`** (NEW)
|
|
- Script to sync ontology/LinkML/UML files to server
|
|
- Loads data into Oxigraph
|
|
|
|
### Usage
|
|
|
|
**Local Development:**
|
|
```bash
|
|
# Start Oxigraph locally
|
|
cd frontend && ./scripts/start-oxigraph.sh
|
|
|
|
# Start frontend dev server
|
|
npm run dev
|
|
|
|
# Visit http://localhost:5173/database
|
|
```
|
|
|
|
**Production Deployment:**
|
|
```bash
|
|
cd infrastructure/terraform
|
|
|
|
# Configure
|
|
cp terraform.tfvars.example terraform.tfvars
|
|
# Edit terraform.tfvars with your values
|
|
|
|
# Deploy
|
|
terraform init
|
|
terraform apply
|
|
|
|
# Sync data
|
|
cd ..
|
|
./deploy-data.sh
|
|
```
|
|
|
|
---
|
|
|
|
## User
|
|
|
|
Please use CD/DI and IaC plus SSH to automatically update the server instance, the .env contains the Hetzner api token for this project under the HETZNER_HC_API_TOKEN key
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
I'll set up CI/CD with GitHub Actions to automatically deploy to the Hetzner server. Let me first check the existing setup and create the necessary configuration.
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: glob
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: bash
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now let me create a local deployment script that can use the Hetzner API token from .env:
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: bash
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now let me update the Terraform configuration to support remote state and add a backend configuration for CI/CD:
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now let me create a setup script for generating SSH keys for CI/CD:
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: bash
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Now let me update the infrastructure README with complete CI/CD documentation:
|
|
|
|
```
|
|
Tool: write
|
|
```
|
|
|
|
---
|
|
|
|
## User
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
## Summary of Work Completed
|
|
|
|
### 1. Database Page Refactoring (IndexedDB → Oxigraph)
|
|
|
|
**Goal**: Change the `/database` page from showing IndexedDB browser storage to showing Oxigraph SPARQL triplestore status.
|
|
|
|
**Files Created/Modified**:
|
|
|
|
| File | Action | Description |
|
|
|------|--------|-------------|
|
|
| `frontend/src/hooks/useOxigraph.ts` | **NEW** | React hook for Oxigraph SPARQL endpoint interaction - provides `executeSparql`, `loadRdfData`, `clearGraph`, `exportGraph` functions and auto-fetches statistics |
|
|
| `frontend/src/pages/Database.tsx` | **REWRITTEN** | Complete redesign - now shows Oxigraph connection status, triple counts, graphs, classes, predicates, namespaces tabs, SPARQL query editor, RDF upload |
|
|
| `frontend/src/pages/Database.css` | **REWRITTEN** | New styles for Oxigraph dashboard with dark mode support |
|
|
| `frontend/vite.config.ts` | **MODIFIED** | Added proxy: `/sparql` → `http://127.0.0.1:7878` |
|
|
|
|
### 2. Infrastructure as Code (Hetzner + Oxigraph)
|
|
|
|
**Goal**: Create Terraform IaC to deploy Oxigraph on Hetzner Cloud server.
|
|
|
|
**Files Created**:
|
|
|
|
| File | Description |
|
|
|------|-------------|
|
|
| `infrastructure/README.md` | Architecture diagram, setup instructions, cost estimates |
|
|
| `infrastructure/terraform/main.tf` | Hetzner server (CX21), volume (50GB), firewall, SSH key |
|
|
| `infrastructure/terraform/variables.tf` | Configurable inputs (server type, domain, etc.) |
|
|
| `infrastructure/terraform/outputs.tf` | Server IP, SPARQL endpoints, cost estimation |
|
|
| `infrastructure/terraform/cloud-init.yaml` | Server bootstrap: installs Caddy + Oxigraph, systemd services, backup cron |
|
|
| `infrastructure/terraform/terraform.tfvars.example` | Example configuration |
|
|
| `infrastructure/terraform/.gitignore` | Ignore tfstate and secrets |
|
|
| `infrastructure/terraform/backend.tf.example` | Remote state options |
|
|
| `infrastructure/deploy-data.sh` | Script to sync data to server |
|
|
|
|
### 3. CI/CD with GitHub Actions
|
|
|
|
**Goal**: Automatic deployments using GitHub Actions + SSH + Hetzner API.
|
|
|
|
**Files Created**:
|
|
|
|
| File | Description |
|
|
|------|-------------|
|
|
| `.github/workflows/deploy.yml` | **NEW** - Full CI/CD pipeline that triggers on pushes to `main` for infra/data/frontend changes |
|
|
| `infrastructure/deploy.sh` | **NEW** - Local deployment script using Hetzner API from `.env` |
|
|
| `infrastructure/setup-cicd.sh` | **NEW** - Setup helper that generates SSH keys and prints GitHub secrets instructions |
|
|
|
|
**CI/CD Features**:
|
|
- Detects changes in `infrastructure/`, `schemas/`, `data/ontology/`, `frontend/`
|
|
- Runs Terraform for infra changes
|
|
- Builds frontend with npm
|
|
- Syncs data via rsync over SSH
|
|
- Manual trigger option with reload Oxigraph checkbox
|
|
- Uses `HETZNER_HC_API_TOKEN` from GitHub secrets (same key as in `.env`)
|
|
|
|
### 4. Updated .gitignore
|
|
|
|
Added entries for:
|
|
- `infrastructure/terraform/terraform.tfvars`
|
|
- `infrastructure/terraform/*.tfstate*`
|
|
- `infrastructure/terraform/.terraform/`
|
|
|
|
---
|
|
|
|
## What Was Being Done When Interrupted
|
|
|
|
I was about to **update the infrastructure README** with complete CI/CD documentation but got an error because I needed to read the file first before overwriting.
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
1. **Read and update** `infrastructure/README.md` with full CI/CD documentation
|
|
2. **Test the setup** - Run `./infrastructure/setup-cicd.sh` to generate SSH keys
|
|
3. **Verify frontend compiles** with the new `useOxigraph` hook
|
|
4. **Optional**: Add environment variable `VITE_OXIGRAPH_URL` to support production URLs
|
|
|
|
---
|
|
|
|
## Key Environment Variables
|
|
|
|
The project expects these in `.env`:
|
|
```
|
|
HETZNER_HC_API_TOKEN=xxx # Hetzner Cloud API token
|
|
GLAM_DOMAIN=sparql.glam-ontology.org # Domain for SPARQL endpoint
|
|
ADMIN_EMAIL=admin@example.org # For Let's Encrypt
|
|
```
|
|
|
|
---
|
|
|
|
## User
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
I'll continue with the next steps. Let me first read the infrastructure README to update it with CI/CD documentation, and check the frontend compiles.
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: bash
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Frontend builds successfully. Now let me update the infrastructure README with CI/CD documentation:
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
```
|
|
Tool: read
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
Both the CI/CD workflow and setup script look complete. Now let me update the infrastructure README with CI/CD documentation:
|
|
|
|
```
|
|
Tool: edit
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
```
|
|
Tool: todowrite
|
|
```
|
|
|
|
---
|
|
|
|
## Assistant
|
|
|
|
---
|
|
|