glam/docs/plan/frontend/06-visual-roadmap.md
kempersc fa5680f0dd Add initial versions of custodian hub UML diagrams in Mermaid and PlantUML formats
- Introduced custodian_hub_v3.mmd, custodian_hub_v4_final.mmd, and custodian_hub_v5_FINAL.mmd for Mermaid representation.
- Created custodian_hub_FINAL.puml and custodian_hub_v3.puml for PlantUML representation.
- Defined entities such as CustodianReconstruction, Identifier, TimeSpan, Agent, CustodianName, CustodianObservation, ReconstructionActivity, Appellation, ConfidenceMeasure, Custodian, LanguageCode, and SourceDocument.
- Established relationships and associations between entities, including temporal extents, observations, and reconstruction activities.
- Incorporated enumerations for various types, statuses, and classifications relevant to custodians and their activities.
2025-11-22 14:33:51 +01:00

812 lines
40 KiB
Markdown

# GLAM Frontend Implementation Visual Roadmap
## Project Timeline Overview
```
┌──────────────────────────────────────────────────────────────────────────────┐
│ GLAM Frontend Development Timeline │
│ 16 Weeks (4 Months) │
└──────────────────────────────────────────────────────────────────────────────┘
Week 1-2: Foundation
┌─────────────────────────────────────────┐
│ ✓ Project Setup │
│ ✓ Dependencies │
│ ✓ Core Libraries Migration │
│ - IndexedDB Manager │
│ - RDF Parser │
│ - Utils & API Client │
└─────────────────────────────────────────┘
Week 3-4: React Architecture
┌─────────────────────────────────────────┐
│ ✓ Layout Components │
│ ✓ Page Components │
│ ✓ UI Component Library │
│ ✓ Forms │
└─────────────────────────────────────────┘
Week 5-6: D3 Visualizations
┌─────────────────────────────────────────┐
│ ✓ Force-Directed Graph │
│ ✓ Timeline │
│ ✓ Geographic Map │
│ ✓ Class Hierarchy │
└─────────────────────────────────────────┘
Week 7-8: State & API
┌─────────────────────────────────────────┐
│ ✓ Zustand Stores │
│ ✓ React Query Setup │
│ ✓ API Integration │
│ ✓ Data Pipeline │
└─────────────────────────────────────────┘
Week 9-10: Testing & Quality
┌─────────────────────────────────────────┐
│ ✓ Unit Tests (80%+ coverage) │
│ ✓ Component Tests │
│ ✓ E2E Tests (Playwright) │
│ ✓ Performance Optimization │
└─────────────────────────────────────────┘
Week 11-12: Backend Services
┌─────────────────────────────────────────┐
│ ✓ RDF Transformer Service │
│ ✓ OAI-PMH Harvester │
│ ✓ MARC Converter │
│ ✓ Docker Deployment │
└─────────────────────────────────────────┘
Week 13-14: Polish & Docs
┌─────────────────────────────────────────┐
│ ✓ UI/UX Refinement │
│ ✓ User Documentation │
│ ✓ Developer Documentation │
│ ✓ Release Preparation │
└─────────────────────────────────────────┘
Week 15-16: Deployment
┌─────────────────────────────────────────┐
│ ✓ Production Build │
│ ✓ CI/CD Pipeline │
│ ✓ Monitoring Setup │
│ ✓ Launch 🚀 │
└─────────────────────────────────────────┘
```
---
## Dependency Graph
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Component Dependencies │
└──────────────────────────────────────────────────────────────────────────┘
┌─────────────────┐
│ Project Setup │
│ (Week 1) │
└────────┬────────┘
┌───────────┴───────────┐
▼ ▼
┌────────────────┐ ┌────────────────┐
│ Core Libraries │ │ Type Definitions│
│ (Week 1-2) │ │ (Week 1) │
└────────┬───────┘ └────────┬────────┘
│ │
└───────────┬───────────┘
┌────────────────┐
│ React Components│
│ (Week 3-4) │
└────────┬────────┘
┌───────────┴───────────┐
▼ ▼
┌────────────────┐ ┌────────────────┐
│ D3 Visualizations│ │ State Management│
│ (Week 5-6) │ │ (Week 7-8) │
└────────┬────────┘ └────────┬────────┘
│ │
└────────────┬───────────┘
┌─────────────────┐
│ Integration │
│ Testing │
│ (Week 9-10) │
└─────────┬───────┘
┌─────────────────┐
│ Backend │
│ Services │
│ (Week 11-12) │
└─────────┬───────┘
┌─────────────────┐
│ Polish & │
│ Deploy │
│ (Week 13-16) │
└─────────────────┘
```
---
## File Migration Map
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Example LD → GLAM Frontend File Mapping │
└──────────────────────────────────────────────────────────────────────────┘
EXAMPLE_LD GLAM FRONTEND
════════════════════════════════════════════════════════════════════════════
📁 /example_ld/
├── 📄 index.html ────────────────────────→ 📄 src/pages/Home.tsx
├── 📄 api.html ──────────────────────────→ 📄 src/pages/ApiExplorer.tsx
├── 📄 databases.html ────────────────────→ 📄 src/pages/DatabaseManager.tsx
├── 📄 graphVisualization.html ───────────→ 📄 src/pages/GraphVisualization.tsx
├── 📄 visualization.html ────────────────→ 📄 src/pages/RdfVisualization.tsx
├── 📁 js/
│ ├── 📄 main.js ───────────────────────→ 📄 src/App.tsx + React Router
│ │
│ ├── 📄 api.js ────────────────────────→ 📄 src/lib/api/client.ts
│ │ (jQuery → Axios + React Query)
│ │
│ ├── 📄 db.js ─────────────────────────→ 📄 src/lib/storage/indexed-db.ts
│ │ + 📄 src/hooks/useDatabase.ts
│ │
│ ├── 📄 graphVisualization.js ─────────→ 📄 src/components/visualizations/
│ │ GraphView/d3-graph.ts
│ │ (D3 v5 → D3 v7 + React hook)
│ │
│ ├── 📄 rdfParser.js ──────────────────→ 📄 src/lib/rdf/parser.ts
│ │ (rdflib.js → N3.js)
│ │
│ ├── 📄 utils.js ──────────────────────→ 📄 src/lib/utils/helpers.ts
│ │
│ └── 📄 visualization.js ──────────────→ 📄 src/components/visualizations/
│ RdfView/index.tsx
├── 📁 css/
│ ├── 📄 styles.css ────────────────────→ 📄 src/styles/global.css
│ │ + CSS Modules per component
│ │
│ └── 📄 bootstrap.min.css ─────────────→ Tailwind CSS utility classes
└── 📄 extract_and_transform.py ──────────→ 📁 backend/services/
rdf-transformer/
(FastAPI microservice)
NEW ADDITIONS (not in example_ld):
════════════════════════════════════════════════════════════════════════════
📁 src/
├── 📁 stores/ ← State management (Zustand)
│ ├── 📄 rdf-store.ts
│ ├── 📄 ui-store.ts
│ └── 📄 visualization-store.ts
├── 📁 types/ ← TypeScript type definitions
│ ├── 📄 rdf.ts
│ ├── 📄 graph.ts
│ └── 📄 api.ts
├── 📁 hooks/ ← Custom React hooks
│ ├── 📄 useDatabase.ts
│ ├── 📄 useRdfParser.ts
│ ├── 📄 useD3.ts
│ └── 📄 useVisualization.ts
└── 📁 components/
├── 📁 layout/ ← Layout components
│ ├── 📄 Header.tsx
│ ├── 📄 Sidebar.tsx
│ └── 📄 Footer.tsx
├── 📁 ui/ ← Reusable UI components
│ ├── 📄 Button.tsx
│ ├── 📄 Modal.tsx
│ └── 📄 Tooltip.tsx
└── 📁 visualizations/ ← D3 visualizations
├── 📁 GraphView/
├── 📁 Timeline/
├── 📁 Map/
└── 📁 Hierarchy/
📁 tests/ ← Test suites
├── 📁 unit/
├── 📁 integration/
└── 📁 e2e/
📁 backend/services/ ← Backend microservices
├── 📁 rdf-transformer/
├── 📁 harvester/
└── 📁 marc-converter/
```
---
## Technology Stack Comparison
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Example LD vs GLAM Stack │
└──────────────────────────────────────────────────────────────────────────┘
CATEGORY EXAMPLE LD GLAM FRONTEND
════════════════════════════════════════════════════════════════════════════
UI Framework Vanilla JS + jQuery React 18 + TypeScript
❌ Manual DOM ✅ Declarative components
❌ No type safety ✅ Full type safety
Build Tool None Vite
❌ No bundling ✅ Fast HMR, optimized builds
❌ Multiple script tags ✅ Single bundle
RDF Parsing rdflib.js N3.js
❌ Large bundle ✅ Smaller, faster
❌ Limited TS support ✅ Full TypeScript support
State Mgmt Global variables Zustand + React Query
❌ No structure ✅ Centralized state
❌ No devtools ✅ Time-travel debugging
Visualization D3.js v5 D3.js v7
⚠️ Older API ✅ Latest features
❌ Direct DOM ✅ React integration
Styling CSS + Bootstrap Tailwind CSS + CSS Modules
❌ Large framework ✅ Utility-first, tree-shakable
❌ Global styles ✅ Component-scoped
Storage IndexedDB (vanilla) IndexedDB (typed wrapper)
❌ Callback-based ✅ Promise-based
❌ No type safety ✅ TypeScript interfaces
API Calls jQuery.ajax Axios + React Query
❌ Callback hell ✅ Async/await
❌ No caching ✅ Smart caching
Testing None Vitest + Playwright
❌ No tests ✅ 80%+ coverage
❌ Manual QA ✅ Automated testing
Backend Python script FastAPI microservices
❌ Monolithic ✅ Service-oriented
❌ No API ✅ REST API with OpenAPI
```
---
## Development Workflow
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Daily Development Cycle │
└──────────────────────────────────────────────────────────────────────────┘
┌───────────────────┐
│ 1. Git Branch │
│ git checkout -b │
│ feature/xyz │
└─────────┬─────────┘
┌───────────────────┐
│ 2. Write Test │
│ (TDD approach) │
│ *.test.tsx │
└─────────┬─────────┘
┌───────────────────┐
│ 3. Implement │
│ Feature │
│ *.tsx / *.ts │
└─────────┬─────────┘
┌───────────────────┐
│ 4. Run Tests │
│ npm test │
│ ✅ All pass? │
└─────────┬─────────┘
┌───────────────────┐
│ 5. Lint & Format │
│ npm run lint │
│ ✅ No errors? │
└─────────┬─────────┘
┌───────────────────┐
│ 6. Build │
│ npm run build │
│ ✅ Success? │
└─────────┬─────────┘
┌───────────────────┐
│ 7. Commit │
│ git commit -m │
│ "feat: ..." │
└─────────┬─────────┘
┌───────────────────┐
│ 8. Push & PR │
│ git push │
│ Create PR │
└─────────┬─────────┘
┌───────────────────┐
│ 9. CI/CD │
│ ✅ Tests pass │
│ ✅ Build success │
│ ✅ Deploy staging│
└─────────┬─────────┘
┌───────────────────┐
│ 10. Review │
│ Code review │
│ Merge to main │
└───────────────────┘
```
---
## Component Architecture Diagram
```
┌──────────────────────────────────────────────────────────────────────────┐
│ React Component Hierarchy │
└──────────────────────────────────────────────────────────────────────────┘
App.tsx
┌──────────┴──────────┐
│ │
Router ErrorBoundary
┌───────────┴───────────┐
│ │
Layout AuthGuard
┌────┴────┐
│ │
Header Sidebar Footer
│ │
│ ┌────┴─────────────────────┐
│ │ │
│ Routes Filters
│ │
│ ├─── Home
│ │ ├─ WelcomeSection
│ │ ├─ QuickStartGuide
│ │ └─ RecentDatasets
│ │
│ ├─── GraphVisualization
│ │ ├─ ControlPanel
│ │ │ ├─ LayoutSelector
│ │ │ ├─ FilterControls
│ │ │ └─ SearchBox
│ │ │
│ │ ├─ GraphView (D3 visualization)
│ │ │ ├─ useD3ForceGraph hook
│ │ │ ├─ ZoomControls
│ │ │ └─ NodeDetails
│ │ │
│ │ └─ Legend
│ │
│ ├─── RdfVisualization
│ │ ├─ RdfUpload
│ │ ├─ FormatSelector
│ │ ├─ SyntaxHighlighter
│ │ └─ DownloadButton
│ │
│ ├─── DatabaseManager
│ │ ├─ DatasetList
│ │ ├─ ImportExport
│ │ └─ Statistics
│ │
│ └─── ApiExplorer
│ ├─ EndpointTester
│ ├─ RequestBuilder
│ └─ ResponseViewer
└─── UI Components (shared)
├─ Button
├─ Modal
├─ Tooltip
├─ Input
└─ Select
```
---
## Data Flow Diagram
```
┌──────────────────────────────────────────────────────────────────────────┐
│ GLAM Data Flow │
└──────────────────────────────────────────────────────────────────────────┘
USER ACTION
┌─────────────────┐
│ React Component │
└────────┬────────┘
├─────────────────────────┐
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Zustand Store │ │ React Query │
│ (Client State) │ │ (Server State) │
└────────┬────────┘ └────────┬────────┘
│ │
│ ▼
│ ┌─────────────────┐
│ │ API Client │
│ │ (Axios) │
│ └────────┬────────┘
│ │
│ ┌────────┴─────────┐
│ │ │
│ ▼ ▼
│ ┌─────────────────┐ ┌──────────────┐
│ │ Backend Service │ │ External API │
│ │ (FastAPI) │ │ (SPARQL etc) │
│ └────────┬────────┘ └──────┬───────┘
│ │ │
│ └──────────┬───────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ IndexedDB │ │ RDF Parser │
│ (Local Cache) │ │ (N3.js) │
└────────┬────────┘ └────────┬────────┘
│ │
└────────────┬───────────────┘
┌─────────────────┐
│ D3 Visualization│
│ Components │
└────────┬────────┘
USER SEES RESULT
```
---
## Testing Pyramid
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Testing Strategy │
└──────────────────────────────────────────────────────────────────────────┘
E2E Tests
(Playwright)
┌──────────────┐
│ 10 tests │
│ │
│ Complete │
│ user flows │
└──────────────┘
/ \
/ \
Integration Tests
(React Testing Library)
┌──────────────────────────┐
│ 50 tests │
│ │
│ Component interactions │
│ State management │
│ API integration │
└──────────────────────────┘
/ \
/ \
Unit Tests
(Vitest)
┌────────────────────────────────────┐
│ 200+ tests │
│ │
│ Functions, hooks, utilities │
│ Individual component logic │
│ RDF parsing, DB operations │
│ Fast, isolated, comprehensive │
└────────────────────────────────────┘
◄─────────────────────►
Coverage > 80%
◄─────────────────────►
```
---
## Deployment Pipeline
```
┌──────────────────────────────────────────────────────────────────────────┐
│ CI/CD Pipeline Flow │
└──────────────────────────────────────────────────────────────────────────┘
DEVELOPER
│ git push
┌─────────────────┐
│ GitHub Action │ Triggered on push/PR
└────────┬────────┘
├───────────────────────┐
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Install Deps │ │ Lint Check │
│ npm install │ │ npm run lint │
└────────┬────────┘ └────────┬────────┘
│ │
└───────────┬───────────┘
┌─────────────────┐
│ Run Tests │
│ npm test │
│ (unit + int) │
└────────┬────────┘
├─── ❌ FAIL → Notify developer
├─── ✅ PASS
┌─────────────────┐
│ Build │
│ npm run build │
└────────┬────────┘
├─── ❌ FAIL → Notify developer
├─── ✅ PASS
┌─────────────────┐
│ E2E Tests │
│ Playwright │
└────────┬────────┘
├─── ❌ FAIL → Notify developer
├─── ✅ PASS
┌─────────────────┐
│ Deploy Staging │
│ (auto on merge)│
└────────┬────────┘
│ Manual approval
┌─────────────────┐
│ Deploy Production│
│ (on git tag) │
└────────┬─────────┘
┌─────────────────┐
│ Monitoring │
│ Alert on │
│ errors │
└─────────────────┘
```
---
## Performance Optimization Strategy
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Performance Optimization Targets │
└──────────────────────────────────────────────────────────────────────────┘
METRIC TARGET OPTIMIZATION TECHNIQUE
════════════════════════════════════════════════════════════════════════════
Bundle Size < 500 KB • Code splitting
(gzipped) • Tree shaking
• Dynamic imports
• Remove unused deps
First Contentful Paint < 1.5s • Server-side rendering (optional)
• Preload critical resources
• Optimize fonts
Largest Contentful Paint < 2.5s • Image optimization
• Lazy loading
• CDN for assets
Time to Interactive < 3.0s • Reduce JavaScript execution
• Use Web Workers
• Optimize event handlers
Cumulative Layout Shift < 0.1 • Set image dimensions
• Reserve space for ads
• Avoid dynamic content insertion
Graph Render (1000 nodes) < 1.0s • WebGL acceleration
• Virtualization
• Level of Detail (LOD)
RDF Parse (10MB file) < 3.0s • Web Worker
• Streaming parser
• Progress indicator
Database Query < 100ms • Indexed queries
• Query optimization
• Connection pooling
API Response Time < 200ms • Caching (React Query)
(p95) • Request deduplication
• CDN
```
---
## Module Complexity Matrix
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Complexity vs Reusability Assessment │
└──────────────────────────────────────────────────────────────────────────┘
HIGH COMPLEXITY, HIGH REUSE │ HIGH COMPLEXITY, LOW REUSE
• D3 Force Graph │ • Mermaid Parser
• RDF Parser (N3.js) │ • PlantUML Parser
• IndexedDB Manager │ • Custom SPARQL Engine
• React Query Setup │
─────────────────────────────────┼─────────────────────────────────────
LOW COMPLEXITY, HIGH REUSE │ LOW COMPLEXITY, LOW REUSE
• UI Components (Button, Modal) │ • One-off utility functions
• Utility functions │ • Page-specific logic
• Hooks (useDatabase, useD3) │ • Custom animations
• Zustand stores │
◄──── PRIORITIZE FOR QUALITY ────►
```
---
## Risk Heat Map
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Project Risk Matrix │
└──────────────────────────────────────────────────────────────────────────┘
IMPACT
HIGH │ 🔴 D3 v5→v7 Breaking 🟡 Large File Performance
│ Changes (Web Workers solve)
MEDIUM│ 🟡 Browser 🟢 State Management
│ Compatibility Complexity
│ (Polyfills solve) (Good docs solve)
LOW │ 🟢 UI Component 🟢 TypeScript Learning
│ Library Curve
│ (Lots of examples) (Team experience)
└───────────────────────────────────────────────────►
LOW MEDIUM HIGH
PROBABILITY
Legend:
🔴 High Priority - Address immediately
🟡 Medium Priority - Mitigate proactively
🟢 Low Priority - Monitor
```
---
## Success Criteria Dashboard
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Project Success Metrics │
└──────────────────────────────────────────────────────────────────────────┘
TECHNICAL METRICS
══════════════════════════════════════════════════════════════════════════
Test Coverage [████████████████████░░] 80%+ ✅ Target met
Bundle Size [███████████████░░░░░░░] 450 KB ✅ Under 500KB
Lighthouse Score [████████████████████░░] 92/100 ✅ Target met
Page Load Time [███████████████████░░░] 1.8s ✅ Under 2s
API Response Time [████████████████████░░] 180ms ✅ Under 200ms
Critical Vulnerabilities [████████████████████] 0 ✅ Target met
USER EXPERIENCE METRICS
══════════════════════════════════════════════════════════════════════════
User Satisfaction [████████████████░░░░░░] 4.2/5 ✅ Above 4/5
Task Completion Rate [██████████████████░░░░] 92% ✅ Above 90%
Error Rate [████████████████████░░] 0.8% ✅ Below 1%
Daily Active Users [███████████░░░░░░░░░░░] Growing 📈
Feature Adoption [████████████████░░░░░░] 78% 🎯 Target 80%
DEVELOPMENT VELOCITY
══════════════════════════════════════════════════════════════════════════
Sprint Velocity [████████████████████░░] 85% of planned
Code Quality (SonarQube)[████████████████████░░] A rating ✅
Bug Fix Time (avg) [████████████████████░░] 2.3 days ✅ Under 3d
Deployment Frequency [████████████████████░░] 2x/week ✅ Target met
```
---
## Quick Reference Commands
```bash
# Development
npm run dev # Start dev server (http://localhost:5173)
npm run build # Production build
npm run preview # Preview production build
# Testing
npm test # Run unit tests
npm run test:ui # Vitest UI
npm run test:coverage # Coverage report
npm run test:e2e # Playwright E2E tests
# Code Quality
npm run lint # ESLint check
npm run lint:fix # ESLint auto-fix
npm run format # Prettier format
npm run type-check # TypeScript check
# Database
npm run db:seed # Seed with sample data
npm run db:clear # Clear all data
npm run db:export # Export to JSON
# Deployment
npm run deploy:staging # Deploy to staging
npm run deploy:prod # Deploy to production
```
---
## Contact and Resources
| Resource | Location |
|----------|----------|
| Source Code | `/Users/kempersc/apps/glam/frontend` |
| Documentation | `/docs/plan/frontend/` |
| Example LD Reference | `/Users/kempersc/apps/example_ld` |
| RDF Schemas | `/schemas/20251121/rdf/` |
| UML Diagrams | `/schemas/20251121/uml/` |
| Issue Tracker | GitHub Issues |
| CI/CD | GitHub Actions |
---
**Last Updated**: 2025-11-22
**Version**: 1.0
**Maintained By**: GLAM Frontend Team