glam/COMPLETE_SCHEMA_MERMAID_GENERATION.md

Complete Schema Mermaid Diagram Generation

Date: 2025-11-24
Feature: Extended LinkML's MermaidRenderer to generate complete schema diagrams

---

Overview

LinkML's built-in MermaidClassDiagramGenerator generates individual diagrams per class (53 separate .mmd files). While this is useful for detailed class-by-class documentation, it doesn't provide a holistic view of the entire schema.

We've created a custom script that generates a single comprehensive Mermaid diagram showing:

• ✅ All 53 classes
• ✅ All 149 relationships (inheritance, associations)
• ✅ Key attributes for each class (limited to 10 per class for readability)
• ✅ Abstract classes marked with <<abstract>>
• ✅ Relationship cardinalities (1, 1..*)

---

Usage

Generate Complete Diagram

cd /Users/kempersc/apps/glam
python3 scripts/generate_complete_mermaid_diagram.py

Output:

schemas/20251121/uml/mermaid/complete_schema_YYYYMMDD_HHMMSS.mmd

View the Diagram

Option 1: Mermaid Live Editor

1. Open https://mermaid.live/
2. Copy contents of complete_schema_YYYYMMDD_HHMMSS.mmd
3. Paste into editor
4. Explore interactively (zoom, pan, export to SVG/PNG)

Option 2: VS Code (with Mermaid extension)

1. Install "Markdown Preview Mermaid Support" extension
2. Open .mmd file in VS Code
3. Right-click → "Open Preview"

Option 3: GitHub (automatic rendering)

1. Push .mmd file to GitHub repository
2. View file on GitHub (automatic Mermaid rendering)

---

Generated Diagram Statistics

Latest Generation (2025-11-24 00:41:43):

• Classes: 53
• Relationships: 149
  • Inheritance (<|--): 42
  • Associations (-->): 107
• Lines: 697
• File Size: 31 KB

---

Key Relationships Captured

EncompassingBody Hierarchy

classDiagram
  class EncompassingBody <<abstract>>
  EncompassingBody <|-- UmbrellaOrganisation : inherits
  EncompassingBody <|-- NetworkOrganisation : inherits
  EncompassingBody <|-- Consortium : inherits
  
  Custodian --> EncompassingBody : encompassing_body

CustodianType Hierarchy (19 types)

classDiagram
  class CustodianType
  CustodianType <|-- MuseumType
  CustodianType <|-- LibraryType
  CustodianType <|-- ArchiveOrganizationType
  CustodianType <|-- GalleryType
  CustodianType <|-- OfficialInstitutionType
  CustodianType <|-- ResearchOrganizationType
  CustodianType <|-- CommercialOrganizationType
  CustodianType <|-- UnspecifiedType
  CustodianType <|-- BioCustodianType
  CustodianType <|-- EducationalProviderType
  CustodianType <|-- CollectingSocietyType
  CustodianType <|-- FeaturePlaceType
  CustodianType <|-- IntangibleHeritageType
  CustodianType <|-- MixedCustodianType
  CustodianType <|-- PersonalCollectionType
  CustodianType <|-- HolySiteType
  CustodianType <|-- DigitalPlatformType
  CustodianType <|-- NonProfitType
  CustodianType <|-- TasteScentHeritageType

Hub Architecture (Custodian as central reference)

classDiagram
  class Custodian
  CustodianObservation --> Custodian : identifies_custodian
  CustodianLegalStatus --> Custodian : refers_to_custodian
  CustodianName --> Custodian : refers_to_custodian
  CustodianPlace --> Custodian : refers_to_custodian
  CustodianCollection --> Custodian : refers_to_custodian
  OrganizationalStructure --> Custodian : refers_to_custodian

---

Script Details

Location: scripts/generate_complete_mermaid_diagram.py

Key Features:

1. Two-pass generation:

   • Pass 1: Define all classes with attributes
   • Pass 2: Define all relationships (inheritance + associations)

2. Smart filtering:

   • Limits attributes to 10 per class (prevents diagram explosion)
   • Only includes class-to-class relationships (skips primitive types)

3. Metadata embedded:

   • Schema name and version
   • Generation timestamp
   • Class/relationship counts

4. Timestamped output:

   • Format: complete_schema_YYYYMMDD_HHMMSS.mmd
   • Allows multiple generations without conflicts

---

Comparison: Per-Class vs Complete Diagrams

Feature	Per-Class Diagrams (gen-yuml)	Complete Diagram (this script)
Files generated	53 (one per class)	1 (entire schema)
Total size	212 KB (53 files)	31 KB (1 file)
Use case	Detailed class documentation	Schema overview and architecture
Relationships shown	Immediate class relationships	All schema relationships
Holistic view	❌ Requires viewing 53 files	✅ Single comprehensive view
Detail level	High (all slots, all docs)	Medium (10 slots per class)
Maintainability	Auto-generated by LinkML	Custom script (easy to extend)

Recommendation: Use both:

• Per-class diagrams: Detailed reference for developers
• Complete diagram: Architecture overview for onboarding, presentations, ontology consultations

---

Extending the Script

Add More Attributes Per Class

Edit scripts/generate_complete_mermaid_diagram.py line 72:

for slot_name in class_slots[:10]:  # Change 10 to 20, 30, etc.

Include Enum Values

Add after line 86:

# Add enums
for enum_name in sorted(schemaview.all_enums()):
    enum = schemaview.get_enum(enum_name)
    mermaid_lines.append(f"  class {enum_name} <<enumeration>>")
    for pv in enum.permissible_values.values():
        mermaid_lines.append(f"    {enum_name} : {pv.text}")
    mermaid_lines.append("")

Filter to Specific Classes

Add after line 69:

# Only include core classes
core_classes = ['Custodian', 'CustodianObservation', 'CustodianLegalStatus', 
                'CustodianName', 'CustodianPlace', 'EncompassingBody']
if class_name not in core_classes:
    continue

---

Troubleshooting

Diagram Too Large (Mermaid.live won't render)

Solution 1: Reduce attributes per class

for slot_name in class_slots[:5]:  # Fewer attributes

Solution 2: Filter to core classes only (see "Extending" above)

Solution 3: Generate multiple diagrams by module

# Filter by module
if 'CustodianType' in class_name:
    # CustodianType hierarchy diagram

Missing Relationships

Check: Ensure slot range is a class name:

if slot.range and slot.range in schemaview.all_classes():
    # Only creates relationship if range is a class

Fix: Some relationships may use uriorcurie (string reference) instead of class range. Script currently filters these out to avoid noise.

---

Next Steps

Generate SVG/PNG Exports

Install Mermaid CLI:

npm install -g @mermaid-js/mermaid-cli

Convert to SVG:

mmdc -i complete_schema_20251124_004143.mmd -o complete_schema.svg

Add to Documentation Website

If building with Sphinx/MkDocs:

## Schema Overview

```mermaid
{{ include('schemas/20251121/uml/mermaid/complete_schema_20251124_004143.mmd') }}
\```

Automate Generation in CI/CD

Add to .github/workflows/schema-docs.yml:

- name: Generate Complete Mermaid Diagram
  run: python3 scripts/generate_complete_mermaid_diagram.py

---

References

• LinkML Documentation: https://linkml.io/linkml/generators/mermaid.html
• Mermaid Class Diagrams: https://mermaid.js.org/syntax/classDiagram.html
• Mermaid Live Editor: https://mermaid.live/
• Script Location: scripts/generate_complete_mermaid_diagram.py
• Output Directory: schemas/20251121/uml/mermaid/

---

Changelog

2025-11-24: Initial implementation

• Created custom script extending LinkML's MermaidRenderer
• Generated complete schema diagram (53 classes, 149 relationships)
• Documented usage and extension patterns