glam/frontend/PHASE3_TASK6_QUERY_BUILDER.md

Phase 3 - Task 6: Advanced Query Builder

Date: 2025-11-22
Priority: Medium
Estimated Time: 4-5 hours
Status: Ready to Start ⏳

---

Overview

Build a visual SPARQL query builder that allows users to construct queries through a graphical interface, with live syntax preview and pre-built templates.

---

Goals

1. Visual Query Construction: Drag-and-drop or form-based interface for building SPARQL queries
2. Live Syntax Preview: Real-time SPARQL syntax display with syntax highlighting
3. Query Templates: Library of pre-built queries for common use cases
4. Syntax Validation: Client-side query validation before execution
5. Query Management: Save, load, and share queries

---

Deliverables

1. Query Builder Component (src/components/query/QueryBuilder.tsx)

Features:

• Subject-Predicate-Object triple pattern builder
• Filter conditions (FILTER, REGEX, LANG, etc.)
• Graph patterns (OPTIONAL, UNION, MINUS)
• Aggregation (COUNT, SUM, AVG, etc.)
• Ordering and pagination (ORDER BY, LIMIT, OFFSET)
• Prefix management (common prefixes pre-configured)

UI Elements:

interface QueryBuilderState {
  prefixes: Map<string, string>;  // PREFIX declarations
  selectVariables: string[];      // SELECT ?var1 ?var2
  wherePatterns: TriplePattern[]; // WHERE { ... }
  filters: FilterExpression[];    // FILTER(...)
  groupBy: string[];              // GROUP BY ?var
  orderBy: OrderClause[];         // ORDER BY ?var
  limit: number | null;           // LIMIT n
  offset: number | null;          // OFFSET n
}

interface TriplePattern {
  subject: string;    // ?var or URI
  predicate: string;  // ?var or URI
  object: string;     // ?var or URI or literal
  optional: boolean;  // OPTIONAL { ... }
}

---

2. Query Editor Component (src/components/query/QueryEditor.tsx)

Features:

• Syntax highlighted text editor (using @codemirror/lang-sparql)
• Line numbers and bracket matching
• Auto-completion for variables and prefixes
• Error highlighting for invalid syntax
• Format/beautify query button

Libraries:

npm install @uiw/react-codemirror @codemirror/lang-sparql

Example:

import CodeMirror from '@uiw/react-codemirror';
import { sparql } from '@codemirror/lang-sparql';

export function QueryEditor({ value, onChange }: QueryEditorProps) {
  return (
    <CodeMirror
      value={value}
      height="400px"
      extensions={[sparql()]}
      onChange={onChange}
      theme="light"
    />
  );
}

---

3. Query Templates Library (src/lib/sparql/templates.ts)

Pre-built Queries:

export interface QueryTemplate {
  id: string;
  name: string;
  description: string;
  category: 'basic' | 'advanced' | 'aggregation' | 'geographic';
  query: string;
  variables: string[];  // Replaceable variables (e.g., %CITY%, %TYPE%)
}

export const QUERY_TEMPLATES: QueryTemplate[] = [
  {
    id: 'find-all-museums',
    name: 'Find All Museums',
    description: 'List all museums with names',
    category: 'basic',
    query: `
PREFIX schema: <http://schema.org/>

SELECT ?museum ?name WHERE {
  ?museum a schema:Museum .
  ?museum schema:name ?name .
}
ORDER BY ?name
LIMIT 100
    `,
    variables: [],
  },
  
  {
    id: 'institutions-in-city',
    name: 'Institutions in City',
    description: 'Find all institutions in a specific city',
    category: 'geographic',
    query: `
PREFIX schema: <http://schema.org/>

SELECT ?institution ?name ?type WHERE {
  ?institution a ?type .
  ?institution schema:name ?name .
  ?institution schema:address ?addr .
  ?addr schema:addressLocality "%CITY%" .
}
ORDER BY ?name
    `,
    variables: ['%CITY%'],
  },
  
  {
    id: 'count-by-type',
    name: 'Count Institutions by Type',
    description: 'Aggregate count of institutions grouped by type',
    category: 'aggregation',
    query: `
PREFIX schema: <http://schema.org/>
PREFIX cpov: <http://data.europa.eu/m8g/>

SELECT ?type (COUNT(?institution) AS ?count) WHERE {
  ?institution a ?type .
  FILTER(?type IN (schema:Museum, schema:Library, schema:ArchiveOrganization))
}
GROUP BY ?type
ORDER BY DESC(?count)
    `,
    variables: [],
  },
  
  {
    id: 'wikidata-links',
    name: 'Find Wikidata Links',
    description: 'Institutions with Wikidata identifiers',
    category: 'advanced',
    query: `
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX schema: <http://schema.org/>

SELECT ?institution ?name ?wikidataURI WHERE {
  ?institution schema:name ?name .
  ?institution owl:sameAs ?wikidataURI .
  FILTER(STRSTARTS(STR(?wikidataURI), "http://www.wikidata.org/entity/Q"))
}
LIMIT 100
    `,
    variables: [],
  },
];

---

4. Query Validator (src/lib/sparql/validator.ts)

Validation Rules:

• Syntax validation (basic SPARQL grammar)
• Prefix validation (all prefixes declared)
• Variable validation (all variables bound)
• Filter validation (correct syntax)

Implementation:

export interface ValidationResult {
  isValid: boolean;
  errors: ValidationError[];
  warnings: ValidationWarning[];
}

export interface ValidationError {
  line: number;
  column: number;
  message: string;
  severity: 'error' | 'warning';
}

export function validateSparqlQuery(query: string): ValidationResult {
  const errors: ValidationError[] = [];
  const warnings: ValidationWarning[] = [];
  
  // Check for required SELECT/CONSTRUCT/ASK/DESCRIBE
  if (!query.match(/\b(SELECT|CONSTRUCT|ASK|DESCRIBE)\b/i)) {
    errors.push({
      line: 1,
      column: 1,
      message: 'Query must contain SELECT, CONSTRUCT, ASK, or DESCRIBE',
      severity: 'error',
    });
  }
  
  // Check for WHERE clause
  if (!query.match(/\bWHERE\s*\{/i)) {
    errors.push({
      line: 1,
      column: 1,
      message: 'Query must contain WHERE clause',
      severity: 'error',
    });
  }
  
  // Check for unbalanced braces
  const openBraces = (query.match(/\{/g) || []).length;
  const closeBraces = (query.match(/\}/g) || []).length;
  if (openBraces !== closeBraces) {
    errors.push({
      line: 1,
      column: 1,
      message: 'Unbalanced braces in query',
      severity: 'error',
    });
  }
  
  // Check for undefined prefixes
  const usedPrefixes = new Set<string>();
  const declaredPrefixes = new Set<string>();
  
  // Find all PREFIX declarations
  const prefixRegex = /PREFIX\s+(\w+):/gi;
  let match;
  while ((match = prefixRegex.exec(query)) !== null) {
    declaredPrefixes.add(match[1]);
  }
  
  // Find all prefix usages
  const usageRegex = /(\w+):[^\s]+/g;
  while ((match = usageRegex.exec(query)) !== null) {
    if (match[0].startsWith('PREFIX')) continue;
    usedPrefixes.add(match[1]);
  }
  
  // Check for undeclared prefixes
  for (const prefix of usedPrefixes) {
    if (!declaredPrefixes.has(prefix)) {
      warnings.push({
        line: 1,
        column: 1,
        message: `Prefix '${prefix}' used but not declared`,
        severity: 'warning',
      });
    }
  }
  
  return {
    isValid: errors.length === 0,
    errors,
    warnings,
  };
}

---

5. Query Builder Page (src/pages/QueryBuilder.tsx)

Layout:

+----------------------------------+
| Navigation Header                |
+----------------------------------+
|                                  |
| +-----------------------------+  |
| | Template Selection Sidebar  |  |
| | - Basic Queries             |  |
| | - Advanced Queries          |  |
| | - Aggregation               |  |
| | - Geographic                |  |
| +-----------------------------+  |
|                                  |
| +-----------------------------+  |
| | Query Builder Panel         |  |
| | - Add Triple Pattern        |  |
| | - Add Filter                |  |
| | - Set Limit/Offset          |  |
| +-----------------------------+  |
|                                  |
| +-----------------------------+  |
| | SPARQL Syntax Preview       |  |
| | (CodeMirror with syntax     |  |
| |  highlighting)              |  |
| +-----------------------------+  |
|                                  |
| +-----------------------------+  |
| | Validation Results          |  |
| | ✓ Valid SPARQL              |  |
| | ⚠ 2 warnings                |  |
| +-----------------------------+  |
|                                  |
| [Execute Query] [Save] [Clear]   |
+----------------------------------+

Component Structure:

export function QueryBuilderPage() {
  const [queryState, setQueryState] = useState<QueryBuilderState>(initialState);
  const [syntaxPreview, setSyntaxPreview] = useState('');
  const [validation, setValidation] = useState<ValidationResult | null>(null);
  
  // Generate SPARQL from visual builder state
  useEffect(() => {
    const sparql = generateSparql(queryState);
    setSyntaxPreview(sparql);
    
    // Validate on change
    const result = validateSparqlQuery(sparql);
    setValidation(result);
  }, [queryState]);
  
  return (
    <div className="query-builder-page">
      <TemplateSidebar onSelectTemplate={loadTemplate} />
      <QueryBuilderPanel state={queryState} onChange={setQueryState} />
      <QueryEditor value={syntaxPreview} onChange={setSyntaxPreview} />
      <ValidationPanel result={validation} />
      <ActionButtons onExecute={executeQuery} onSave={saveQuery} />
    </div>
  );
}

---

Implementation Steps

Step 1: Create Query Templates (30 min)

touch src/lib/sparql/templates.ts

Tasks:

• Define QueryTemplate interface
• Create 10-15 pre-built queries covering:
  • Basic SELECT queries
  • Aggregation (COUNT, GROUP BY)
  • Geographic filtering (city, country)
  • Wikidata links
  • Relationship queries (branches, partnerships)
  • Collection queries

---

Step 2: Create Query Validator (45 min)

touch src/lib/sparql/validator.ts
touch tests/unit/sparql-validator.test.ts

Tasks:

• Implement basic syntax validation
• Check for required clauses (SELECT, WHERE)
• Validate prefix declarations
• Check for unbalanced braces
• Variable binding checks
• Write 10+ unit tests

---

Step 3: Install CodeMirror (15 min)

cd /Users/kempersc/apps/glam/frontend
npm install @uiw/react-codemirror @codemirror/lang-sparql

Tasks:

• Install dependencies
• Test CodeMirror integration in simple component
• Configure syntax highlighting theme
• Add auto-completion for common prefixes

---

Step 4: Create Query Editor Component (30 min)

touch src/components/query/QueryEditor.tsx
touch src/components/query/QueryEditor.css

Tasks:

• Implement CodeMirror wrapper
• Add syntax highlighting
• Add line numbers
• Add bracket matching
• Style for dark/light theme compatibility

---

Step 5: Create Query Builder Component (90 min)

touch src/components/query/QueryBuilder.tsx
touch src/components/query/QueryBuilder.css

Tasks:

• Implement QueryBuilderState interface
• Create triple pattern builder UI
• Add filter condition builder
• Implement LIMIT/OFFSET controls
• Add prefix management UI
• Create generateSparql() function to convert state to SPARQL string

---

Step 6: Create Query Builder Page (45 min)

touch src/pages/QueryBuilder.tsx
touch src/pages/QueryBuilder.css

Tasks:

• Layout page structure
• Integrate all components
• Add template selection sidebar
• Wire up state management
• Add save/load functionality
• Implement validation display

---

Step 7: Add Routing and Navigation (15 min)

Update src/App.tsx:

{
  path: '/query-builder',
  element: <QueryBuilderPage />,
}

Update src/components/layout/Navigation.tsx:

<NavLink to="/query-builder">Query Builder</NavLink>

---

Step 8: Write Tests (30 min)

touch tests/unit/query-validator.test.ts
touch tests/unit/query-templates.test.ts
touch tests/unit/query-builder.test.tsx

Test Coverage:

• Validator: 15+ tests (syntax, prefixes, variables)
• Templates: 5+ tests (template loading, variable substitution)
• Builder: 10+ tests (state management, SPARQL generation)

---

Testing Checklist

Manual Testing

• Load query from template
• Edit query in visual builder
• See live SPARQL preview update
• Validation shows errors for invalid syntax
• Validation shows warnings for undeclared prefixes
• Execute query (shows error if Oxigraph not running)
• Save query to localStorage
• Load saved query
• Export query as .sparql file
• Syntax highlighting works in dark/light theme

Automated Testing

• All validator tests pass
• All template tests pass
• All builder component tests pass
• Zero TypeScript errors
• Build succeeds

---

Success Criteria

1. ✅ User can select from 10+ pre-built query templates
2. ✅ User can visually construct queries (triple patterns, filters, LIMIT)
3. ✅ Live SPARQL syntax preview updates in real-time
4. ✅ Syntax validation shows errors and warnings
5. ✅ User can save/load queries
6. ✅ User can edit raw SPARQL in CodeMirror editor
7. ✅ Syntax highlighting works correctly
8. ✅ All tests pass
9. ✅ Zero build errors
10. ✅ Documentation complete

---

Libraries and Tools

Required Dependencies

# Syntax highlighting
npm install @uiw/react-codemirror @codemirror/lang-sparql

# Optional: SPARQL parser for advanced validation
npm install sparqljs

Optional Enhancements (Future)

• sparqljs - Full SPARQL parser for advanced validation
• @rdfjs/types - TypeScript types for RDF data model
• yasqe - Yet Another SPARQL Editor (alternative to CodeMirror)

---

Example Queries to Support

Query 1: Find All Museums in Denmark

PREFIX schema: <http://schema.org/>

SELECT ?museum ?name ?city WHERE {
  ?museum a schema:Museum .
  ?museum schema:name ?name .
  ?museum schema:address ?addr .
  ?addr schema:addressLocality ?city .
  ?addr schema:addressCountry "DK" .
}
ORDER BY ?city ?name
LIMIT 100

Query 2: Count Institutions by Type

PREFIX schema: <http://schema.org/>

SELECT ?type (COUNT(?inst) AS ?count) WHERE {
  ?inst a ?type .
  FILTER(?type IN (schema:Museum, schema:Library, schema:ArchiveOrganization))
}
GROUP BY ?type
ORDER BY DESC(?count)

Query 3: Institutions with Wikidata and ISIL

PREFIX schema: <http://schema.org/>
PREFIX owl: <http://www.w3.org/2002/07/owl#>

SELECT ?inst ?name ?wikidata ?isil WHERE {
  ?inst schema:name ?name .
  ?inst owl:sameAs ?wikidata .
  ?inst schema:identifier ?isil .
  
  FILTER(STRSTARTS(STR(?wikidata), "http://www.wikidata.org/entity/Q"))
  FILTER(CONTAINS(?isil, "ISIL:"))
}
LIMIT 50

---

Notes

Oxigraph Not Required Yet

Important: Task 6 (Query Builder) does NOT require Oxigraph to be running. The query builder generates SPARQL strings and validates syntax, but does not execute queries.

Query Execution: Implemented in Task 7 (SPARQL Execution)

State Persistence

Save query builder state to localStorage:

// Save current query
const savedQuery = {
  name: 'My Query',
  query: syntaxPreview,
  timestamp: Date.now(),
};

localStorage.setItem('saved-queries', JSON.stringify([savedQuery]));

Integration with GraphContext

Query builder should integrate with existing GraphContext:

const { graphData } = useGraphContext();

// Use graph data to suggest variables and URIs in autocomplete
const availableTypes = extractTypesFromGraph(graphData);
const availablePredicates = extractPredicatesFromGraph(graphData);

---

References

SPARQL Documentation

• W3C SPARQL 1.1 Query: https://www.w3.org/TR/sparql11-query/
• SPARQL Tutorial: https://www.w3.org/2009/Talks/0615-qbe/
• SPARQL Examples: https://www.linkeddatatools.com/querying-semantic-data

CodeMirror Documentation

• React CodeMirror: https://uiwjs.github.io/react-codemirror/
• SPARQL Language Support: https://codemirror.net/try/?example=SPARQL

Project Documentation

• Triplestore Setup: TRIPLESTORE_OXIGRAPH_SETUP.md
• RDF Datasets: data/rdf/README.md
• Schema Documentation: schemas/20251121/rdf/

---

Timeline

Step	Task	Time	Status
1	Create query templates	30 min	⏳
2	Create query validator	45 min	⏳
3	Install CodeMirror	15 min	⏳
4	Create query editor	30 min	⏳
5	Create query builder	90 min	⏳
6	Create query builder page	45 min	⏳
7	Add routing/navigation	15 min	⏳
8	Write tests	30 min	⏳
Total		4-5 hours

---

Next Steps

After completing Task 6:

Task 7: SPARQL Query Execution (6-8 hours)

1. Install and configure Oxigraph server
2. Load Denmark dataset (43,429 triples)
3. Create SPARQL client module
4. Implement query execution hook
5. Create results viewer component
6. Add export functionality (CSV, JSON, RDF)
7. Add query performance metrics

See: TRIPLESTORE_OXIGRAPH_SETUP.md for detailed Task 7 plan

---

Status: Ready to Start ⏳
Estimated Completion: 4-5 hours
Dependencies: None (can start immediately)
Next Task: Task 7 - SPARQL Execution (requires Oxigraph)

Last Updated: 2025-11-22
Author: OpenCode AI Agent