glam/frontend/PHASE3_TASK6_QUERY_BUILDER.md

# Phase 3 - Task 6: Advanced Query Builder

**Date**: 2025-11-22  
**Priority**: Medium  
**Estimated Time**: 4-5 hours  
**Status**: Ready to Start ⏳

---

## Overview

Build a visual SPARQL query builder that allows users to construct queries through a graphical interface, with live syntax preview and pre-built templates.

---

## Goals

1. **Visual Query Construction**: Drag-and-drop or form-based interface for building SPARQL queries
2. **Live Syntax Preview**: Real-time SPARQL syntax display with syntax highlighting
3. **Query Templates**: Library of pre-built queries for common use cases
4. **Syntax Validation**: Client-side query validation before execution
5. **Query Management**: Save, load, and share queries

---

## Deliverables

### 1. Query Builder Component (`src/components/query/QueryBuilder.tsx`)

**Features**:
- Subject-Predicate-Object triple pattern builder
- Filter conditions (FILTER, REGEX, LANG, etc.)
- Graph patterns (OPTIONAL, UNION, MINUS)
- Aggregation (COUNT, SUM, AVG, etc.)
- Ordering and pagination (ORDER BY, LIMIT, OFFSET)
- Prefix management (common prefixes pre-configured)

**UI Elements**:
```typescript
interface QueryBuilderState {
  prefixes: Map<string, string>;  // PREFIX declarations
  selectVariables: string[];      // SELECT ?var1 ?var2
  wherePatterns: TriplePattern[]; // WHERE { ... }
  filters: FilterExpression[];    // FILTER(...)
  groupBy: string[];              // GROUP BY ?var
  orderBy: OrderClause[];         // ORDER BY ?var
  limit: number | null;           // LIMIT n
  offset: number | null;          // OFFSET n
}

interface TriplePattern {
  subject: string;    // ?var or URI
  predicate: string;  // ?var or URI
  object: string;     // ?var or URI or literal
  optional: boolean;  // OPTIONAL { ... }
}
```

---

### 2. Query Editor Component (`src/components/query/QueryEditor.tsx`)

**Features**:
- Syntax highlighted text editor (using `@codemirror/lang-sparql`)
- Line numbers and bracket matching
- Auto-completion for variables and prefixes
- Error highlighting for invalid syntax
- Format/beautify query button

**Libraries**:
```bash
npm install @uiw/react-codemirror @codemirror/lang-sparql
```

**Example**:
```typescript
import CodeMirror from '@uiw/react-codemirror';
import { sparql } from '@codemirror/lang-sparql';

export function QueryEditor({ value, onChange }: QueryEditorProps) {
  return (
    <CodeMirror
      value={value}
      height="400px"
      extensions={[sparql()]}
      onChange={onChange}
      theme="light"
    />
  );
}
```

---

### 3. Query Templates Library (`src/lib/sparql/templates.ts`)

**Pre-built Queries**:

```typescript
export interface QueryTemplate {
  id: string;
  name: string;
  description: string;
  category: 'basic' | 'advanced' | 'aggregation' | 'geographic';
  query: string;
  variables: string[];  // Replaceable variables (e.g., %CITY%, %TYPE%)
}

export const QUERY_TEMPLATES: QueryTemplate[] = [
  {
    id: 'find-all-museums',
    name: 'Find All Museums',
    description: 'List all museums with names',
    category: 'basic',
    query: `
PREFIX schema: <http://schema.org/>

SELECT ?museum ?name WHERE {
  ?museum a schema:Museum .
  ?museum schema:name ?name .
}
ORDER BY ?name
LIMIT 100
    `,
    variables: [],
  },
  
  {
    id: 'institutions-in-city',
    name: 'Institutions in City',
    description: 'Find all institutions in a specific city',
    category: 'geographic',
    query: `
PREFIX schema: <http://schema.org/>

SELECT ?institution ?name ?type WHERE {
  ?institution a ?type .
  ?institution schema:name ?name .
  ?institution schema:address ?addr .
  ?addr schema:addressLocality "%CITY%" .
}
ORDER BY ?name
    `,
    variables: ['%CITY%'],
  },
  
  {
    id: 'count-by-type',
    name: 'Count Institutions by Type',
    description: 'Aggregate count of institutions grouped by type',
    category: 'aggregation',
    query: `
PREFIX schema: <http://schema.org/>
PREFIX cpov: <http://data.europa.eu/m8g/>

SELECT ?type (COUNT(?institution) AS ?count) WHERE {
  ?institution a ?type .
  FILTER(?type IN (schema:Museum, schema:Library, schema:ArchiveOrganization))
}
GROUP BY ?type
ORDER BY DESC(?count)
    `,
    variables: [],
  },
  
  {
    id: 'wikidata-links',
    name: 'Find Wikidata Links',
    description: 'Institutions with Wikidata identifiers',
    category: 'advanced',
    query: `
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX schema: <http://schema.org/>

SELECT ?institution ?name ?wikidataURI WHERE {
  ?institution schema:name ?name .
  ?institution owl:sameAs ?wikidataURI .
  FILTER(STRSTARTS(STR(?wikidataURI), "http://www.wikidata.org/entity/Q"))
}
LIMIT 100
    `,
    variables: [],
  },
];
```

---

### 4. Query Validator (`src/lib/sparql/validator.ts`)

**Validation Rules**:
- Syntax validation (basic SPARQL grammar)
- Prefix validation (all prefixes declared)
- Variable validation (all variables bound)
- Filter validation (correct syntax)

**Implementation**:
```typescript
export interface ValidationResult {
  isValid: boolean;
  errors: ValidationError[];
  warnings: ValidationWarning[];
}

export interface ValidationError {
  line: number;
  column: number;
  message: string;
  severity: 'error' | 'warning';
}

export function validateSparqlQuery(query: string): ValidationResult {
  const errors: ValidationError[] = [];
  const warnings: ValidationWarning[] = [];
  
  // Check for required SELECT/CONSTRUCT/ASK/DESCRIBE
  if (!query.match(/\b(SELECT|CONSTRUCT|ASK|DESCRIBE)\b/i)) {
    errors.push({
      line: 1,
      column: 1,
      message: 'Query must contain SELECT, CONSTRUCT, ASK, or DESCRIBE',
      severity: 'error',
    });
  }
  
  // Check for WHERE clause
  if (!query.match(/\bWHERE\s*\{/i)) {
    errors.push({
      line: 1,
      column: 1,
      message: 'Query must contain WHERE clause',
      severity: 'error',
    });
  }
  
  // Check for unbalanced braces
  const openBraces = (query.match(/\{/g) || []).length;
  const closeBraces = (query.match(/\}/g) || []).length;
  if (openBraces !== closeBraces) {
    errors.push({
      line: 1,
      column: 1,
      message: 'Unbalanced braces in query',
      severity: 'error',
    });
  }
  
  // Check for undefined prefixes
  const usedPrefixes = new Set<string>();
  const declaredPrefixes = new Set<string>();
  
  // Find all PREFIX declarations
  const prefixRegex = /PREFIX\s+(\w+):/gi;
  let match;
  while ((match = prefixRegex.exec(query)) !== null) {
    declaredPrefixes.add(match[1]);
  }
  
  // Find all prefix usages
  const usageRegex = /(\w+):[^\s]+/g;
  while ((match = usageRegex.exec(query)) !== null) {
    if (match[0].startsWith('PREFIX')) continue;
    usedPrefixes.add(match[1]);
  }
  
  // Check for undeclared prefixes
  for (const prefix of usedPrefixes) {
    if (!declaredPrefixes.has(prefix)) {
      warnings.push({
        line: 1,
        column: 1,
        message: `Prefix '${prefix}' used but not declared`,
        severity: 'warning',
      });
    }
  }
  
  return {
    isValid: errors.length === 0,
    errors,
    warnings,
  };
}
```

---

### 5. Query Builder Page (`src/pages/QueryBuilder.tsx`)

**Layout**:
```
+----------------------------------+
| Navigation Header                |
+----------------------------------+
|                                  |
| +-----------------------------+  |
| | Template Selection Sidebar  |  |
| | - Basic Queries             |  |
| | - Advanced Queries          |  |
| | - Aggregation               |  |
| | - Geographic                |  |
| +-----------------------------+  |
|                                  |
| +-----------------------------+  |
| | Query Builder Panel         |  |
| | - Add Triple Pattern        |  |
| | - Add Filter                |  |
| | - Set Limit/Offset          |  |
| +-----------------------------+  |
|                                  |
| +-----------------------------+  |
| | SPARQL Syntax Preview       |  |
| | (CodeMirror with syntax     |  |
| |  highlighting)              |  |
| +-----------------------------+  |
|                                  |
| +-----------------------------+  |
| | Validation Results          |  |
| | ✓ Valid SPARQL              |  |
| | ⚠ 2 warnings                |  |
| +-----------------------------+  |
|                                  |
| [Execute Query] [Save] [Clear]   |
+----------------------------------+
```

**Component Structure**:
```typescript
export function QueryBuilderPage() {
  const [queryState, setQueryState] = useState<QueryBuilderState>(initialState);
  const [syntaxPreview, setSyntaxPreview] = useState('');
  const [validation, setValidation] = useState<ValidationResult | null>(null);
  
  // Generate SPARQL from visual builder state
  useEffect(() => {
    const sparql = generateSparql(queryState);
    setSyntaxPreview(sparql);
    
    // Validate on change
    const result = validateSparqlQuery(sparql);
    setValidation(result);
  }, [queryState]);
  
  return (
    <div className="query-builder-page">
      <TemplateSidebar onSelectTemplate={loadTemplate} />
      <QueryBuilderPanel state={queryState} onChange={setQueryState} />
      <QueryEditor value={syntaxPreview} onChange={setSyntaxPreview} />
      <ValidationPanel result={validation} />
      <ActionButtons onExecute={executeQuery} onSave={saveQuery} />
    </div>
  );
}
```

---

## Implementation Steps

### Step 1: Create Query Templates (30 min)

```bash
touch src/lib/sparql/templates.ts
```

**Tasks**:
- Define `QueryTemplate` interface
- Create 10-15 pre-built queries covering:
  - Basic SELECT queries
  - Aggregation (COUNT, GROUP BY)
  - Geographic filtering (city, country)
  - Wikidata links
  - Relationship queries (branches, partnerships)
  - Collection queries

---

### Step 2: Create Query Validator (45 min)

```bash
touch src/lib/sparql/validator.ts
touch tests/unit/sparql-validator.test.ts
```

**Tasks**:
- Implement basic syntax validation
- Check for required clauses (SELECT, WHERE)
- Validate prefix declarations
- Check for unbalanced braces
- Variable binding checks
- Write 10+ unit tests

---

### Step 3: Install CodeMirror (15 min)

```bash
cd /Users/kempersc/apps/glam/frontend
npm install @uiw/react-codemirror @codemirror/lang-sparql
```

**Tasks**:
- Install dependencies
- Test CodeMirror integration in simple component
- Configure syntax highlighting theme
- Add auto-completion for common prefixes

---

### Step 4: Create Query Editor Component (30 min)

```bash
touch src/components/query/QueryEditor.tsx
touch src/components/query/QueryEditor.css
```

**Tasks**:
- Implement CodeMirror wrapper
- Add syntax highlighting
- Add line numbers
- Add bracket matching
- Style for dark/light theme compatibility

---

### Step 5: Create Query Builder Component (90 min)

```bash
touch src/components/query/QueryBuilder.tsx
touch src/components/query/QueryBuilder.css
```

**Tasks**:
- Implement `QueryBuilderState` interface
- Create triple pattern builder UI
- Add filter condition builder
- Implement LIMIT/OFFSET controls
- Add prefix management UI
- Create `generateSparql()` function to convert state to SPARQL string

---

### Step 6: Create Query Builder Page (45 min)

```bash
touch src/pages/QueryBuilder.tsx
touch src/pages/QueryBuilder.css
```

**Tasks**:
- Layout page structure
- Integrate all components
- Add template selection sidebar
- Wire up state management
- Add save/load functionality
- Implement validation display

---

### Step 7: Add Routing and Navigation (15 min)

**Update `src/App.tsx`**:
```typescript
{
  path: '/query-builder',
  element: <QueryBuilderPage />,
}
```

**Update `src/components/layout/Navigation.tsx`**:
```typescript
<NavLink to="/query-builder">Query Builder</NavLink>
```

---

### Step 8: Write Tests (30 min)

```bash
touch tests/unit/query-validator.test.ts
touch tests/unit/query-templates.test.ts
touch tests/unit/query-builder.test.tsx
```

**Test Coverage**:
- Validator: 15+ tests (syntax, prefixes, variables)
- Templates: 5+ tests (template loading, variable substitution)
- Builder: 10+ tests (state management, SPARQL generation)

---

## Testing Checklist

### Manual Testing

- [ ] Load query from template
- [ ] Edit query in visual builder
- [ ] See live SPARQL preview update
- [ ] Validation shows errors for invalid syntax
- [ ] Validation shows warnings for undeclared prefixes
- [ ] Execute query (shows error if Oxigraph not running)
- [ ] Save query to localStorage
- [ ] Load saved query
- [ ] Export query as .sparql file
- [ ] Syntax highlighting works in dark/light theme

### Automated Testing

- [ ] All validator tests pass
- [ ] All template tests pass
- [ ] All builder component tests pass
- [ ] Zero TypeScript errors
- [ ] Build succeeds

---

## Success Criteria

1. ✅ User can select from 10+ pre-built query templates
2. ✅ User can visually construct queries (triple patterns, filters, LIMIT)
3. ✅ Live SPARQL syntax preview updates in real-time
4. ✅ Syntax validation shows errors and warnings
5. ✅ User can save/load queries
6. ✅ User can edit raw SPARQL in CodeMirror editor
7. ✅ Syntax highlighting works correctly
8. ✅ All tests pass
9. ✅ Zero build errors
10. ✅ Documentation complete

---

## Libraries and Tools

### Required Dependencies

```bash
# Syntax highlighting
npm install @uiw/react-codemirror @codemirror/lang-sparql

# Optional: SPARQL parser for advanced validation
npm install sparqljs
```

### Optional Enhancements (Future)

- `sparqljs` - Full SPARQL parser for advanced validation
- `@rdfjs/types` - TypeScript types for RDF data model
- `yasqe` - Yet Another SPARQL Editor (alternative to CodeMirror)

---

## Example Queries to Support

### Query 1: Find All Museums in Denmark
```sparql
PREFIX schema: <http://schema.org/>

SELECT ?museum ?name ?city WHERE {
  ?museum a schema:Museum .
  ?museum schema:name ?name .
  ?museum schema:address ?addr .
  ?addr schema:addressLocality ?city .
  ?addr schema:addressCountry "DK" .
}
ORDER BY ?city ?name
LIMIT 100
```

### Query 2: Count Institutions by Type
```sparql
PREFIX schema: <http://schema.org/>

SELECT ?type (COUNT(?inst) AS ?count) WHERE {
  ?inst a ?type .
  FILTER(?type IN (schema:Museum, schema:Library, schema:ArchiveOrganization))
}
GROUP BY ?type
ORDER BY DESC(?count)
```

### Query 3: Institutions with Wikidata and ISIL
```sparql
PREFIX schema: <http://schema.org/>
PREFIX owl: <http://www.w3.org/2002/07/owl#>

SELECT ?inst ?name ?wikidata ?isil WHERE {
  ?inst schema:name ?name .
  ?inst owl:sameAs ?wikidata .
  ?inst schema:identifier ?isil .
  
  FILTER(STRSTARTS(STR(?wikidata), "http://www.wikidata.org/entity/Q"))
  FILTER(CONTAINS(?isil, "ISIL:"))
}
LIMIT 50
```

---

## Notes

### Oxigraph Not Required Yet

**Important**: Task 6 (Query Builder) does NOT require Oxigraph to be running. The query builder generates SPARQL strings and validates syntax, but does not execute queries.

**Query Execution**: Implemented in Task 7 (SPARQL Execution)

### State Persistence

Save query builder state to localStorage:
```typescript
// Save current query
const savedQuery = {
  name: 'My Query',
  query: syntaxPreview,
  timestamp: Date.now(),
};

localStorage.setItem('saved-queries', JSON.stringify([savedQuery]));
```

### Integration with GraphContext

Query builder should integrate with existing `GraphContext`:
```typescript
const { graphData } = useGraphContext();

// Use graph data to suggest variables and URIs in autocomplete
const availableTypes = extractTypesFromGraph(graphData);
const availablePredicates = extractPredicatesFromGraph(graphData);
```

---

## References

### SPARQL Documentation

- **W3C SPARQL 1.1 Query**: https://www.w3.org/TR/sparql11-query/
- **SPARQL Tutorial**: https://www.w3.org/2009/Talks/0615-qbe/
- **SPARQL Examples**: https://www.linkeddatatools.com/querying-semantic-data

### CodeMirror Documentation

- **React CodeMirror**: https://uiwjs.github.io/react-codemirror/
- **SPARQL Language Support**: https://codemirror.net/try/?example=SPARQL

### Project Documentation

- **Triplestore Setup**: `TRIPLESTORE_OXIGRAPH_SETUP.md`
- **RDF Datasets**: `data/rdf/README.md`
- **Schema Documentation**: `schemas/20251121/rdf/`

---

## Timeline

| Step | Task | Time | Status |
|------|------|------|--------|
| 1 | Create query templates | 30 min | ⏳ |
| 2 | Create query validator | 45 min | ⏳ |
| 3 | Install CodeMirror | 15 min | ⏳ |
| 4 | Create query editor | 30 min | ⏳ |
| 5 | Create query builder | 90 min | ⏳ |
| 6 | Create query builder page | 45 min | ⏳ |
| 7 | Add routing/navigation | 15 min | ⏳ |
| 8 | Write tests | 30 min | ⏳ |
| **Total** | | **4-5 hours** | |

---

## Next Steps

After completing Task 6:

### Task 7: SPARQL Query Execution (6-8 hours)

1. Install and configure Oxigraph server
2. Load Denmark dataset (43,429 triples)
3. Create SPARQL client module
4. Implement query execution hook
5. Create results viewer component
6. Add export functionality (CSV, JSON, RDF)
7. Add query performance metrics

**See**: `TRIPLESTORE_OXIGRAPH_SETUP.md` for detailed Task 7 plan

---

**Status**: Ready to Start ⏳  
**Estimated Completion**: 4-5 hours  
**Dependencies**: None (can start immediately)  
**Next Task**: Task 7 - SPARQL Execution (requires Oxigraph)

**Last Updated**: 2025-11-22  
**Author**: OpenCode AI Agent