Commit graph

12 commits

Author SHA1 Message Date
kempersc
4d5641b6c5 feat(rag): add database routing configuration to templates
- Add 'databases' field to TemplateDefinition and TemplateMatchResult
- Support values: 'oxigraph' (SPARQL/KG), 'qdrant' (vector search)
- Add helper methods use_oxigraph() and use_qdrant()
- Default to both databases for backward compatibility
- Allows templates to skip vector search for factual/geographic queries
2026-01-09 11:54:17 +01:00
kempersc
c88fd3af70 Refactor code structure for improved readability and maintainability 2026-01-09 11:05:26 +01:00
kempersc
6608a207d4 update frontend 2026-01-08 15:56:28 +01:00
kempersc
0b0ea75070 feat(rag): add factual query fast path - skip LLM for count/list queries
- Add ontology cache warming at startup in lifespan() function
- Add is_factual_query() detection in template_sparql.py (12 templates)
- Add factual_result and sparql_query fields to DSPyQueryResponse
- Skip LLM generation for factual templates (count, list, compare)
- Execute SPARQL directly and return results as table (~15s → ~2s latency)
- Update ConversationPanel.tsx to render factual results table
- Add CSS styling for factual results with green theme

For queries like 'hoeveel archieven zijn er in Den Haag', the SPARQL
results ARE the answer - no need for expensive LLM prose generation.
2026-01-08 13:34:23 +01:00
kempersc
99dc608826 Refactor RAG to template-based SPARQL generation
Major architectural changes based on Formica et al. (2023) research:
- Add TemplateClassifier for deterministic SPARQL template matching
- Add SlotExtractor with synonym resolution for slot values
- Add TemplateInstantiator using Jinja2 for query rendering
- Refactor dspy_heritage_rag.py to use template system
- Update main.py with streamlined pipeline
- Fix semantic_router.py ordering issues
- Add comprehensive metrics tracking

Template-based approach achieves 65% precision vs 10% LLM-only
per Formica et al. research on SPARQL generation.
2026-01-07 22:04:43 +01:00
kempersc
11983014bb Enhance specificity scoring system integration with existing infrastructure
- Updated documentation to clarify integration points with existing components in the RAG pipeline and DSPy framework.
- Added detailed mapping of SPARQL templates to context templates for improved specificity filtering.
- Implemented wrapper patterns around existing classifiers to extend functionality without duplication.
- Introduced new tests for the SpecificityAwareClassifier and SPARQLToContextMapper to ensure proper integration and functionality.
- Enhanced the CustodianRDFConverter to include ISO country and subregion codes from GHCID for better geospatial data handling.
2026-01-05 17:37:49 +01:00
kempersc
4f0cafe98a enrich HC profiles 2026-01-02 02:11:04 +01:00
kempersc
1d8fd68e3a backup custodian web profiles 2025-12-30 23:53:16 +01:00
kempersc
30b701a5ec backup HC data 2025-12-30 23:41:15 +01:00
kempersc
90b402dba6 enrich AR en Czech files 2025-12-30 23:01:01 +01:00
kempersc
d64f857aa9 add sparql validator and RAG injector 2025-12-30 03:43:31 +01:00
kempersc
84904e344b Make AGENTS more succint by referring to opencode rules & enrich custodians 2025-12-28 14:56:35 +01:00