Commit graph

20 commits

Author SHA1 Message Date
kempersc
9034045e9a feat: update manifest.json timestamp and refactor Custodian and PersonObservation schemas with new slots for entity status and observation source documents
Some checks failed
Deploy Frontend / build-and-deploy (push) Successful in 5m10s
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 8m6s
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 2s
2026-01-16 14:33:06 +01:00
kempersc
f9f3cc8e74 fix: resolve YAML import indentation and add missing slot descriptions
Schema Improvements:
- Fix YAML import indentation across 800+ class files (sed: '^- ../' → '  - ../')
- Add descriptions to 26 inline slots missing them (lint warnings)
- Fix malformed imports in BirthPlace.yaml and CustodianObservation.yaml

Validation Results:
- linkml-lint: 4 warnings (intentional SCREAMING_CASE tier names)
- gen-owl: SUCCESS (164,069 lines generated)
- gen-json-schema: SUCCESS (9.4MB generated)

Files affected: 1,034 files, +23,908 -15,200 lines
2026-01-16 00:09:28 +01:00
kempersc
c2629f6d29 Fix LinkML schema validation errors (0 errors, 30 warnings)
Schema Migration Fixes:
- Fix YAML import indentation in ~650 slot files (linkml:types and enum imports)
- Rename slot reference: has_or_had_holds_record_set_type → hold_or_held_record_set_type
  (70+ archive class files, main schema, manifest.json)
- Fix ProvenanceBlock.yaml: remove invalid any_of range, use string with multivalued
- Fix has_or_had_provenance.yaml: remove nested template_specificity from annotations

Validation Status:
- 0 errors (was multiple import/reference errors)
- 30 warnings (missing descriptions on inline slots, intentional SCREAMING_CASE names)

Files changed: ~3,850 (slots, classes, main schema, manifest)
2026-01-15 23:21:38 +01:00
kempersc
0cc8c8ca8f Add archived slot definitions for various attributes in the HC ontology
- Introduced new YAML files for slots including typical_scope, typical_technical_feature, unit_affiliation, used, used_by, user_community, verified, web_observation, whatsapp_business_likelihood, wikidata_alignment, wikidata, wikidata_entity, wikidata_equivalent, wikidata_id, wikidata_mapping, stores_or_stored, and time_of_destruction.
- Each slot includes detailed descriptions, mappings, and examples to enhance the ontology's semantic structure.
- Migrated and centralized the 'stores_object' slot into 'stores_or_stored' to comply with RiC-O naming conventions.
- Added comprehensive documentation for temporal-aware slots to support better data integration and querying capabilities.
2026-01-15 20:44:51 +01:00
kempersc
8174c9692e Refactor and Archive Deprecated Slots
- Removed deprecated slots:
  - accepts_or_accepted_external_work
  - accepts_or_accepted_payment_method
  - accepts_or_accepted_visiting_scholar
  - parent_collection
  - parent_custodian
  - storage_description
  - storage_type_description
  - sub_guide_description
  - transfer_location
  - transfer_location_text
  - transfer_policy
  - transfer_to_collection_date
  - unit_description

- Archived corresponding YAML files for the removed slots with detailed notes on migration and replacements.
- Updated slot fixes to reflect the migration of deprecated slots to new structures and naming conventions.
- Introduced new slots and classes to replace deprecated ones, ensuring compliance with RiC-O standards.
2026-01-15 13:00:27 +01:00
kempersc
043ea868b5 fix(schema): Resolve broken imports after slot migration
All checks were successful
Deploy Frontend / build-and-deploy (push) Successful in 4m31s
- Fix empty import list elements (- # comment pattern) in Laptop, Expenses,
  FunctionType, Overview, WebLink, Photography classes
- Replace valid_from/valid_to slots with temporal_extent in class slots lists
- Update slot_usage to use temporal_extent with TimeSpan range
- Update examples to use temporal_extent with begin_of_the_begin/end_of_the_end
- Fix typo is_or_was_is_or_was_archived_at → is_or_was_archived_at in WebObservation
- Add TimeSpan imports to classes using temporal_extent
- Fix relative import paths for Timestamp in temporal slots
- Fix CustodianIdentifier → Identifier imports in FundingAgenda, ReadingRoomAnnex

Schema validates successfully with 902 classes and 2043 slots.
2026-01-15 12:25:27 +01:00
kempersc
6c3fa6b5a3 Remove deprecated slots and add new slot definitions for enhanced data modeling
- Deleted obsolete slot definitions for work_location and workshop_space.
- Introduced new TaxonName class to represent scientific taxonomic names with detailed attributes.
- Archived existing slots related to surname_prefix, target_name, taxon_name, terminal_count, text_region_count, title, title_proper, total_chapter, total_characters_extracted, total_connections_extracted, track_name, transcript_format, traveling_venue, type_label, type_status, typical_responsibility, unesco_domain, unesco_inscription_year, unesco_list_status, uniform_title, unit_name, used_by_custodian, uv_filtered_required, valid_from_geo, valid_to_geo, validation_status, variant_of_name, verification_date, viability_status, within_auxiliary_place, and within_place.
- Updated slot descriptions and structures to improve clarity and compliance with standards.
2026-01-15 11:42:35 +01:00
kempersc
ad5fbe82cf Migrate valid_from and valid_to slots to temporal_extent across multiple classes
- Consolidated valid_from and valid_to slots into a single temporal_extent slot in FundingRequirement, GiftShop, OrganizationBranch, OrganizationalChangeEvent, OrganizationalStructure, SocialMediaProfile, Storage, StorageUnit classes.
- Updated slot definitions to use TimeSpan for temporal_extent, providing structured validity periods.
- Removed deprecated slots: valid_from, valid_to, verified_by, wikidata_entity_id, and worldcat_id, archiving their definitions for reference.
- Adjusted related documentation and examples to reflect the new temporal_extent structure.
2026-01-14 22:33:36 +01:00
kempersc
58940582c3 Refactor warehouse and video slots: Migrate and archive multiple slots including warehouse_security_level, warehouse_name, and video_id. Introduce UnitIdentifier class to replace unit_id and unit_identifier slots. Update consuming classes and document migration details. Archive obsolete slot definitions for track_id, tracking_ids_assigned, vendor_name, and others. Ensure compliance with migration rules and maintain historical records in the archive. 2026-01-14 22:20:44 +01:00
kempersc
6da794ee38 feat: Introduce new slots and classes for enhanced heritage data modeling
- Added `has_or_had_place_of_birth` slot to capture structured birth place information with historical context.
- Introduced `has_or_had_quantity` slot for capturing quantified values with units and provenance.
- Created `has_or_had_service_area` slot to define geographic service areas for heritage custodians.
- Implemented `is_or_was_approximate` slot to indicate uncertainty in values (dates, quantities).
- Added `is_or_was_asserted_by` slot to track the agent responsible for assertions.
- Introduced `Asserter` class to model agents making assertions, including types like human, automated, and AI.
- Created `Quantity` class to represent quantified values with optional units and types.
- Added enums for `AsserterTypeEnum` and `QuantityTypeEnum` to standardize types of asserters and quantities.
- Archived outdated slots and replaced them with new structured alternatives following RiC-O conventions.
2026-01-14 16:54:10 +01:00
kempersc
7c7d8c0270 feat: Add SchemaElementPopup component for displaying LinkML schema element previews
- Implemented a draggable, resizable, and minimizable popup component for displaying previews of LinkML schema elements (classes, slots, enums).
- Integrated loading states and error handling for fetching element information.
- Added navigation functionality to go to full element view.
- Enhanced user experience with type badges and detailed descriptions for each element type.

chore: Migrate AudioEventSegment, BayNumber, BoxNumber, and BudgetStatus classes to new YAML schema format

- Created new YAML definitions for AudioEventSegment, BayNumber, BoxNumber, and BudgetStatus classes with detailed descriptions and attributes.
- Migrated from deprecated slots to new class structures as part of Rule 53.
- Updated imports and prefixes for consistency across schemas.

chore: Archive deprecated slots for audio_event_segments, bay_number, and box_number

- Archived previous slot definitions for audio_event_segments, bay_number, and box_number to maintain historical records.
- Updated slot descriptions and ensured proper URI mappings for future reference.
2026-01-14 15:13:06 +01:00
kempersc
355d8be51d centralise slots 2026-01-12 14:33:56 +01:00
kempersc
0d5d48568d refactor(schema): centralize slot definitions per Rule 38
- Remove slot_uri, description, mappings from slot_usage sections
- Move these properties to centralized slot files in modules/slots/
- Keep only class-specific overrides in slot_usage (required, inlined, examples)
- Update 1,499 centralized slot files with enriched definitions
- Clean 188 class files

Violations fixed:
- slot_uri in slot_usage: 1,676 → 0
- description in slot_usage: 2,287 → 0 (moved to centralized)

Schema still validates: 816 classes, 2028 slots, 127 enums
2026-01-11 23:27:17 +01:00
kempersc
174a420c08 refactor(schema): centralize 1515 inline slot definitions per Rule 48
All checks were successful
Deploy Frontend / build-and-deploy (push) Successful in 3m57s
- Remove inline slot definitions from 144 class files
- Create 7 new centralized slot files in modules/slots/:
  - custodian_type_broader.yaml
  - custodian_type_narrower.yaml
  - custodian_type_related.yaml
  - definition.yaml
  - finding_aid_access_restriction.yaml
  - finding_aid_description.yaml
  - finding_aid_temporal_coverage.yaml
- Add centralize_inline_slots.py automation script
- Update manifest with new timestamp

Rule 48: Class files must NOT define inline slots - all slots
must be imported from modules/slots/ directory.

Note: Pre-existing IdentifierFormat duplicate class definition
(in Standard.yaml and IdentifierFormat.yaml) not addressed in
this commit - requires separate schema refactor.
2026-01-11 22:02:14 +01:00
kempersc
626bd3a095 refactor(schemas): apply naming conventions to 261 class files
- Apply Rule 39: RiC-O style hasOrHad*/isOrWas* for temporal slots
- Apply Rule 43: Singular noun convention (keywords → keyword)
- Update slot references to match renamed slot files
- Maintain schema integrity across all class definitions
2026-01-10 15:36:33 +01:00
kempersc
0393b321c9 refactor(schema): unify custodian_type slots into has_or_had_custodian_type (Rule 39, 43)
- Migrate 236+ class files from custodian_types to has_or_had_custodian_type
- Archive deprecated slots: custodian_type, custodian_types, custodian_type_broader/narrower/related
- Update main schema and manifest imports
- Fix Custodian.yaml class to use new slot
- Fix annotation format (list→scalar) in has_or_had_custodian_type.yaml

Rules applied:
- Rule 39: RiC-O naming convention (hasOrHad pattern)
- Rule 43: Slot nouns must be singular (multivalued:true for cardinality)
- Rule 38: Slot centralization with semantic URI
2026-01-09 10:55:21 +01:00
kempersc
b34992b1d3 Migrate all 293 class files to ontology-aligned slots
Extends migration to all class types (museums, libraries, galleries, etc.)

New slots added to class_metadata_slots.yaml:
- RiC-O: rico_record_set_type, rico_organizational_principle,
  rico_has_or_had_holder, rico_note
- Multilingual: label_de, label_es, label_fr, label_nl, label_it, label_pt
- Scope: scope_includes, scope_excludes, custodian_only,
  organizational_level, geographic_restriction
- Notes: privacy_note, preservation_note, legal_note

Migration script now handles 30+ annotation types.
All migrated schemas pass linkml-validate.

Total: 387 class files now use proper slots instead of annotations.
2026-01-06 12:24:54 +01:00
kempersc
11983014bb Enhance specificity scoring system integration with existing infrastructure
- Updated documentation to clarify integration points with existing components in the RAG pipeline and DSPy framework.
- Added detailed mapping of SPARQL templates to context templates for improved specificity filtering.
- Implemented wrapper patterns around existing classifiers to extend functionality without duplication.
- Introduced new tests for the SpecificityAwareClassifier and SPARQLToContextMapper to ensure proper integration and functionality.
- Enhanced the CustodianRDFConverter to include ISO country and subregion codes from GHCID for better geospatial data handling.
2026-01-05 17:37:49 +01:00
kempersc
242bc8bb35 Add new slots for heritage custodian entities
- Created deliverables_slot for expected or achieved deliverable outputs.
- Introduced event_id_slot for persistent unique event identifiers.
- Added follow_up_date_slot for scheduled follow-up action dates.
- Implemented object_ref_slot for references to heritage objects.
- Established price_slot for price information across entities.
- Added price_currency_slot for currency codes in price information.
- Created protocol_slot for API protocol specifications.
- Introduced provenance_text_slot for full provenance entry text.
- Added record_type_slot for classification of record types.
- Implemented response_formats_slot for supported API response formats.
- Established status_slot for current status of entities or activities.
- Added FactualCountDisplay component for displaying count query results.
- Introduced ReplyTypeIndicator component for visualizing reply types.
- Created approval_date_slot for formal approval dates.
- Added authentication_required_slot for API authentication status.
- Implemented capacity_items_slot for maximum storage capacity.
- Established conservation_lab_slot for conservation laboratory information.
- Added cost_usd_slot for API operation costs in USD.
2026-01-05 00:49:05 +01:00
kempersc
2dca28d8c1 enrich CH entries with mission statements 2026-01-04 13:12:32 +01:00