Commit graph

474 commits

Author SHA1 Message Date
kempersc
f61b8d5493 chore: update slot_fixes.yaml processed status and manifest timestamps
All checks were successful
Deploy Frontend / build-and-deploy (push) Successful in 2m13s
2026-01-19 13:12:01 +01:00
kempersc
e2d4666221 feat(schema): add Classroom and RoomUnit classes for classroom_count migration
Some checks failed
Deploy Frontend / build-and-deploy (push) Has been cancelled
- Create Classroom.yaml (subclass of Facility)
- Create RoomUnit.yaml (subclass of Unit for room counting)
- Create RoomUnitTypeEnum.yaml with room type values
- Update EducationCenter.yaml to use has_or_had_facility with Classroom
- Archive classroom_count.yaml slot

Rule 53 compliance: Replaces simple integer count with structured facility model.
2026-01-19 13:10:38 +01:00
kempersc
3da90b940e feat(schema): complete multiple slot_fixes.yaml migrations
All checks were successful
Deploy Frontend / build-and-deploy (push) Successful in 2m4s
Session 2026-01-19: Completed remaining migrations per Rules 53/56/60.

Major migrations:
1. claim_type → has_or_had_type + ClaimType/ClaimTypes (60+ concrete types in 11 categories)
2. circumstances_of_death → is_deceased + DeceasedStatus + CauseOfDeath
3. claims_count → has_or_had_quantity + Quantity (with based_on_claim for provenance)
4. classification_status → has_or_had_type + ClassificationStatusType

Created files:
- ClaimType.yaml, ClaimTypes.yaml (abstract base + 60+ concrete subclasses)
- DeceasedStatus.yaml, CauseOfDeath.yaml, CauseOfDeathTypeEnum.yaml
- ClassificationStatus.yaml, ClassificationStatusType.yaml, ClassificationStatusTypes.yaml
- CITESAppendix.yaml, City.yaml, CertaintyLevel.yaml
- is_deceased.yaml, is_or_was_caused_by.yaml, based_on_claim.yaml

Archived slots:
- claim_type, circumstances_of_death, claims_count, classification_status

Added Rule 60 to AGENTS.md: No Migration Deferral - agents MUST execute all migrations.

All 527 slot_fixes.yaml entries now complete (100%).
2026-01-19 13:05:53 +01:00
kempersc
4a277d7d42 standardise slots 2026-01-19 00:09:28 +01:00
kempersc
6812524ae5 feat(entity-review): add 'provides match' toggle for source URLs
All checks were successful
Deploy Frontend / build-and-deploy (push) Successful in 2m23s
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Successful in 5m37s
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Successful in 7m24s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Successful in 5m47s
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Successful in 6m52s
DSPy RAG Evaluation / Quality Gate (push) Successful in 1s
- Add toggle in source URL form to indicate when a source provides
  sufficient information to create a person profile without LinkedIn
- Store provides_match boolean in source observation data
- Display green badge on existing sources that have provides_match: true
- Include bilingual tooltip (EN/NL) explaining the toggle purpose
2026-01-18 18:25:45 +01:00
kempersc
b11223277c fix(entity-review): persist source URLs for WCMS-only profiles
All checks were successful
Deploy Frontend / build-and-deploy (push) Successful in 2m1s
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Successful in 5m34s
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Successful in 7m43s
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Successful in 5m54s
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Successful in 6m58s
DSPy RAG Evaluation / Quality Gate (push) Successful in 1s
- Add source_urls to WCMS-only profile detail response
- Update _candidates_by_wcms cache when creating new WCMS-only entries
- Use correct refresh method (fetchWcmsOnlyProfileDetail) after adding source URL

Fixes issue where source URLs added to WCMS-only profiles were not
displayed after page refresh because:
1. The wcms-only-profile/{email} endpoint wasn't returning source_urls
2. The frontend was calling fetchProfileDetail instead of
   fetchWcmsOnlyProfileDetail after adding a source URL
3. New WCMS-only entries weren't added to the lookup cache
2026-01-18 15:27:04 +01:00
kempersc
a31b89f672 refactor: update manifest.json timestamp, enhance schema definitions, and migrate publication_activity slot to structured format 2026-01-18 13:16:44 +01:00
kempersc
9a00160264 refactor: update manifest.json timestamp and enhance schema definitions across multiple YAML files 2026-01-18 02:02:54 +01:00
kempersc
a6a9ba58b8 standardise slots 2026-01-18 01:23:32 +01:00
kempersc
f30b1777f4 Enhance schema definitions and introduce new classes for DigitalPlatformV2
- Added detailed descriptions for slots: collecting_scope, collection_access, custody_history, education_level, membership_size, and publication_activity to improve clarity and usability.
- Removed the publication_date slot due to migration to a new structure.
- Updated slot fixes with migration notes and adjustments for various slots, ensuring alignment with new ontology standards.
- Introduced new classes for DigitalPlatformV2, including DigitalPlatformV2DataQualityNotes, DigitalPlatformV2DataSource, DigitalPlatformV2KeyContact, DigitalPlatformV2OrganizationProfile, DigitalPlatformV2OrganizationStatus, DigitalPlatformV2PrimaryPlatform, DigitalPlatformV2Provenance, DigitalPlatformV2ServiceDetails, and DigitalPlatformV2TransformationMetadata, each with comprehensive attributes and descriptions.
- Added classes for EnrichmentProvenance and EnrichmentProvenanceEntry to track provenance for enrichment sources, including detailed attributes for verification and source tracking.
- Created LogoClaim, LogoEnrichment, and LogoEnrichmentSummary classes to manage logo and favicon data extracted from web scraping, with attributes for claims and summary statistics.
- Archived the publication_date slot to maintain historical records.
2026-01-18 00:59:51 +01:00
kempersc
146f3c5c4e refactor: update generated timestamp in manifest.json and fix spelling errors in slot_fixes.yaml 2026-01-18 00:39:01 +01:00
kempersc
44b0771936 Remove incompatible equals_string values for has_or_had_identifier across multiple archive classes to comply with uriorcurie range requirements. 2026-01-17 23:02:17 +01:00
kempersc
aac0372aaa refactor: update tax deductibility schema and migrate slots per Rule 48 2026-01-17 21:45:52 +01:00
kempersc
2f707d224b refactor: update timestamp and enhance publication slot migration notes in slot_fixes.yaml 2026-01-17 21:11:23 +01:00
kempersc
47663e7c79 Refactor schema definitions and slots for improved temporal modeling and publisher representation
- Migrated `published_at` to `is_or_was_published_at` with structured `PublicationEvent` class for enhanced temporal accuracy.
- Introduced `has_or_had_publisher` slot to replace the string-based `publisher` slot, allowing for detailed publisher information.
- Added new slots: `deduction_percentage`, `regulatory_body`, `expiration_date`, and `jurisdiction` to support tax scheme documentation.
- Archived outdated slots: `published_by` and `publisher`, ensuring compliance with updated naming conventions and ontology alignment.
- Updated `Database` types to `DatabaseSystem` for consistency in technological infrastructure classification.
- Broadened range types for slots `allows_or_allowed` and `includes_or_included` from `string` to `uriorcurie` to resolve OWL ambiguities.
- Enhanced documentation and examples across various classes and slots to clarify usage and improve understanding.
2026-01-17 21:10:50 +01:00
kempersc
ed80fb316e refactor: migrate cataloging_standard to complies_or_complied_with and create CatalogingStandard class per Rule 53/56 2026-01-17 20:58:12 +01:00
kempersc
47834df7a3 refactor: update catalog_url to has_or_had_url and migrate per Rule 53/56 2026-01-17 19:53:55 +01:00
kempersc
441a096243 Implement feature X to enhance user experience and fix bug Y in module Z 2026-01-17 19:50:28 +01:00
kempersc
4319f38c05 Add archived slots for audience size, audience type, and capacity metrics
- Created new YAML files for audience size and audience type slots, defining their properties and annotations.
- Added archived capacity slots including cubic meters, linear meters, item count, and descriptions, with appropriate URIs and ranges.
- Introduced a template specificity slot for context-aware RAG filtering.
- Consolidated capacity-related slots into a unified structure, including has_or_had_capacity, capacity_type, and capacity_value, with detailed descriptions and examples.
2026-01-17 18:53:23 +01:00
kempersc
1b829fbe82 standardise slot names 2026-01-17 17:49:56 +01:00
kempersc
f71ef52432 refactor: migrate catalog slots to has_or_had_* structure and archive deprecated slots 2026-01-17 15:44:25 +01:00
kempersc
f18c7a5c3a Refactor address and asserter slots; migrate to has_or_had_* structure
- Updated manifest.json with new generated timestamp.
- Removed deprecated address_type and algorithm_name slots; migrated to has_or_had_type and has_or_had_label respectively.
- Updated Asserter.yaml to use has_or_had_* slots for asserter_contact, asserter_type, and asserter_version.
- Introduced IndexEntry class in Index.yaml for hierarchical index entries.
- Added DigitalPlatformType import to MailingListArchive.yaml and OnlineNewsArchive.yaml.
- Removed obsolete unit_type, algorithm_name, algorithm_version, asserter_contact, asserter_type, and asserter_version slot files.
- Archived removed slots in respective archive files.
- Updated slot_fixes.yaml to reflect migration statuses for asserter slots.
2026-01-17 15:34:11 +01:00
kempersc
46757be964 Refactor ontology schema: Migrate slots and update references
- Replaced deprecated slot 'broader_type' with 'has_or_had_hypernym' in MuseumType, OrganizationBranch, and ResearchOrganizationType schemas, ensuring all references are updated accordingly.
- Removed obsolete slots: 'binding_description', 'binding_type', 'borrower', 'borrower_contact', 'bounding_box', 'branch_description', 'branch_type', and 'taxonomic_rank', archiving them for future reference.
- Introduced new generic slots: 'has_or_had_contact_point', 'has_or_had_geographic_extent', and 'has_or_had_rank' to standardize contact and spatial information, aligning with RiC-O naming conventions.
- Updated slot_fixes.yaml to reflect migration status and ensure immutability of revision entries.
- Enhanced documentation and examples for new slots to facilitate understanding and usage.
2026-01-17 15:18:34 +01:00
kempersc
69373b5a13 Refactor and archive slots; migrate to generic temporal slots
- Refactored WomensArchivesRecordSetTypes.yaml to streamline imports and slot usage.
- Deleted obsolete slots: approximation_level, benefit, bio_custodian_subtype, bio_type_classification, business_criticality, business_model, cached_token.
- Archived deleted slots to respective archive directories for future reference.
- Introduced new generic slots: has_or_had_benefit, has_or_had_classification, has_or_had_level, has_or_had_model, has_or_had_token to standardize temporal naming conventions and improve semantic clarity.
- Updated slot descriptions and annotations to reflect new structures and usage.
2026-01-17 14:41:16 +01:00
kempersc
d99a7800e3 feat: enhance entity profile saving with PPID generation and backward compatibility 2026-01-17 01:55:38 +01:00
kempersc
dc95c7f7b7 fix: update generated timestamp in manifest.json and add feedback comments in slot_fixes.yaml 2026-01-17 00:11:38 +01:00
kempersc
54b26343c9 Add initial version of QUDT ontology file 2026-01-17 00:08:39 +01:00
kempersc
196f8a1023 Refactor schema slots and classes for improved semantic clarity and consistency
- Migrated `temperature_tolerance` to `allows_or_allowed` with `TemperatureDeviation` class for structured temperature tolerance representation.
- Replaced `temporal_coverage` with `has_or_had_content` to enhance temporal modeling using the `Content` class.
- Updated `FindingAid`, `LegalResponsibilityCollection`, and `EnvironmentalZone` schemas to reflect new slot structures.
- Archived obsolete slots: `temperature_tolerance`, `temporal_coverage`, `typical_http_methods`, and `typical_response_formats`.
- Introduced `has_or_had_technological_infrastructure` slot to replace `technology_stack`, providing a structured approach to modeling technological components.
- Enhanced documentation and examples across affected schemas to ensure clarity on new structures and their usage.
2026-01-16 20:09:58 +01:00
kempersc
cbdf2c2b2b feat: Introduce new slots and classes for heritage content and modes
- Added `has_or_had_direction` slot to represent directional orientations of entities, migrating from `text_direction`.
- Introduced `has_or_had_mode` slot for linking entities to operational modes, migrating from `thinking_mode`.
- Created `Content` class to encapsulate intellectual content of heritage materials, migrating from `temporal_coverage`.
- Added `TextDirection` class to define text flow orientations, migrating from `text_direction` slot.
- Introduced `ThinkingMode` class for LLM thinking mode configurations, migrating from `thinking_mode` slot.
- Archived previous slots related to text direction, languages detected, and thinking modes for historical reference.
- Updated documentation and annotations for clarity and compliance with RiC-O naming conventions.
2026-01-16 19:43:50 +01:00
kempersc
2c1ab0b4e6 fix: update generated timestamp and totalFiles count in manifest.json 2026-01-16 19:42:51 +01:00
kempersc
d47bb5b097 standardise slots 2026-01-16 18:57:52 +01:00
kempersc
c1748d3b11 feat: broaden slot ranges to 'Any' to resolve OWL ambiguity per Rule 55 2026-01-16 15:16:29 +01:00
kempersc
db389ed0a3 Refactor schema slots to resolve OWL ambiguity and enhance flexibility
- Updated ranges for multiple slots from `string` to `uriorcurie` to address OWL "Ambiguous type" warnings and allow for URI/CURIE references.
- Removed specialized slots for subtitle and transcript formats, consolidating them under broader predicates.
- Introduced new slots for structured descriptions, observation source documents, and entity statuses to improve data modeling.
- Implemented Rule 54 to broaden generic predicate ranges instead of creating bespoke predicates, promoting schema reuse and reducing complexity.
- Added a script for generating OWL ontology with type-object handling to ensure consistent ObjectProperty treatment for polymorphic slots.
2026-01-16 15:06:36 +01:00
kempersc
0a1f6c6f34 fix: update generated timestamp in manifest.json 2026-01-16 15:04:18 +01:00
kempersc
9034045e9a feat: update manifest.json timestamp and refactor Custodian and PersonObservation schemas with new slots for entity status and observation source documents
Some checks failed
Deploy Frontend / build-and-deploy (push) Successful in 5m10s
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 8m6s
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 2s
2026-01-16 14:33:06 +01:00
kempersc
bbef8eeac3 feat: add --frontend-server flag for server-side builds
Builds frontend on server (91.98.224.44) instead of locally.
Recommended for low-RAM machines where Vite builds hang.

- Added DEPLOY_FRONTEND_SERVER variable
- Added --frontend-server CLI flag
- Updated --all to use server build by default
- Server build: git pull → pnpm install → pnpm build → rsync to /var/www/
2026-01-16 14:17:46 +01:00
kempersc
77fb2ba9bf fix: update identifier references to external_identifier in WebPortal.yaml and update generated timestamp in manifest.json
Some checks failed
Deploy Frontend / build-and-deploy (push) Successful in 4m34s
DSPy RAG Evaluation / Layer 1 - Unit Tests (push) Failing after 9m20s
DSPy RAG Evaluation / Layer 2 - DSPy Module Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 3 - Integration Tests (push) Has been skipped
DSPy RAG Evaluation / Layer 4 - Comprehensive Evaluation (push) Has been skipped
DSPy RAG Evaluation / Quality Gate (push) Failing after 2s
2026-01-16 13:29:24 +01:00
kempersc
620b521f1b chore: update generated timestamp in manifest.json 2026-01-16 13:27:42 +01:00
kempersc
24cddb82dc enrich ppid profiles 2026-01-16 12:50:50 +01:00
kempersc
7424b85352 Add new slots for heritage custodian entities
- Introduced setpoint_max, setpoint_min, setpoint_tolerance, setpoint_type, setpoint_unit, setpoint_value, temperature_target, track_id, typical_http_methods, typical_metadata_standard, typical_response_formats, typical_scope, typical_technical_feature, unit_code, unit_symbol, unit_type, wikidata_entity, wikidata_equivalent, and wikidata_id slots.
- Each slot includes a unique identifier, name, title, description, and annotations for custodian types and specificity score.
2026-01-16 01:04:38 +01:00
kempersc
f9f3cc8e74 fix: resolve YAML import indentation and add missing slot descriptions
Schema Improvements:
- Fix YAML import indentation across 800+ class files (sed: '^- ../' → '  - ../')
- Add descriptions to 26 inline slots missing them (lint warnings)
- Fix malformed imports in BirthPlace.yaml and CustodianObservation.yaml

Validation Results:
- linkml-lint: 4 warnings (intentional SCREAMING_CASE tier names)
- gen-owl: SUCCESS (164,069 lines generated)
- gen-json-schema: SUCCESS (9.4MB generated)

Files affected: 1,034 files, +23,908 -15,200 lines
2026-01-16 00:09:28 +01:00
kempersc
6d961feafa chore: update manifest.json timestamp after schema sync 2026-01-15 23:23:11 +01:00
kempersc
c2629f6d29 Fix LinkML schema validation errors (0 errors, 30 warnings)
Schema Migration Fixes:
- Fix YAML import indentation in ~650 slot files (linkml:types and enum imports)
- Rename slot reference: has_or_had_holds_record_set_type → hold_or_held_record_set_type
  (70+ archive class files, main schema, manifest.json)
- Fix ProvenanceBlock.yaml: remove invalid any_of range, use string with multivalued
- Fix has_or_had_provenance.yaml: remove nested template_specificity from annotations

Validation Status:
- 0 errors (was multiple import/reference errors)
- 30 warnings (missing descriptions on inline slots, intentional SCREAMING_CASE names)

Files changed: ~3,850 (slots, classes, main schema, manifest)
2026-01-15 23:21:38 +01:00
kempersc
555949798d Update manifest.json timestamp, remove deprecated slot imports, and archive obsolete slots 2026-01-15 21:01:16 +01:00
kempersc
027871c070 Update generated timestamp in manifest.json and adjust imports in ExhibitionCatalog.yaml 2026-01-15 20:46:02 +01:00
kempersc
0cc8c8ca8f Add archived slot definitions for various attributes in the HC ontology
- Introduced new YAML files for slots including typical_scope, typical_technical_feature, unit_affiliation, used, used_by, user_community, verified, web_observation, whatsapp_business_likelihood, wikidata_alignment, wikidata, wikidata_entity, wikidata_equivalent, wikidata_id, wikidata_mapping, stores_or_stored, and time_of_destruction.
- Each slot includes detailed descriptions, mappings, and examples to enhance the ontology's semantic structure.
- Migrated and centralized the 'stores_object' slot into 'stores_or_stored' to comply with RiC-O naming conventions.
- Added comprehensive documentation for temporal-aware slots to support better data integration and querying capabilities.
2026-01-15 20:44:51 +01:00
kempersc
416aa407cc Add new slots for financial and heritage documentation
- Introduced total expense, total frames analyzed, total investment, total liability, total net asset, and traditional product slots to enhance financial reporting capabilities.
- Added transition types detected, treatment description, type hypothesis, typical condition, typical HTTP methods, typical response formats, and typical scope slots for improved heritage documentation.
- Implemented user community, verified, web observation, WhatsApp business likelihood, wikidata equivalent, and wikidata mapping slots to enrich institutional data representation.
- Established has_or_had_asset, has_or_had_budget, has_or_had_expense, and is_or_was_threatened_by slots to capture asset, budget, expense relationships, and threats to heritage forms.
2026-01-15 19:35:39 +01:00
kempersc
37d923cae1 Refactor slot names and update imports for consistency
- Migrated `was_generated_by` to `is_or_was_generated_by` and `was_derived_from` to `is_or_was_derived_from` across multiple YAML schema files as per Rule 53.
- Updated relevant imports, slot lists, and slot usage keys to reflect the new naming conventions.
- Added migration comments for clarity and tracking.
- Introduced a migration script to automate the changes across all affected files.
2026-01-15 15:07:53 +01:00
kempersc
3fb27c15e2 Refactor and archive deprecated slots; update migration records
- Removed deprecated slots: storage_security_level, version_number, video_comment, visiting_hour, was_asserted_by, was_revision_of, writing_system.
- Archived corresponding YAML files for deprecated slots with detailed migration notes.
- Updated slot definitions for has_collection and encompassing_body to reflect new naming conventions and temporal patterns.
- Enhanced metadata extraction in index_persons_qdrant.py to include WCMS registration and data sources.
- Modified hybrid_retriever and multi_embedding_retriever to support filtering by WCMS registration status.
2026-01-15 13:16:59 +01:00
kempersc
8174c9692e Refactor and Archive Deprecated Slots
- Removed deprecated slots:
  - accepts_or_accepted_external_work
  - accepts_or_accepted_payment_method
  - accepts_or_accepted_visiting_scholar
  - parent_collection
  - parent_custodian
  - storage_description
  - storage_type_description
  - sub_guide_description
  - transfer_location
  - transfer_location_text
  - transfer_policy
  - transfer_to_collection_date
  - unit_description

- Archived corresponding YAML files for the removed slots with detailed notes on migration and replacements.
- Updated slot fixes to reflect the migration of deprecated slots to new structures and naming conventions.
- Introduced new slots and classes to replace deprecated ones, ensuring compliance with RiC-O standards.
2026-01-15 13:00:27 +01:00