glam/docs/plan/prompt-query_template_mapping/conversation-context.md

31 KiB

Conversation Context and Follow-up Prompts

Overview

This document describes how the template-based SPARQL system handles follow-up prompts - short questions that only make sense in the context of previous conversation turns. DSPy's History mechanism is crucial for resolving these elliptical references.

The Problem: Elliptical Follow-up Questions

Consider this conversation:

User: Welke archieven zijn er in Den Haag?
System: In Den Haag zijn de volgende archieven: Nationaal Archief, Haags Gemeentearchief...

User: En in Enschede?

The second prompt "En in Enschede?" is elliptical - it omits information that must be inferred from context:

  • What is being asked about? → archives (from previous question)
  • What kind of question? → geographic listing (same intent)
  • Full resolved question → "Welke archieven zijn er in Enschede?"

Without context resolution, "En in Enschede?" would be unmatchable to any template.

DSPy History Integration

The History Object

DSPy provides a History class for conversation context:

from dspy import History

# History contains previous turns as messages
history = History(messages=[
    {"role": "user", "content": "Welke archieven zijn er in Den Haag?"},
    {"role": "assistant", "content": "In Den Haag zijn de volgende archieven: ..."},
])

Context Resolution Signature

"""DSPy signatures for conversation context resolution."""

import dspy
from dspy import History
from pydantic import BaseModel, Field


class ResolvedQuestion(BaseModel):
    """Resolved question with context filled in."""
    
    original_question: str = Field(
        description="The original (possibly elliptical) question"
    )
    resolved_question: str = Field(
        description="Fully resolved question with all implicit references expanded"
    )
    is_follow_up: bool = Field(
        description="Whether this was a follow-up to a previous question"
    )
    inherited_intent: str | None = Field(
        default=None,
        description="Intent inherited from previous turn (if follow-up)"
    )
    inherited_entities: list[str] = Field(
        default_factory=list,
        description="Entities inherited from previous turns"
    )
    slot_overrides: dict[str, str] = Field(
        default_factory=dict,
        description="Slots that changed from the previous question"
    )
    confidence: float = Field(
        ge=0.0, le=1.0,
        description="Confidence in the resolution"
    )


class ContextResolutionSignature(dspy.Signature):
    """Resolve elliptical follow-up questions using conversation history.
    
    Users often ask follow-up questions that omit information from previous turns.
    Your task is to resolve these elliptical references into fully specified questions.
    
    EXAMPLES OF ELLIPTICAL PATTERNS:
    
    1. Location change (most common):
       Previous: "Welke archieven zijn er in Den Haag?"
       Follow-up: "En in Enschede?"
       Resolved: "Welke archieven zijn er in Enschede?"
       
    2. Type change:
       Previous: "Hoeveel musea zijn er in Utrecht?"
       Follow-up: "En bibliotheken?"
       Resolved: "Hoeveel bibliotheken zijn er in Utrecht?"
       
    3. Pronoun reference:
       Previous: "Wat is het oudste archief in Nederland?"
       Follow-up: "Wie is daar de directeur?"
       Resolved: "Wie is de directeur van [oldest archive name]?"
       
    4. Implicit continuation:
       Previous: "Welke musea zijn er in Amsterdam?"
       Follow-up: "Met een Van Gogh collectie?"
       Resolved: "Welke musea in Amsterdam hebben een Van Gogh collectie?"
       
    5. Comparative follow-up:
       Previous: "Hoeveel archieven zijn er in Noord-Holland?"
       Follow-up: "En in Zuid-Holland?"
       Resolved: "Hoeveel archieven zijn er in Zuid-Holland?"
       
    6. Detail request:
       Previous: "Welke bibliotheken zijn er in Rotterdam?"
       Follow-up: "Meer informatie over de eerste"
       Resolved: "Meer informatie over [first library name from results]"
    
    IMPORTANT:
    - If the question is self-contained, return it unchanged
    - Only inherit context when the question is clearly elliptical
    - Preserve the user's language (Dutch/English)
    - Include ALL necessary information in the resolved question
    """
    
    question: str = dspy.InputField(
        desc="Current user question (may be elliptical)"
    )
    history: History = dspy.InputField(
        desc="Previous conversation turns",
        default=History(messages=[])
    )
    language: str = dspy.InputField(
        desc="Language code (nl, en, de, fr)",
        default="nl"
    )
    previous_results: str = dspy.InputField(
        desc="Results from the previous query (for pronoun resolution)",
        default=""
    )
    
    resolved: ResolvedQuestion = dspy.OutputField(
        desc="Fully resolved question with context"
    )


class ConversationContextResolver(dspy.Module):
    """Resolve elliptical follow-up questions to full questions.
    
    This module is critical for template matching: templates match against
    resolved questions, not the original elliptical input.
    """
    
    def __init__(self, fast_lm: dspy.LM | None = None):
        """Initialize the context resolver.
        
        Args:
            fast_lm: Optional fast LM for context resolution.
                     Recommended: gpt-4o-mini for speed while maintaining quality.
        """
        super().__init__()
        self.fast_lm = fast_lm
        self.resolver = dspy.TypedPredictor(ContextResolutionSignature)
    
    def forward(
        self,
        question: str,
        history: History | None = None,
        language: str = "nl",
        previous_results: str = "",
    ) -> dspy.Prediction:
        """Resolve elliptical question using conversation context.
        
        Args:
            question: Current user question
            history: Previous conversation turns
            language: Language code
            previous_results: Formatted results from previous query
            
        Returns:
            Prediction with resolved question and metadata
        """
        if history is None:
            history = History(messages=[])
        
        # Quick check: if no history, question is self-contained
        if not history.messages:
            return dspy.Prediction(
                original_question=question,
                resolved_question=question,
                is_follow_up=False,
                inherited_intent=None,
                inherited_entities=[],
                slot_overrides={},
                confidence=1.0,
            )
        
        # Use fast LM if configured
        if self.fast_lm:
            with dspy.settings.context(lm=self.fast_lm):
                result = self.resolver(
                    question=question,
                    history=history,
                    language=language,
                    previous_results=previous_results,
                )
        else:
            result = self.resolver(
                question=question,
                history=history,
                language=language,
                previous_results=previous_results,
            )
        
        resolved = result.resolved
        
        return dspy.Prediction(
            original_question=resolved.original_question,
            resolved_question=resolved.resolved_question,
            is_follow_up=resolved.is_follow_up,
            inherited_intent=resolved.inherited_intent,
            inherited_entities=resolved.inherited_entities,
            slot_overrides=resolved.slot_overrides,
            confidence=resolved.confidence,
        )

Template Matching with Context

Integration Flow

User Follow-up: "En in Enschede?"
        |
        v
+------------------------+
| ConversationContext    |  <-- Resolve elliptical reference
| Resolver               |
+------------------------+
        |
        | resolved_question = "Welke archieven zijn er in Enschede?"
        v
+------------------------+
| TemplateClassifier     |  <-- Match against resolved question
+------------------------+
        |
        | template_id = "city_institution_search"
        | slots = {city: "Enschede", type: "A"}
        v
+------------------------+
| TemplateInstantiator   |  <-- Generate SPARQL
+------------------------+
        |
        v
Valid SPARQL Query

Slot Override Detection

When a follow-up changes just one slot, we can optimize by detecting the change:

"""Detect which slots changed in a follow-up question."""

from dataclasses import dataclass


@dataclass
class SlotChange:
    """A change in slot value between turns."""
    
    slot_name: str
    previous_value: str
    new_value: str


class SlotChangeDetector:
    """Detect slot changes between conversation turns."""
    
    # Patterns that indicate slot changes
    LOCATION_CHANGE_PATTERNS = [
        r"^en in (.+)\??$",  # "en in Enschede?"
        r"^in (.+)\??$",     # "in Utrecht?"
        r"^(.+) dan\??$",    # "Utrecht dan?"
    ]
    
    TYPE_CHANGE_PATTERNS = [
        r"^en (.+)\??$",     # "en bibliotheken?"
        r"^(.+) dan\??$",    # "bibliotheken dan?"
    ]
    
    def detect_changes(
        self,
        current_question: str,
        previous_slots: dict[str, str],
        resolved_slots: dict[str, str],
    ) -> list[SlotChange]:
        """Detect which slots changed between turns.
        
        Args:
            current_question: The follow-up question
            previous_slots: Slots from the previous turn
            resolved_slots: Slots extracted from resolved question
            
        Returns:
            List of slot changes
        """
        changes = []
        
        for slot_name, new_value in resolved_slots.items():
            old_value = previous_slots.get(slot_name)
            if old_value and old_value != new_value:
                changes.append(SlotChange(
                    slot_name=slot_name,
                    previous_value=old_value,
                    new_value=new_value,
                ))
        
        return changes


# Example
detector = SlotChangeDetector()

previous_slots = {"institution_type_code": "A", "city_lower": "den haag"}
resolved_slots = {"institution_type_code": "A", "city_lower": "enschede"}

changes = detector.detect_changes(
    "En in Enschede?",
    previous_slots,
    resolved_slots,
)
# -> [SlotChange(slot_name="city_lower", previous_value="den haag", new_value="enschede")]

Conversation State Management

Tracking Previous Turn Context

"""Manage conversation state for follow-up handling."""

from dataclasses import dataclass, field
from datetime import datetime
from typing import Any


@dataclass
class ConversationTurn:
    """A single turn in the conversation."""
    
    timestamp: datetime
    question: str
    resolved_question: str
    intent: str
    template_id: str | None
    slots: dict[str, str]
    sparql_query: str | None
    results: list[dict]
    answer: str


@dataclass
class ConversationState:
    """State of an ongoing conversation."""
    
    session_id: str
    turns: list[ConversationTurn] = field(default_factory=list)
    current_intent: str | None = None
    current_slots: dict[str, str] = field(default_factory=dict)
    entity_cache: dict[str, Any] = field(default_factory=dict)
    
    def add_turn(self, turn: ConversationTurn) -> None:
        """Add a turn and update current state."""
        self.turns.append(turn)
        self.current_intent = turn.intent
        self.current_slots = turn.slots.copy()
        
        # Cache entities from results for pronoun resolution
        for i, result in enumerate(turn.results[:5]):
            key = f"result_{i}"
            self.entity_cache[key] = result
            
            # Also cache by name for "het eerste" / "the first" references
            if name := result.get("name"):
                self.entity_cache[f"name:{name.lower()}"] = result
    
    def get_previous_turn(self) -> ConversationTurn | None:
        """Get the most recent turn."""
        return self.turns[-1] if self.turns else None
    
    def get_history(self, max_turns: int = 5) -> dspy.History:
        """Get conversation history for DSPy."""
        messages = []
        for turn in self.turns[-max_turns:]:
            messages.append({"role": "user", "content": turn.question})
            messages.append({"role": "assistant", "content": turn.answer})
        return dspy.History(messages=messages)
    
    def format_previous_results(self, max_results: int = 5) -> str:
        """Format previous results for pronoun resolution."""
        prev = self.get_previous_turn()
        if not prev or not prev.results:
            return ""
        
        lines = ["Previous results:"]
        for i, result in enumerate(prev.results[:max_results], 1):
            name = result.get("name", "Unknown")
            typ = result.get("type", "")
            lines.append(f"{i}. {name} ({typ})")
        
        return "\n".join(lines)

Session-Aware RAG Module

"""Heritage RAG with conversation context handling."""

import dspy
from uuid import uuid4


class ConversationAwareHeritageRAG(dspy.Module):
    """Heritage RAG that handles follow-up questions using DSPy History."""
    
    def __init__(self):
        super().__init__()
        self.context_resolver = ConversationContextResolver()
        self.fyke = FykeFilter()
        self.router = HeritageQueryRouter()
        self.template_classifier = TemplateClassifier()
        self.template_instantiator = TemplateInstantiator()
        self.answer_generator = dspy.ChainOfThought(HeritageAnswerGenerator)
        
        # Session storage (in production: Redis or similar)
        self._sessions: dict[str, ConversationState] = {}
    
    def get_or_create_session(self, session_id: str | None = None) -> ConversationState:
        """Get existing session or create new one."""
        if session_id is None:
            session_id = str(uuid4())
        
        if session_id not in self._sessions:
            self._sessions[session_id] = ConversationState(session_id=session_id)
        
        return self._sessions[session_id]
    
    async def answer(
        self,
        question: str,
        session_id: str | None = None,
        language: str = "nl",
    ) -> dspy.Prediction:
        """Answer question with conversation context awareness.
        
        Args:
            question: User's current question (may be elliptical)
            session_id: Session ID for conversation continuity
            language: Language code
            
        Returns:
            Prediction with answer and metadata
        """
        # Get session state
        session = self.get_or_create_session(session_id)
        
        # Step 1: Resolve elliptical references using DSPy History
        history = session.get_history()
        previous_results = session.format_previous_results()
        
        resolution = self.context_resolver(
            question=question,
            history=history,
            language=language,
            previous_results=previous_results,
        )
        
        resolved_question = resolution.resolved_question
        
        # Step 2: Check relevance (fyke)
        relevance = self.fyke(question=resolved_question, language=language)
        
        if relevance.caught_by_fyke:
            return dspy.Prediction(
                answer=relevance.fyke_response,
                caught_by_fyke=True,
                session_id=session.session_id,
            )
        
        # Step 3: Route using resolved question
        routing = self.router(
            question=resolved_question,
            language=language,
            history=history,  # Pass history to router for additional context
        )
        
        # Step 4: Template classification
        # Use inherited intent from context resolution if available
        effective_intent = resolution.inherited_intent or routing.intent
        
        template_match = self.template_classifier(
            question=resolved_question,
            intent=effective_intent,
            entities=routing.entities,
        )
        
        # Step 5: Handle slot inheritance for follow-ups
        if resolution.is_follow_up and session.current_slots:
            # Merge: current slots from session + overrides from new question
            merged_slots = session.current_slots.copy()
            merged_slots.update(resolution.slot_overrides)
            merged_slots.update(template_match.slots)
            effective_slots = merged_slots
        else:
            effective_slots = template_match.slots
        
        # Step 6: Generate and execute SPARQL
        sparql_query = None
        results = []
        
        if template_match.template_id and template_match.confidence > 0.7:
            try:
                sparql_query = self.template_instantiator.instantiate(
                    template_id=template_match.template_id,
                    slots=effective_slots,
                )
                results = await self.execute_sparql(sparql_query)
            except Exception as e:
                logger.warning(f"Template instantiation failed: {e}")
        
        # Step 7: Fallback to vector search if needed
        if not results:
            results = await self.qdrant_search(resolved_question)
        
        # Step 8: Generate answer
        context = self._format_results(results)
        answer_result = self.answer_generator(
            question=resolved_question,
            context=context,
            history=history,
            sources=["sparql" if sparql_query else "qdrant"],
            language=language,
        )
        
        # Step 9: Update session state
        turn = ConversationTurn(
            timestamp=datetime.now(),
            question=question,  # Original question
            resolved_question=resolved_question,
            intent=effective_intent,
            template_id=template_match.template_id,
            slots=effective_slots,
            sparql_query=sparql_query,
            results=results,
            answer=answer_result.answer,
        )
        session.add_turn(turn)
        
        return dspy.Prediction(
            answer=answer_result.answer,
            original_question=question,
            resolved_question=resolved_question,
            is_follow_up=resolution.is_follow_up,
            slot_changes=resolution.slot_overrides,
            template_id=template_match.template_id,
            sparql_query=sparql_query,
            confidence=answer_result.confidence,
            session_id=session.session_id,
            citations=answer_result.citations,
            follow_up_suggestions=answer_result.follow_up,
        )

Follow-up Pattern Taxonomy

Common Elliptical Patterns

Pattern Example Resolution Strategy
Location swap "En in Enschede?" Replace location slot, keep other slots
Type swap "En bibliotheken?" Replace type slot, keep location
Comparative "En in Zuid-Holland?" Replace region, keep intent and type
Pronoun reference "Wie is daar de directeur?" Resolve "daar" to entity from results
Ordinal reference "Meer over de eerste" Map "eerste" to first result
Implicit continuation "Met een Van Gogh collectie?" Add filter to previous query
Negation "Welke niet?" Negate previous filter
Quantifier change "Hoeveel dan?" Change SELECT to COUNT

Pattern Detection Rules

"""Rules for detecting follow-up patterns."""

import re
from dataclasses import dataclass
from enum import Enum


class FollowUpPattern(Enum):
    """Types of follow-up patterns."""
    
    LOCATION_SWAP = "location_swap"
    TYPE_SWAP = "type_swap"
    PRONOUN_REFERENCE = "pronoun_reference"
    ORDINAL_REFERENCE = "ordinal_reference"
    IMPLICIT_CONTINUATION = "implicit_continuation"
    NEGATION = "negation"
    QUANTIFIER_CHANGE = "quantifier_change"
    DETAIL_REQUEST = "detail_request"
    SELF_CONTAINED = "self_contained"


@dataclass
class PatternRule:
    """Rule for detecting a follow-up pattern."""
    
    pattern: FollowUpPattern
    regex: str
    slot_affected: str | None
    description: str


# Dutch follow-up pattern rules
DUTCH_PATTERN_RULES = [
    PatternRule(
        pattern=FollowUpPattern.LOCATION_SWAP,
        regex=r"^en\s+in\s+(.+?)\??$",
        slot_affected="location",
        description="Location change: 'en in X?'",
    ),
    PatternRule(
        pattern=FollowUpPattern.LOCATION_SWAP,
        regex=r"^in\s+(.+?)\??$",
        slot_affected="location",
        description="Location query: 'in X?'",
    ),
    PatternRule(
        pattern=FollowUpPattern.TYPE_SWAP,
        regex=r"^en\s+(musea|bibliotheken|archieven|galerieën)\??$",
        slot_affected="institution_type",
        description="Type change: 'en bibliotheken?'",
    ),
    PatternRule(
        pattern=FollowUpPattern.PRONOUN_REFERENCE,
        regex=r".*(daar|die|deze|dat|daarin|hiervan).*",
        slot_affected=None,
        description="Pronoun reference requiring entity resolution",
    ),
    PatternRule(
        pattern=FollowUpPattern.ORDINAL_REFERENCE,
        regex=r".*(eerste|tweede|derde|laatste|vorige).*",
        slot_affected=None,
        description="Ordinal reference to results",
    ),
    PatternRule(
        pattern=FollowUpPattern.DETAIL_REQUEST,
        regex=r"^meer\s+(informatie|details|over)\s+.*$",
        slot_affected=None,
        description="Request for more details",
    ),
    PatternRule(
        pattern=FollowUpPattern.QUANTIFIER_CHANGE,
        regex=r"^hoeveel\s+(dan|er)?\??$",
        slot_affected=None,
        description="Change to count query",
    ),
]


class FollowUpPatternDetector:
    """Detect follow-up patterns in questions."""
    
    def __init__(self, language: str = "nl"):
        self.language = language
        self.rules = DUTCH_PATTERN_RULES  # Could extend for other languages
    
    def detect(self, question: str) -> tuple[FollowUpPattern, re.Match | None]:
        """Detect the follow-up pattern in a question.
        
        Args:
            question: The user's question
            
        Returns:
            Tuple of (pattern_type, regex_match)
        """
        question_lower = question.lower().strip()
        
        for rule in self.rules:
            match = re.match(rule.regex, question_lower, re.IGNORECASE)
            if match:
                return rule.pattern, match
        
        return FollowUpPattern.SELF_CONTAINED, None
    
    def get_slot_to_change(self, pattern: FollowUpPattern) -> str | None:
        """Get which slot should change for a pattern."""
        for rule in self.rules:
            if rule.pattern == pattern:
                return rule.slot_affected
        return None

DSPy Optimization for Context Resolution

Training Data for Context Resolver

"""Training examples for context resolution optimization."""

CONTEXT_RESOLUTION_TRAINING = [
    # Location swap examples
    dspy.Example(
        question="En in Enschede?",
        history=dspy.History(messages=[
            {"role": "user", "content": "Welke archieven zijn er in Den Haag?"},
            {"role": "assistant", "content": "In Den Haag zijn de volgende archieven: ..."},
        ]),
        language="nl",
        resolved=ResolvedQuestion(
            original_question="En in Enschede?",
            resolved_question="Welke archieven zijn er in Enschede?",
            is_follow_up=True,
            inherited_intent="geographic",
            inherited_entities=["archive"],
            slot_overrides={"city_lower": "enschede"},
            confidence=0.95,
        ),
    ).with_inputs("question", "history", "language"),
    
    # Type swap examples
    dspy.Example(
        question="En bibliotheken?",
        history=dspy.History(messages=[
            {"role": "user", "content": "Hoeveel musea zijn er in Utrecht?"},
            {"role": "assistant", "content": "Er zijn 15 musea in Utrecht."},
        ]),
        language="nl",
        resolved=ResolvedQuestion(
            original_question="En bibliotheken?",
            resolved_question="Hoeveel bibliotheken zijn er in Utrecht?",
            is_follow_up=True,
            inherited_intent="statistical",
            inherited_entities=["library", "Utrecht"],
            slot_overrides={"institution_type_code": "L"},
            confidence=0.95,
        ),
    ).with_inputs("question", "history", "language"),
    
    # Pronoun resolution
    dspy.Example(
        question="Wie is daar de directeur?",
        history=dspy.History(messages=[
            {"role": "user", "content": "Wat is het oudste archief in Nederland?"},
            {"role": "assistant", "content": "Het oudste archief is het Nationaal Archief, opgericht in 1802."},
        ]),
        previous_results="Previous results:\n1. Nationaal Archief (Archive)",
        language="nl",
        resolved=ResolvedQuestion(
            original_question="Wie is daar de directeur?",
            resolved_question="Wie is de directeur van het Nationaal Archief?",
            is_follow_up=True,
            inherited_intent="entity_lookup",
            inherited_entities=["Nationaal Archief"],
            slot_overrides={"institution_slug": "nationaal-archief"},
            confidence=0.90,
        ),
    ).with_inputs("question", "history", "language", "previous_results"),
    
    # Self-contained question (no context needed)
    dspy.Example(
        question="Hoeveel musea zijn er in Amsterdam?",
        history=dspy.History(messages=[
            {"role": "user", "content": "Welke archieven zijn er in Den Haag?"},
            {"role": "assistant", "content": "In Den Haag zijn de volgende archieven: ..."},
        ]),
        language="nl",
        resolved=ResolvedQuestion(
            original_question="Hoeveel musea zijn er in Amsterdam?",
            resolved_question="Hoeveel musea zijn er in Amsterdam?",
            is_follow_up=False,
            inherited_intent=None,
            inherited_entities=[],
            slot_overrides={},
            confidence=0.98,
        ),
    ).with_inputs("question", "history", "language"),
]


def optimize_context_resolver():
    """Optimize the context resolver using GEPA."""
    
    resolver = ConversationContextResolver()
    
    # Define metric
    def resolution_metric(example: dspy.Example, prediction: dspy.Prediction) -> float:
        """Score context resolution quality."""
        score = 0.0
        
        # Check resolved question matches
        if prediction.resolved_question == example.resolved.resolved_question:
            score += 0.4
        elif example.resolved.resolved_question.lower() in prediction.resolved_question.lower():
            score += 0.2
        
        # Check is_follow_up detection
        if prediction.is_follow_up == example.resolved.is_follow_up:
            score += 0.2
        
        # Check inherited intent
        if prediction.inherited_intent == example.resolved.inherited_intent:
            score += 0.2
        
        # Check slot overrides
        expected_overrides = set(example.resolved.slot_overrides.items())
        actual_overrides = set(prediction.slot_overrides.items())
        if expected_overrides == actual_overrides:
            score += 0.2
        elif expected_overrides & actual_overrides:
            score += 0.1
        
        return score
    
    # Optimize
    from dspy import GEPA
    
    optimizer = GEPA(
        metric=resolution_metric,
        auto="light",
    )
    
    optimized_resolver = optimizer.compile(
        resolver,
        trainset=CONTEXT_RESOLUTION_TRAINING,
    )
    
    return optimized_resolver

Testing Follow-up Handling

Test Cases

"""Test cases for follow-up handling."""

import pytest


@pytest.mark.parametrize("question,history_q,expected_resolved", [
    # Location swap
    (
        "En in Enschede?",
        "Welke archieven zijn er in Den Haag?",
        "Welke archieven zijn er in Enschede?",
    ),
    # Type swap
    (
        "En bibliotheken?",
        "Hoeveel musea zijn er in Utrecht?",
        "Hoeveel bibliotheken zijn er in Utrecht?",
    ),
    # Province swap
    (
        "En in Zuid-Holland?",
        "Hoeveel archieven zijn er in Noord-Holland?",
        "Hoeveel archieven zijn er in Zuid-Holland?",
    ),
    # Self-contained (should not change)
    (
        "Hoeveel musea zijn er in Amsterdam?",
        "Welke archieven zijn er in Den Haag?",
        "Hoeveel musea zijn er in Amsterdam?",
    ),
])
def test_context_resolution(question, history_q, expected_resolved):
    """Test that follow-ups are correctly resolved."""
    resolver = ConversationContextResolver()
    
    history = dspy.History(messages=[
        {"role": "user", "content": history_q},
        {"role": "assistant", "content": "..."},
    ])
    
    result = resolver(question=question, history=history, language="nl")
    
    assert result.resolved_question == expected_resolved


@pytest.mark.parametrize("question,expected_pattern", [
    ("En in Enschede?", FollowUpPattern.LOCATION_SWAP),
    ("En bibliotheken?", FollowUpPattern.TYPE_SWAP),
    ("Wie is daar de directeur?", FollowUpPattern.PRONOUN_REFERENCE),
    ("Meer over de eerste", FollowUpPattern.DETAIL_REQUEST),
    ("Hoeveel dan?", FollowUpPattern.QUANTIFIER_CHANGE),
    ("Welke musea zijn er in Amsterdam?", FollowUpPattern.SELF_CONTAINED),
])
def test_pattern_detection(question, expected_pattern):
    """Test follow-up pattern detection."""
    detector = FollowUpPatternDetector(language="nl")
    pattern, _ = detector.detect(question)
    assert pattern == expected_pattern

Summary

Handling follow-up prompts requires:

Component Purpose DSPy Integration
ConversationContextResolver Resolve elliptical references TypedPredictor with History
ConversationState Track slots across turns Enables slot inheritance
FollowUpPatternDetector Detect pattern type Guides resolution strategy
SlotChangeDetector Identify changed slots Enables efficient query modification

Key Principles

  1. Always resolve before matching - Templates match against resolved questions
  2. Inherit slots from previous turns - Only override what changed
  3. Use DSPy History - Native support for conversation context
  4. Optimize with training data - GEPA can improve resolution quality
  5. Cache entity results - Enable pronoun/ordinal resolution

Example Conversation Flow

Turn 1:
  User: "Welke archieven zijn er in Den Haag?"
  Resolved: "Welke archieven zijn er in Den Haag?"
  Template: region_institution_search
  Slots: {type: "A", city: "den haag"}

Turn 2:
  User: "En in Enschede?"
  Resolved: "Welke archieven zijn er in Enschede?"  <-- Context used
  Template: region_institution_search (same)
  Slots: {type: "A", city: "enschede"}  <-- Only city changed

Turn 3:
  User: "En bibliotheken?"
  Resolved: "Welke bibliotheken zijn er in Enschede?"  <-- Context used
  Template: region_institution_search (same)
  Slots: {type: "L", city: "enschede"}  <-- Only type changed