Detectors

Every production detector along with its supporting types.

Base

Detector

Bases: ABC

Abstract base class for a chatbot failure detector.

Subclasses must declare the class-level attributes name, description, failure_mode, and requires_llm, then implement detect().

Example::

class MyDetector(Detector):
    name = "my_detector"
    description = "Detects my specific failure pattern"
    failure_mode = FailureMode.DEATH_LOOP
    requires_llm = False

    def detect(self, conversation: Conversation) -> list[Detection]:
        ...

detect abstractmethod

detect(conversation: Conversation) -> list[Detection]

Analyze a conversation and return zero or more detections.

Return an empty list if no failures are found. A single conversation may produce multiple detections (e.g., both a death loop and an escalation burial).

Implementations must not mutate the input conversation.

DetectorRegistry

DetectorRegistry(detectors: list[Detector] | None = None)

Holds the set of detectors to run on a conversation.

register

register(detector: Detector) -> None

Add a detector to the registry.

unregister

unregister(name: str) -> None

Remove a detector by name. No-op if not present.

get

get(name: str) -> Detector | None

Look up a detector by name.

all

all() -> list[Detector]

Return all registered detectors in insertion order.

run

run(conversation: Conversation) -> list[Detection]

Run every registered detector against a single conversation.

Phase 1 — statistical

DeathLoopDetector

DeathLoopDetector(*, similarity_threshold: float = 0.85, min_repeats: int = 3, min_content_length: int = 15, similarity_fn: SimilarityFn | None = None, frustration_keywords: Sequence[str] | None = None)

Bases: Detector

Detect consecutive similar bot responses indicating a conversational loop.

Configure the detector.

Parameters:

Name	Type	Description	Default
similarity_threshold	float	Pairwise similarity (0.0–1.0) above which two bot responses count as "the same answer." Default 0.85.	0.85
min_repeats	int	Minimum number of consecutive similar responses required to trigger a detection. Default 3.	3
min_content_length	int	Bot messages shorter than this (after normalization would be trivial) are ignored — short messages like "OK" trivially match and cause false positives. Default 15 characters.	15
similarity_fn	SimilarityFn | None	Function comparing two strings, returning [0.0, 1.0]. Defaults to lexical_similarity. Inject a semantic function (e.g. sentence-transformers-backed) to catch paraphrased loops.	None
frustration_keywords	Sequence[str] | None	Sequence of lowercase substrings that indicate customer frustration. Detections containing these get a small confidence bonus. Defaults to a curated English list.	None

SilentChurnDetector

SilentChurnDetector(*, confirmation_keywords: Sequence[str] | None = None, min_user_messages: int = 2, min_bot_messages: int = 2)

Bases: Detector

Detect conversations that ended without a customer confirmation signal.

Configure the detector.

Parameters:

Name	Type	Description	Default
confirmation_keywords	Sequence[str] | None	Lowercase substrings that indicate customer satisfaction / resolution. If any appears in any user message the conversation is considered healthy.	None
min_user_messages	int	Minimum number of user-role messages required to count as a "substantive" interaction worth flagging. Default 2.	2
min_bot_messages	int	Minimum number of bot-role messages required to count as a "substantive" interaction worth flagging. Default 2.	2

EscalationBurialDetector

EscalationBurialDetector(*, escalation_keywords: Sequence[str] | None = None, transfer_confirmation_keywords: Sequence[str] | None = None)

Bases: Detector

Detect when the customer asks for a human and the bot refuses to transfer.

Configure the detector.

Parameters:

Name	Type	Description	Default
escalation_keywords	Sequence[str] | None	Lowercase substrings indicating an explicit human-agent request in a user message.	None
transfer_confirmation_keywords	Sequence[str] | None	Lowercase substrings indicating the bot acknowledged the transfer (i.e. did NOT bury the request).	None

Phase 2 — content-aware

SentimentCollapseDetector

SentimentCollapseDetector(*, scorer: SentimentScorer | None = None, min_user_messages: int = 3, min_drop: float = 0.4, end_threshold: float = -0.2)

Bases: Detector

Detect a customer-sentiment drop over the course of a conversation.

Configure the detector.

Parameters:

Name	Type	Description	Default
scorer	SentimentScorer | None	Sentiment scoring backend. Defaults to KeywordSentimentScorer.	None
min_user_messages	int	Minimum user-role messages required to compute a meaningful trend. Default 3.	3
min_drop	float	Minimum difference (early-third average - late-third average) that counts as a collapse. Default 0.4.	0.4
end_threshold	float	The late third must also be below this absolute threshold. Default -0.2 — ensures the conversation actually ended in negative territory, not just less positive.	-0.2

BrandDamageDetector

BrandDamageDetector(*, checker: ContentSafetyChecker | None = None, competitor_names: Sequence[str] | None = None)

Bases: Detector

Detect bot messages likely to damage brand trust if seen publicly.

Configure the detector.

Parameters:

Name	Type	Description	Default
checker	ContentSafetyChecker | None	Pluggable content safety checker. Defaults to PatternSafetyChecker with the competitor_names argument.	None
competitor_names	Sequence[str] | None	Names (case-insensitive) of competitors the bot should not endorse. Only used if checker is not provided.	None

ConfidentLiesDetector

ConfidentLiesDetector(*, policy: PolicyBase | None = None, commitment_patterns: Sequence[Pattern[str]] | None = None)

Bases: Detector

Detect bot commitments that may not match company policy.

Configure the detector.

Parameters:

Name	Type	Description	Default
policy	PolicyBase | None	Optional knowledge base describing allowed and disallowed commitments. Without one, all commitments are flagged at medium confidence for human review.	None
commitment_patterns	Sequence[Pattern[str]] | None	Custom list of regex patterns matching commitment-style language. Defaults to a curated set.	None

ConfidentMisinformationDetector

ConfidentMisinformationDetector(*, facts: FactBase | None = None, claim_patterns: Sequence[Pattern[str]] | None = None)

Bases: Detector

Detect confident factual claims that may contradict ground truth.

Configure the detector.

Parameters:

Name	Type	Description	Default
facts	FactBase | None	Optional ground-truth facts. Without one, confident claims are flagged as "review recommended."	None
claim_patterns	Sequence[Pattern[str]] | None	Custom claim detection patterns. Defaults to a curated set covering pricing, hours, availability, and policy.	None

Pluggable backends

SentimentScorer

Bases: Protocol

Callable contract: given a text, return sentiment in [-1.0, 1.0].

KeywordSentimentScorer dataclass

KeywordSentimentScorer(positive: frozenset[str] = _POSITIVE_TOKENS, negative: frozenset[str] = _NEGATIVE_TOKENS)

Default SentimentScorer implementation using keyword counts.

ContentSafetyChecker

Bases: Protocol

A pluggable safety checker. Return a list of violation labels (empty if the text is safe).

PatternSafetyChecker dataclass

PatternSafetyChecker(profanity: tuple[str, ...] = _DEFAULT_PROFANITY, self_deprecation: tuple[str, ...] = _DEFAULT_SELF_DEPRECATION, offbrand_patterns: tuple[Pattern[str], ...] = _DEFAULT_OFFBRAND_PATTERNS, competitor_names: tuple[str, ...] = tuple())

Default stdlib-only content safety checker.