Skip to content

Self-host the HTTP server

chatbot-auditor ships with a FastAPI server so you can run audits as an HTTP service inside your own infrastructure. This tutorial walks through running it locally with Docker, configuring auth, and calling the /analyze endpoint from another service.

Install

pip install "chatbot-auditor[server]"

The [server] extra pulls in FastAPI and Uvicorn.

Run locally (no Docker)

uvicorn chatbot_auditor.server:app --host 0.0.0.0 --port 8000

In another terminal:

curl http://localhost:8000/healthz
# {"status":"ok"}

curl -X POST http://localhost:8000/analyze \
  -H "Content-Type: application/json" \
  -d @example.json

Run with Docker

The repository includes a Dockerfile and docker-compose.yml:

docker compose up --build

This builds the wheel, installs it with the [server] extra, runs as an unprivileged user, and exposes port 8000 with a liveness healthcheck.

Endpoints

Method Path Purpose
GET /healthz Liveness probe
GET /readyz Readiness probe — registry initialized
GET /version Version + attribution
GET /detectors Enumerate active detectors
POST /analyze Audit a batch of conversations

Auto-generated OpenAPI docs are available at /docs (Swagger UI) and /redoc (ReDoc) when the server is running.

POST /analyze

Request body:

{
  "conversations": [
    {
      "id": "c-42",
      "platform": "intercom",
      "reported_resolved": true,
      "messages": [
        {"role": "user", "content": "I need a refund"},
        {"role": "bot", "content": "Please check our FAQ."}
      ]
    }
  ],
  "detector_names": ["death_loop", "silent_churn"]
}

detector_names is optional. Omit it to run the full default registry.

Response:

{
  "total_conversations": 1,
  "total_detections": 2,
  "detections": [
    {
      "conversation_id": "c-42",
      "detector": "silent_churn",
      "failure_mode": "silent_churn",
      "severity": "high",
      "confidence": 0.85,
      "explanation": "Conversation ended without a customer confirmation...",
      "evidence": [...],
      "recommended_action": "follow_up",
      "metadata": {...}
    }
  ]
}

Authentication

Set CHATBOT_AUDITOR_API_KEYS to a comma-separated list of secrets:

docker run -p 8000:8000 \
  -e CHATBOT_AUDITOR_API_KEYS="prod-key-A,prod-key-B" \
  chatbot-auditor

With keys configured, /analyze requires Authorization: Bearer <key>. Health and metadata endpoints are unaffected — they remain public so probes and load balancers can check them without credentials.

When CHATBOT_AUDITOR_API_KEYS is unset or empty, /analyze accepts unauthenticated requests. Useful for local development behind a VPN or an existing API gateway; never expose unauthenticated to the public internet.

Limits

CHATBOT_AUDITOR_MAX_CONVERSATIONS_PER_REQUEST (default: 1000) caps the batch size. Requests above the limit return HTTP 413.

Deployment checklist

  • [ ] Run behind TLS (reverse proxy: nginx, Caddy, Cloudflare, ALB, …)
  • [ ] Set CHATBOT_AUDITOR_API_KEYS to rotate-able values from your secrets manager
  • [ ] Wire /healthz and /readyz to your orchestrator (K8s, ECS, Nomad)
  • [ ] Aggregate logs — the server writes to stdout in JSON-friendly format
  • [ ] Monitor request latency; the default five detectors run in pure Python at tens of thousands of conversations/second per core — CPU-bind first

Calling from another service

A minimal Python client:

import httpx

resp = httpx.post(
    "https://auditor.internal/analyze",
    headers={"Authorization": "Bearer prod-key-A"},
    json={
        "conversations": [...],
    },
    timeout=30.0,
)
resp.raise_for_status()
for d in resp.json()["detections"]:
    print(d["severity"], d["detector"], d["explanation"])