rag system
#fastapi#qdrant#redis#openai#sentence-transformers#deberta#docker#prometheus
production rag system with zero-hallucination verification pipeline. nli entailment checking, hybrid search (vector + bm25 + cross-encoder reranking), multi-tenant architecture, and abstention decider.
/ verification pipeline
- nli entailment checking using deberta v3 — verifies every answer is logically supported by source documents before returning to user
- citation validation ensures each cited passage actually supports the claim it's attached to
- multi-signal abstention decider with 6 detection signals — system refuses to answer rather than hallucinate
- faithfulness threshold at 0.7, citation support threshold at 0.5 — tunable per deployment
/ hybrid search architecture
- vector search (bge-small-en-v1.5 embeddings) for semantic similarity
- bm25 keyword search for exact term matching
- reciprocal rank fusion combines both retrieval methods
- cross-encoder reranking (ms-marco-minilm-l-6-v2) refines top candidates
/ multi-tenant system
- per-tenant document isolation with database-backed tenancy
- tenant authentication and authorization via api keys
- admin api for tenant management, document ingestion, and usage tracking
- async background job processing with redis for document ingestion status tracking
/ evaluation framework
- offline evaluation metrics: faithfulness, context utilization, citation precision
- cli evaluation runner with configurable thresholds
- 35+ integration tests covering retrieval, verification, and end-to-end query flows
- prometheus metrics endpoint and structured logging for production observability
/ how it works
01documents ingested via api — parsed, chunked, deduplicated, embedded, stored in qdrant
02query arrives — rewritten with conversation history if multi-turn
03hybrid retrieval: vector search + bm25, fused via reciprocal rank fusion
04cross-encoder reranks top candidates for precision
05llm generates answer grounded in retrieved context with citations
06verification pipeline checks entailment, validates citations, decides whether to return or abstain
/ features
zero-hallucination verification
nli entailment checking verifies every answer against source documents. the system prefers abstention over hallucination — refuses uncertain answers rather than guessing.
hybrid retrieval + reranking
vector search + bm25 keyword search combined via reciprocal rank fusion, then refined by cross-encoder reranking. captures both semantic similarity and exact term matches.
abstention decider
6 detection signals determine whether the system should answer or refuse. prevents confident-sounding but unsupported responses — the core failure mode of naive rag.
conversational context
redis-backed session store with query rewriting using chat history. supports multi-turn conversations with context-aware retrieval.
multi-format ingestion
supports pdf, txt, docx, pptx via pymupdf. configurable chunking (512 chars with 64 overlap), content deduplication via hashing, async background processing with status tracking.
production observability
structured logging with context, prometheus metrics endpoint, per-request timing instrumentation, and health checks for api, qdrant, and database.