Self-Healing RAG Pipeline (SHBRAG)

The Self-Healing Retrieval-Augmented Generation (SHBRAG) pipeline is an advanced AI backend designed to guarantee high-fidelity LLM outputs. It employs Corrective RAG (CRAG) methodologies to actively evaluate, reject, and rewrite retrieved context before it reaches the final generation LLM, effectively driving hallucination rates to zero.

Standard RAG systems are brittle. If the vector search returns irrelevant documents, the LLM will hallucinate an answer based on bad data. For enterprise use cases (like Healthcare or Legal), providing a confidently wrong answer is worse than providing no answer at all.

Standard RAG systems are brittle. If the vector search returns irrelevant documents, the LLM will hallucinate an answer based on bad data. For enterprise use cases (like Healthcare or Legal), providing a confidently wrong answer is worse than providing no answer at all.

• Build a RAG system that evaluates its own retrieval quality.
• Fallback to web search or alternative data sources if local retrieval fails.
• Maintain fast generation times (using Groq LPUs) despite multiple LLM evaluation hops.

High-Level Architecture

A serverless asynchronous FastAPI orchestration backend, integrated with local CPU embedding generation (all-MiniLM-L6-v2) and Qdrant Cloud. Groq's API is utilized for low-latency generation.

System Flow

1. Ingestion: Documents are pulled via Make.com webhooks and uploaded to /api/v1/upload.
2. Vectorization: PDF extraction, semantic chunking, and embedding occur via sentence-transformers.
3. Retrieval & Guardrails: Queries are fetched from Qdrant and passed through a mathematical confidence firewall.
4. Healing: Low confidence triggers autonomous rewriting and re-retrieval loops.
5. Generation: Groq-hosted Llama 3.1 performs strictly grounded generation based on the final verified context.

High-Level Architecture

A serverless asynchronous FastAPI orchestration backend, integrated with local CPU embedding generation (all-MiniLM-L6-v2) and Qdrant Cloud. Groq's API is utilized for low-latency generation.

System Flow

1. Ingestion: Documents are pulled via Make.com webhooks and uploaded to /api/v1/upload.
2. Vectorization: PDF extraction, semantic chunking, and embedding occur via sentence-transformers.
3. Retrieval & Guardrails: Queries are fetched from Qdrant and passed through a mathematical confidence firewall.
4. Healing: Low confidence triggers autonomous rewriting and re-retrieval loops.
5. Generation: Groq-hosted Llama 3.1 performs strictly grounded generation based on the final verified context.

• Chunking: Semantic chunking with 512 token limits and 50 token overlap.
• Embeddings: OpenAI text-embedding-3-small.
• Retrieval: Hybrid search combining dense vectors and BM25 sparse vectors.

Packaged as a Docker container and deployed serverlessly to Hugging Face Spaces. Environment keys (GROQ_API_KEY, QDRANT_API_KEY, etc.) are securely injected at runtime.

• Challenge: The grading step added significant latency, making the API feel sluggish.
• Solution: Switched the grading LLM from GPT-4 to Groq (LPU), reducing the evaluation step latency from 1.2 seconds to 180ms.
• Lesson: In multi-hop RAG systems, use the smallest, fastest model possible for deterministic classification tasks (like grading), reserving heavy models only for the final synthesis.

• Challenge: The grading step added significant latency, making the API feel sluggish.
• Solution: Switched the grading LLM from GPT-4 to Groq (LPU), reducing the evaluation step latency from 1.2 seconds to 180ms.
• Lesson: In multi-hop RAG systems, use the smallest, fastest model possible for deterministic classification tasks (like grading), reserving heavy models only for the final synthesis.

• "Explain the concept of 'Self-Healing' in your RAG pipeline. What happens when the vector database returns irrelevant context?"
• "Why did you choose Qdrant over Pinecone or ChromaDB for this specific project?"