AI Agents Made Simple reached #1 in Software Development, Software Engineering and Software Design Tools on Amazon.com's Kindle Store. A decision guide to building AI agents that work: 28 chapters, 148 illustrations, no code. Get the book on Amazon →

Sophisticated Controllable Agent for Complex RAG Tasks 🧠📚

An advanced Retrieval-Augmented Generation (RAG) solution designed to tackle complex questions that simple semantic similarity-based retrieval cannot solve. This project showcases a sophisticated deterministic graph acting as the "brain" of a highly controllable autonomous agent capable of answering non-trivial questions from your own data.

RAG Made Simple - the 400-page visual guide that extends this repo. Hit #1 in Generative AI on Amazon at launch

Get it on Amazon (paperback · Kindle · free on Kindle Unlimited) →

---

🚀 Level up with my Agents Towards Production repository. It delivers horizontal, code-first tutorials that cover every tool and step in the lifecycle of building production-grade GenAI agents, guiding you from spark to scale with proven patterns and reusable blueprints for real-world launches, making it the smartest place to start if you're serious about shipping agents to production.

📚 Explore my comprehensive guide on RAG techniques to complement this advanced agent implementation with many other RAG techniques.

🤖 Explore my GenAI Agents Repository to complement this advanced agent implementation with many other AI Agents implementations and tutorials.

📫 Stay Updated!

Join over 20,000 of AI enthusiasts getting unique cutting-edge insights and free tutorials! Plus, subscribers get exclusive early access and special 33% discounts on my books!

Prefer video? I break these ideas down into short, one-idea-per-episode explainers on YouTube. Watch on YouTube →

🌟 Key Features

• Sophisticated Deterministic Graph: Acts as the "brain" of the agent, enabling complex reasoning.
• Controllable Autonomous Agent: Capable of answering non-trivial questions from custom datasets.
• Hallucination Prevention: Ensures answers are solely based on provided data, avoiding AI hallucinations.
• Multi-step Reasoning: Breaks down complex queries into manageable sub-tasks.
• Adaptive Planning: Continuously updates its plan based on new information.
• Performance Evaluation: Utilizes Ragas metrics for comprehensive quality assessment.

🧠 How It Works

1. PDF Loading and Processing: Load PDF documents and split them into chapters.
2. Text Preprocessing: Clean and preprocess the text for better summarization and encoding.
3. Summarization: Generate extensive summaries of each chapter using large language models.
4. Book Quotes Database Creation: Create a database for specific questions that will need access to quotes from the book.
5. Vector Store Encoding: Encode the book content and chapter summaries into vector stores for efficient retrieval.
6. Question Processing:
   • Anonymize the question by replacing named entities with variables.
   • Generate a high-level plan to answer the anonymized question.
   • De-anonymize the plan and break it down into retrievable or answerable tasks.
7. Task Execution:
   • For each task, decide whether to retrieve information or answer based on context.
   • If retrieving, fetch relevant information from vector stores and distill it.
   • If answering, generate a response using chain-of-thought reasoning.
8. Verification and Re-planning:
   • Verify that generated content is grounded in the original context.
   • Re-plan remaining steps based on new information.
9. Final Answer Generation: Produce the final answer using accumulated context and chain-of-thought reasoning.

📊 Evaluation

The solution is evaluated using Ragas metrics:

• Answer Correctness
• Faithfulness
• Answer Relevancy
• Context Recall
• Answer Similarity

This is the beginning of the README. Read the rest at the original. Read the full README ↗