End-to-End RAG (Retrieval-Augmented Generation) Application Development

Project Overview: We need an experienced AI/ML developer to build a robust, dynamic Retrieval-Augmented Generation (RAG) system for querying unstructured internal documents (PDFs, DOCX, TXT) with high factual accuracy and low latency. Key Technical Requirements: Document Ingestion & Chunking: Dynamic text extraction, cleaning, and semantic chunking. Avoid rigid hardcoded templates so it works across various document types. Vector Database Integration: Embedding generation and storage using solutions such as PostgreSQL (pgvector), ChromaDB, Pinecone, or Qdrant. Orchestration & Retrieval: Implemented via LangChain or LangGraph to handle multi-step reasoning, query expansion, and similarity search. LLM Integration: Connect with local/cloud LLMs (e.g., Groq API, Ollama, OpenAI) with proper guardrails against hallucinations. Interface / API: A clean REST API (FastAPI) or an interactive prototype (Streamlit / Next.js) for testing query-response flows. Deliverables: Clean, modular, and well-documented source code (Python). Containerized setup (Dockerfile & docker-compose.yml). Brief documentation explaining setup, chunking strategy, and vector retrieval flow. Preferred Skills: Python, LangChain, LangGraph, pgvector / Vector DBs, FastAPI, LLM API Integration, Docker.