Beyond Chatbots: Building a Local RAG Application

My Proposal: Build Your Own Local AI Knowledge Assistant is a hands-on workshop that introduces participants to the fundamentals of Large Language Models (LLMs), local AI, and Retrieval-Augmented Generation (RAG).

The workshop begins with a brief introduction to how modern AI and LLMs work, followed by an exploration of local LLM inference using Ollama and Qwen. Participants will then learn how private documents can be made accessible to an LLM through a RAG pipeline, covering document ingestion, chunking, embeddings, vector databases, semantic search, retrieval, context augmentation, and grounded generation.

Participants will progressively build each component and integrate them into a complete application using Python, ChromaDB, LangChain, Ollama, Qwen, and Streamlit. By the end of the workshop, they will have built and tested their own local AI knowledge assistant capable of answering questions based on uploaded documents and displaying relevant sources.

The workshop also introduces practical considerations such as retrieval quality, common RAG failure modes, hallucinations, RAG evaluation, security considerations, and the differences between RAG and fine-tuning.

The emphasis is on learning by building, with approximately 20–30% conceptual explanation and 60–70% live coding and hands-on experimentation. No prior RAG experience is required; basic Python familiarity is recommended.

Key takeaway: Participants will leave with a working local RAG application and a practical understanding of how document-aware AI systems are built from the ground up.

By the end of the workshop, participants will be able to:

• Understand the fundamentals of LLMs, Generative AI, and local AI inference.

• Explain what RAG is and how it differs from a traditional LLM-based application.

• Run and interact with local LLMs using Ollama and Qwen.

• Understand document ingestion, chunking, embeddings, and semantic search.

• Use FAISS and ChromaDB as a vector database for storing and retrieving document information.

• Build a complete RAG pipeline from documents to grounded answers.

• Integrate an LLM and RAG pipeline into a Python application.

• Build a simple, interactive RAG application using Streamlit.

• Evaluate retrieval and answer quality and understand common RAG failure modes.

• Understand the practical differences between RAG and fine-tuning.

• Gain hands-on experience building and experimenting with a local AI application.