Optimizing Retrieval Augmented Generation Chatbots: A Comparative Analysis

Abstract

This thesis investigates the optimization of Retrieval Augmented Generation (RAG) chatbots with a focus on improving information preparation within the retrieval process. As part of an adapted Systematic Literature Review (SLR), the current state of research on retrieval methods and optimization approaches in the RAG context is systematically collected and analyzed. Based on this analysis, suitable methods are identified and classified. For the subsequent comparative evaluation of the selected retrieval approaches, an evaluation framework based on an adapted version of the Software Architecture Comparison Method (SACAM) is developed, which includes criteria such as document relevance, answer precision, response latency, and hallucination resistance. The implementation focuses on vector-based retrieval methods (e.g., dense retrieval and hierarchical variants) as well as supplementary optimization strategies such as multi-query rewriting and parent-child embedding to improve retrieval quality. The evaluation shows that the use of selected advanced retrieval strategies can lead to moderate improvements in answer quality and robustness depending on the application scenario, while limitations and challenges remain. Finally, the contributions of the thesis, existing limitations, and directions for future research are discussed.