QueryMinds
- #SaaS
An LLM-powered chatbot solution integrating the company’s knowledge base and external sources, ensuring data privacy and accurate responses.
- Machine Learning
- Natural Language Processing
Impact
Our solution offered substantial benefits to the client, including:
- Equipping employees with a tool to efficiently search for answers within the knowledge base, thereby saving time and enhancing worker productivity.
- Cost savings by reducing the time employees spend on information searches
Services we provided
LLM-powered chatbot integrates company knowledge and external data for privacy and precise responses
Tech Stack
Python
Flask
Llama 2
LangChain
Challenges and Solutions
🧐 Challenges
- Creating an LLM-based employee query solution, using the company’s knowledge base and external sources, emphasizing data security and prompt responses
- Tackling data cleanliness, vital due to markup and noise in knowledge base files, ensured accurate LLM-driven responses.
💡 Solutions
To ensure the privacy of the company’s data, the decision was made to use a locally run LLM instead of OpenAI’s GPT models. Llama 2 was chosen due to being the most advanced open-source LLM.
The development of the solution involved:
- Splitting the knowledge base into chunks, which were then encoded and stored in a vector database for semantic search.
- Prompt engineering to utilize LLMs for answering questions in a chat format.
User flow
1. User asks the question in the chat
2. The user’s question get’s encoded and top chunks, semantically matching the user’s question are retrieved
3. The chunks are passed to an LLM, which tries to answer the question
4. The model’s answer appears in the chat, along with buttons leading to sources on which the answer is based on
5. The user can ask further questions to get a more detailed response