A top 50 startup, serving millions of users, develops personalized and contextual RAG chatbots that can participate in group conversations, with each chatbot conversing as a specific persona. Invisible performed evaluations and provided improved search prompting, enabling the client to enhance model performance and context efficiently.
Conversations Evaluated per Week: 300
The client faced a critical issue: their chatbots, while performing well in one-on-one chats, struggled in group settings. Customers noticed that the personas of these chatbots would shift unpredictably, creating confusion and undermining trust.
The client needed a strategic partner—one that could scale, ensure quality, and provide precision in assessing and refining chatbot responses.
Invisible's team worked within the client's platform to review model conversations from the research team and assess the level of accuracy of the model responses. We evaluated conversations for flow and factual accuracy, and provided revised search prompting.
These prompts were especially important and niche for the client - prompts had to match the tone and persona associated with their chatbots. Subsequent model responses were improved by integrating these new search results.
By guiding the AI to seamlessly incorporate search results, the client’s RAG chatbot transformed the quality of group chats—making them more engaging and deeply relevant.
"Invisible is flexible and attentive to our needs. They have been close working partners and have been able to work closely with us to improve processes and annotation outputs."
- Head of Trust and Safety
