P-145: Innovative AI-Driven Decision Support Tool for Multiple Myeloma using Retrieval Augmented Generation

Introduction: Managing Multiple Myeloma (MM) requires precise and current treatment information, especially given the rapid advances in genomic research. Existing decision-support tools often lack the latest research and personalized insights. To address this, we developed an advanced Retrieval-Augmented Generation (RAG)-based chatbot uniquely designed for MM management. This tool uses RAG, which combines information retrieval with text generation, leveraging the largest multimodal literature repository and integrating proprietary data and databases like Semantic Scholar.

Methods: Our chatbot utilizes the unstructr.io framework to systematically analyze and manage over 5 million MM-specific research papers dating from 1932 to 2024. A continuously updated large language model (LLM)-based web scraper populates our repository with the latest studies. The chatbot employs the DSPy framework to dynamically generate and optimize queries, using reasoning loops that reflect physician-like cognitive processes. The Reflexion framework enhances learning from interaction histories, improving response accuracy. The chatbot can address specific queries posed by oncologists, such as those related to rare genetic variants or complex treatment scenarios.

Results: In a benchmark study comparing the RAG chatbot to existing medical AI platforms, including MedPalm by Google, our tool showed superior performance in several areas. It delivered precise medical recommendations within seconds, reducing average research time by 30% compared to other MM-specific applications. The chatbot achieved a high relevance score as evaluated by MM specialists, providing deep dives into patient-specific genomic data that influenced treatment decisions. It also exhibited a significantly faster adaptation to new MM research, integrating the latest findings and clinical trials into its recommendations.

For complex queries involving patients with relapsed MM resistant to standard treatments and specific genetic mutations, the chatbot explored the implications of these genetic changes on standard therapies, suggested potential experimental treatments or clinical trials, and assessed the potential benefits and limitations of combining different targeted therapies.

Unlike general models like ChatGPT, our chatbot continuously updates its repository with the latest research and clinical data while implementing strategies to mitigate data misinterpretation, ensuring more accurate, context-specific, and reliable recommendations.

Conclusions: Our chatbot offers clear advantages over generic LLMs and lays the groundwork for enhanced precision medicine. By tailoring responses to unique genetic and clinical profiles, it will improve clinical decision-making and patient care. Fine-tuning it on genomic and clinical data allows the chatbot to analyze patient data and prioritize findings based on the latest literature. Additionally, this system can serve as a foundation for tools that educate patients about their condition and treatment options.