Email: [email protected]
The goal of MediSyn is to design, develop, and validate a multi-agent chatbot system capable of simulating educational conversations representing diverse team members in typical medical and public health settings. The MediSyn project seeks to enhance Interprofessional Education (IPE) experiences, which foster collaboration and mutual learning among students from multiple professional domains. Key objectives include:
- Customizing an open-source LLM (e.g., LLaMA3) using Retrieval-Augmented Generation (RAG) and/or fine-tuning techniques (e.g., LoRA) to create chatbot agents tailored for IPE.
- Evaluating the performance of the chatbot system (e.g., adherence to professional guidelines, quality of interaction, and depth of knowledge representation).
- Conducting usability and user interface (UI) testing to improve learner experiences.
- Optionally integrating Learning Tools Interoperability (LTI) features to seamlessly connect the system with Learning Management Systems (LMS).
Figure 1: Concept visualization
The dataset for fine-tuning and customization will include open-source academic texts, dialogue datasets, and domain-specific medical and public health content to build contextually accurate chatbot agents. Specific datasets are to be determined based on project needs.
Generative AI offers transformative potential for IPE by addressing challenges such as limited diversity among team participants in training scenarios. A multi-agent chatbot system can simulate diverse roles within interprofessional teams, enhancing the realism and inclusivity of educational interventions. This approach has the potential to improve both the delivery of learning experiences and the outcomes for students as they transition to real-world professional environments.
The project is structured into several key phases:
- Requirement Analysis: Collaborate with researchers to define core IPE components and chatbot performance criteria.
- Development: Customize one or more open-source LLMs to develop a prototype system using RAG and fine-tuning techniques.
- Validation: Conduct multi-agent testing to evaluate performance metrics, including adherence to professional guidelines, domain accuracy, and conversational depth.
- Usability Testing: Assess user experience with focus on satisfaction, ease of use, and educational impact.
- Integration (Optional): Implement LTI features for compatibility with existing LMS platforms.
The estimated timeline for MediSyn development:
- Weeks 1-2: Initial requirement analysis and tool selection.
- Weeks 3-8: Prototype design and development.
- Weeks 9-12: Iterative testing and refinement based on feedback.
- Weeks 13-16: Multi-agent evaluation and usability testing.
- Weeks 17-18: Final reporting, documentation, and presentation.
A team of 2-3 students is ideal for the scope and interdisciplinary nature of the project.
Potential issues and their mitigations include:
- Limited Training Data: Locating relevant datasets for customization and RAG will require careful research.
- Interdisciplinary Complexity: Developing effective prompts and designing interactions to represent diverse professional roles will require iterative refinement.
- Cross-Disciplinary Collaboration: Ensuring smooth communication between team members from various fields is crucial to project success.
- Author: Amir Jafari
- Email: [email protected]
- GitHub: https://github.com/amir-jafari/Capstone