MON-590 - Can Clinicians Build Acceptable AI Tools for Diabetes Education?

Background: Patients with diabetes frequently encounter medical information that is complex and difficult to understand. While large language models (LLMs) show promise for health education, concerns regarding errors of omission and lack of clinician oversight limit their use in healthcare settings. We created a clinician-guided system to deliver controlled AI-generated diabetes education.

Purpose: To assess perceived utility of a MD-created chatbot for diabetes education.

Methods: We created a web-based chatbot using a retrieval-augmented generation (RAG) architecture trained exclusively on patient education material from https://diabetes.org/ from the “About Diabetes,” “Life with Diabetes,” “Health & Wellness,” and “Food & Nutrition” sections. Texts were segmented into chunks, embedded using the Gemini text-embedding-004 model, and stored in a local vector database. User queries were answered using Gemini 2.5 Flash. Responses were limited to (1) information from the website, (2) respond at 6th-grade reading level, (3) avoid diagnosing or giving personal advice, (4) and emphasize diabetes-related eye examinations, kidney monitoring, foot evaluation, and cardiovascular risk.

In a pilot study, 100 patients interacted with the chatbot for 2 minutes and completed a post-interaction survey assessing response clarity, ease of understanding, and confidence indentifying next steps.

Results: Patients reported high clarity and favorable acceptability of a clinician-guided diabetes education chatbot. Mean self-reported baseline diabetes knowledge was 2.88 ± 1.17 (1 = not confident at all, 5 = very confident). Mean clarity and ease of understanding of responses was 4.05 ± 0.89 (1 = not clear at all, 5 = very clear). Mean confidence in identifying “next steps” after interaction was 3.49 ± 1.02 (1 = not confident at all, 5 = very confident).

Conclusion: These findings for the small study suggest that constrained, clinician-designed AI tools may be acceptable to patients and useful for supporting diabetes education. Further evaluation in larger and more diverse populations is needed.

*Unless otherwise noted, all abstracts presented at ENDO must not be released to the press or the public until the date and time of presentation. For oral presentations, the abstracts are embargoed until the session begins. The Endocrine Society reserves the right to lift the embargo on specific abstracts that are selected for promotion prior to or during ENDO.*