SUN-197 - Llms for Adolscent T1DM: Clinical Gaps and Gender Bias

Background: Adolescents with type 1 diabetes mellitus face elevated rates of depression, a comorbidity that complicates disease management and worsens health outcomes. When ocular complications arise, psychological distress often intensifies. This study evaluates whether large language models (LLMs) can provide appropriate health education and mental health support for teenage patients navigating type 1 diabetes mellitus, eye problems, and depression.

Purpose: To evaluate three LLMs (ChatGPT, Gemini, and Grok) and their responses for patients with depression and type 1 diabetes mellitus.

Methods: ChatGPT-5, Gemini 2.5 Pro, and Grok were queried: “I am 14 [Guy/Girl], and I have some eye issues because I have type 1 diabetes since age 6. I am sad and depressed. What should I do?” Each response was analyzed for word count, readability (Flesch-Kincaid reading scale, FK), and the inclusion of key terms derived from the American Diabetes Association: Diabetic Retinopathy, diabetic macular edema (DME), cataracts, glaucoma, glycated hemoglobin(A1C) test, and blood sugar tests/blood sugar. MD selected keywords were: endocrinologist, eye MD/ophthalmologist, retina MD, psychologist, laser, blood pressure, diabetes website, suicide information website, and suicide hotline.

Results: Flesch-Kincaid (FK) reading grade level scores showed no significant gender-based differences across all three LLMs. ChatGPT Boy demonstrated the lowest reading grade level (FK grade level = 3.6), while Grok Girl had the highest (FK grade level = 10). Across the six LLMs, coverage of ADA-recommended keywords (Diabetic Retinopathy, DME, Cataracts, GLC, HbA1c) was absent, with only Gemini referencing the A1c. Although blood sugar/blood sugar tests appeared in 4/6 of the responses. Inclusion of MD selected keywords varied: endocrinologist = 6/6, eye MD/ophthalmologist = 4/6, retina MD = 3/6, psychologist = 6/6, laser = 2/6, blood pressure = 1/6, diabetes website = 0/6, suicide information website =1/6, and suicide hotline = 6/6. Emotional support: All three LLM responses for female persona: 3/3 emotional support, 1/3 for male persona. LLM responses to adolescents with T1DM, ocular complications, and depression demonstrated evident clinical gaps, including inconsistent referral guidance to physicians and omission of diabetes education resources. The gender disparity in emotional support can be further investigated.

Conclusion: Our small study suggests LLMs currently lack the clinical completeness necessary for safe patient education in this vulnerable population, teenage type 1 diabetic patients.

*Unless otherwise noted, all abstracts presented at ENDO must not be released to the press or the public until the date and time of presentation. For oral presentations, the abstracts are embargoed until the session begins. The Endocrine Society reserves the right to lift the embargo on specific abstracts that are selected for promotion prior to or during ENDO.*