Free papers detail | 11th Asia Pacific Tele Ophthalmology Society Symposium

Title
From visual question answering to intelligent AI agents in ophthalmology

Authors
Xiaolan Chen, Danli Shi, Mingguang He

Presenting
Xiaolan Chen

PURPOSE:
Clinical ophthalmic practice requires integrating diverse multimodal data, yet traditional artificial intelligence (AI) systems struggle to address this complexity. We aim to explore the evolution from visual question answering (VQA) to multimodal AI agents, providing eye care professionals with deeper insights into these models and their applications.

METHODS:
A narrative review was conducted, synthesizing the current literature on ophthalmic conversational AI, including VQA and AI agents, from both theoretical and practical perspectives. The analysis focused on their technical basis, the landscape of available datasets, progress in system development, and potential clinical applications.

RESULTS:
Current VQA systems in ophthalmology show promise in education, clinical support, and patient engagement but are limited by single-turn interactions and static data. AI agents can offer dynamic, context-aware support through tool integration, memory, and continuous dialogue. Meanwhile, large language models (LLMs) can enhance reasoning and interaction within such systems by generating synthetic data, serving as the foundation model for task adaptation, integrating into VQA systems, and acting as the control core of AI agents. Despite notable progress, significant challenges remain, including limited high-quality multimodal datasets, the lack of standardized evaluation frameworks, and barriers to real-world deployment. Advancing the field requires robust, clinically relevant data, multidimensional evaluation, and explainable, workflow-ready AI systems.

RESULTS:

Current VQA systems in ophthalmology show promise in education, clinical support, and patient engagement but are limited by single-turn interactions and static data. AI agents can offer dynamic, context-aware support through tool integration, memory, and continuous dialogue. Meanwhile, large language models (LLMs) can enhance reasoning and interaction within such systems by generating synthetic data, serving as the foundation model for task adaptation, integrating into VQA systems, and acting as the control core of AI agents. Despite notable progress, significant challenges remain, including limited high-quality multimodal datasets, the lack of standardized evaluation frameworks, and barriers to real-world deployment. Advancing the field requires robust, clinically relevant data, multidimensional evaluation, and explainable, workflow-ready AI systems.

CONCLUSIONS:
While recent advances in VQA and multimodal AI agents are encouraging, their application in ophthalmology remains largely experimental. Addressing these gaps through close collaboration between AI researchers and the ophthalmic community may pave the way for systems that enhance access to and quality of eye care worldwide.