Funding period 03/2021 – 03/2024
Ophthalmo-AI focuses on the two most common causes of blindness in people aged 50 and older: Age-related macula degeneration (AMD) and diabetic retinopathy (DR). An intelligent assistance system is under development to support ophthalmologists using image data and clinical data to reach a correct medical diagnosis and determine the best possible treatment.
Age-related macula degeneration is a result of the aging process in the cells of the central retina (macula = point of sharpest vision). As every person ages, cellular debris accumulates in this area and, sometimes, irregular blood vessels grow from the choroid behind the retina. Fluids and blood can leak from them and cause the pathology of “distortion” and progressive vision loss (“wet” form of “late” AMD).
Diabetic retinopathy can occur as a result of diabetes. Similarly, blood components can leak into the retinal tissue, causing pathological vascular neoplasms – so-called “proliferations” – to form on the retina. These are very vulnerable vessels that can cause strong hemorrhages in the eye, which may result in retinal detachment. Diabetic retinopathy often leads to edema (= swelling) in the area of the macula, a so-called „diabetic maculopathy,“ which causes vision problems.
Optical Coherence Tomography, (= cross-sectional imag ing of the retina) is able to detect the smallest changes and swellings in the area of the macula, such as the presence of irregular blood vessels. This non-invasive procedure is essential in the diagnosis and monitoring of the treatments of both diseases. New drugs available today suppress the growth of abnormal blood vessels and reduce the swelling of the macula. These drugs are injected directly into the vitreous cavity in a procedure called intravitreal injection of medicine, or IVOM. IVOM must be repeated at varying intervals, usually monthly, until the patient‘s vision stabilizes.
The first phase of the project involved marking relevant information in the patient files and images of the retinal structures for later machine processing using tools developed in-house. An existing DFKI tool helps to minimize the time required for these complex annotations and is used to support physicians with segmentation models and special graphical algorithms. In addition, Active Learning algorithms select the image data that promises the greatest potential benefit to the continuous improvement of the models. The resulting neural networks are used to support diagnoses and therapy recommendations and to predict the success of the therapy. To enable physicians to understand the AI‘s decision, in addition to assessing the presence of disease, the system provides explanations of the results, in the form of interactive segmentation maps of the features relevant to the diagnosis.
The Augmented Intelligence system developed by the research areas Interactive Machine Learning (IML) and Cognitive Assistants (COS) is tested in clinical demonstrators for practical use. In addition to the human involvement in the annotation of input data, a second aspect of the human-in-the-loop concept is applicable: physicians provide constructive feedback, for example, through corrections to the segmentation maps, which permits continuous improvement of the models.
The feedback of expert knowledge into the models for diagnosis and therapy recommendation by means of interactive machine learning represents the major scientific innovation of Ophthalmo-AI. Until now, no market-ready system existed in medical imaging that can combine all aspects. State-of-the-art systems for visualization and explanation help to make diagnoses and recommended therapies of the underlying ML model more transparent. Physician participation in improving diagnosis and therapy forecasts is instrumental in enhancing the transparency of the ML model as well as in increasing the general acceptance of the system.
References
Kadir, Md Abdul, Hasan Md Tusfiqur Alam, and Daniel Sonntag. “EdgeAL: An Edge Estimation Based Active Learning Approach for OCT Segmentation.” arXiv preprint arXiv:2307.10745 (2023).
Duy M. H. Nguyen, Hoang Nguyen, Mai T. N. Truong, Tri Cao, Binh T. Nguyen, Nhat Ho, Paul Swoboda, Shadi Albarqouni, Pengtao Xie, Daniel Sonntag: Joint Self-Supervised Image-Volume Representation Learning with Intra-inter Contrastive Clustering. Proceedings of the thirty-seventh AAAI Conference on Artifical Intelligence (2023).
Hasan Md Tusfiqur Alam, Duy M. H. Nguyen, Mai Thanh Nhat Truong, Triet A. Nguyen, Binh T. Nguyen, Michael Barz, Hans-Jürgen Profitlich, Ngoc T. T. Than, Ngan Le, Pengtao Xie, Daniel Sonntag:
DRG-Net: Interactive Joint Learning of Multi-lesion Segmentation and Classification for Diabetic Retinopathy Grading. CoRR abs/2212.14615 (2022).
Duy M. H. Nguyen, Mai Thanh Nhat Truong, Ngoc T. T. Than, Alexander Prange, Daniel Sonntag: Self-Supervised Domain Adaptation for Diabetic Retinopathy Grading using Vessel Image Reconstruction. Proceedings of the 44th German Conference on Artificial Intelligence (2021).
Sponsored by BMBF – Federal Ministry of Education and Research
