December 2020
Keenan TDL, Clemons TE, Domalpally A, Elman MJ, Havilio M, Agrón E, Benyamini G, Chew EY. Retinal Specialist versus Artificial Intelligence Detection of Retinal Fluid from OCT: Age-Related Eye Disease Study 2: 10-Year Follow-On Study. Ophthalmology. 2020 Jun 27:S0161-6420(20)30580-7. doi: 10.1016/j.ophtha.2020.06.038. Epub ahead of print. PMID: 32598950.
Artificial intelligence-based or machine learning analyses are beginning to have an impact on the diagnosis, management, and outcome measurements of retinal diseases. Automated algorithms are attractive because they have the potential to increase the accuracy, consistency, and efficiency of OCT interpretation.
In an effort to better assess one particular software – the Notal OCT Analyzer (NOA, Notal Vision Ltd, Tel Aviv, Israel) – the authors designed this study with two main goals:
The “ground truth” to which the performance of the retina specialists and NOA software was compared was the interpretation provided by expert graders at a reading center.
The patients included in this study were derived from the AREDS2 10-year follow-on study (AREDS2-10Y). This subset of patients included 709 participants from 19 sites who had a repeat evaluation 10 years after their initial enrollment in the AREDS2 study (of which there were 4203 total participants). Of these, 1127 eyes were included in the study.
The Cirrus and Spectralis OCT images from these patients were analyzed for the presence or absence of IRF and SRF by:
The main takeaway points are as follows:
Accuracy | Sensitivity | Specificity | Prescision | |
Human investigators | 0.805 | 0.468 | 0.970 | 0.883 |
NOA Software | 0.851 | 0.822 | 0.865 | 0.749 |
This was a robust and well-conducted prospective study featuring masked expert graders from a reading center to whom the artificial intelligence software and human investigators (retina specialists) were compared. It is interesting, and somewhat surprising, to see the low accuracy and sensitivity of retinal specialists in identifying retinal fluid compared to the NOA software. It is not unreasonable, as the authors also postulated, that the real-life performance of other retina specialists in busy clinics may be even lower than the performance of the retina specialists evaluated here who were serving as study investigators in a clinical trial.
The results of this study would suggest that artificial intelligence softwares may potentially serve as valuable tools in enhancing the interpretation of OCT images of wet AMD patients. It may not be long before we find further studies that also reveal that the assistance of artificial intelligence software such as this may perhaps improve the clinical management and outcomes of patients with retinal disease.
Mohsin H. Ali, MD
The Retina Group of Washington
Sterling & Reston, VA