Retinal Specialist versus Artificial Intelligence Detection of Retinal Fluid from OCT
December 2020
Keenan TDL, Clemons TE, Domalpally A, Elman MJ, Havilio M, Agrón E, Benyamini G, Chew EY. Retinal Specialist versus Artificial Intelligence Detection of Retinal Fluid from OCT: Age-Related Eye Disease Study 2: 10-Year Follow-On Study. Ophthalmology. 2020 Jun 27:S0161-6420(20)30580-7. doi: 10.1016/j.ophtha.2020.06.038. Epub ahead of print. PMID: 32598950.
Artificial intelligence-based or machine learning analyses are beginning to have an impact on the diagnosis, management, and outcome measurements of retinal diseases. Automated algorithms are attractive because they have the potential to increase the accuracy, consistency, and efficiency of OCT interpretation.
In an effort to better assess one particular software – the Notal OCT Analyzer (NOA, Notal Vision Ltd, Tel Aviv, Israel) – the authors designed this study with two main goals:
- To compare the performance of retina specialists with the performance of the NOA software in identifying the presence of absence of retinal fluid.
- To compare the performance of retina specialists with the performance of the NOA software in separately identifying intraretinal and subretinal fluid.
The “ground truth” to which the performance of the retina specialists and NOA software was compared was the interpretation provided by expert graders at a reading center.
The patients included in this study were derived from the AREDS2 10-year follow-on study (AREDS2-10Y). This subset of patients included 709 participants from 19 sites who had a repeat evaluation 10 years after their initial enrollment in the AREDS2 study (of which there were 4203 total participants). Of these, 1127 eyes were included in the study.
The Cirrus and Spectralis OCT images from these patients were analyzed for the presence or absence of IRF and SRF by:
- Study investigators (i.e. the human retina specialists seeing the clinical trial patients in the clinic)
- NOA software (this software had previously been validated in a separate study, PMID: 27206840).
- Expert graders at the University of Wisconsin Fundus Photography Reading Center (considered the “ground truth”)
The main takeaway points are as follows:
- The accuracy and sensitivity of the NOA software in detecting the presence of retinal fluid and intraretinal fluid was higher than that of the human investigators (retina specialists). For subretinal fluid, the accuracy of human investigators was higher and the sensitivity was lower than the NOA software.
- For detecting the presence of retinal fluid (either IRF or SRF), the following table summarizes the results:
| Accuracy | Sensitivity | Specificity | Prescision | |
| Human investigators | 0.805 | 0.468 | 0.970 | 0.883 |
| NOA Software | 0.851 | 0.822 | 0.865 | 0.749 |
- Human investigators correctly identified retinal fluid in less than half of the cases.
- Human investigators tended to miss retinal fluid in (a) images with lower total fluid volumes and (b) images containing only intraretinal fluid.
- The NOA software tended to miss retinal fluid in (a) images considered more challenging by the reading center (i.e. those that required a senior grader adjudication) and (b) images containing only intraretinal fluid.
This was a robust and well-conducted prospective study featuring masked expert graders from a reading center to whom the artificial intelligence software and human investigators (retina specialists) were compared. It is interesting, and somewhat surprising, to see the low accuracy and sensitivity of retinal specialists in identifying retinal fluid compared to the NOA software. It is not unreasonable, as the authors also postulated, that the real-life performance of other retina specialists in busy clinics may be even lower than the performance of the retina specialists evaluated here who were serving as study investigators in a clinical trial.
The results of this study would suggest that artificial intelligence softwares may potentially serve as valuable tools in enhancing the interpretation of OCT images of wet AMD patients. It may not be long before we find further studies that also reveal that the assistance of artificial intelligence software such as this may perhaps improve the clinical management and outcomes of patients with retinal disease.
