Expert level of detection of interictal discharges with a deep neural network

Abstract

Objective: Deep learning methods have shown potential in automating the detection of interictal epileptiform discharges (IEDs) in electroencephalography (EEG). We compared IED detection using our previously trained deep neural network with a group of experts to assess its potential applicability.

Methods: First, we performed clinical validation on an internal data set. Seven experts reviewed all EEG studies. Performance agreement between experts and the network was compared at both the EEG and IED levels. All EEG recordings were also processed with Persyst. Subsequently, we performed external validation, with data from four centers, using a hybrid approach, where detections by the deep neural network were reviewed by an expert. In case of disagreement with the original report, the EEG recording was annotated independently by five experts.

Results: For internal validation we included 22 EEG studies with IEDs and 28 EEG studies from controls. At the EEG level, our network showed performance similar to that of the experts. For individual IED detection, the sensitivities between experts ranged from 20.7%-86.4%, whereas the sensitivity of our network was 82.5% (confidence interval [CI]: 77.7%-87.4%) at 99% specificity and a false detection rate (FDR) of <.2/min, outperforming Persyst, with 64.6% sensitivity (CI: 61.4%-67.9%) at 98% specificity. External validation in 174 EEG studies demonstrated that all 85 EEG recordings classified as normal in the original report were classified correctly, with an FDR of .10/min. Of the 89 EEG studies with IEDs according to the report, 56 were correctly classified (Cohen's κ = .62). Visual analysis of the remaining 33 EEG recordings showed high interobserver variability among the five experts (Fleiss' κ = .13).

Significance: Our deep neural network detects IEDs on par with clinical experts. The external validation in a hybrid approach showed substantial agreement with the original report. Disagreement was due mainly to high interobserver variability. Our deep neural network may support visual EEG analysis and assist in diagnostics, particularly when human resources are limited.

Keywords: clinical validation; deep learning; epilepsy; interictal epileptiform discharge.