Lausanne University Hospital and University of Lausanne, Switzerland
Department of Radiology

Author Of 1 Presentation

Machine Learning/Network Science Oral Presentation

PS16.04 - RimNet: A deep 3D multimodal MRI architecture for paramagnetic rim lesions assessment in multiple sclerosis

Presentation Number
Presentation Topic
Machine Learning/Network Science
Lecture Time
13:27 - 13:39



In multiple sclerosis (MS), perilesional chronic inflammation appears on in vivo 3T susceptibility-based magnetic resonance imaging (MRI) as non-gadolinium-enhancing paramagnetic rim lesions (PRL). A higher PRL burden has been recently associated with a more aggressive disease course. The visual detection of PRL by experts is time-consuming and can be subjective.


To develop a multimodal convolutional neural network (CNN) capable of automatically detecting PRL on 3D-T2*w-EPI unwrapped phase and 3D-T2w-FLAIR images.


124 MS cases (87 relapsing remitting MS, 16 primary progressive MS and 21 secondary progressive MS) underwent 3T MRI (MAGNETOM Prisma and MAGNETOM Skyra, Siemens Healthcare, Erlangen, Germany). Two neurologists visually inspected FLAIR magnitude and EPI phase images and annotated 462 PRL. 4857 lesions detected by an automatic segmentation (La Rosa et al. 2019) without overlap with PRL were considered non-PRL. The prototype RimNet was built upon two single CNNs, each fed with 3D patches centered on candidate lesions in phase and FLAIR images, respectively. A two-step feature-map fusion, initially after the first convolutional block and then before the fully connected layers, enhances the extraction of low and high-level multimodal features. For comparison, two unimodal CNNs were trained with phase and FLAIR images. The areas under the ROC curve (AUC) were used for evaluation (DeLong et al. 1988). The operating point was set at a lesion-wise specificity of 0.95. The patient-wise assessment was conducted by using a clinically relevant threshold of four rim+ lesions per patient (Absinta et al. 2019).


RimNet (AUC=0.943) outperformed the phase and FLAIR image unimodal networks (AUC=0.913 and 0.855, respectively, P’s <0.0001). At the operating point, RimNet showed higher lesion-wise sensitivity (70.6%) than the unimodal phase network (62.1%), but lower than the experts (77.7%). At the patient level, RimNet performed with sensitivity of 86.8% and specificity of 90.7%. Individual expert ratings yielded averaged sensitivity and specificity values of 76.3% and 99.4%, respectively.


The excellent performance of RimNet supports its further development as an assessment tool to automatically detect PRL in MS. Interestingly, the unimodal FLAIR network performed reasonably well despite the absence of a paramagnetic rim, suggesting that morphometric features such as volume or shape might be a distinguishable feature of PRL.