Annals of Clinical and Experimental NeurologyAnnals of Clinical and Experimental NeurologyАнналы клинической и экспериментальной неврологии2075-54732409-2533Eco-Vector64710.54101/ACEN.2021.4.6ReviewsОбзорыReview ArticleThe quality of artificial intelligence algorithms for identifying manifestations of multiple sclerosis on magnetic resonance imaging (systematic review)Качество алгоритмов искусственного интеллекта для выявления признаков рассеянного склероза на магнитно-резонансных томограммах (систематический обзор)ChernyaevaGalina N.ЧерняеваГалина Николаевна
Russian Federation

junior researcher

м.н.с.

a.vladzimirsky@npcmr.ruhttps://orcid.org/0000-0002-5066-5997https://orcid.org/0000-0001-6545-6170MorozovSergey P.МорозовСергей Павлович
Russian Federation

D. Sci. (Med), Prof., Director

д.м.н., проф., директор

a.vladzimirsky@npcmr.ruhttps://orcid.org/0000-0001-6545-6170https://orcid.org/0000-0002-2990-7736VladzimirskyyAnton V.ВладзимирскийАнтон Вячеславович
Russian Federation

D. Sci. (Med), Deputy Director for R&D

д.м.н., заместитель директора по научной работе

a.vladzimirsky@npcmr.ruhttps://orcid.org/0000-0002-2990-7736Research and Practical Clinical Center for Diagnostics and Telemedicine TechnologiesГБУЗ г. Москвы «Научно-практический клинический центр диагностики и телемедицинских технологий Департамента здравоохранения Москвы»I.M. Sechenov First Moscow State Medical University (Sechenov University)ФГАОУ ВО «Первый Московский государственный медицинский университет имени И.М. Сеченова» (Сеченовский Университет)23122021154546507052020Copyright ©; 2021, Chernyaeva G.N., Morozov S.P., Vladzimirskyy A.V.Copyright ©; 2021, Черняева Г.Н., Морозов С.П., Владзимирский А.В.2021Chernyaeva G.N., Morozov S.P., Vladzimirskyy A.V.Черняева Г.Н., Морозов С.П., Владзимирский А.В.https://creativecommons.org/licenses/by/4.0https://annaly-nevrologii.com/pathID/article/view/647

A systematic review was undertaken to summarize the data regarding accuracy and effectiveness of artificial intelligence algorithms for identifying MRI manifestations of multiple sclerosis. The review included 39 papers, whose authors put forth a multitude of corresponding algorithms and mathematical models. However, quality assessment of these developments was limited by retrospective testing on repeat data sets. Clinical test results were almost entirely absent, and there were no prospective independent studies of accuracy and applicability. The relatively high values obtained for the main measures (similarity, sensitivity and specificity coefficients, which were 75–85%) were offset by the methodological errors when creating the baseline data sets, and lack of validation using independent data. Due to small sample sizes and methodological errors when measuring the result accuracy, most of the studies did not meet the criteria for evidence-based research. Studies with the highest methodological quality had algorithms that achieved a sensitivity of 51.6–77.0%, with a Sørensen–Dice coefficient of 53.5–56.0%. These numbers are not high, but they indicate that automatic identification of multiple sclerosis manifestations on magnetic resonance imaging may be achievable. Further development of computer-aided analysis requires the creation of clinical use scenarios and testing methodology, and prospective clinical testing.

Выполнен систематический обзор литературы с целью обобщения данных о точности и результативности применения алгоритмов искусственного интеллекта для выявления рассеянного склероза по результатам магнитно-резонансной томографии. В обзор включены 39 статей, авторами которых предложено большое количество соответствующих алгоритмов и математических моделей. Однако оценка качества таких разработок ограничена ретроспективным тестированием на повторяющихся наборах данных. Практически полностью отсутствуют результаты клинической апробации, нет проспективных независимых научных исследований точности и применимости. Довольно высокие уровни основных метрик (коэффициенты сходства, чувствительность, специфичность — 75–85%) нивелируются методическими ошибками при формировании исходных наборов данных, отсутствием валидации на независимых данных. В силу малых объёмов выборок и методических дефектов оценки точности результаты подавляющего большинства статей не отвечают критериям доказательности. В наиболее качественных, с методической точки зрения, исследованиях достигнута чувствительность алгоритмов 51,6–77,0%, значение коэффициента Дайса–Сёренсена — 53,5–56,0%. Значение невысоки, но они свидетельствуют о потенциальной реализуемости задачи автоматизированного выявления признаков рассеянного склероза на магнитно-резонансных томограммах. Для дальнейшего развития автоматизированного анализа требуется разработка клинических сценариев применения, формирование методологии тестирования, проведение проспективных клинических апробаций.

multiple sclerosisartificial intelligencemachine learningmagnetic resonance imagingрассеянный склерозискусственный интеллектмашинное обучениемагнитно-резонансная томографияКремнева Елена Игоревна, ФГБНУ «Научный центр неврологии»Abdurakhmanova R.F., Izzatov Kh.N., Khadibaeva G.R. et al. Multiple sclerosis: etiology, pathogenesis and clinics (part I). Vestnik poslediplomnogo obrazo- vaniya v sfere zdravookhraneniya. 2016;(3):68-74. (In Russ).Абдурахманова Р.Ф., Иззатов Х.Н., Хадибаева Г.Р., Шарипова Б.А., Кахарова М.Х. Рассеянный склероз: этиология, патогенез и клиника (часть I). Вестник последипломного образования в сфере здравоохранения. 2016;(3):68–74.Popova E.V., Boyko A.N., Barabanova M.A. et al. Primary progressive multiple sclerosis: current issues of timely diagnosis. Zhurnal nevrologii i psikhiatrii im. S.S. Korsakova. 2017;117(10-12):35-40. (In Russ). DOI: 10.17116/jnev- ro201711710235-40.Попова Е.В., Бойко А.Н., Барабанова М.А. и соавт. Первично-прогресси-рующий рассеянный склероз: современное состояние проблемы своевременной постановки диагноза. Журнал неврологии и психиатрии им. C.C. Корсакова. 2017;117(10-12):35–40. DOI: 10.17116/jnevro201711710235-40.Howard J., Trevick S., Younger D.S. Epidemiology of multiple sclerosis. Neurol Clin. 2016;34(4):919-939. DOI: 10.1016/j.ncl.2016.06.016. PMID: 27720001.Howard J., Trevick S., Younger D.S. Epidemiology of multiple sclerosis. Neurol Clin. 2016;34(4):919–939. DOI: 10.1016/j.ncl.2016.06.016. PMID: 27720001.Borodin A.V. Differential diagnostics of multiple sclerosis. Prakticheskaya meditsina. 2018;(10):59-63. (In Russ).Бородин А.В. Дифференциальная диагностика рассеянного склероза. Практическая медицина. 2018;(10):59–63.Siva A. Common clinical and imaging conditions misdiagnosed as multiple sclerosis: a current approach to the differential diagnosis of multiple sclerosis. Neurol Clin. 2018;36(1):69-117. DOI: 10.1016/j.ncl.2017.08.014. PMID: 29157405.Siva A. Common clinical and imaging conditions misdiagnosed as multiple sclerosis: a current approach to the differential diagnosis of multiple sclerosis. Neurol Clin. 2018;36(1):69–117. DOI: 10.1016/j.ncl.2017.08.014. PMID: 29157405.Solomon A.J., Naismith R.T., Cross A.H. Misdiagnosis of multiple sclerosis: impact of the 2017 McDonald criteria on clinical practice. Neurology. 2019;92(1):26-33. DOI: 10.1212/WNL.0000000000006583. PMID: 30381369.Solomon A.J., Naismith R.T., Cross A.H. Misdiagnosis of multiple sclerosis: impact of the 2017 McDonald criteria on clinical practice. Neurology. 2019;92(1):26–33. DOI: 10.1212/WNL.0000000000006583. PMID: 30381369.Morozov S.P., Vladzymyrsky A.V., Klyashtorny V.G. et al. Clinical studies of intelligence technology-based software (radiology). Moscow, 2019. 33 p. (In Russ).Морозов С.П., Владзимирский А.В., Кляшторный В.Г. и др. Клинические испытания программного обеспечения на основе интеллектуальных технологий (лучевая диагностика). М., 2019. 33 с.Udupa J.K., Wei L., Samarasekera S., Miki Y. et al. Multiple sclerosis lesion quantification using fuzzy-connectedness principles. IEEE Trans Med Imaging. 1997;16(5):598-609. DOI: 10.1109/42.640750. PMID: 9368115.Udupa J.K., Wei L., Samarasekera S., Miki Y. et al. Multiple sclerosis lesion quantification using fuzzy-connectedness principles. IEEE Trans Med Imaging. 1997;16(5):598–609. DOI: 10.1109/42.640750. PMID: 9368115.Datta S., Sajja B.R., He R. et al. Segmentation and quantification of black holes in multiple sclerosis. Neuroimage. 2006;29(2):467-474. DOI: 10.1016/j. neuroimage.2005.07.042. PMID: 16126416.Datta S., Sajja B.R., He R. et al. Segmentation and quantification of black holes in multiple sclerosis. Neuroimage. 2006;29(2):467–474. DOI: 10.1016/j.neuroimage.2005.07.042. PMID: 16126416.He R., Sajja B.R., Narayana P.A. Implementation of high-dimensional feature map for segmentation of MR images. Ann Biomed Eng. 2005;33(10):1439- 1448. DOI: 10.1007/s10439-005-5888-3. PMID: 16240091.He R., Sajja B.R., Narayana P.A. Implementation of high-dimensional feature map for segmentation of MR images. Ann Biomed Eng. 2005;33(10):1439–1448. DOI: 10.1007/s10439-005-5888-3. PMID: 16240091.Kawa J., Pietka E. Kernelized fuzzy c-means method in fast segmentation of demyelination plaques in multiple sclerosis. Conf Proc IEEE Eng Med Biol Soc. 2007;2007:5616-5619. DOI: 10.1109/IEMBS.2007.4353620. PMID: 18003286.Kawa J., Pietka E. Kernelized fuzzy c-means method in fast segmentation of demyelination plaques in multiple sclerosis. Conf Proc IEEE Eng Med Biol Soc. 2007;2007:5616–5619. DOI: 10.1109/IEMBS.2007.4353620. PMID: 18003286.Ait-Ali L.S., Prima S., Hellier P. et al. STREM: a robust multidimensional parametric method to segment MS lesions in MRI. Med Image Comput Comput Assist Interv. 2005; 8(Pt 1):409-416. DOI: 10.1007/11566465_51. PMID: 16685872.Aït-Ali L.S., Prima S., Hellier P. et al. STREM: a robust multidimensional parametric method to segment MS lesions in MRI. Med Image Comput Comput Assist Interv. 2005; 8(Pt 1):409–416. DOI: 10.1007/11566465_51. PMID: 16685872.Raff U., Newman F.D. Automated lesion detection and lesion quantitation in MR images using autoassociative memory. Med Phys. 1992;19(1):71-77. DOI: 10.1118/1.596853. PMID: 1620061.Raff U., Newman F.D. Automated lesion detection and lesion quantitation in MR images using autoassociative memory. Med Phys. 1992;19(1):71–77. DOI: 10.1118/1.596853. PMID: 1620061.Raff U., Newman F.D. Lesion detection in radiologic images using an autoassociative paradigm: preliminary results. Med Phys. 1990; 17(5):926-928. DOI: 10.1118/1.596449. PMID: 2233581.Raff U., Newman F.D. Lesion detection in radiologic images using an autoassociative paradigm: preliminary results. Med Phys. 1990; 17(5):926–928. DOI: 10.1118/1.596449. PMID: 2233581.Chase H.S., Mitrani L.R., Lu G.G., Fulgieri D.J. Early recognition of multiple sclerosis using natural language processing of the electronic health record. BMC Med Inform Decis Mak. 2017;17(1):24. DOI: 10.1186/s12911-017-0418-4.Nelson R.E., Butler J., LaFleur J. et al. Determining multiple sclerosis phenotype from electronic medical records. J Manag Care Spec Pharm. 2016;22(12):1377-1382. DOI: 10.18553/jmcp.2016.22.12.1377. PMID: 27882837.Nelson R.E., Butler J., LaFleur J. et al. Determining multiple sclerosis phenotype from electronic medical records. J Manag Care Spec Pharm. 2016;22(12):1377–1382. DOI: 10.18553/jmcp.2016.22.12.1377. PMID: 27882837.Davis M.F., Sriram S., Bush W.S. et al. Automated extraction of clinical traits of multiple sclerosis in electronic medical records. J Am Med Inform Assoc. 2013;20(e2):e334-e340. DOI: 10.1136/amiajnl-2013-001999. PMID: 24148554.Davis M.F., Sriram S., Bush W.S. et al. Automated extraction of clinical traits of multiple sclerosis in electronic medical records. J Am Med Inform Assoc. 2013;20(e2):e334–e340. DOI: 10.1136/amiajnl-2013-001999. PMID: 24148554.Wottschel V., Alexander D.C., Kwok P.P. et al. Predicting outcome in clinically isolated syndrome using machine learning. Neuroimage Clin. 2014;7:281-287. DOI: 10.1016/j.nicl.2014.11.021. PMID: 25610791.Wottschel V., Alexander D.C., Kwok P.P. et al. Predicting outcome in clinically isolated syndrome using machine learning. Neuroimage Clin. 2014;7:281–287. DOI: 10.1016/j.nicl.2014.11.021. PMID: 25610791.Zhao Y., Healy B.C., Rotstein D. et al. Exploration of machine learning techniques in predicting multiple sclerosis disease course. PLoS One. 2017;12(4):e0174866. DOI: 10.1371/journal.pone.0174866. PMID: 28379999.Wu F.Y., Slater J.D., Honig L.S. et al. A neural network design for event-related potential diagnosis. Comput Biol Med. 1993;23(3):251-264. DOI: 10.1016/0010-4825(93)90024-u. PMID: 8334865.Wu F.Y., Slater J.D., Honig L.S. et al. A neural network design for event-related potential diagnosis. Comput Biol Med. 1993;23(3):251–264. DOI: 10.1016/0010-4825(93)90024-u. PMID: 8334865.Ahmadi A., Davoudi S., Daliri M.R. Computer Aided Diagnosis System for multiple sclerosis disease based on phase to amplitude coupling in covert visual attention. Comput Methods Programs Biomed. 2019;169:9-18. DOI: 10.1016/j. cmpb.2018.11.006. PMID: 30638593.Ahmadi A., Davoudi S., Daliri M.R. Computer Aided Diagnosis System for multiple sclerosis disease based on phase to amplitude coupling in covert visual attention. Comput Methods Programs Biomed. 2019;169:9–18. DOI: 10.1016/j.cmpb.2018.11.006. PMID: 30638593.Kiiski H., Jollans L., Donnchadha S.O. et al. Machine learning EEG to predict cognitive functioning and processing speed over a 2-year period in multiple sclerosis patients and controls. Brain Topogr. 2018;31(3):346-363. DOI: 10.1007/s10548-018-0620-4. PMID: 29380079.Kiiski H., Jollans L., Donnchadha S.Ó. et al. Machine learning EEG to predict cognitive functioning and processing speed over a 2-year period in multiple sclerosis patients and controls. Brain Topogr. 2018;31(3):346–363. DOI: 10.1007/s10548-018-0620-4. PMID: 29380079.Liberati A., Altman D.G., Tetzlaff J. et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1- e34. DOI: 10.1016/j.jclinepi.2009.06.006. PMID: 19631507.Liberati A., Altman D.G., Tetzlaff J. et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1–e34. DOI: 10.1016/j.jclinepi.2009.06.006. PMID: 19631507.Wang S.H., Tang C., Sun J. et al. Multiple sclerosis identification by 14-layer convolutional neural network with batch normalization, dropout, and stochastic pooling. Front Neurosci. 2018;12:818. DOI: 10.3389/fnins.2018.00818. PMID: 30467462.Zurita M., Montalba C., Labbe T. et al. Characterization of relapsing-remitting multiple sclerosis patients using support vector machine classifications of functional and diffusion MRI data. Neuroimage Clin. 2018;20:724-730. DOI: 10.1016/j.nicl.2018.09.002. PMID: 30238916.Zurita M., Montalba C., Labbé T. et al. Characterization of relapsing-remitting multiple sclerosis patients using support vector machine classifications of functional and diffusion MRI data. Neuroimage Clin. 2018;20:724–730. DOI: 10.1016/j.nicl.2018.09.002. PMID: 30238916.Yoo Y., Tang L.Y.W., Brosch T. et al. Deep learning of joint myelin and T1w MRI features in normal-appearing brain tissue to distinguish between multiple sclerosis patients and healthy controls. Neuroimage Clin. 2017;17:169-178. DOI: 10.1016/j.nicl.2017.10.015. PMID: 29071211.Yoo Y., Tang L.Y.W., Brosch T. et al. Deep learning of joint myelin and T1w MRI features in normal-appearing brain tissue to distinguish between multiple sclerosis patients and healthy controls. Neuroimage Clin. 2017;17:169–178. DOI: 10.1016/j.nicl.2017.10.015. PMID: 29071211.Nakamura K., Fisher E. Segmentation of brain magnetic resonance images for measurement of gray matter atrophy in multiple sclerosis patients. Neuroimage. 2009;44(3):769-776. DOI: 10.1016/j.neuroimage.2008.09.059. PMID: 19007895.Nakamura K., Fisher E. Segmentation of brain magnetic resonance images for measurement of gray matter atrophy in multiple sclerosis patients. Neuroimage. 2009;44(3):769–776. DOI: 10.1016/j.neuroimage.2008.09.059. PMID: 19007895.Hackmack K., Paul F., Weygandt M. et al. Alzheimer’s disease neuroimaging initiative. Multi-scale classification of disease using structural MRI and wavelet transform. Neuroimage. 2012;62(1):48-58. DOI: 10.1016/j.neuroim- age.2012.05.022. PMID: 22609452.Hackmack K., Paul F., Weygandt M. et al. Alzheimer’s disease neuroimaging initiative. Multi-scale classification of disease using structural MRI and wavelet transform. Neuroimage. 2012;62(1):48–58. DOI: 10.1016/j.neuroimage.2012.05.022. PMID: 22609452.Wei W., Poirion E., Bodini B. et al. Predicting PET-derived demyelination from multimodal MRI using sketcher-refiner adversarial training for multiple sclerosis. Med Image Anal. 2019;58:101546. DOI: 10.1016/j.media.2019.101546. PMID: 31499318.Theocharakis P., Glotsos D., Kalatzis I. et al. Pattern recognition system for the discrimination of multiple sclerosis from cerebral microangiopathy lesions based on texture analysis of magnetic resonance images. Magn Reson Imaging. 2009;27(3):417-422. DOI: 10.1016/j.mri.2008.07.014. PMID: 18786795.Theocharakis P., Glotsos D., Kalatzis I. et al. Pattern recognition system for the discrimination of multiple sclerosis from cerebral microangiopathy lesions based on texture analysis of magnetic resonance images. Magn Reson Imaging. 2009;27(3):417–422. DOI: 10.1016/j.mri.2008.07.014. PMID: 18786795.Crimi A., Commowick O., Maarouf A. et al. Predictive value of imaging markers at multiple sclerosis disease onset based on gadolinium- and USPIO-enhanced MRI and machine learning. PLoS One. 2014;9(4):e93024. DOI: 10.1371/ journal.pone.0093024. PMID: 24691080.Crimi A., Commowick O., Maarouf A. et al. Predictive value of imaging markers at multiple sclerosis disease onset based on gadolinium- and USPIO-enhanced MRI and machine learning. PLoS One. 2014;9(4):e93024. DOI: 10.1371/journal.pone.0093024. PMID: 24691080.Aslani S., Dayan M., Storelli L. et al. Multi-branch convolutional neural network for multiple sclerosis lesion segmentation. Neuroimage. 2019;196:1-15. DOI: 10.1016/j.neuroimage.2019.03.068. PMID: 30953833.Aslani S., Dayan M., Storelli L. et al. Multi-branch convolutional neural network for multiple sclerosis lesion segmentation. Neuroimage. 2019;196:1–15. DOI: 10.1016/j.neuroimage.2019.03.068. PMID: 30953833.McKinley R., Wepfer R., Grunder L. et al. Automatic detection of lesion load change in Multiple Sclerosis using convolutional neural networks with segmentation confidence. Neuroimage Clin. 2020;25:102104. DOI: 10.1016/j. nicl.2019.102104. PMID: 31927500.McKinley R., Wepfer R., Grunder L. et al. Automatic detection of lesion load change in Multiple Sclerosis using convolutional neural networks with segmentation confidence. Neuroimage Clin. 2020;25:102104. DOI: 10.1016/j.nicl.2019.102104. PMID: 31927500.Jain S., Ribbens A., Sima D.M. et al. D. Two time point MS lesion segmentation in brain MRI: an expectation-maximization framework. Front Neurosci. 2016;10:576. DOI: 10.3389/fnins.2016.00576. PMID: 28066162.Cabezas M., Oliver A., Valverde S. et al. BOOST: a supervised approach for multiple sclerosis lesion segmentation. J Neurosci Methods. 2014;237:108-117. DOI: 10.1016/j.jneumeth.2014.08.024. PMID: 25194638.Cabezas M., Oliver A., Valverde S. et al. BOOST: a supervised approach for multiple sclerosis lesion segmentation. J Neurosci Methods. 2014;237:108–117. DOI: 10.1016/j.jneumeth.2014.08.024. PMID: 25194638.Gabr R.E., Coronado I., Robinson M. et al. Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study. Mult Scler. 2020;26(10): 1217-1226. DOI: 10.1177/1352458519856843. PMID: 31190607.Gabr R.E., Coronado I., Robinson M. et al. Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study. Mult Scler. 2020;26(10): 1217–1226. DOI: 10.1177/1352458519856843. PMID: 31190607.Brosch T., Tang L.Y., Youngjin Yoo et al. Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans Med Imaging. 2016;35(5):1229-1239. DOI: 10.1109/TMI.2016.2528821. PMID: 26886978.Brosch T., Tang L.Y., Youngjin Yoo et al. Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans Med Imaging. 2016;35(5):1229–1239. DOI: 10.1109/TMI.2016.2528821. PMID: 26886978.Valverde S., Cabezas M., Roura E. et al. Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach. Neuroimage. 2017;155:159-168. DOI: 10.1016/j.neuroimage.2017.04.034. PMID: 28435096.Valverde S., Cabezas M., Roura E. et al. Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach. Neuroimage. 2017;155:159–168. DOI: 10.1016/j.neuroimage.2017.04.034. PMID: 28435096.Bossuyt P.M., Reitsma J.B., Bruns D.E. et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. Radiology. 2015;277(3):826-832. DOI: 10.1148/radiol.2015151516. PMID: 26509226.Bossuyt P.M., Reitsma J.B., Bruns D.E. et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. Radiology. 2015;277(3):826–832. DOI: 10.1148/radiol.2015151516. PMID: 26509226.Yamamoto D., Arimura H., Kakeda S. et al. Computer-aided detection of multiple sclerosis lesions in brain magnetic resonance images: false positive reduction scheme consisted of rule-based, level set method, and support vector machine. Comput Med Imaging Graph. 2010;34(5):404-413. DOI: 10.1016/j. compmedimag.2010.02.001. PMID: 20189353.Yamamoto D., Arimura H., Kakeda S. et al. Computer-aided detection of multiple sclerosis lesions in brain magnetic resonance images: false positive reduction scheme consisted of rule-based, level set method, and support vector machine. Comput Med Imaging Graph. 2010;34(5):404–413. DOI: 10.1016/j.compmedimag.2010.02.001. PMID: 20189353.Goldberg-Zimring D., Achiron A., Miron S. et al. Automated detection and characterization of multiple sclerosis lesions in brain MR images. Magn Reson Imaging. 1998;16(3):311-318. DOI: 10.1016/s0730-725x(97)00300-7. PMID: 9621972.Goldberg-Zimring D., Achiron A., Miron S. et al. Automated detection and characterization of multiple sclerosis lesions in brain MR images. Magn Reson Imaging. 1998;16(3):311–318. DOI: 10.1016/s0730-725x(97)00300-7. PMID: 9621972.Bendfeldt K., Kloppel S., Nichols T.E. et al. Multivariate pattern classification of gray matter pathology in multiple sclerosis. Neuroimage. 2012;60(1):400- 408. DOI: 10.1016/j.neuroimage.2011.12.070. PMID: 22245259.Bendfeldt K., Klöppel S., Nichols T.E. et al. Multivariate pattern classification of gray matter pathology in multiple sclerosis. Neuroimage. 2012;60(1):400–408. DOI: 10.1016/j.neuroimage.2011.12.070. PMID: 22245259.Salem M., Cabezas M., Valverde S. et al. A supervised framework with intensity subtraction and deformation field features for the detection of new T2-w lesions in multiple sclerosis. Neuroimage Clin. 2017;17:607-615. DOI: 10.1016/j. nicl.2017.11.015. PMID: 29234597.Salem M., Cabezas M., Valverde S. et al. A supervised framework with intensity subtraction and deformation field features for the detection of new T2-w lesions in multiple sclerosis. Neuroimage Clin. 2017;17:607–615. DOI: 10.1016/j.nicl.2017.11.015. PMID: 29234597.Boudraa A.O., Dehak S.M., Zhu Y.M. et al. Automated segmentation of multiple sclerosis lesions in multispectral MR imaging using fuzzy clustering. Comput Biol Med. 2000;30(1):23-40. DOI: 10.1016/s0010-4825(99)00019-0. PMID: 10695813.Boudraa A.O., Dehak S.M., Zhu Y.M. et al. Automated segmentation of multiple sclerosis lesions in multispectral MR imaging using fuzzy clustering. Comput Biol Med. 2000;30(1):23–40. DOI: 10.1016/s0010-4825(99)00019-0. PMID: 10695813.Geremia E., Menze B.H., Clatz O. et al. Spatial decision forests for MS lesion segmentation in multi-channel MR images. Med Image Comput Comput Assist Interv. 2010;13(Pt 1):111-118. DOI: 10.1007/978-3-642-15705-9_14. PMID: 20879221.Geremia E., Menze B.H., Clatz O. et al. Spatial decision forests for MS lesion segmentation in multi-channel MR images. Med Image Comput Comput Assist Interv. 2010;13(Pt 1):111–118. DOI: 10.1007/978-3-642-15705-9_14. PMID: 20879221.Shiee N., Bazin P.L., Ozturk A. et al. A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions. Neuroimage. 2010;49(2):1524-1535. DOI: 10.1016/j.neuroimage.2009.09.005. PMID: 19766196.Shiee N., Bazin P.L., Ozturk A. et al. A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions. Neuroimage. 2010;49(2):1524–1535. DOI: 10.1016/j.neuroimage.2009.09.005. PMID: 19766196.Garcia-Lorenzo D., Lecoeur J., Arnold D.L. et al. Multiple sclerosis lesion segmentation using an automatic multimodal graph cuts. Med Image Comput Comput Assist Interv. 2009;12(Pt 2):584-591. DOI: 10.1007/978-3-642-04271- 3_71. PMID: 20426159.García-Lorenzo D., Lecoeur J., Arnold D.L. et al. Multiple sclerosis lesion segmentation using an automatic multimodal graph cuts. Med Image Comput Comput Assist Interv. 2009;12(Pt 2):584–591. DOI: 10.1007/978-3-642-04271-3_71. PMID: 20426159.Salem M., Cabezas M., Valverde S. et al. A supervised framework with intensity subtraction and deformation field features for the detection of new T2-w lesions in multiple sclerosis. Neuroimage Clin. 2017;17:607-615. DOI: 10.1016/j. nicl.2017.11.015. PMID: 29234597.Salem M., Cabezas M., Valverde S. et al. A supervised framework with intensity subtraction and deformation field features for the detection of new T2-w lesions in multiple sclerosis. Neuroimage Clin. 2017;17:607–615. DOI: 10.1016/j.nicl.2017.11.015. PMID: 29234597.Khayati R., Vafadust M., Towhidkhah F. et al. A novel method for automatic determination of different stages of multiple sclerosis lesions in brain MR FLAIR images. Comput Med Imaging Graph. 2008;32(2):124-133. PMID: 18055174.Khayati R., Vafadust M., Towhidkhah F. et al. A novel method for automatic determination of different stages of multiple sclerosis lesions in brain MR FLAIR images. Comput Med Imaging Graph. 2008;32(2):124–133. PMID: 18055174.Weiss N., Rueckert D., Rao A. Multiple sclerosis lesion segmentation using dictionary learning and sparse coding. Med Image Comput Comput Assist Interv. 2013;16(Pt 1):735-742. DOI: 10.1007/978-3-642-40811-3_92. PMID: 24505733.Weiss N., Rueckert D., Rao A. Multiple sclerosis lesion segmentation using dictionary learning and sparse coding. Med Image Comput Comput Assist Interv. 2013;16(Pt 1):735–742. DOI: 10.1007/978-3-642-40811-3_92. PMID: 24505733.Gao J., Li C., Feng C. et al. Non-locally regularized segmentation of multiple sclerosis lesion from multi-channel MRI data. Magn Reson Imaging. 2014;32(8):1058-1066. DOI: 10.1016/j.mri.2014.03.006. PMID: 24948583.Gao J., Li C., Feng C. et al. Non-locally regularized segmentation of multiple sclerosis lesion from multi-channel MRI data. Magn Reson Imaging. 2014;32(8):1058–1066. DOI: 10.1016/j.mri.2014.03.006. PMID: 24948583.Bijar A., Khayati R., Penalver Benavent A. Increasing the contrast of the brain MR FLAIR images using fuzzy membership functions and structural similarity indices in order to segment MS lesions. PLoS One. 2013;8(6):e65469. DOI: 10.1371/journal.pone.0065469. PMID: 23799015.Bijar A., Khayati R., Peñalver Benavent A. Increasing the contrast of the brain MR FLAIR images using fuzzy membership functions and structural similarity indices in order to segment MS lesions. PLoS One. 2013;8(6):e65469. DOI: 10.1371/journal.pone.0065469. PMID: 23799015.Cerasa A., Bilotta E., Augimeri A. et al. A Cellular Neural Network methodology for the automated segmentation of multiple sclerosis lesions. J Neurosci Methods. 2012;203(1):193-199. DOI: 10.1016/j.jneumeth.2011.08.047. PMID: 21920384.Cerasa A., Bilotta E., Augimeri A. et al. A Cellular Neural Network methodology for the automated segmentation of multiple sclerosis lesions. J Neurosci Methods. 2012;203(1):193–199. DOI: 10.1016/j.jneumeth.2011.08.047. PMID: 21920384.Aymerich F.X., Sobrevilla P., Montseny E. et al. Fuzzy approach toward reducing false positives in the detection of small multiple sclerosis lesions in magnetic resonance images. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:5694- 5697. DOI: 10.1109/IEMBS.2011.6091378. PMID: 22255632.Aymerich F.X., Sobrevilla P., Montseny E. et al. Fuzzy approach toward reducing false positives in the detection of small multiple sclerosis lesions in magnetic resonance images. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:5694–5697. DOI: 10.1109/IEMBS.2011.6091378. PMID: 22255632.Kuwazuru J., Arimura H., Kakeda S. et al. Automated detection of multiple sclerosis candidate regions in MR images: false-positive removal with use of an ANN-controlled level-set method. Radiol Phys Technol. 2012;5(1):105-113. DOI: 10.1007/s12194-011-0141-2. PMID: 22139608.Kuwazuru J., Arimura H., Kakeda S. et al. Automated detection of multiple sclerosis candidate regions in MR images: false-positive removal with use of an ANN-controlled level-set method. Radiol Phys Technol. 2012;5(1):105–113. DOI: 10.1007/s12194-011-0141-2. PMID: 22139608.Kanber B., Nachev P., Barkhof F. et al. High-dimensional detection of imaging response to treatment in multiple sclerosis. NPJ Digit Med. 2019;2:49. DOI: 10.1038/s41746-019-0127-8. PMID: 31304395.Freire P.GL, Ferrari R.J. Multiple sclerosis lesion enhancement and white matter region estimation using hyperintensities in FLAIR images. arXiv. 2018:1807.09619.Feng Y., Pan H., Meyer C. et al. A self-adaptive network for multiple sclerosis lesion segmentation from multi-contrast MRI with various imaging protocols. arXiv. 2018:1811.07491.Kazancli E., Prchkovska V., Rodrigues P. et al. Multiple sclerosis lesion segmentation using improved convolutional neural networks. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018). 2018; 4; VISAPP: 260269. DOI: 10.5220/0006540902600269.Kazancli E., Prchkovska V., Rodrigues P. et al. Multiple sclerosis lesion segmentation using improved convolutional neural networks. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018). 2018; 4; VISAPP: 260–269. DOI: 10.5220/0006540902600269.Roy S., Butman J.A., Reich D.S. et al. Multiple sclerosis lesion segmentation from brain MRI via fully convolutional neural networks. arXiv. 2018:1803.09172v1.Sacca V, Sarica A., Novellino F. et al. Evaluation of machine learning algorithms performance for the prediction of early multiple sclerosis from resting-state FMRI connectivity data. Brain Imaging Behav. 2019;13(4):1103-1114. DOI: 10.1007/s11682-018-9926-9. PMID: 29992392.Saccà V., Sarica A., Novellino F. et al. Evaluation of machine learning algorithms performance for the prediction of early multiple sclerosis from resting-state FMRI connectivity data. Brain Imaging Behav. 2019;13(4):1103–1114. DOI: 10.1007/s11682-018-9926-9. PMID: 29992392.Zhang H., Zhang J., Zhang Q. et al. RSANet: Recurrent Slice-wise Attention Network for multiple sclerosis lesion segmentation. arXiv; 2020:2002.12470v1.Gheshlaghi S.H., Ranjbar A., Suratgar A.A. et al. A superpixel segmentation based technique for multiple sclerosis lesion detection. arXiv; 2020:1907.03109v1.