Isfahan University of Medical Sciences Health Information Management 1735-7853 21 2 2024 06 21 Evaluating AI Performance in COVID-19 Diagnosis: A Systematic Review Evaluating AI Performance in COVID-19 Diagnosis: A Systematic Review 103 111 31384 10.48305/him.2025.43286.1247 FA Azadeh Abkar PhD student,, Department of computer engineering, Shiraz Branch,Islamic Azad University, Shiraz, Iran 0000-0003-0597-2223 Mahdi Mehrabi Assistant Professor,, Department of computer engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran 0000-0002-1551-5723 Amin Golabpour Assistant Professor, Department of Health Information Technology, Shahrood University of Medical Sciences, Shahrood, Iran 0000-0001-7649-4033 Mohammad Amin Shayegan Associate Professor, Department of Computer Science, Shiraz Branch, Islamic Azad University, Shiraz, Iran 0000-0001-7394-4772 Journal Article 2024 12 15 Introduction: Early detection of COVID-19 plays a crucial role in reducing mortality. With its ability to analyze large volumes of medical data, artificial intelligence (AI) can aid in the rapid and accurate diagnosis of this disease. This review study aims to assess the performance of AI models in detecting COVID-19. Methods: A systematic search was conducted in the PubMed, Scopus, and WOS databases, yielding 143 relevant articles. The studies were analyzed based on the type of input data, algorithms used, performance evaluation metrics, the presentation of clinical rules, and the clinical validation of the models. Additionally, 451 review and meta-analysis articles were examined to determine the extent to which these algorithms have been integrated into clinical guidelines. Results: The input data in the articles consisted of clinical records, medical images, and audio processing. More than 96% of the algorithms were found to be black-box models, lacking clinical validation by specialists. However, the average performance of these models was reported to be above 90%. The review of meta-analysis articles revealed that none of the algorithms had undergone formal clinical evaluation; only their performance on available data was assessed. Conclusion: The review of studies shows that AI performs well in COVID-19 detection but faces limitations, including lack of explainability, reliance on training data, and the absence of extensive clinical evaluations. The development of white-box models and conducting broad clinical studies are crucial for ensuring the widespread acceptance of these algorithms in clinical settings. Introduction: Early detection of COVID-19 plays a crucial role in reducing mortality. With its ability to analyze large volumes of medical data, artificial intelligence (AI) can aid in the rapid and accurate diagnosis of this disease. This review study aims to assess the performance of AI models in detecting COVID-19. Methods: A systematic search was conducted in the PubMed, Scopus, and WOS databases, yielding 143 relevant articles. The studies were analyzed based on the type of input data, algorithms used, performance evaluation metrics, the presentation of clinical rules, and the clinical validation of the models. Additionally, 451 review and meta-analysis articles were examined to determine the extent to which these algorithms have been integrated into clinical guidelines. Results: The input data in the articles consisted of clinical records, medical images, and audio processing. More than 96% of the algorithms were found to be black-box models, lacking clinical validation by specialists. However, the average performance of these models was reported to be above 90%. The review of meta-analysis articles revealed that none of the algorithms had undergone formal clinical evaluation; only their performance on available data was assessed. Conclusion: The review of studies shows that AI performs well in COVID-19 detection but faces limitations, including lack of explainability, reliance on training data, and the absence of extensive clinical evaluations. The development of white-box models and conducting broad clinical studies are crucial for ensuring the widespread acceptance of these algorithms in clinical settings.

https://him.mui.ac.ir/article_31384_9189e075289f180149ff1107d6d48f78.pdf