https://doi.org/10.35845/kmuj.2024.23503                            ORIGINAL ARTICLE

Comparison of Artificial Intelligence-based learning with the traditional method in the diagnosis of COVID-19 chest radiographs among postgraduate radiology residents

Ayesha Isani Majeed  ¹

 

1: Department of Radiology, Pakistan Institute of Medical Sciences (PIMS) Islamabad, Pakistan   Email : ayeshamajeed1@gmail.com Contact #: +92-300-5226086   Date Submitted: September 29, 2023 Date Revised: April 12, 2024 Date Accepted: May 27, 2024

THIS ARTICLE MAY BE CITED AS: Majeed AI. Comparison of Artificial Intelligence-based learning with the traditional method in the diagnosis of COVID-19 chest radiographs among postgraduate radiology residents. Khyber Med Univ J 2024;16(2):140-44. https://doi.org/10.35845/kmuj.2024.23503

ABSTRACT

OBJECTIVE: To compare Artificial Intelligence (AI)-based teaching with traditional approach in chest radiographs to detect COVID-19 pneumonia.

METHODS: This prospective experimental randomized controlled trial was conducted at Pakistan Institute of Medical Sciences, Islamabad from July to November 2021, following ethical approval. Forty postgraduate radiology residents were randomly assigned into Group A (traditional teaching; n=20) or Group B (AI-based teaching; n=20) using a lottery method. Group A engaged in one-on-one sessions for COVID X-ray reporting, while Group-B trained in AI-deep learning methods. Pre-tests assessed baseline knowledge, and post-training assessments compared learning outcomes. Statistical analysis using SPSS v25 included Independent sample t-tests and chi square test. Following initial assessments, teaching methods were exchanged between groups for comparison.

RESULTS: Out of 40 participants60% were males and 40% were females, with mean age of 27.45±1.7 years.  Group B showed significantly higher post-test scores (9.40±0.598) compared to Group-A (7.75 ± 1.118) (p < 0.001).The average improvement in scores was significantly higher in Group B based on the change from pre-test to post-test scores (p < 0.05). Significant score improvements favored Group-B across all training years (p <0.05). Gender analysis indicated similar score gains among males but significantly higher improvements in females in Group B (4.09±1.868 vs. 2.00±1.414, p <0.05).

CONCLUSION: AI approach proves significantly more time and cost efficient compared to traditional teaching methods in enhancing the ability of radiology residents. This highlight the potential of AI to optimize medical education by integration of AI technologies into radiology training programs, providing efficient, scalable, and effective learning experiences.

KEYWORDS: Artificial intelligence (MeSH); COVID-19 (MeSH); Radiographs (Non-MeSH); Traditional method (Non-MeSH);Visual Perception (MeSH); Annotated image (Non-MeSH), Computer System (MeSH);Learning (MeSH); Radiologists (MeSH).

INTRODUCTION

The rapid integration of artificial intelligence (AI) into healthcare has revolutionized traditional approaches to delivering medical services.¹ Today, educators in medical fields advocate for a fundamental shift in medical education, emphasizing the transition from "knowledge acquisition" to "knowledge management and communication" to effectively address the challenges of the 21st century.²Over recent decades, AI has significantly impacted various aspects of healthcare, including data collection, interpretation, and image analysis, particularly in fields like diagnostic radiology. AI technologies have streamlined workflows by automating tasks such as image sorting, prioritization, and detection of urgent cases, thereby reducing the time radiologists spend on routine activities.³ Moreover, AI-driven computer programs have been developed to aid in diagnosing abnormalities and alerting radiologists to potential issues, thereby minimizing the occurrence of false negative readings.^4,5

In response to the increased burden of chest radiographs during the COVID-19 pandemic, AI-based software solutions have emerged to detect COVID-19-related findings on both radiographs and high-resolution CT scans of the chest. These tools have proven invaluable to radiologists, alleviating workload pressures, saving time, and enhancing diagnostic accuracy by reducing the likelihood of misinterpretations. Additionally, they serve as effective educational aids, facilitating the teaching of disease diagnosis to radiology students.⁶

The rising expectations for medical education, influenced by social media, industry leaders, and healthcare professionals, are driving the demand for the integration of AI. Additionally, escalating costs in medical education and healthcare underscore the necessity for AI to alleviate financial burdens. Critics argue that the traditional model of health education is antiquated, costly, and excessively time-consuming. Previous research has highlighted AI's potential to personalize learning, automate grading, and provide intelligent tutoring.^7,8Various studies have sought to develop a policy framework for an integrated learning approach that leverages personalized interaction, respects patient privacy, and utilizes efficient AI techniques to save time.^9-11

The COVID-19 pandemic has accelerated the adoption of online teaching and remote work, prompting widespread acceptance and integration of digital technologies. This shift has not only reshaped education and work dynamics but also accelerated the incorporation of AI into various facets of life. Researchers have developed AI-based tools for detecting COVID-19-associated pneumonia through the interpretation of chest radiographs. Studies have documented AI's promising role in achieving high accuracy in this area.^12-14

Given the high level of expertise traditionally required in radiology for detecting COVID-19 pneumonia on chest radiographs, this study was planned to compare the effectiveness of AI-based learning methods with traditional methods among radiology residents. It evaluates and compares the academic performance and learning outcomes of residents in the Radiology Department of Pakistan Institute of Medical Sciences,a tertiary care facility located in Islamabad, Pakistan.

METHODS

This prospective experimental randomized controlled trial was conducted in the Radiology Department of Pakistan Institute of Medical Sciences (PIMS) Islamabad, Pakistan, from July 2021 to November 2021, following approval from the Institutional Ethical Review Board (ERB letter No. F.1-1/2015/ERB/SZABMU/594). The sample size was determined according to the methodology outlined by Suresh and Chandrashekara (2012),¹⁵ aligning with the study's research objectives.

All postgraduate residents of Radiology Department (both MD and FCPS), Pakistan Institute of Medical Sciences, PIMS Islamabad were eligible to be included in the study. Residents working in other institutes/departments, students who have already taken the course on AI and those leaving training before completion of the study were excluded from the sample population.

Forty postgraduate students were recruited for the study after obtaining informed consent from each participant. Initially, these students were grouped based on their year of training, resulting in four groups. Subsequently, a lottery method was employed to randomize these groups into two groups, A and B, each comprising 20 students.

In Group A, residents participated in traditional one-on-one teaching sessions for COVID X-ray reporting. This approach involved joint review of chest radiographs by students and supervisors, where students presented their findings and received immediate feedback from experienced radiologists. Feedback was systematically collected through structured questionnaires distributed among senior doctors, focusing on progress, satisfaction levels, and overall performance.

On the other hand, Group B underwent training in deep learning, a specialized branch of AI, for detecting COVID-19 pneumonia. Using AI-equipped software, students independently analyzed chest radiographs, identifying abnormal areas with annotation tools and selecting corresponding diagnoses from provided options. After completing their interpretation, the software displayed an AI-annotated image of the radiograph, highlighting abnormal findings in various pathologies with color-coded indicators. Students compared their initial interpretations with AI-generated insights to identify and correct errors. The software operated without time constraints, providing feedback only after the student completed their interpretation, allowing for iterative practice and skill refinement.

Following random allocation, all participating radiology trainees underwent an initial assessment (pre-test)to ensure they understood the parameters and fundamentals of identifying COVID-19 pneumonia on chest X-rays (figure 1). This assessment aimed to evaluate their knowledge of radiological terminology such as ground glass opacification, consolidation, and pleural effusion, which were crucial for subsequent testing.

After training, residents' responses during the identification of COVID-19 pneumonia cases on chest radiographs were analyzed to assess the effectiveness of learning methods, comparing traditional one-on-one teaching with AI-based approaches.During the process, points were awarded for each finding. Each correct identification earned one point, with a maximum of 10 points per resident, facilitating a clear comparison between the groups.The accumulated total points of both groups were compared using a t-test. Following the t-test, the role of AI in medical education was evaluated and compared with the traditional one-on-one teaching methodology. Data analysis was performed using SPSS version 25, including statistical tests to identify significant differences in learning outcomes between the two groups. The p-value of < 0.05 was considered significant.

To explore alternative teaching approaches, Group A and Group B were subsequently switched: Group A experienced AI-based teaching, while Group B received traditional teaching methods. No additional scoring was conducted post-intervention, as the study objectives were considered achieved with the initial assessments and comparative analysis of learning outcomes.

Figure 1. Flow chart demonstrating the methodology

RESULTS

The data is evenly distributed around the mean with approximately equal frequencies of values on both sides. Out of 40 participants, 24 (60%) were males and 16 (40%) were females. Mean age of study participants was 27.73±1.71 years, ranging from 25 to 33 years (Table 1).

 

Table I:  Comparison of Age Distribution between Traditional Teaching and Deep Learning Methods

^{Group A received traditional one-on-one teaching sessions}

^{Group B: received training in deep learning (AI-based)}

Table II summarizes the comparison of pre-test and post-test scores in both Group A and Group B, showing significant improvements in post-test scores for both groups (p < 0.001). Mean Post-Test Score in group B was 9.40±0.59 as compared to 7.75±1.12 in group A(p < 0.001).The comparison indicated that the average improvement in scores was significantly higher in Group B (p < 0.05) based on the change from pre-test to post-test scores (Table III).

Table II:  Comparison of Pre-test and Post-test Scores between Traditional Teaching and Deep Learning Methods

^{Group A received traditional one-on-one teaching sessions}

^{Group B: received training in deep learning (AI-based)}

 

Table III: Comparison of Pre-and Post-Test Score Improvement between Traditional Teaching and Deep Learning Methods

^{Group A received traditional one-on-one teaching sessions}

^{Group B: received training in deep learning (AI-based)}

 

The comparison of improvement in pre- and post-test scores based on year of training revealed that participants in the 1^stand 3^rd years had significantly higher mean differences in test scores, while those in the 2^nd and 4^th years showed similar average difference scores (Table IV).

 

Table IV: Comparison of Year-wise Improvement in Test Scores between Traditional Teaching and Deep Learning Methods

Group A received traditional one-on-one teaching sessions

Group B: received training in deep learning (AI-based)

 

The comparison using a Chi-square test indicated similar improvement in scores among males in both groups (2.73 ± 1.280 vs. 3.89 ± 1.616, p > 0.05). However, among females, the improvement in scores differed significantly, with Group B showing a significantly higher mean difference in pre- and post-test scores compared to Group A (4.09 ± 1.868 vs. 2.00 ± 1.414, p < 0.05) [Table V].

Table V: Comparison of Gender-based Improvement in Scores between Traditional Teaching and Deep Learning Methods

^{Group A received traditional one-on-one teaching sessions}

^{Group B: received training in deep learning (AI-based)}

DISCUSSION

AI-based learning for radiology residents has demonstrated superior results compared to traditional methods, offering immediate feedback and allowing students to progress at their own pace. According to Andersen and Ponti,¹⁶ AI-driven platforms can analyze students' strengths and weaknesses, providing personalized learning experiences. Furthermore, AI is more effective than clinicians in assessing medical students' performance.¹⁷ Siemens G,.¹⁸ documented that AI systems reduce the time between learning and evaluation, offering instant feedback that helps students identify areas for improvement and facilitates iterative learning. Compared to traditional approaches, AI also consumes less time. Varma JR, et al.'s research showed a significant 32% increase in multiple-choice test scores (2.24 to 2.96, p < 0.001) using CIRCSIM(cardiovascular Integrated Real-Time Computational Simulation; a computer-based simulation program), a computer-based simulation program, alongside didactic teaching¹⁹ which supports our findings. Another study found a 70% improvement in the evaluation of imaging interpretation, including transthoracic echocardiograms, when compared to human assessment.²⁰

Somaliland and Malaysia have implemented effective initiatives to integrate AI, aiming to enhance the training of healthcare professionals in a cost-effective, time-efficient, and impactful manner.^21,22

Past research emphasizes the need for more interventions in the education system to promote Artificial Intelligence in Education (AIED).²³ The specific impacts of AI in education remain largely unexplored.²⁴ AI software can adapt teaching methods to suit individual student needs, effectively identifying areas of weakness and promoting student self-awareness in specific subjects. An important benefit of integrating AI in education is its potential to eliminate bias, ensuring equitable treatment of all students regardless of their background. In radiology, AI datasets provide a vast number of images for student training, surpassing the quantity available in human-curated libraries or personal collections.

The findings of this research should be interpreted considering its inherent limitations. Firstly, the relatively small sample size may restrict its generalizability to a broader population. Secondly, since the results are based on chest radiographs, any variations in the quality of these images could impact the study outcomes. Additionally, as the study was conducted at a single institution, factors specific to that institution and potential biases cannot be disregarded, potentially limiting applicability across diverse educational settings. Furthermore, the study focused exclusively on one department, overlooking variations in educational needs and outcomes across different disciplines.

While the research explores the innovative use of AI-based teaching compared to traditional methods, these constraints advise caution in applying the findings to a broader educational context. It is recommended that similar research be conducted on a larger scale across different departments and institutions to gain deeper insights into AI-based teaching.

CONCLUSION

This study highlights AI's pivotal role in medical education, particularly in training radiology residents to diagnose COVID-19 pneumonia from chest radiographs. AI-based learning provides immediate feedback, personalized experiences, and superior error correction compared to traditional methods, enhancing diagnostic skills effectively. The findings underscore AI's potential to optimize medical education through integration into radiology training programs. However, caution is necessary due to the study's limitations in sample size and single-institution focus. Future research should explore AI's broader impact across diverse educational settings and medical specialties to fully leverage its potential in global healthcare training.

REFERENCES

 

1.  Fong SJ, Dey N, Chaki J, Fong SJ, Dey N, Chaki J. An introduction to COVID-19. ArtifIntell Coronavirus Outbreak 2020:1-22.https://doi.org/10.1007%2F978-981-15-5936-5_1

2.    Harris M, Qi A, Jeagal L, Torabi N, Menzies D, Korobitsyn A, et al. A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis. PLoS One 2019;14(9):e0221339.https://doi.org/10.1371/journal.pone.0221339

3. Tang X. The role of artificial intelligence in medical imaging research. BJR Open 2019;2(1):20190031. https://doi.org/10.1259/bjro.20190031

4.  Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. https://doi.org/10.1016/j.media.2017.07.005

5.  Paul R, Hawkins SH, Balagurunathan Y, Schabath MB, Gillies RJ, Hall LO, et al. Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma.  Tomography 2016;2:388-95. https://doi.org/10.18383/j.tom.2016.00211

6.  Putha P, Tadepalli M, Reddy B, Raj T, Chiramal JA, Govil S, et al. Can artificial intelligence reliably report chest x-rays?: radiologist validation of an algorithm trained on 2.3 million x-rays. A Xiv preprint. 2018 Jul 19. https://doi.org/10.48550/arXiv.1807.07455

7. Paranjape K, Schinkel M, Panday RN, Car J, Nanayakkara P. Introducing artificial intelligence training in medical education. JMIR Med Educ 2019;5(2):e16048. https://doi.org/10.2196/16048

8.  Sapci AH, Sapci HA. Artificial intelligence education and tools for medical and health informatics students [systematic review]. JMIR Med Educ 2020;6(1):e19285. https://doi.org/10.2196/19285

9.  Van der Niet AG, Bleakley A. Where medical education meets artificial intelligence: 'Does technology care? Med Educ 2021;55(1):30-36. https://doi.org/10.1111/medu.1413

10. Varma JR, Fernando S, Ting BY, Aamir S, Sivaprakasam R. the global use of artificial intelligence in the undergraduate medical curriculum [a systematic review]. Cureus 2023;15(5):e39701. https://doi.org/10.7759/cureus.39701

11. Webster CS. Artificial intelligence and the adoption of new technology in medical education. Med Educ 2021;55(1):6-7. https://doi.org/10.1111/medu.14409

12.  Dorr F, Chaves H, Serra MM, Ramirez A, Costa ME, Seia J, et al. COVID-19 pneumonia accurately detected on chest radiographs with artificial intelligence. Intell-Based Med 2020;3:100014. https://doi.org/10.1016/j.ibmed.2020.100014

13. Rangarajan K, Muku S, Garg AK, Gabra P, Shankar SH, Nischal N, et al. Artificial Intelligence–assisted chest X-ray assessment scheme for COVID-19. EurRadiol 2021;31:6039-48. https://doi.org/10.1007/s00330-020-07628-5

14. Baruah D, Runge L, Jones RH, Collins HR, Kabakus IM, McBee MP. COVID-19 Diagnosis on chest radiograph using artificial intelligence. Cureus 2022;14(11):e31897. https://doi,org/10.7759/cureus.

15. Suresh K, Chandrashekara S. Sample size estimation and power analysis for clinical research studies. J Hum ReprodSci 2012;5(1):7-13. https://doi.org/10.4103/0974-1208.97779

16. Andersen, S Ponti, M Personalized adaptive learning and artificial intelligence in Higher education [A systematic literature review]. Educ Sci 2020;10(8), 205.

17. Stevens RH, Lopo AC. Artificial neural network comparison of expert and novice problem-solving strategies. ProcAnnuSympComputAppl Med Care 1994;64-8.

18. Siemens, G Long, P Penetrating the Fog: Analytics in learning and education. educause Review 2011;46(5):30-32.

19. Varma JR, Fernando S, Ting BY, Aamir S, Sivaprakasam R. the global use of artificial intelligence in the undergraduate medical curriculum [A systematic review]. Cureus 2023;15(5). https://doi.org/10.7759/cureus.39701

20. Langet H, Bonopera M, De Craene M, Popoff A, Denis E, Pizaine G, et al.Turning novices into experts: can artificial intelligence transform echocardiography training?.Eur Heart J Cardiovasc. Imaging 2020;21(Suppl 1): jez319.275 https://doi.org/10.1093/ehjci/jez319.275

21.Keynejad RC. Global health partnership for student peer-to-peer psychiatry e-learning: lessons learned. Glob Health 2016;12(1):1-7.https://doi.org/10.1186/s12992-016-0221-5

22.O’donovan J, Maruthappu M. Distant peer-tutoring of clinical skills, using tablets with instructional videos and Skype: A pilot study in the UK and Malaysia. Med Teach 2015;37(5):463-9. https://org.doi/10.3109/0142159X.2014.956063

23.Lombardi, D. Evaluation of traditional and online learning in artificial intelligence. Proccedings of the Third Workshop on Technology Enhanced Learning Environments for Blended Education. [ Accessed on : November 30, 2021]. Available from URL: www.CEUR-WS.org

24.Xie H, Hwang GJ, Wong TL. Editorial note: from conventional AI to modern AI in education: reexamining AI and analytic techniques for teaching and learning. J Educ Techno Soc 2021;24(3).

AUTHORS' CONTRIBUTIONS Following author have made substantial contributions to the manuscript as under: AIM: Concept and study design, acquisition, analysis and interpretation of data, drafting the manuscript, critical review, approval of the final version to be published Author agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

 

CONFLICT OF INTEREST Authors declared no conflict of interest, whether financial or otherwise, that could influence the integrity, objectivity, or validity of their research work. GRANT SUPPORT AND FINANCIAL DISCLOSURE Authors declared no specific grant for this research from any funding agency in the public, commercial or non-profit sectors

 

DATA SHARING STATEMENT The data that support the findings of this study are available from the corresponding author upon reasonable request

 

This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License.

 

KMUJ web address: www.kmuj.kmu.edu.pk Email address: kmuj@kmu.edu.pk