ViT-ResNet Fusion: An Explainable Hybrid Framework for High-Accuracy Multiclass Lung Disease Classification in Chest X-Rays

This study introduces ViT-ResNet Fusion, a novel hybrid deep learning framework aimed at high-accuracy multiclass lung disease classification from chest X-ray images. Accurate identification of lung conditions such as pneumonia, tuberculosis, COVID-19, and other pulmonary disorders is crucial for timely clinical intervention. Traditional convolutional neural networks (CNNs) excel at capturing local spatial features but often fail to model global contextual relationships effectively. Conversely, Vision Transformers (ViTs) provide superior global attention capabilities but may underutilize fine-grained local patterns. To leverage the complementary strengths of both architectures, we propose a fusion strategy that integrates ResNet for robust local feature extraction with ViT for global contextual awareness. Three specialized models are developed within this framework: PulmoFusion‑X, which emphasizes comprehensive pulmonary features; BronchoViT‑R, optimized for bronchial pattern recognition; and AeroNet‑S, designed to capture subtle aeration-related abnormalities. These models are trained and evaluated on the publicly available SIPAKM Chest X-Ray Dataset, demonstrating substantial improvements in classification accuracy, precision, recall, and F1-score compared to baseline CNN and transformer models. Additionally, explainable AI techniques such as Grad-CAM are employed to visualize the regions contributing to model predictions, enhancing interpretability and clinical trust. The proposed hybrid framework not only achieves state-of-the-art performance but also provides an interpretable and scalable solution for automated chest X-ray analysis, potentially aiding radiologists in faster and more reliable disease diagnosis.

Keywords: Vision Transformer, ResNet, Hybrid Deep Learning, Chest X-Rays, Multiclass Lung Disease Classification, Explainable AI, PulmoFusion‑X, BronchoViT‑R, AeroNet‑S, Grad-CAM