Integrating Advanced Techniques RFESVM Feature Engineering and NelderMead Optimized XGBoost for Accurate Lung Cancer Prediction

Lung cancer remains a leading cause of cancer-related mortality worldwide, primarily due to late-stage diagnosis and subtle early symptoms. Early detection is crucial for improving treatment outcomes, increasing survival rates, and reducing healthcare costs. Traditional diagnostic methods are often invasive and may lack the sensitivity required for early-stage identification. Leveraging advances in machine learning, this study proposes a comprehensive lung cancer prediction system using minimal, non-invasive patient data. The dataset consists of 1000 patient records with 23 features, including demographic details and lifestyle indicators such as smoking habits, anxiety, chest pain, and fatigue. The methodology begins with Recursive Feature Elimination (RFE) combined with Support Vector Machine (SVM) for effective feature selection. XGBoost, known for its ability to model complex relationships, is optimized using the Nelder-Mead algorithm to improve classification performance. Additionally, Naive Bayes is incorporated due to its simplicity and fast training capabilities, providing complementary strengths to the ensemble. These base learners are integrated within a stacking classifier framework with a meta-learner, enhancing predictive accuracy, generalization, and reducing overfitting. The proposed model is designed to be scalable, interpretable, and suitable for practical deployment in healthcare analytics, enabling clinicians to make early and accurate lung cancer diagnoses efficiently.

Keywords

Lung Cancer Prediction, Machine Learning, Recursive Feature Elimination (RFE), Support Vector Machine (SVM), XGBoost, Stacking Classifier, Nelder-Mead Optimization, Ensemble Learning, Feature Engineering, Cancer Risk Assessment, Early Diagnosis, Clinical Dataset, Healthcare Analytics.