This repository contains two main stages of the stroke prediction workflow:

• final_clean.py prepares the dataset for modelling.
• final_stroke_ensemble.py trains and evaluates the final leakage-safe ensemble on the real imbalanced dataset.

The project evolved from an earlier balanced setup into a more realistic medical classification workflow that keeps the original class imbalance and focuses on recall-sensitive evaluation.

• healthcare_data.csv - raw source dataset.
• healthcare_data_cleaned.csv - cleaned dataset exported by final_clean.py.
• final_clean.py - deduplicates records, imputes BMI, adds clinically useful interaction features, and saves the cleaned dataset.
• final_stroke_ensemble.py - final ensemble training script with threshold tuning, stacking, and test-set evaluation.
• visualization_report.py - standalone script that generates polished EDA figures, statistical-test visuals, and a markdown report.
• label_shuffle_test.py - sanity check to confirm the pipeline is not leaking label information.
• stroke_ensemble_roc_pr_fi.png - ROC, precision-recall, and XGBoost feature importance plot.
• stroke_ensemble_confusion.png - confusion matrices for all tuned models.
• stroke_ensemble_threshold_sweep.png - threshold sweep for the stacking model.

final_clean.py performs the following steps:

• removes the accidental Unnamed: 0 index column if present;
• deduplicates rows using all clinical columns except patient_id;
• imputes missing bmi_value values using grouped medians by gender and age band;
• adds interaction features such as age_squared, age_x_glucose, age_x_hypertension, age_x_heart, glucose_x_hypertension, and age_risk_score;
• saves the result to healthcare_data_cleaned.csv.

final_stroke_ensemble.py runs a full leakage-safe ensemble pipeline on the imbalanced dataset:

• loads healthcare_data.csv;
• removes duplicate clinical rows;
• drops leakage-prone or unnecessary columns such as patient_id, age_group, bmi_category, high_glucose, risk_score, and lifestyle_risk when present;
• engineers predictive features including:
  • age_squared
  • age_over_10
  • age_x_hypertension
  • age_x_heart_disease
  • glucose_x_bmi
  • glucose_per_bmi
  • bmi_deviation
  • cvd_count
  • is_senior
  • log_glucose
• splits the data into train, validation, and test partitions with stratification;
• handles class imbalance with SMOTE on the training folds only;
• trains three base models:
  • Logistic Regression
  • Random Forest
  • XGBoost
• combines them with:
  • Soft Voting
  • Stacking
• tunes decision thresholds on the validation set for two operating modes:
  • High Sensitivity
  • Balanced
• evaluates the final models on an untouched test set.

• Original rows: 9722
• After deduplication: 5110
• Duplicates removed: 4612
• Stroke cases: 249
• Stroke prevalence: 4.87%

Train/validation/test split from the latest run:

• Train: 3270
• Validation: 818
• Test: 1022
• Train stroke cases: 159 (4.9%)
• Test stroke cases: 50 (4.9%)
• scale_pos_weight: 19.54

Stacking was the best overall model in the latest run:

• Best AUC-ROC in both modes: 0.8383
• Best AUC-PR in both modes: 0.2150
• Best recall in balanced mode: 0.7800
• Best recall in high-sensitivity mode: 0.9200

Top features from the latest run:

1. age_squared - 0.155550
2. is_senior - 0.131076
3. age - 0.114347
4. cvd_count - 0.088688
5. residence - 0.071208
6. gender - 0.067930
7. has_hypertension - 0.046056
8. smoking_habit - 0.038471
9. age_over_10 - 0.036362
10. employment_type - 0.034493

Running final_stroke_ensemble.py produces:

• stroke_ensemble_roc_pr_fi.png
• stroke_ensemble_confusion.png
• stroke_ensemble_threshold_sweep.png

Running visualization_report.py produces:

• visuals/01_class_balance.png
• visuals/02_numeric_distributions.png
• visuals/03_categorical_counts.png
• visuals/04_boxplots.png
• visuals/05_correlation_heatmap.png
• visuals/06_statistical_tests.png
• visuals/07_model_metric_heatmap.png
• visuals/08_xgb_feature_importance.png
• visuals/visualization_report.md

label_shuffle_test.py was used as a sanity check. When labels are shuffled, the models drop to chance-level performance, which supports the claim that the pipeline is not trivially leaking target information.

Observed shuffled-label behavior from the latest check:

• Logistic Regression ROC-AUC: 0.4943
• Random Forest ROC-AUC: 0.4998
• XGBoost ROC-AUC: 0.4980

This is the expected outcome for a leakage-safe pipeline.

Install the project dependencies with:

pip install -r requirements.txt

Run the cleaning step:

python final_clean.py

Run the final ensemble:

python final_stroke_ensemble.py

Run the label shuffle sanity check:

python label_shuffle_test.py

Generate the standalone visuals and report:

python visualization_report.py

• The latest ensemble run keeps the natural class imbalance instead of forcing a 50/50 split.
• Threshold tuning is used to make the model more practical for stroke screening, where missing true stroke cases is costly.
• final_stroke_ensemble.py is the main reference implementation for the final reported results in this repository.