Halide Yılmaz Evaluation
4 posts
Evaluation9 minBenchmark Contamination: Why Leaderboard Scores Are Inflated
Test set leakage is no longer one team's cross-validation mistake but a defect in the whole field's evaluation infrastructure. How contamination and saturation inflate scores, and how to build your own private held-out set.
Evaluation9 minIs That 1 % Real? Confidence Intervals and Bootstrap for Comparing Models
One model beating another by a point is usually noise. How test set sampling error, bootstrap confidence intervals, paired tests and multiple comparison correction tell you whether the gap between two models is real.
Evaluation3 minClass Imbalance: Building Models That Do Not Miss the Rare Event
In fraud, failure or rare-disease problems the positive class is tiny. How resampling, class weights, threshold tuning and the precision-recall curve let you build a model that actually works on imbalanced data.
Evaluation3 minData Leakage and Cross-Validation: Why Your Test Score Is Lying
A validation score that looks too good is usually a sign of leakage. Temporal leakage, group leakage, scaling mistakes, and how to choose the right cross-validation scheme.