A Comparative Analysis of Hyperparameter Optimization Strategies for CatBoost Classifier: TPE, CMA-ES, and Grid Search Approaches

Tunç B. N., Güngör S., Atasever Ü. H., Elzein A.

45th EARSeL Symposium, Athens, Yunanistan, 29 Eylül - 02 Ekim 2026, (Tam Metin Bildiri)

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Basıldığı Şehir: Athens
  • Basıldığı Ülke: Yunanistan
  • Erciyes Üniversitesi Adresli: Evet

Özet

Precise hyperparameter configuration is a decisive factor in the predictive performance of gradient-boosting classifiers applied to hyperspectral imagery, yet the search strategy used to obtain that configuration is rarely subjected to the same methodological scrutiny as the classifier itself. This paper presents a systematic, repeated-run comparison of three hyperparameter optimization (HPO) strategies — the Tree-structured Parzen Estimator (TPE), the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), and exhaustive Grid Search — applied to a CatBoost classifier over a five-dimensional search space (number of iterations, tree depth, learning rate, L2 leaf regularization, border count) for the classification of the Salinas-A hyperspectral benchmark. Each sampler was executed for five independent repetitions of fifty trials, and the resulting 750 CatBoost fits were evaluated on a held-out 2,141-pixel partition using Accuracy, F1-score, Cohen’s Kappa, the Matthews Correlation Coefficient, the Structural Similarity Index (SSIM) of the resulting classification maps, and total wall-clock search duration. TPE and CMA-ES reached an identical peak test accuracy of 0.8963, marginally above the 0.8935 achieved by Grid Search, with mean accuracies across the five repetitions differing by only 0.06 percentage points. CMA-ES, however, required only 3,037 s on average to reach this performance while also producing the spatially most consistent classification maps (SSIM = 0.8171). These results indicate that, on hyperspectral benchmarks of this scale, sampler choice for CatBoost should be driven primarily by search efficiency and solution stability rather than by an expectation of accuracy gain.