Effect of Hyperparameter Tuning on Performance on Classification model

Muhammad Sholeh(1*), Uning Lestari(2), Dina Andayati(3),

(1) Universitas AKPRIND Indonesia
(2) Universitas AKPRIND Indonesia
(3) Universitas AKPRIND Indonesia
(*) Corresponding Author

Abstract


This research aims to analyze the effect of hyperparameter tuning on the performance of Logistic Regression, K-Nearest Neighbours, Support Vector Machine, Decision Tree, Random Forest, Random Forest Classifier, Naive Bayes algorithms.  These six algorithms were tested both using hyperparameter tuning and not using hyperparameter tuning. The dataset used in this research is a public dataset, namely the heart datasheet. This datasheet contains information about features related to the diagnosis of heart disease. Hyperparameter tuning is performed using a grid search technique to determine the best combination of hyperparameter values that can improve model accuracy. Performance comparison is done by measuring the accuracy, precision, recall, and F1-score of each algorithm before and after tuning. The research method follows the stages in the Knowledge Discovery in Databases (KDD) methodology. The KDD methodology consists of several stages of data collection, data cleaning to remove errors, data integration from various sources, and data selection and transformation to be ready for analysis. Next, data mining is performed to find patterns or relationships in the data and evaluation and interpretation of the results to ensure their validity. The results show that hyperparameter tuning applied to the six algorithms does not necessarily improve performance. In the algorithm. SVM and decision tree algorithms, the performance results before hyperparameter tuning actually have a higher accuracy value. The performance of algorithms that experienced an increase after hyperparameter tuning was logistic regression and K-Nearest neighbours. The same performance results are generated in the Random Forest and Naive Bayes algorithms. Based on testing the six algorithms and using the heart datasheet, the hyperparameter tuning process does not always result in a better performance value.

Full Text:

PDF

References


[1] D. Cielen, A. D. B. Meysman, and M. Ali, Introducing Data Science. 2016.

[2] M. Arhami and M. Nasir, Data Mining - Algoritma dan Implementasi. Yogyakarta: Penerbit Andi, 2020.

[3] K. Pinaryanto, R. A. Nugroho, and Y. Basilius, “Classification of Toddler Nutrition Using C4 . 5 Decision Tree Method,” Int. J. Appl. Sci. Smart Technol., vol. 3, no. 1, pp. 131–142, 2021.

[4] B. L. Qasthari, E. Susanti, and M. Sholeh, “Classification of Lung and Colon Cancer Histopathological Images Using Convolutional Neural Network ( CNN ) Method on a Pre-Trained Models,” Int. J. Appl. Sci. Smart Technol., vol. 5, no. 1, pp. 133–142, 2023.

[5] M. Barlow, Learning to Love Data Science. Gravenstein Highway North, Sebastopol: O’Reilly Media, Inc, 2015.

[6] D. Sarkar, R. Bali, and T. Sharma, Practical Machine Learning with Python. Bangalore, Karnataka, India: Apress, 2018.

[7] D. N. Ardelia, H. D. Arifin, S. Daniswara, and A. P. Sari, “Klasifikasi Harga Ponsel Menggunakan Algoritma Logistic Regression,” J. INFORMATICS Electron. Eng., vol. 04, no. 01, pp. 37–43, 2024.

[8] R. A. Putri and N. S. Fatonah, “Perbandingan Metode Klasifikasi serta Analisis Faktor Akademis Pola Kelulusan Mahasiswa di Perguruan Tinggi,” J. Inform. J. Pengemb. IT, vol. 7, no. 2, pp. 109–117, 2022, doi: 10.30591/jpit.v7i2.3082.

[9] J. A Ilemobayo et al., “Hyperparameter Tuning in Machine Learning: A Comprehensive Review,” J. Eng. Res. Reports, vol. 26, no. 6, pp. 388–395, 2024, doi: 10.9734/jerr/2024/v26i61188.

[10] M. Arifin and S. Adiyono, “Hyperparameter Tuning in Machine Learning to Predicting Student Academic Achievement,” Int. J. Artif. Intelegence Res., vol. 8, no. 1, pp. 1–8, 2024.

[11] M. Fajri and A. Primajaya, “Komparasi Teknik Hyperparameter Optimization pada SVM untuk Permasalahan Klasifikasi dengan Menggunakan Grid Search dan Random Search,” J. Appl. Informatics Comput., vol. 7, no. 1, pp. 14–19, 2023, doi: 10.30871/jaic.v7i1.5004.

[12] N. Amalia and Asmunin, “Optimasi Algoritma Random Forest dengan Hyperparameter Tuning Menggunakan GridSearchCV untuk Prediksi Nasabah Churn pada Industri Perbankan,” Manaj. Inf., vol. 16, no. 1, pp. 1–9, 2024.

[13] A. Hamied Nababan and M. Y. Hutagalung, “Hyperparameter Tuning Pada Model Stance Detection Menggunakan GridSearchCV,” J. Sains dan Teknol., vol. 5, no. 1, pp. 205–209, 2023, [Online]. Available: https://doi.org/10.55338/saintek.v5i1.1505

[14] T. A. E. Putri, T. Widiharih, and R. Santoso, “Penerapan Tuning Hyperparameter Randomsearchcv Pada Adaptive Boosting Untuk Prediksi Kelangsungan Hidup Pasien Gagal Jantung,” J. Gaussian, vol. 11, no. 3, pp. 397–406, 2023, doi: 10.14710/j.gauss.11.3.397-406.

[15] A. Baita, I. A. Prasetyo, and N. Cahyono, “Hyperparameter Tuning on Random Forest for Diagnose Covid-19,” JIKO (Jurnal Inform. dan Komputer), vol. 6, no. 2, pp. 138–143, 2023, doi: 10.33387/jiko.v6i2.6389.

[16] S. W. Lubis, P. P. Adikara, and B. Darma, “Optimasi Hyperparameter Support Vector Machine Dengan Particle Swarm Optimization Terhadap Klasifikasi Citra Digital Imbalanced,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. Vol 8 No 3, 2024, [Online]. Available: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/13508

[17] G. F. Fahrudin, S. Suroso, and S. Soim, “Pengembangan Model Support Vector Machine untuk Meningkatkan Akurasi Klasifikasi Diagnosis Penyakit Jantung,” J. Teknol. Sist. Inf. dan Apl., vol. 7, no. 3, pp. 1418–1428, 2024, doi: 10.32493/jtsi.v7i3.42254.

[18] W. Firgiawan, D. Yustianisa, and N. A. Nur, “Hyperparameter Tuning for Optimizing Stunting Classification with KNN , SVM , and Naïve Bayes Algorithms,” J. TEKNO KOMPAK, vol. 19, no. 1, pp. 92–104, 2025.

[19] P. J. Deo and Y. B. D. Setianto, “K-Nearest Neighbor ( KNN ) Algorithm Performance In Diabetes Case Study,” PROXIES, vol. 6, no. 1, pp. 93–102, 2022.

[20] M. Rizki, A. Hermawan, and D. Avianto, “Optimization of Hyperparameter K in K-Nearest Neighbor Using Particle Swarm Optimization,” JUITA J. Inform., vol. 12, no. 1, p. 71, 2024, doi: 10.30595/juita.v12i1.20688.

[21] N. Hendradinata, I. Gede, and S. Astawa, “Hyperparameter Tuning Algoritma KNN Untuk Klasifikasi Kanker Payudara Dengan Grid Search CV,” JNATIA Vol., vol. 1, no. November, pp. 397–402, 2022.

[22] S. Suraya, M. Sholeh, and D. Andayati, “Penerapan Metode Clustering Dengan Algoritma K-Means Pada Pengelompokan Indeks Prestasi Akademik Mahasiswa,” Skanika, vol. 6, no. 1, pp. 51–60, 2023, doi: 10.36080/skanika.v6i1.2982.




DOI: https://doi.org/10.24071/ijasst.v7i1.11735

Refbacks

  • There are currently no refbacks.









Publisher : Faculty of Science and Technology

Society/Institution : Sanata Dharma University

 

 

 

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.