SVM and Ensemble Majority Voting Algorithm on Sentiment Analysis of Using chatGPT in Education

Hildegardis Yayukristi Weko(1), Hari Suparwito(2*),

(1) Sanata Dharma University
(2) Sanata Dharma University
(*) Corresponding Author

Abstract


The pros and cons of using ChatGPT in education have caused academic debate as it has influenced current educational praxis. Discussions about the possibility of ChatGPT for writing manuscripts or doing assignments are rife on social media, one of which is Twitter. The purpose of this study is to understand the public perception of the use of ChatGPT in education. The proposed method is sentiment analysis with SVM and Majority Voting algorithms. SVM is one of the superior algorithms in pattern recognition and is suitable for use in classification. The Majority Voting ensemble algorithm combines independent algorithms' prediction results. In this research, majority voting uses three base classifiers, namely Naïve Bayes, Random Forest, and KNN. The results of the study showed that the accuracy of SVM is 83.6% and Majority Voting is 85.4%, with the accuracy of the NB, RF, and KNN base classifiers of 76.82%, 80.91%, and 74.5%, respectively. This proved that the Majority Voting Ensemble is superior to individual algorithms with higher accuracy values. This follows the results of previous research, where the ensemble performs better than the individual algorithm. The accuracy values of SVM and the Ensemble Majority Voting models showed that both models could successfully classify sentiment on tweet data for using ChatGPT in education.


Full Text:

PDF

References


[1] M. Dowling and B. Lucey, “ChatGPT for (Finance) research: The Bananarama Conjecture,” Financ Res Lett, vol. 53, May 2023, doi: 10.1016/j.frl.2023.103662.

[2] Ö. Aydın and E. Karaarslan, “OpenAI ChatGPT Generated Literature Review: Digital Twin in Healthcare,” SSRN Electronic Journal, Dec. 2022, doi: 10.2139/ssrn.4308687.

[3] D. Baidoo-Anu and L. Ansah, “Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning,” Oct. 2023.

[4] U. Bukar, M. S. Sayeed, S. F. Abdul Razak, S. Yogarayan, and O. Amodu, “Text Analysis of Chatgpt as a Tool for Academic Progress or Exploitation,” SSRN Electronic Journal, Oct. 2023, doi: 10.2139/ssrn.4381394.

[5] I. Arifdarma, “Pengaruh Teknologi ChatGPT Terhadap Dunia Pendidikan: Potensi dan Tantangan,” Jurnal Agriwidya, vol. 4, no. 1, Mar. 2023, Accessed: Oct. 18, 2023. [Online]. Available: https://repository.pertanian.go.id/handle/123456789/20278

[6] G. Koca, “Sentiment analysis with twitter data on bitcoin,” Anadolu Üniversitesi İktisadi ve İdari Bilim. Fakültesi Derg, vol. 22, no. 4, pp. 19–30, 2021.

[7] B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Foundations and Trends® in information retrieval, vol. 2, no. 1–2, pp. 1–135, 2008.

[8] A. Sarker, M. S. U. Zaman, and M. A. Y. Srizon, “Twitter data classification by applying and comparing multiple machine learning techniques,” International Journal of Innovative Research in Computer Science & Technology (IJIRCST) ISSN, pp. 2347–5552, 2019.

[9] M. Tubishat, F. Al-Obeidat, and A. Shuhaiber, “Sentiment Analysis of Using ChatGPT in Education,” WSEAS TRANSACTIONS ON ADVANCES in ENGINEERING EDUCATION, vol. 20, pp. 60–66, Oct. 2023, doi: 10.37394/232010.2023.20.9.

[10] A. Alsaeedi and M. Z. Khan, “A study on sentiment analysis techniques of Twitter data,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 2, pp. 361–374, 2019, doi: 10.14569/ijacsa.2019.0100248.

[11] A. K. Putri and H. Suparwito, “Uji Algoritma Stacking Ensemble Classifier pada Kemampuan Adaptasi Mahasiswa Baru dalam Pembelajaran Online,” KONSTELASI: Konvergensi Teknologi dan Sistem Informasi, vol. 3, no. 1, pp. 1–12, 2023.

[12] T. Wolf et al., “Transformers: State-of-the-Art Natural Language Processing,” Oct. 2020, pp. 38–45. doi: 10.18653/v1/2020.emnlp-demos.6.

[13] Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach.” Oct. 2019.

[14] Wilson Wongso, “indonesian-roberta-base-sentiment-classifier (Revision e402e46).” Hugging Face, Indonesia, 2023. Accessed: Oct. 18, 2023. [Online]. Available: https://huggingface.co/w11wo/indonesian-roberta-base-sentiment-classifier

[15] A. Indrawati, H. Subagyo, A. Sihombing, and S. A. Wagiyah, “Analyzing the Impact of Resampling Method for Imbalanced Data Text in Indonesian Scientific Articles Categorization,” BACA: Jurnal Dokumentasi dan Informasi, vol. 42, no. 2, pp. 133–141, 2020.

[16] C. Zhai and S. Massung, Text data management and analysis: a practical introduction to information retrieval and text mining. Morgan & Claypool, 2016.

[17] F. Alzami, E. D. Udayanti, D. P. Prabowo, and R. A. Megantara, “Document Preprocessing with TF-IDF to Improve the Polarity Classification Performance of Unstructured Sentiment Analysis,” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, pp. 235–242, Aug. 2020, doi: 10.22219/kinetik.v5i3.1066.

[18] M. U. Muhammad et al., “Sentiment Analysis of Students’ Feedback before and after COVID-19 Pandemic,” International Journal on Emerging Technologies, vol. 12, no. 2, pp. 177–182, 2021, [Online]. Available: www.researchtrend.net

[19] S. Shalev-Shwartz and S. Ben-David, Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.

[20] A. L. Hananto, B. Priyatna, A. Hananto, and A. P. Nardilasari, DATA MINING : Penerapan Algoritma (SVM, Naive Bayes, KNN) Dan Implementasi Menggunakan Rapid Miner, . Bandung, Jawa Barat: Media Sains Indonesia, 2023.

[21] N. Pavitha et al., “Movie recommendation and sentiment analysis using machine learning,” Global Transitions Proceedings, vol. 3, no. 1, pp. 279–284, 2022.

[22] F. Rahmadayana and Y. Sibaroni, “Sentiment analysis of work from home activity using SVM with randomized search optimization,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 5, no. 5, pp. 936–942, 2021.

[23] J. Sadhasivam and R. B. Kalivaradhan, “Sentiment analysis of Amazon products using ensemble machine learning algorithm,” International Journal of Mathematical, Engineering and Management Sciences, vol. 4, no. 2, p. 508, 2019.

[24] Z.-H. Zhou, Ensemble methods: foundations and algorithms. CRC press, 2012.

[25] M. Khalid, I. Ashraf, A. Mehmood, S. Ullah, M. Ahmad, and G. S. Choi, “GBSVM: Sentiment classification from unstructured reviews using ensemble classifier,” Applied Sciences (Switzerland), vol. 10, no. 8, Apr. 2020, doi: 10.3390/APP10082788.

[26] M. Hosni, I. Abnane, A. Idri, J. M. Carrillo de Gea, and J. L. Fernández Alemán, “Reviewing ensemble classification methods in breast cancer,” Comput Methods Programs Biomed, vol. 177, pp. 89–112, 2019, doi: https://doi.org/10.1016/j.cmpb.2019.05.019.

[27] R. M. K. Saeed, S. Rady, and T. F. Gharib, “An ensemble approach for spam detection in Arabic opinion texts,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 1, pp. 1407–1416, 2022, doi: https://doi.org/10.1016/j.jksuci.2019.10.002.

[28] J. Brownlee, “Why Use Ensemble Learning?,” machinelearningmastery.com. Accessed: Nov. 06, 2023. [Online]. Available: https://machinelearningmastery.com/why-use-ensemble-learning/#:~:text=There%20are%20two%20main%20reasons,the%20predictions%20and%20model%20performance




DOI: https://doi.org/10.24071/ijasst.v7i2.12680

Refbacks

  • There are currently no refbacks.









Publisher : Faculty of Science and Technology

Society/Institution : Sanata Dharma University

 

 

 

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.