Comparative study of NBC, SVM, and CNN performance in sentiment analysis of the millennial farmers program on platform X
Keywords:
Classification, Sentiment Analysis, Naïve Bayes, Deep Learning, Machine LearningAbstract
Public perception of government programs is often expressed on social media, making sentiment analysis a vital tool for understanding public opinion. This research presents a comparative study of three machine learning algorithms, Naive Bayes Classifier (NBC), Support Vector Machine (SVM) and Convolutional Neural Network (CNN), applied to sentiment analysis on the Millennial Farmers Program 2024 from Platform X. The study aims to measure the performance of each algorithm in terms of accuracy, execution time, and memory efficiency by adopted a quantitative approach with a comparative experimental design. Sentiment data were collected from platform X (Twitter) between October 2024 and January 2025, with a total of 5,177 raw tweets acquired using the keyword "petani_milenial". After data preprocessing steps, including cleaning, case folding, tokenization, stopword removal, and stemming, the data's sentiment (positive, negative, neutral) was automatically labeled using the IndoBERT model. The sentiment analysis results revealed a dominance of neutral sentiment (832 tweets), followed by negative (331 tweets) and positive (197 tweets). Model performance evaluation showed that SVM achieved the highest overall accuracy at 79%, demonstrating superior capability in classifying neutral sentiment. CNN, with an accuracy of 74.26%, stood out in recall for positive sentiment (73%), making it an effective choice for comprehensive identification of positive sentiment. Meanwhile, NBC, with 78% accuracy, proved to be the most efficient in terms of computation time and memory usage. The study concludes that the optimal model selection greatly depends on the specific use-case priority, whether overall accuracy, positive sentiment identification, or computational efficiency. The results show that CNN outperforms NBC and SVM in accuracy, while NBC is the fastest and consumes the least memory. These findings can help inform future research and implementation in large-scale sentiment analysis applications.
Downloads
References
R. Tiwari and A. Ahamad, “Twitter sentiment analysis,” no. 05, pp. 8977–8980, 2023.
J. Mishra, U. Pradesh, A. Kumar, and U. Pradesh, “Twitter sentiment analysis,” 2023, doi: 10.55041/IJSREM24071.
L. Islam, M. Mustufa, S. Shaikh, U. Shaikh, and Z. Kazi, “Twitter Sentiment Analysis,” pp. 476–481, 2023, doi: 10.48175/IJARSCT-9448.
L. Hutao, “Machine Learning-based Voting Classifier for Improving Sentiment Analysis on Twitter Data,” Trans. Comput. Sci. Intell. Syst. Res., vol. 5, pp. 2960–2238, 2024.
I. Ilyas, “Optimalisasi peran petani milenial dan digitalisasi pertanian dalam pengembangan pertanian di Indonesia,” Forum Ekon., vol. 24, no. 2, pp. 259–266, 2022, doi: 10.30872/jfor.v24i2.10364.
T. Prasetiawan, “Memantik Minat Petani Milenial,” 2024.
F. Nurrizky and S. Dwiasnati, “Comparison of Naive Bayes and Support Vector Machine (SVM) Algorithms Regarding The Popularity of Presidential Candidates In The Upcoming 2024 Presidential Election,” Comput. Eng. Appl. J., vol. 13, no. 1, pp. 17–28, 2024, doi: 10.18495/comengapp.v13i1.459.
S. Sambhaji Patil, A. Rodrigues, R. Telangi, and V. Chavan, “Convolutional Neural Networks for Text Classification: A Comprehensive Analysis,” vol. 11, no. May, 2023, [Online]. Available: www.ijraset.com
S. Bogle, “Using Sentiment Analysis and Machine Learning Algorithms to Determine Citizens’ Perceptions,” 2018.
T. Agustini, “Sentiment Analysis on Social Media using Machine Learning-Based Approach,” no. June, p. 544437, 2021.
M. Tabany and M. Gueffal, “Sentiment Analysis and Fake Amazon Reviews Classification Using SVM Supervised Machine Learning Model,” J. Adv. Inf. Technol., vol. 15, no. 1, pp. 49–58, 2024, doi: 10.12720/jait.15.1.49-58.
A. S. Rahayu, A. Fauzi, and R. Rahmat, “Komparasi Algoritma Naïve Bayes Dan Support Vector Machine (SVM) Pada Analisis Sentimen Spotify,” J. Sist. Komput. dan Inform., vol. 4, no. 2, p. 349, 2022, doi: 10.30865/json.v4i2.5398.
I. W. D. Wiguna, D. V. Waas, I. K. A. G. Wiguna, and M. L. Radhitya, “Comparison of SVM & Naïve Bayes Methods in Sentiment Analysis of Electric Vehicle Subsidy Policy Based on X Data,” J. Eng. Sci. Res., vol. 6, no. 1, pp. 23–30, 2024, doi: 10.23960/jesr.v6i1.158.
S. Pokhrel, “ANALISIS SENTIMEN TEXT DENGAN METODE CNN STUDY KASUS TEMPAT WISATA MAKASSAR,” Αγαη, vol. 15, no. 1, pp. 37–48, 2024.
L. K. Ramasamy, S. Kadry, Y. Nam, and M. N. Meqdad, “Performance analysis of sentiments in Twitter dataset using SVM models,” Int. J. Electr. Comput. Eng., vol. 11, no. 3, pp. 2275–2284, 2021, doi: 10.11591/ijece.v11i3.pp2275-2284.
U. E. Orji, M. E. Ezema, and J. C. Agbo, “Bayes Algorithm for Sentiment Analysis,” pp. 412–419, 2021.
R. Risnantoyo, A. Nugroho, and K. Mandara, “Sentiment Analysis on Corona Virus Pandemic Using Machine Learning Algorithm,” J. Informatics Telecommun. Eng., vol. 4, no. 1, pp. 86–96, 2020, doi: 10.31289/jite.v4i1.3798.
I. Hakim, A. Nugroho, S. H. Sukmana, and W. Gata, “Sentimen Analisis Stay Home menggunakan metode klasifikasi Naive Bayes, Support Vector Machine, dan k-Nearest Neighbor,” Paradig. - J. Komput. dan Inform., vol. 22, no. 2, pp. 169–174, 2020, doi: 10.31294/p.v22i2.8237.
T. Fadiyah Basar, D. E. Ratnawati, and I. Arwani, “Analisis Sentimen Pengguna Twitter terhadap Pembayaran Cashless menggunakan Shopeepay dengan Algoritma Random Forest,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 6, no. 3, pp. 1426–1433, 2022, [Online]. Available: http://j-ptiik.ub.ac.id
Y. A. Singgalen, “Pemilihan Metode dan Algoritma dalam Analisis Sentimen di Media Sosial : Sistematic Literature Review,” J. Inf. Syst. Informatics, vol. 3, no. 2, pp. 278–302, 2021, doi: 10.33557/journalisi.v3i2.125.
F. S. Pamungkas and I. Kharisudin, “Analisis Sentimen dengan SVM, NAIVE BAYES dan KNNuntuk Studi Tanggapan Masyarakat Indonesia TerhadapPandemi Covid-19 pada Media Sosial Twitter,” Pros. Semin. Nas. Mat., vol. 4, pp. 1–7, 2021, [Online]. Available: https://journal.unnes.ac.id/sju/prisma/article/view/45038
I. Wulansari and R. Arief, “Analisis Performa Metode Convolutional Neural Network (Cnn) Dengan Word Embedding Glove Pada Klasifikasi Sentimen Dari Twitter,” J. Ilm. Teknol. dan Rekayasa, vol. 28, no. 3, pp. 252–264, 2023, doi: 10.35760/tr.2023.v28i3.6090.
B. A. Yuniarossy, K. M. Hindrayani, and A. T. Damaliana, “Analisis Sentimen Terhadap Isu Feminisme Di Twitter Menggunakan Model Convolutional Neural Network (Cnn),” J. Lebesgue J. Ilm. Pendidik. Mat. Mat. dan Stat., vol. 5, no. 1, pp. 477–491, 2024, doi: 10.46306/lb.v5i1.585.
S. R. Gayam, “Artificial Intelligence for Natural Language Processing : Techniques for Sentiment Analysis , Language Translation , and Conversational Agents,” vol. 1, no. 1, pp. 175–215, 2021.
E. Retnoningsih and R. Pramudita, “Mengenal Machine Learning Dengan Teknik Supervised Dan Unsupervised Learning Menggunakan Python,” Bina Insa. Ict J., vol. 7, no. 2, p. 156, 2020, doi: 10.51211/biict.v7i2.1422.
A. V. Ramana, K. S. N. Prasad, and A. S. Rao, “Nonlinear Deep Learning Framework for Precise Sentiment Classification in Textual Data,” Commun. Appl. Nonlinear Anal., vol. 32, no. 1S, pp. 573–584, 2025, doi: 10.52783/cana.v32.2345.
Z. Kastrati, F. Dalipi, A. S. Imran, K. P. Nuci, and M. A. Wani, “Sentiment analysis of students’ feedback with nlp and deep learning: A systematic mapping study,” Appl. Sci., vol. 11, no. 9, 2021, doi: 10.3390/app11093986.
F. Şengül, K. Adem, and E. Kavalcı Yılmaz, “Sentiment Analysis Based On Machine Learning Methods On Twitter Data Using OneAPI,” pp. 207–213, 2023, [Online]. Available: http://as-proceeding.com/:Konya,Turkeyhttps://as-proceeding.com/index.php/iccar
R. Vidhya, P. Gopalakrishnan, and N. Vallamkondu, “Sentiment Analysis Using Machine Learning Classifiers: Evaluation of Performance,” 2021, doi: 10.4108/eai.7-6-2021.2308565.
R. Patil, S. Boit, V. Gudivada, and J. Nandigam, “A Survey of Text Representation and Embedding Techniques in NLP,” IEEE Access, vol. 11, no. March, pp. 36120–36146, 2023, doi: 10.1109/ACCESS.2023.3266377.
K. L. Tan, C. P. Lee, and K. M. Lim, “A Survey of Sentiment Analysis: Approaches, Datasets, and Future Research,” Appl. Sci., vol. 13, no. 7, 2023, doi: 10.3390/app13074550.
M. Bibi, W. Aziz, M. Almaraashi, I. H. Khan, M. S. A. Nadeem, and N. Habib, “A Cooperative Binary-Clustering Framework Based on Majority Voting for Twitter Sentiment Analysis,” IEEE Access, vol. 8, pp. 68580–68592, 2020, doi: 10.1109/ACCESS.2020.2983859.
E. Sutoyo and A. Almaarif, “Twitter sentiment analysis of the relocation of Indonesia’s capital city,” Bull. Electr. Eng. Informatics, vol. 9, no. 4, pp. 1620–1630, 2020, doi: 10.11591/eei.v9i4.2352.
G. Y. Sitio, A. Rumapea, and D. P. Lumbanraja, “Analisis Sentimen Pemindahan Ibu Kota Negara Di Media Sosial Twitter Menggunakan metode Convolutional Neural Network (CNN),” Methotika J. Ilm. Tek. Inform., vol. 3, no. 2, pp. 97–104, 2023, [Online]. Available: http://ojs.fikom-methodist.net/index.php/methotika
G. W. Lindsay, “Convolutional neural networks as a model of the visual system: Past, present, and future,” J. Cogn. Neurosci., vol. 33, no. 10, pp. 2017–2031, 2021, doi: 10.1162/jocn_a_01544.
M. Metode, “Analisis Sentimen Berbasis Aspek pada EDOM Pembelajaran Aspect-Based Sentiment Analysis in EDOM Learning Using CNN and Word2vec Methods,” vol. 12, no. 3, 2024, doi: 10.26418/justin.v12i3.75610.
N. W. Ernawati, I. N. Satya Kumara, and W. Setiawan, “Perbandingan Metode Klasifikasi Support Vector Machine Dan Naïve Bayes Pada Analisis Sentimen Kendaraan Listrik,” J. SPEKTRUM, vol. 10, no. 3, p. 106, 2023, doi: 10.24843/spektrum.2023.v10.i03.p12.
M. A. Ayu and A. H. Muhendra, “Preprocessing of Slang Words for Sentiment Analysis on Public Perceptions in Twitter”.
F. Hassan et al., “Performance evolution for sentiment classification using machine learning algorithm,” J. Appl. Res. Technol. Eng., vol. 4, no. 2, pp. 97–110, 2023, doi: 10.4995/jarte.2023.19306.
M. M. Hamed Taherdoost, “Artificial Intelligence and Sentiment Analysis : A Review in,” Comput. 2023, 12, 37. https// doi.org/10.3390/computers12020037, no. 0, 2023.
A. Nur Oktavia, M. Iqbal, R. W. Saputra, M. I. Zulfikar, and A. Saifudin, “Implementasi Metode Natural Language Processing Dalam Studi Analisis,” J. Ilm. Ilmu Komput. dan Multimed., vol. 2, no. 1, pp. 154–159, 2024, [Online]. Available: https://jurnalmahasiswa.com/index.php/biikma
E. D. N. Sari and I. Irhamah, “Analisis Sentimen Nasabah pada Layanan Perbankan Menggunakan Metode Regresi Logistik Biner, Naïve Bayes Classifier (NBC), dan Support Vector Machine (SVM),” J. Sains dan Seni ITS, vol. 8, no. 2, 2020, doi: 10.12962/j23373520.v8i2.44565.
R. A. Rudiyanto and E. B. Setiawan, “Sentiment Analysis Using Convolutional Neural Network (CNN) and Particle Swarm Optimization on Twitter,” JITK (Jurnal Ilmu Pengetah. dan Teknol. Komputer), vol. 9, no. 2, pp. 188–195, 2024, doi: 10.33480/jitk.v9i2.5201.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0) 
