Image-Based Facial Skin Undertone Classification Using ResNet50 and MobileNetV2
Keywords:
classification, skin tone, deep learning, ResNet50, MobileNetV2Abstract
This study aims to develop an automatic facial skin undertone classification system based on deep learning using facial images. The problem stems from the manual undertone identification process, which is subjective and potentially inconsistent when determining skin color categories. The method uses a Convolutional Neural Network (CNN) and compares two architectures, ResNet50 and MobileNetV2, for classifying warm, cool, and neutral skin undertones. The dataset combines secondary data from the Hugging Face platform and primary data obtained through direct facial image capture. The study follows the CRISP-DM process, including business understanding, data understanding, data preparation, modeling, evaluation, and deployment. ResNet50 achieved an accuracy of 89.3% and a weighted F1-score of 0.893, whereas MobileNetV2 achieved an accuracy of 68.3% and a weighted F1-score of 0.684. Thus, ResNet50 demonstrated better and more stable performance for facial skin undertone classification.
Downloads
References
Butt, M.A., & Zaman, M. (2023). AttentionRes-UNet: Attention residual UNet with focal tverskyloss for skin lesion segmentation. International Journal of Decision Support System Technology, 15(1), 1–17.
Dhewayani, F. N., Amelia, D., Alifah, D. N., Sari, B. N., & Jajuli, M. (2022). Implementasi k-means clustering untuk pengelompokan daerah rawan bencana kebakaran menggunakan model CRISP-DM. Jurnal Teknologi Dan Informasi (JATI), 12, 64–77.
Faritzie, H.A., Firda, A., & Aprilyanti, S. (2022). Identifikasi dan analisis kerusakan jalan pada ruas jalan siaransakokotapalembang. BEARING: Jurnal Penelitian Dan Kajian Teknik Sipil, 7, 223–229.
Fatih, M., Suciati, N., & Navastara, D. A. (2021). Deteksi kerumunan menggunakan metode Fully Convolutional Network pada kamera. Jurnal Teknik ITS, 10(2), 339–344.
Hari, E., Prasetyo, A., & Rosyidah, A.N. (2025). Deteksi kerusakan jalan menggunakan model deep learning YOLOv8. INOVATE: Jurnal Ilmiah Inovasi Teknologi Informasi, 10(1), 105–114.
Huang, H., Ho, G.T.S., Tang, V., Chow, E.W.H., Ip, W.H., & Yung, K.L. (2025). A lightweight fully convolutional neural network for efficient and accurate tunnel crack detection. Journal of Advanced Research.
Huang, S.-Y., Hsu, W.-L., Hsu, R.-J., & Liu, D.-W. (2022). Fully convolutional network for the semantic segmentation of medical images: A survey. Diagnostics, 1–21.
Li, H., & Liang, B. (2023). Liver tumor computed tomography image segmentation based on an improved U-Net model. Applied Sciences, 13(20), 11283.
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. doi: 10.1109/5.726791.
Lin, T., Ai, F., Doll, P. (n.d.). Focal loss for dense object detection. (Original publication information not fully provided).
Mart, F., Contreras-Ochando, L., & Lachiche, N. (2021). CRISP-DM twenty years later: From data mining processes to data science trajectories. IEEE Transactions on Knowledge and Data Engineering, 33(8), 3048–3061. doi: 10.1109/TKDE.2019.2962680.
Nurapipah, N., & Yuliana, S.S. (2025). Skin tone classification in digital images using cnn for make-up and color recommendation. Journal of Intelligent Systems Technology and Informatics, 1(2), 78–85. doi: 10.64878/jistics.v1i2.29.
Prabowo, F.W., Homaidi, A., & Lutfi, A. (2024). Deteksi warna kulit menggunakan metode deep learning dengan cnn (convolutional neural network). Jurnal Teknik Elektro Dan Informatika, 19, 186–190.
Sandler, M., Zhu, M., Zhmoginov, A., & Mar, C.V. (n.d.). Mobilenetv2: Inverted residuals and linear bottlenecks. (Original publication information not fully provided).
Santosa, R.R., Sasikirana, A., Ramadhan, Z.L., Putra, Y., & Satria, B. (2025). Perbandingan kinerja mobilenetv2 dan resnet50v2 dalam klasifikasi tingkat kematangan tomat. Indonesian Journal of Science, Technology and Humanities, 3, 49–56. Retrieved from https://jurnal.intekom.id/index.php/ijstech
Sathyanarayanan, S., Tantri, B.R. (2024). Confusion matrix-based performance evaluation metrics. AJBR, 27(4S). doi: 10.53555/AJBR.v27i4S.4345.
Schröer, C., Kruse, F., & Gomez, J.M. (2021). A systematic literature review on applying process models. Procedia Computer Science, 181, 526–534. doi: 10.1016/j.procs.2021.01.199.
Sihombing, P.R., & Arsani, A.M. (2021). Perbandingan metode machine learning dalam klasifikasi kemiskinan di indonesia tahun 2018. Jurnal Teknik Informatika (JUTIF), 2(1), 51–56. doi: 10.20884/1.jutif.2021.2.1.52.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Intelligent Systems for Community Development
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0) 
.
