A Prompt-Optimized Vision-Language Model System for Fine-Grained Fashion Style Recognition
DOI:
https://doi.org/10.64509/jdi.14.118Keywords:
Clothing Style Recognition, Vision-Language Model, Prompt Engineering, Confidence Distribution, Lightweight DeploymentAbstract
With the accelerated digital transformation of the fashion industry, accurate clothing style recognition has become essential for intelligent e-commerce retrieval, personalized outfit recommendation, and fashion trend analysis. Vision-language models (VLMs), with their cross-modal semantic alignment and zero-shot inference capabilities, offer an efficient technical pathway for this task. However, when applied to the fashion domain, general-purpose VLMs face several challenges, including vague prompt semantics, insufficient fine-grained discrimination, difficulty in capturing composite style characteristics, and high deployment costs. To address these issues, this paper presents a comprehensive clothing style recognition system based on Qwen3-VL-Plus. A four-level structured prompt template—comprising style definitions, visual feature guidance, and confidence distribution constraints—is introduced to direct the model's attention toward fine-grained attributes such as silhouette, fabric, color, and decoration. A confidence distribution modeling mechanism based on Softmax normalization is proposed to convert cross-modal similarity scores into interpretable confidence distributions, enabling quantitative representation of composite styles. Furthermore, INT8 quantization and inference pipeline optimization are employed to achieve lightweight deployment, allowing the system to operate stably in an ordinary CPU environment. On the FashionStyle14 dataset, the proposed system achieves a Top-1 accuracy of 69.21% and a macro-average F1 score of 66.38%, outperforming CLIP by 17.98 and 18.21 percentage points, respectively. INT8 quantization yields a 36.77% reduction in inference time and a 75.12% reduction in model size. In a user experience evaluation involving 30 participants, the system attains an overall satisfaction score of 8.60 out of 10, with 96.67% of users recognizing the interpretability of the recognition results. Experimental results demonstrate that the proposed approach offers significant advantages in recognition accuracy, interpretability, and deployment practicality, providing a complete solution with both technical feasibility and application value for intelligent clothing style recognition in the fashion industry.
Downloads
References
[1] Casciani, D., Chkanikova, O., Pal, R.: Exploring the nature of digital transformation in the fashion industry: Opportunities for supply chains, business models, and sustainability-oriented innovations. Sustainability: Science, Practice and Policy 18(1), 773-795 (2022). https://doi.org/10.1080/15487733.2022.2125640 DOI: https://doi.org/10.1080/15487733.2022.2125640
[2] Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: DeepFashion: Powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1096-1104 (2016). https://doi.org/10.1109/CVPR.2016.124 DOI: https://doi.org/10.1109/CVPR.2016.124
[3] Wang, L.C., Zeng, X.Y., Koehl, L., Chen, Y.: Intelligent fashion recommender system: Fuzzy logic in personalized garment design. IEEE Transactions on Human-Machine Systems, 45(1), 95-109 (2015). https://doi.org/10.1109/THMS.2014.2364398 DOI: https://doi.org/10.1109/THMS.2014.2364398
[4] Yue, X., Zhang, C., Fujita, H., Liu, J., Li, Y.: Clothing fashion style recognition with design issue graph. Applied Intelligence, 51(6), 3548-3560 (2021). https://doi.org/10.1007/s10489-020-01950-7 DOI: https://doi.org/10.1007/s10489-020-01950-7
[5] Ortega Ballesteros, G.: Personalized style-specific outfit generation from garment wardrobes. Master's Thesis, Universitat Politecnica de Catalunya, Spain (2025)
[6] Paulauskaite-Taraseviciene, A., Noreika, E., Purtokas, R., Lagzdinyte-Budnike, I., Daniulaitis, V., Salickaite-Zukauskiene, R.: An intelligent solution for automatic garment measurement using image recognition technologies. Applied Sciences, 12(9), 4470 (2022). https://doi.org/10.3390/app12094470 DOI: https://doi.org/10.3390/app12094470
[7] Hafner, M., Katsantoni, M., Köster, T., Marks, J., Mukherjee, J., Staiger, D., Ule, J., Zavolan, M.: CLIP and complementary methods. Nature Reviews Methods Primers, 1(1), 20 (2021). https://doi.org/10.1038/s43586-021-00018-1 DOI: https://doi.org/10.1038/s43586-021-00018-1
[8] Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-VL technical report. arXiv preprint arXiv:2511.21631 (2025). https://doi.org/10.48550/arXiv.2511.21631
[9] Shen, S., Li, L.H., Tan, H., Bansal, M., Rohrbach, A., Chang, K.W., Yao, Z., Keutzer, K.: How much can CLIP benefit vision-and-language tasks? arXiv preprint arXiv:2107.06383 (2021). https://doi.org/10.48550/arXiv.2107.06383
[10] Ghosh, A., Acharya, A., Saha, S., Jain, V., Chadha, A.: Exploring the frontier of vision-language models: A survey of current methodologies and future directions. arXiv preprint arXiv:2404.07214 (2024). https://doi.org/10.48550/arXiv.2404.07214 DOI: https://doi.org/10.2139/ssrn.4783140
[11] Wang, C., Wang, Z., Xu, X., Tang, Y., Zhou, J., Lu, J.: Q-VLM: Post-training Quantization for Large Vision-Language Models. In Proceedings of the 38th International Conference on Neural Information Processing Systems, pp.114553-114573 (2024). https://doi.org/10.52202/079017-3638 DOI: https://doi.org/10.52202/079017-3638
[12] Lee, J., Sul, I.H.: Construction of garment pattern shape information system using image analysis and shape recognition techniques. International Journal of Clothing Science and Technology, 28(4), 543-555 (2016). https://doi.org/10.1108/IJCS-10-2015-0114 DOI: https://doi.org/10.1108/IJCST-10-2015-0114
[13] Cheng, C., Liu, D.S.-M.: An Intelligent clothing search system based on fashion style. In 2008 International Conference on Machine Learning and Cybernetics, pp.1592-1597 (2008). https://doi.org/10.1109/ICMLC.2008.4620660 DOI: https://doi.org/10.1109/ICMLC.2008.4620660
[14] Hidayati, S.C., You, C.W., Cheng, W.H., Hua, K.L.: Learning and recognition of clothing genres from full-body images. IEEE Transactions on Cybernetics, 48(5), 1647-1659 (2017). https://doi.org/10.1109/TCYB.2017.2712634 DOI: https://doi.org/10.1109/TCYB.2017.2712634
[15] Wang, Z., Wang, X.Y., Xu, J.P.: Personalized clothing recommendation via multi-feature fusion. Wool Textile Journal, 53(12), 101-108 (2025). https://doi.org/10.19333/j.mfk.20240707401 DOI: https://doi.org/10.1186/s40537-025-01224-4
[16] Hua, Y.Y., Zhang, D.C., Ge, S.M.: Research progress on interpretability of deep learning models. Journal of Information Security, 5(3), 1-13 (2020). https://doi.org/10.19363/J.cnki.cn10-1380/tn.2020.05.01
[17] Ji, S.L., Li, J.F., Du, T.Y., Li, B.: Survey on Techniques, Applications and Security of Machine Learning Interpretability. Journal of Computer Research and Development, 56(10), 2071-2096 (2019). https://doi.org/10.7544/issn1000-1239.2019.20190540
[18] Kim, E., Kim, S., Jung, C., Hahm, Y., Cho, S.: Fashion image retrieval with vision-language model guided fine-grained textual attributes and cross-domain contrastive optimization. IEEE Access, 13, 192403-192415 (2025). https://doi.org/10.1109/ACCESS.2025.3630632 DOI: https://doi.org/10.1109/ACCESS.2025.3630632
[19] Wang, M., Mahjoubar, A., Joshi, A.: FashionVQA: A domain-specific visual question answering system. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 3514-3519 (2023). https://doi.org/10.1109/CVPRW59228.2023.00356 DOI: https://doi.org/10.1109/CVPRW59228.2023.00356
[20] Liu, V., Chilton, L.B.: Design guidelines for prompt engineering text-to-image generative models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1-23 (2022). https://doi.org/10.1145/3491102.3501825 DOI: https://doi.org/10.1145/3491102.3501825
[21] Cao, Y., Zhang, J., Frittoli, L., Cheng, Y., Shen, W., Boracchi, G.: AdaCLIP: Adapting CLIP with hybrid learnable prompts for zero-shot anomaly detection. In European Conference on Computer Vision, pp. 55-72 (2024). https://doi.org/10.1007/978-3-031-72761-0_4 DOI: https://doi.org/10.1007/978-3-031-72761-0_4
[22] Zhao, K., Huang, S., Pan, P., Li, Y., Zhang, Y., Gu, Z., Xu, Y.: Distribution adaptive INT8 quantization for training CNNs. In Proceedings of the AAAI Conference on Artificial Intelligence. Virtual Event, pp. 3483-3491 (2021). https://doi.org/10.1609/aaai.v35i4.16462 DOI: https://doi.org/10.1609/aaai.v35i4.16462
[23] Takagi, M., Simo-Serra, E., Iizuka S., Ishikawa, H.: What Makes a Style: Experimental Analysis of Fashion Prediction. In 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 2247-2253 (2017). https://doi.org/10.1109/ICCVW.2017.263 DOI: https://doi.org/10.1109/ICCVW.2017.263
[24] Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations (ICLR), pp. 1-14 (2015)
[25] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510-4520 (2018). https://doi.org/10.1109/CVPR.2018.00474 DOI: https://doi.org/10.1109/CVPR.2018.00474
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
