Person detection and re-identification in open-world settings of retail stores and public spaces

AUTOR(I) / AUTHOR(S): Branko BRKLJAČ , Milan BRKLJAČ  

Download Full Pdf   

DOI:  10.46793/ALFATECHproc25.264B

SAŽETAK / ABSTRACT:

Practical applications of computer vision in smart cities usually assume system integration and operation in challenging open-world environments. In the case of person re-identification task the main goal is to retrieve information whether the specific person has appeared in another place at a different time instance of the same video, or over multiple camera feeds. This typically assumes collecting raw data from video surveillance cameras in different places and under varying illumination conditions. In the considered open-world setting it also requires detection and localization of the person inside the analyzed video frame before the main re-identification step. With multi-person and multi-camera setups the system complexity becomes higher, requiring sophisticated tracking solutions and re-identification models. In this work we will discuss existing challenges in system design architectures, consider possible solutions based on different computer vision techniques, and describe applications of such systems in retail stores and public spaces for improved marketing analytics. In order to analyse sensitivity of person re-identification task under different open-world environments, a performance of one close to real-time solution will be demonstrated over several video captures and live camera feeds. Finally, based on conducted experiments we will indicate further research directions and possible system improvements.

KLJUČNE REČI / KEYWORDS:

person re-identification; computer vision; retail stores; public spaces; marketing analytics

PROJEKAT / ACKNOWLEDGEMENT:

The authors would like to especially thank the conference organizers for their patience and support throughout the manuscript preparation process. This research was partially supported by the Science Fund of the Republic of Serbia, project” Multimodal multilingual human-machine speech communication – AI SPEAK”, grant no. 7449; and by the Ministry of Science, Technological Development and Innovation of the Republic of Serbia (contract no. 451-03-137/2025-03/200156); and by the Faculty of Technical Sciences, University of Novi Sad, through the project “Scientific and artistic research work of researchers in teaching and associate positions at the Faculty of Technical Sciences, University of Novi Sad 2025” (no. 0150/295).

LITERATURA / REFERENCES:

• Dollar, P.,Wojek, C.,Schiele, B. & Perona, P. (2011). “Pedestrian detection: An evaluation of the state of the art,”IEEE Transactions on Pattern AnalysisandMachineIntelligence,vol.34,no.4,pp.743– 761.
• Gong, S., Cristani, M., Shuicheng, Y. & Change Loy, C. (2014). “Person re-identification,”, ISBN978-1-4471-6295-7.
• Yurtsever, E.,Lambert, J., Carballo, A. &Takeda, K. (2020). “A survey ofautonomous driving: Common practices and emerging technologies,”IEEE access, vol. 8, pp. 58 443–58 469.
• Kwon, J., Lee, Y. &Lee, J. (2021). “Comparative study of markerless vision- based gait analyses for person Reidentification,”  Sensors,  21, no. 24, p. 8208.
• Behera, N. K. S., Behera, T. K., Nappi, M. Bakshi, S.&Sa, P. K. (2021).“Futuristic person re-identification over internet of biometrics things(iobt): Technical potential versus practical reality,” Pattern Recognition Letters,vol.151,pp.163–171.
• Chen, S., Fern, A. &  Todorovic, S. (2015). “Personcountlocalizationinvideosfrom noisy foreground and detections,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1364–1372.
• Gong, S., Loy, C. C.&Xiang, T. (2011). “Security and surveillance,” in Visual analysisofhumans: Lookingatpeople.Springer,pp.455–472.
• Yang, H. F.,Cai, J., Liu, C., Ke, R. &Wang, Y. (2023).“Cooperative multi– camera vehicle tracking and traffic surveillance with edge artificial intelligence and representation learning,”Transportation Research partC: EmergingTechnologies, vol.148 ,p.103982.
• Almazan, J., Gajić, B., Murray, N. & Larlus, D. “Re-ID done right: Towards good practices for person re-identification,” arXiv preprintarXiv:1801.05339.
• Shili, M., Sohaib, O. & Hammedi, S. (2024). “Yolo v5 and  deep simpleonline and real-time tracking algorithms for real-time customer behaviortracking and retail optimization,”Algorithms, vol. 17, no. 11, p. 525.
• Del Carpio, A. F. (2024). “Analyzing computer vision models for detecting customers: A practical experience in a Mexican retail,”International Journal of Advancesin Intelligent Informatics, vol.10, no.1, pp. 131– 174.
• Quintana, M., Menendez, J. M., Alvarez, F.,& Lopez, J. P. (2016). “Improv– ing retail efficiency through sensing technologies: A survey,” Pattern Recognition Letters, vol. 81, pp. 3–10.
• Liu, X., Jiang, Y., Jain, P. & Kim, K.-H. (2018). “TAR: Enabling fine-grainedtargeted advertising in retail stores,” in Proceedings of the 16th Annual International Conference on Mobile Systems, Applications and Services, pp.323–336.
• Ghose, A., Li, B., Li, R. & Xu, K. (2022). “Real-time purchase prediction using retail video analytics,” in ICIS Proceedings, vol. 6, pp. 1–17. [Online]. Available: https://aisel.aisnet.org/icis2022/ data analytics/dataanalytics/6
• Martini, M., Paolanti, M. & Frontoni, E. (2020). “Open-world person re- identification with RGBD camera in top-view configuration for retailapplications,” IEEE Access, vol. 8, pp. 67 756–67 765.
• Van Rooijen, A., Bouma, H., Pruim, R., Baan, J., et al. (2020). “Anonymizedperson re-identification in surveillance cameras,” in Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies IV, vol. 115 42. SPIE, pp.63–67.
• Hashemifard, F. Florez-Revuelta, G. Lacey et al., “A fallen person detector with a privacy-preserving edge-AI camera,” pp. 262–269,2023.
• Pietrini, R. (2019). “Deep understanding of shopper behaviours and interactions in intelligent retail environment,”, doctoral thesis, Universita`Politecnicadelle Marche, Ancona, Italy. [Online]. Available: http://hd l.handle.net/11566/274602
• Lee L., et al. (2015). “The emotional shopper: Assessing the effectiveness of retail therapy,”Foundations and Trends® in Marketing, vol. 8, no. 2, pp. 69–145.
• Yamashiro, T., Honma, Y.,Togo, R., Ogawa, T. & Haseyama, M. (2023). “Customerinterestestimationmethodinrealstoreusingreidentification and 3D posture estimation models,” in International Workshop on Advanced Imaging Technology (IWAIT) 2023, vol. 12592. SPIE, pp.42–47.
• Becattini, F., Becchi, G., Ferracani, A., Bimbo, A.D., Presti, L.L.,               Mazzola, G., et al. (2022).“Imallaneffectiveframeworkforpersonalizedvisits: Improving the customer experience in stores,” in Proceedings of the 1st WorkshoponMultimediaComputingtowardsFashionRecommen dation, pp.11–19.
• Luxonis (2025). OAK-D lite. [Online]. Available: https://docs.luxonicom/hardware/products/OAK-D%20  Lite (Accessed2025-01-11).
• Luxonis (2022). OAK lite datasheet. [Online]. Available: https://github.com/luxonis/depthai-hardware/tree/master/
•         DM9095   OAK-D-LITE DepthAI   USB3C/Datasheet (Accessed2025-01-11).
• Zafeiriou, S., Zhang, C., & Zhang, Z. (2015). “A survey on face detection in the wild: Past, present and future,”Computer Vision and Image Understanding, vol.138, pp.1–24.
• Feng, Y. ,Yu, S.,Peng, H., Li, Y.-R. &Zhang, J. (2022). “Detect facesefficiently: A survey and evaluations,”IEEE Transactions on Biometrics, Behavior, andIdentityScience,vol.4,no.1,pp.1– 18.
• Zou, Z., Chen, K., Shi, Z., Guo, Y. &Ye, J. (2023).“Object detection in 20 years: A survey,”Proceedings of the IEEE, vol. 111, no. 3, pp. 257–276.
• Wang, X., Peng, J., Zhang, S., Chen, B., Wang, Y. & Guo, Y. (2022). “A survey offacerecognition,”arXivpreprintarXiv: 2212.13038.
• Qian, Y. , Barthelemy, J., Karuppiah, E., & Perez, P.  (2024). “Identifying re-identification challenges: Past, current and future trends,” SN Computer Science, vol.5, no.7, p.937.
• Zahra, A., Perwaiz, N., Shahzad, M. & Fraz, M. M. (2023). “Person re-identification: A retrospective on domain specific open challenges and futuretrends,”Pattern Recognition,vol.142,p.109669.
• Yaghoubi, E., Kumar, A., & Proenc¸a, H. (2021), “SSSPR:Ashortsurveyofsurveys in person re-identification,”Pattern Recognition Letters, vol. 143, pp. 50–57.
• …