Intelligent video surveillance for shoplifting detection using deep learning

Abstract

Shoplifting remains a major challenge for the retail industry, causing significant financial losses. While video surveillance is widely used, manual monitoring is inefficient, and traditional 2D automated systems fail to recognize the continuous movements involved in theft. This project proposes a complete, automated deep learning pipeline designed for spatiotemporal behavior analysis and shoplifting detection. The system first processes continuous video streams to isolate individuals using advanced human detection (RT-DETR) and object tracking (BoT-SORT). To analyze the resulting sequences, we developed a specialized Dual-Branch feature extraction architecture. This combines a 3D Vision Transformer (VideoMAE) to understand the global behavioral context, and a 3D Convolutional Neural Network (R3D-18) to capture rapid, localized movements. These features are fused to provide a complete representation of the shopper’s actions. To ensure robustness and generalizability, the models were trained on a unified dataset combining two distinct shoplifting datasets. Experimental results demonstrated the high effectiveness of the Dual-Branch architecture, achieving an Accuracy of 92.86%, an F1-score of 93.47%, a Precision of 95.88%, and an AUC of 99.06%. These findings confirm that combining 3D transformers with 3D convolutions offers a highly reliable solution for intelligent retail surveillance