Adaptive RoI with pretrained models for Automated Retail Checkout

Abstract

In this paper, we present a solution for automatic checkout in a retail store as a part of AI City Challenge 2023 Track 4. We propose a methodology which involves usage of pretrained Yolov5 models to detect person and media pipe models to detect hands of the person. This information is utilized to compute the Region of Interest (RoI) which is adaptive in nature. Afterwards, a custom trained object detection model is used detect products in the frame. We then use a tracker to track the products across video frames to avoid duplicated counting. The method is evaluated on the AI City challenge 2023 – Track 4 and gets the F1 score 0.6571 on the test A set, which places us on 6th place on the public leader board. The code is made public and available on GitHub. ¹