Automatic cattle identification system based on color point cloud using hybrid PointNet++ Siamese network

Abstract

Cattle health monitoring and management systems are essential for farmers and veterinarians, as traditional manual health checks can be time-consuming and labor-intensive. A critical aspect of such systems is accurate cattle identification, which enables effective health monitoring. Existing 2D vision-based identification methods have demonstrated promising results; however, their performance is often compromised by environmental factors, variations in cattle texture, and noise. Moreover, these approaches require model retraining to recognize newly introduced cattle, limiting their adaptability in dynamic farm environments. To overcome these challenges, this study presents a novel cattle identification system based on color point clouds captured using RGB-D cameras. The proposed approach employs a hybrid detection method that first applies a 2D depth image detection model before converting the detected region into a color point cloud, allowing for robust feature extraction. A customized lightweight tracking approach is implemented, leveraging Intersection over Union (IoU)-based bounding box matching and mask size analysis to consistently track individual cattle across frames. The identification framework is built upon a hybrid PointNet ++ Siamese Network trained with a triplet loss function, ensuring the extraction of discriminative features for accurate cattle identification. By comparing extracted features against a pre-stored database, the system successfully predicts cattle IDs without requiring model retraining. The proposed method was evaluated on a dataset consisting predominantly of Holstein cow along with a few Jersey cows, achieving an average identification accuracy of 99.55% over a 13-day testing period. Notably, the system can successfully detect and identify unknown cattle without requiring model retraining. This cattle identification research aims to integrate the comprehensive cattle health monitoring system, encompassing lameness detection, body condition score evaluation, and weight estimation, all based on point cloud data and deep learning techniques.