Adapting Object Detection to Fisheye Cameras: A Knowledge Distillation with Semi-Pseudo-Label Approach

In this paper, we introduce a lightweight object detection system, custom-designed for fisheye cameras and optimized for quick deployment on embedded systems. Given the constraints of training solely on standard images, our methodology centers on the effective knowledge transfer to accentuate object detection in fisheye scenarios. The integration of the Parallel Residual Bi-Fusion (PRB) Feature Pyramid Network (FPN) into the state-of-the-art YOLOv7 backbone specifically addresses the challenges of detecting tiny objects often present in fisheye images. Our unique two-phase training strategy operates as follows: Firstly, a comprehensive Teacher Model is trained on standard images, setting the stage for knowledge acquisition. Subsequently, in the second phase, this knowledge is distilled to a more compact Student Model. The twist is in using fisheye images as pseudo-information, ensuring the model’s adaptability to fisheye-centric environments. Combining knowledge distillation with semi-pseudo-label semi-supervised learning, this strategy guarantees optimal performance and embraces a lightweight design perfect for real-time applications on constrained devices. In essence, our contributions span the crafting of a specialized object detection framework for fisheye cameras, the proposition of a novel two-tiered training strategy, and the synergetic use of PRB with YOLOv7. Empirical results reinforce the efficacy of our approach, illustrating that while our model retains a compact footprint, it doesn’t compromise on performance, excelling in tasks with a comparable nature and offering swift inference.