A Study of Self Distillation for Mango Image Classification

We study a knowledge transfer approach called self distillation on a mango image dataset. Taking the deepest part of a convolutional neural network as the teacher, the self distillation approach transfers the relatively richer knowledge of the deepest part to shallow parts of this network, which are viewed as the students. We verify that this approach is effective in the target mango image dataset. Furthermore, we propose two more losses to improve performance considering data characteristics. In the discussion, we not only verify effectiveness of self distillation, but also point out weakness of the current approach, which unveils potential improvement for self distillation in the future.