TY - GEN
T1 - A Study of Self Distillation for Mango Image Classification
AU - Chen, Wei Chi
AU - Chu, Wei Ta
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/12
Y1 - 2020/12
N2 - We study a knowledge transfer approach called self distillation on a mango image dataset. Taking the deepest part of a convolutional neural network as the teacher, the self distillation approach transfers the relatively richer knowledge of the deepest part to shallow parts of this network, which are viewed as the students. We verify that this approach is effective in the target mango image dataset. Furthermore, we propose two more losses to improve performance considering data characteristics. In the discussion, we not only verify effectiveness of self distillation, but also point out weakness of the current approach, which unveils potential improvement for self distillation in the future.
AB - We study a knowledge transfer approach called self distillation on a mango image dataset. Taking the deepest part of a convolutional neural network as the teacher, the self distillation approach transfers the relatively richer knowledge of the deepest part to shallow parts of this network, which are viewed as the students. We verify that this approach is effective in the target mango image dataset. Furthermore, we propose two more losses to improve performance considering data characteristics. In the discussion, we not only verify effectiveness of self distillation, but also point out weakness of the current approach, which unveils potential improvement for self distillation in the future.
UR - http://www.scopus.com/inward/record.url?scp=85102183718&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85102183718&partnerID=8YFLogxK
U2 - 10.1109/ICS51289.2020.00077
DO - 10.1109/ICS51289.2020.00077
M3 - Conference contribution
AN - SCOPUS:85102183718
T3 - Proceedings - 2020 International Computer Symposium, ICS 2020
SP - 356
EP - 359
BT - Proceedings - 2020 International Computer Symposium, ICS 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 International Computer Symposium, ICS 2020
Y2 - 17 December 2020 through 19 December 2020
ER -