An adaptive computation framework of distributed deep learning models for internet-of-things applications

Mu Hsuan Cheng, Qihui Sun, ChiaHeng Tu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose the computation framework that facilitates the inference of the distributed deep learning model to be performed collaboratively by the devices in a distributed computing hierarchy. For example, in Internet-of-Things (IoT) applications, the three-tier computing hierarchy consists of end devices, gateways, and server(s), and the model inference could be done adaptively by one or more computing tiers from the bottom to the top of the hierarchy. By allowing the trained models to run on the actually distributed systems, which has not done by the previous work, the proposed framework enables the co-design of the distributed deep learning models and systems. In particular, in addition to the model accuracy, which is the major concern for the model designers, we found that as various types of computing platforms are present in IoT applications fields, measuring the delivered performance of the developed models on the actual systems is also critical to making sure that the model inference does not cost too much time on the end devices. Furthermore, the measured performance of the model (and the system) would be a good input to the model/system design in the next design cycle, e.g., to determine a better mapping of the network layers onto the hierarchy tiers. On top of the framework, we have built the surveillance system for detecting objects as a case study. In our experiments, we evaluate the delivered performance of model designs on the two-tier computing hierarchy, show the advantages of the adaptive inference computation, analyze the system capacity under the given workloads, and discuss the impact of the model parameter setting on the system capacity. We believe that the enablement of the performance evaluation expedites the design process of the distributed deep learning models/systems.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages85-91
Number of pages7
ISBN (Electronic)9781538677599
DOIs
Publication statusPublished - 2019 Jan 9
Event24th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018 - Hakodate, Japan
Duration: 2018 Aug 292018 Aug 31

Publication series

NameProceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018

Conference

Conference24th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018
CountryJapan
CityHakodate
Period18-08-2918-08-31

Fingerprint

Internet of things
Deep learning
Gateways (computer networks)
Network layers
Distributed computer systems
Servers
Systems analysis
Costs
Experiments

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Cheng, M. H., Sun, Q., & Tu, C. (2019). An adaptive computation framework of distributed deep learning models for internet-of-things applications. In Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018 (pp. 85-91). [8607237] (Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/RTCSA.2018.00019
Cheng, Mu Hsuan ; Sun, Qihui ; Tu, ChiaHeng. / An adaptive computation framework of distributed deep learning models for internet-of-things applications. Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 85-91 (Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018).
@inproceedings{10e7a69916884ca7ad08b8c9ac1332b7,
title = "An adaptive computation framework of distributed deep learning models for internet-of-things applications",
abstract = "We propose the computation framework that facilitates the inference of the distributed deep learning model to be performed collaboratively by the devices in a distributed computing hierarchy. For example, in Internet-of-Things (IoT) applications, the three-tier computing hierarchy consists of end devices, gateways, and server(s), and the model inference could be done adaptively by one or more computing tiers from the bottom to the top of the hierarchy. By allowing the trained models to run on the actually distributed systems, which has not done by the previous work, the proposed framework enables the co-design of the distributed deep learning models and systems. In particular, in addition to the model accuracy, which is the major concern for the model designers, we found that as various types of computing platforms are present in IoT applications fields, measuring the delivered performance of the developed models on the actual systems is also critical to making sure that the model inference does not cost too much time on the end devices. Furthermore, the measured performance of the model (and the system) would be a good input to the model/system design in the next design cycle, e.g., to determine a better mapping of the network layers onto the hierarchy tiers. On top of the framework, we have built the surveillance system for detecting objects as a case study. In our experiments, we evaluate the delivered performance of model designs on the two-tier computing hierarchy, show the advantages of the adaptive inference computation, analyze the system capacity under the given workloads, and discuss the impact of the model parameter setting on the system capacity. We believe that the enablement of the performance evaluation expedites the design process of the distributed deep learning models/systems.",
author = "Cheng, {Mu Hsuan} and Qihui Sun and ChiaHeng Tu",
year = "2019",
month = "1",
day = "9",
doi = "10.1109/RTCSA.2018.00019",
language = "English",
series = "Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "85--91",
booktitle = "Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018",
address = "United States",

}

Cheng, MH, Sun, Q & Tu, C 2019, An adaptive computation framework of distributed deep learning models for internet-of-things applications. in Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018., 8607237, Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018, Institute of Electrical and Electronics Engineers Inc., pp. 85-91, 24th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018, Hakodate, Japan, 18-08-29. https://doi.org/10.1109/RTCSA.2018.00019

An adaptive computation framework of distributed deep learning models for internet-of-things applications. / Cheng, Mu Hsuan; Sun, Qihui; Tu, ChiaHeng.

Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 85-91 8607237 (Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - An adaptive computation framework of distributed deep learning models for internet-of-things applications

AU - Cheng, Mu Hsuan

AU - Sun, Qihui

AU - Tu, ChiaHeng

PY - 2019/1/9

Y1 - 2019/1/9

N2 - We propose the computation framework that facilitates the inference of the distributed deep learning model to be performed collaboratively by the devices in a distributed computing hierarchy. For example, in Internet-of-Things (IoT) applications, the three-tier computing hierarchy consists of end devices, gateways, and server(s), and the model inference could be done adaptively by one or more computing tiers from the bottom to the top of the hierarchy. By allowing the trained models to run on the actually distributed systems, which has not done by the previous work, the proposed framework enables the co-design of the distributed deep learning models and systems. In particular, in addition to the model accuracy, which is the major concern for the model designers, we found that as various types of computing platforms are present in IoT applications fields, measuring the delivered performance of the developed models on the actual systems is also critical to making sure that the model inference does not cost too much time on the end devices. Furthermore, the measured performance of the model (and the system) would be a good input to the model/system design in the next design cycle, e.g., to determine a better mapping of the network layers onto the hierarchy tiers. On top of the framework, we have built the surveillance system for detecting objects as a case study. In our experiments, we evaluate the delivered performance of model designs on the two-tier computing hierarchy, show the advantages of the adaptive inference computation, analyze the system capacity under the given workloads, and discuss the impact of the model parameter setting on the system capacity. We believe that the enablement of the performance evaluation expedites the design process of the distributed deep learning models/systems.

AB - We propose the computation framework that facilitates the inference of the distributed deep learning model to be performed collaboratively by the devices in a distributed computing hierarchy. For example, in Internet-of-Things (IoT) applications, the three-tier computing hierarchy consists of end devices, gateways, and server(s), and the model inference could be done adaptively by one or more computing tiers from the bottom to the top of the hierarchy. By allowing the trained models to run on the actually distributed systems, which has not done by the previous work, the proposed framework enables the co-design of the distributed deep learning models and systems. In particular, in addition to the model accuracy, which is the major concern for the model designers, we found that as various types of computing platforms are present in IoT applications fields, measuring the delivered performance of the developed models on the actual systems is also critical to making sure that the model inference does not cost too much time on the end devices. Furthermore, the measured performance of the model (and the system) would be a good input to the model/system design in the next design cycle, e.g., to determine a better mapping of the network layers onto the hierarchy tiers. On top of the framework, we have built the surveillance system for detecting objects as a case study. In our experiments, we evaluate the delivered performance of model designs on the two-tier computing hierarchy, show the advantages of the adaptive inference computation, analyze the system capacity under the given workloads, and discuss the impact of the model parameter setting on the system capacity. We believe that the enablement of the performance evaluation expedites the design process of the distributed deep learning models/systems.

UR - http://www.scopus.com/inward/record.url?scp=85061776848&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061776848&partnerID=8YFLogxK

U2 - 10.1109/RTCSA.2018.00019

DO - 10.1109/RTCSA.2018.00019

M3 - Conference contribution

T3 - Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018

SP - 85

EP - 91

BT - Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Cheng MH, Sun Q, Tu C. An adaptive computation framework of distributed deep learning models for internet-of-things applications. In Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 85-91. 8607237. (Proceedings - 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2018). https://doi.org/10.1109/RTCSA.2018.00019