ONNC: A Compilation Framework Connecting ONNX to Proprietary Deep Learning Accelerators

Wei Fen Lin, Der Yu Tsai, Luba Tang, Cheng Tao Hsieh, Cheng Yi Chou, Ping Hao Chang, Luis Hsu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

37 Citations (Scopus)

Abstract

This paper presents ONNC (Open Neural Network Compiler), a retargetable compilation framework designed to connect ONNX (Open Neural Network Exchange) models to proprietary deep learning accelerators (DLAs). The intermediate representations (IRs) of ONNC have one-to-one mapping to ONNX IRs, thus making porting ONNC to proprietary DLAs much simpler than other compilation frameworks such as TVM and Glow especially for hardware with coarse-grained operators that are not part of the generic IRs in the LLVM backend. ONNC also has a flexible pass manager designed to support compiler optimizations at all levels. A docker image of ONNC bundled with a Vanilla backend is released with this paper to enable fast porting to new hardware targets. To illustrate how an ONNC-based toolkit guides our research and development in DLA design, we present a case study on compiler optimizations for activation memory consumption. The study shows that the Best-Fit algorithm with a proposed heuristic and a reordering scheme may act as a near-optimal strategy, getting the memory consumption close to the ideal lower bound in 11 of 12 models from the ONNX model zoo. To our best knowledge, ONNC is the first open source compilation framework that is specially designed to support the ONNX-based models for both commercial and research projects for deep learning applications.

Original languageEnglish
Title of host publicationProceedings 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages214-218
Number of pages5
ISBN (Electronic)9781538678848
DOIs
Publication statusPublished - 2019 Mar
Event1st IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2019 - Hsinchu, Taiwan
Duration: 2019 Mar 182019 Mar 20

Publication series

NameProceedings 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2019

Conference

Conference1st IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2019
Country/TerritoryTaiwan
CityHsinchu
Period19-03-1819-03-20

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'ONNC: A Compilation Framework Connecting ONNX to Proprietary Deep Learning Accelerators'. Together they form a unique fingerprint.

Cite this