Performance Acceleration of Secure Machine Learning Computations for Edge Applications

Edge appliances built with machine learning applications have been gradually adopted in a wide variety of application fields, such as intelligent transportation, the banking industry, and medical diagnosis. Privacy-preserving computation approaches can be used on smart appliances in order to secure the privacy of sensitive data, including application data and the parameters of machine learning models. Nevertheless, the data privacy is achieved at the cost of execution time. That is, the execution speed of a secure machine learning application is several orders of magnitude slower than that of the application in plaintext. Especially, the performance gap is enlarged for edge appliances. In this work, in order to improve the execution efficiency of secure applications, an open-source software framework CrypTen is targeted, which is widely used for building secure machine learning applications using the Secure Multi-Party Computation (SMPC) based privacy-preserving computation approach. We analyze the performance characteristics of the secure machine learning applications built with CrypTen, and the analysis reveals that the communication overhead hinders the execution of the secure applications. To tackle the issue, a communication library, OpenMPI, is added to the CrypTen framework as a new communication backend to boost the application performance by up to 50%. We further develop a hybrid communication scheme by combining the OpenMPI backend with the original communication backend with the CrypTen framework. The experimental results show that the enhanced CrypTen framework is able to provide better performance for the small-size data (LeNet5 on MNIST dataset by up to 50% of speedup) and maintain similar performance for large-size data (AlexNet on CIFAR-10), compared to the original CrypTen framework.