Anticipating Autonomous Vehicle Driving based on Multi-Modal Multiple Motion Tasks Network

Abida Khanum, Chao Yang Lee, Chih Chung Hus, Chu Sing Yang

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


Recently, research concerning autonomous self-driving vehicles has become very popular. In autonomous vehicles (AVs), different decision-making and learning architectures have been proposed to predict multiple tasks (MTs) or MTs from various datasets or to improve performance among different MTs. In this paper, a novel and unified multitask learning framework, called Multi-Modal DenseNet (M2-DenseNet), is proposed to predict MTs in a single network in which three long short-term memory units act as the output (MTs). Accordingly, the proposed M2-DenseNet can predict three different motion decision-making tasks, i.e., the steering angle, speed, and throttle, to control AV driving. Moreover, M2-DenseNet can greatly reduce the time complexity (e.g., to less than 5 ms) because the different prediction tasks can be predicted simultaneously. We conduct comprehensive experiments with the lane-keeping task based on two control mechanisms using the proposed M2-DenseNet and other existing methods to evaluate the performance. The experiments demonstrate that M2-DenseNet significantly outperforms other state-of-the-art methods with the accuracies of the three control tasks being approximately 98%, 99%, and 98%, respectively. The mean squared error between the predicted value and the ground truth is reported in the experiments, with values for the steering angle, speed, and throttle of 0.0250, 0.0210, and 0.0242, respectively.

Original languageEnglish
Article number69
JournalJournal of Intelligent and Robotic Systems: Theory and Applications
Issue number3
Publication statusPublished - 2022 Jul

All Science Journal Classification (ASJC) codes

  • Software
  • Mechanical Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering
  • Control and Systems Engineering
  • Industrial and Manufacturing Engineering


Dive into the research topics of 'Anticipating Autonomous Vehicle Driving based on Multi-Modal Multiple Motion Tasks Network'. Together they form a unique fingerprint.

Cite this