Anticipating Autonomous Vehicle Driving based on Multi-Modal Multiple Motion Tasks Network

Recently, research concerning autonomous self-driving vehicles has become very popular. In autonomous vehicles (AVs), different decision-making and learning architectures have been proposed to predict multiple tasks (MTs) or MTs from various datasets or to improve performance among different MTs. In this paper, a novel and unified multitask learning framework, called Multi-Modal DenseNet (M²-DenseNet), is proposed to predict MTs in a single network in which three long short-term memory units act as the output (MTs). Accordingly, the proposed M²-DenseNet can predict three different motion decision-making tasks, i.e., the steering angle, speed, and throttle, to control AV driving. Moreover, M²-DenseNet can greatly reduce the time complexity (e.g., to less than 5 ms) because the different prediction tasks can be predicted simultaneously. We conduct comprehensive experiments with the lane-keeping task based on two control mechanisms using the proposed M²-DenseNet and other existing methods to evaluate the performance. The experiments demonstrate that M²-DenseNet significantly outperforms other state-of-the-art methods with the accuracies of the three control tasks being approximately 98%, 99%, and 98%, respectively. The mean squared error between the predicted value and the ground truth is reported in the experiments, with values for the steering angle, speed, and throttle of 0.0250, 0.0210, and 0.0242, respectively.