Involvement of Deep Learning for Vision Sensor-Based Autonomous Driving Control: A Review

Currently, autonomous vehicles (AVs) have gained considerable research interest in motion planning (MP) to control driving. Deep learning (DL) is a subset of machine learning motivated through neural networks. This article provides the latest survey on theories and applications of DL, reinforcement learning (RL), and deep RL, and it summarizes different DL methods. In addition, we present the main issues in autonomous driving (AD) and analyze DL-based architectures for decision-making frameworks in MP tasks, such as lane assist, lane following, overtaking, collision avoidance, emergency braking, and MP. Furthermore, we introduce well-known publicly available datasets collected on public roads and simulators suitable for different AD purposes and discuss simulator environments, activation functions, and DL-based libraries for output control in AVs. Moreover, we discuss challenges in terms of hardware and software, safety, computational time and cost, balanced data, multitask learning, and technology issues. Finally, we present future directions for MP.