Autonomous Driving in Traffic with End-to-End Vision-based Deep Learning

Neurocomputing, 2024

Sergio Paniego^1,2, Enrique Shinohara^1,2, JoseMaria Cañas^1,2

DOI: 10.1016/j.neucom.2024.127874

Abstract

This paper presents a shallow end-to-end vision-based deep learning approach for autonomous vehicle driving in traffic scenarios. The primary objectives include lane keeping and maintaining a safe distance from preceding vehicles. This study leverages an imitation learning approach, creating a supervised dataset for robot control from expert agent demonstrations using the state-of-the-art Carla simulator in different traffic conditions. This dataset encompasses three different versions complementary to each other and we have made it publicly available along with the rest of the materials. The PilotNet neural model is utilized in two variants: the first one with complementary outputs for brake and throttle control commands along with dropout; the second one incorporates these improvements and adds the vehicle speed. Both models have been trained with the aforementioned dataset. The experimental results demonstrate that the models, despite their simplicity and shallow architecture, including only small-scale changes, successfully drive in traffic conditions without sacrificing performance in free-road environments, broadening their area of application widely. Additionally, the second model adeptly maintains a safe distance from leading cars and exhibits satisfactory generalization capabilities to diverse vehicle types. A new evaluation metric to measure the distance to the front vehicle has been created and added to Behavior Metrics; an open-source autonomous driving assessment tool built on CARLA that performs experimental validations of autonomous driving solutions.

Videos

Materials

Paper

Dataset

Code

Comparison software tool with simulator

Models' weights

Citation

@article{Paniego2024,
  title = {Autonomous Driving in Traffic with End-to-End Vision-based Deep Learning},
  journal = {Neurocomputing},
  pages = {127874},
  year = {2024},
  issn = {0925-2312},
  doi = {10.1016/j.neucom.2024.127874},
  url = {https://www.sciencedirect.com/science/article/pii/S0925231224006453},
  author = {Paniego, Sergio and Shinohara, Enrique and Cañas, JoséMaría},
  keywords = {End-to-end autonomous driving, Imitation learning, Deep learning, Lane-following},
}