Week 15 - DAgger-Based Dataset Generation and Enhanced Model Evaluation
January 9, 2026
Generating targeted driving examples for lane return behavior using DAgger methodology and comprehensive performance analysis
This week's work focused on generating specific driving examples demonstrating how the ego vehicle should return to the right lane from various starting positions and vehicle orientations. A DAgger (Dataset Aggregation) philosophy was employed to provide the necessary randomization for this purpose, creating a more diverse and representative dataset for challenging lane-return scenarios.
1. Dataset Generation with DAgger Methodology:
The DAgger approach was implemented to systematically collect corrective driving examples, particularly focusing on lane return maneuvers. This methodology allows the autonomous system to learn from its own mistakes by aggregating expert corrections to problematic situations encountered during autonomous operation. The generated dataset includes examples from multiple starting positions and orientations, ensuring comprehensive coverage of lane-return scenarios.
2. Offline Testing Results - Scenario 1: STRAIGHT - RIGHT - STRAIGHT - RIGHT
Figure 1: Smoothed temporal evolution of real vs inferred steering for MobileNet in Scenario 1. The error analysis shows smoothed error averaging around 0.000004.
Figure 2: Smoothed temporal evolution of real vs inferred steering for PilotNet in Scenario 1. The smoothed error averages around -0.02887.
3. Offline Testing Results - Scenario 2: SOFT LEFT - STRAIGHT LEFT
Figure 3: Smoothed temporal evolution of real vs inferred steering for MobileNet in Scenario 2. The smoothed error averages around 0.007719.
Figure 4: Smoothed temporal evolution of real vs inferred steering for PilotNet in Scenario 2.
4. Online Testing Results:
Both trained models (MobileNet and PilotNet) were tested in CARLA Simulator to validate their online performance. The following behaviors were observed:
MOBILENET (PYTORCH) - ONLINE PERFORMANCE:
• Stable driving with appropriate lane correction
• Correct execution of turns
• Limitation: In specific routes, the vehicle "gets lost" and leaves the road without being able to correct its trajectory
PILOTNET (TENSORFLOW) - ONLINE PERFORMANCE:
• Faster maneuver execution but less stable driving
• Correct turns but occasional premature turn initiation
• Issues: Early turns cause lane departure from the inner side, inadequate corrections, and occasional omission of turns
5. Autonomous Driving Demonstrations:
Below are video demonstrations showing both models in operation within CARLA Simulator:
MOBILENET AUTONOMOUS DRIVING DEMONSTRATION
Video 1: MobileNet model demonstrating stable driving with appropriate lane correction and correct turns. Note that in some specific routes, the vehicle "gets lost" and exits the road without correction capability.
PILOTNET AUTONOMOUS DRIVING DEMONSTRATION
Video 2: PilotNet model demonstrating faster but less stable maneuvers. The model shows correct turns but occasionally initiates turns prematurely, causing lane departure from the inner side. It also exhibits inadequate corrections and omits some turns.
6. Analysis and Next Steps:
The DAgger-based dataset generation has provided valuable examples for improving lane-return behavior, though both models still exhibit specific limitations. MobileNet shows superior stability but lacks robustness in unfamiliar routes, while PilotNet offers faster response but suffers from timing and consistency issues.
Conclusion:
Week 15 demonstrated the effectiveness of the DAgger methodology for generating targeted training examples for challenging driving scenarios. While both models show improved performance with the enhanced dataset, distinct behavioral patterns emerge: MobileNet prioritizes stability at the cost of adaptability, while PilotNet favors responsiveness with reduced consistency. The next phase will focus on addressing these specific limitations while expanding the dataset to cover a wider range of challenging driving conditions, ultimately working toward a more robust and adaptable autonomous driving system.