Large‑Scale Dataset Training & Methodology Comparison
June 19, 2026
⏳ Training in Progress
Current status: The model training with the new 62k dataset started this week. As of today, approximately 28 hours remain until completion (on NVIDIA GeForce GTX 1080). Results are not expected before our usual meeting time tomorrow.
This dataset is significantly larger and more varied than previous weeks, which explains the extended training time.
1. Horizontal Flip Verification
Two critical checks were performed to ensure the horizontal flip augmentation works correctly:
✅ Steering sign change: After flipping, the steering value changes sign (e.g., +0.5 → –0.5). Verified: OK
✅ Coherence of trained model: The model produces steering values that keep the vehicle in the right lane when presented with flipped images. Verified: OK
This confirms that the augmented data is physically meaningful and that the model learns the correct mapping.
2. Manual Driving – Left‑Hand Traffic (LHT) & Horizontal Flip
Following last week's recommendations, manual driving was performed in Town 01 and Town 04 using only simple maneuvers (straight, left/right turns) to facilitate horizontal flip augmentation.
- Data was collected for both left‑hand and right‑hand traffic scenarios.
- After collection, horizontal flip was applied to all images, and the steering sign was inverted accordingly.
- Coherence was verified as described above.
3. Image Cropping (35% top removed)
Following the standard PilotNet preprocessing pipeline, the top 35% of each image was cropped to remove the sky and other non‑road elements.
# Crop top 35% of image (height=120 → 42 pixels removed)
cropped = image[42:, :, :] # shape becomes (78, 160, 3)
This reduces input dimensionality and focuses the network on the road and lane markings.
4. Actuator Noise Injection
To improve robustness, Gaussian noise was added to the steering actuators with a normal distribution of variance 0.1.
# Noise injection during training
noise = np.random.normal(0, 0.1, size=steering.shape)
noisy_steering = steering + noise
This helps the model cope with small sensor errors and actuation delays.
5. Dataset Composition – 62,000 Samples
Based on the best results from previous weeks, the following distribution was chosen:
| Maneuver | Percentage | # Samples (approx) |
|---|---|---|
| Straight (forward) | 45% | 27,900 |
| Soft Drunk‑DAgger | 25% | 15,500 |
| Turns (left + right) | 30% | 18,600 |
This composition was selected because it yielded the best validation performance in Weeks 33 and 34.
📦 Total dataset size: 62,000 samples (fixed, no online balancing).
6. Training Status – NVIDIA GTX 1080
Started: June 17, 2026
Estimated remaining time: ~28 hours
Hardware: NVIDIA GeForce GTX 1080 (8 GB VRAM)
Batch size: 64 | Epochs: 30
Due to the larger dataset, training is slower than in previous weeks. Results will be reported as soon as they are available.
7. New Hardware – NVIDIA RTX 5070
⚡ A new PC with NVIDIA GeForce RTX 5070 has been set up and is now running CARLA 0.9.14 in headless mode.
Starting next week, we expect to migrate training to this machine, reducing training times significantly (estimated 2–3× faster).
8. Methodology Comparison – Carlos Andrés Velasquez
This week we discussed dataset generation and training strategies with Carlos Andrés. The key differences are summarised below:
| Aspect | Armando’s Methodology | Carlos Andrés’ Methodology |
|---|---|---|
| Main focus | Offline data augmentation and static balancing. A fixed dataset with predetermined composition before training. | Dynamic balancing and hot‑selection. Dataset is balanced during training, strategically selecting the most relevant samples. |
| Augmentation techniques | Geometric: horizontal flip (with steering sign adjustment). Cropping: 35% top. Noise: actuator noise (normal, var 0.1). | Semantic segmentation: binary mask to isolate the road (white) from background (black). Cropping: centred on the road. |
| Data handling | Fixed composition: 45% straight, 25% soft drunk dagger, 30% turns. | Peak elimination: over‑represented data is removed to avoid flattening. No duplication; naturalness is prioritised. |
| Use of DAgger | Incorporated as a fixed 25% “soft drunk dagger” component. | Recovery strategy: during DAgger execution, random predefined actions are triggered. Images produced just after these actions are saved unlabelled, then later labelled as “recovery” and mixed with nominal data. |
| Data volume | Fixed dataset of 62,000 samples with immutable composition. | Variable and dynamic composition: nominal datasets (“bubble”) mixed with DAgger in varying proportions (e.g., 80% bubble + 20% DAgger). |
| Balancing philosophy | Balance by quantity: seeks equal number of samples per type. | Balance by relevance: removes redundant samples, prioritises rarity or utility, preserving the natural distribution of human behaviour. |
| Input data | Direct RGB image. | Semantic image + steering/throttle commands. The image is obtained from CARLA and processed to keep only the road. |
| Human behaviour representation | Replicates the behaviour of a human expert. | Preserves the prevalence of natural human behaviour, avoiding artefacts that distort the original data distribution. |
Table 1 – Comparison of dataset generation and training methodologies.
9. Summary & Next Steps
- ✅ Horizontal flip verification completed (steering sign and coherence OK).
- ✅ LHT manual driving data collected (Town 01 & Town 02) and flipped.
- ✅ Image cropping (35% top) and actuator noise (var=0.1) applied.
- ✅ 62k dataset built with composition: 45% straight, 25% soft drunk dagger, 30% turns.
- ⏳ Training in progress (~28 h remaining on GTX 1080).
- 🖥️ New RTX 5070 PC set up for future faster training.
- 📊 Methodology comparison with Carlos Andrés documented.
🔜 Next week (Week 36):
- Evaluate the results of the 62k model once training finishes.
- If successful, test on Town 02 and Town 06 (generalisation).
- Begin migrating training pipeline to the RTX 5070 PC.
- Consider incorporating semantic segmentation (inspired by Carlos Andrés) if beneficial.
References
- [1] Bojarski, M., et al. (2016): End to end learning for self‑driving cars (PilotNet). arXiv:1604.07316
- [2] Mateus, A. (2026): Week 34 report – Fine‑tuning vs new model & dataset compositions.
- [3] Velasquez, C.A. (2026): Personal communication on dynamic balancing and semantic segmentation.
— Armando Mateus, Robotics Lab URJC
📌 WEEK 35 SUMMARY – JUNE 19, 2026
⏳ Training in progress: 62k dataset, ~28 h remaining on GTX 1080.
✅ Horizontal flip: Steering sign change and coherence verified.
✅ LHT driving: Manual data collected and flipped.
✅ Preprocessing: Cropping (35% top) + actuator noise (var=0.1).
📊 Dataset: 45% straight, 25% soft drunk dagger, 30% turns.
🖥️ New hardware: RTX 5070 PC ready for upcoming training.
🧑🤝🧑 Methodology comparison with Carlos Andrés Velasquez completed.