Robust Driving at 15 km/h: Clean Data, the Right Camera Height and a Stable Recipe
November 17, 2026
π Part 1 β Executive Summary
π What Changed This Week
The models are now very robust at 15 km/h. 35 of 36 new models drive the full five minutes of the Town02 test route (about 1,239 m) without a collision, in each of three repetitions. The same model now drives the same way every time: distances repeat within 0.1 m.
The seed problem is under control. Distillation removed the dependence on the training seed while the models were fragile. With the new, larger and cleaner training sets, almost every normally trained model already succeeds (29 of 30), so distillation adds little: it removes the single failure, but it does not improve lane keeping.
Two hidden problems were fixed. A coherence filter, which checks that every recorded steering label matches how the car really turned, found frozen-steering recordings and duplicated drives in the base data; they were removed. And the driving camera was mounted 1 m higher (2.4 m) than the camera used to generate the training images (1.4 m); it now drives at 1.4 m.
First robustness tests (speed, starting position, starting orientation, wrong-way start and low morning sun) were run and recorded, as a quick check before the formal evaluation in Behavior Metrics.
Short answer: with coherent data and the right camera, a standard training recipe produces models that complete the route reliably; the open questions are now how far this robustness goes and which of the changes made the difference.
π Part 2 β Fixing the Data: Coherence Filter and Camera Height
Before this week, almost no model could pass the three left-hand L-corners of the route (A at about 65 m, C at about 250 m and the gas station at about 445 m), and which model succeeded seemed to depend on luck. Instead of changing the training method again, the base recordings and the driving setup were checked one by one.
The coherence test
When a dataset is generated, every recording is replayed by placing the car pose by pose at the recorded position and heading. The recorded heading therefore tells what the car really did, and a faithful steering label must follow it: when the car turns, the steering moves, with the same sign. For every recording we measured how many different steering values it contains, how much the car turns in total, and the correlation between steering and turning rate.
Frozen: the steering hardly changes (three values or fewer, or the same value in more than 80 % of the frames) while the car turns more than 15Β°.
No turn: the car turns 15Β° or less during the whole recording.
Mirroring check: mirroring a British-side recording flips the steering but not the heading, so a correctly mirrored recording must show a negative correlation.
What the filter found
Frozen guardrail recordings inside the left-turn category. 62 guardrail captures from October 25 had been stored as left turns, 23 % of that category. In 28 of them the steering stayed frozen near β0.06 while the car turned; in 26 the car did not turn at all. They taught the model not to turn in front of a curve next to a guardrail, which is exactly the gas-station scene. Every generated dataset contained about 2 % of these frames.
Repeated drives. The July categories existed three times (original, evening and morning copies): same drives and labels with different lighting. Some straight-road drives were even stored twice in the same folder; one of 16,848 frames reached 11 % of the straight-road category.
What was fine: all 2,441 British recordings are mirrored with the correct sign, the steering scale is the same in every model, and no recording hits a steering-sensitivity limit.
Base recordings
Before
After
Recordings
7,182
3,165
Samples
1,476,298
637,405
Frozen-steering recordings
28
0
Left-turn samples
30,898
9,853
The 57 bad guardrail recordings (5,201 samples) were deleted and the 8 valid ones kept. The evening and morning copies were removed after checking, drive by drive, that each one also existed in its original folder. Lighting variation will be handled separately.
The camera was 1 m too high
The training images are generated with the camera at 1.4 m, but the driving camera was mounted at 2.4 m. This was confirmed by re-rendering the same poses at both heights: 10 of 10 matched 1.4 m. With the camera at 1.4 m, the lateral error of the same models dropped from 0.744 m to 0.357 m. Driving now uses 1.4 m by default.
π± Part 3 β A New Series: Comp0 on Clean Data, with Distillation
The reference mixture (Comp0: 45 % straight road, 25 % gentle weaving, 20 % right turns, 10 % left turns) was rebuilt from the clean base with six sampling seeds of 75,285 images each: 52,700 for training and 22,585 (30 %) for validation. For each dataset, five normal models (the teachers, training seeds 42, 1, 2, 3 and 4) and one distilled student were trained. The recipe was simplified: no weight averaging (EMA), no horizontal flip, no brightness augmentation, and the checkpoint with the best validation loss is kept.
Teachers
Students
Models
30
6
Complete 5 minutes in all 3 repetitions
29 (97 %)
6 (100 %)
Datasets with a failing model
1 of 6
0 of 6
Mean lateral error |cte|
0.294 m
0.310 m
Minimum steering at corner A
β0.32
β0.26
Within one dataset the training seed still decides which model comes out: for dataset seed 4, the best teacher is first in the ranking and another teacher is the worst that completes the route. The only failing teacher (dataset seed 2) always crashes at the gas station, and its student, trained on the average of the five teachers, completes the route like the four good ones. This is the effect distillation was adopted for.
With large training sets, distillation hardly matters
When the teachers already drive well, there is little to correct. Five of the six students are slightly worse than the average of their teachers in lateral error, and only one (dataset seed 1, second in the ranking) beats all its teachers. Averaging five teachers smooths the steering in the corners (β0.26 against β0.32 at A), which keeps the car on the road but slightly further from the lane centre. Six of six against 29 of 30 is not a statistically significant difference on its own (95 % Wilson interval for the students: 61β100 %). The expectation is that distillation matters again when data is scarce, which is the next experiment.
π Part 4 β Driving Results
All drives use Town02 at noon with a clear sky, the camera at 1.4 m and a target speed of 15 km/h. The gas station was the obstacle almost no model could pass.
Share of models that pass the gas station (about 445 m). Every new model that passes it also completes the five minutes of the route.
Series
Models
Conditions
Result
84
313 previous models
camera 1.4 m, 160 s
20 pass the gas station
85β87
those 20, repeated 3 times
camera 1.4 m, 160 s
16 pass in all four series; 4 change outcome
88β90
36 new models
camera 1.4 m, 300 s, 3 repetitions
35 complete 5 minutes in every repetition
All six datasets of the new series produce good drivers:
Dataset seed
0
1
2
4
9
42
Teachers completing the route
5/5
5/5
4/5
5/5
5/5
5/5
Student
β
β
β
β
β
β
Student's ranking position (of 36)
27
2
22
34
25
30
Rank
Model
Dataset seed
Completed
|cte| (m)
|cte| p95 (m)
Off lane
1
teacher
4
3/3
0.263
0.324
0 %
2
student
1
3/3
0.270
0.352
0 %
3
teacher
9
3/3
0.270
0.383
0 %
4
teacher
42
3/3
0.271
0.381
0 %
5
teacher
0
3/3
0.272
0.374
0 %
β¦
positions 6 to 34: |cte| between 0.273 and 0.372 m, no model off lane
35
teacher
4
3/3
0.399
0.700
1.8 %
36
teacher
2
0/3
crashes at the gas station (443 m)
Ranking order: drives completed without a collision, then mean lateral error, then share of the route off the lane. Most models lie between 0.27 and 0.32 m of lateral error, a difference of 3 to 5 cm in a 4 m lane; only the extremes stand apart. The lateral error of a given model changes by less than 0.005 m between repetitions, so these differences are real, but small.
π₯ Part 5 β Preliminary Robustness Tests
Before the formal evaluation in Behavior Metrics, the best student (dataset seed 1) was driven in harder conditions than the ones it was trained and tested on. Every drive was recorded. These are quick, single-run checks, not yet a systematic evaluation.
Speed: the same route with a target speed of 15, 30, 35, 40, 45 and 60 km/h (the training data was driven at about 15 km/h).
Starting position: five starting points in the interior of Town02, away from the test route.
Starting pose: starting turned 45Β° from the lane, starting on the centre line, and starting in the opposite lane (wrong way).
Lighting: the sun at 45Β° in the morning (09:00) and at 40Β° in the afternoon (16:00), instead of overhead.
15 km/h β reference speed
Test route, five minutes.
35 km/h
More than twice the training speed.
45 km/h
Three times the training speed.
Map interior at 15 km/h
Starting point away from the test route.
Wrong-way start
The car starts in the opposite lane.
Start turned 45Β°
The car starts at 45Β° from the lane direction.
Morning sun at 45Β°
Low sun at 09:00 instead of overhead.
β οΈ First Observations
The model completed the five-minute route at 15, 30, 35 and 40 km/h, and failed within seconds at 60 km/h. The morning and afternoon sun runs ended well before the five minutes: lighting looks like the main weakness, which is consistent with training only on noon images and without brightness augmentation. The pose and starting-point tests are being tabulated.
π Part 6 β Future Work
Behavior Metrics evaluation. Run the new models in Behavior Metrics with the metrics used in Carlos VelΓ‘squez's work: successful runs, collisions, lane invasions, mean speed and mean deviation from the ideal trajectory, so that the results are comparable with the lab's previous work.
Smaller datasets and a more formal test of distillation. Train the same mixture with nested subsets of Β½, ΒΌ and β of the training images, with five teachers and three students per dataset. With less data the teachers should become less reliable, and the comparison of the spread between teachers and between students will show whether distillation helps when data is scarce. The same setup will separate the effect of the clean data from the effect of the new training recipe.
Robustness scenarios. Turn the preliminary tests into a systematic battery: speeds, starting positions and orientations, lighting and weather, and maps not used for training, with several runs per condition. Lighting will be the first focus.
π WEEK 51 SUMMARY β NOVEMBER 17, 2026
π― Objective: Understand why models failed at the left-hand corners of the test route, fix the data and the driving setup, and obtain models that complete the route reliably.
π§ͺ Methodologies: Coherence test between recorded steering and the car's real heading change, applied to all 7,182 base recordings, with a mirroring check. Camera height verified by re-rendering poses. Comp0 rebuilt on the clean base with six dataset seeds (75,285 images, 30 % validation), 30 teachers and 6 distilled students without EMA, flip or brightness augmentation. Driving in Town02 at 15 km/h, 300 s, three repetitions, all recorded. Preliminary robustness tests on speed, starting pose and lighting.
π Key Findings: 28 frozen-steering guardrail recordings and 26 no-turn recordings were hidden in the left-turn category, and many drives were stored three to five times. The driving camera was 1 m higher than the training camera. With both fixed, 35 of 36 models complete five minutes in every repetition, and driving is repeatable to 0.1 m. Distillation removes the only failure but does not improve lane keeping when the training set is large. The model drives up to 40 km/h but not 60 km/h, and low sun is its weak point.
β Achievements: Base data cleaned (7,182 β 3,165 recordings, 0 frozen). Camera fixed at 1.4 m. 36 new models trained and driven 108 times. Recording of every drive with OBS. Ranking and reports of the series.
β Limitations: Several changes were made at once (clean data, camera, recipe), so the improvement cannot yet be attributed to one of them. One map, one route and one weather condition. Robustness tests are single runs. Behavior Metrics results and the pull request replies are still pending.
π Next Steps: Behavior Metrics evaluation. Smaller nested datasets with several students to test distillation formally and to separate the effects of data and recipe. A systematic robustness battery, starting with lighting.
π References
H. Jiang, H. Narasimhan, D. Bahri, A. Cotter, A. Rostamizadeh. Churn Reduction via Distillation. ICLR 2022. arXiv:2106.02654
S. Bhojanapalli, K. Wilber, A. Veit, A. S. Rawat, S. Kim, A. Menon, S. Kumar. On the Reproducibility of Neural Network Predictions. 2021. arXiv:2102.03349