Week 28~29 - From good to great: improving the basic model
With each week, we try to expand the functionality of our model by making it more robust and efficient. And is all this the basis for all the work we have been doing this weeks: retrieving more data to have all the needed information to train, to lighten it up so that we can have a smaller and more efficient dataset, improving the behaviour on the road or improving the breaks performance when it encounters a multiple variety of vehicles, and on and on…
Now, arriving to this two weeks we have been focused on this next points:
- Try with a smaller dataset
- Make a more robust dataset having the npc stop more randomly (instead of having it stops on the traffic lights)
- Check what the convolutional layer is watching
- Train and test in multiple weather conditions
The dataset
So, first of all, the dataset we had consisted of roughly 80.000 images making it a total of 36.8 Gb. Now the idea was to test how much could we shrink this for we are currently just dealing with Town02 and if we want to generalize this to all the available Towns, we are going to increment this dataset a lot.
THe first thing was cutting it all to half, being careful to not cut relevant information, keeping it as balanced as possible. So we tried training the 40.000 images dataset and it was not bad! But (yes there is a but) we were noticing weird behaviour like the car wasn’t able to stay sometimes on the lane or that sometimes it stopped when a car was in front of it and once the car in front of it started running, our car was not able to get out of this stop state. Now, the first thought was pretty straight forward, we had cut too much valuable information and in top of that, there was a thought: what if the “staying in a stop situation” was because the war was always stopping in the same spots, the traffic lights. So we needed not to only know how much to increase the dataset information, but to add random stops on the spawned vehicle so that it would not make the mistake to stop on designated areas only because the spawned car always stopped on the traffic lights areas.
In conclusion, we have added the functionality to make the spawned car stop randomly and recollected the necessary dataset that finally added up to 58.000 images.
From the loss graph we saw in the previous few weeks, we pretty much gathered the information needed and tested it, it seemed as if leaving it at 70 epochs we ended up with a pretty stable model.
Convolutional layer
One of the methods proposed to check if we could see the features map from the convolutional layer was the Grad-CAM, a method that produces a heatmap from the gradients of the objective class from the final convolutional layer. The thing is, that by simply using the final layer we weren’t being able to “see” in a more explanatory way what the car was seeing from its surroundings.
In the next four videos we can visualize on the top right corner what the car feed to the model and what the model is seeing. We have extracted as we can see from the summary all the four convolutional layers. On the top left we have the first layer, top right the second layer, bottom left the third layer and finally bottom right the fourth layer.
From the videos we have that the four layer is more easy to understand for the model but we can understand pretty little from it. So in order to see if the other layers were more visually attractive we had to check and it seems to be more understandable than the final layer, that as we saw from the summary it was 18 by 1, basically a straigh line features.