Week 7~11 - Slow but steady wins the race
As we enter a vacation period in our thesis planification, advances will surely slow down when we compared them with the previous weeks. But this does not mean that no progress will be made in the meantime. First, for the sake of organizing the month we need to know what our to-do list is going to be, meaning, what is the primary problem to be solved right now.
As we established in the sixth week, we need to refine even more the brain we are using to make our car, in the Carla Simulator, follow a line. We already made some improvements in the last week but they weren’t enough, for the car wasn’t technically following the line as the model was trained to do. In order to find where the problem might be, we tried to simulate pretty closely the DeepLearningStudio project (even though this one was trained on the Gazebo Simulator), that is, the image we feeded to the model was a close representation of the images from which they trained their models, and the outputs were scaled to try to translate the Gazebo values to Carla values. Because this changes didn’t make the car follow the line as we wanted it to, an assumption was made by thinking that the problem resided in the differences between the both simulations cars dynamics. To pinpoint wether this was true or not, we have to test it by training a model using a Carla Simulation based dataset.
Now that we have landed into the tasks at hand, the next steps can be divided in:
- Create an explicit brain to make our car follow a line. We already had a brain to make the car follow the road, but for the sake of undestanding the problem we explained beforehand, we need to simplify the task, by making it follow a line.
- Create a dataset for the follow-line task in the Carla Simulation. This dataset will be recolected with the use of the explicit follow-line brain.
- Train a model on our own with the recently created dataset, to see if it is capable of following the line better than the PilotNet-based model trained in Gazebo.
Creating the explicit follow-line brain
For this first task, we are going to make our car follow a line, in our case, this line will be the lane that separates the road. In order to solve this problem, we are goint to use what I learned from the Robotic Vision course in my master’s studies, in which luckily enough we were asked to keep a blog to track our progress for the different projects. The project that interests us is one where we were asked to make a Formula 1 car follow a line. Pretty much the same, right?
Few changes were made to the controller, for we don’t need to make our car follow the line as fast as possible. One change that we can notice is that our line in the Carla Simulator is not exactly a single line, but this have a rather simple solution for we only need to get the left side of the left line and the right side of the right line to get the center of this two.
Creating and organizing the data
Now that we have a car that follows a line, we need to collect data from multiple runs. The data we are going to collect is pretty straightforward: the image from the frontal camera, the throttle and the steering value taken from the moment the image was taken.
This gets us to a total of a little more than 15.000 images from the dataset. As always, analyzing the quality of the data is an important step for machine learning in order to understand better if our neural network is going to learn like we want it to, or if we are feeding it some uncomplete data.
We are going to need to polish our dataset, meaning, that we need a better distributed dataset if we are going to make our car follow a line correctly. To improve the dataset we have two options that comes right now to mind. One is increasing our dataset, and the other one is oversampling. The thing with increasing our dataset is that we are still going to need to balance the data, for the variability in our dataset recollections is pretty low. So, by oversampling the weird cases in the dataset we are able to increase the weight of its value in the process of training.
Training and results
With our new balanced dataset, we use the tensorflow library to train a PilotNet model to follow a line. The behaviour of the model can be seen in the next video.
Concluding with the last video, we have created an explicit follow line algorithm, organize and created a dataset and trained a model (such as the PilotNet) to implement onto the car, and in doing so we have established that training the model on the Carla Simulation dataset, improve the behaviour of the car on the task of following a line. But we may need to be able to translate dataset from other simulations (even from real data) if we want to train from them our Carla vehicle. That being said, we have made some interesting progress towards having machine learning methods used on the Carla environment which is pretty cool :)