Week 6 - Refining some code and testing some frequencies among others…
For this week we have multiple work borders open that I am going to explain more in detail in this post. To have a better visual of it, the task were as follows:
- Keep exploring some datasets to train from
- Study of the frames-per-second and frequencies from the server-client model simulation, Carla
- Be able to train a model in my local computer from a dataset made by Sergio Paniego
- Refining the previous PilotNet model implemented in a vehicle from the Carla Simulator
Dataset exploration
First, the exploration of the data, a task that I found more complicated than I thought, for most of the public dataset for self-driving cars were orientated towards segmentation and detection tasks. Despite these complications I could find multiple datasets from Udacity, an educational organization which has available multiple dataset for free, that comes with the images from RGB cameras, the steering and the throttle values among others that I find most useful for future task of my thesis.
Frequency study
Secondly I centered myself most of the week on trying to understand and comprehend, the necessary information that comes from analyzing the frequencies on the server-client simulation, Carla. To better understand how our machine can handle the simulation demand of resources, we need to know if the speed at which the server is rendering new images is fast enought to keep with the client speed to recieve and process the images, and viceversa. In order to do this, we analyze the frequencies in two possible ways, one where the client only recieves the images, and other where the client uses a model to predict the images output. A table containing the frames-per-second from the client and the server without using a PilotNet model and using it can be seen below.
Further study of this results may be needed, for the frames-per-second were seen decaying the longer we kept the simulation running. Also, the results from this version of the code were different compared to an older version of the code, so I would like to deepen my understanding of this little inconsistency.
Model training
A training of a model from the DeepLearningStudio repository was achieved, by adjusting the Python version to 3.7 and 3.6 along with some minor updates for the packages. It was posible to train a model in our local machine using the DeepLearningStudio dataset. It took around 7 minutes to complete the training. More study of the code would be wise to again deepen my knowledge of how to use Tensorflow to train deep neural networks.
Refining the PilotNet-based model for Carla
For the final task of the week, we needed to test wether if we were adapting correctly the PilotNet-based model from DeepLearningStudio to the carla simulation or if we were doing something wrong as we saw in the previous post from the failed neural network video. To do this, two changes were made:
- One of them consisted on cutting the horizon from the recieved image of the camera, and changing the color that delimitates the lanes to red. This changes were made in order to simulate in the closest way possible, the environment in which the model was trained.
- The other big change was basically to tweak the output (throttle and steer) of the model, to adapt it for the Carla Simulation values.
The result of this refining work can be seen in the next video:
Simulating the same path were we saw the model fail in the previous week, we can see that the car follow the lane without doing some weird movements that take it out of its course. It is clear, that the model was going to fail in our case when no lane is detected in the image, but this was the expected behaviour, because the vehicle is also following the lane as the model was train to do.