Week 17-18. Continue with DQN algorithm. Problems with graphic card (Part II).

1 minute read

To Do

  • Pong game using DQN algorithm.
  • Train Vanessa’s models.
  • Take a look at the pilot. Get ideas to implement the based on reinforcement training.


1. Pong game using DQN algorithm

Temporarily, a solution has been added to the PONG game using Deep Reinforcement Learning but in this case following the guidelines of the Packtl book, which uses PyTorch as a framework.

DQN Solution using PyTorch.

The results can be seen in the gif. The training has taken approximately two and a half hours with a team with these characteristics:

  • intel i7 7th generation 2.8 GHz 8-core processor.
  • 16 Gb of RAM.
  • Nvidia 1050Ti GPU.
  • 256Gb M.2 SSD hard disk.

As a future work there is still to solve the same exercise with the TensorFlow framework but solving the bug that appeared in the training.

2. Train Vanessa’s models

This task has not been solved this week.

Before the training, a restructuring of all the code, models and dataset has been carried out to adapt it to an order where more types of similar autonomous driving projects fit.

3. Take a look at the pilot. Get ideas to implement the based on reinforcement training.

Vanessa's master's dissertation diagram.
Workflow of Vanessa’s master’s dissertation.


There is still a lot of work to be done in restructuring the initial code. I keep working on it until I have some minimum stability that allows me to face the problem with ‘reinformenet learning’.


Landing on other people’s code is a difficult job but it forces you to understand how someone else wants their code. Communicating with the other person to reach a common point where the code can grow is an enriching job.