Week 1 - Two frame types

2 minute read

To Do

The tasks proposed for this week are

  • Adapt the sample generator to obtain raw images and modeled images.
  • Start training with the different combinations of input and output images types.

Progress

Modeled and raw images

After the last meeting, the possibility of using two different types of images that represented the same information was considered. For this we introduce the concept of modeled image in which the entire image is transformed into two values (x, y) that represent the position of the active pixel in the image.

If we use the image itself as the network input, the number of parameters is totally dependent on the size of the image, being more complex the more big is the picture. With this transformation we reduce the number of parameters that the network must use trying to make learning more effective and scalable.

As for the code modified in the generator, the change has not been too large since the way to generate the image was to obtain the active position according to the dynamics of movement and subsequently generate the image. Below is the function in the Frames.py code that obtains the samples with the new line that will store the modeled image (self.modeled_sample).

  def get_sample(self):
        positions_x, positions_y = self.get_positions()
        for i in range(self.n_points + 1):
            self.raw_sample.append(self.get_image(positions_x[i], positions_y[i]))
            self.modeled_sample.append([positions_x[i], positions_y[i]])

With the function self.get_positions() the positions of the active pixel in the image are generated according to the movement so, as we said previously, it is only necessary to store these positions in a list with the structure (x,y)

Once the sample is generated, it is necessary to modify the function that stores the samples so that it saves in a file that list of positions of each frame:

  def save(self, image_path, filename, sample_file_path):
        sample_df = pd.DataFrame(columns=['x', 'y'])

        for i, image in enumerate(self.raw_sample):
            if i == 0:
                utils.check_dirs(image_path, True)
            cv2.imwrite(image_path + "/" + str(i) + '.png', image)
            sample_df.loc[i] = self.modeled_sample[i]

        sample_df.to_csv(sample_file_path, index=False)

        ...

In the following video you can see the two types of data.

New trainings

In view of the new data generated, we are presented with different ways of attacking the training that will be explored in the coming weeks:

  • Input: Raw image –> Output: Raw image
  • Input: Modeled image –> Output: Modeled image
  • Input: Modeled image –> Output: Raw image

The next work to be done will be to attack the different trainings using the linear motion database at the initial height is random (3 degrees of freedom: speed, slope and height).

You May Also Enjoy