1 minute read

After the training failure two weeks ago, I investigated how to control the joints that were not directly used during inference. Thanks to Oscar’s advice, I discovered that the correct approach was to control them through events. With that in mind, I trained the network under three different scenarios to evaluate how different elbow behaviors affect the learning process.

Fixed elbows

First, I implemented a step event that forces the elbows to remain at a fixed position throughout every simulation step. This keeps the joints completely static during each training episode.

fixed elbows

Fixed elbows in random positions

Next, I created a reset event that assigns a random position to each elbow at the beginning of every training episode. Once initialized, the elbows remain fixed in those positions until the episode ends.

fixed elbows in random position (In the video, only a single training episode is shown, so the elbow positions do not change.)

Moving elbows

Finally, I implemented a system that continuously moves the elbows during training. For each elbow, a random target position and a random duration are generated. The required velocity is then computed using the following expression: velocity = (target_position - current_position) / time. At every simulation step, this velocity is added to the current joint position until the target is reached. Once the target is reached, a new random target position and duration are generated, allowing the elbows to move continuously throughout the episode.

The function responsible for generating new targets is called both during the reset event (to initialize the first movement) and during the step event whenever an elbow reaches its current target.

moving elbows


Summary

  • Investigated why the uncontrolled joints were not behaving as expected.
  • Trained the network with fixed elbow positions.
  • Trained the network with randomly initialized but fixed elbow positions.
  • Trained the network with continuously moving elbow positions.

Categories:

Updated: