<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/feed.xml" rel="self" type="application/atom+xml" /><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/" rel="alternate" type="text/html" /><updated>2026-10-15T10:58:29+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/feed.xml</id><title type="html">ROBOTICS LAB URJC</title><subtitle>My TFG weekly updates</subtitle><author><name>Diego López Municio</name></author><entry><title type="html">Article Revision and Conference Acceptance</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/Article_revision/" rel="alternate" type="text/html" title="Article Revision and Conference Acceptance" /><published>2026-10-15T00:00:00+00:00</published><updated>2026-10-15T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/Article_revision</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/Article_revision/"><![CDATA[<h2 id="addressing-the-reviewers-feedback">Addressing the Reviewers’ Feedback</h2>

<p>After submitting our article to the conference, we received a mixed review. While one reviewer was positive about our project, the other raised concerns about some features that had not yet been experimentally validated. They also questioned the project’s originality, pointing out similarities with the <a href="https://github.com/fan-ziqi/rl_sar">rl_sar repository</a>.</p>

<p>The conference organizers initially asked us to submit a revised version of the article addressing these concerns.</p>

<h3 id="validating-the-robots-navigation-capabilities">Validating the Robot’s Navigation Capabilities</h3>

<p>One of the reviewer’s main concerns was that some of the capabilities described in our article had not been sufficiently tested. To address this issue, I created a new simulation world with markers placed on the floor and made the robot follow a route connecting them.</p>

<p>This experiment was intended to demonstrate that our locomotion system could be integrated with a navigation system, allowing the robot to follow a predefined route rather than simply walking in place or moving in a specific direction.</p>

<h3 id="how-does-our-project-differ-from-rl_sar">How Does Our Project Differ from rl_sar?</h3>

<p>Addressing the concerns about the project’s originality was more challenging.</p>

<p>The <a href="https://github.com/fan-ziqi/rl_sar">rl_sar repository</a> contains a ROS 2 node that uses neural networks to control a robot’s locomotion, which is also one of the main goals of our project. Although the two projects share some similarities, they differ in their software environments and intended scope.</p>

<p>For example, rl_sar targets ROS 2 Foxy, whereas our project uses ROS 2 Jazzy. However, we considered this difference insufficient on its own to clearly distinguish the two projects. Therefore, we focused on the additional capabilities we plan to develop.</p>

<p>While rl_sar focuses on locomotion, our long-term goal is to develop a more versatile humanoid robot capable of performing three main tasks:</p>

<ul>
  <li><strong>Whole-body locomotion:</strong> using the entire body to maintain balance and improve stability while walking.</li>
  <li><strong>Object carrying:</strong> walking while keeping the arms available to carry objects, such as a box.</li>
  <li><strong>Fall recovery:</strong> getting back up autonomously after falling.</li>
</ul>

<p>These capabilities represent different challenges in humanoid robotics and are intended to extend the functionality of our locomotion system.</p>

<h3 id="conference-acceptance">Conference Acceptance</h3>

<p>While I was working on the object-carrying functionality, we received another email from the conference organizers. A third reviewer had evaluated our article and given it a positive assessment, so the conference accepted our submission without requiring a revised version.</p>

<p>Nevertheless, we considered the previous reviewer’s concerns valid and decided to continue improving the project. We wanted our claims to be supported by experiments and to make the project’s contributions clearer, regardless of whether a revised submission was still required.</p>

<h2 id="working-on-object-carrying">Working on Object Carrying</h2>

<p>Making the robot walk while carrying an object turned out to be more challenging than expected.</p>

<p>To simulate carrying a box, I initially trained the neural network with the robot’s arms held in a fixed position. However, the resulting network did not work correctly in Gazebo.</p>

<p>After checking the ROS 2 nodes for potential errors, I decided to train the network again, this time without providing the arm joint positions and velocities as inputs.</p>

<p>This new network worked in Gazebo, suggesting that the arm-related inputs had been contributing to the problem. During training in MuJoCo, fixing the arms appeared to produce joint-state values that were not suitable for direct use by the network in the Gazebo simulation.</p>

<p>Removing these inputs allowed us to obtain a network that could run in Gazebo. However, this did not necessarily mean that the robot had learned to carry objects reliably.</p>

<h3 id="results-and-next-steps">Results and Next Steps</h3>

<p>Although we managed to run the new network in Gazebo, the results were not satisfactory enough to consider object carrying a completed feature.</p>

<p>We therefore decided not to present it as an implemented capability of the project. Instead, we will continue working on it as part of the project’s future development.</p>

<p>This experiment highlighted an important challenge when training locomotion policies in simulation: the network’s inputs must remain consistent with the robot’s state representation across different simulation environments.</p>

<h2 id="summary">Summary</h2>

<ul>
  <li><strong>Conference acceptance:</strong> Our article was accepted without requiring a revised submission.</li>
  <li><strong>Navigation demonstration:</strong> I created a simulation environment in which the robot follows a predefined route marked on the floor.</li>
  <li><strong>Object carrying:</strong> I trained and tested a new locomotion network in Gazebo, although its performance was not yet sufficient for reliable object carrying.</li>
  <li><strong>Future work:</strong> We will continue developing the robot’s ability to carry objects while improving the validation of our system’s capabilities.</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Addressing the reviewers' feedback, validating the robot's navigation capabilities, and working on object carrying.]]></summary></entry><entry><title type="html">Summer – Writing an article for a conference</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/Summer/" rel="alternate" type="text/html" title="Summer – Writing an article for a conference" /><published>2026-09-03T00:00:00+00:00</published><updated>2026-09-03T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/Summer</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/Summer/"><![CDATA[<p>Summer was not the most productive period of the project. Due to the holidays of the different people involved, development was significantly slowed down.</p>

<p>After successfully training the networks described in the previous post, I tried to train the final version, with the arms completely free. Unfortunately, I was not able to get it working successfully.</p>

<p>After this failure, the training process had to be paused because we decided to write an article about our project and submit it to a conference. Since the free-arm motion is still under development, we decided to focus the article on the full-body network and leave the free-arm motion as part of the future improvements.</p>

<p>My contribution to the article was to write about the robot and simulators we were using, as well as the programs we were developing and the tests we performed on the network.</p>

<p>The tests were the main focus of my work. Although we had tested the network before, we did not have any concrete quantitative results, since this version was still under development.</p>

<p>For this reason, I developed a ROS2 node that sends a Twist command to the robot for 10 seconds and calculates the final position it should reach if the command is followed without any error. Once the robot finishes moving, the node calculates the error between the desired position and the actual obtained position.</p>

<p>I performed this test both in Gazebo and in MuJoCo, the original simulator used for the network. The results showed that the position error was larger in Gazebo than in MuJoCo.</p>

<p>I also repeated the test while changing the inference rate of the network, from 10 Hz to 70 Hz. This showed that the best inference rate in practice was around 60 Hz, while the network remained stable even at 20 Hz.</p>

<p>The errors we observed were acceptable. With a reactive controller on top of the network, the robot can move without any major problems, since small deviations from the desired trajectory can be corrected by sending another Twist command towards the desired position.</p>

<h2 id="the-main-issue-we-noticed-is-that-the-robot-tends-to-turn-slightly-to-the-right-this-means-that-part-of-the-observed-error-could-come-from-the-ros2-code-rather-than-the-network-itself-we-still-have-to-investigate-the-reason-for-this-behavior">The main issue we noticed is that the robot tends to turn slightly to the right. This means that part of the observed error could come from the ROS2 code rather than the network itself. We still have to investigate the reason for this behavior.</h2>

<h3 id="summary">Summary</h3>

<ul>
  <li>Failed to train the final network with completely free arms.</li>
  <li>Contributed to writing the conference article.</li>
  <li>Developed new tests to quantitatively evaluate the network.</li>
  <li>Compared the network’s performance in Gazebo and MuJoCo.</li>
  <li>Tested different inference rates, finding 60 Hz to be the best rate in practice.</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Summer: experiments, testing and writing a conference paper]]></summary></entry><entry><title type="html">Weeks 34-35 – Training controlling the elbows</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks34-35/" rel="alternate" type="text/html" title="Weeks 34-35 – Training controlling the elbows" /><published>2026-07-16T00:00:00+00:00</published><updated>2026-07-16T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks34-35</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks34-35/"><![CDATA[<p>After the training failure two weeks ago, I investigated how to control the joints that were not directly used during inference. Thanks to Oscar’s advice, I discovered that the correct approach was to control them through events. With that in mind, I trained the network under three different scenarios to evaluate how different elbow behaviors affect the learning process.</p>

<h4 id="fixed-elbows">Fixed elbows</h4>
<p>First, I implemented a step event that forces the elbows to remain at a fixed position throughout every simulation step. This keeps the joints completely static during each training episode.</p>

<p><img src="/2025-tfg-diego-lopez/assets/images/G1_fixed_elbows.gif" alt="fixed elbows" /></p>

<h4 id="fixed-elbows-in-random-positions">Fixed elbows in random positions</h4>
<p>Next, I created a reset event that assigns a random position to each elbow at the beginning of every training episode. Once initialized, the elbows remain fixed in those positions until the episode ends.</p>

<p><img src="/2025-tfg-diego-lopez/assets/images/G1_fixed_elbows_rp.gif" alt="fixed elbows in random position" />
(In the video, only a single training episode is shown, so the elbow positions do not change.)</p>

<h4 id="moving-elbows">Moving elbows</h4>
<p>Finally, I implemented a system that continuously moves the elbows during training. For each elbow, a random target position and a random duration are generated. The required velocity is then computed using the following expression: velocity = (target_position - current_position) / time. At every simulation step, this velocity is added to the current joint position until the target is reached. Once the target is reached, a new random target position and duration are generated, allowing the elbows to move continuously throughout the episode.</p>

<p>The function responsible for generating new targets is called both during the reset event (to initialize the first movement) and during the step event whenever an elbow reaches its current target.</p>

<p><img src="/2025-tfg-diego-lopez/assets/images/G1_moving_elbows.gif" alt="moving elbows" /></p>

<hr />

<h3 id="summary">Summary</h3>

<ul>
  <li>Investigated why the uncontrolled joints were not behaving as expected.</li>
  <li>Trained the network with fixed elbow positions.</li>
  <li>Trained the network with randomly initialized but fixed elbow positions.</li>
  <li>Trained the network with continuously moving elbow positions.</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Weeks 34 and 35 of the project: training the network using the elbows in diferent ways]]></summary></entry><entry><title type="html">Week 33 – Training failed</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week33/" rel="alternate" type="text/html" title="Week 33 – Training failed" /><published>2026-07-02T00:00:00+00:00</published><updated>2026-07-02T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week33</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week33/"><![CDATA[<p>After last week’s new network, my goal this week was to train another one with the elbows fixed in position and, if that worked, try to make them move during training.</p>

<p>The first issue appeared while I was reviewing last week’s network. I discovered that the observations did not include any information about the elbows. Because of this, I had to train a new network after finding the reason why the elbow data was missing from the observations. Fortunately, this new training run was successful.</p>

<p>After that, I started training another network with the elbows fixed in position. However, when I tested how well it walked, I found that the elbows were just as “dead” as they had been in the previous training.</p>

<p>Next week, I will continue investigating this issue and hopefully achieve what I originally intended to do this week.</p>

<hr />

<h3 id="summary">Summary</h3>

<ul>
  <li>Fixed the observation space to include elbow information.</li>
  <li>Successfully retrained the previous network.</li>
  <li>Failed to train a network with fixed elbows.</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Week 33 of the project: failed to fix the elbows on a position]]></summary></entry><entry><title type="html">Week 32 – Retraining</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week32/" rel="alternate" type="text/html" title="Week 32 – Retraining" /><published>2026-06-26T00:00:00+00:00</published><updated>2026-06-26T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week32</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week32/"><![CDATA[<p>This week goal was to free the arms from the network control. In order to do that, I firstly try to traing it leaving the elbows “dead”.</p>

<p>The training was succesfull, obtaining this result.</p>

<p><img src="/2025-tfg-diego-lopez/assets/images/G1_walking_mjlab_dead_elbows.gif" alt="g1" /></p>

<p>Next week I’ll be working on keeping the elbows in place and then making the robot able to move while I move them.</p>

<hr />

<h3 id="summary">Summary</h3>

<ul>
  <li>Training the network to not use the elbows</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Week 32 of the project: retraining the network to free the arms]]></summary></entry><entry><title type="html">Week 31 – Testing the Network</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week31/" rel="alternate" type="text/html" title="Week 31 – Testing the Network" /><published>2026-06-10T00:00:00+00:00</published><updated>2026-06-10T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week31</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/week31/"><![CDATA[<p>The main goal this week was to test how the robot reacts to commands in different situations.</p>

<p>First, I checked how it performed without modifying anything. The robot tends to turn to the right even when it is supposed to stay still or move in a straight line. This issue could be addressed with a higher-level movement manager that is aware of the robot’s position and can correct its direction.</p>

<p>In its current state, the robot is functional, but not yet useful for real tasks. In particular, it needs to properly handle arm control. To study this, I tested three different configurations:</p>

<ul>
  <li>Keeping the arms in the default position</li>
  <li>Not controlling the arms at all</li>
  <li>Controlling one arm manually</li>
</ul>

<hr />

<h4 id="keeping-the-arms-in-the-default-position">Keeping the arms in the default position</h4>

<p>In this test, I modified the code to continuously send the default arm position. This is based on how the network computes commands using the formula:</p>

<p><code class="language-plaintext highlighter-rouge">default_pos + action * scale</code></p>

<p>By setting the scale to 0, the network’s output is effectively ignored.</p>

<p>In this configuration, the robot is not able to maintain a stable position and performs erratic movements. However, it is still able to execute commands correctly, so with a higher-level controller (as mentioned above), it could be properly managed.</p>

<hr />

<h4 id="not-controlling-the-arms-at-all">Not controlling the arms at all</h4>

<p>In this case, I stopped sending any commands to the arm topics, so the motors remain inactive.</p>

<p>Here, the robot performs better than in the default mode. Although the difference is not very noticeable during active control, when the command is to stay still, the robot remains more stable and avoids the slight rightward turning observed before.</p>

<hr />

<h4 id="controlling-one-arm-manually">Controlling one arm manually</h4>

<p>Finally, I stopped sending network commands and manually controlled one arm.</p>

<p>As soon as the arm is raised, the robot loses balance and enters a falling state. It begins to move in the direction the arm is pointing. In this condition, the robot cannot be properly controlled, especially if the goal is to move in the opposite direction of the arm, since it is unable to compensate for the imbalance.</p>

<hr />

<h3 id="summary">Summary</h3>

<ul>
  <li>Tested the network under different arm-control configurations</li>
  <li>Identified stability issues related to arm movement and default control</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Week 31 of the project: testing the network's limits]]></summary></entry><entry><title type="html">Weeks 29-30 – Simulating the robot in Gazebo</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks29-30/" rel="alternate" type="text/html" title="Weeks 29-30 – Simulating the robot in Gazebo" /><published>2026-05-28T00:00:00+00:00</published><updated>2026-05-28T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks29-30</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks29-30/"><![CDATA[<p>During these two weeks, I made a major improvement in the project by migrating the robot simulation from MuJoCo to Gazebo and integrating it with ROS2.</p>

<p>After successfully controlling the robot in MuJoCo using a neural network, the next step was to port the system to Gazebo in order to work within a ROS2-based pipeline.</p>

<h2 id="gazebo-simulation-setup">Gazebo simulation setup</h2>

<p>The first challenge was obtaining a compatible model of the Unitree G1 robot for Gazebo. I used two main repositories:</p>
<ul>
  <li><a href="https://github.com/luisdbesteiro/gazebo-harmonic-ros">Luis Dominguez Besteiro - G1 Gazebo simulation for ROS2</a></li>
  <li><a href="https://github.com/unitreerobotics/unitree_ros/tree/master/robots/g1_description">Unitree G1 description</a></li>
</ul>

<p>However, Luis’ repository was designed to run inside a Docker environment using ROS2 Humble, which caused compatibility issues with my local ROS2 Jazzy setup.</p>

<p>Although the migration from ROS2 Humble to Jazzy required minimal changes, the main difficulty came from the heavy use of absolute paths inside the Docker container. These paths were used to locate models, worlds, and ROS-Gazebo bridges, making the project non-portable.</p>

<p>To solve this, I restructured the project to make it compatible with a standard ROS2 workspace:</p>

<ul>
  <li>Moved all required directories inside the package structure</li>
  <li>Replaced absolute Docker paths with dynamically generated ROS2 package paths</li>
  <li>Adjusted launch files to not use Docker-specific rendering constraints as the default lauching mode</li>
  <li>Migrated the build system from <code class="language-plaintext highlighter-rouge">setup.py</code> to <code class="language-plaintext highlighter-rouge">CMakeLists.txt</code> to ensure proper resource installation via <code class="language-plaintext highlighter-rouge">colcon</code></li>
</ul>

<p>After these changes, the G1 model was fully functional in Gazebo.</p>

<h2 id="inference-node-implementation">Inference node implementation</h2>

<p>Once the simulation was ready, I implemented the inference node responsible for controlling the robot using the neural network.</p>

<p>A key concern was latency introduced by ROS2 topic communication. To minimize overhead, I implemented the entire inference pipeline in C++, avoiding multiple distributed nodes and reducing inter-process communication.</p>

<p>The node subscribes to the following topics:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">/cmd_vel</code></li>
  <li><code class="language-plaintext highlighter-rouge">/g1/imu/pelvis</code></li>
  <li><code class="language-plaintext highlighter-rouge">/g1/joint_states</code></li>
  <li><code class="language-plaintext highlighter-rouge">/g1/pelvis/odometry</code></li>
</ul>

<p>Using these inputs, the node constructs the feature vector required by the neural network. The model is executed using the ONNX Runtime library.</p>

<p>The output of the network is then published directly to each joint controller topic, enabling real-time control of the robot.</p>

<h2 id="result">Result</h2>

<p>With this implementation, the G1 robot in Gazebo is able to follow velocity commands published to <code class="language-plaintext highlighter-rouge">/cmd_vel</code> using the trained neural network policy.</p>

<p>Finally, all changes were integrated into the repository, and the README was updated with installation and usage instructions.</p>

<p>Here there is a small GIF of the G1 robot being teleoperated inside the Gazebo simulator.</p>

<p><img src="/2025-tfg-diego-lopez/assets/images/G1_teleoperated.gif" alt="g1 teleoperatd" /></p>

<h3 id="summary">Summary</h3>

<ul>
  <li>Simulating the G1 in Gazebo</li>
  <li>Controlling the network using ROS2</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Weeks 29 to 30 of the project: migrating the neural network control to Gazebo with ROS2]]></summary></entry><entry><title type="html">Weeks 23–28 – Using the Network</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks23-28/" rel="alternate" type="text/html" title="Weeks 23–28 – Using the Network" /><published>2026-05-13T00:00:00+00:00</published><updated>2026-05-13T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks23-28</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks23-28/"><![CDATA[<p>During this month, the project improved significantly.</p>

<p>First, Oscar identified several issues I had while using the network:</p>

<ul>
  <li>Using an incorrect initial position</li>
  <li>Sending the absolute position instead of the offset from the default position</li>
  <li>Projecting gravity onto the base instead of the pelvis</li>
  <li>Assuming that the network returned the values in the same order used by the simulation</li>
</ul>

<p>After identifying these problems, he developed a new version of the code that fixed them.</p>

<p>Once the issues were solved, I spent some time understanding the new implementation and later modified it.</p>

<p>I integrated the three-thread architecture from the previous post into Oscar’s code, introducing two main changes:</p>

<ul>
  <li>The first change was modifying the robot’s movements to better observe its rotation.</li>
  <li>The second, and most important one, was changing how the threads handled timing.</li>
</ul>

<p>The simulation runs much faster than real time. For example, a 16-second simulation only takes around 2.7 seconds to execute. Because of this, the control thread was not behaving correctly: when the initial 2-second waiting state finished, the simulation was already close to ending.</p>

<p>To solve this issue, I started using simulation time instead of real time inside the threads. However, since it is difficult to know exactly how long each thread should wait before its next iteration, I implemented a micro-sleep strategy. The threads repeatedly sleep for very short periods until the simulation time reaches the next scheduled execution point.</p>

<p><img src="/2025-tfg-diego-lopez/assets/images/G1_walking.gif" alt="G1 walking" />
G1 walking routine</p>

<h3 id="summary">Summary</h3>

<ul>
  <li>Understanding Oscar’s code</li>
  <li>Integrating the three-thread architecture</li>
  <li>Fixing timing issues in the threads</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Weeks 23 to 28 of the project: finally using the network]]></summary></entry><entry><title type="html">Weeks 20-22 – Using the Network</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks20-22/" rel="alternate" type="text/html" title="Weeks 20-22 – Using the Network" /><published>2026-04-15T00:00:00+00:00</published><updated>2026-04-15T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks20-22</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks20-22/"><![CDATA[<p>During these three weeks, I continued improving the code in two main areas.</p>

<p>The first one was understanding how the neural network is used. I have now confirmed that it takes 99 inputs, and the heading of the twist command was actually the extra value that should not be included. 
Once this was clarified, I continued working on fixing the way the network is integrated into the simulation.</p>

<p>I discovered that the network output does not directly represent the final joint values. Instead, it produces values between -1 and 1, which are multiplied by a scale factor to obtain an offset. This offset is then applied to the default joint positions to determine the final pose.</p>

<p>Although I found several arrays that seem to correspond to the scaling factors and default joint positions, the network is still not working correctly. This suggests that there may still be an issue either in these arrays or in the way the inputs are being computed.</p>

<p>At the same time, I also changed the way the simulation loop is structured.</p>

<p>Previously, inference was performed during every simulation iteration before updating the simulator. Now, the process has been divided into three separate threads:</p>
<ul>
  <li><strong>Simulation thread</strong>: runs the MuJoCo simulation as fast as possible.</li>
  <li><strong>Network thread</strong>: performs inference at <strong>50 Hz</strong>.</li>
  <li><strong>Control thread</strong>: generates the movement commands used by the network at <strong>10 Hz</strong>.</li>
</ul>

<p>This new structure should improve performance and make the execution flow more realistic by separating simulation, control, and inference tasks.</p>

<h3 id="summary">Summary</h3>
<ul>
  <li>Improved understanding of the network inputs</li>
  <li>Updated how the network outputs are interpreted</li>
  <li>Reworked the simulation architecture using three threads</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Weeks 20 to 22 of the project: continuing the development]]></summary></entry><entry><title type="html">Weeks 18–19 – Using the network</title><link href="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks18-19/" rel="alternate" type="text/html" title="Weeks 18–19 – Using the network" /><published>2026-03-12T00:00:00+00:00</published><updated>2026-03-12T00:00:00+00:00</updated><id>https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks18-19</id><content type="html" xml:base="https://roboticslaburjc.github.io/2025-tfg-diego-lopez/blog/weeks18-19/"><![CDATA[<p>During these two weeks, I focused on learning how to use the neural network.</p>

<p>Since the network was not created by me, the first thing I needed to do was determine what its inputs were.</p>

<p>Using the following code:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">session</span><span class="p">.</span><span class="n">get_inputs</span><span class="p">():</span>
    <span class="k">print</span><span class="p">(</span><span class="n">i</span><span class="p">.</span><span class="n">name</span><span class="p">,</span> <span class="n">i</span><span class="p">.</span><span class="n">shape</span><span class="p">,</span> <span class="n">i</span><span class="p">.</span><span class="nb">type</span><span class="p">)</span>

<span class="k">for</span> <span class="n">o</span> <span class="ow">in</span> <span class="n">session</span><span class="p">.</span><span class="n">get_outputs</span><span class="p">():</span>
    <span class="k">print</span><span class="p">(</span><span class="n">o</span><span class="p">.</span><span class="n">name</span><span class="p">,</span> <span class="n">o</span><span class="p">.</span><span class="n">shape</span><span class="p">,</span> <span class="n">o</span><span class="p">.</span><span class="nb">type</span><span class="p">)</span>
</code></pre></div></div>

<p>I obtained the following result:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>obs <span class="o">[</span>1, 99] tensor<span class="o">(</span>float<span class="o">)</span>
actions <span class="o">[</span>1, 29] tensor<span class="o">(</span>float<span class="o">)</span>
</code></pre></div></div>

<p>Since the robot has 29 joints, the output clearly corresponds to the commands sent to those joints. The problem was understanding the inputs.</p>

<p>After checking how the network was originally designed, I concluded that the inputs were:</p>

<ul>
  <li>IMU linear velocity</li>
  <li>IMU angular velocity</li>
  <li>Projected gravity</li>
  <li>Joint positions</li>
  <li>Joint velocities</li>
  <li>Previous output</li>
  <li>Desired command</li>
</ul>

<p>The issue appeared when checking the dimensions.<br />
If all the data obtained from MuJoCo were placed in a single array, the resulting dimension was <code class="language-plaintext highlighter-rouge">[1, 113]</code>.</p>

<p>While searching online, I found that the positions and velocities returned by MuJoCo include some values that are not related to the joints themselves. After removing those values, the size became <code class="language-plaintext highlighter-rouge">[1, 100]</code>, which is still one element larger than expected.</p>

<p>Since I could not find where the extra value was coming from, I tried removing the last element of the twist command. This value represents the desired facing direction, so I thought it might not be essential.</p>

<p>It clearly did not work.</p>

<p><img src="/2025-tfg-diego-lopez/assets/images/G2_first_try.gif" alt="G2_failure" /></p>

<hr />

<h3 id="summary">Summary</h3>

<ul>
  <li>Identified the network inputs and outputs</li>
  <li>Investigated how to obtain the required data</li>
  <li>Attempted to run the network</li>
</ul>]]></content><author><name>Diego López Municio</name></author><category term="blog" /><summary type="html"><![CDATA[Weeks 18 and 19 of the project: using the neural network to infer how to walk]]></summary></entry></feed>