Week 6. 14/09/2020

This week I had to do:

1. Understand the D1 Kibotics how works simcore, brains and JavaScrip-Editor. –> I have a look at all the files that are part of the Kibotics D1 and trying to understad how Kibotics works. More or less
2. Making a ‘Acustic Teleoperator’ Robot has to understand Go, Stop and other word for Back with the tutorial I made of TensorFlowjs audio recognition. –> Before anything I had to know what word is predicted in the audio recognition tensorflowjs

In this code: […]

    recognizer.listen(({scores}) => {
     // Turn scores into a list of (score,word) pairs.
     scores = Array.from(scores).map((s, i) => ({score: s, word: words[i]}));
     console.log(scores);
     // Find the most probable word.
     scores.sort((s1, s2) => s2.score - s1.score);
     document.querySelector('#console').textContent = scores[0].word;
    //Te most likely word
      console.log(scores[0].word);

    //Position of go:   words.indexOf(2)
    //'go'= words[words.indexOf('go')
    // 'stop'= words[words.indexOf('stop')
    // My BACK is SEVEN ' seven '= words[words.indexOf('seven') 
    
    if (scores[0].word == words[words.indexOf('go')]){
      var texto = 'ADELANTE!!';
      var h1 = document.getElementById("palabra");
      h1.innerHTML = texto;
      console.log(texto);
      localRobot.setV(0.9);


    }else if (scores[0].word == words[words.indexOf('stop')]){
      var texto = 'PARAAAA!!';
      var h1 = document.getElementById("palabra");
      h1.innerHTML = texto;
      console.log(texto);
      localRobot.setV(0);


    }else if (scores[0].word == words[words.indexOf('seven')]){
      var texto = 'Atrás!!';
      var h1 = document.getElementById("palabra");
      h1.innerHTML = texto;
      console.log(texto);
      localRobot.setV(-0.9);
    [...] 

Then I saw the examples of teleoperators in the kibotics D1 platform and they didn’t work because the build folder was missing so I copy the build folder in to the teleoperators folder and everything worked!!!. My model is based on the drone.html of this teleoperators so I insert my new code in the js folder –> in teleop.js I insert my audio recognition code when the robot is loaded.

Here is the video of how works:
‘GO’ is going onward Robot.setV(0.9). ‘STOP’ is going 0 velocity Robot.setV(0) so the robot stops. ‘SEVEN’ is going back Robot.setV(-0.9).

• Problems/Issues: I have some problems with the audio recognition with the noise of the computer fan when Websim is running.

The next week goals are:

1.Refine Acustic Teleoperator:

-Include a fence or wall or futbol goal
-Try with training model, record examples.
Also, try Physics Motor of Natalia Monforte.

2.Inicial soundtrack in JavaScript Editor.