The robots doesn't move during iteration

[speedhawk]

Hi, I am using deepbots to construct an environment for my path planning project. I have successfully executed the CartPole game as the tutorial. When I want to construct an action space of my own, however, I always failed. Just as the same sight of CartPole, I set my action in to discrete 3 instead of 2 meaning my robots got three actions totally:

# six elements include x-axis position, y-axis position, ps0 value, ps1 value, ps6 value, ps7 value.
self.observation_space = Box(low=np.array([-0.25, -0.88, 0.00, 0.00, 0.00, 0.00]),
                                     high=np.array([0.75, 0.12, 4095.00, 4095.00, 4095.00, 4095.00]),
                                     dtype=np.float64)

self.action_space = Discrete(3)

self.robot = self.getSelf()

self.leftMotor = self.getDevice("left wheel motor")
self.rightMotor = self. getDevice("right wheel motor")
self.leftMotor.setPosition(float('inf'))
self.rightMotor.setPosition(float('inf'))
self.leftMotor.setVelocity(0.0)
self.rightMotor.setVelocity(0.0)`

Additionally, I did the same setting in apply_action() method according to the tutorial, in which I mapped the three actions to my robots with detailed behaviors:

def apply_action(self, action):
        action = int(action[0])
        print("action: ", action)

        l_speed = 3.14
        r_speed = 3.14

        if action == 0:     # go straight
            l_speed = 3.14
            r_speed = 3.14
        if action == 1:     # turn right
            l_speed = 3.14
            r_speed = -3.14
        if action == 2:     # turn left
            l_speed = -3.14
            r_speed = 3.14

        self.leftMotor.setVelocity(l_speed)
        self.rightMotor.setVelocity(r_speed)

Here, The control algorithms may be not correct but the robots should at least stochastically move. During iteration, however, I found that my robot (in my projects, I used an e-punk robot) never moved even one step no matter how many timestep I set. I don't know what happened to my code and hope anyone can help me to fix it. Many thanks!

[tsampazk]

Hello! You could try setting the positions of the motors to infinite every time you use setVelocity, you can see an example here.

Other than that, make sure that you get proper action output from the agent (i can see that you are already printing action) and/or try setting the motor speeds directly to make sure they are properly set on Webots. Let me know if that helps 😄 .

[tsampazk]

I will convert this to a discussion as it is not really an issue with any of the tutorials 😃

[speedhawk]

Hi, thank you for your reply! Based on your answer, I can clarify that my understanding of the data structure of deepbots is correct. 😃

[speedhawk]

Hi, thank you for your reply! Based on your answer, I can clarify that my understanding of the data structure of deepbots is correct. 😃

[tsampazk]

Did you try setting the position to infinite similar to the example i linked? Can you also verify that the e-puck motor indeed has the correct controller and visually check whether the wheels are turning through the 3d view of the simulation? You could start with the example i linked and work your way up from there.

[speedhawk]

Yes of course. Now everything is OK. Thank you for your help! Additionally, I have reviewed the documentation of setVelocity() and setPosition() function for why it is required to reset the position each time I call the action. But unfortunately, I only know it is because of PID control algorithms for reseting but don't know the crucial reasons why the exclusive reseting of position is required in my project but not in CartPole game 😂😂.

[speedhawk]

Oh, BTW, during this delving of fixing, I have to say that I found there are quite few of path planning instances in gym environment. Detaily, what I mean is that most of gym environment about path planning cannot achieves as well as true 3D simulations (i.g. the cliff walking game can be regarded as a path planning environment but in real-world the robots cannot perform grid-by-grid walking, which is really like turn-based game😂). Therefore, are there any existed existed of real-time 3D path planning in deepbots?

[tsampazk]

Alright, happy to help! Can you please "mark as answer" the relevant answer that fixed your issue?

Path planning is a complicated matter to be honest. You could translate grid positions to discrete positions (in the simulator or in the real world) for a robot to move. It is not really path-planning per-se but you can check out our new find and avoid v2 example here.