End effector control action space limits

[nicholscrawford]

It seems like the EE control action space has incorrect limits -- I'm using pd_ee_delta_pose on tabletop tasks, and actions on the scale of [-1, 1] seem like they would violate joint constraints. I can only get some of my algorithms to work with much more realistic scales.

I think ee actions should be limited so they don't violate joint constraints. Is anyone working on this, or is there an appetite for contributions here?

[StoneT2000]

CPU or GPU sim?

Also happy to take on new contributions if they work out well!

[nicholscrawford]

I think it's the same for either, right? Currently testing on CPU

[StoneT2000]

CPU simulation uses a different IK solver (pinnochio) which should be handling joint limits. The controller design is

• user sends delta action (position, RPY)
• convert to delta pose, applied to the current end-effector pose
• use pinnochio to generate the joint positions that would achieve the current end-effector pose + delta pose, solution initialized at the current robot joint position (to get closest solution)
• Internally set the ik solved joint position for all robot joint angles. If IK solver fails (due to being out of range of robot) then robot is not give a new joint position and tries to stay at whatever joint position it was told to go to last.

GPU simulation uses pytorch_kinematics but I think some other users have observed that sometimes the robot will go out of joint limits.

Oh as I write this I think perhaps you are looking at the normalized action space. Most controllers normalize action spaces to -1, 1 range, but an action of +1 in x direction is probably more like 0.1 meters in x direction (this is the case for the panda).

[nicholscrawford]

Sorry yeah I should be more specific. I'm thinking about the joint velocity limits, and using the normalized space. My issue is that the normalized space seems to allow for commands that are too fast, not that they violate joint pos limits.

I haven't confirmed whether or not this is true though!

[StoneT2000]

If you want true joint velocity limits you probably need to write a new PDEEPoseController that doesn't use joint position control?

The alternative is to modify the pd ee pose controller directly. Create a new robot copying the old one / inheriting it and override the controllers. If the max position/rotation changes permitted per action are smaller, then the velocity will be smaller.

Another way to do it (although a little less recommended) is to tune the stiffness/damping values of the robot for each joint to be smaller, which should cause the robot move slower to the desired new end effector pose.

Also one other possible issue is the videos you record are too fast. Recently i find since we use a control frequency of 20 by default, but a video FPS of 30 on the RecordEpisode wrapper, the robot moves faster than it should by 1.5x when in reality it will move slower than that. We should probably make video fps default to None in the wrapper and guess video fps based on the control frequency

[nicholscrawford]

Here's the slowed down random actions with joint position control:

1.mp4

And here's with ee control:

1.mp4

Which I think looks a bit unrealistic. I'll look into the first two options, it sounds interesting.

[StoneT2000]

If you want more realistic try recording the video with RecordEpisode(..., render_substeps=True, video_fps=100) (the env simulation frequency is 100 by default). It might look "realistic" that way since it is recording every single physics step. By default it records every control step (each call to env.step) but by default for PickCube each env.step calls the physics step 5 times, so you are actually seeing just 1/5 total frames and looks not so smooth.