[Question] Best / most efficient way to disable one arm in training in bimanual robot

[CreativeNick]

I'm using a bimanual allegro robot and am only training on picking up an object on a table using one arm (right arm). Currently I just positioned the left arm (the arm that's disabled from being considered in RL training) all the way to the edge of the table and am used a policy that discourages any movement but I'm not sure if this is the most efficient / best way of doing so.

Since this is for sim to real, I don't want to just delete the arm entirely in the UDRF file or enable the right arm to clip through the left arm since that can't actually happen in reality. I also don't want to wildly move the left arm away in a position that would hit a wall or another object in real life.

Even with my current setup, the left arm still moves a bit but generally remains in roughly the same position. However, I'm thinking it would be computationally more efficient if the left arm wasn't even considered in the RL training process at all. Is there a better method that could achieve this? Would it be a wrapper that only enables the right arm and disables the left arm? I know there's also the method of simply commenting out all left arm/hand related joints and such but I want to set the initial position of the left arm to be away from the object/area so it doesn't get in the way. I saw in the Simulation and Robotics 101 documentation that kinematic actors "are partially physically simulated. If any force is placed on this actor, it will not deform or move a single centimeter. However, dynamic objects that interact with this actor will receive reactionary forces. Compared to dynamic objects however kinematic objects use less cpu/gpu memory and are faster to simulate." However, I couldn't find a way to make certain joints kinematic.

Thank you!

Current setup:
In ycb_env.py:

...


def compute_dense_reward(self, obs: Any, action: np.ndarray, info: Dict):

  ...

  # penalty for left arm movement
  left_hand_positions = torch.stack([link.pose.p for link in self.left_hand_link], dim=1)
  left_hand_movement = torch.linalg.norm(left_hand_positions - left_hand_positions.detach(), dim=-1).mean(dim=-1)
  left_arm_penalty = -left_hand_movement * 50.0
  total_reward += left_arm_penalty

  ...
  

...

In bimanual_allegro:

...

@property
    def _controller_configs(self):
        # different properties for left and right arm
        arm_pd_joint_delta_pos = PDJointPosControllerConfig(
            self.arm_joint_names,
            lower=[-0.0, -0.1] * 6, # left arm no movement (0), right arm normal (-0.1)
            upper=[0.0, 0.1] * 6, # left arm no movement (0), right arm normal (0.1)
            stiffness=[self.arm_stiffness * 100, self.arm_stiffness] * 6, # high stiffness for left, normal for right
            damping=[self.arm_damping * 100, self.arm_damping] * 6, # high damping for left, normal for right
            force_limit=self.arm_force_limit,
            use_delta=True,
            friction=self.arm_friction,
        )

        arm_pd_joint_pos = PDJointPosControllerConfig(
            self.arm_joint_names,
            lower=None,
            upper=None,
            stiffness=[self.arm_stiffness * 100, self.arm_stiffness] * 6,
            damping=[self.arm_damping * 100, self.arm_damping] * 6,
            force_limit=self.arm_force_limit,
            normalize_action=False,
            friction=self.arm_friction,
        )

        hand_pd_joint_delta_pos = PDJointPosControllerConfig(
            self.hand_joint_names,
            lower=-0.1,
            upper=0.1,
            stiffness=self.hand_stiffness,
            damping=self.hand_damping,
            force_limit=self.hand_force_limit,
            use_delta=True,
            friction=self.hand_friction,
        )

        hand_pd_joint_pos = PDJointPosControllerConfig(
            self.hand_joint_names,
            lower=None,
            upper=None,
            stiffness=self.hand_stiffness,
            damping=self.hand_damping,
            force_limit=self.hand_force_limit,
            normalize_action=True,
            friction=self.hand_friction,
        )

        controller_configs = dict(
            pd_joint_delta_pos=dict(
                arm=arm_pd_joint_delta_pos,
                hand=hand_pd_joint_delta_pos
            ),
            pd_joint_pos=dict(
                arm=arm_pd_joint_pos,
                hand=hand_pd_joint_pos
            )
        )
        return deepcopy_dict(controller_configs)

...

[StoneT2000]

If you are not controlling the left arm at all but still want in sim, use the PassiveController in mani_skill/agents/controllers/passive_controller.py class and apply that to all the left arm joints. The robot will simply stay at wherever the set qpos is and if it is pushed around it will stay where it was pushed to.

You will need a new controller config such as pd_joint_delta_pos_right_arm=dict(left_arm=passive_controller, right_arm=..., hand=...)