Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

how can we export actor policy and depoly it in the real world for h1 MaskedMimic #54

Open
dbdxnuliba opened this issue Feb 6, 2025 · 1 comment
Open

how can we export actor policy and depoly it in the real world for h1 MaskedMimic #54

dbdxnuliba opened this issue Feb 6, 2025 · 1 comment

Comments

@dbdxnuliba
Copy link

dbdxnuliba commented Feb 6, 2025 •
edited
Loading

Greate work!
after train h1 MaskedMimic
how can we export actor policy and depoly it in the real world or geneis simulator for h1 MaskedMimic

I print the obs dict contents ,shows many keys when Inference for h1 MaskedMimic:

def evaluate_policy(self):
        self.create_eval_callbacks()
        self.pre_evaluate_policy()

        done_indices = []
        step = 0
        while self.config.max_eval_steps is None or step < self.config.max_eval_steps:
            print('ppo evaluate_policy')
            self.handle_reset(done_indices)

            # Update obs incase any environments were reset
            obs = self.create_agent_obs()
            # print('obs:', obs)

            for key in obs:
                tensor_data = obs[key]
                
                if isinstance(tensor_data, torch.Tensor):
                    converted = tensor_data.cpu().detach().numpy()
                else:
                    converted = tensor_data
                
                # 打印基本信息
                print(f"Key: {key}")
                print(f"Shape: {converted.shape}")
                print(f"Data type: {converted.dtype}")
                # print("Sample values:", converted[0][:5] if len(converted.shape) > 1 else converted[:5])  # 打印前5个值
                print("Sample values:", converted[0][:] if len(converted.shape) > 1 else converted[:]) 


                print("-" * 50)


            # Obtain actor predictions
            actor_outs = self.pre_eval_env_step(obs)

            # Step env
            rewards, dones, extras = self.env_step(actor_outs["actions"])

            all_done_indices = dones.nonzero(as_tuple=False)
            done_indices = all_done_indices.squeeze(-1)

            self.post_eval_env_step(actor_outs)

            step += 1

        self.post_evaluate_policy()

when inference
PYTHON_PATH phys_anim/eval_agent.py +robot=h1 +backbone=isaacgym +opt=[masked_mimic/tasks/user_control] +motion_file=phys_anim/data/motions/h1_punch.npy +checkpoint=/home/rob/rl/ProtoMotions/results/h1_masked_mimic/lightning_logs/version_0/last.ckpt

Key: obs
Shape: (1, 343)
Data type: float32
Sample values: [ 9.72779989e-01  5.41139208e-02 -2.69925948e-02  5.96944690e-01
 -1.53534319e-02  9.52443257e-02 -1.69392422e-01  2.39392258e-02
  9.63765830e-02 -1.72903970e-01  2.02930328e-02  2.11611003e-01
 -1.76863745e-01  2.12467581e-01  2.05644459e-01 -5.27627170e-01
  7.42711127e-02  1.88379556e-01 -9.02599037e-01 -1.60764232e-02
 -7.95771703e-02 -1.77233294e-01  2.30912045e-02 -7.59857371e-02
 -1.80620745e-01  3.36467363e-02 -1.90861255e-01 -1.80472881e-01
  1.38744399e-01 -1.81698352e-01 -5.66310167e-01 -7.63845667e-02
 -2.01903209e-01 -9.02925849e-01 -0.00000000e+00  0.00000000e+00
 -5.96046590e-08  5.72040007e-02  1.34919390e-01  4.33105618e-01
  5.60295954e-02  1.92752659e-01  4.45289463e-01  3.47755961e-02
  2.21215308e-01  3.15774113e-01  2.15447266e-02  2.61489630e-01
  1.21482268e-01  2.17788219e-01  2.80784398e-01 -1.04610942e-01
  3.12734321e-02 -1.74484119e-01  4.21465665e-01  2.05564834e-02
 -2.32094675e-01  4.29237992e-01 -4.81630117e-03 -2.68420666e-01
  3.02455455e-01 -2.40576714e-02 -3.23421597e-01  1.12322114e-01
  1.67117894e-01 -3.93254936e-01 -1.08076356e-01  9.95915830e-01
  3.16649675e-08 -9.02861878e-02  9.01947692e-02 -4.49889824e-02
  9.94907439e-01  9.95915830e-01  3.16649675e-08 -9.02861878e-02
  9.01947692e-02 -4.49889824e-02  9.94907439e-01  9.95626509e-01
  2.84982435e-02 -8.89711380e-02  9.01946276e-02 -4.49889898e-02
  9.94907558e-01  9.95626509e-01  2.84982566e-02 -8.89711827e-02
  8.78959000e-02  3.69923525e-02  9.95442688e-01  8.76463711e-01
  4.42535803e-02  4.79430139e-01 -4.80426192e-01  1.49067687e-02
  8.76908600e-01  9.37888801e-01  1.78194754e-02 -3.46477866e-01
  3.45488250e-01  4.31630947e-02  9.37429845e-01  9.99458730e-01
  3.19471546e-02  7.86217116e-03 -8.95020738e-03  3.40583511e-02
  9.99379814e-01  9.92139697e-01  9.10647288e-02 -8.58259276e-02
  9.01947245e-02 -4.49889936e-02  9.94907498e-01  9.92139697e-01
  9.10647735e-02 -8.58258829e-02  8.53489414e-02  9.12605599e-03
  9.96309340e-01  9.60515976e-01  8.86085853e-02  2.63738602e-01
 -2.62745351e-01 -2.29037441e-02  9.64593589e-01  8.38071823e-01
  7.63213784e-02 -5.40194154e-01  5.37829161e-01  5.05087376e-02
  8.41538906e-01  9.95803416e-01  9.15127322e-02  4.94055683e-04
 -6.09319075e-04  1.23159867e-03  9.99998689e-01  9.92421627e-01
 -7.96505511e-02 -9.35711190e-02  9.01947543e-02 -4.49889749e-02
  9.94907677e-01  9.84624147e-01 -3.92190628e-02 -1.70227587e-01
  1.32573485e-01 -4.66794789e-01  8.74372423e-01  9.84624565e-01
 -3.92190367e-02 -1.70227647e-01  1.58283263e-01 -2.11972520e-01
  9.64372933e-01  9.79222357e-01 -9.16933194e-02 -1.80875167e-01
  1.58283308e-01 -2.11972445e-01  9.64372337e-01  6.54128909e-01
  6.43112063e-02 -7.53643692e-01  7.45684206e-01 -2.21819922e-01
  6.28291726e-01  6.54129446e-01  6.43112436e-02 -7.53643870e-01
  7.45684385e-01 -2.21819997e-01  6.28292203e-01  9.80259240e-01
 -1.09076053e-01 -1.64908633e-01  1.94125876e-01  3.72692943e-01
  9.07422602e-01  9.80259180e-01 -1.09076023e-01 -1.64908633e-01
  1.88904569e-01  2.70432085e-01  9.44024384e-01  9.81615543e-01
 -7.87417293e-02 -1.73869953e-01  1.88904628e-01  2.70432115e-01
  9.44024205e-01  6.37248158e-01 -2.32781753e-01 -7.34661460e-01
  7.70174503e-01  1.58576265e-01  6.17806435e-01  6.37248158e-01
 -2.32781753e-01 -7.34661460e-01  7.70174503e-01  1.58576265e-01
  6.17806435e-01 -3.31024118e-02 -1.06188040e-02  4.34005633e-02
  5.11404872e-02 -2.48120874e-02  3.50943096e-02 -4.76441346e-02
 -1.40878130e-02  5.29315211e-02 -4.93843704e-02 -3.71082989e-03
  4.27560806e-02 -5.65778166e-02  1.97394774e-03  4.42998745e-02
 -1.03885941e-02  1.10530406e-02  4.50412296e-02  1.52042843e-02
  3.63329682e-03 -1.09860171e-02 -4.96434122e-02 -1.90965971e-03
  4.96922880e-02 -5.02469577e-02 -1.17254145e-02  3.86247598e-02
 -6.15281872e-02 -1.68481432e-02  3.76518145e-02 -4.14545909e-02
 -1.45494398e-02  2.22698431e-02  5.57093741e-03 -1.06915622e-03
  1.01675396e-03 -5.05155534e-04 -1.61057431e-02  4.01371159e-02
  2.86730751e-02 -2.12551150e-02  3.89515758e-02  2.85186004e-02
 -2.00809799e-02  3.93374600e-02  2.77278926e-02 -1.87235717e-02
  3.97121049e-02  2.43259948e-02 -2.05353480e-02  3.72961126e-02
  2.18619127e-02 -2.30562910e-02  3.44856046e-02  3.59692797e-02
 -2.11890601e-02  3.35267335e-02  4.08709086e-02 -9.13435034e-03
  2.84492727e-02  5.10976687e-02  1.90826766e-02  1.87190473e-02
  5.71410507e-02  4.21770141e-02  1.49049247e-02  6.02237992e-02
  5.52582145e-02  1.40100662e-02  2.19672248e-02  1.31308436e-01
 -1.57888338e-03  2.19672248e-02  1.31308436e-01 -1.57888338e-03
  3.30526605e-02  1.25779048e-01  1.20700695e-01 -1.59173496e-02
  1.24377355e-01  1.25076756e-01 -9.63102560e-03 -7.39667043e-02
  1.31892487e-01 -5.55733312e-03 -2.02498212e-01  1.36309266e-01
 -2.97787599e-02  5.61729670e-01  1.10047817e-01  1.09907882e-02
  1.36783451e-01 -1.22656003e-01 -1.10449297e-02  1.34760872e-01
 -1.20749786e-01  1.12638879e-03  2.31605163e-03 -1.20579258e-01
  1.96380690e-02 -1.99122027e-01 -1.20319918e-01  1.05818445e-02
 -1.00575000e-01 -1.20446794e-01  2.40175463e-02  1.30285740e-01
  2.10374296e-02  8.81343149e-03  1.21992650e-02 -3.96992713e-02
  2.70917336e-03  1.24424081e-02 -3.86439338e-02  3.66684189e-03
  1.11599034e-02 -3.28091569e-02  5.77628380e-03  2.73503195e-02
 -2.95966715e-02  5.77628380e-03  2.73503195e-02 -2.95966715e-02
  1.78859755e-02 -2.03181040e-02  8.42045993e-02  1.58310428e-01
 -3.59435044e-02  6.05810359e-02  1.45992145e-01 -5.35781123e-02
 -9.77836316e-04  1.46521896e-01 -3.49685997e-02 -6.41486095e-03
  1.46521896e-01 -3.49685997e-02 -6.41486095e-03]
--------------------------------------------------
Key: terrain
Shape: (1, 256)
Data type: float32
Sample values: [0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278 0.97278
 0.97278 0.97278 0.97278 0.97278]
--------------------------------------------------
Key: mimic_target_poses
Shape: (1, 4140)
Data type: float32
Sample values: [-0.02575041 -0.0354844   0.09536458 ...  0.17878556  0.23773971
  0.95473343]
--------------------------------------------------
Key: masked_mimic_target_poses
Shape: (1, 2024)
Data type: float32
Sample values: [-0.02575041 -0.0354844   0.09536458 ...  0.          0.619529
  1.        ]
--------------------------------------------------
Key: masked_mimic_target_poses_masks
Shape: (1, 11)
Data type: bool
Sample values: [ True False False False False False False False False False False]
--------------------------------------------------
Key: masked_mimic_target_bodies_masks
Shape: (1, 140)
Data type: bool
Sample values: [False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False False False False False False False False False False
 False False False  True False False False False]
--------------------------------------------------
Key: motion_text_embeddings
Shape: (1, 512)
Data type: float32
Sample values: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0.]
--------------------------------------------------
Key: motion_text_embeddings_mask
Shape: (1, 1)
Data type: bool
Sample values: [False]
--------------------------------------------------
Key: historical_pose_obs
Shape: (1, 3120)
Data type: float32
Sample values: [ 6.9573941e-04  1.8151345e-04 -9.3954813e-04 ... -1.4375220e-01
  4.5952749e-01  2.2600000e+00]
--------------------------------------------------
Key: historical_pose_obs_mask
Shape: (1, 15)
Data type: bool
Sample values: [ True  True  True  True  True  True  True  True False False False False
 False False False]
--------------------------------------------------
Key: vae_noise
Shape: (1, 64)
Data type: float32
Sample values: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

Can you please explain the meaning of each element of every key in obs when inference for h1 masked_mimic,

And how to obtain the obs use protomotion when depoly the policy to real h1 robot, and do you have a plan to give an example for depoly the policy trained by protomoion to real robot, if so, I would be very grateful!
@tesslerc
Copy link
Collaborator

tesslerc commented Feb 6, 2025

We don't have an example for deploying, so I can't help with that.
If you have a real robot and are interested in deploying, I'd be happy to chat (please contact me via email) and try figure a path together.

Regarding Genesis, I believe we'll add support. But can't commit to any timeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tesslerc @dbdxnuliba