Training Kit-ML encountered dimension mismatch problem

[lixiang927047]

@ChenFengYe I want to use the framework of MotionGPT to train Kit-ML, but I have encountered a lot of dimensional incompatibility problems, most of which are due to the fact that the data set is only 251 dimensional and the network is set to 263 dimensional (probably because the number of summary joints in the Kit-ML data set is 21). The number of joints for HumanML3D is 22). What do I need to do with the Kit-ML data set before training? Or do different Settings in the training script?

[billl-jiang]

Hello,

For training Kit-ML with MotionGPT and addressing the dimension mismatch issue, you simply need to update the configuration.

MotionGPT/configs/config_h3d_stage1.yaml

Lines 33 to 34 in a56b09f

	DATASET:
	target: mGPT.data.HumanML3D.HumanML3DDataModule

Change mGPT.data.HumanML3D.HumanML3DDataModule to mGPT.data.Kit.KitDataModule in the config file. This should align the dimensions by adapting to the Kit dataset's structure.

[XGraphKhipu]

Hello!, where can I find the pretrained model for KIT?

[fyyakaxyy]

Hello!, where can I find the pretrained model for KIT?

Hello, have you solved this problem? When I train on KIT_ML, the bug is:

Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/mgpt/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/ubuntu/anaconda3/envs/mgpt/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/MotionGPT/train.py", line 94, in <module>
    main()
  File "/home/ubuntu/MotionGPT/train.py", line 45, in main
    model = build_model(cfg, datamodule)
  File "/home/ubuntu/MotionGPT/mGPT/models/build_model.py", line 8, in build_model
    return instantiate_from_config(model_config)
  File "/home/ubuntu/MotionGPT/mGPT/config.py", line 42, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/home/ubuntu/MotionGPT/mGPT/models/mgpt.py", line 37, in __init__
    super().__init__()
  File "/home/ubuntu/MotionGPT/mGPT/models/base.py", line 17, in __init__
    self.configure_metrics()
  File "/home/ubuntu/MotionGPT/mGPT/models/base.py", line 142, in configure_metrics
    self.metrics = BaseMetrics(datamodule=self.datamodule, **self.hparams)
  File "/home/ubuntu/MotionGPT/mGPT/metrics/base.py", line 24, in __init__
    self.M2TMetrics = M2TMetrics(
  File "/home/ubuntu/MotionGPT/mGPT/metrics/m2t.py", line 98, in __init__
    self._get_t2m_evaluator(cfg)
  File "/home/ubuntu/MotionGPT/mGPT/metrics/m2t.py", line 132, in _get_t2m_evaluator
    self.t2m_moveencoder.load_state_dict(
  File "/home/ubuntu/anaconda3/envs/mgpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MovementConvEncoder:
  size mismatch for main.0.weight: copying a param with shape torch.Size([512, 259, 4]) from checkpoint, the shape in current model is torch.Size([512, 247, 4]).