Training from Scratch (without pretrained weights/checkpoint)

[gelbling]

Hello! Thanks for creating this amazing library.

I've been working on a pose estimator that only includes both arms keypoints: 2 shoulder + 2 elbows + 2 wrists = 6 keypoints.
PLEASE NOTE: I modified the COCO dataset to accomodate only 6 keypoints and training works fine.

I've been training with a CSPNeXt backbone & checkpoint file, but am wondering if this affects the training of the keypoints I care about (see above)
I have tried training without specifying a init_cfg dictionary, but I get an error every time.

Please see the model config code details below!!

QUESTIONS:

1. Does using a whole body pose estimation backbone affect the training outcome of a model with less keypoints?

MODEL CONFIG:

model = dict(
type='TopdownPoseEstimator',
data_preprocessor=dict(
type='PoseDataPreprocessor',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
bgr_to_rgb=True),
backbone=dict(
scope='mmdet',
type='CSPNeXt',
arch='P5',
expand_ratio=0.5,
deepen_factor=0.67,
widen_factor=0.75,
out_indices=(4, ),
channel_attention=True,
norm_cfg=dict(type='SyncBN'),
act_cfg=dict(type='SiLU'),
),
head=dict(
type='RTMCCHead',
in_channels=768,
out_channels=7,
input_size=codec['input_size'],
in_featuremap_size=(6, 8),
simcc_split_ratio=codec['simcc_split_ratio'],
final_layer_kernel_size=7,
gau_cfg=dict(
hidden_dims=256,
s=128,
expansion_factor=2,
dropout_rate=0.,
drop_path=0.,
act_fn='SiLU',
use_rel_bias=False,
pos_enc=False),
loss=dict(
type='KLDiscretLoss',
use_target_weight=True,
beta=10.,
label_softmax=True),
decoder=codec),
test_cfg=dict(flip_test=True))

[Ben-Louis]

Hi @gelbling, sorry for the late reply. If you use a model checkpoint that’s trained with the whole body pose, it usually doesn’t affect the outcome of training a model with fewer keypoints. By the way, you mentioned getting an error when you don’t set init_cfg. Can you tell us more about that error so we can figure out what’s going on?

[gelbling]

Hi @Ben-Louis, thanks for clarifying! I do have a follow-up question.

If you use a model checkpoint that’s trained with the whole body pose, it usually doesn’t affect the outcome of training a model with fewer keypoints.

I'm trying to create a pose estimator mainly for arm positioning. I am currently training the model with the following six keypoints: left-shoulder, right-shoulder, left elbow, right elbow, left wrist, right wrist. I've tested having the nose as the 7th keypoint and noticed a weird correlation between having the nose and the performance of the arm estimation. Do you know if including any other keypoints (such as the nose) increasse accuracy and consistency of arm pose estimation?

By the way, you mentioned getting an error when you don’t set init_cfg. Can you tell us more about that error so we can figure out what’s going on?

With regards to this issue, we figured out whats going on and have fixed it!