Language-Assisted Human Part Motion Learning for
Skeleton-Based Temporal Action Segmentation
(Under Review)

Authors: Bowen Chen, Haoyu Ji, Zhiyong Wang, Benjamin Filtjens, Chunzhuo Wang, Weihong Ren, Bart Vanrumste Honghai Liu (*Corresponding author)

Note：This repository is still under construction. All the pretrained checkpoints will be uploaded soon.

Enviroment

Pytorch == 1.10.1+cu111, torchvision == 0.11.2, python == 3.8.13, CUDA==11.4

Download ViT-B/32.pt into the clip folder

Datasets

All datasets can be downloaded from GoogleDrive or BaiduNetdisk. (~4.3GB). Please contact the author of "Traffic Control Gesture Recognition for Autonomous Vehicles" (IROS 2020, oral) for the TCG dataset. Link: TCG

Note：These datasets have been openly collected, curated, and subsequently released on a cloud platform by the authors of "A Decoupled Spatio-Temporal Framework for Skeleton-based Action Segmentation", rather than provided by the authors of the present paper.

Get Started

NOTE: The training and evaluation are integrated into a single main.py

python main.py {config} {dataset} --log {log}

Here, the dataset can be one of the following: LARA, TCG, PKU-subject, or PKU-view.

If you wish to modify other parameters, please make the necessary changes in csv/{dataset}/{config}.yaml.

Acknowledgement

Our code and repository are built upon:

• DeST
• ASRF
• LaSA

We extend our gratitude to the authors for openly sharing their code and datasets.