crosstyan/OpenGait
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

SkeletonGait

Browse Source
This commit is contained in:
Jingzhe Ma
2024-03-07 21:10:22 +08:00
parent 662ffb25b9
commit 29578ccfd7
12 changed files with 59 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
configs/skeletongait/README.md
+30 -4
View File
@@ -2,7 +2,9 @@
This [paper](https://arxiv.org/abs/2311.13444) has been accepted by AAAI 2024.
## Step 1: Generating Heatmap
## Generating Heatmap and Training Steps
### Step 1: Generating Heatmap
Leveraging the power of Distributed Data Parallel (DDP), we've streamlined the heatmap generation process. Below is the script to initiate the generation:
```
CUDA_VISIBLE_DEVICES=0,1,2,3 \
@@ -21,9 +23,12 @@ Parameter Guide:
- `--ext_name`: An **optional** suffix for the 'save_root' directory to facilitate identification. Defaults to an empty string.
- `--heatmap_cfg_path`: Path to the configuration file of the heatmap generator. The default setting is `configs/skeletongait/pretreatment_heatmap.yaml`.
Note: If your pose data follows the COCO 18 format (for instance, OU-MVLP pose data or data extracted using [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) in COCO format), ensure to set `transfer_to_coco17` to True in the configuration file `configs/skeletongait/pretreatment_heatmap.yaml`.
**Optional**
## Step 2: Creating Symbolic Links for Heatmap and Silhouette Data
### Step 2: Creating Symbolic Links for Heatmap and Silhouette Data
The script to symlink heatmaps and silouettes is as follows:
@@ -38,9 +43,9 @@ Parameter Guide:
- `--heatmap_data_path`: The **absolute** path to your heatmap data. This is **required**.
- `--silhouette_data_path`: The **absolute** path to your silhouette data. This is **required**.
- `--output_path`: Designates the directory for linked output data. This is **required**.
- `--dataset_pkl_ext_name`: An **optional** parameter to specify the extension for `.pkl` silhouette files. Defaults to `.pkl`.
- `--dataset_pkl_ext_name`: An **optional** parameter to specify the extension for `.pkl` silhouette files. Defaults to `.pkl`. CCPG is `aligned-sils.pkl`, SUSTech-1K is `Camera-Sils_aligned.pkl`, and other is `.pkl`.
## Step3: Training SkeletonGait or SkeletonGait++
### Step3: Training SkeletonGait or SkeletonGait++
The script to SkeletonGait is as follows:
@@ -61,3 +66,24 @@ CUDA_VISIBLE_DEVICES=0,1,2,3 \
--cfgs ./configs/skeletongait/skeletongait++_Gait3D.yaml \
--phase train --log_to_file
```
## Performance for SkeletonGait and SkeletonGait++
### SkeletonGait
| Datasets | `Rank1` | Configuration |
|---------------------|---------|----------------------------------------------|
| CCPG | CL: 52.4, UP: 65.4, DN: 72.8, BG: 80.9 | [skeletongait_CCPG.yaml](./skeletongait_CCPG.yaml) |
| OU-MVLP (AlphaPose) | TODO | [skeletongait_OUMVLP.yaml](./skeletongait_OUMVLP.yaml) |
| SUSTech-1K | Normal: 54.2, Bag: 51.7, Clothing: 21.34, Carrying: 51.59, Umberalla: 44.5, Uniform: 53.37, Occlusion: 67.07, Night: 44.15, Overall: 51.46 | [skeletongait_SUSTech1K.yaml](./skeletongait_SUSTech1K.yaml) |
| Gait3D | 38.1 | [skeletongait_Gait3D.yaml](./skeletongait_Gait3D.yaml) |
| GREW | TODO | [skeletongait_GREW.yaml](./skeletongait_GREW.yaml) |
### SkeletonGait++
| Datasets | `Rank1` | Configuration |
|---------------------|---------|-------------------------------------------------|
| CCPG | CL: 90.1, UP: 95.0, DN: 92.9, BG: 97.0 | [skeletongait++_CCPG.yaml](./skeletongait++_CCPG.yaml) |
| SUSTech-1K | Normal: 85.09, Bag: 82.90, Clothing: 46.53, Carrying: 81.88, Umberalla: 80.76, Uniform: 82.50, Occlusion: 86.16, Night: 47.48, Overall: 81.33 | [skeletongait++_SUSTech1K.yaml](./skeletongait++_SUSTech1K.yaml) |
| Gait3D | 77.40 | [skeletongait++_Gait3D.yaml](./skeletongait++_Gait3D.yaml) |
| GREW | 87.04 | [skeletongait++_GREW.yaml](./skeletongait++_GREW.yaml) |
configs/skeletongait/skeletongait++_CCPG.yaml
+2 -2
View File
@@ -1,6 +1,6 @@
data_cfg:
dataset_name: CCPG
dataset_root: your_path
dataset_root: /data3/gait_heatmap_data/CCPG/heatmap_sil
dataset_partition: ./datasets/CCPG/CCPG.json
num_workers: 1
data_in_use: [True, True] # heatmap, sil
@@ -10,7 +10,7 @@ data_cfg:
evaluator_cfg:
enable_float16: true
restore_ckpt_strict: true
restore_hint: 60000
restore_hint: /home/jdy/fanchao/OpenGait_230701/output/CCPG/DeepGaitV2_P3D_GaitMap_MM_FeatureLevel/DeepGaitV2_P3D_GaitMap_B1C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True_ML_LowLevel/checkpoints/DeepGaitV2_P3D_GaitMap_B1C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True_ML_LowLevel-60000.pt
save_name: SkeletonGaitPP
eval_func: evaluate_CCPG
sampler:
configs/skeletongait/skeletongait++_GREW.yaml
+2 -2
View File
@@ -1,6 +1,6 @@
data_cfg:
dataset_name: GREW
dataset_root: your_path
dataset_root: /data3/gait_heatmap_data/GREW/heatmap_sil
dataset_partition: ./datasets/GREW/GREW.json
num_workers: 1
data_in_use: [True, True] # heatmap, sil
@@ -10,7 +10,7 @@ data_cfg:
evaluator_cfg:
enable_float16: true
restore_ckpt_strict: true
restore_hint: 180000
restore_hint: /home/jdy/fanchao/OpenGait_230701/output/GREW/DeepGaitV2_P3D_GaitMap_MM_FeatureLevel/DeepGaitV2_P3D_GaitMap_B4C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True_LowLevel-AttentionFusion/checkpoints/DeepGaitV2_P3D_GaitMap_B4C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True_LowLevel-AttentionFusion-180000.pt
save_name: SkeletonGaitPP
eval_func: GREW_submission
sampler:
configs/skeletongait/skeletongait++_Gait3D.yaml
+2 -2
View File
@@ -10,7 +10,7 @@ data_cfg:
evaluator_cfg:
enable_float16: true
restore_ckpt_strict: true
restore_hint: 60000
restore_hint: /home/jdy/fanchao/OpenGait_230701/output/Gait3D/DeepGaitV2_P3D_GaitMap_MM_FeatureLevel/DeepGaitV2_P3D_GaitMap_B4C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True_MM_FeatureLevel_AttentionFusion/checkpoints/DeepGaitV2_P3D_GaitMap_B4C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True_MM_FeatureLevel_AttentionFusion-60000.pt
save_name: SkeletonGaitPP # LowLevel
eval_func: evaluate_Gait3D
sampler:
@@ -73,7 +73,7 @@ trainer_cfg:
sampler:
batch_shuffle: true
batch_size:
- 4 # TripletSampler, batch_size[0] indicates Number of Identity
- 32 # TripletSampler, batch_size[0] indicates Number of Identity
- 4 # batch_size[1] indicates Samples sequqnce for each Identity
frames_num_fixed: 30 # fixed frames number for training
frames_skip_num: 4
configs/skeletongait/skeletongait++_SUSTech1K.yaml
+2 -2
View File
@@ -1,6 +1,6 @@
data_cfg:
dataset_name: SUSTech1K
dataset_root: your_path
dataset_root: /data3/gait_heatmap_data/SUSTech1K/heatmap_sil
dataset_partition: ./datasets/SUSTech1K/SUSTech1K.json
num_workers: 4
data_in_use: [True, True] # heatmap, sil
@@ -10,7 +10,7 @@ data_cfg:
evaluator_cfg:
enable_float16: true
restore_ckpt_strict: true
restore_hint: 50000
restore_hint: /home/jdy/fanchao/OpenGait_230701/output/SUSTech1K/DeepGaitV2_P3D_GaitMap_MM_FeatureLevel/DeepGaitV2_P3D_GaitMap_B1C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True_ML_LowLevel/checkpoints/DeepGaitV2_P3D_GaitMap_B1C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True_ML_LowLevel-50000.pt
save_name: SkeletonGaitPP
eval_func: evaluate_indoor_dataset #evaluate_Gait3D
sampler:
configs/skeletongait/skeletongait_Gait3D.yaml
+2 -2
View File
@@ -1,6 +1,6 @@
data_cfg:
dataset_name: Gait3D
dataset_root: /data3/gait_heatmap_data/Gait3D/heatmap_sil
dataset_root: your_path
dataset_partition: ./datasets/Gait3D/Gait3D.json
num_workers: 1
data_in_use: [True, False] # heatmap, sil
@@ -78,7 +78,7 @@ trainer_cfg:
sampler:
batch_shuffle: true
batch_size:
- 4 # TripletSampler, batch_size[0] indicates Number of Identity
- 32 # TripletSampler, batch_size[0] indicates Number of Identity
- 4 # batch_size[1] indicates Samples sequqnce for each Identity
frames_num_fixed: 30 # fixed frames number for training
frames_skip_num: 4
configs/skeletongait/skeletongait_OUMVLP.yaml
+2 -2
View File
@@ -1,6 +1,6 @@
data_cfg:
dataset_name: OUMVLP
dataset_root: your_path
dataset_root: /data3/gait_heatmap_data/OUMVLP/OUMVLP_sigma_8.0_/pkl
dataset_partition: ./datasets/OUMVLP/OUMVLP.json
num_workers: 1
data_in_use: [True, False] # heatmap, sil
@@ -10,7 +10,7 @@ data_cfg:
evaluator_cfg:
enable_float16: true
restore_ckpt_strict: true
restore_hint: 120000
restore_hint: /home/jdy/fanchao/OpenGait_230701/output/OUMVLP/DeepGaitV2_P3D_GaitMap/DeepGaitV2_P3D_GaitMap_B4C2_Sigma-8.0_Hot_False_Align-True/checkpoints/DeepGaitV2_P3D_GaitMap_B4C2_Sigma-8.0_Hot_False_Align-True-120000.pt
save_name: SkeletonGait
sampler:
batch_shuffle: false
configs/skeletongait/skeletongait_SUSTech1K.yaml
+2 -2
View File
@@ -1,6 +1,6 @@
data_cfg:
dataset_name: SUSTech1K
dataset_root: your_path
dataset_root: /data3/gait_heatmap_data/SUSTech1K/SUSTech1K_sigma_8.0_/pkl
dataset_partition: ./datasets/SUSTech1K/SUSTech1K.json
num_workers: 4
data_in_use: [True, False] # heatmap, sil
@@ -10,7 +10,7 @@ data_cfg:
evaluator_cfg:
enable_float16: true
restore_ckpt_strict: true
restore_hint: 50000
restore_hint: /home/jdy/fanchao/OpenGait_230701/output/SUSTech1K/DeepGaitV2_P3D_GaitMap/DeepGaitV2_P3D_GaitMap_B1C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True/checkpoints/DeepGaitV2_P3D_GaitMap_B1C2_Sigma-8.0_Hot_False_Align-True_OpenGaitDA-True-50000.pt
save_name: SkeletonGait
eval_func: evaluate_indoor_dataset #evaluate_Gait3D
sampler: