1.0.0 official release (#18)
* fix bug in fix_BN * gaitgl OUMVLP support. * update ./doc/3.advance_usage.md Cross-Dataset Evalution & Data Agumentation * update config * update docs.3 * update docs.3 * add loss doc and gather input decorator * refine the create model doc * support rearrange directory of unzipped OUMVLP * fix some bugs in loss_aggregator.py * refine docs and little fix * add oumvlp pretreatment description * pretreatment dataset fix oumvlp description * add gaitgl oumvlp result * assert gaitgl input size * add pipeline * update the readme. * update pipeline and readme * Corrigendum. * add logo and remove path * update new logo * Update README.md * modify logo size Co-authored-by: 12131100 <12131100@mail.sustech.edu.cn> Co-authored-by: noahshen98 <77523610+noahshen98@users.noreply.github.com> Co-authored-by: Noah <595311942@qq.com>
This commit is contained in:
Junhao Liang
GitHub
@@ -1,40 +1,49 @@
|
|||||||
**Note:**
|
<img src="./assets/logo2.png" width = "320" height = "110" alt="logo" />
|
||||||
This code is only used for **academic purposes**, people cannot use this code for anything that might be considered commercial use.
|
|
||||||
|
|
||||||
# OpenGait
|
<div align="center"><img src="./assets/nm.gif" width = "100" height = "100" alt="nm" /><img src="./assets/bg.gif" width = "100" height = "100" alt="bg" /><img src="./assets/cl.gif" width = "100" height = "100" alt="cl" /></div>
|
||||||
|
|
||||||
OpenGait is a flexible and extensible gait recognition project provided by the [Shiqi Yu Group](https://faculty.sustech.edu.cn/yusq/) and supported in part by [WATRIX.AI](http://www.watrix.ai). Just the pre-beta version is released now, and more documentations as well as the reproduced methods will be offered as soon as possible.
|
------------------------------------------
|
||||||
|
|
||||||
|
OpenGait is a flexible and extensible gait recognition project provided by the [Shiqi Yu Group](https://faculty.sustech.edu.cn/yusq/) and supported in part by [WATRIX.AI](http://www.watrix.ai).
|
||||||
|
|
||||||
**Highlighted features:**
|
**Highlighted features:**
|
||||||
- **Multiple Models Support**: We reproduced several SOTA methods, and reached the same or even better performance.
|
- **Multiple Models Support**: We reproduced several SOTA methods, and reached the same or even the better performance.
|
||||||
- **DDP Support**: The officially recommended [`Distributed Data Parallel (DDP)`](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) mode is used during the training and testing phases.
|
- **DDP Support**: The officially recommended [`Distributed Data Parallel (DDP)`](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) mode is used during both the training and testing phases.
|
||||||
- **AMP Support**: The [`Auto Mixed Precision (AMP)`](https://pytorch.org/tutorials/recipes/recipes/amp_recipe.html?highlight=amp) option is available.
|
- **AMP Support**: The [`Auto Mixed Precision (AMP)`](https://pytorch.org/tutorials/recipes/recipes/amp_recipe.html?highlight=amp) option is available.
|
||||||
- **Nice log**: We use [`tensorboard`](https://pytorch.org/docs/stable/tensorboard.html) and `logging` to log everything, which looks pretty.
|
- **Nice log**: We use [`tensorboard`](https://pytorch.org/docs/stable/tensorboard.html) and `logging` to log everything, which looks pretty.
|
||||||
|
|
||||||
|
|
||||||
# Model Zoo
|
## Model Zoo
|
||||||
|
|
||||||
|
### CASIA-B
|
||||||
| Model | NM | BG | CL | Configuration | Input Size | Inference Time | Model Size |
|
| Model | NM | BG | CL | Configuration | Input Size | Inference Time | Model Size |
|
||||||
| :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------: | :--------: | :--------: | :------------------------------------------------------------------------------------------- | :--------: | :------------: | :------------: |
|
| :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------: | :--------: | :--------: | :------------------------------------------------------------------------------------------- | :--------: | :------------: | :------------: |
|
||||||
| Baseline | 96.3 | 92.2 | 77.6 | [baseline.yaml](config/baseline.yaml) | 64x44 | 12s | 3.78M |
|
| Baseline | 96.3 | 92.2 | 77.6 | [baseline.yaml](config/baseline.yaml) | 64x44 | 12s | 3.78M |
|
||||||
| [GaitSet(AAAI2019)](https://arxiv.org/pdf/1811.06186.pdf) | 95.8(95.0) | 90.0(87.2) | 75.4(70.4) | [gaitset.yaml](config/gaitset.yaml) | 64x44 | 11s | 2.59M |
|
| [GaitSet(AAAI2019)](https://arxiv.org/pdf/1811.06186.pdf) | 95.8(95.0) | 90.0(87.2) | 75.4(70.4) | [gaitset.yaml](config/gaitset.yaml) | 64x44 | 13s | 2.59M |
|
||||||
| [GaitPart(CVPR2020)](http://home.ustc.edu.cn/~saihui/papers/cvpr2020_gaitpart.pdf) | 96.1(96.2) | 90.7(91.5) | 78.7(78.7) | [gaitpart.yaml](config/gaitpart.yaml) | 64x44 | 22s | 1.20M |
|
| [GaitPart(CVPR2020)](http://home.ustc.edu.cn/~saihui/papers/cvpr2020_gaitpart.pdf) | 96.1(96.2) | 90.7(91.5) | 78.7(78.7) | [gaitpart.yaml](config/gaitpart.yaml) | 64x44 | 56s | 1.20M |
|
||||||
| [GLN*(ECCV2020)](http://home.ustc.edu.cn/~saihui/papers/eccv2020_gln.pdf) | 96.4(95.6) | 93.1(92.0) | 81.0(77.2) | [gln_phase1.yaml](config/gln/gln_phase1.yaml), [gln_phase2.yaml](config/gln/gln_phase2.yaml) | 128x88 | 14s | 8.54M / 14.70M |
|
| [GLN*(ECCV2020)](http://home.ustc.edu.cn/~saihui/papers/eccv2020_gln.pdf) | 96.4(95.6) | 93.1(92.0) | 81.0(77.2) | [gln_phase1.yaml](config/gln/gln_phase1.yaml), [gln_phase2.yaml](config/gln/gln_phase2.yaml) | 128x88 | 47s/46s | 8.54M / 14.70M |
|
||||||
| [GaitGL(ICCV2021)](https://openaccess.thecvf.com/content/ICCV2021/papers/Lin_Gait_Recognition_via_Effective_Global-Local_Feature_Representation_and_Local_Temporal_ICCV_2021_paper.pdf) | 97.4(97.4) | 94.5(94.5) | 83.8(83.6) | [gaitgl.yaml](config/gaitgl.yaml) | 64x44 | 31s | 3.10M |
|
| [GaitGL(ICCV2021)](https://openaccess.thecvf.com/content/ICCV2021/papers/Lin_Gait_Recognition_via_Effective_Global-Local_Feature_Representation_and_Local_Temporal_ICCV_2021_paper.pdf) | 97.4(97.4) | 94.5(94.5) | 83.8(83.6) | [gaitgl.yaml](config/gaitgl.yaml) | 64x44 | 38s | 3.10M |
|
||||||
|
|
||||||
|
### OUMVLP
|
||||||
|
| Model | Rank@1 | Configuration | Input Size | Inference Time | Model Size |
|
||||||
|
| :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------: | :------------------------------------------: | :--------: | :------------- | :--------: |
|
||||||
|
| Baseline | 86.7 | [baseline.yaml](config/OUMVLP/baseline.yaml) | 64x44 | 1m13s | 44.11M |
|
||||||
|
| [GaitSet(AAAI2019)](https://arxiv.org/pdf/1811.06186.pdf) | 87.2(87.1) | [gaitset.yaml](config/OUMVLP/gaitset.yaml) | 64x44 | 1m26s | 6.31M |
|
||||||
|
| [GaitPart(CVPR2020)](http://home.ustc.edu.cn/~saihui/papers/cvpr2020_gaitpart.pdf) | 88.6(88.7) | [gaitpart.yaml](config/OUMVLP/gaitpart.yaml) | 64x44 | 8m04s | 3.78M |
|
||||||
|
| [GaitGL(ICCV2021)](https://openaccess.thecvf.com/content/ICCV2021/papers/Lin_Gait_Recognition_via_Effective_Global-Local_Feature_Representation_and_Local_Temporal_ICCV_2021_paper.pdf) | 89.9(89.7) | [gaitgl.yaml](config/OUMVLP/gaitgl.yaml) | 64x44 | 5m23s | 95.62M |
|
||||||
|
|
||||||
|
|
||||||
The results in the parentheses are mentioned in the papers
|
The results in the parentheses are mentioned in the papers
|
||||||
|
|
||||||
|
|
||||||
**Note**:
|
**Note**:
|
||||||
- All the models were tested on [CASIA-B](http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp) (Rank@1, excluding identical-view cases).
|
- All results are Rank@1, excluding identical-view cases.
|
||||||
- The shown result of GLN is implemented without compact block.
|
- The shown result of GLN is implemented without compact block.
|
||||||
- Only 2 RTX6000 are used during the inference phase.
|
- Only two RTX3090 are used for infering CASIA-B, and eight are used for infering OUMVLP.
|
||||||
- The results on [OUMVLP](http://www.am.sanken.osaka-u.ac.jp/BiometricDB/GaitMVLP.html) will be released soon.
|
|
||||||
It's inference process just cost about 90 secs(Baseline & 8 RTX6000).
|
|
||||||
|
|
||||||
|
|
||||||
# Get Started
|
|
||||||
## Installation
|
## Get Started
|
||||||
|
### Installation
|
||||||
1. clone this repo.
|
1. clone this repo.
|
||||||
```
|
```
|
||||||
git clone https://github.com/ShiqiYu/OpenGait.git
|
git clone https://github.com/ShiqiYu/OpenGait.git
|
||||||
@@ -57,56 +66,66 @@ It's inference process just cost about 90 secs(Baseline & 8 RTX6000).
|
|||||||
pip install tqdm pyyaml tensorboard opencv-python
|
pip install tqdm pyyaml tensorboard opencv-python
|
||||||
pip install torch==1.6.0 torchvision==0.7.0
|
pip install torch==1.6.0 torchvision==0.7.0
|
||||||
```
|
```
|
||||||
## Prepare dataset
|
### Prepare dataset
|
||||||
See [prepare dataset](doc/prepare_dataset.md).
|
See [prepare dataset](docs/0.prepare_dataset.md).
|
||||||
|
|
||||||
## Get trained model
|
### Get trained model
|
||||||
- Option 1:
|
- Option 1:
|
||||||
```
|
```
|
||||||
python misc/download_pretrained_model.py
|
python misc/download_pretrained_model.py
|
||||||
```
|
```
|
||||||
- Option 2: Go to the [release page](https://github.com/ShiqiYu/OpenGait/releases/), then download the model file and uncompress it to `output`.
|
- Option 2: Go to the [release page](https://github.com/ShiqiYu/OpenGait/releases/), then download the model file and uncompress it to [output](output).
|
||||||
|
|
||||||
## Train
|
### Train
|
||||||
Train a model by
|
Train a model by
|
||||||
```
|
```
|
||||||
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/baseline.yaml --phase train
|
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/baseline.yaml --phase train
|
||||||
```
|
```
|
||||||
- `python -m torch.distributed.launch` Our implementation uses DistributedDataParallel.
|
- `python -m torch.distributed.launch` [DDP](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) launch instruction.
|
||||||
- `--nproc_per_node` The number of gpu to use, it must equal the length of `CUDA_VISIBLE_DEVICES`.
|
- `--nproc_per_node` The number of gpus to use, and it must equal the length of `CUDA_VISIBLE_DEVICES`.
|
||||||
- `--cfgs` The path of config file.
|
- `--cfgs` The path to config file.
|
||||||
- `--phase` Specified as `train`.
|
- `--phase` Specified as `train`.
|
||||||
- `--iter` You can specify a number of iterations or use `restore_hint` in the configuration file and resume training from there.
|
<!-- - `--iter` You can specify a number of iterations or use `restore_hint` in the config file and resume training from there. -->
|
||||||
- `--log_to_file` If specified, log will be written on disk simultaneously.
|
- `--log_to_file` If specified, the terminal log will be written on disk simultaneously.
|
||||||
|
|
||||||
You can run commands in [train.sh](train.sh) for training different models.
|
You can run commands in [train.sh](train.sh) for training different models.
|
||||||
|
|
||||||
## Test
|
### Test
|
||||||
Use trained model to evaluate by
|
Evaluate the trained model by
|
||||||
```
|
```
|
||||||
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/baseline.yaml --phase test
|
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/baseline.yaml --phase test
|
||||||
```
|
```
|
||||||
- `--phase` Specified as `test`.
|
- `--phase` Specified as `test`.
|
||||||
- `--iter` You can specify a number of iterations or or use `restore_hint` in the configuration file and restore model from there.
|
- `--iter` Specify a iteration checkpoint.
|
||||||
|
|
||||||
**Tip**: Other arguments are the same as train phase.
|
**Tip**: Other arguments are the same as train phase.
|
||||||
|
|
||||||
You can run commands in [test.sh](test.sh) for testing different models.
|
You can run commands in [test.sh](test.sh) for testing different models.
|
||||||
|
|
||||||
## Customize
|
## Customize
|
||||||
1. First, you need to read the [config documentation](doc/detailed_config.md) to figure out the usage of every item.
|
1. Read the [detailed config](docs/1.detailed_config.md) to figure out the usage of needed setting items;
|
||||||
2. If you want create your own model, see [here](doc/how_to_create_your_model.md).
|
2. See [how to create your model](docs/2.how_to_create_your_model.md);
|
||||||
|
3. There are some advanced usages, refer to [advanced usages](docs/3.advanced_usages.md), please.
|
||||||
|
|
||||||
# Warning
|
## Warning
|
||||||
- Some models may not be compatible with `AMP`, you can disable it by setting `enable_float16` **False**.
|
- Some models may not be compatible with `AMP`, you can disable it by setting `enable_float16` **False**.
|
||||||
- In `DDP` mode, zombie processes may be generated when the program terminates abnormally. You can use this command `kill $(ps aux | grep main.py | grep -v grep | awk '{print $2}')` to clear them.
|
- In `DDP` mode, zombie processes may be generated when the program terminates abnormally. You can use this command [sh misc/clean_process.sh](./misc/clean_process.sh) to clear them.
|
||||||
- We implemented the functionality about testing while training, but it slightly affected the results. None of our published models use this functionality. You can disable it by setting `with_test` **False**.
|
- We implemented the functionality about testing while training, but it slightly affected the results. None of our published models use this functionality. You can disable it by setting `with_test` **False**.
|
||||||
|
- Recommended Pytorch version: 1.6-1.8
|
||||||
|
|
||||||
# Authors:
|
## Authors:
|
||||||
**Open Gait Team (OGT)**
|
**Open Gait Team (OGT)**
|
||||||
- [Chao Fan (樊超)](https://faculty.sustech.edu.cn/?p=128578&tagid=yusq&cat=2&iscss=1&snapid=1&orderby=date)
|
- [Chao Fan (樊超)](https://faculty.sustech.edu.cn/?p=128578&tagid=yusq&cat=2&iscss=1&snapid=1&orderby=date), 12131100@mail.sustech.edu.cn
|
||||||
- [Chuanfu Shen (沈川福)](https://faculty.sustech.edu.cn/?p=95396&tagid=yusq&cat=2&iscss=1&snapid=1&orderby=date)
|
- [Chuanfu Shen (沈川福)](https://faculty.sustech.edu.cn/?p=95396&tagid=yusq&cat=2&iscss=1&snapid=1&orderby=date), 11950016@mail.sustech.edu.cn
|
||||||
- [Junhao Liang (梁峻豪)](https://faculty.sustech.edu.cn/?p=95401&tagid=yusq&cat=2&iscss=1&snapid=1&orderby=date)
|
- [Junhao Liang (梁峻豪)](https://faculty.sustech.edu.cn/?p=95401&tagid=yusq&cat=2&iscss=1&snapid=1&orderby=date), 12132342@mail.sustech.edu.cn
|
||||||
|
|
||||||
# Acknowledgement
|
## Acknowledgement
|
||||||
- GLN: [Saihui Hou (侯赛辉)](http://home.ustc.edu.cn/~saihui/index_english.html)
|
- GLN: [Saihui Hou (侯赛辉)](http://home.ustc.edu.cn/~saihui/index_english.html)
|
||||||
- GaitGL: Beibei Lin (林贝贝)
|
- GaitGL: [Beibei Lin (林贝贝)](https://scholar.google.com/citations?user=KyvHam4AAAAJ&hl=en&oi=ao)
|
||||||
|
|
||||||
|
<!-- ## Citation
|
||||||
|
```
|
||||||
|
``` -->
|
||||||
|
|
||||||
|
**Note:**
|
||||||
|
This code is only used for **academic purposes**, people cannot use this code for anything that might be considered commercial use.
|
||||||
|
|||||||
Binary file not shown.
|
After Width: | Height: | Size: 53 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 70 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 76 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 41 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 61 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 237 KiB |
@@ -1,6 +1,6 @@
|
|||||||
data_cfg:
|
data_cfg:
|
||||||
dataset_name: CASIA-B
|
dataset_name: CASIA-B
|
||||||
dataset_root: your_path
|
dataset_root: your_path
|
||||||
dataset_partition: ./misc/partitions/CASIA-B_include_005.json
|
dataset_partition: ./misc/partitions/CASIA-B_include_005.json
|
||||||
num_workers: 1
|
num_workers: 1
|
||||||
remove_no_gallery: false # Remove probe if no gallery for it
|
remove_no_gallery: false # Remove probe if no gallery for it
|
||||||
@@ -18,9 +18,9 @@ evaluator_cfg:
|
|||||||
sample_type: all_ordered # all indicates whole sequence used to test, while ordered means input sequence by its natural order; Other options: fixed_unordered
|
sample_type: all_ordered # all indicates whole sequence used to test, while ordered means input sequence by its natural order; Other options: fixed_unordered
|
||||||
frames_all_limit: 720 # limit the number of sampled frames to prevent out of memory
|
frames_all_limit: 720 # limit the number of sampled frames to prevent out of memory
|
||||||
metric: euc # cos
|
metric: euc # cos
|
||||||
# transform:
|
transform:
|
||||||
# - type: BaseSilCuttingTransform
|
- type: BaseSilCuttingTransform
|
||||||
# img_w: 128
|
img_w: 64
|
||||||
|
|
||||||
loss_cfg:
|
loss_cfg:
|
||||||
- loss_term_weights: 1.0
|
- loss_term_weights: 1.0
|
||||||
@@ -79,7 +79,7 @@ scheduler_cfg:
|
|||||||
scheduler: MultiStepLR
|
scheduler: MultiStepLR
|
||||||
trainer_cfg:
|
trainer_cfg:
|
||||||
enable_float16: true # half_percesion float for memory reduction and speedup
|
enable_float16: true # half_percesion float for memory reduction and speedup
|
||||||
fix_BN: false
|
fix_BN: true
|
||||||
log_iter: 100
|
log_iter: 100
|
||||||
restore_ckpt_strict: true
|
restore_ckpt_strict: true
|
||||||
restore_hint: 0
|
restore_hint: 0
|
||||||
@@ -97,6 +97,6 @@ trainer_cfg:
|
|||||||
frames_num_min: 25 # min frames number for unfixed traing
|
frames_num_min: 25 # min frames number for unfixed traing
|
||||||
sample_type: fixed_unordered # fixed control input frames number, unordered for controlling order of input tensor; Other options: unfixed_ordered or all_ordered
|
sample_type: fixed_unordered # fixed control input frames number, unordered for controlling order of input tensor; Other options: unfixed_ordered or all_ordered
|
||||||
type: TripletSampler
|
type: TripletSampler
|
||||||
# transform:
|
transform:
|
||||||
# - type: BaseSilCuttingTransform
|
- type: BaseSilCuttingTransform
|
||||||
# img_w: 128
|
img_w: 64
|
||||||
|
|||||||
@@ -0,0 +1,102 @@
|
|||||||
|
data_cfg:
|
||||||
|
dataset_name: OUMVLP
|
||||||
|
dataset_root: your_path
|
||||||
|
dataset_partition: ./misc/partitions/OUMVLP.json
|
||||||
|
num_workers: 1
|
||||||
|
remove_no_gallery: false # Remove probe if no gallery for it
|
||||||
|
test_dataset_name: OUMVLP
|
||||||
|
|
||||||
|
evaluator_cfg:
|
||||||
|
enable_float16: true
|
||||||
|
restore_ckpt_strict: true
|
||||||
|
restore_hint: 150000
|
||||||
|
save_name: Baseline
|
||||||
|
eval_func: identification
|
||||||
|
sampler:
|
||||||
|
batch_shuffle: false
|
||||||
|
batch_size: 4
|
||||||
|
sample_type: all_ordered # all indicates whole sequence used to test, while ordered means input sequence by its natural order; Other options: fixed_unordered
|
||||||
|
frames_all_limit: 720 # limit the number of sampled frames to prevent out of memory
|
||||||
|
metric: euc # cos
|
||||||
|
# transform:
|
||||||
|
# - type: BaseSilCuttingTransform
|
||||||
|
# img_w: 128
|
||||||
|
|
||||||
|
loss_cfg:
|
||||||
|
- loss_term_weights: 1.0
|
||||||
|
margin: 0.2
|
||||||
|
type: TripletLoss
|
||||||
|
log_prefix: triplet
|
||||||
|
- loss_term_weights: 0.1
|
||||||
|
scale: 16
|
||||||
|
type: CrossEntropyLoss
|
||||||
|
log_prefix: softmax
|
||||||
|
log_accuracy: true
|
||||||
|
|
||||||
|
model_cfg:
|
||||||
|
model: Baseline
|
||||||
|
backbone_cfg:
|
||||||
|
in_channels: 1
|
||||||
|
layers_cfg: # Layers configuration for automatically model construction
|
||||||
|
- BC-32
|
||||||
|
- BC-32
|
||||||
|
- M
|
||||||
|
- BC-64
|
||||||
|
- BC-64
|
||||||
|
- M
|
||||||
|
- BC-128
|
||||||
|
- BC-128
|
||||||
|
- BC-256
|
||||||
|
- BC-256
|
||||||
|
type: Plain
|
||||||
|
SeparateFCs:
|
||||||
|
in_channels: 256
|
||||||
|
out_channels: 256
|
||||||
|
parts_num: 31
|
||||||
|
SeparateBNNecks:
|
||||||
|
class_num: 5153
|
||||||
|
in_channels: 256
|
||||||
|
parts_num: 31
|
||||||
|
bin_num:
|
||||||
|
- 16
|
||||||
|
- 8
|
||||||
|
- 4
|
||||||
|
- 2
|
||||||
|
- 1
|
||||||
|
|
||||||
|
optimizer_cfg:
|
||||||
|
lr: 0.1
|
||||||
|
momentum: 0.9
|
||||||
|
solver: SGD
|
||||||
|
weight_decay: 0.0005
|
||||||
|
|
||||||
|
scheduler_cfg:
|
||||||
|
gamma: 0.1
|
||||||
|
milestones: # Learning Rate Reduction at each milestones
|
||||||
|
- 50000
|
||||||
|
- 100000
|
||||||
|
scheduler: MultiStepLR
|
||||||
|
trainer_cfg:
|
||||||
|
enable_float16: true # half_percesion float for memory reduction and speedup
|
||||||
|
fix_BN: false
|
||||||
|
log_iter: 100
|
||||||
|
with_test: true
|
||||||
|
restore_ckpt_strict: true
|
||||||
|
restore_hint: 0
|
||||||
|
save_iter: 10000
|
||||||
|
save_name: Baseline
|
||||||
|
sync_BN: true
|
||||||
|
total_iter: 150000
|
||||||
|
sampler:
|
||||||
|
batch_shuffle: true
|
||||||
|
batch_size:
|
||||||
|
- 32 # TripletSampler, batch_size[0] indicates Number of Identity
|
||||||
|
- 16 # batch_size[1] indicates Samples sequqnce for each Identity
|
||||||
|
frames_num_fixed: 30 # fixed frames number for training
|
||||||
|
frames_num_max: 50 # max frames number for unfixed training
|
||||||
|
frames_num_min: 25 # min frames number for unfixed traing
|
||||||
|
sample_type: fixed_unordered # fixed control input frames number, unordered for controlling order of input tensor; Other options: unfixed_ordered or all_ordered
|
||||||
|
type: TripletSampler
|
||||||
|
# transform:
|
||||||
|
# - type: BaseSilCuttingTransform
|
||||||
|
# img_w: 128
|
||||||
@@ -49,7 +49,6 @@ scheduler_cfg:
|
|||||||
trainer_cfg:
|
trainer_cfg:
|
||||||
enable_distributed: true
|
enable_distributed: true
|
||||||
enable_float16: false
|
enable_float16: false
|
||||||
fix_BN: false
|
|
||||||
log_iter: 100
|
log_iter: 100
|
||||||
restore_ckpt_strict: true
|
restore_ckpt_strict: true
|
||||||
restore_hint: 0
|
restore_hint: 0
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
# Note : *** the batch_size should be equal to the gpus number at the test phase!!! ***
|
||||||
|
data_cfg:
|
||||||
|
dataset_name: OUMVLP
|
||||||
|
dataset_root: your_path
|
||||||
|
dataset_partition: ./misc/partitions/OUMVLP.json
|
||||||
|
num_workers: 1
|
||||||
|
remove_no_gallery: false
|
||||||
|
test_dataset_name: OUMVLP
|
||||||
|
|
||||||
|
evaluator_cfg:
|
||||||
|
enable_distributed: true
|
||||||
|
enable_float16: false
|
||||||
|
restore_ckpt_strict: true
|
||||||
|
restore_hint: 210000
|
||||||
|
save_name: GaitGL
|
||||||
|
sampler:
|
||||||
|
batch_size: 2
|
||||||
|
sample_type: all_ordered
|
||||||
|
type: InferenceSampler
|
||||||
|
|
||||||
|
loss_cfg:
|
||||||
|
- loss_term_weights: 1.0
|
||||||
|
margin: 0.2
|
||||||
|
type: TripletLoss
|
||||||
|
log_prefix: triplet
|
||||||
|
- loss_term_weights: 1.0
|
||||||
|
scale: 1
|
||||||
|
type: CrossEntropyLoss
|
||||||
|
log_accuracy: true
|
||||||
|
label_smooth: true
|
||||||
|
log_prefix: softmax
|
||||||
|
|
||||||
|
model_cfg:
|
||||||
|
model: GaitGL
|
||||||
|
channels: [32, 64, 128, 256]
|
||||||
|
class_num: 5153
|
||||||
|
|
||||||
|
optimizer_cfg:
|
||||||
|
lr: 1.0e-4
|
||||||
|
solver: Adam
|
||||||
|
weight_decay: 0
|
||||||
|
|
||||||
|
scheduler_cfg:
|
||||||
|
gamma: 0.1
|
||||||
|
milestones:
|
||||||
|
- 150000
|
||||||
|
- 200000
|
||||||
|
scheduler: MultiStepLR
|
||||||
|
|
||||||
|
trainer_cfg:
|
||||||
|
enable_distributed: true
|
||||||
|
enable_float16: true
|
||||||
|
with_test: false
|
||||||
|
log_iter: 100
|
||||||
|
restore_ckpt_strict: true
|
||||||
|
restore_hint: 0
|
||||||
|
save_iter: 10000
|
||||||
|
save_name: GaitGL
|
||||||
|
sync_BN: true
|
||||||
|
total_iter: 210000
|
||||||
|
sampler:
|
||||||
|
batch_shuffle: true
|
||||||
|
batch_size:
|
||||||
|
- 32
|
||||||
|
- 8
|
||||||
|
frames_num_fixed: 30
|
||||||
|
frames_skip_num: 0
|
||||||
|
sample_type: fixed_ordered
|
||||||
|
type: TripletSampler
|
||||||
@@ -58,7 +58,6 @@ scheduler_cfg:
|
|||||||
|
|
||||||
trainer_cfg:
|
trainer_cfg:
|
||||||
enable_float16: true
|
enable_float16: true
|
||||||
fix_BN: false
|
|
||||||
log_iter: 100
|
log_iter: 100
|
||||||
restore_ckpt_strict: true
|
restore_ckpt_strict: true
|
||||||
restore_hint: 0
|
restore_hint: 0
|
||||||
|
|||||||
@@ -0,0 +1,82 @@
|
|||||||
|
data_cfg:
|
||||||
|
dataset_name: OUMVLP
|
||||||
|
dataset_root: your_path
|
||||||
|
dataset_partition: ./misc/partitions/OUMVLP.json
|
||||||
|
num_workers: 4
|
||||||
|
remove_no_gallery: false
|
||||||
|
test_dataset_name: OUMVLP
|
||||||
|
|
||||||
|
evaluator_cfg:
|
||||||
|
enable_float16: false
|
||||||
|
restore_ckpt_strict: true
|
||||||
|
restore_hint: 250000
|
||||||
|
save_name: GaitPart
|
||||||
|
sampler:
|
||||||
|
batch_size: 4
|
||||||
|
sample_type: all_ordered
|
||||||
|
type: InferenceSampler
|
||||||
|
metric: euc # cos
|
||||||
|
|
||||||
|
loss_cfg:
|
||||||
|
loss_term_weights: 1.0
|
||||||
|
margin: 0.2
|
||||||
|
type: TripletLoss
|
||||||
|
log_prefix: triplet
|
||||||
|
|
||||||
|
model_cfg:
|
||||||
|
model: GaitPart
|
||||||
|
backbone_cfg:
|
||||||
|
in_channels: 1
|
||||||
|
layers_cfg:
|
||||||
|
- BC-32
|
||||||
|
- BC-32
|
||||||
|
- M
|
||||||
|
- BC-64
|
||||||
|
- BC-64
|
||||||
|
- M
|
||||||
|
- FC-128-3
|
||||||
|
- FC-128-3
|
||||||
|
- FC-256-3
|
||||||
|
- FC-256-3
|
||||||
|
type: Plain
|
||||||
|
SeparateFCs:
|
||||||
|
in_channels: 256
|
||||||
|
out_channels: 256
|
||||||
|
parts_num: 16
|
||||||
|
bin_num:
|
||||||
|
- 16
|
||||||
|
|
||||||
|
optimizer_cfg:
|
||||||
|
lr: 0.0001
|
||||||
|
momentum: 0.9
|
||||||
|
solver: Adam
|
||||||
|
weight_decay: 0.0
|
||||||
|
|
||||||
|
scheduler_cfg:
|
||||||
|
gamma: 0.1
|
||||||
|
milestones:
|
||||||
|
- 150000
|
||||||
|
scheduler: MultiStepLR
|
||||||
|
|
||||||
|
trainer_cfg:
|
||||||
|
enable_float16: true
|
||||||
|
fix_BN: false
|
||||||
|
log_iter: 100
|
||||||
|
with_test: true
|
||||||
|
restore_ckpt_strict: true
|
||||||
|
restore_hint: 0
|
||||||
|
save_iter: 10000
|
||||||
|
save_name: GaitPart
|
||||||
|
sync_BN: false
|
||||||
|
total_iter: 250000
|
||||||
|
sampler:
|
||||||
|
batch_shuffle: false
|
||||||
|
batch_size:
|
||||||
|
- 32
|
||||||
|
- 16
|
||||||
|
frames_num_fixed: 30
|
||||||
|
frames_num_max: 50
|
||||||
|
frames_num_min: 25
|
||||||
|
frames_skip_num: 10
|
||||||
|
sample_type: fixed_ordered
|
||||||
|
type: TripletSampler
|
||||||
@@ -56,7 +56,6 @@ scheduler_cfg:
|
|||||||
|
|
||||||
trainer_cfg:
|
trainer_cfg:
|
||||||
enable_float16: true
|
enable_float16: true
|
||||||
fix_BN: false
|
|
||||||
log_iter: 100
|
log_iter: 100
|
||||||
restore_ckpt_strict: true
|
restore_ckpt_strict: true
|
||||||
restore_hint: 0
|
restore_hint: 0
|
||||||
|
|||||||
@@ -0,0 +1,76 @@
|
|||||||
|
data_cfg:
|
||||||
|
dataset_name: OUMVLP
|
||||||
|
dataset_root: your_path
|
||||||
|
dataset_partition: ./misc/partitions/OUMVLP.json
|
||||||
|
num_workers: 4
|
||||||
|
remove_no_gallery: false
|
||||||
|
test_dataset_name: OUMVLP
|
||||||
|
|
||||||
|
evaluator_cfg:
|
||||||
|
enable_float16: true
|
||||||
|
restore_ckpt_strict: true
|
||||||
|
restore_hint: 250000
|
||||||
|
save_name: GaitSet
|
||||||
|
sampler:
|
||||||
|
batch_size: 4
|
||||||
|
sample_type: all_ordered
|
||||||
|
type: InferenceSampler
|
||||||
|
metric: euc # cos
|
||||||
|
|
||||||
|
loss_cfg:
|
||||||
|
loss_term_weights: 1.0
|
||||||
|
margin: 0.2
|
||||||
|
type: TripletLoss
|
||||||
|
log_prefix: triplet
|
||||||
|
|
||||||
|
model_cfg:
|
||||||
|
model: GaitSet
|
||||||
|
in_channels:
|
||||||
|
- 1
|
||||||
|
- 64
|
||||||
|
- 128
|
||||||
|
- 256
|
||||||
|
SeparateFCs:
|
||||||
|
in_channels: 256
|
||||||
|
out_channels: 256
|
||||||
|
parts_num: 62
|
||||||
|
bin_num:
|
||||||
|
- 16
|
||||||
|
- 8
|
||||||
|
- 4
|
||||||
|
- 2
|
||||||
|
- 1
|
||||||
|
|
||||||
|
optimizer_cfg:
|
||||||
|
lr: 0.0001
|
||||||
|
momentum: 0.9
|
||||||
|
solver: Adam
|
||||||
|
weight_decay: 0
|
||||||
|
|
||||||
|
scheduler_cfg:
|
||||||
|
gamma: 0.1
|
||||||
|
milestones:
|
||||||
|
- 150000
|
||||||
|
scheduler: MultiStepLR
|
||||||
|
|
||||||
|
trainer_cfg:
|
||||||
|
enable_float16: true
|
||||||
|
fix_BN: false
|
||||||
|
with_test: true
|
||||||
|
log_iter: 100
|
||||||
|
restore_ckpt_strict: true
|
||||||
|
restore_hint: 0
|
||||||
|
save_iter: 10000
|
||||||
|
save_name: GaitSet
|
||||||
|
sync_BN: false
|
||||||
|
total_iter: 250000
|
||||||
|
sampler:
|
||||||
|
batch_shuffle: false
|
||||||
|
batch_size:
|
||||||
|
- 32
|
||||||
|
- 16
|
||||||
|
frames_num_fixed: 30
|
||||||
|
frames_num_max: 50
|
||||||
|
frames_num_min: 25
|
||||||
|
sample_type: fixed_unordered
|
||||||
|
type: TripletSampler
|
||||||
@@ -76,7 +76,7 @@ scheduler_cfg:
|
|||||||
trainer_cfg:
|
trainer_cfg:
|
||||||
enable_distributed: true
|
enable_distributed: true
|
||||||
enable_float16: true
|
enable_float16: true
|
||||||
fix_BN: false
|
fix_layers: false
|
||||||
with_test: false
|
with_test: false
|
||||||
log_iter: 100
|
log_iter: 100
|
||||||
optimizer_reset: false
|
optimizer_reset: false
|
||||||
|
|||||||
@@ -71,7 +71,7 @@ scheduler_cfg:
|
|||||||
trainer_cfg:
|
trainer_cfg:
|
||||||
enable_distributed: true
|
enable_distributed: true
|
||||||
enable_float16: true
|
enable_float16: true
|
||||||
fix_BN: false
|
fix_layers: false
|
||||||
log_iter: 100
|
log_iter: 100
|
||||||
optimizer_reset: true
|
optimizer_reset: true
|
||||||
scheduler_reset: true
|
scheduler_reset: true
|
||||||
|
|||||||
@@ -1,4 +0,0 @@
|
|||||||
# How to Create Your Own Model
|
|
||||||
This section of documentation will be **refined in the future**. For now, you can refer these files: [default config](../config/default.yaml), [baseline config](../config/baseline.yaml), [loss aggregator](../lib/modeling/loss_aggregator.py), [base_model](../lib/modeling/base_model.py), and [baseline model](../lib/modeling/models/baseline.py).
|
|
||||||
|
|
||||||
Then, you can write your own model in `lib\modeling\models`, and use it in configuration file.
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
# Prepare dataset
|
|
||||||
Suppose you have downloaded the original dataset, we need to preprocess the data and save it as pickle file. Remember to set your path to the root of processed dataset in [config/*.yaml](config/).
|
|
||||||
|
|
||||||
## Preprocess
|
|
||||||
**CASIA-B**
|
|
||||||
|
|
||||||
Download URL: http://www.cbsr.ia.ac.cn/GaitDatasetB-silh.zip
|
|
||||||
|
|
||||||
- Original
|
|
||||||
```
|
|
||||||
CASIA-B
|
|
||||||
001 (subject)
|
|
||||||
bg-01 (type)
|
|
||||||
000 (view)
|
|
||||||
001-bg-01-000-001.png (frame)
|
|
||||||
001-bg-01-000-002.png (frame)
|
|
||||||
......
|
|
||||||
......
|
|
||||||
......
|
|
||||||
......
|
|
||||||
```
|
|
||||||
- Run `python misc/pretreatment.py --input_path CASIA-B --output_path CASIA-B-pkl`
|
|
||||||
- Processed
|
|
||||||
```
|
|
||||||
CASIA-B-pkl
|
|
||||||
001 (subject)
|
|
||||||
bg-01 (type)
|
|
||||||
000 (view)
|
|
||||||
000.pkl (contains all frames)
|
|
||||||
......
|
|
||||||
......
|
|
||||||
......
|
|
||||||
```
|
|
||||||
|
|
||||||
## Split dataset
|
|
||||||
You can use the partition file in [misc/partitions](misc/partitions/) directly, or you can create yours. Remember to set your path to the partition file in [config/*.yaml](config/).
|
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# Prepare dataset
|
||||||
|
Suppose you have downloaded the original dataset, we need to preprocess the data and save it as pickle file. Remember to set your path to the root of processed dataset in [config/*.yaml](config/).
|
||||||
|
|
||||||
|
## Preprocess
|
||||||
|
**CASIA-B**
|
||||||
|
|
||||||
|
Download URL: http://www.cbsr.ia.ac.cn/GaitDatasetB-silh.zip
|
||||||
|
- Original
|
||||||
|
```
|
||||||
|
CASIA-B
|
||||||
|
001 (subject)
|
||||||
|
bg-01 (type)
|
||||||
|
000 (view)
|
||||||
|
001-bg-01-000-001.png (frame)
|
||||||
|
001-bg-01-000-002.png (frame)
|
||||||
|
......
|
||||||
|
......
|
||||||
|
......
|
||||||
|
......
|
||||||
|
```
|
||||||
|
- Run `python misc/pretreatment.py --input_path CASIA-B --output_path CASIA-B-pkl`
|
||||||
|
- Processed
|
||||||
|
```
|
||||||
|
CASIA-B-pkl
|
||||||
|
001 (subject)
|
||||||
|
bg-01 (type)
|
||||||
|
000 (view)
|
||||||
|
000.pkl (contains all frames)
|
||||||
|
......
|
||||||
|
......
|
||||||
|
......
|
||||||
|
```
|
||||||
|
**OUMVLP**
|
||||||
|
|
||||||
|
Step1: Download URL: http://www.am.sanken.osaka-u.ac.jp/BiometricDB/GaitMVLP.html
|
||||||
|
|
||||||
|
Step2: Unzip the dataset, you will get a structure directory like:
|
||||||
|
|
||||||
|
- Original
|
||||||
|
```
|
||||||
|
OUMVLP-raw
|
||||||
|
Silhouette_000-00 (view-sequence)
|
||||||
|
00001 (subject)
|
||||||
|
0001.png (frame)
|
||||||
|
0002.png (frame)
|
||||||
|
......
|
||||||
|
00002
|
||||||
|
0001.png (frame)
|
||||||
|
0002.png (frame)
|
||||||
|
......
|
||||||
|
......
|
||||||
|
Silhouette_000-01
|
||||||
|
00001
|
||||||
|
0001.png (frame)
|
||||||
|
0002.png (frame)
|
||||||
|
......
|
||||||
|
00002
|
||||||
|
0001.png (frame)
|
||||||
|
0002.png (frame)
|
||||||
|
......
|
||||||
|
......
|
||||||
|
Silhouette_015-00
|
||||||
|
......
|
||||||
|
Silhouette_015-01
|
||||||
|
......
|
||||||
|
......
|
||||||
|
```
|
||||||
|
Step3 : To rearrange directory of OUMVLP dataset, turning to id-type-view structure, Run
|
||||||
|
```
|
||||||
|
python misc/rearrange_OUMVLP.py --input_path OUMVLP-raw --output_path OUMVLP-rearrange
|
||||||
|
```
|
||||||
|
|
||||||
|
Step4: Transforming images to pickle file, run
|
||||||
|
```
|
||||||
|
python misc/pretreatment.py --input_path OUMVLP-rearrange --output_path OUMVLP-pkl
|
||||||
|
```
|
||||||
|
|
||||||
|
- Processed
|
||||||
|
```
|
||||||
|
OUMVLP-pkl
|
||||||
|
00001 (subject)
|
||||||
|
00 (sequence)
|
||||||
|
000 (view)
|
||||||
|
000.pkl (contains all frames)
|
||||||
|
015 (view)
|
||||||
|
015.pkl (contains all frames)
|
||||||
|
...
|
||||||
|
01 (sequence)
|
||||||
|
000 (view)
|
||||||
|
000.pkl (contains all frames)
|
||||||
|
015 (view)
|
||||||
|
015.pkl (contains all frames)
|
||||||
|
......
|
||||||
|
00002 (subject)
|
||||||
|
......
|
||||||
|
......
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## Split dataset
|
||||||
|
You can use the partition file in [misc/partitions](misc/partitions/) directly, or you can create yours. Remember to set your path to the partition file in [config/*.yaml](config/).
|
||||||
@@ -4,7 +4,7 @@
|
|||||||
* Data configuration
|
* Data configuration
|
||||||
>
|
>
|
||||||
> * Args
|
> * Args
|
||||||
> * dataset_name: Dataset name. Only support `CASIA-B`.
|
> * dataset_name: Only support `CASIA-B` and `OUMVLP` now.
|
||||||
> * dataset_root: The path of storing your dataset.
|
> * dataset_root: The path of storing your dataset.
|
||||||
> * num_workers: The number of workers to collect data.
|
> * num_workers: The number of workers to collect data.
|
||||||
> * dataset_partition: The path of storing your dataset partition file. It splits the dataset to two parts, including train set and test set.
|
> * dataset_partition: The path of storing your dataset partition file. It splits the dataset to two parts, including train set and test set.
|
||||||
@@ -15,7 +15,7 @@
|
|||||||
### loss_cfg
|
### loss_cfg
|
||||||
* Loss function
|
* Loss function
|
||||||
> * Args
|
> * Args
|
||||||
> * type: Loss function type, support `TripletLoss` and `CrossEntropyLoss`
|
> * type: Loss function type, support `TripletLoss` and `CrossEntropyLoss`.
|
||||||
> * loss_term_weights: loss weight.
|
> * loss_term_weights: loss weight.
|
||||||
> * log_prefix: the prefix of loss log.
|
> * log_prefix: the prefix of loss log.
|
||||||
|
|
||||||
@@ -23,36 +23,36 @@
|
|||||||
### optimizer_cfg
|
### optimizer_cfg
|
||||||
* Optimizer
|
* Optimizer
|
||||||
> * Args
|
> * Args
|
||||||
> * solver: Optimizer type, example: `SGD`, `Adam`
|
> * solver: Optimizer type, example: `SGD`, `Adam`.
|
||||||
> * **others**: Please refer to `torch.optim`
|
> * **others**: Please refer to `torch.optim`.
|
||||||
|
|
||||||
|
|
||||||
### scheduler_cfg
|
### scheduler_cfg
|
||||||
* Learning rate scheduler
|
* Learning rate scheduler
|
||||||
> * Args
|
> * Args
|
||||||
> * scheduler : Learning rate scheduler, example: `MultiStepLR`
|
> * scheduler : Learning rate scheduler, example: `MultiStepLR`.
|
||||||
> * **others** : Please refer to `torch.optim.lr_scheduler`
|
> * **others** : Please refer to `torch.optim.lr_scheduler`.
|
||||||
----
|
----
|
||||||
### model_cfg
|
### model_cfg
|
||||||
* Model to be trained
|
* Model to be trained
|
||||||
> * Args
|
> * Args
|
||||||
> * model : Model type, please refer to [Model Library](../lib/modeling/models) for the supported values
|
> * model : Model type, please refer to [Model Library](../lib/modeling/models) for the supported values.
|
||||||
> * **others** : Please refer to [Training Configuration File of Corresponding Model](../config)
|
> * **others** : Please refer to the [Training Configuration File of Corresponding Model](../config).
|
||||||
----
|
----
|
||||||
### evaluator_cfg
|
### evaluator_cfg
|
||||||
* Evaluator configuration
|
* Evaluator configuration
|
||||||
> * Args
|
> * Args
|
||||||
> * enable_float16: If `True`, enable auto mixed precision.
|
> * enable_float16: If `True`, enable the auto mixed precision mode.
|
||||||
> * restore_ckpt_strict: If `True`, check whether the checkpoint is the same as the model.
|
> * restore_ckpt_strict: If `True`, check whether the checkpoint is the same as the defined model.
|
||||||
> * restore_hint: `int` value indicates the iteration number of restored checkpoint; `str` value indicates the path of restored checkpoint.
|
> * restore_hint: `int` value indicates the iteration number of restored checkpoint; `str` value indicates the path to restored checkpoint.
|
||||||
> * save_name: The name of the experiment.
|
> * save_name: The name of the experiment.
|
||||||
> * eval_func: The function name of evaluation. For `CASIA-B`, choose `identification`.
|
> * eval_func: The function name of evaluation. For `CASIA-B`, choose `identification`.
|
||||||
> * sampler:
|
> * sampler:
|
||||||
> - type: The name of sampler. Choose `InferenceSampler`
|
> - type: The name of sampler. Choose `InferenceSampler`.
|
||||||
> - sample_type: In general, we use `all_ordered` to input all frames by its natural order, which makes sure the tests are consistent.
|
> - sample_type: In general, we use `all_ordered` to input all frames by its natural order, which makes sure the tests are consistent.
|
||||||
> - batch_size: In general, it should equal to the number of utilized GPU.
|
> - batch_size: `int` values.
|
||||||
> - **others**: Please refer to [data.sampler](../lib/data/sampler.py) and [data.collate_fn](../lib/data/collate_fn.py)
|
> - **others**: Please refer to [data.sampler](../lib/data/sampler.py) and [data.collate_fn](../lib/data/collate_fn.py)
|
||||||
> * transform: support `BaseSilCuttingTransform`, `BaseSilTransform`. The difference between them is `BaseSilCuttingTransform` cut the pixels on both sides horizontally.
|
> * transform: Support `BaseSilCuttingTransform`, `BaseSilTransform`. The difference between them is `BaseSilCuttingTransform` cut out the black pixels on both sides horizontally.
|
||||||
> * metric: `euc` or `cos`, generally, `euc` performs better.
|
> * metric: `euc` or `cos`, generally, `euc` performs better.
|
||||||
|
|
||||||
----
|
----
|
||||||
@@ -60,25 +60,25 @@
|
|||||||
* Trainer configuration
|
* Trainer configuration
|
||||||
> * Args
|
> * Args
|
||||||
> * fix_BN: If `True`, we fix the weight of all `BatchNorm` layers.
|
> * fix_BN: If `True`, we fix the weight of all `BatchNorm` layers.
|
||||||
> * log_iter: Every `log_iter` iterations, log the information.
|
> * log_iter: Log the information per `log_iter` iterations.
|
||||||
> * save_iter: Every `save_iter` iterations, save the model.
|
> * save_iter: Save the checkpoint per `save_iter` iterations.
|
||||||
> * with_test: If `True`, we test the model every `save_iter` iterations. A bit of performance impact.(*To Be Fixed*)
|
> * with_test: If `True`, we test the model every `save_iter` iterations. A bit of performance impact.(*Disable in Default*)
|
||||||
> * optimizer_reset: If `True` and `restore_hint!=0`, reset the optimizer while restoring the model.
|
> * optimizer_reset: If `True` and `restore_hint!=0`, reset the optimizer while restoring the model.
|
||||||
> * scheduler_reset: If `True` and `restore_hint!=0`, reset the scheduler while restoring the model.
|
> * scheduler_reset: If `True` and `restore_hint!=0`, reset the scheduler while restoring the model.
|
||||||
> * sync_BN: If `True`, applies Batch Normalization as described in the paper [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://arxiv.org/abs/1502.03167).
|
> * sync_BN: If `True`, applies Batch Normalization synchronously.
|
||||||
> * total_iter: The total number of training iterations.
|
> * total_iter: The total training iterations, `int` values.
|
||||||
> * sampler:
|
> * sampler:
|
||||||
> - type: The name of sampler. Choose `TripletSampler`
|
> - type: The name of sampler. Choose `TripletSampler`.
|
||||||
> - sample_type: `[all, fixed, unfixed]` indicates the number of frames used to test, while `[unordered, ordered]` means whether input sequence by its natural order. Example: `fixed_unordered` means selecting fixed number of frames randomly.
|
> - sample_type: `[all, fixed, unfixed]` indicates the number of frames used to test, while `[unordered, ordered]` means whether input sequence by its natural order. Example: `fixed_unordered` means selecting fixed number of frames randomly.
|
||||||
> - batch_size: *[P,K]*\
|
> - batch_size: *[P,K]* where `P` denotes the subjects in training batch while the `K` represents the sequences every subject owns. **Example**:
|
||||||
> **example**:
|
|
||||||
> - 8
|
> - 8
|
||||||
> - 16
|
> - 16
|
||||||
> - **others**: Please refer to [data.sampler](../lib/data/sampler.py) and [data.collate_fn](../lib/data/collate_fn.py)
|
> - **others**: Please refer to [data.sampler](../lib/data/sampler.py) and [data.collate_fn](../lib/data/collate_fn.py).
|
||||||
> * **others**: Please refer to `evaluator_cfg`
|
> * **others**: Please refer to `evaluator_cfg`.
|
||||||
---
|
---
|
||||||
**Note**: All configuatrarion items will merged into [default.yaml](../config/default.yaml), and the current configuration is preferable.
|
**Note**:
|
||||||
|
- All the config items will be merged into [default.yaml](../config/default.yaml), and the current config is preferable.
|
||||||
|
- The output directory, which includes the log, checkpoint and summary files, is depended on the defined `dataset_name`, `model` and `save_name` settings, like `output/${dataset_name}/${model}/${save_name}`.
|
||||||
# Example
|
# Example
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
@@ -102,7 +102,9 @@ evaluator_cfg:
|
|||||||
sample_type: all_ordered # all indicates whole sequence used to test, while ordered means input sequence by its natural order; Other options: fixed_unordered
|
sample_type: all_ordered # all indicates whole sequence used to test, while ordered means input sequence by its natural order; Other options: fixed_unordered
|
||||||
frames_all_limit: 720 # limit the number of sampled frames to prevent out of memory
|
frames_all_limit: 720 # limit the number of sampled frames to prevent out of memory
|
||||||
metric: euc # cos
|
metric: euc # cos
|
||||||
|
transform:
|
||||||
|
- type: BaseSilCuttingTransform
|
||||||
|
img_w: 64
|
||||||
|
|
||||||
loss_cfg:
|
loss_cfg:
|
||||||
- loss_term_weights: 1.0
|
- loss_term_weights: 1.0
|
||||||
@@ -128,6 +130,9 @@ model_cfg:
|
|||||||
- M
|
- M
|
||||||
- BC-256
|
- BC-256
|
||||||
- BC-256
|
- BC-256
|
||||||
|
# - M
|
||||||
|
# - BC-512
|
||||||
|
# - BC-512
|
||||||
type: Plain
|
type: Plain
|
||||||
SeparateFCs:
|
SeparateFCs:
|
||||||
in_channels: 256
|
in_channels: 256
|
||||||
@@ -158,7 +163,7 @@ scheduler_cfg:
|
|||||||
scheduler: MultiStepLR
|
scheduler: MultiStepLR
|
||||||
trainer_cfg:
|
trainer_cfg:
|
||||||
enable_float16: true # half_percesion float for memory reduction and speedup
|
enable_float16: true # half_percesion float for memory reduction and speedup
|
||||||
fix_BN: false
|
fix_layers: false
|
||||||
log_iter: 100
|
log_iter: 100
|
||||||
restore_ckpt_strict: true
|
restore_ckpt_strict: true
|
||||||
restore_hint: 0
|
restore_hint: 0
|
||||||
@@ -176,6 +181,8 @@ trainer_cfg:
|
|||||||
frames_num_min: 25 # min frames number for unfixed traing
|
frames_num_min: 25 # min frames number for unfixed traing
|
||||||
sample_type: fixed_unordered # fixed control input frames number, unordered for controlling order of input tensor; Other options: unfixed_ordered or all_ordered
|
sample_type: fixed_unordered # fixed control input frames number, unordered for controlling order of input tensor; Other options: unfixed_ordered or all_ordered
|
||||||
type: TripletSampler
|
type: TripletSampler
|
||||||
|
transform:
|
||||||
|
- type: BaseSilCuttingTransform
|
||||||
|
img_w: 64
|
||||||
|
|
||||||
```
|
```
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
# How to Create Your Own Model
|
||||||
|
## Pipeline
|
||||||
|
|
||||||
|
|
||||||
|
## A new model
|
||||||
|
If you want to design a new model, you need to write a class inherited from `BaseModel`, e.g, NewModel in newmodel.py:
|
||||||
|
```python
|
||||||
|
from ..base_model import BaseModel
|
||||||
|
|
||||||
|
class NewModel(BaseModel):
|
||||||
|
def __init__(self, cfgs, is_training):
|
||||||
|
super().__init__(cfgs, is_training)
|
||||||
|
|
||||||
|
def build_network(self, model_cfg):
|
||||||
|
self.encoder = ...
|
||||||
|
|
||||||
|
def forward(self, inputs):
|
||||||
|
ipts, labs, typs, viws, seqL = inputs
|
||||||
|
sils = ipts[0]
|
||||||
|
if len(sils.size()) == 4:
|
||||||
|
sils = sils.unsqueeze(2)
|
||||||
|
del ipts
|
||||||
|
n, s, c, h, w = sils.size()
|
||||||
|
|
||||||
|
embed_1, logits, embed = self.encoder(sils)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'training_feat': {
|
||||||
|
'triplet': {'embeddings': embed_1, 'labels': labs},
|
||||||
|
'softmax': {'logits': logits, 'labels': labs}
|
||||||
|
},
|
||||||
|
'visual_summary': {
|
||||||
|
'image/sils': sils.view(n*s, 1, h, w)
|
||||||
|
},
|
||||||
|
'inference_feat': {
|
||||||
|
'embeddings': embed
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
```
|
||||||
|
In your model class, at least you need to implement `build_network()` and `forward()` functions. The first is used to build the netwroks, and it does not need `return value`. Another is used to calculate the features, the `return value` is fixed in dictionary format
|
||||||
|
|
||||||
|
> `training_feat` is for the loss computing, and it must be a `dict` object.
|
||||||
|
>
|
||||||
|
> `visual_summary` is for visualization, and it must be a `dict` object.
|
||||||
|
>
|
||||||
|
> `inference_feat` is for the inference, and it must be a `dict` object.
|
||||||
|
>
|
||||||
|
> `triplet` and `softmax` are the prefixes (or names) of the loss function.
|
||||||
|
>
|
||||||
|
> `embeddings`, `logits` and `labels` are the input arguments of the loss function.
|
||||||
|
|
||||||
|
More information should be seen in [base_model.py](../lib/modeling/base_model.py) and [loss_aggregator.py](../lib/modeling/loss_aggregator.py).
|
||||||
|
|
||||||
|
After finishing the model file, you have two steps left to do:
|
||||||
|
|
||||||
|
**Step 1**: Put your newmodel.py under `lib/modeling/models`.
|
||||||
|
|
||||||
|
**Step 2**: Specify the model name in a yaml file:
|
||||||
|
```yaml
|
||||||
|
model_cfg:
|
||||||
|
model: NewModel
|
||||||
|
param1: ...
|
||||||
|
param2: ...
|
||||||
|
param3: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## A new loss
|
||||||
|
If you want to write a new loss, you need to write a class inherited from `lib/modeling/losses`, like this
|
||||||
|
```python
|
||||||
|
from .base import BaseLoss
|
||||||
|
|
||||||
|
class NewLoss(BaseLoss):
|
||||||
|
def __init__(self, *args, **kwargs):
|
||||||
|
super(NewLoss, self).__init__(*args, **kargs)
|
||||||
|
|
||||||
|
@gather_and_scale_wrapper
|
||||||
|
def forward(self, embeddings, labels):
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
Remember to use `gather_and_scale_wrapper` to wrap your forward function if your loss is computed by pairs like `triplet`. By this, we gather all features to one GPU card and scale the loss by the number of GPUs.
|
||||||
|
|
||||||
|
Then, put your loss in `lib/modeling/losses` so that you can use it in config file.
|
||||||
|
|
||||||
|
Moreover, refer to [loss_aggregator.py](../lib/modeling/loss_aggregator.py) to explore how does your defined loss work in the model.
|
||||||
@@ -0,0 +1,88 @@
|
|||||||
|
# Advanced Usages
|
||||||
|
### Cross-Dataset Evalution
|
||||||
|
> You can conduct cross-dataset evalution by just modifying several arguments in your [data_cfg](../config/baseline.yaml#L1).
|
||||||
|
>
|
||||||
|
> Take [baseline.yaml](../config/baseline.yaml) as an example:
|
||||||
|
> ```yaml
|
||||||
|
> data_cfg:
|
||||||
|
> dataset_name: CASIA-B
|
||||||
|
> dataset_root: your_path
|
||||||
|
> dataset_partition: ./misc/partitions/CASIA-B_include_005.json
|
||||||
|
> num_workers: 1
|
||||||
|
> remove_no_gallery: false # Remove probe if no gallery for it
|
||||||
|
> test_dataset_name: CASIA-B
|
||||||
|
> ```
|
||||||
|
> Now, suppose we get the model trained on [CASIA-B](http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp), and then we want to test it on [OUMVLP](http://www.am.sanken.osaka-u.ac.jp/BiometricDB/GaitMVLP.html).
|
||||||
|
>
|
||||||
|
> We should alter the `dataset_root`, `dataset_partition` and `test_dataset_name`, just like:
|
||||||
|
> ```yaml
|
||||||
|
> data_cfg:
|
||||||
|
> dataset_name: CASIA-B
|
||||||
|
> dataset_root: your_OUMVLP_path
|
||||||
|
> dataset_partition: ./misc/partitions/OUMVLP.json
|
||||||
|
> num_workers: 1
|
||||||
|
> remove_no_gallery: false # Remove probe if no gallery for it
|
||||||
|
> test_dataset_name: OUMVLP
|
||||||
|
> ```
|
||||||
|
---
|
||||||
|
>
|
||||||
|
<!-- ### Identification Function
|
||||||
|
> Sometime, your test dataset may be neither the popular [CASIA-B](http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp) nor the largest [OUMVLP](http://www.am.sanken.osaka-u.ac.jp/BiometricDB/GaitMVLP.html). Meanwhile, you need to customize a special identification function to fit your dataset.
|
||||||
|
>
|
||||||
|
> * If your path structure is similar to [CASIA-B](http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp) (the 3-flod style: `id-type-view`), we recommand you to -->
|
||||||
|
|
||||||
|
### Data Augmentation
|
||||||
|
> In OpenGait, there is a basic transform class almost called by all the models, this is [BaseSilCuttingTransform](../lib/data/transform.py#L20), which is used to cut the input silhouettes.
|
||||||
|
>
|
||||||
|
> Accordingly, by referring to this implementation, you can easily customize the data agumentation in just two steps:
|
||||||
|
> * *Step1*: Define the transform function or class in [transform.py](../lib/data/transform.py), and make sure it callable. The style of [torchvision.transforms](https://pytorch.org/vision/stable/_modules/torchvision/transforms/transforms.html) is recommanded, and following shows a demo;
|
||||||
|
>> ```python
|
||||||
|
>> import torchvision.transforms as T
|
||||||
|
>> class demo1():
|
||||||
|
>> def __init__(self, args):
|
||||||
|
>> pass
|
||||||
|
>>
|
||||||
|
>> def __call__(self, seqs):
|
||||||
|
>> '''
|
||||||
|
>> seqs: with dimension of [sequence, height, width]
|
||||||
|
>> '''
|
||||||
|
>> pass
|
||||||
|
>> return seqs
|
||||||
|
>>
|
||||||
|
>> class demo2():
|
||||||
|
>> def __init__(self, args):
|
||||||
|
>> pass
|
||||||
|
>>
|
||||||
|
>> def __call__(self, seqs):
|
||||||
|
>> pass
|
||||||
|
>> return seqs
|
||||||
|
>>
|
||||||
|
>> def TransformDemo(base_args, demo1_args, demo2_args):
|
||||||
|
>> transform = T.Compose([
|
||||||
|
>> BaseSilCuttingTransform(**base_args),
|
||||||
|
>> demo1(args=demo1_args),
|
||||||
|
>> demo2(args=demo2_args)
|
||||||
|
>> ])
|
||||||
|
>> return transform
|
||||||
|
>> ```
|
||||||
|
> * *Step2*: Reset the [`transform`](../config/baseline.yaml#L100) arguments in your config file:
|
||||||
|
>> ```yaml
|
||||||
|
>> transform:
|
||||||
|
>> - type: TransformDemo
|
||||||
|
>> base_args: {'img_w': 64}
|
||||||
|
>> demo1_args: false
|
||||||
|
>> demo2_args: false
|
||||||
|
>> ```
|
||||||
|
|
||||||
|
### Visualization
|
||||||
|
> To learn how does the model work, sometimes, you need to visualize the intermediate result.
|
||||||
|
>
|
||||||
|
> For this purpose, we have defined a built-in instantiation of [`torch.utils.tensorboard.SummaryWriter`](https://pytorch.org/docs/stable/tensorboard.html), that is [`self.msg_mgr.writer`](../lib/utils/msg_manager.py#L24), to make sure you can log the middle information everywhere you want.
|
||||||
|
>
|
||||||
|
> Demo: if we want to visualize the output feature of [baseline's backbone](../lib/modeling/models/baseline.py#L27), we could just insert the following codes at [baseline.py#L28](../lib/modeling/models/baseline.py#L28):
|
||||||
|
>> ```python
|
||||||
|
>> summary_writer = self.msg_mgr.writer
|
||||||
|
>> if torch.distributed.get_rank() == 0 and self.training and self.iteration % 100==0:
|
||||||
|
>> summary_writer.add_video('outs', outs.mean(2).unsqueeze(2), self.iteration)
|
||||||
|
>> ```
|
||||||
|
> Note that this example requires the [`moviepy`](https://github.com/Zulko/moviepy) package, and hence you should run `pip install moviepy` first.
|
||||||
+5
-5
@@ -52,14 +52,14 @@ class DataSet(tordata.Dataset):
|
|||||||
|
|
||||||
def __getitem__(self, idx):
|
def __getitem__(self, idx):
|
||||||
if not self.cache:
|
if not self.cache:
|
||||||
data_lst = self.__loader__(self.seqs_info[idx][-1])
|
data_list = self.__loader__(self.seqs_info[idx][-1])
|
||||||
elif self.seqs_data[idx] is None:
|
elif self.seqs_data[idx] is None:
|
||||||
data_lst = self.__loader__(self.seqs_info[idx][-1])
|
data_list = self.__loader__(self.seqs_info[idx][-1])
|
||||||
self.seqs_data[idx] = data_lst
|
self.seqs_data[idx] = data_list
|
||||||
else:
|
else:
|
||||||
data_lst = self.seqs_data[idx]
|
data_list = self.seqs_data[idx]
|
||||||
seq_info = self.seqs_info[idx]
|
seq_info = self.seqs_info[idx]
|
||||||
return data_lst, seq_info
|
return data_list, seq_info
|
||||||
|
|
||||||
def __load_all_data(self):
|
def __load_all_data(self):
|
||||||
for idx in range(len(self)):
|
for idx in range(len(self)):
|
||||||
|
|||||||
@@ -1,7 +1,28 @@
|
|||||||
|
"""The plain backbone.
|
||||||
|
|
||||||
|
The plain backbone only contains the BasicConv2d, FocalConv2d and MaxPool2d and LeakyReLU layers.
|
||||||
|
"""
|
||||||
|
|
||||||
import torch.nn as nn
|
import torch.nn as nn
|
||||||
from ..modules import BasicConv2d, FocalConv2d
|
from ..modules import BasicConv2d, FocalConv2d
|
||||||
|
|
||||||
|
|
||||||
class Plain(nn.Module):
|
class Plain(nn.Module):
|
||||||
|
"""
|
||||||
|
The Plain backbone class.
|
||||||
|
|
||||||
|
An implicit LeakyRelu appended to each layer except maxPooling.
|
||||||
|
The kernel size, stride and padding of the first convolution layer are 5, 1, 2, the ones of other layers are 3, 1, 1.
|
||||||
|
|
||||||
|
Typical usage:
|
||||||
|
- BC-64: Basic conv2d with output channel 64. The input channel is the output channel of previous layer.
|
||||||
|
|
||||||
|
- M: nn.MaxPool2d(kernel_size=2, stride=2)].
|
||||||
|
|
||||||
|
- FC-128-1: Focal conv2d with output channel 64 and halving 1(divided to 2^1=2 parts).
|
||||||
|
|
||||||
|
Use it in your configuration file.
|
||||||
|
"""
|
||||||
|
|
||||||
def __init__(self, layers_cfg, in_channels=1):
|
def __init__(self, layers_cfg, in_channels=1):
|
||||||
super(Plain, self).__init__()
|
super(Plain, self).__init__()
|
||||||
@@ -13,9 +34,11 @@ class Plain(nn.Module):
|
|||||||
def forward(self, seqs):
|
def forward(self, seqs):
|
||||||
out = self.feature(seqs)
|
out = self.feature(seqs)
|
||||||
return out
|
return out
|
||||||
|
|
||||||
# torchvision/models/vgg.py
|
|
||||||
def make_layers(self):
|
def make_layers(self):
|
||||||
|
"""
|
||||||
|
Reference: torchvision/models/vgg.py
|
||||||
|
"""
|
||||||
def get_layer(cfg, in_c, kernel_size, stride, padding):
|
def get_layer(cfg, in_c, kernel_size, stride, padding):
|
||||||
cfg = cfg.split('-')
|
cfg = cfg.split('-')
|
||||||
typ = cfg[0]
|
typ = cfg[0]
|
||||||
@@ -27,7 +50,8 @@ class Plain(nn.Module):
|
|||||||
return BasicConv2d(in_c, out_c, kernel_size=kernel_size, stride=stride, padding=padding)
|
return BasicConv2d(in_c, out_c, kernel_size=kernel_size, stride=stride, padding=padding)
|
||||||
return FocalConv2d(in_c, out_c, kernel_size=kernel_size, stride=stride, padding=padding, halving=int(cfg[2]))
|
return FocalConv2d(in_c, out_c, kernel_size=kernel_size, stride=stride, padding=padding, halving=int(cfg[2]))
|
||||||
|
|
||||||
Layers = [get_layer(self.layers_cfg[0], self.in_channels, 5, 1, 2), nn.LeakyReLU(inplace=True)]
|
Layers = [get_layer(self.layers_cfg[0], self.in_channels,
|
||||||
|
5, 1, 2), nn.LeakyReLU(inplace=True)]
|
||||||
in_c = int(self.layers_cfg[0].split('-')[1])
|
in_c = int(self.layers_cfg[0].split('-')[1])
|
||||||
for cfg in self.layers_cfg[1:]:
|
for cfg in self.layers_cfg[1:]:
|
||||||
if cfg == 'M':
|
if cfg == 'M':
|
||||||
@@ -37,6 +61,3 @@ class Plain(nn.Module):
|
|||||||
Layers += [conv2d, nn.LeakyReLU(inplace=True)]
|
Layers += [conv2d, nn.LeakyReLU(inplace=True)]
|
||||||
in_c = int(cfg.split('-')[1])
|
in_c = int(cfg.split('-')[1])
|
||||||
return nn.Sequential(*Layers)
|
return nn.Sequential(*Layers)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+84
-30
@@ -1,3 +1,14 @@
|
|||||||
|
"""The base model definition.
|
||||||
|
|
||||||
|
This module defines the abstract meta model class and base model class. In the base model,
|
||||||
|
we define the basic model functions, like get_loader, build_network, and run_train, etc.
|
||||||
|
The api of the base model is run_train and run_test, they are used in `lib/main.py`.
|
||||||
|
|
||||||
|
Typical usage:
|
||||||
|
|
||||||
|
BaseModel.run_train(model)
|
||||||
|
BaseModel.run_test(model)
|
||||||
|
"""
|
||||||
import torch
|
import torch
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import os.path as osp
|
import os.path as osp
|
||||||
@@ -13,7 +24,6 @@ from abc import abstractmethod
|
|||||||
|
|
||||||
from . import backbones
|
from . import backbones
|
||||||
from .loss_aggregator import LossAggregator
|
from .loss_aggregator import LossAggregator
|
||||||
from modeling.modules import fix_BN
|
|
||||||
from data.transform import get_transform
|
from data.transform import get_transform
|
||||||
from data.collate_fn import CollateFn
|
from data.collate_fn import CollateFn
|
||||||
from data.dataset import DataSet
|
from data.dataset import DataSet
|
||||||
@@ -28,80 +38,97 @@ __all__ = ['BaseModel']
|
|||||||
|
|
||||||
|
|
||||||
class MetaModel(metaclass=ABCMeta):
|
class MetaModel(metaclass=ABCMeta):
|
||||||
|
"""The necessary functions for the base model.
|
||||||
|
|
||||||
|
This class defines the necessary functions for the base model, in the base model, we have implemented them.
|
||||||
|
"""
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def get_loader(self, data_cfg):
|
def get_loader(self, data_cfg):
|
||||||
'''
|
"""Based on the given data_cfg, we get the data loader."""
|
||||||
Build your data Loader here.
|
|
||||||
Inputs: data_cfg, dict
|
|
||||||
Return: Loader
|
|
||||||
'''
|
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def build_network(self, model_cfg):
|
def build_network(self, model_cfg):
|
||||||
'''
|
"""Build your network here."""
|
||||||
Build your Model here.
|
|
||||||
Inputs: model_cfg, dict
|
|
||||||
Return: Network, nn.Module(s)
|
|
||||||
'''
|
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def init_parameters(self):
|
def init_parameters(self):
|
||||||
|
"""Initialize the parameters of your network."""
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def get_optimizer(self, optimizer_cfg):
|
def get_optimizer(self, optimizer_cfg):
|
||||||
'''
|
"""Based on the given optimizer_cfg, we get the optimizer."""
|
||||||
Build your Optimizer here.
|
|
||||||
Inputs: optimizer_cfg, dict
|
|
||||||
Return: Optimizer, a optimizer object
|
|
||||||
'''
|
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def get_scheduler(self, scheduler_cfg):
|
def get_scheduler(self, scheduler_cfg):
|
||||||
'''
|
"""Based on the given scheduler_cfg, we get the scheduler."""
|
||||||
Build your Scheduler.
|
|
||||||
Inputs: scheduler_cfg, dict
|
|
||||||
Optimizer, your optimizer
|
|
||||||
Return: Scheduler, a scheduler object
|
|
||||||
'''
|
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def save_ckpt(self, iteration):
|
def save_ckpt(self, iteration):
|
||||||
|
"""Save the checkpoint, including model parameter, optimizer and scheduler."""
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def resume_ckpt(self, restore_hint):
|
def resume_ckpt(self, restore_hint):
|
||||||
|
"""Resume the model from the checkpoint, including model parameter, optimizer and scheduler."""
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def inputs_pretreament(self, inputs):
|
def inputs_pretreament(self, inputs):
|
||||||
|
"""Transform the input data based on transform setting."""
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def train_step(self, loss_num) -> bool:
|
def train_step(self, loss_num) -> bool:
|
||||||
|
"""Do one training step."""
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def inference(self):
|
def inference(self):
|
||||||
|
"""Do inference (calculate features.)."""
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def run_train(model):
|
def run_train(model):
|
||||||
|
"""Run a whole train schedule."""
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def run_test(model):
|
def run_test(model):
|
||||||
|
"""Run a whole test schedule."""
|
||||||
raise NotImplementedError
|
raise NotImplementedError
|
||||||
|
|
||||||
|
|
||||||
class BaseModel(MetaModel, nn.Module):
|
class BaseModel(MetaModel, nn.Module):
|
||||||
|
"""Base model.
|
||||||
|
|
||||||
|
This class inherites the MetaModel class, and implements the basic model functions, like get_loader, build_network, etc.
|
||||||
|
|
||||||
|
Attributes:
|
||||||
|
msg_mgr: the massage manager.
|
||||||
|
cfgs: the configs.
|
||||||
|
iteration: the current iteration of the model.
|
||||||
|
engine_cfg: the configs of the engine(train or test).
|
||||||
|
save_path: the path to save the checkpoints.
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
def __init__(self, cfgs, training):
|
def __init__(self, cfgs, training):
|
||||||
|
"""Initialize the base model.
|
||||||
|
|
||||||
|
Complete the model initialization, including the data loader, the network, the optimizer, the scheduler, the loss.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
cfgs:
|
||||||
|
All of the configs.
|
||||||
|
training:
|
||||||
|
Whether the model is in training mode.
|
||||||
|
"""
|
||||||
|
|
||||||
super(BaseModel, self).__init__()
|
super(BaseModel, self).__init__()
|
||||||
self.msg_mgr = get_msg_mgr()
|
self.msg_mgr = get_msg_mgr()
|
||||||
self.cfgs = cfgs
|
self.cfgs = cfgs
|
||||||
@@ -132,8 +159,6 @@ class BaseModel(MetaModel, nn.Module):
|
|||||||
"cuda", self.device))
|
"cuda", self.device))
|
||||||
|
|
||||||
if training:
|
if training:
|
||||||
if cfgs['trainer_cfg']['fix_BN']:
|
|
||||||
fix_BN(self)
|
|
||||||
self.loss_aggregator = LossAggregator(cfgs['loss_cfg'])
|
self.loss_aggregator = LossAggregator(cfgs['loss_cfg'])
|
||||||
self.optimizer = self.get_optimizer(self.cfgs['optimizer_cfg'])
|
self.optimizer = self.get_optimizer(self.cfgs['optimizer_cfg'])
|
||||||
self.scheduler = self.get_scheduler(cfgs['scheduler_cfg'])
|
self.scheduler = self.get_scheduler(cfgs['scheduler_cfg'])
|
||||||
@@ -142,7 +167,12 @@ class BaseModel(MetaModel, nn.Module):
|
|||||||
if restore_hint != 0:
|
if restore_hint != 0:
|
||||||
self.resume_ckpt(restore_hint)
|
self.resume_ckpt(restore_hint)
|
||||||
|
|
||||||
|
if training:
|
||||||
|
if cfgs['trainer_cfg']['fix_BN']:
|
||||||
|
self.fix_BN()
|
||||||
|
|
||||||
def get_backbone(self, model_cfg):
|
def get_backbone(self, model_cfg):
|
||||||
|
"""Get the backbone of the model."""
|
||||||
def _get_backbone(backbone_cfg):
|
def _get_backbone(backbone_cfg):
|
||||||
if is_dict(backbone_cfg):
|
if is_dict(backbone_cfg):
|
||||||
Backbone = get_attr_from([backbones], backbone_cfg['type'])
|
Backbone = get_attr_from([backbones], backbone_cfg['type'])
|
||||||
@@ -266,7 +296,20 @@ class BaseModel(MetaModel, nn.Module):
|
|||||||
"Error type for -Restore_Hint-, supported: int or string.")
|
"Error type for -Restore_Hint-, supported: int or string.")
|
||||||
self._load_ckpt(save_name)
|
self._load_ckpt(save_name)
|
||||||
|
|
||||||
|
def fix_BN(self):
|
||||||
|
for module in self.modules():
|
||||||
|
classname = module.__class__.__name__
|
||||||
|
if classname.find('BatchNorm') != -1:
|
||||||
|
module.eval()
|
||||||
|
|
||||||
def inputs_pretreament(self, inputs):
|
def inputs_pretreament(self, inputs):
|
||||||
|
"""Conduct transforms on input data.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
inputs: the input data.
|
||||||
|
Returns:
|
||||||
|
tuple: training data including inputs, labels, and some meta data.
|
||||||
|
"""
|
||||||
seqs_batch, labs_batch, typs_batch, vies_batch, seqL_batch = inputs
|
seqs_batch, labs_batch, typs_batch, vies_batch, seqL_batch = inputs
|
||||||
trf_cfgs = self.engine_cfg['transform']
|
trf_cfgs = self.engine_cfg['transform']
|
||||||
seq_trfs = get_transform(trf_cfgs)
|
seq_trfs = get_transform(trf_cfgs)
|
||||||
@@ -293,9 +336,13 @@ class BaseModel(MetaModel, nn.Module):
|
|||||||
return ipts, labs, typs, vies, seqL
|
return ipts, labs, typs, vies, seqL
|
||||||
|
|
||||||
def train_step(self, loss_sum) -> bool:
|
def train_step(self, loss_sum) -> bool:
|
||||||
'''
|
"""Conduct loss_sum.backward(), self.optimizer.step() and self.scheduler.step().
|
||||||
Conduct loss_sum.backward(), self.optimizer.step() and self.scheduler.step().
|
|
||||||
'''
|
Args:
|
||||||
|
loss_sum:The loss of the current batch.
|
||||||
|
Returns:
|
||||||
|
bool: True if the training is finished, False otherwise.
|
||||||
|
"""
|
||||||
|
|
||||||
self.optimizer.zero_grad()
|
self.optimizer.zero_grad()
|
||||||
if loss_sum <= 1e-9:
|
if loss_sum <= 1e-9:
|
||||||
@@ -322,6 +369,13 @@ class BaseModel(MetaModel, nn.Module):
|
|||||||
return True
|
return True
|
||||||
|
|
||||||
def inference(self, rank):
|
def inference(self, rank):
|
||||||
|
"""Inference all the test data.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
rank: the rank of the current process.Transform
|
||||||
|
Returns:
|
||||||
|
Odict: contains the inference results.
|
||||||
|
"""
|
||||||
total_size = len(self.test_loader)
|
total_size = len(self.test_loader)
|
||||||
if rank == 0:
|
if rank == 0:
|
||||||
pbar = tqdm(total=total_size, desc='Transforming')
|
pbar = tqdm(total=total_size, desc='Transforming')
|
||||||
@@ -355,9 +409,7 @@ class BaseModel(MetaModel, nn.Module):
|
|||||||
|
|
||||||
@ staticmethod
|
@ staticmethod
|
||||||
def run_train(model):
|
def run_train(model):
|
||||||
'''
|
"""Accept the instance object(model) here, and then run the train loop."""
|
||||||
Accept the instance object(model) here, and then run the train loop handler.
|
|
||||||
'''
|
|
||||||
for inputs in model.train_loader:
|
for inputs in model.train_loader:
|
||||||
ipts = model.inputs_pretreament(inputs)
|
ipts = model.inputs_pretreament(inputs)
|
||||||
with autocast(enabled=model.engine_cfg['enable_float16']):
|
with autocast(enabled=model.engine_cfg['enable_float16']):
|
||||||
@@ -390,6 +442,8 @@ class BaseModel(MetaModel, nn.Module):
|
|||||||
|
|
||||||
@ staticmethod
|
@ staticmethod
|
||||||
def run_test(model):
|
def run_test(model):
|
||||||
|
"""Accept the instance object(model) here, and then run the test loop."""
|
||||||
|
|
||||||
rank = torch.distributed.get_rank()
|
rank = torch.distributed.get_rank()
|
||||||
with torch.no_grad():
|
with torch.no_grad():
|
||||||
info_dict = model.inference(rank)
|
info_dict = model.inference(rank)
|
||||||
|
|||||||
@@ -1,3 +1,5 @@
|
|||||||
|
"""The loss aggregator."""
|
||||||
|
|
||||||
import torch
|
import torch
|
||||||
from . import losses
|
from . import losses
|
||||||
from utils import is_dict, get_attr_from, get_valid_args, is_tensor, get_ddp_module
|
from utils import is_dict, get_attr_from, get_valid_args, is_tensor, get_ddp_module
|
||||||
@@ -6,18 +8,48 @@ from utils import get_msg_mgr
|
|||||||
|
|
||||||
|
|
||||||
class LossAggregator():
|
class LossAggregator():
|
||||||
|
"""The loss aggregator.
|
||||||
|
|
||||||
|
This class is used to aggregate the losses.
|
||||||
|
For example, if you have two losses, one is triplet loss, the other is cross entropy loss,
|
||||||
|
you can aggregate them as follows:
|
||||||
|
loss_num = tripley_loss + cross_entropy_loss
|
||||||
|
|
||||||
|
Attributes:
|
||||||
|
losses: A dict of losses.
|
||||||
|
"""
|
||||||
|
|
||||||
def __init__(self, loss_cfg) -> None:
|
def __init__(self, loss_cfg) -> None:
|
||||||
|
"""
|
||||||
|
Initialize the loss aggregator.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
loss_cfg: Config of losses. List for multiple losses.
|
||||||
|
"""
|
||||||
self.losses = {loss_cfg['log_prefix']: self._build_loss_(loss_cfg)} if is_dict(loss_cfg) \
|
self.losses = {loss_cfg['log_prefix']: self._build_loss_(loss_cfg)} if is_dict(loss_cfg) \
|
||||||
else {cfg['log_prefix']: self._build_loss_(cfg) for cfg in loss_cfg}
|
else {cfg['log_prefix']: self._build_loss_(cfg) for cfg in loss_cfg}
|
||||||
|
|
||||||
def _build_loss_(self, loss_cfg):
|
def _build_loss_(self, loss_cfg):
|
||||||
|
"""Build the losses from loss_cfg.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
loss_cfg: Config of loss.
|
||||||
|
"""
|
||||||
Loss = get_attr_from([losses], loss_cfg['type'])
|
Loss = get_attr_from([losses], loss_cfg['type'])
|
||||||
valid_loss_arg = get_valid_args(
|
valid_loss_arg = get_valid_args(
|
||||||
Loss, loss_cfg, ['type', 'pair_based_loss'])
|
Loss, loss_cfg, ['type', 'gather_and_scale'])
|
||||||
loss = get_ddp_module(Loss(**valid_loss_arg))
|
loss = get_ddp_module(Loss(**valid_loss_arg).cuda())
|
||||||
return loss
|
return loss
|
||||||
|
|
||||||
def __call__(self, training_feats):
|
def __call__(self, training_feats):
|
||||||
|
"""Compute the sum of all losses.
|
||||||
|
|
||||||
|
The input is a dict of features. The key is the name of loss and the value is the feature and label. If the key not in
|
||||||
|
built losses and the value is torch.Tensor, then it is the computed loss to be added loss_sum.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
training_feats: A dict of features. The same as the output["training_feat"] of the model.
|
||||||
|
"""
|
||||||
loss_sum = .0
|
loss_sum = .0
|
||||||
loss_info = Odict()
|
loss_info = Odict()
|
||||||
|
|
||||||
@@ -28,14 +60,12 @@ class LossAggregator():
|
|||||||
for name, value in info.items():
|
for name, value in info.items():
|
||||||
loss_info['scalar/%s/%s' % (k, name)] = value
|
loss_info['scalar/%s/%s' % (k, name)] = value
|
||||||
loss = loss.mean() * loss_func.loss_term_weights
|
loss = loss.mean() * loss_func.loss_term_weights
|
||||||
if loss_func.pair_based_loss:
|
|
||||||
loss = loss * torch.distributed.get_world_size()
|
|
||||||
loss_sum += loss
|
loss_sum += loss
|
||||||
|
|
||||||
else:
|
else:
|
||||||
if isinstance(v, dict):
|
if isinstance(v, dict):
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
"The key %s in -Trainng-Feat- should be stated as the log_prefix of a certain loss defined in your loss_cfg."
|
"The key %s in -Trainng-Feat- should be stated as the log_prefix of a certain loss defined in your loss_cfg."%v
|
||||||
)
|
)
|
||||||
elif is_tensor(v):
|
elif is_tensor(v):
|
||||||
_ = v.mean()
|
_ = v.mean()
|
||||||
|
|||||||
@@ -1,13 +1,54 @@
|
|||||||
|
from ctypes import ArgumentError
|
||||||
import torch.nn as nn
|
import torch.nn as nn
|
||||||
|
import torch
|
||||||
from utils import Odict
|
from utils import Odict
|
||||||
|
import functools
|
||||||
|
from utils import ddp_all_gather
|
||||||
|
|
||||||
class BasicLoss(nn.Module):
|
|
||||||
def __init__(self, loss_term_weights=1.0):
|
|
||||||
super(BasicLoss, self).__init__()
|
|
||||||
|
|
||||||
self.loss_term_weights = loss_term_weights
|
def gather_and_scale_wrapper(func):
|
||||||
self.pair_based_loss = True
|
"""Internal wrapper: gather the input from multple cards to one card, and scale the loss by the number of cards.
|
||||||
self.info = Odict()
|
"""
|
||||||
|
|
||||||
|
@functools.wraps(func)
|
||||||
|
def inner(*args, **kwds):
|
||||||
|
try:
|
||||||
|
|
||||||
|
for k, v in kwds.items():
|
||||||
|
kwds[k] = ddp_all_gather(v)
|
||||||
|
|
||||||
|
loss, loss_info = func(*args, **kwds)
|
||||||
|
loss *= torch.distributed.get_world_size()
|
||||||
|
return loss, loss_info
|
||||||
|
except:
|
||||||
|
raise ArgumentError
|
||||||
|
return inner
|
||||||
|
|
||||||
|
|
||||||
|
class BaseLoss(nn.Module):
|
||||||
|
"""
|
||||||
|
Base class for all losses.
|
||||||
|
|
||||||
|
Your loss should also subclass this class.
|
||||||
|
|
||||||
|
Attribute:
|
||||||
|
loss_term_weights: the weight of the loss.
|
||||||
|
info: the loss info.
|
||||||
|
"""
|
||||||
|
loss_term_weights = 1.0
|
||||||
|
info = Odict()
|
||||||
|
|
||||||
def forward(self, logits, labels):
|
def forward(self, logits, labels):
|
||||||
raise NotImplementedError
|
"""
|
||||||
|
The default forward function.
|
||||||
|
|
||||||
|
This function should be overridden by the subclass.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
logits: the logits of the model.
|
||||||
|
labels: the labels of the data.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
tuple of loss and info.
|
||||||
|
"""
|
||||||
|
return .0, self.info
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
import torch
|
import torch
|
||||||
import torch.nn.functional as F
|
import torch.nn.functional as F
|
||||||
|
|
||||||
from .base import BasicLoss
|
from .base import BaseLoss
|
||||||
|
|
||||||
|
|
||||||
class CrossEntropyLoss(BasicLoss):
|
class CrossEntropyLoss(BaseLoss):
|
||||||
def __init__(self, scale=2**4, label_smooth=True, eps=0.1, loss_term_weights=1.0, log_accuracy=False):
|
def __init__(self, scale=2**4, label_smooth=True, eps=0.1, loss_term_weights=1.0, log_accuracy=False):
|
||||||
super(CrossEntropyLoss, self).__init__()
|
super(CrossEntropyLoss, self).__init__()
|
||||||
self.scale = scale
|
self.scale = scale
|
||||||
@@ -13,7 +13,6 @@ class CrossEntropyLoss(BasicLoss):
|
|||||||
self.log_accuracy = log_accuracy
|
self.log_accuracy = log_accuracy
|
||||||
|
|
||||||
self.loss_term_weights = loss_term_weights
|
self.loss_term_weights = loss_term_weights
|
||||||
self.pair_based_loss = False
|
|
||||||
|
|
||||||
def forward(self, logits, labels):
|
def forward(self, logits, labels):
|
||||||
"""
|
"""
|
||||||
@@ -26,7 +25,7 @@ class CrossEntropyLoss(BasicLoss):
|
|||||||
one_hot_labels = self.label2one_hot(
|
one_hot_labels = self.label2one_hot(
|
||||||
labels, c).unsqueeze(0).repeat(p, 1, 1) # [p, n, c]
|
labels, c).unsqueeze(0).repeat(p, 1, 1) # [p, n, c]
|
||||||
loss = self.compute_loss(log_preds, one_hot_labels)
|
loss = self.compute_loss(log_preds, one_hot_labels)
|
||||||
self.info.update({'loss': loss})
|
self.info.update({'loss': loss.detach().clone()})
|
||||||
if self.log_accuracy:
|
if self.log_accuracy:
|
||||||
pred = logits.argmax(dim=-1) # [p, n]
|
pred = logits.argmax(dim=-1) # [p, n]
|
||||||
accu = (pred == labels.unsqueeze(0)).float().mean()
|
accu = (pred == labels.unsqueeze(0)).float().mean()
|
||||||
|
|||||||
@@ -1,22 +1,19 @@
|
|||||||
import torch
|
import torch
|
||||||
import torch.nn.functional as F
|
import torch.nn.functional as F
|
||||||
|
|
||||||
from .base import BasicLoss
|
from .base import BaseLoss, gather_and_scale_wrapper
|
||||||
from utils import ddp_all_gather
|
|
||||||
|
|
||||||
|
|
||||||
class TripletLoss(BasicLoss):
|
class TripletLoss(BaseLoss):
|
||||||
def __init__(self, margin, loss_term_weights=1.0):
|
def __init__(self, margin, loss_term_weights=1.0):
|
||||||
super(TripletLoss, self).__init__()
|
super(TripletLoss, self).__init__()
|
||||||
self.margin = margin
|
self.margin = margin
|
||||||
|
|
||||||
self.loss_term_weights = loss_term_weights
|
self.loss_term_weights = loss_term_weights
|
||||||
self.pair_based_loss = True
|
|
||||||
|
|
||||||
|
@gather_and_scale_wrapper
|
||||||
def forward(self, embeddings, labels):
|
def forward(self, embeddings, labels):
|
||||||
# embeddings: [n, p, c], label: [n]
|
# embeddings: [n, p, c], label: [n]
|
||||||
embeddings = ddp_all_gather(embeddings)
|
|
||||||
labels = ddp_all_gather(labels)
|
|
||||||
embeddings = embeddings.permute(
|
embeddings = embeddings.permute(
|
||||||
1, 0, 2).contiguous() # [n, p, c] -> [p, n, c]
|
1, 0, 2).contiguous() # [n, p, c] -> [p, n, c]
|
||||||
embeddings = embeddings.float()
|
embeddings = embeddings.float()
|
||||||
@@ -32,10 +29,10 @@ class TripletLoss(BasicLoss):
|
|||||||
loss_avg, loss_num = self.AvgNonZeroReducer(loss)
|
loss_avg, loss_num = self.AvgNonZeroReducer(loss)
|
||||||
|
|
||||||
self.info.update({
|
self.info.update({
|
||||||
'loss': loss_avg,
|
'loss': loss_avg.detach().clone(),
|
||||||
'hard_loss': hard_loss,
|
'hard_loss': hard_loss.detach().clone(),
|
||||||
'loss_num': loss_num,
|
'loss_num': loss_num.detach().clone(),
|
||||||
'mean_dist': mean_dist})
|
'mean_dist': mean_dist.detach().clone()})
|
||||||
|
|
||||||
return loss_avg, self.info
|
return loss_avg, self.info
|
||||||
|
|
||||||
|
|||||||
@@ -63,8 +63,8 @@ class GeMHPP(nn.Module):
|
|||||||
|
|
||||||
class GaitGL(BaseModel):
|
class GaitGL(BaseModel):
|
||||||
"""
|
"""
|
||||||
Title: Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation
|
GaitGL: Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation
|
||||||
ICCV2021: https://openaccess.thecvf.com/content/ICCV2021/papers/Lin_Gait_Recognition_via_Effective_Global-Local_Feature_Representation_and_Local_Temporal_ICCV_2021_paper.pdf
|
Arxiv : https://arxiv.org/pdf/2011.01461.pdf
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, *args, **kargs):
|
def __init__(self, *args, **kargs):
|
||||||
@@ -73,31 +73,71 @@ class GaitGL(BaseModel):
|
|||||||
def build_network(self, model_cfg):
|
def build_network(self, model_cfg):
|
||||||
in_c = model_cfg['channels']
|
in_c = model_cfg['channels']
|
||||||
class_num = model_cfg['class_num']
|
class_num = model_cfg['class_num']
|
||||||
|
dataset_name = self.cfgs['data_cfg']['dataset_name']
|
||||||
|
|
||||||
# For CASIA-B
|
if dataset_name == 'OUMVLP':
|
||||||
self.conv3d = nn.Sequential(
|
# For OUMVLP
|
||||||
BasicConv3d(1, in_c[0], kernel_size=(3, 3, 3),
|
self.conv3d = nn.Sequential(
|
||||||
stride=(1, 1, 1), padding=(1, 1, 1)),
|
BasicConv3d(1, in_c[0], kernel_size=(3, 3, 3),
|
||||||
nn.LeakyReLU(inplace=True)
|
stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
)
|
nn.LeakyReLU(inplace=True),
|
||||||
self.LTA = nn.Sequential(
|
BasicConv3d(in_c[0], in_c[0], kernel_size=(3, 3, 3),
|
||||||
BasicConv3d(in_c[0], in_c[0], kernel_size=(
|
stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
3, 1, 1), stride=(3, 1, 1), padding=(0, 0, 0)),
|
nn.LeakyReLU(inplace=True),
|
||||||
nn.LeakyReLU(inplace=True)
|
)
|
||||||
)
|
self.LTA = nn.Sequential(
|
||||||
|
BasicConv3d(in_c[0], in_c[0], kernel_size=(
|
||||||
|
3, 1, 1), stride=(3, 1, 1), padding=(0, 0, 0)),
|
||||||
|
nn.LeakyReLU(inplace=True)
|
||||||
|
)
|
||||||
|
|
||||||
self.GLConvA0 = GLConv(in_c[0], in_c[1], halving=3, fm_sign=False, kernel_size=(
|
self.GLConvA0 = nn.Sequential(
|
||||||
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1))
|
GLConv(in_c[0], in_c[1], halving=1, fm_sign=False, kernel_size=(
|
||||||
self.MaxPool0 = nn.MaxPool3d(kernel_size=(1, 2, 2), stride=(1, 2, 2))
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
|
GLConv(in_c[1], in_c[1], halving=1, fm_sign=False, kernel_size=(
|
||||||
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
|
)
|
||||||
|
self.MaxPool0 = nn.MaxPool3d(
|
||||||
|
kernel_size=(1, 2, 2), stride=(1, 2, 2))
|
||||||
|
|
||||||
self.GLConvA1 = GLConv(in_c[1], in_c[2], halving=3, fm_sign=False, kernel_size=(
|
self.GLConvA1 = nn.Sequential(
|
||||||
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1))
|
GLConv(in_c[1], in_c[2], halving=1, fm_sign=False, kernel_size=(
|
||||||
self.GLConvB2 = GLConv(in_c[2], in_c[2], halving=3, fm_sign=True, kernel_size=(
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1))
|
GLConv(in_c[2], in_c[2], halving=1, fm_sign=False, kernel_size=(
|
||||||
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
|
)
|
||||||
|
self.GLConvB2 = nn.Sequential(
|
||||||
|
GLConv(in_c[2], in_c[3], halving=1, fm_sign=False, kernel_size=(
|
||||||
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
|
GLConv(in_c[3], in_c[3], halving=1, fm_sign=True, kernel_size=(
|
||||||
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
# For CASIA-B or other unstated datasets.
|
||||||
|
self.conv3d = nn.Sequential(
|
||||||
|
BasicConv3d(1, in_c[0], kernel_size=(3, 3, 3),
|
||||||
|
stride=(1, 1, 1), padding=(1, 1, 1)),
|
||||||
|
nn.LeakyReLU(inplace=True)
|
||||||
|
)
|
||||||
|
self.LTA = nn.Sequential(
|
||||||
|
BasicConv3d(in_c[0], in_c[0], kernel_size=(
|
||||||
|
3, 1, 1), stride=(3, 1, 1), padding=(0, 0, 0)),
|
||||||
|
nn.LeakyReLU(inplace=True)
|
||||||
|
)
|
||||||
|
|
||||||
self.Head0 = SeparateFCs(64, in_c[2], in_c[2])
|
self.GLConvA0 = GLConv(in_c[0], in_c[1], halving=3, fm_sign=False, kernel_size=(
|
||||||
self.Bn = nn.BatchNorm1d(in_c[2])
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1))
|
||||||
self.Head1 = SeparateFCs(64, in_c[2], class_num)
|
self.MaxPool0 = nn.MaxPool3d(
|
||||||
|
kernel_size=(1, 2, 2), stride=(1, 2, 2))
|
||||||
|
|
||||||
|
self.GLConvA1 = GLConv(in_c[1], in_c[2], halving=3, fm_sign=False, kernel_size=(
|
||||||
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1))
|
||||||
|
self.GLConvB2 = GLConv(in_c[2], in_c[2], halving=3, fm_sign=True, kernel_size=(
|
||||||
|
3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1))
|
||||||
|
|
||||||
|
self.Head0 = SeparateFCs(64, in_c[-1], in_c[-1])
|
||||||
|
self.Bn = nn.BatchNorm1d(in_c[-1])
|
||||||
|
self.Head1 = SeparateFCs(64, in_c[-1], class_num)
|
||||||
|
|
||||||
self.TP = PackSequenceWrapper(torch.max)
|
self.TP = PackSequenceWrapper(torch.max)
|
||||||
self.HPP = GeMHPP()
|
self.HPP = GeMHPP()
|
||||||
@@ -105,7 +145,9 @@ class GaitGL(BaseModel):
|
|||||||
def forward(self, inputs):
|
def forward(self, inputs):
|
||||||
ipts, labs, _, _, seqL = inputs
|
ipts, labs, _, _, seqL = inputs
|
||||||
seqL = None if not self.training else seqL
|
seqL = None if not self.training else seqL
|
||||||
|
if not self.training and len(labs) != 1:
|
||||||
|
raise ValueError(
|
||||||
|
'The input size of each GPU must be 1 in testing mode, but got {}!'.format(len(labs)))
|
||||||
sils = ipts[0].unsqueeze(1)
|
sils = ipts[0].unsqueeze(1)
|
||||||
del ipts
|
del ipts
|
||||||
n, _, s, h, w = sils.size()
|
n, _, s, h, w = sils.size()
|
||||||
|
|||||||
@@ -191,10 +191,3 @@ def RmBN2dAffine(model):
|
|||||||
if isinstance(m, nn.BatchNorm2d):
|
if isinstance(m, nn.BatchNorm2d):
|
||||||
m.weight.requires_grad = False
|
m.weight.requires_grad = False
|
||||||
m.bias.requires_grad = False
|
m.bias.requires_grad = False
|
||||||
|
|
||||||
|
|
||||||
def fix_BN(model):
|
|
||||||
for module in model.modules():
|
|
||||||
classname = module.__class__.__name__
|
|
||||||
if classname.find('BatchNorm2d') != -1:
|
|
||||||
module.eval()
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
from .common import get_ddp_module, ddp_all_gather
|
from .common import get_ddp_module, ddp_all_gather
|
||||||
from .common import Odict, Ntuple
|
from .common import Odict, Ntuple
|
||||||
from .common import get_valid_args
|
from .common import get_valid_args
|
||||||
from .common import is_list_or_tuple, is_str, is_list, is_dict, is_tensor, is_array, config_loader, init_seeds, handler, params_count
|
from .common import is_list_or_tuple, is_bool, is_str, is_list, is_dict, is_tensor, is_array, config_loader, init_seeds, handler, params_count
|
||||||
from .common import ts2np, ts2var, np2var, list2var
|
from .common import ts2np, ts2var, np2var, list2var
|
||||||
from .common import mkdir, clones
|
from .common import mkdir, clones
|
||||||
from .common import MergeCfgsDict
|
from .common import MergeCfgsDict
|
||||||
|
|||||||
@@ -74,6 +74,10 @@ def is_list_or_tuple(x):
|
|||||||
return isinstance(x, (list, tuple))
|
return isinstance(x, (list, tuple))
|
||||||
|
|
||||||
|
|
||||||
|
def is_bool(x):
|
||||||
|
return isinstance(x, bool)
|
||||||
|
|
||||||
|
|
||||||
def is_str(x):
|
def is_str(x):
|
||||||
return isinstance(x, str)
|
return isinstance(x, str)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
kill $(ps aux | grep main.py | grep -v grep | awk '{print $2}')
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,59 @@
|
|||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
from tqdm import tqdm
|
||||||
|
import argparse
|
||||||
|
|
||||||
|
|
||||||
|
parser = argparse.ArgumentParser(description='Test')
|
||||||
|
parser.add_argument('--input_path', default='/home1/data/OUMVLP_raw', type=str,
|
||||||
|
help='Root path of raw dataset.')
|
||||||
|
parser.add_argument('--output_path', default='/home1/data/OUMVLP_rearranged', type=str,
|
||||||
|
help='Root path for output.')
|
||||||
|
|
||||||
|
|
||||||
|
opt = parser.parse_args()
|
||||||
|
|
||||||
|
INPUT_PATH = opt.input_path
|
||||||
|
OUTPUT_PATH = opt.output_path
|
||||||
|
|
||||||
|
|
||||||
|
def mv_dir(src, dst):
|
||||||
|
shutil.copytree(src, dst)
|
||||||
|
print(src, dst)
|
||||||
|
|
||||||
|
|
||||||
|
sils_name_list = os.listdir(INPUT_PATH)
|
||||||
|
name_space = 'Silhouette_'
|
||||||
|
views = sorted(list(
|
||||||
|
set([each.replace(name_space, '').split('-')[0] for each in sils_name_list])))
|
||||||
|
seqs = sorted(list(
|
||||||
|
set([each.replace(name_space, '').split('-')[1] for each in sils_name_list])))
|
||||||
|
ids = list()
|
||||||
|
for each in sils_name_list:
|
||||||
|
ids.extend(os.listdir(os.path.join(INPUT_PATH, each)))
|
||||||
|
|
||||||
|
|
||||||
|
progress = tqdm(total=len(set(ids)))
|
||||||
|
|
||||||
|
|
||||||
|
results = list()
|
||||||
|
pid = 0
|
||||||
|
for _id in sorted(set(ids)):
|
||||||
|
progress.update(1)
|
||||||
|
for _view in views:
|
||||||
|
for _seq in seqs:
|
||||||
|
seq_info = [_id, _seq, _view]
|
||||||
|
name = name_space + _view + '-' + _seq + '/' + _id
|
||||||
|
src = os.path.join(INPUT_PATH, name)
|
||||||
|
dst = os.path.join(OUTPUT_PATH, *seq_info)
|
||||||
|
if os.path.exists(src):
|
||||||
|
try:
|
||||||
|
if os.path.exists(dst):
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
os.makedirs(dst)
|
||||||
|
for subfile in os.listdir(src):
|
||||||
|
os.symlink(os.path.join(src, subfile),
|
||||||
|
os.path.join(dst, subfile))
|
||||||
|
except OSError as err:
|
||||||
|
print(err)
|
||||||
@@ -1,5 +1,6 @@
|
|||||||
|
# # **************** For CASIA-B ****************
|
||||||
# # Baseline
|
# # Baseline
|
||||||
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/baseline.yaml --phase test
|
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/baseline.yaml --phase test
|
||||||
|
|
||||||
# # GaitSet
|
# # GaitSet
|
||||||
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gaitset.yaml --phase test
|
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gaitset.yaml --phase test
|
||||||
@@ -8,10 +9,24 @@
|
|||||||
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gaitpart.yaml --phase test
|
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gaitpart.yaml --phase test
|
||||||
|
|
||||||
# GaitGL
|
# GaitGL
|
||||||
# CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --master_port 12345 --nproc_per_node=4 lib/main.py --cfgs ./config/gaitgl.yaml --iter 80000 --phase test
|
# CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --master_port 12345 --nproc_per_node=4 lib/main.py --cfgs ./config/gaitgl.yaml --phase test
|
||||||
|
|
||||||
# # GLN
|
# # GLN
|
||||||
# # Phase 1
|
# # Phase 1
|
||||||
CUDA_VISIBLE_DEVICES=3,4 python -m torch.distributed.launch --master_port 12345 --nproc_per_node=2 lib/main.py --cfgs ./config/gln/gln_phase1.yaml --phase test
|
# CUDA_VISIBLE_DEVICES=3,4 python -m torch.distributed.launch --master_port 12345 --nproc_per_node=2 lib/main.py --cfgs ./config/gln/gln_phase1.yaml --phase test
|
||||||
# # Phase 2
|
# # Phase 2
|
||||||
# CUDA_VISIBLE_DEVICES=2,5 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gln/gln_phase2.yaml --phase test
|
# CUDA_VISIBLE_DEVICES=2,5 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gln/gln_phase2.yaml --phase test
|
||||||
|
|
||||||
|
|
||||||
|
# # **************** For OUMVLP ****************
|
||||||
|
# # Baseline
|
||||||
|
# CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 lib/main.py --cfgs ./config/baseline_OUMVLP.yaml --phase test
|
||||||
|
|
||||||
|
# # GaitSet
|
||||||
|
# CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 lib/main.py --cfgs ./config/gaitset_OUMVLP.yaml --phase test
|
||||||
|
|
||||||
|
# # GaitPart
|
||||||
|
# CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 lib/main.py --cfgs ./config/gaitpart_OUMVLP.yaml --phase test
|
||||||
|
|
||||||
|
# GaitGL
|
||||||
|
# CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 lib/main.py --cfgs ./config/gaitgl_OUMVLP.yaml --phase test
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
|
# # **************** For CASIA-B ****************
|
||||||
# # Baseline
|
# # Baseline
|
||||||
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/baseline.yaml --phase train
|
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/baseline.yaml --phase train
|
||||||
|
|
||||||
# # GaitSet
|
# # GaitSet
|
||||||
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gaitset.yaml --phase train
|
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gaitset.yaml --phase train
|
||||||
@@ -8,10 +9,24 @@
|
|||||||
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gaitpart.yaml --phase train
|
# CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 lib/main.py --cfgs ./config/gaitpart.yaml --phase train
|
||||||
|
|
||||||
# GaitGL
|
# GaitGL
|
||||||
#CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 lib/main.py --cfgs ./config/gaitgl.yaml --phase train
|
# CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 lib/main.py --cfgs ./config/gaitgl.yaml --phase train
|
||||||
|
|
||||||
# # GLN
|
# # GLN
|
||||||
# # Phase 1
|
# # Phase 1
|
||||||
CUDA_VISIBLE_DEVICES=2,5,6,7 python -m torch.distributed.launch --nproc_per_node=4 lib/main.py --cfgs ./config/gln/gln_phase1.yaml --phase train
|
# CUDA_VISIBLE_DEVICES=2,5,6,7 python -m torch.distributed.launch --nproc_per_node=4 lib/main.py --cfgs ./config/gln/gln_phase1.yaml --phase train
|
||||||
# # Phase 2
|
# # Phase 2
|
||||||
CUDA_VISIBLE_DEVICES=2,5,6,7 python -m torch.distributed.launch --nproc_per_node=4 lib/main.py --cfgs ./config/gln/gln_phase2.yaml --phase train
|
# CUDA_VISIBLE_DEVICES=2,5,6,7 python -m torch.distributed.launch --nproc_per_node=4 lib/main.py --cfgs ./config/gln/gln_phase2.yaml --phase train
|
||||||
|
|
||||||
|
|
||||||
|
# # **************** For OUMVLP ****************
|
||||||
|
# # Baseline
|
||||||
|
# CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 lib/main.py --cfgs ./config/baseline_OUMVLP.yaml --phase train
|
||||||
|
|
||||||
|
# # GaitSet
|
||||||
|
# CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 lib/main.py --cfgs ./config/gaitset_OUMVLP.yaml --phase train
|
||||||
|
|
||||||
|
# # GaitPart
|
||||||
|
# CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 lib/main.py --cfgs ./config/gaitpart_OUMVLP.yaml --phase train
|
||||||
|
|
||||||
|
# GaitGL
|
||||||
|
# CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 lib/main.py --cfgs ./config/gaitgl_OUMVLP.yaml --phase train
|
||||||
Reference in New Issue
Block a user